Files
doczyai-pipelines/fieldExtraction
Katon Minhas 3bb6abbcef Merged in feature/simplify-b-field-chunking (pull request #348)
feature/simplify b field chunking

* Add chunk_by_exhibit function and unit tests

* Refactor preprocess

* Updated to run BU primary on entire Exhibit

* Makes BU secondary compatible with exhibit chunk mapping

* Added unit tests for get_exhibit_chunk

* Trimmed TD funcs

* Downstream changes to run_top_down

* Removed conditional funcs

* Merged main into feature/simplify-b-field-chunking

* removed conditional funcs import

* Merged main into feature/simplify-b-field-chunking

* Fixed run_bottom_up_primary arguments

* Fixed TD primary input

* Fixed grouped keyword mappings

* Remove filename tracking until postprocessing

* Removed postprocessing for client columns

* Removed col-based postprocessing

* Merged main into feature/simplify-b-field-chunking

* Fixed NoneType

* Add docstrings

* Updated unit tests for removal of 'Filename'

* Updated comments


Approved-by: Alex Galarce
2025-01-10 17:24:44 +00:00
..

Field Extraction

Running the Code

This project uses Python's module system for imports. To run the code properly:

  1. DO NOT run files directly like this:

    python src/client/main.py  # ❌ This won't work
    
  2. INSTEAD, use Python's module flag (-m) from the project root:

    python -m src.client.main  # ✅ This is correct
    

Why?

The code uses absolute imports (e.g., from src.utils import llm_utils) to maintain a clear and consistent package structure. Running with python -m ensures Python can properly resolve these imports.

Development

  • All imports should use the src. prefix (e.g., from src.utils import llm_utils)
  • Always run code from the project root directory using the -m flag
  • Tests are configured to handle these imports automatically via pytest settings in pyproject.toml

Configuration

The project uses poetry for dependency management and pytest for testing. Key configurations in pyproject.toml:

[tool.pytest.ini_options]
pythonpath=["."]
testpaths = [
    "tests"
]

Dependencies

To install dependencies:

poetry install

To run tests:

poetry run pytest

To run type checking:

poetry run mypy .