Merged in feature/integrate-standard-and-complex (pull request #356)

Feature/integrate standard and complex

* last edits to merge_tables_draft_v2 before moving to table_funcs

* migrate to table_utils.py

* fix typehint that caused mypy error

* fix mypy errors

* black and isort

* fix error with multi-table pages

* update poetry.lock

* add tests, fix bug when table start marker or table end marker is missing

* added more tests

* tests for get_str_dictionaries_from_text()

* fix edge case in get_str_dictionaries_from_text

* fix mypy error

* more tests

* add tests, add handling for invalid dictionaries

* add tests

* add tests for insert_column_headers()

* update requirements

* add new simple/complex logic to preprocess.py

* remove files used solely for testing

* split pages into sub-pages by tables

* add table split on end marker.  Also add tests

* add docstrings, change control flow

* remove unused tests, add TODO for new tests

* get in prompt changes and intermediate decisions

* TODO for future enhancement for get_exhibit_pages


Approved-by: Katon Minhas

fieldExtraction/pyproject.toml

@@ -16,7 +16,6 @@ psutil = "^6.1.0"
 rapidfuzz = "^3.10.1"
 pyxlsb = "^1.0.10"
 openpyxl = "^3.1.5"
-pytest-mock = "^3.14.0"
 
 [tool.poetry.group.dev.dependencies]
 black = "^24.10.0"
@@ -27,6 +26,7 @@ isort = "^5.13.2"
 [tool.poetry.group.test.dependencies]
 pytest = "^8.3.3"
 pytest-mock = "^3.14.0"
+pytest-cov = "^6.0.0"
 
 [build-system]
 requires = ["poetry-core"]