feature/dynamic and generalized branch
* included txt files for nltk_data
* move nltk_data to src
* Fix last upload count
* Last upload count bugfix
* Fixed B processing
* Remove client-specific postprocessing
* split consolidate_output
* Run by file
* Fix output
* Reconfigure smart_chunk fields
* Add Full Context
* Fix merge conflicts
* Regex, Smart-Chunked, and Full working - not adding smart-chunked-->full when necessary
* Modernized run_full_context_fields()
* Switched set to list in field_context
* Move fields from smart_chunked to full_context as part of 'field_context' function
* Working version with placeholders
* Update poetry and pyproject
* Update s3 output
* Remove deprecated unit test
* Updated error messages
* Updated smart chunk ac name to one to one
* Update dependencies - end-to-end test for write s3 functional
* Add basic multithreading
* Send individual output to s3/local
Approved-by: Alex Galarce
Feature/integrate standard and complex
* last edits to merge_tables_draft_v2 before moving to table_funcs
* migrate to table_utils.py
* fix typehint that caused mypy error
* fix mypy errors
* black and isort
* fix error with multi-table pages
* update poetry.lock
* add tests, fix bug when table start marker or table end marker is missing
* added more tests
* tests for get_str_dictionaries_from_text()
* fix edge case in get_str_dictionaries_from_text
* fix mypy error
* more tests
* add tests, add handling for invalid dictionaries
* add tests
* add tests for insert_column_headers()
* update requirements
* add new simple/complex logic to preprocess.py
* remove files used solely for testing
* split pages into sub-pages by tables
* add table split on end marker. Also add tests
* add docstrings, change control flow
* remove unused tests, add TODO for new tests
* get in prompt changes and intermediate decisions
* TODO for future enhancement for get_exhibit_pages
Approved-by: Katon Minhas
Task/unit tests investment and client
* move io_utils.py
* add __init__.py
* refactor: move string_funcs and update import paths to use utils namespace
* rename string_funcs to string_utils
* move and rename table_utils.py
* refactor: rename claude_funcs to llm_utils in multiple files
* update unit tests
* added unit tests for src/string_utils.py
* added unit tests for src/table_utils.py
* added unit tests for src/llm_utils.py
* added unit tests for src/io_utils.py
* Merge branch 'main' into task/unit-tests-investment-and-client; TODO: fix unit tests
* Unit tests to fix
* fix unit tests
* fix invalid references
* add unit tests for preprocessing_funcs.py
* move pytest-mock to test dependencies
* Merge remote-tracking branch 'origin/main' into task/unit-tests-investment-and-client
* add tests/string_utils_test.py
* removed filename parameters for a few functions
* save postprocessing_funcs_test midway
* save postprocessing_funcs_test
* postprocessing_funcs unit tests complete
* Merge branch 'main' into task/unit-tests-investment-and-client
Approved-by: Katon Minhas
Alex Galarce