Feature/multithreading
* fixes
* Merge main into feature/multithreading
Resolved conflicts:
- Kept timing instrumentation in file_processing.py
- Kept new 3-step Exhibit-based approach for one-to-n processing
- Maintained parallelization improvements (20 workers)
Changes include:
- Timing utils integration for performance monitoring
- Increased max_workers from 5 to 20 across all components
- Parallelized code_breakout and grouper_breakout
- Fixed tin_npi_funcs function call parameters
* Fix max_workers error for empty documents
- Add check to skip parallel processing when no pages exist
- Use min(len(all_page_tasks), 20) to prevent max_workers=0
- Handles edge case of documents with no exhibits or pages
* Fix max_workers=0 errors in one_to_n_funcs
- Add checks before all ThreadPoolExecutor creations
- Prevents errors when processing empty lists:
- carveout_and_special_case
- breakout
- special_case_breakout
- filter_services_without_reimbursements
- run_lob_relationship
- Ensures executor only created when there are items to process
* Reduce code processing parallelism to prevent API throttling
Lower max_workers from 20 to 10 for code_breakout and grouper_breakout
to prevent overwhelming Bedrock API with concurrent requests
* fixed
* Merged main into feature/multithreading
* Move documentation into folder
* Fix logging statements
* Merge branch 'main' into feature/multithreading
* Refactor for clarity
* Update previous exhibit passing logic
* properly simplify exhibits
* Merged main into feature/multithreading
* update conditional for None
* exhibit multithreading changed
* exhibit multithreading changed
* Merge remote-tracking branch 'origin/main' into feature/multithreading
* synced with main
* Parallelize dynamic assignment, refactor HSC field worker, add timing/exhibit unit tests, tidy imports/ignore helpers
* analyze_regression.py edited online with Bitbucket
* count_pages.py edited online with Bitbucket
* compare_regressed_with_baseline.py edited online with Bitbucket
* simple_testbed_compare.py edited online with Bitbucket
* run_testbed_metrics_regressed.py edited online with Bitbucket
Approved-by: Katon Minhas
Faizan Mohiuddin
8c9060e425