Bugfix/prov info issues
* Initial commit
* Passed payer name to tin npi llm call to avoid assigning tin npi to payer
* Removed record that contains NO_IDENTIFIERS_FOUND from PROV_INFO_JSON
* Updated unit test for test_tin_npi_funcs
* Changed name check code back to original to be consistent
* Updated test_tin_npi_funcs.py to pass in payer_name in functions
* Add fix to unit test
* Updated output in unit tests
* Merged main into bugfix/prov-info-issues
Approved-by: Katon Minhas
Bugfix/one to one and prov info
* Add post-processing logic to update PROV_INFO_JSON if needed; Updated prompt to verify the number of digitis in TIN and NPI
* Added update_prov_info_with_group_name to postprocessing_funcs.py
* Updated FIRST_PAGE_PROVIDER_NAME_TEMPLATE to ask LLM to explain resoning before answering
* Updated logic to use LLM to check if provider name matches
* Use only Hybrid Smart Chunking to get provider group name
* Cleaned up code
* Removed commented out code
* Updated unit test
* Merged main into bugfix/one_to_one_and_prov_info
* Edited prompt to distinguish provider names and to enclose answer in pipes
* Updated test_tin_npi_funcs
Approved-by: Katon Minhas
Bugfix/prov info
* Added logic to extract provider name from first page
* Added prompt to extract provider name from first page
* Added prompts and provider name extraction using Hybrid Smart Chunking
* Cleaned up code
* Added additional logic to check if the provider name on tin/npi page matches provider name extracted from page 1.
* Added fix for IS_GROUP issue in provider info json
* Remove commented out code
* unit test fix
* uni test fix
* Merge branch 'main' into bugfix/prov_info
* Change to string_utils.is_empty
* Minor refactor
Approved-by: Katon Minhas
Feature/black and isort
* black and isort constants
* black and isort codes
* black and isort crosswalks
* black and isort investment
* black and isort prompts
* isort testbed
* isort tracking
* black and isort utils
* poetry and black tests
Approved-by: Katon Minhas
Feature/tin enhancements
* Add OCR-correctable TIN patterns and substitution mapping
* Merge remote-tracking branch 'origin/main' into feature/tin-enhancements
* Refactor OCR-correctable TIN pattern and enhance documentation for clarity
* Enhance get_all_matches function with type hints and improve documentation; add get_all_matches_with_ocr for OCR-correctable matches
* Add attempt_ocr_correction function for TIN OCR error handling and validation
* Merge remote-tracking branch 'origin/main' into feature/tin-enhancements
* Integrate OCR correction for TIN extraction in run_provider_info_fields; log corrections for monitoring
* Refactor OCR-correctable patterns in regex_patterns.py; add NPI_OCR_PATTERN and enhance documentation for clarity
* Enhance OCR correction in attempt_ocr_correction to support both TIN and NPI; update validation logic for identifier lengths
* Integrate NPI OCR correction in run_provider_info_fields; log corrections for monitoring
* Enhance logging in run_provider_info_fields to summarize TIN and NPI extraction quality; include counts of exact and OCR-corrected identifiers
* Add unit and integration tests for OCR correction
* isort
* Merge remote-tracking branch 'origin/main' into feature/tin-enhancements
* Merged main into feature/tin-enhancements
Approved-by: Katon Minhas
Bugfix/dynamic primary issues
* Refactor TRIGGER_CAP special case configuration to utilize only trigger cap questions for two-stage processing
* Refactor get_special_cases to improve handling of N/A responses and merge methodology results with special case results
* Fix field type in get_special_cases for fee schedule breakout configuration
* Create fieldset.py - refactor
* Fix placeholders
* string concatenate new questions
* Update trigger cap questions output format
* Update TRIGGER_CAP_THRESHOLD_AMT prompt for clarity and specificity in reimbursement methodology
* Update constants class
* Update Field class
* Resolve constants passing
* Update field class - change Null criteria
* Add default
* Enhance documentation for get_special_cases function with detailed processing flow and architecture decisions
* Enhance get_special_cases and run_trigger_cap_breakout functions with detailed docstrings for clarity on processing methodologies
* Update constants passing
* Fix questions
* Update fs and mb questions"
* pass constanst correctly for dynamic
* Merge branch 'feature/trigger_cap_breakout' into bugfix/dynamic-primary-issues
* update constants
* only add questions if present
* ensure answer is a valid special_case
* Fix unit tests
* Remove test file
Approved-by: Alex Galarce
Refactor/one time io
* Delete client work
* Streamline imports
* Fix list class
* Update tests
* Clear code funcs test
* Fix unit tests
* Single read code funcs
* Single-read model
* Single-load embeddings
* Successful E2E test
* update unit tests
* Update importsg
* Reload poetry.lock
* remove old exhibit header function
* Remove aarete_derived generic function
* remove align and format tables
* Remove strings to dict
* references
* Clear code
* Move preprocessing_funcs
* remove keywords
* refactor postprocessing_funcs
* Pass unit test - remove qa_qc directory
* Black and isort
* remove print
Approved-by: Alex Galarce
Feature/prov group tin
* don't prompt for INTRO_PAGE
* test complete
* test 2
* Update test script
* Merge branch 'main' into feature/prov-group-tin
* Merged main into feature/prov-group-tin
* refactor
* cleaning
* Update save
* Fix return type
* Update for reimbursement_tin_npi
* Comment out reimbursement-tin-npi
* Pass unit tests
* Merge branch 'main' into feature/prov-group-tin
* Merged main into feature/prov-group-tin
* Remove length validation
* Merge branch 'main' into feature/prov-group-tin
* Remove ON_INTRO from prompt
* Restore
* Remove test
* Move reimb prov info to dynamic reimb info
* e2e test
* Fix for unit test
* remove prints
* update docstrings and typehints
Approved-by: Alex Galarce
Feature/align tin npi
* Add merge_provider_info function to consolidate provider data into one-to-one results
* Merge remote-tracking branch 'origin/main' into feature/align-tin-npi
* Remove debug print statement from run_one_to_one_prompts function
* turn run_regex_fields into an orchestrator function by breaking parts of it into separate functions
* Remove unused import of get_matches function and add docstring to chunk_on_matches for clarity
* Add docstring to run_regex_fields for improved clarity and documentation
* fix provider name keys in merge_provider_info
* Remove unused functions get_matches, tin_npi_prompt, and clean_tin_npi from tin_npi_funcs.py
* Set provider name to "UNKNOWN" if cleaned name is empty in clean_provider_info
* Add unit tests for clean_provider_info function to validate data processing
* Enhance docstring for get_all_matches function
* Add tests for get_all_matches and chunk_on_matches functions
* Merge remote-tracking branch 'origin/main' into feature/align-tin-npi
* Implement merge_provider_info function to consolidate provider data into one-to-one results
* Refactor merge_provider_info to simplify list conversion for group and other provider information
* Remove unnecessary TIN and NPI postprocessing step from postprocess function
* Remove clean_tin_npi_other function and its invocation from postprocess function
* Use json prompts for TIN/NPI/Name and inject them into TIN_NPI_TEMPLATE
* Rename run_regex_fields to run_provider_info_fields
* Enhance TIN_NPI_TEMPLATE to include detailed instructions for extracting provider entities and their identifying information, including IS_GROUP logic for main contracting parties.
* moved TIN/NPI regexes to regex_patterns.py
* Remove unnecessary blank line in investment_values.py
* Remove debug print statements from run_provider_info_fields and get_provider_info functions
* Enhance clean_provider_info to handle various representations of IS_GROUP as boolean
* Enhance get_provider_info to include error handling for JSON parsing and ensure consistent return format
Approved-by: Katon Minhas
Sha Brown