Bugfix/molina nv reimb method issues to main
* added asc and apc full forms in prompt
* updated grouper instructions for reimb method
* updated reimb method prompt
* Merged main into bugfix/molina_nv_reimb_method_issues
Approved-by: Katon Minhas
Bugfix/process only after title
* Update print
* Update exhibit dict creation
* Detect Participating Provider Agreement
* Remove test file
* Merged main into bugfix/process-only-after-title
* Merged main into bugfix/process-only-after-title
Approved-by: Faizan Mohiuddin
Feature/field type config
* Added field type procesing conditional logic
* Updated field type naming convention in config.py; Updated date_postprocessing to avoid column key missing issue
* Cleaned up code; Move merging functionality to one to one processing block
* Merged main into feature/field_type_config
Approved-by: Katon Minhas
Bugfix/ut issues
* Update exhibit linking
* prompt_calls refactor
* remove test
* Add Healthy Kids Program
* Move Healthy Kids to UT specifically
* Remove header check from Dynamic process
* Merged main into bugfix/ut-issues
* prep for merge
* remove deprecated tests
* Merged main into bugfix/ut-issues
* Merged main into bugfix/ut-issues
Approved-by: Faizan Mohiuddin
Bugfix/special cases to Main
* prompt update
* Merge branch 'main' into bugfix/json_and_unit_of_measure
* minor bugfix
* minor bugfix
* prompt update for outlier rate
* added outlier testing jupyter notebook
* Remove special case from any empty
* updated outlier breakout prompt
* Merged main into bugfix/special-cases
* reimb_method minor fix
* Merged main into bugfix/special-cases
* bugfix field to add dynamic funcs
* merged with main
* latest outlier changes
* Merged main into bugfix/special-cases
* added table guidelines for special case primary
* updated special case primary prompt
* Merged main into bugfix/special-cases
* updated special case primary table handling
* merged main into bugfix special cases
* Merged main into bugfix/special-cases
* Remove test notebooks from tracking
* trying out pipeline fix
* reverted chnages
* pipeline fix
Approved-by: Katon Minhas
Bugfix/mi issue fixes
* Update mapping to allow for , or | lists
* Pipe-separation for full-context
* Update VALIDATE_REIMBURSEMENTS
* Designate Provider Submission Form as cover sheet language - make cover sheet detection more efficient
* remove deprecated functions
* remove test
Approved-by: Faizan Mohiuddin
Bugfix/mi dynamic
* Refactor - split out one_to_n_cleaning
* Rework postprocess
* Update postprocessing - fill dynamic values if there is only 1 non-empty value
* Update to use string_utils
* Group by AARETE_DERIVED_LOB as well
* Update crosswalk mapping to handle | lists
* Update constants - run with MI flag
Approved-by: Siddhant Medar
bugfix/molina_tracker_issues to main
* provider name updated prompt
* updated provider name prompt
* Merged main into bugfix/molina_tracker_issues
Approved-by: Katon Minhas
Feature/one to one funcs docs and cleanup
* Refactor imports and remove unused filter_payer_from_providers function
* Update docstring for date_postprocessing function to clarify processing steps
* Augment docstrings for check_and_update_effective_date and run_full_context_fields
* Enhance docstring for one_to_one_breakout function to clarify processing steps and side effects
* remove unused import and augment docstrings
Approved-by: Katon Minhas
Update mapping to detect if field is aarete_derived
* Update mapping to detect if field is aarete_derived
* Merged main into bugfix/network-mapping
Approved-by: Alex Galarce
Feature/black and isort
* black and isort constants
* black and isort codes
* black and isort crosswalks
* black and isort investment
* black and isort prompts
* isort testbed
* isort tracking
* black and isort utils
* poetry and black tests
Approved-by: Katon Minhas
Feature/one to n docs
* Refactor function signatures and improve imports in one_to_n_funcs.py
* Enhance type hints and documentation for carveout_check and methodology_breakout functions
* Add docstring to methodology_breakout_secondary function for clarity on parameters and return values
* Add docstring to special_case_breakout function for clarity on parameters and return values
* Enhance docstring for deduplicate_with_accumulator function to clarify behavior and parameters
* Enhance docstring for filter_services_without_reimbursements function to clarify validation process and answer structure
* Enhance docstring for filter_services_without_reimbursements function to clarify filtering criteria and validation process
* Enhance docstring for split_compound_reimbursements function to clarify the splitting process and provide an example
* Enhance docstring for split_reimb_dates function to clarify date processing, parameters, return values, and provide an example
* Remove unused regex import from one_to_n_funcs.py
* Remove unused Delimiter import from one_to_n_funcs.py
Approved-by: Katon Minhas
io_utils documentation
* add docstring to read_local function and handle file read errors gracefully by returning None on failure
* update read_local function to support .xlsx files and improve error handling
* add docstring to read_s3 function to clarify its purpose and return value
* refactor read_s3 and read_s3_csv functions to improve error handling and add type hints
* improve error handling in read_s3_csv function for CSV file parsing
* enhance read_input function with error handling and detailed docstring
* Add TODOs for read_input_csv and filter_already_processed functions; add type hints and docstring for remove_txt_extension
* enhance write_local function documentation; make docstrings more consistent when there is a "notes" section
* enhance write_local and write_s3 functions; improve documentation, add error output type handling, and ensure consistent CSV writing options
* refactor test_io_utils.py; reorganize imports and enhance read_input_local test with additional mock for file existence
Approved-by: Katon Minhas
Bugfix/provider info to refactor/one-to-n
* Applied suggestion to fieldExtraction/src/prompts/investment_prompts.json
* update unit test
* Update Carveout IND and Default IND logic
* Split out exhibit-level lesser
* Bugfix
* update lesser of exhibit prompt
* Bugfix
* exhibit lesser of varible name bugfix
* Update exhibit lesser of passing
* Update Exhibit-level prompt template
* Shorten full_context prompt length if the questions are long
* Special case assignment index
* Resolve unit tests
* updated UC IND prompt
* Remove print
* update one-to-n unit test
* added dot based date example
* updated date fix prompt
* Merged in bugfix/json_error_and_unit_of_measure (pull request #708)
Bugfix/json error and unit of measure
* json parser fixes and prompt changes
* universal json load fixed
* json error fixed
* json error fixed 2
* Merged refactor/one-to-n into bugfix/json_error_and_unit_of_measure
Approved-by: Katon Minhas
* added new tin format
* updated signature page prompt
* updated tin search and tin duplication
* Merge remote-tracking branch 'origin/main' into bugfix/provider_info
* extended duplicate logic to npi and names
* Merged main into bugfix/provider_info
Approved-by: Katon Minhas
json parsing fix
* OUTLIER_TERMS-->OUTLIER_TERM standardization
* Applied suggestion to fieldExtraction/src/prompts/investment_prompts.json
* update unit test
* Update Carveout IND and Default IND logic
* Split out exhibit-level lesser
* Bugfix
* update lesser of exhibit prompt
* Bugfix
* exhibit lesser of varible name bugfix
* Update exhibit lesser of passing
* Update Exhibit-level prompt template
* Shorten full_context prompt length if the questions are long
* Special case assignment index
* Resolve unit tests
* updated UC IND prompt
* Remove print
* update one-to-n unit test
* added dot based date example
* updated date fix prompt
* Merged in bugfix/json_error_and_unit_of_measure (pull request #708)
Bugfix/json error and unit of measure
* json parser fixes and prompt changes
* universal json load fixed
* json error fixed
* json error fixed 2
* Merged refactor/one-to-n into bugfix/json_error_and_unit_of_measure
Approved-by: Katon Minhas
* json parsing fix
* prompt update
* Merge branch 'main' into bugfix/json_and_unit_of_measure
* minor bugfix
* minor bugfix
Approved-by: Alex Galarce
Bugfix/fee schedule
* Initialize grouper pct and base rate prmopts
* Update postprocessing
* Merged main into bugfix/fee-schedule
* Prep for merge
Approved-by: Alex Galarce
Feature/improve logging docs
* Enhance PerFileHandler documentation and functionality for improved logging context management
* Add more docuementation
* Update logging_utils.py documentation for clarity and usage examples
* Enhance logging comments for clarity on per-file logging setup and concurrent processing
* Enhance logging comments for clarity on per-file logging context and its implications
Approved-by: Katon Minhas
Refactor/one to n
* remove print
* Update logging for fsbreakout and grouper breakout
* Additional type hints
* updated special case primary
* Remove deprecated special case check
* OUTLIER_TERMS-->OUTLIER_TERM standardization
* Applied suggestion to fieldExtraction/src/prompts/investment_prompts.json
* update unit test
* Update Carveout IND and Default IND logic
* Split out exhibit-level lesser
* Bugfix
* update lesser of exhibit prompt
* Bugfix
* exhibit lesser of varible name bugfix
* Update exhibit lesser of passing
* Update Exhibit-level prompt template
* Shorten full_context prompt length if the questions are long
* Special case assignment index
* Resolve unit tests
* updated UC IND prompt
* Remove print
* update one-to-n unit test
* added dot based date example
* updated date fix prompt
* Merged in bugfix/json_error_and_unit_of_measure (pull request #708)
Bugfix/json error and unit of measure
* json parser fixes and prompt changes
* universal json load fixed
* json error fixed
* json error fixed 2
* Merged refactor/one-to-n into bugfix/json_error_and_unit_of_measure
Approved-by: Katon Minhas
Approved-by: Alex Galarce
Bugfix/provider info tracker issues to main
* testing TIN AND NPI
* print statements added
* updated condition for header and footer tin and npi check
* tin testing for hyphen format
* removed print statements
* updated comments
Approved-by: Katon Minhas
Add time-based rate handling rules and update section labels in prompt templates
* Add time-based rate handling rules and update section labels in prompt templates
* Add mapping for MyCare Ohio in crosswalk_program.json and update state mapping for Ohio in crosswalk_program_lob.json
* Update test parameters and state configuration in testing notebook
* Update exhibit-wide payment constraint prompt to exclude administrative processing language
* Merge remote-tracking branch 'origin/main' into bugfix/reimb-term-issues
* Refine exhibit-wide payment constraint prompt to clarify extraction criteria and examples
* Enhance exhibit-wide payment constraint prompt with guidance for abbreviated terms and clarify acceptable constraints
* Update SERVICE_TERM prompt to specify healthcare services and clarify extraction criteria
Approved-by: Katon Minhas
Bugfix/trigger cap fixes
* stoploss prompt added again
* stoploss prompt added again
* stoploss prompt updated
* reverted back to one stage for trigger cap cases
* Merge branch 'main' into bugfix/trigger_cap_fixes
* pipeline error fixed
* incorrect discount special case fixed
Approved-by: Alex Galarce
Change logging level from info to debug for OCR correction and extraction summaries
* Change logging level from info to debug for OCR correction and extraction summaries
Approved-by: Katon Minhas
Bugfix/multi tin npi fix
* Add debug logging for OCR candidates and provider info extraction
* Update TIN and NPI extraction instructions to handle multiple values in a single record
* Add validation layer to cross-check TINs and NPIs with regex findings in provider info extraction
* Enhance exact match extraction by normalizing identifiers and validating lengths for TIN and NPI
Approved-by: Katon Minhas
Feature/tin enhancements
* Add OCR-correctable TIN patterns and substitution mapping
* Merge remote-tracking branch 'origin/main' into feature/tin-enhancements
* Refactor OCR-correctable TIN pattern and enhance documentation for clarity
* Enhance get_all_matches function with type hints and improve documentation; add get_all_matches_with_ocr for OCR-correctable matches
* Add attempt_ocr_correction function for TIN OCR error handling and validation
* Merge remote-tracking branch 'origin/main' into feature/tin-enhancements
* Integrate OCR correction for TIN extraction in run_provider_info_fields; log corrections for monitoring
* Refactor OCR-correctable patterns in regex_patterns.py; add NPI_OCR_PATTERN and enhance documentation for clarity
* Enhance OCR correction in attempt_ocr_correction to support both TIN and NPI; update validation logic for identifier lengths
* Integrate NPI OCR correction in run_provider_info_fields; log corrections for monitoring
* Enhance logging in run_provider_info_fields to summarize TIN and NPI extraction quality; include counts of exact and OCR-corrected identifiers
* Add unit and integration tests for OCR correction
* isort
* Merge remote-tracking branch 'origin/main' into feature/tin-enhancements
* Merged main into feature/tin-enhancements
Approved-by: Katon Minhas
Feature/initialize phase 5
* Initialize at Exhibit-Level
* Merge branch 'main' into feature/initialize-phase-5
* Add all breakout fields
* Prep for merge
Approved-by: Alex Galarce
bugfix/multiple_outlier_fixes to main
* updated prompts
* Merged main into bugfix/multiple_outliers_fixes
* updated outlier maximum prompts
Approved-by: Katon Minhas
Bugfix/lob fixes
* Refactor update_lob_for_duals and update_grouper_base_rate_and_grouper_pct_rate functions for improved clarity and logic; add handling for 'neither' and 'n/a' cases.
* Change logging level from info to debug for various functions to reduce log verbosity.
Approved-by: Katon Minhas
Feature/payer state
* fix indentation for PROVIDER_STATE prompt
* Add PAYER_STATE prompt and normalize state abbreviation in run_full_context_fields
* Add PAYER_STATE to COLUMN_ORDER in investment_columns.py
* Merged main into feature/payer-state
* Only run PAYER_STATE prompt when more than one state in the batch
Faizan Mohiuddin