Do not run Exhibit level prompt if there are no exhibit level fields
* Do not run Exhibit level prompt if there are no exhibit level fields
* Update dynamic test
* Remove test from one-to-n test module
* Update test
* Fix unit test
Approved-by: Alex Galarce
Collect TIN/NPI/Name page-by-page and run a second pass for IS_GROUP
* configure first pass: no providers marked IS_GROUP. Allows simplification of prompt
* Add group provider identification logic in second pass
* group identification: don't take 20% of pages. Take first 3 and last 3
* Refine group provider identification logic to use pipes for JSON formatting and improve error handling
* fix mypy error for `identify_group_provider`
* Merge remote-tracking branch 'origin/main' into feature/tin-npi-prompt-formatting
* Add "PROV_INFO_JSON" to investment column order
* Refactor group provider identification to use signature page extraction and fix JSON format
* move signature page extraction to `string_utils.py` and add unit tests for it
* Update identify_group_provider to return provider name along with TIN and NPI in JSON format
* Update identify_group_provider to always use group-provided name when available
* move out GROUP_TIN_NPI_TEMPLATE to `investment_prompts.py`
* Remove deprecated PROV_GROUP_TIN_CHECK and PROV_GROUP_NPI_CHECK prompts
* Move out provider info fields from `one_to_one_funcs` to `tin_npi_funcs`
* isort
* remove debug print statements
* Remove debug print statements from provider info functions
* add note to regex_utils.py about where functions are used
* Add page_key_sort function and corresponding tests for sorting page keys
* Refactor identify_group_provider to sort pages and avoid duplicates in extracted sections
* Remove debug print statements from identify_group_provider function
* Merge remote-tracking branch 'origin/main' into feature/tin-npi-prompt-formatting
* extract_signature_page function: prioritize Textract marker for signatures and fallback to keyword search
* fix mypy error
* Update test cases to reflect changes in signature extraction logic
Approved-by: Katon Minhas
Field/initialize patient age
* Update split_text so that it works on single-page contracts (discovered when testing)
* Add PATIENT_AGE_RANGE intermediate column
* Remove PATIENT_AGE_RANGE in postprocessing
* Dont convert columns to int type
* Add unit tests
* Remove test
* Update unit test
* Update unit test
* Update unit test
* Update unit test
Approved-by: Alex Galarce
Field/dynamic codes
* remove test
* Merge branch 'main' into field/dynamic-codes
* Merged main into field/dynamic-codes
* Update test
* Fix unit test
* Merged main into field/dynamic-codes
* Merged main into field/dynamic-codes
* Merged main into field/dynamic-codes
* Update fields with crosswalks
* Update mapping to return empty string if no mapping
* Update dynamic_funcs
* Restructure
* Update file_processing
* Update dynamic primary
* Fix dynamic primary
* Update base fields
* Update one-to-n process
* Merged main into field/dynamic-codes
* genericized get_dynamic_answers
* Update tests
* Remove test file
* dynamic_funcs cleanup
* remove prints
* add exhibit_header
* Fix imports
Approved-by: Alex Galarce
Bugfix/flatten singleton list ints problem
* Fix flatten_singleton_string_list to handle non-list int strings and improve test coverage
* Fix test_flatten_singleton_string_list to handle integer input correctly
Approved-by: Katon Minhas
Feature/align tin npi
* Add merge_provider_info function to consolidate provider data into one-to-one results
* Merge remote-tracking branch 'origin/main' into feature/align-tin-npi
* Remove debug print statement from run_one_to_one_prompts function
* turn run_regex_fields into an orchestrator function by breaking parts of it into separate functions
* Remove unused import of get_matches function and add docstring to chunk_on_matches for clarity
* Add docstring to run_regex_fields for improved clarity and documentation
* fix provider name keys in merge_provider_info
* Remove unused functions get_matches, tin_npi_prompt, and clean_tin_npi from tin_npi_funcs.py
* Set provider name to "UNKNOWN" if cleaned name is empty in clean_provider_info
* Add unit tests for clean_provider_info function to validate data processing
* Enhance docstring for get_all_matches function
* Add tests for get_all_matches and chunk_on_matches functions
* Merge remote-tracking branch 'origin/main' into feature/align-tin-npi
* Implement merge_provider_info function to consolidate provider data into one-to-one results
* Refactor merge_provider_info to simplify list conversion for group and other provider information
* Remove unnecessary TIN and NPI postprocessing step from postprocess function
* Remove clean_tin_npi_other function and its invocation from postprocess function
* Use json prompts for TIN/NPI/Name and inject them into TIN_NPI_TEMPLATE
* Rename run_regex_fields to run_provider_info_fields
* Enhance TIN_NPI_TEMPLATE to include detailed instructions for extracting provider entities and their identifying information, including IS_GROUP logic for main contracting parties.
* moved TIN/NPI regexes to regex_patterns.py
* Remove unnecessary blank line in investment_values.py
* Remove debug print statements from run_provider_info_fields and get_provider_info functions
* Enhance clean_provider_info to handle various representations of IS_GROUP as boolean
* Enhance get_provider_info to include error handling for JSON parsing and ensure consistent return format
Approved-by: Katon Minhas
Hotfix/flatten list to string
* updated
* rate fields fix
* updated the singleton_list
* updated
* added
* Checked_files
* added
* Merged main into hotfix/flatten-list-to-string
* format_rate_fields_with_commas
* Update test
* flatten_singleton_string_list - new cases
* Update
* Removed unneccessary case
* Remove case
* Updated the test cases
* Merge remote-tracking branch 'origin/main' into hotfix/flatten-list-to-string
* Refactor flatten_singleton_string_list to handle empty input and return a comma-separated string for multiple elements
Approved-by: Katon Minhas
Hotfix/reimb issue fixes
* updated DYNAMIC_CHECK prompt
* updated date_fix_prompt to YYYYMMDD format
* filled N/A values for REIMB_PROV_TIN
* changed YYYYMMDD to YYYY/MM/DD
* extracting TIN, NPI,NAME as a tuple
* updated TIN, NPI,NAME as a tuple prompt
* Merge remote-tracking branch 'origin/main' into hotfix/reimb_issue_fixes
* fix: correct spelling of 'memorize' to 'memoize' in cache decorator comment
* combined postprocess date format check functions
* Merged main into hotfix/reimb_issue_fixes
* updated postprocessing
* rename and slight refactor
* remove unused imports, re-sort
* remove unused imports from postprocess.py
* Fix unit tests
* re-add imports which are needed for tests to pass (?)
* Fix column name in auto-renewal termination date handling
* Fix formatting in postprocess function
Approved-by: Katon Minhas
Feature/dynamic one to one
* Pass N/A to one-to-one
* E2E test passed
* Merged main into feature/dynamic-one-to-one
* Pass ANY non N/A answer to reimbursement-level
* Update test
* Remove test
* Merge branch 'main' into feature/dynamic-one-to-one
* Wrap clean_dates
* Revert postprocess
* Merged main into feature/dynamic-one-to-one
* Merged main into feature/dynamic-one-to-one
* Remove prints
Approved-by: Alex Galarce
Bugfix/table error fix
* handle multipage table data correctly
* same metadata for all tables on the page
* Multiple tables on page, non-continuous, split
* Add back align_and_format_tables
* update unit tests
* Merged main into bugfix/table-error-fix
* improved align and format table prompt
* Successful E2E test on San Joaquin - without running align and format table prompt
* San Joaquin e2e test passed
* Merged main into bugfix/table-error-fix
* Merge branch 'main' into bugfix/table-error-fix
* Add same number of columns condition
* Merged main into bugfix/table-error-fix
* Clean investment_values
* Remove import
* docstrings
* Remove test doc
* Add back original ALIGN_AND_FORMAT_TABLES prompt for client work
* Remove import
* Add config reference
* Merged main into bugfix/table-error-fix
* Move table_utils to investment.table_funcs
* restore table_utils.py
* Update tests
* update preprocess
Approved-by: Alex Galarce
Update date formatting in prompts to YYYY/MM/DD
* Update date formatting in prompts to YYYY/MM/DD
* fix tests for new date formatting
Approved-by: Katon Minhas
Bugfix/dynamic assignment issue
* Merged main into feature/proc-desc
* Uncomment test code
* Bugfix
* script optimisation for codes
* modified is empty function in string utils
* Remove test.py
* bug fix to handle list of proc_codes
* Merged main into bugfix/proc-code-lists
* changed any method to all method in list handling
* code mapping moved outside to run once
* Merge branch 'feature/daip2-97' into bugfix/proc-code-lists
* merge conflict fixed
* merge conflicts fixed
* incorporated latest changes from main
* Merged main into bugfix/proc-code-lists
* Move load all dataset to io_utils
* Fix list is_empty
* Updated poetry.lock
* Bugfix
* Docstring
* Docstrings
* Merge branch 'main' into bugfix/proc-code-lists
* Merge branch 'bugfix/proc-code-lists' into bugfix/dynamic-assignment-issue
* Update test for no default exhibit_level_answer_dict
* Merged main into bugfix/dynamic-assignment-issue
Approved-by: Alex Galarce
Document dates optimization
* simplified file processing for testing
* modified derived_term_date_prompt
* fixed day values with spaces or leading zeros
* modified contract termination date prompt
* updated termination date prompt
* reranking of chunks for better context
* exclude last date while calculating term date
* clean up
* clean up 2
* Merge branch 'main' into document_dates_optimization
pulled latest changes from main
* Merged main into document_dates_optimization
* fix unit tests
* Merged main into document_dates_optimization
* refactor: update termination date prompt for clarity and accuracy
* Merged main into document_dates_optimization
* fix pipeline
* Merge remote-tracking branch 'origin/main' into document_dates_optimization
* Merge branch 'document_dates_optimization' of https://bitbucket.org/aarete/doczy.ai into document_dates_optimization
Approved-by: Katon Minhas
Test/check doc dates against test bed
* first commit
* Enhance prompts and processing for contract termination dates
- Updated prompt to include 'year to year' as a valid example output.
- Implemented case-insensitive processing for contract text and field keywords for smart chunking.
- Clarified prompt for derived termination date to specify it is exclusive of renewals
* Merge remote-tracking branch 'origin/main' into test/check-doc-dates-against-test-bed
* Update termination date prompt tests
* Refine keywords for contract termination date prompt in investment prompts
Approved-by: Katon Minhas
Initialize Reimbursement-Level TINs
* Initialized Reimbursement-Level TIN and NPI
* Update unit tests
* Docstrings and cleanup
* Update test
Approved-by: Alex Galarce
Fields/document dates further iteration
* add concise date fix prompt
* Refactor and prepare for postprocessing formatting
* Merged main into fields/document-dates-further-iteration
* Merged main into fields/document-dates-further-iteration
* Implement memoization for date postprocessing and add derived termination date calculation
* Merge branch 'fields/document-dates-further-iteration' of https://bitbucket.org/aarete/doczy.ai into fields/document-dates-further-iteration
* fix mypy error
* tweaks to derived termination date
* Add docstring to derived_term_date_prompt and create unit tests for prompt generation
* fix memoization in postprocess.py
* Merged main into fields/document-dates-further-iteration
Approved-by: Katon Minhas
Feature/doc date iterations
* add parse_chunk()
* add stitch_chunks() and tests
* create new constant
* put in new stitch_chunks() function
* resolve output validation TODO
* Merged main into feature/clean-up-smart-chunking
* fix Mayank's callout of noncontiguous chunks, fix typos, add tests
* add comment on test
* Merge remote-tracking branch 'origin/main' into feature/doc-date-iterations
* add 'signature page' to effective date keywords. loosen instructions on termination date.
* enhance prompts with examples and clarify instructions for termination date
* fix for group_fields()
* clean up test imports
Approved-by: Katon Minhas
Feature/clean up smart chunking
* add parse_chunk()
* add stitch_chunks() and tests
* create new constant
* put in new stitch_chunks() function
* resolve output validation TODO
* Merged main into feature/clean-up-smart-chunking
* fix Mayank's callout of noncontiguous chunks, fix typos, add tests
* add comment on test
* Merged main into feature/clean-up-smart-chunking
Approved-by: Katon Minhas
feature/dynamic and generalized branch
* included txt files for nltk_data
* move nltk_data to src
* Fix last upload count
* Last upload count bugfix
* Fixed B processing
* Remove client-specific postprocessing
* split consolidate_output
* Run by file
* Fix output
* Reconfigure smart_chunk fields
* Add Full Context
* Fix merge conflicts
* Regex, Smart-Chunked, and Full working - not adding smart-chunked-->full when necessary
* Modernized run_full_context_fields()
* Switched set to list in field_context
* Move fields from smart_chunked to full_context as part of 'field_context' function
* Working version with placeholders
* Update poetry and pyproject
* Update s3 output
* Remove deprecated unit test
* Updated error messages
* Updated smart chunk ac name to one to one
* Update dependencies - end-to-end test for write s3 functional
* Add basic multithreading
* Send individual output to s3/local
Approved-by: Alex Galarce
Feature/integrate standard and complex
* last edits to merge_tables_draft_v2 before moving to table_funcs
* migrate to table_utils.py
* fix typehint that caused mypy error
* fix mypy errors
* black and isort
* fix error with multi-table pages
* update poetry.lock
* add tests, fix bug when table start marker or table end marker is missing
* added more tests
* tests for get_str_dictionaries_from_text()
* fix edge case in get_str_dictionaries_from_text
* fix mypy error
* more tests
* add tests, add handling for invalid dictionaries
* add tests
* add tests for insert_column_headers()
* update requirements
* add new simple/complex logic to preprocess.py
* remove files used solely for testing
* split pages into sub-pages by tables
* add table split on end marker. Also add tests
* add docstrings, change control flow
* remove unused tests, add TODO for new tests
* get in prompt changes and intermediate decisions
* TODO for future enhancement for get_exhibit_pages
Approved-by: Katon Minhas
Task/unit tests investment and client
* move io_utils.py
* add __init__.py
* refactor: move string_funcs and update import paths to use utils namespace
* rename string_funcs to string_utils
* move and rename table_utils.py
* refactor: rename claude_funcs to llm_utils in multiple files
* update unit tests
* added unit tests for src/string_utils.py
* added unit tests for src/table_utils.py
* added unit tests for src/llm_utils.py
* added unit tests for src/io_utils.py
* Merge branch 'main' into task/unit-tests-investment-and-client; TODO: fix unit tests
* Unit tests to fix
* fix unit tests
* fix invalid references
* add unit tests for preprocessing_funcs.py
* move pytest-mock to test dependencies
* Merge remote-tracking branch 'origin/main' into task/unit-tests-investment-and-client
* add tests/string_utils_test.py
* removed filename parameters for a few functions
* save postprocessing_funcs_test midway
* save postprocessing_funcs_test
* postprocessing_funcs unit tests complete
* Merge branch 'main' into task/unit-tests-investment-and-client
Approved-by: Katon Minhas
feature/simplify b field chunking
* Add chunk_by_exhibit function and unit tests
* Refactor preprocess
* Updated to run BU primary on entire Exhibit
* Makes BU secondary compatible with exhibit chunk mapping
* Added unit tests for get_exhibit_chunk
* Trimmed TD funcs
* Downstream changes to run_top_down
* Removed conditional funcs
* Merged main into feature/simplify-b-field-chunking
* removed conditional funcs import
* Merged main into feature/simplify-b-field-chunking
* Fixed run_bottom_up_primary arguments
* Fixed TD primary input
* Fixed grouped keyword mappings
* Remove filename tracking until postprocessing
* Removed postprocessing for client columns
* Removed col-based postprocessing
* Merged main into feature/simplify-b-field-chunking
* Fixed NoneType
* Add docstrings
* Updated unit tests for removal of 'Filename'
* Updated comments
Approved-by: Alex Galarce
Task/unit tests investment and client
* move io_utils.py
* add __init__.py
* refactor: move string_funcs and update import paths to use utils namespace
* rename string_funcs to string_utils
* move and rename table_utils.py
* refactor: rename claude_funcs to llm_utils in multiple files
* update unit tests
* added unit tests for src/string_utils.py
* added unit tests for src/table_utils.py
* added unit tests for src/llm_utils.py
* added unit tests for src/io_utils.py
* Merge branch 'main' into task/unit-tests-investment-and-client; TODO: fix unit tests
* Unit tests to fix
* fix unit tests
* fix invalid references
Approved-by: Alex Galarce
Refactor/split investment and client
* refactor: clean up tests
* Renamed tests/qcqa to tests/qa_qc for consistency
* comment out faulty test - see docstring note at top
* split to client/investment
* add __init__.py to investment and client to avoid mypy confusion
* fix imports
* update imports for consistency across investment module
* move back to one config
* try changing import to relative
* add init.py to src
* try relative import
* try one more sys.path.append
* fix path for client main
* fix imports in file_processing
* make prompts common
* move keywords out to `src`
* move smart_chunking_funcs to `src`
* fix mypy error
* start moving to explicit imports
* remove old keywords and fix imports for keywords and prompts
* change all imports to full paths
* fix unit tests
* add readme for new way of running things
* isort after all that import work
* remove unused sys.path.append from some tests
Approved-by: Katon Minhas
Refactor/daip2-9 code refactor
* add missing import
* refactor ac_smart_chunking.py
* update tests
* removed old and unused imports
* removed outdated import
* forgot to import re
* refactor bottom up funcs
* remove unused imports from file_processing.py
* refactor dict_operations.py
* remove unused import from conditional_funcs.py
* replace top_down_funcs with one_to_n_funcs and remove top_down_funcs.py
* remove duplicate import from one_to_n_funcs.py
* refactor all utils.py imports
* refactor error handling by removing last code in utils and integrating InvalidDateException into postprocessing_funcs
* refactor import in claude_funcs.py to use string_funcs directly and (hopefully) resolve circular import
* refactor regex_funcs.py to use postprocessing_funcs for add_hyphen_if_needed calls
* refactor hotfix_helper_funcs.py to use postprocessing_funcs for add_hyphen_if_needed calls
* refactor postprocess.py to remove unused imports
* add numpy import to string_funcs
* fix tests after utils refactor
* move adhoc from tests to scripts
* remove unused imports
* remove scripts from mypy checking
* isort
* Merged main into refactor/daip-2-9-code-refactor
Approved-by: Katon Minhas
Feature/self repair main
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* new testing
* new testing
* added unit tests under fieldExtraction/tests/main-unit-tests
* unit test fixes
* unit test fixes
* merge into main
* modified consolidate_output and slight changes to qa_qc_helpers to accomodate qaqc functionality w faizan and michael's changes
* merging self repair into main
* synced with main now
* fixes
* provider billing removed
* provider billing removed
* provider billing fixed
* commented out load dotenv in config
* Merged main into feature/self-repair-main
* modified check_s3_bucket_exists function by returning False if credentials not found for pipeline
* Merge branch 'feature/self-repair-main' of https://bitbucket.org/aarete/doczy.ai into feature/self-repair-main
Approved-by: Michael McGuinness
QCQA Additions
* tested on batch 10, small fixes + change to s3 output
* Merged main into QCQA_Additions
* unit tests for qc qa helpers
* static error fix
* another static fix
* static fix again
* fixed None check
* static fix none
* Merged main into QCQA_Additions
* Small changes based on Katon's review
Approved-by: Katon Minhas
QCQA Integration
* Merged in bugfix/npi_hotfix (pull request #305)
double filter for duplicates
* double filter for duplicates
Approved-by: Katon Minhas
* Merged in chore/npi (pull request #309)
npi hotfix into main
* npi hotfix into main
* npi hotfix into main
Approved-by: Alex Galarce
* Merged in chore/irs (pull request #310)
irs hotfix into main
* irs hotfix into main
* pass top sheet to irs hotfix func
* docstring updates
Approved-by: Alex Galarce
* slight fix to methodology short fix
* Merged in bugfix/all_cnc_hotfixes (pull request #302)
DRAFT: Bugfix/all cnc hotfixes
* Pipeline fix - health plan state
* Update for prompt vs non-prompt fixes
* Revert utils.find_regex_matches()
* Remove prints
* Non-prompt run
* Health Plan State, IRS, NPI, LOB, Lesser, Prov 2, Default, Rate Standard, Contract Effective Date, Term Group
* Merged in bugfix/agreement_name (pull request #294)
contract title fixed
* contract title fixed
* Merged bugfix/all_cnc_hotfixes into bugfix/agreement_name
Approved-by: Katon Minhas
* All hotfixes added
* Fixed utils
* fix for clean_contract_effective_date
* Merged in default-fix (pull request #296)
moved default funcs to the top, dropped intermediate columns
* moved default funcs to the top, dropped intermediate columns
Approved-by: Katon Minhas
* quick fix to dropping extra columns
* Merged in bugfix/effective_date_meridian (pull request #297)
fixed effective date for meridian contracts
* fixed effective date for meridian contracts
Approved-by: Kat…
* Merged in hotfix/irs_tweak (pull request #303)
returning None instead on str(N/A)
* returning None instead on str(N/A)
Approved-by: Katon Minhas
* Merged in bugfix/npi_hotfix (pull request #305)
double filter for duplicates
* double filter for duplicates
Approved-by: Katon Minhas
* Merged in bugfix/all_cnc_hotfixes (pull request #302)
DRAFT: Bugfix/all cnc hotfixes
* Pipeline fix - health plan state
* Update for prompt vs non-prompt fixes
* Revert utils.find_regex_matches()
* Remove prints
* Non-prompt run
* Health Plan State, IRS, NPI, LOB, Lesser, Prov 2, Default, Rate Standard, Contract Effective Date, Term Group
* Merged in bugfix/agreement_name (pull request #294)
contract title fixed
* contract title fixed
* Merged bugfix/all_cnc_hotfixes into bugfix/agreement_name
Approved-by: Katon Minhas
* All hotfixes added
* Fixed utils
* fix for clean_contract_effective_date
* Merged in default-fix (pull request #296)
moved default funcs to the top, dropped intermediate columns
* moved default funcs to the top, dropped intermediate columns
Approved-by: Katon Minhas
* quick fix to dropping extra columns
* Merged in bugfix/effective_date_meridian (pull request #297)
fixed effective date for meridian contracts
* fixed effective date for meridian contracts
Approved-by: Kat…
* Merged in hotfix/irs_tweak (pull request #303)
returning None instead on str(N/A)
* returning None instead on str(N/A)
Approved-by: Katon Minhas
* Merged in bugfix/npi_hotfix (pull request #305)
double filter for duplicates
* double filter for duplicates
Approved-by: Katon Minhas
* Merged in chore/npi (pull request #309)
npi hotfix into main
* npi hotfix into main
* npi hotfix into main
Approved-by: Alex Galarce
* Merged in chore/irs (pull request #310)
irs hotfix into main
* irs hotfix into main
* pass top sheet to irs hotfix func
* docstring updates
Approved-by: Alex Galarce
* Merged in bugfix/clean_short_methodology (pull request #312)
changed regex pattern in get_short_methodology
* changed regex pattern in
Approved-by: Katon Minhas
* Merged in hotfix/filename_fix (pull request #314)
filename corrected
* filename corrected
* Merged main into hotfix/filename_fix
Approved-by: Katon Minhas
* Merge remote-tracking branch 'remotes/origin/main' into HEAD
* rearranged qcqa scripts/funcs and prep for merge w main, moved lots of stuff to postprocess
* slight change to icm_number_fix
* transitioned some qcqa funcs to postprocessing, added fields to qcqa report
* Merged main into QCQA_Integration
* reorganization of qa qc helpers, consolidate_output, qa_qc from Katon's feedback
* adhoc and at scale qcqa ran successfully
* fix to consolidate_output
* Merge remote-tracking branch 'remotes/origin/main' into QCQA_Integration
* Merged main into QCQA_Integration
* small changes to npi_check, changed name of config.CONTRACT_FILE_NAME to config.FILE_NAME_COLUMN
Approved-by: Katon Minhas
New Prompt Method (starting with contract effective date)
* add todos for Siddhant
* moved effective date prompts to prompts.py and added docstring and type hints
* moved effective date prompts to prompts.py and added docstring and type hints
* fixed conflict
Co-authored-by: Alex Galarce <agalarce@aarete.com>
* modified contract effective date logic in existing smart chunking
* Merged main into chore/contract_effective_date
* updated extract_text_from_delimiters function and unit tests for it
* Merge remote-tracking branch 'origin/main' into chore/contract_effective_date
* added comments to preprocessing_funcs
* preprocessing_funcs.py edited online with Bitbucket
* utils.py edited online with Bitbucket
* add debug line
* add NA handling into fix_date_formatting()
* add debug print statements
* worked on review comments
* fixed date checking
* fixed date checking and added unit tests
* Merged main into chore/contract_effective_date
* added logic to handle case when smart chunking returns empty context for effective date
* added comments to file_processing.py
* optimized imports and removed a TODO
* Merged main into chore/contract_effective_date
* made prompt changes for effective date
* keywords.py edited online with Bitbucket
* Removed excess print statements
Approved-by: Katon Minhas
DRAFT: Bugfix/all cnc hotfixes
* Pipeline fix - health plan state
* Update for prompt vs non-prompt fixes
* Revert utils.find_regex_matches()
* Remove prints
* Non-prompt run
* Health Plan State, IRS, NPI, LOB, Lesser, Prov 2, Default, Rate Standard, Contract Effective Date, Term Group
* Merged in bugfix/agreement_name (pull request #294)
contract title fixed
* contract title fixed
* Merged bugfix/all_cnc_hotfixes into bugfix/agreement_name
Approved-by: Katon Minhas
* All hotfixes added
* Fixed utils
* fix for clean_contract_effective_date
* Merged in default-fix (pull request #296)
moved default funcs to the top, dropped intermediate columns
* moved default funcs to the top, dropped intermediate columns
Approved-by: Katon Minhas
* quick fix to dropping extra columns
* Merged in bugfix/effective_date_meridian (pull request #297)
fixed effective date for meridian contracts
* fixed effective date for meridian contracts
Approved-by: Katon Minhas
* Default rate and Lesser fix
* Merged in hotfix/irs_npi_updated (pull request #298)
updated irs and npi hotfix funcs
* updated irs and npi hotfix funcs
* Merged bugfix/all_cnc_hotfixes into hotfix/irs_npi_updated
Approved-by: Katon Minhas
* non-functional commit - default only
* debug commit - premerge
* Batch 1 Dup File Reconciliation
* Default rate fix
* Full CNC 1A Run Config
* Standard column refactor
* Working stitching for 1A
* Merged in Alex-Galarce/cnc_hotfix_effective_date_utilspy-edited-1732659983687 (pull request #301)
Remove "license" from effective date smart chunking
* Remove "license" from effective date smart chunking
* Merged bugfix/all_cnc_hotfixes into Alex-Galarce/cnc_hotfix_effective_date_utilspy-edited-1732659983687
Approved-by: Katon Minhas
* Merged in hotfix/irs_batch2_feedback (pull request #300)
IRS fix for feedback on batch2
* IRS fix for feedback on batch2
* Merged bugfix/all_cnc_hotfixes into hotfix/irs_batch2_feedback
Approved-by: Katon Minhas
* Merged main into bugfix/all_cnc_hotfixes
Approved-by: Katon Minhas
update inputs for smart chunking
* update inputs for smart chunking
* updated test case inputs
* updated test case inputs
* Add keywords back to argument for ac smart chunking
* Test case fix
Approved-by: Katon Minhas
Bugfix/health plan state
* minor bugfix to cover an edge-case
* updated state_check function and unit tests
* Merged main into bugfix/health_plan_state
* Merged main into bugfix/health_plan_state
Approved-by: Katon Minhas
Add cost logging
* add inspect logging
* change comment
* add simple token estimation
* rudimentary costlog
* compute cost
* float formatting
* put costlog to s3
* float formatting attempt 2
* fix formatting, remove a couple prints
* add missing comma
* add filename
* make costlog a .csv instead of a .log
* use csv library instead of from-scratch csv writing
* switch to import math, add test
* pyproject.toml Added src to pythonpath
* Merged main into feature/costlog
* add jupyter to requirements, add test path to pyproject.toml
* Merge remote-tracking branch 'refs/remotes/origin/feature/costlog' into feature/costlog
* Merge remote-tracking branch 'origin/main' into feature/costlog
* merge in main
* add TODOs
* Merged main into feature/costlog
* Merged main into feature/costlog
Approved-by: Michael McGuinness
Approved-by: Katon Minhas
Update for full-context AC chunks to return exact contract_text instead of concatenated text_dict pages
* Update for full-context AC chunks to return exact contract_text instead of concatenated text_dict pages
* Updated test cases
* Updated case 1
* Merged main into bugfix/full_context_check
Approved-by: Alex Galarce
plan state hotfix
* updated state_check function
* added a wrapper function apply_state_check_to_df function to enable state_check function to work with dataframe and also wrote unit tests for both the functions.
* Merged main into hotfix/plan_state
* append extra column with new health plan state corrected
* Merge remote-tracking branch 'refs/remotes/origin/hotfix/plan_state' into hotfix/plan_state
* fix tests
* move apply_state_check_to_df() to cnc_hotfix.py
* move all functionality out of utils and into cnc_hotfix.py
* Merge remote-tracking branch 'origin/main' into hotfix/plan_state
* remove duplicated pythonpath in pyproject.toml
* format cnc_hotfix.py
* changed to use valid.STATE_MAP to not duplicate functionality
Approved-by: Katon Minhas
grouped keywords and IRS regex chunking
* abandon old branch and rebuild
* removed print for field names, prompts
* pushing as part of leftovers
* IRS regex chunking
* Merged in feature/chunk_term_clean (pull request #262)
chunk_log_added
* chunk_log_added
* Merge remote-tracking branch 'remotes/origin/feature_ac_chunking_clean' into feature/chunk_term_clean
Approved-by: Alex Galarce
* added function for regex based IRS chunking
* shifted regex_match_chunk function to utils
* added helper functions for regex based chunking
* replaced tin_regex function to ac_funcs
* removed prompt for irs_others
* added regex chunking for irs
* removed irs group from smart chunking
* testing functionality for regex based irs fields
* updated return N/A condition
* Merged main into feature_ac_chunking_clean
* file_processing.py edited online with Bitbucket
* added back conditional prompts
* minor fixed for PR
* remove install types
* mering clean branch
* with passed test casses
* added flag for regex based tin execution
Approved-by: Michael McGuinness
Katon Minhas