Bugfix/unify prompt exhibit header
* Remove dead prompt_exhibit_header overrides and legacy get_exhibit_pages
Investigation (AST call tree analysis) proved both were dead code:
1. get_exhibit_pages() in preprocessing_funcs.py had ZERO callers in the
entire codebase. It was superseded by get_exhibit_pages_new() which is
the only live path (called by preprocess.one_to_n_exhibit_chunking).
2. Client prompt_exhibit_header overrides (2-arg) in bcbs_promise and
clover prompt_calls.py were dead for two independent reasons:
a) Their only consumer (get_exhibit_pages) has zero callers
b) The live consumer (get_exhibit_pages_new) calls with 3 args —
if the resolver returned the client 2-arg function, Python would
raise TypeError. The SaaS 3-arg version was always running.
Verification:
- Programmatic AST call tree (local_scripts/call_tree_analysis.py)
confirms zero callers and signature mismatch
- Resolver test confirms all clients resolve to SaaS 3-arg implementation
- Full unit suite: 1077 passed, 0 failures
- E2E: BCBS P…
* Merged dev into bugfix/unify_prompt_exhibit_header
* Add dev dependencies for local code index
Add tree-sitter, lancedb, and tiktoken as dev dependencies for a local
codebase indexing system. The index provides hybrid search (symbol table,
call graph, semantic embeddings, BM25) for token-efficient code exploration.
The index lives in a gitignored .index/ folder — each developer builds
their own locally.
Approved-by: Katon Minhas
Bugfix/implicit cpt update
* DAIP2-2310: enrich service term before implicit code extraction
- Add SERVICE_ENRICHMENT prompt + enrich_service_for_codes() that runs
after explicit extraction and before the implicit pipeline, stripping
provider/facility/contractual noise while preserving billing codes.
- Tighten CODE_IMPLICIT_ARBITRATION instructions to require strict
synonymy and prefer no_match over best-effort guesses.
- Remove single-candidate auto-accept bypass so all implicit candidates
go through LLM arbitration.
- Update unit tests for new arbitration path and enrichment mock.
* DAIP2-2310: refine service enrichment and track implicit mapping source
- Service enrichment prompt: bare 'Inpatient'/'Outpatient' place-of-service
rows now collapse to 'Covered Services' to prevent spurious Level 1 matches.
When paired with a real procedure (e.g. 'Outpatient Surgery'), the place
modifier is preserved.
- code_implicit_rag: record per-match mapping origin (CPT_LEVEL1,
HCPCS_LEVEL1, CPT_LEVEL2, HCPCS_LEVEL2) on the code answer dict for
debugging which mapping produced each implicit match.
- constants: add TODO next to HCPCS_LEVEL1_MAPPING load flagging the
upcoming DME code range change.
* Merged dev into DAIP2-2310-implicit-cpt-update
* DAIP2-2310: fix 3-digit revenue code extraction and enrichment exclusion rule
Two related fixes surfaced during short-circuit analysis of the NICU rows:
1. 3-digit revenue codes were being dropped by validate_explicit_codes,
causing the explicit path to return empty and the row to fall through
to implicit RAG (which would then semantic-match against CPT/HCPCS
and produce 'Specific - No Match'). Root causes:
- _revenue_code_format_valid required exactly 4 digits, rejecting the
common contract form like 'Revenue Code 173'.
- REV_MAPPING keys are loaded as ints by pandas (leading zero stripped
from all-numeric CSV columns), so even after accepting 3-digit form
the mapping lookup missed.
Fixes in src/codes/code_funcs.py:
- _revenue_code_format_valid now accepts 3 or 4 digits.
- New _normalize_revenue_code helper returns canonical 4-digit form.
- validate_explicit_codes stores the normalized form in REVENUE_CD.
- _has_unmatched_codes normalizes before membership chec…
* Merged dev into bugfix/implicit-cpt-update
* DAIP2-2310: accept revenue code wildcards and ranges; add unit tests
Extends the earlier 3-digit revenue code fix to cover two more forms the
CODE_EXPLICIT prompt instructs the LLM to return:
1. X-suffix wildcards (e.g. '17X', '173X') - standard UB-04 shorthand
meaning any digit across a revenue family. The prompt explicitly says
'may end in X' but the validator was rejecting them, causing the same
silent drop + implicit-RAG fall-through as the 3-digit bug.
2. Ranges (e.g. '173-174', '0170-0179') - the CODE_EXPLICIT prompt
instructs ranges in 'LOW-HIGH' form. Procedure codes support ranges
but revenue codes did not. The Gulf Coast 'Rev Codes 173-174' rows
were landing on Specific - No Match because of this.
New helpers in src/codes/code_funcs.py:
- _revenue_wildcard_format_valid / _revenue_range_format_valid /
_revenue_code_any_format_valid: targeted format checks.
- _normalize_revenue_any: canonical form for any valid kind
(single code -> '0173', wildcard -> '017X', range -> '0173-0…
* DAIP2-2310: prevent enrichment from hallucinating or dropping descriptors
Two related prompt changes to SERVICE_ENRICHMENT_INSTRUCTION based on
review feedback:
1. Add a 'no information not in input' rule. The enrichment is a
faithful rewrite that strips noise — not a knowledge-augmented
expansion. The LLM must NOT look up what a code means and inject
the description, infer category names from codes, or translate
medical shorthand into clinical terms not present in the input.
2. Add a 'preserve descriptive labels' rule. Things like 'Level 1',
'Level 2-5', 'Tier A', 'Class III', 'low complexity' etc. are
meaningful service descriptors written by the contract author and
must remain in the output even when codes are also present. The
prior ER few-shot example showed the LLM stripping 'Level 1'..
'Level 5' labels — that's information loss.
Updated few-shot examples to match the new rules:
- Emergency Room example output now preserves 'Level 1 - 99281,
Level 2 - 99282, ...' verbati…
Approved-by: Katon Minhas
DAIP2-2346 update client prompt calls
* DAIP2-2346: retire accidental client prompt_calls drift
Delete 14 client prompt_calls overrides in bcbs_promise and clover that were
stale forks of old SaaS implementations. Shared callers all expect SaaS return
shapes; client shims were either redundant with parser-side normalization or
outright broken dead code (prompt_dynamic_assignment unpack crash,
prompt_lesser_of_check double-parse). Resolver shim at
src/pipelines/shared/prompts/prompt_calls.py now falls through to SaaS.
Preserved with annotations:
- validate_reimbursements_for_llm: real business rule (strict YES equality)
- prompt_exhibit_header: required by 2-arg call site at preprocessing_funcs.py:192
* DAIP2-2346: also retire prompt_exhibit_level and prompt_dynamic overrides
Follow-up to the Bucket A cleanup. Both client overrides only drifted by
omitting the optional field_names kwarg on prompt_templates.EXHIBIT_LEVEL.
SaaS passes field_names and gets the format-aware parser
(_create_json_dict_parser); falling through to SaaS is a strict improvement
in normalization. Client files now contain only the two truly
client-specific overrides.
* Changes to gitignore
* Merged dev into DAIP2-2346-update-client-prompt-calls
Approved-by: Katon Minhas
Feature/DAIP2-1575 set up dev prod uat branches
* Add branch promotion gates and release pipelines
* Add resolve_source_branch.sh script and update .gitignore; fix pipeline step name casing 'dev'
* Add AI code review step and refactor branch resolution in pipelines
* Merged dev into feature/DAIP2-1575-set-up-dev-prod-uat-branches-
* Merged dev into feature/DAIP2-1575-set-up-dev-prod-uat-branches-
* Merged dev into feature/DAIP2-1575-set-up-dev-prod-uat-branches-
Approved-by: Katon Minhas
Feature/new output format
* bugfix
* Merge branch 'DEV' into Optimize/DAIP2-1474-restructure-postprocess
* requested postprocessing changes
* prompt changes reverted
* fix pipeline issues
* fix pipeline issues
* fix pipeline issues
* fixed formatting
* fixed formatting
* Merge branch 'DEV' into Optimize/DAIP2-1474-restructure-postprocess
* Merge branch 'DEV' into Optimize/DAIP2-1474-restructure-postprocess
* save dashboard and cc output separately
* save dashboard output in s3
* pipeline error fixed
* json list through postprocessing
* Merged DEV into Optimize/DAIP2-1474-restructure-postprocess
* Merge remote-tracking branch 'origin/Optimize/DAIP2-1474-restructure-postprocess' into feature/new_output_format
* Restructure output file organization and add standard field sanitization
Output Structure Changes:
- Reorganize output files into hierarchical directory structure:
- full_outputs/cc_results/ for consolidated CC results
- full_outputs/dashboard_results/ for consolidated dashboard results
- full_outputs/ for error files
- automation_qa-qc/ for QC/QA validated results and statistics
- parent-child/ for parent-child mapping outputs
- tracking/ for usage and cost tracking data
- individual/ for per-file CC results (dashboard individual files removed)
- Update file naming conventions to match new structure
- Remove QC/QA processing for error files (error files are saved without validation)
Post-Processing Changes:
- Add standard N/A value cleaning: remove placeholder values (N/A, UNKNOWN, etc.)
when they are the only value in a cell (applies before CC/dashboard split)
- Normalize all _IND fields to contain only 'Y' or 'N' values (no blanks)
- Ensure standard cleaning runs before splitting into CC …
* Ran Black for CI
* Refactor file splitting logic and add comprehensive tests
- Refactored splitting logic in io_utils.py:
- Consolidated repeated splitting code into two focused helper functions:
- _write_local_split_files() for local file writing with splitting
- _write_s3_split_files() for S3 file writing with splitting
- Both helpers use shared split_dataframe_by_filename() function
- Added MAX_ROWS_PER_SPLIT configuration (default: 70000) in config.py
- Added comprehensive test coverage for splitting logic:
- Tests for split_dataframe_by_filename() with various scenarios
- Tests for local and S3 write operations with single and multiple splits
- Tests for cc_results_full, dashboard_results_full, and qc_qa_cc_full output types
- Fixed existing test failures (write_s3 error handling, path assertions)
- Improved code maintainability and readability
* Fix failing tests in test_postprocess.py
- Updated standard_postprocess tests to use actual columns from FIELD_FORMAT_MAPPING
(PAYER_NAME, CONTRACT_TITLE) instead of custom test columns that get dropped
- Added FILE_NAME column to all file structure test DataFrames (required for splitting logic)
- Added MAX_ROWS_PER_SPLIT mock configuration for splitting tests
- Fixed patch decorators for S3 tests to properly mock logging
All 41 tests now passing.
* Black for CI
* Blank [] and ['[]'] in output instead of displaying them
- Add placeholder patterns in clean_na_values for [], ['[]'], ["[]"]
- Update format_as_json_list to return blank for empty lists instead of []
- Filter out empty-list placeholder items from list values in format_as_json_list
- Add tests for clean_na_values empty list handling and format_as_json_list
Co-authored-by: Cursor <cursoragent@cursor.com>
* Merged DEV into feature/new_output_format
* Pipeline config, parent-child, dashboard, and runner fixes
- Parent-child: enable by default, write to run directory, S3 upload via io_utils
- Dashboard: optional (CC only by default), run_dashboard=True to enable
- Add io_utils.upload_local_file_to_s3 for centralized file uploads
- Parent-child returns (row_count, local_path); pipeline handles S3 upload
- Add TODO in config for WRITE_PC_TO_S3 removal after approval
- Fix indentation errors in runner.py
Co-authored-by: Cursor <cursoragent@cursor.com>
Approved-by: Katon Minhas
Feature/lesser table caching refactor hybrid
* chore: Remove unused duplicate main.py from shared pipeline
* fix: Correct crosswalk paths in aarete_derived.py
* chore: Remove unused documentation files from fieldExtraction
* docs: Add documentation files to documentation folder
* docs: Update README with uv setup, expanded project structure, and branching conventions
* docs: Add uv installation steps with Ubuntu/WSL emphasis
* Enable prompt caching for all remaining LLM calls
- Add _INSTRUCTION() functions for: EXHIBIT_HEADER, EXHIBIT_LINKAGE,
EXHIBIT_TITLE_MATCH, DATE_FIX, DERIVED_TERM_DATE, CHECK_PROVIDER_NAME_MATCH,
SPECIAL_CASE_ASSIGNMENT
- Update all invoke_claude() calls in saas and clover pipelines to use
cache=True with corresponding _INSTRUCTION() functions
- Add new instructions to get_cacheable_instructions() for cache warming
- Update tests for new instruction functions
Functions now using caching:
- prompt_exhibit_level
- prompt_exhibit_lesser (EXHIBIT_LEVEL_LESSER_OF)
- prompt_fee_schedule_breakout
- prompt_grouper_breakout
- prompt_special_case_assignment
- prompt_exhibit_linkage
- prompt_exhibit_header
- prompt_smart_chunked (ONE_TO_ONE templates)
- prompt_date_fix
- prompt_derived_term_date
- prompt_exhibit_title_match
- provider_name_match_check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Reorder
* feat: Add bcbs_promise client pipeline with OFFSET_TERM extraction
- Add new bcbs_promise client with HSC-based OFFSET_TERM field extraction
- Extract full paragraph text of offset/recoupment provisions from contracts
- Derive OFFSET_INDICATOR (Y/N) from OFFSET_TERM presence
- Fix reorder_columns to preserve extra columns not in COLUMN_ORDER
- Update QC/QA output path to outputs/qc_qa/
* fix: Update dev deps and test assertions for QC/QA output path
- Add pytest/pytest-mock to dev dependencies for mypy type checking
- Update test assertions to expect outputs/qc_qa instead of qa_qc_output
* style: Apply black formatting to prompt_templates.py
* Merge main, move scripts
* Archive some scripts
* update py version
* remove .py version file
* Remove ASCII characters
* Restore testbed code
* restore tracking
* Update testbed metrics
* Enable prompt caching for CODE_LAST_CHECK, FILL_BILL_TYPE, DUAL_LOB_CHECK, and GROUPER_BREAKOUT
- Add CODE_LAST_CHECK_INSTRUCTION() for service specificity classification
- Add FILL_BILL_TYPE_INSTRUCTION() for bill type code determination
- Add DUAL_LOB_CHECK_INSTRUCTION() for Medicare/Medicaid classification
- Update code_funcs.py to use caching for CODE_LAST_CHECK, FILL_BILL_TYPE, GROUPER_BREAKOUT
- Update postprocessing_funcs.py to use caching for DUAL_LOB_CHECK
- Add new instructions to get_cacheable_instructions() for cache warming
- Add unit tests for new instruction functions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix postprocessing_funcs to remove invalid columns
* Merge branch 'main' into feature/lesser-table-caching-refactor-hybrid
* Revert prompt caching changes from aed1b73c
* update formatting
* Update imports
Approved-by: Sha Brown
Approved-by: Praneel Panchigar
Feature/multithreading
* fixes
* Merge main into feature/multithreading
Resolved conflicts:
- Kept timing instrumentation in file_processing.py
- Kept new 3-step Exhibit-based approach for one-to-n processing
- Maintained parallelization improvements (20 workers)
Changes include:
- Timing utils integration for performance monitoring
- Increased max_workers from 5 to 20 across all components
- Parallelized code_breakout and grouper_breakout
- Fixed tin_npi_funcs function call parameters
* Fix max_workers error for empty documents
- Add check to skip parallel processing when no pages exist
- Use min(len(all_page_tasks), 20) to prevent max_workers=0
- Handles edge case of documents with no exhibits or pages
* Fix max_workers=0 errors in one_to_n_funcs
- Add checks before all ThreadPoolExecutor creations
- Prevents errors when processing empty lists:
- carveout_and_special_case
- breakout
- special_case_breakout
- filter_services_without_reimbursements
- run_lob_relationship
- Ensures executor only created when there are items to process
* Reduce code processing parallelism to prevent API throttling
Lower max_workers from 20 to 10 for code_breakout and grouper_breakout
to prevent overwhelming Bedrock API with concurrent requests
* fixed
* Merged main into feature/multithreading
* Move documentation into folder
* Fix logging statements
* Merge branch 'main' into feature/multithreading
* Refactor for clarity
* Update previous exhibit passing logic
* properly simplify exhibits
* Merged main into feature/multithreading
* update conditional for None
* exhibit multithreading changed
* exhibit multithreading changed
* Merge remote-tracking branch 'origin/main' into feature/multithreading
* synced with main
* Parallelize dynamic assignment, refactor HSC field worker, add timing/exhibit unit tests, tidy imports/ignore helpers
* analyze_regression.py edited online with Bitbucket
* count_pages.py edited online with Bitbucket
* compare_regressed_with_baseline.py edited online with Bitbucket
* simple_testbed_compare.py edited online with Bitbucket
* run_testbed_metrics_regressed.py edited online with Bitbucket
Approved-by: Katon Minhas
Fix LOB_PROGRAM_RELATIONSHIP extraction and preserve debugging code
* Fix LOB_PROGRAM_RELATIONSHIP extraction and preserve debugging code
- Add format requirements to LOB_RELATIONSHIP_INSTRUCTION for proper caching
- Strengthen LOB_RELATIONSHIP prompt with explicit pipe delimiter requirements
- Preserve all DEBUG blocks and print statements in dynamic_funcs.py and file_processing.py
- Update crosswalk mappings and gitignore
- Ensure format instructions are cached at API level to prevent missing pipe delimiters
* Merge main into bugfix/dynamic_issuefixes-clean and remove all debug print statements
* Remove redundant one_to_one_fields initialization to match main
Approved-by: Katon Minhas
Feature/automated cost logging csv
* feat: Add usage and cost tracking with CSV export
- Add usage_tracking.py module for thread-safe token and cost tracking
- Extract actual tokens from Bedrock API responses (replaces word count estimation)
- Integrate usage tracking into llm_utils.py for all LLM invocations
- Add CSV export functionality (per-file/per-model and batch summary)
- Integrate CSV export into main.py for local runs
- Update .gitignore to exclude PRD
Phase 1: Data collection and local CSV export implemented
- Tracks all LLM calls including utility functions (date_fix, derive_term_date)
- Calculates costs per model using centralized cost constants
- Generates two CSV files: USAGE.csv and USAGE-SUMMARY.csv
- Exports to same directory as final results when write_to_s3=False
* - Removed some temporary logging code
- Removed unused imports (uuid, datetime) from llm_utils.py
- Added cache token columns (cache_creation_tokens, cache_read_tokens) to CSV exports
- Added tracking for cache warming
- Updated usage_tracking.py to track cache tokens separately in data structure
All token tracking functionality remains intact.
* Split input tokens into fresh and cache in summary section
- Add total_fresh_input_tokens and total_cache_tokens to GLOBAL_USAGE
- Update summary CSV to include split token columns for better tractability
- Add averages for fresh_input_tokens and cache_tokens
- Maintain total_input_tokens = fresh_input_tokens + cache_tokens relationship
- All calculations verified and match per-file totals correctly
* upload files functionality
* Add comprehensive unit tests for usage_tracking module
- Implement Phase 1-4 tests covering all 10 functions in usage_tracking.py
- Add 98 test cases with parametrized tests for comprehensive coverage
- Fix extract_usage_from_response to handle None input gracefully
- Add python-dotenv dependency to pyproject.toml
- Tests cover: core functions, data access, export functions, and utilities
- All tests passing (98/98)
* Add S3 upload tests for main.py and fix f-string syntax error
- Add tests/test_investment_main_s3.py with 3 test cases:
* test_main_writes_final_and_error_to_s3: validates both final and error DataFrames uploaded when processing has errors
* test_main_writes_only_final_when_no_errors: validates only final DataFrame uploaded when all files succeed
* test_main_writes_usage_when_present: validates usage tracking files uploaded when usage data exists
- Tests use monkeypatching and module stubs to isolate S3 write logic without external dependencies
- Fix syntax error in src/codes/code_funcs.py: nested f-string quotes (level_dict['level_suffix'])
- All tests passing
* Add comprehensive test for usage_tracking.get_usage_dataframes S3 uploads
- Add test_usage_dataframes_written_to_s3 to validate per_file_usage_df and batch_summary_df
- Test verifies both dataframes from usage_tracking.get_usage_dataframes() are written to S3
- Validates 'usage' suffix writes per_file_usage_df with correct schema (file_name, tokens, cost)
- Validates 'usage_summary' suffix writes batch_summary_df with correct schema (totals, averages, region_mode)
- Uses pd.testing.assert_frame_equal to ensure exact dataframes are uploaded
- Fix module caching issue between tests by clearing sys.modules cache
- All 4 tests passing
* Add cleanup fixture to prevent test pollution
- Add autouse pytest fixture to clean up stubbed modules after each test
- Remove manual cache clearing from individual tests (now handled by fixture)
- Prevents our module stubs from affecting other test files in CI/CD pipeline
- Fixes Bitbucket pipeline failures where other tests couldn't find llm_utils attributes
* Refactor tests: add comprehensive io_utils coverage; disable usage_tracking suite
- Added 22 focused tests for write_local and write_s3 plus existing IO behaviors
- Introduced FakeS3Client to avoid boto3 network/credential dependency
- Added preserve_config fixture (no monkeypatch) to isolate config side effects
- Marked test_usage_tracking.py skipped per new consolidation approach
- Verified 26 tests (io_utils + investment_main) all pass locally without AWS exceptions
* Fix S3_CLIENT mocking: use mocker.patch instead of direct assignment
- Changed preserve_config fixture to NOT save/restore S3_CLIENT
- Updated all 5 write_s3 tests to use mocker.patch('src.config.S3_CLIENT', fake)
- This prevents pollution of config.S3_CLIENT across test modules
- Fixes NoCredentialsError in other tests that were importing config after our tests set FakeS3Client
- All 22 io_utils tests pass + 4 investment_main_s3 tests pass
- test_llm_utils::test_invoke_claude_local_mode now passes
* Add tests for write_s3 function
* Resolve merge conflict: keep mocker-based write_s3 tests
* Remove rogue skip mark from test_usage_tracking.py
The skip mark was incorrectly added by remote branch claiming tests were
migrated to io_utils. However, test_io_utils.py only tests io_utils functions
(read/write operations), not usage_tracking functions.
Restore the comprehensive usage_tracking tests from commit 26a556f0 which
includes 98 test cases covering all 10 functions in usage_tracking.py.
* Fix test pollution: add config cleanup fixture and restore RUN_MODE
- Add autouse fixture in test_io_utils.py to restore config values after each test
- Fix test_llm_utils.py to restore RUN_MODE after test_invoke_claude_local_mode
- Prevents config patches from leaking into subsequent tests causing NoCredentialsError
* Remove redundant test_investment_main_s3.py
- File was testing same write_s3 functionality already covered in test_io_utils.py
- Was causing module pollution by stubbing src.utils.llm_utils
- Caused test failures in other test files due to module cache issues
- Integration testing was minimal and mocked write_s3 anyway
* Merged main into feature/automated-cost-logging-csv
Approved-by: Karan Desai
Approved-by: Katon Minhas
Reimbursement Exhibit Prefiltering
* Add IDENTIFY_REIMBURSEMENT_EXHIBITS function to analyze exhibit headers for reimbursement information
* Add Zone.Identifier to .gitignore to prevent Windows zone information files from being tracked
* Add functionality to identify reimbursement exhibits
* Add functionality to identify reimbursement exhibits in run_one_to_n_prompts
* Output YAML parsing
* Refactor identify_reimbursement_exhibits to improve YAML parsing and error handling
* New reimbursement exhibit logic
* Update IDENTIFY_REIMBURSEMENT_EXHIBITS_PROMPT to return PAGE NUMBERS instead of exhibit headers
* Fix identify_reimbursement_exhibits to return page numbers as strings after YAML parsing
* Remove debugging print statement for reimbursement exhibits in run_one_to_n_prompts
* Remove commented debugging print statements in identify_reimbursement_exhibits
* Increase read_timeout in EC2 configuration from 2000 to 4000
* Merge remote-tracking branch 'origin/main' into feature/reimb-primary-improvements
* Remove debugging print statements in get_reimbursement_primary function
* mypy error
* Fix return type in identify_reimbursement_exhibits to ensure keys are returned as a list
* Merge remote-tracking branch 'origin/main' into feature/reimb-primary-improvements
Approved-by: Katon Minhas
Feature/load crosswalks
* allow .json in fieldExtraction/crosswalk
* clean up gitignores, add rule to allow .jsons in crosswalk, create first crosswalk
* create mappings directory
* add to_csv functionality for MCS to check results
* Merged main into feature/load_crosswalks
Approved-by: Katon Minhas
DRAFT: Bugfix/all cnc hotfixes
* Pipeline fix - health plan state
* Update for prompt vs non-prompt fixes
* Revert utils.find_regex_matches()
* Remove prints
* Non-prompt run
* Health Plan State, IRS, NPI, LOB, Lesser, Prov 2, Default, Rate Standard, Contract Effective Date, Term Group
* Merged in bugfix/agreement_name (pull request #294)
contract title fixed
* contract title fixed
* Merged bugfix/all_cnc_hotfixes into bugfix/agreement_name
Approved-by: Katon Minhas
* All hotfixes added
* Fixed utils
* fix for clean_contract_effective_date
* Merged in default-fix (pull request #296)
moved default funcs to the top, dropped intermediate columns
* moved default funcs to the top, dropped intermediate columns
Approved-by: Katon Minhas
* quick fix to dropping extra columns
* Merged in bugfix/effective_date_meridian (pull request #297)
fixed effective date for meridian contracts
* fixed effective date for meridian contracts
Approved-by: Katon Minhas
* Default rate and Lesser fix
* Merged in hotfix/irs_npi_updated (pull request #298)
updated irs and npi hotfix funcs
* updated irs and npi hotfix funcs
* Merged bugfix/all_cnc_hotfixes into hotfix/irs_npi_updated
Approved-by: Katon Minhas
* non-functional commit - default only
* debug commit - premerge
* Batch 1 Dup File Reconciliation
* Default rate fix
* Full CNC 1A Run Config
* Standard column refactor
* Working stitching for 1A
* Merged in Alex-Galarce/cnc_hotfix_effective_date_utilspy-edited-1732659983687 (pull request #301)
Remove "license" from effective date smart chunking
* Remove "license" from effective date smart chunking
* Merged bugfix/all_cnc_hotfixes into Alex-Galarce/cnc_hotfix_effective_date_utilspy-edited-1732659983687
Approved-by: Katon Minhas
* Merged in hotfix/irs_batch2_feedback (pull request #300)
IRS fix for feedback on batch2
* IRS fix for feedback on batch2
* Merged bugfix/all_cnc_hotfixes into hotfix/irs_batch2_feedback
Approved-by: Katon Minhas
* Merged main into bugfix/all_cnc_hotfixes
Approved-by: Katon Minhas
Feature/add dotenv
* transferred sensitive information to .env file from src/config.py
* replace model IDs, AWS account info
* Merge remote-tracking branch 'origin/main' into feature/add-dotenv
* add explanatory comment
* Merge remote-tracking branch 'origin/main' into feature/add-dotenv
* remove tests that were just for local
Approved-by: Katon Minhas
grouped keywords and IRS regex chunking
* abandon old branch and rebuild
* removed print for field names, prompts
* pushing as part of leftovers
* IRS regex chunking
* Merged in feature/chunk_term_clean (pull request #262)
chunk_log_added
* chunk_log_added
* Merge remote-tracking branch 'remotes/origin/feature_ac_chunking_clean' into feature/chunk_term_clean
Approved-by: Alex Galarce
* added function for regex based IRS chunking
* shifted regex_match_chunk function to utils
* added helper functions for regex based chunking
* replaced tin_regex function to ac_funcs
* removed prompt for irs_others
* added regex chunking for irs
* removed irs group from smart chunking
* testing functionality for regex based irs fields
* updated return N/A condition
* Merged main into feature_ac_chunking_clean
* file_processing.py edited online with Bitbucket
* added back conditional prompts
* minor fixed for PR
* remove install types
* mering clean branch
* with passed test casses
* added flag for regex based tin execution
Approved-by: Michael McGuinness
Cnc fl main
* prov type moved
* comment out invalid prov2 bc it's overly restrictive
* Update prompt template to ensure N/A rather than blanks
* Update AC multi-prompt template. Update prompts to ensure all keys are present and N/As are filled
* Update AC postprocessing
* Update clean_tin() to add N/A for values other than 10 digits
* update tin postprocessing
* Update npi postprocessing
* Add 'Filename:' to the start of all filenames
* Update group TIN/NPI postprocessing
* Update parent agreement code
* fix health plan state mapping
* Update B postprocessing
* Updated indicators, re-ordering
* Update postprocessing - all columns included and reordered
* Ensure Pages is included in AC-only output
* Remove excess print statements
* remove example_test to pass pipeline
* Fix parent agreement code for AC
* Updated postprocessing for NPI/TIN other
* Update prompts for N explicitely on AC prompt-based indicators
* Updated main and utils for more intuitive filtering
* Updated file_counting to match new main process
* update test
* Add example_test
Approved-by: Michael McGuinness
Siddhant Medar