Feature/llm utils refactor
* - Removed some temporary logging code
- Removed unused imports (uuid, datetime) from llm_utils.py
- Added cache token columns (cache_creation_tokens, cache_read_tokens) to CSV exports
- Added tracking for cache warming
- Updated usage_tracking.py to track cache tokens separately in data structure
All token tracking functionality remains intact.
* Split input tokens into fresh and cache in summary section
- Add total_fresh_input_tokens and total_cache_tokens to GLOBAL_USAGE
- Update summary CSV to include split token columns for better tractability
- Add averages for fresh_input_tokens and cache_tokens
- Maintain total_input_tokens = fresh_input_tokens + cache_tokens relationship
- All calculations verified and match per-file totals correctly
* upload files functionality
* Add comprehensive unit tests for usage_tracking module
- Implement Phase 1-4 tests covering all 10 functions in usage_tracking.py
- Add 98 test cases with parametrized tests for comprehensive coverage
- Fix extract_usage_from_response to handle None input gracefully
- Add python-dotenv dependency to pyproject.toml
- Tests cover: core functions, data access, export functions, and utilities
- All tests passing (98/98)
* Add S3 upload tests for main.py and fix f-string syntax error
- Add tests/test_investment_main_s3.py with 3 test cases:
* test_main_writes_final_and_error_to_s3: validates both final and error DataFrames uploaded when processing has errors
* test_main_writes_only_final_when_no_errors: validates only final DataFrame uploaded when all files succeed
* test_main_writes_usage_when_present: validates usage tracking files uploaded when usage data exists
- Tests use monkeypatching and module stubs to isolate S3 write logic without external dependencies
- Fix syntax error in src/codes/code_funcs.py: nested f-string quotes (level_dict['level_suffix'])
- All tests passing
* Add comprehensive test for usage_tracking.get_usage_dataframes S3 uploads
- Add test_usage_dataframes_written_to_s3 to validate per_file_usage_df and batch_summary_df
- Test verifies both dataframes from usage_tracking.get_usage_dataframes() are written to S3
- Validates 'usage' suffix writes per_file_usage_df with correct schema (file_name, tokens, cost)
- Validates 'usage_summary' suffix writes batch_summary_df with correct schema (totals, averages, region_mode)
- Uses pd.testing.assert_frame_equal to ensure exact dataframes are uploaded
- Fix module caching issue between tests by clearing sys.modules cache
- All 4 tests passing
* Add cleanup fixture to prevent test pollution
- Add autouse pytest fixture to clean up stubbed modules after each test
- Remove manual cache clearing from individual tests (now handled by fixture)
- Prevents our module stubs from affecting other test files in CI/CD pipeline
- Fixes Bitbucket pipeline failures where other tests couldn't find llm_utils attributes
* Refactor tests: add comprehensive io_utils coverage; disable usage_tracking suite
- Added 22 focused tests for write_local and write_s3 plus existing IO behaviors
- Introduced FakeS3Client to avoid boto3 network/credential dependency
- Added preserve_config fixture (no monkeypatch) to isolate config side effects
- Marked test_usage_tracking.py skipped per new consolidation approach
- Verified 26 tests (io_utils + investment_main) all pass locally without AWS exceptions
* Fix S3_CLIENT mocking: use mocker.patch instead of direct assignment
- Changed preserve_config fixture to NOT save/restore S3_CLIENT
- Updated all 5 write_s3 tests to use mocker.patch('src.config.S3_CLIENT', fake)
- This prevents pollution of config.S3_CLIENT across test modules
- Fixes NoCredentialsError in other tests that were importing config after our tests set FakeS3Client
- All 22 io_utils tests pass + 4 investment_main_s3 tests pass
- test_llm_utils::test_invoke_claude_local_mode now passes
* Add tests for write_s3 function
* Resolve merge conflict: keep mocker-based write_s3 tests
* Remove rogue skip mark from test_usage_tracking.py
The skip mark was incorrectly added by remote branch claiming tests were
migrated to io_utils. However, test_io_utils.py only tests io_utils functions
(read/write operations), not usage_tracking functions.
Restore the comprehensive usage_tracking tests from commit 26a556f0 which
includes 98 test cases covering all 10 functions in usage_tracking.py.
* Fix test pollution: add config cleanup fixture and restore RUN_MODE
- Add autouse fixture in test_io_utils.py to restore config values after each test
- Fix test_llm_utils.py to restore RUN_MODE after test_invoke_claude_local_mode
- Prevents config patches from leaking into subsequent tests causing NoCredentialsError
* Remove redundant test_investment_main_s3.py
- File was testing same write_s3 functionality already covered in test_io_utils.py
- Was causing module pollution by stubbing src.utils.llm_utils
- Caused test failures in other test files due to module cache issues
- Integration testing was minimal and mocked write_s3 anyway
* Phase 1: Remove unused code and align cost constants
- Remove count_tokens() function (unused, replaced by exact API counts)
- Remove unused cost constants from invoke_claude() and invoke_claude_with_image_array()
- Remove unused math import
- Update usage_tracking.py to match correct cost constants from llm_utils.py:
- Corrected Haiku cache_read: 0.000025 -> 0.00003
- Implemented correct cache creation cost calculation:
* Sonnet models: 25% higher than input (0.00375 vs 0.003)
* Haiku: 20% higher than input (0.0003 vs 0.00025)
- Remove test_count_tokens() test
- Update test_track_usage_single_call() to account for corrected cache creation costs
All 437 tests passing. Token counts verified against previous runs.
* Phase 2: Remove redundant update_stats() tracking
- Remove all update_stats() calls from EC2 mode (3 locations)
- Remove update_stats() function definition
- Remove estimated token counting (tokens_sent, tokens_received)
- Remove elapsed_time calculations
- Remove unused start_time assignments
- Remove test_update_stats() test case
- Clean up setUp() method in test_llm_utils.py
- Update docstrings to reflect usage_tracking.py as source of truth
All token tracking now uses exact API counts via usage_tracking.track_usage().
All 436 tests passing.
* Merge main into feature/llm-utils-refactor
Resolved merge conflicts:
- llm_utils.py: Kept refactored code (removed legacy update_stats, math import)
- usage_tracking.py: Kept corrected cost constants and cache creation pricing logic
- io_utils.py: Merged main's new 'dtc' and 'dtc_summary' output types
- test_usage_tracking.py: Kept corrected test values for cache_read and cache_creation costs
All conflicts resolved while preserving Phase 1 & 2 refactoring work.
* Fix merge conflict artifact in test_usage_tracking.py
Removed leftover merge conflict marker (<<<<<<< HEAD) that was missed
during merge resolution. The test now correctly calculates cache creation
costs for Sonnet (25% higher) and Haiku (20% higher) models.
* Phase 3: Add thread safety and enhance cache key generation
Step 3.1: Add thread safety to cache operations
- Added _cache_lock = threading.Lock() at module level
- Wrapped all cache operations (get, set, move_to_end, popitem) with lock
- All 4 cache operation locations now thread-safe:
* Cache check/retrieval in invoke_claude()
* Cache storage in invoke_claude()
* Cache check/retrieval in invoke_claude_with_image_array()
* Cache storage in invoke_claude_with_image_array()
Step 3.2: Enhance cache key generation
- Updated get_cache_key() to include instruction, max_tokens, and cache flag
- Updated get_cache_key_with_image_array() with same enhancements
- Cache keys now differentiate based on all relevant parameters
- Backward compatible (all new parameters are optional)
Step 3.3: Update call sites
- Updated invoke_claude() to pass instruction, max_tokens, cache to get_cache_key()
- Updated invoke_claude_with_image_array() to pass max_tokens
All 412 tests passing. Cache operations are now thread-saf…
* Phase 4: Add comprehensive tests for cache operations and token tracking
Step 4.1: Remove legacy test cases
- Already completed in Phase 2 (test_update_stats, test_count_tokens removed)
- setUp() cleaned up (no MODEL_STATS/GLOBAL_STATS setup needed)
Step 4.2: Add tests for thread-safe cache operations
- test_cache_thread_safety_concurrent_reads: 10 concurrent cache reads
- test_cache_thread_safety_concurrent_writes: 10 concurrent cache writes
- test_cache_thread_safety_mixed_operations: mixed read/write operations
Step 4.3: Add tests for enhanced cache key generation
- test_get_cache_key_basic: backward compatibility test
- test_get_cache_key_with_instruction: instruction parameter
- test_get_cache_key_with_max_tokens: max_tokens parameter
- test_get_cache_key_with_cache_flag: cache flag
- test_get_cache_key_all_parameters: all parameters together
- test_get_cache_key_with_image_array: image array variant
- test_invoke_claude_uses_enhanced_cache_key: integration test
Step 4.4: Verify token tracking uses e…
* Phase 5: Replace legacy CSV export with usage_tracking.py
Step 5.1: Replace write_stats_to_csv() implementation
- Updated write_stats_to_csv() to use usage_tracking.get_usage_dataframes() instead of config.MODEL_STATS/GLOBAL_STATS
- Added import for usage_tracking module
- Function now reads from usage_tracking.py as the single source of truth
Step 5.2: Add optional run_id and timestamp parameters
- Added optional run_id parameter (defaults to config.BATCH_ID)
- Added optional timestamp parameter (defaults to current timestamp)
- Updated upload_tracking_logs() to pass batch_id and run_timestamp
Step 5.3: Transform usage_tracking format to legacy CSV format
- Aggregates per-file, per-model data to per-file data
- Maps model names to Claude 2/3/3.5 request counts
- Calculates tokens_sent (fresh input tokens) and tokens_received (output tokens)
- Maintains backward compatibility with legacy CSV structure
- Handles empty data gracefully (writes headers-only CSV)
Step 5.4: Verify call sites work with updated f…
* refactor(llm_utils): extract helper functions and reduce code duplication
- Add private helper functions: _track_usage_from_response(), _build_claude_3_request_body(), _store_in_cache()
- Simplify routing logic using model sets (_CLAUDE_2_MODELS, _CLAUDE_3_AND_UP_MODELS)
- Replace 8 instances of usage tracking pattern with single function call
- Replace 3 instances of request body building with helper function
- Replace 2 instances of cache storage with helper function
- Add Claude 4.5 Sonnet to cache support list
- Reduce file size from 1,004 to 860 lines (14.3% reduction, 144 lines removed)
- All tests passing (422 tests)
- Maintains backward compatibility and functionality
* Re-support Claude 3 Haiku and add comprehensive tests for new helper functions
- Re-added Claude 3 Haiku to supported models (removed from deprecated patterns)
- Updated model validation to include Haiku in supported models list
- Added comprehensive test coverage for new helper functions:
- _validate_model_support() - 5 tests covering deprecation, support, and context
- _supports_prompt_cache() - 2 tests for cache support detection
- _build_claude_3_request_body() - 4 tests for request body building with/without cache
- Updated error messages to reflect Claude 3+ models (Haiku, Sonnet 3.5/3.7/4/4.5)
- All 23 tests passing
* Merged main into feature/llm-utils-refactor
* Merged main into feature/llm-utils-refactor
Approved-by: Siddhant Medar
Approved-by: Katon Minhas
Add time-based rate handling rules and update section labels in prompt templates
* Add time-based rate handling rules and update section labels in prompt templates
* Add mapping for MyCare Ohio in crosswalk_program.json and update state mapping for Ohio in crosswalk_program_lob.json
* Update test parameters and state configuration in testing notebook
* Update exhibit-wide payment constraint prompt to exclude administrative processing language
* Merge remote-tracking branch 'origin/main' into bugfix/reimb-term-issues
* Refine exhibit-wide payment constraint prompt to clarify extraction criteria and examples
* Enhance exhibit-wide payment constraint prompt with guidance for abbreviated terms and clarify acceptable constraints
* Update SERVICE_TERM prompt to specify healthcare services and clarify extraction criteria
Approved-by: Katon Minhas
Feature/per file logging
* Add custom per-file logging handler
* Add close method in PerFileHandler to properly close file handlers
* Add return type annotation to setup_per_file_logging function
* Set up per-file logging in main and process_file functions
* Add per-file logging configuration options and update setup logic
* Refactor logging setup in testing notebook to use per-file logging and improve organization
* Clean up whitespace and improve organization in testing notebook
* Merge remote-tracking branch 'origin/main' into feature/per-file-logging
* Enhance per-file logging setup by adding debug level configuration and informative print statement
* Change logging from info to debug level in table processing functions for improved verbosity control
* isort
Approved-by: Katon Minhas
Refactor/one time io
* Delete client work
* Streamline imports
* Fix list class
* Update tests
* Clear code funcs test
* Fix unit tests
* Single read code funcs
* Single-read model
* Single-load embeddings
* Successful E2E test
* update unit tests
* Update importsg
* Reload poetry.lock
* remove old exhibit header function
* Remove aarete_derived generic function
* remove align and format tables
* Remove strings to dict
* references
* Clear code
* Move preprocessing_funcs
* remove keywords
* refactor postprocessing_funcs
* Pass unit test - remove qa_qc directory
* Black and isort
* remove print
Approved-by: Alex Galarce
Table Continuation Fix
* Add EXHIBIT_HEADER_MARKERS for improved document header recognition in table handling
* Refactor EXHIBIT_HEADER_MARKERS definition for improved clarity and organization
* remove unused page span tracking code; add unit tests for new functionality
* remove debugging logs from combine_continuous_tables function
* Refactor logging setup and update test file configuration in investment field testing notebook
* Merge remote-tracking branch 'origin/main' into feature/further-phase-1-and-2-fixes
Approved-by: Katon Minhas
Feature/global lesser of
* Merge remote-tracking branch 'origin/feature/reimb-primary-updates'
* First try at global "lesser of"
* Generate reimb_ids further upstream
* Implement global lesser of injection and extraction logic in file processing
* Update logic to change LESSER_OF_IND in-place for global affected rows; add docstrings
* Refactor global lesser of injection logic to create new rows for REIMB_IDs lacking lesser of logic; update original rows in place and improve tracking flags.
* Merge remote-tracking branch 'origin/main' into feature/global-lesser-of
* Add logging for global lesser of logic injection and update handling of original rows
* Refactor global lesser of logic: move functions to one_to_one_funcs.py and update inject_global_lesser_of_rows to use new implementations
* Move imports for METHODOLOGY_BREAKOUT_QUESTIONS and REIMB_TERM_BREAKOUT to the top of one_to_one_funcs.py for better organization and to remove prototype comments.
* isort
* Refactor import statement for generate_reimb_ids to use the full module path for clarity
* load field from JSON instead of hardcoded values
* Update logging levels in investment_field_testing notebook
* Merged main into feature/global-lesser-of
* Remove unnecessary comment from investment_postprocessing_funcs import statement for clarity
* Merge branch 'feature/global-lesser-of' of https://bitbucket.org/aarete/doczy.ai into feature/global-lesser-of
* Merge remote-tracking branch 'origin/main' into feature/global-lesser-of
* Remove debug print statement from process_file function
* fix call to Field.load_from_file
* Merge remote-tracking branch 'origin/main' into feature/global-lesser-of
* Add tracking flag for global lesser of logic when none is found
* explicitly sort by FILE_NAME, EXHIBIT_PAGE, and REIMB_LESSER_OF_ID columns
* Add new field type, "full_context_separate", to make sure that we don't run it twice. Also formatting fix
* Fix iteration over separate fields in run_full_context_fields function
Approved-by: Katon Minhas
Rework reimbursement ID to associate parts of "lesser of" (or other such coupled statement) with one another
* refactor reimbursement ID generation with hashed service terms
* reduce hash length for unique reimbursement ID generation
* update reimbursement ID format to include 'exh_pg' for clarity
* clean up testing.ipynb
* one more formatting fix to testing.ipynb
* Move reimb_id generation to postprocessing
* move reimb_id function to postprocessing_funcs.py
* refactor generate_reimb_ids to group by FILE_NAME and EXHIBIT_PAGE for ID generation
* clean testing.ipynb further
Approved-by: Katon Minhas
Better visibility and testing functionality for optimizing prompts
* first commit
* Enhance prompts and processing for contract termination dates
- Updated prompt to include 'year to year' as a valid example output.
- Implemented case-insensitive processing for contract text and field keywords for smart chunking.
- Clarified prompt for derived termination date to specify it is exclusive of renewals
* Merge remote-tracking branch 'origin/main' into test/check-doc-dates-against-test-bed
* Update termination date prompt tests
* Refine keywords for contract termination date prompt in investment prompts
* Docs on using the testing notebook
* add 'input_files' option to test_params
* Merge remote-tracking branch 'origin/main' into test/check-doc-dates-against-test-bed
* comment out debug line
* refactor: adjust input_files filtering logic in main function
Approved-by: Katon Minhas
Test/check doc dates against test bed
* first commit
* Enhance prompts and processing for contract termination dates
- Updated prompt to include 'year to year' as a valid example output.
- Implemented case-insensitive processing for contract text and field keywords for smart chunking.
- Clarified prompt for derived termination date to specify it is exclusive of renewals
* Merge remote-tracking branch 'origin/main' into test/check-doc-dates-against-test-bed
* Update termination date prompt tests
* Refine keywords for contract termination date prompt in investment prompts
Approved-by: Katon Minhas
Refactor/daip2-9 code refactor
* add missing import
* refactor ac_smart_chunking.py
* update tests
* removed old and unused imports
* removed outdated import
* forgot to import re
* refactor bottom up funcs
* remove unused imports from file_processing.py
* refactor dict_operations.py
* remove unused import from conditional_funcs.py
* replace top_down_funcs with one_to_n_funcs and remove top_down_funcs.py
* remove duplicate import from one_to_n_funcs.py
* refactor all utils.py imports
* refactor error handling by removing last code in utils and integrating InvalidDateException into postprocessing_funcs
* refactor import in claude_funcs.py to use string_funcs directly and (hopefully) resolve circular import
* refactor regex_funcs.py to use postprocessing_funcs for add_hyphen_if_needed calls
* refactor hotfix_helper_funcs.py to use postprocessing_funcs for add_hyphen_if_needed calls
* refactor postprocess.py to remove unused imports
* add numpy import to string_funcs
* fix tests after utils refactor
* move adhoc from tests to scripts
* remove unused imports
* remove scripts from mypy checking
* isort
* Merged main into refactor/daip-2-9-code-refactor
Approved-by: Katon Minhas
Feature/self repair main
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* new testing
* new testing
* added unit tests under fieldExtraction/tests/main-unit-tests
* unit test fixes
* unit test fixes
* merge into main
* modified consolidate_output and slight changes to qa_qc_helpers to accomodate qaqc functionality w faizan and michael's changes
* merging self repair into main
* synced with main now
* fixes
* provider billing removed
* provider billing removed
* provider billing fixed
* commented out load dotenv in config
* Merged main into feature/self-repair-main
* modified check_s3_bucket_exists function by returning False if credentials not found for pipeline
* Merge branch 'feature/self-repair-main' of https://bitbucket.org/aarete/doczy.ai into feature/self-repair-main
Approved-by: Michael McGuinness
QCQA Additions
* tested on batch 10, small fixes + change to s3 output
* Merged main into QCQA_Additions
* unit tests for qc qa helpers
* static error fix
* another static fix
* static fix again
* fixed None check
* static fix none
* Merged main into QCQA_Additions
* Small changes based on Katon's review
Approved-by: Katon Minhas
QCQA Integration
* Merged in bugfix/npi_hotfix (pull request #305)
double filter for duplicates
* double filter for duplicates
Approved-by: Katon Minhas
* Merged in chore/npi (pull request #309)
npi hotfix into main
* npi hotfix into main
* npi hotfix into main
Approved-by: Alex Galarce
* Merged in chore/irs (pull request #310)
irs hotfix into main
* irs hotfix into main
* pass top sheet to irs hotfix func
* docstring updates
Approved-by: Alex Galarce
* slight fix to methodology short fix
* Merged in bugfix/all_cnc_hotfixes (pull request #302)
DRAFT: Bugfix/all cnc hotfixes
* Pipeline fix - health plan state
* Update for prompt vs non-prompt fixes
* Revert utils.find_regex_matches()
* Remove prints
* Non-prompt run
* Health Plan State, IRS, NPI, LOB, Lesser, Prov 2, Default, Rate Standard, Contract Effective Date, Term Group
* Merged in bugfix/agreement_name (pull request #294)
contract title fixed
* contract title fixed
* Merged bugfix/all_cnc_hotfixes into bugfix/agreement_name
Approved-by: Katon Minhas
* All hotfixes added
* Fixed utils
* fix for clean_contract_effective_date
* Merged in default-fix (pull request #296)
moved default funcs to the top, dropped intermediate columns
* moved default funcs to the top, dropped intermediate columns
Approved-by: Katon Minhas
* quick fix to dropping extra columns
* Merged in bugfix/effective_date_meridian (pull request #297)
fixed effective date for meridian contracts
* fixed effective date for meridian contracts
Approved-by: Kat…
* Merged in hotfix/irs_tweak (pull request #303)
returning None instead on str(N/A)
* returning None instead on str(N/A)
Approved-by: Katon Minhas
* Merged in bugfix/npi_hotfix (pull request #305)
double filter for duplicates
* double filter for duplicates
Approved-by: Katon Minhas
* Merged in bugfix/all_cnc_hotfixes (pull request #302)
DRAFT: Bugfix/all cnc hotfixes
* Pipeline fix - health plan state
* Update for prompt vs non-prompt fixes
* Revert utils.find_regex_matches()
* Remove prints
* Non-prompt run
* Health Plan State, IRS, NPI, LOB, Lesser, Prov 2, Default, Rate Standard, Contract Effective Date, Term Group
* Merged in bugfix/agreement_name (pull request #294)
contract title fixed
* contract title fixed
* Merged bugfix/all_cnc_hotfixes into bugfix/agreement_name
Approved-by: Katon Minhas
* All hotfixes added
* Fixed utils
* fix for clean_contract_effective_date
* Merged in default-fix (pull request #296)
moved default funcs to the top, dropped intermediate columns
* moved default funcs to the top, dropped intermediate columns
Approved-by: Katon Minhas
* quick fix to dropping extra columns
* Merged in bugfix/effective_date_meridian (pull request #297)
fixed effective date for meridian contracts
* fixed effective date for meridian contracts
Approved-by: Kat…
* Merged in hotfix/irs_tweak (pull request #303)
returning None instead on str(N/A)
* returning None instead on str(N/A)
Approved-by: Katon Minhas
* Merged in bugfix/npi_hotfix (pull request #305)
double filter for duplicates
* double filter for duplicates
Approved-by: Katon Minhas
* Merged in chore/npi (pull request #309)
npi hotfix into main
* npi hotfix into main
* npi hotfix into main
Approved-by: Alex Galarce
* Merged in chore/irs (pull request #310)
irs hotfix into main
* irs hotfix into main
* pass top sheet to irs hotfix func
* docstring updates
Approved-by: Alex Galarce
* Merged in bugfix/clean_short_methodology (pull request #312)
changed regex pattern in get_short_methodology
* changed regex pattern in
Approved-by: Katon Minhas
* Merged in hotfix/filename_fix (pull request #314)
filename corrected
* filename corrected
* Merged main into hotfix/filename_fix
Approved-by: Katon Minhas
* Merge remote-tracking branch 'remotes/origin/main' into HEAD
* rearranged qcqa scripts/funcs and prep for merge w main, moved lots of stuff to postprocess
* slight change to icm_number_fix
* transitioned some qcqa funcs to postprocessing, added fields to qcqa report
* Merged main into QCQA_Integration
* reorganization of qa qc helpers, consolidate_output, qa_qc from Katon's feedback
* adhoc and at scale qcqa ran successfully
* fix to consolidate_output
* Merge remote-tracking branch 'remotes/origin/main' into QCQA_Integration
* Merged main into QCQA_Integration
* small changes to npi_check, changed name of config.CONTRACT_FILE_NAME to config.FILE_NAME_COLUMN
Approved-by: Katon Minhas
Feature/hotfix stitch
* Add hotfix scripts
* Merged main into feature/hotfix_stitch
* Moved stitching scripts to scripts folder
* Update for 2
* Fixed flat fee issue
* Fix payer name issue
* All hotfix updates
* Merged main into feature/hotfix_stitch
* Add rapidfuzz
Approved-by: Alex Galarce
DRAFT: Bugfix/all cnc hotfixes
* Pipeline fix - health plan state
* Update for prompt vs non-prompt fixes
* Revert utils.find_regex_matches()
* Remove prints
* Non-prompt run
* Health Plan State, IRS, NPI, LOB, Lesser, Prov 2, Default, Rate Standard, Contract Effective Date, Term Group
* Merged in bugfix/agreement_name (pull request #294)
contract title fixed
* contract title fixed
* Merged bugfix/all_cnc_hotfixes into bugfix/agreement_name
Approved-by: Katon Minhas
* All hotfixes added
* Fixed utils
* fix for clean_contract_effective_date
* Merged in default-fix (pull request #296)
moved default funcs to the top, dropped intermediate columns
* moved default funcs to the top, dropped intermediate columns
Approved-by: Katon Minhas
* quick fix to dropping extra columns
* Merged in bugfix/effective_date_meridian (pull request #297)
fixed effective date for meridian contracts
* fixed effective date for meridian contracts
Approved-by: Katon Minhas
* Default rate and Lesser fix
* Merged in hotfix/irs_npi_updated (pull request #298)
updated irs and npi hotfix funcs
* updated irs and npi hotfix funcs
* Merged bugfix/all_cnc_hotfixes into hotfix/irs_npi_updated
Approved-by: Katon Minhas
* non-functional commit - default only
* debug commit - premerge
* Batch 1 Dup File Reconciliation
* Default rate fix
* Full CNC 1A Run Config
* Standard column refactor
* Working stitching for 1A
* Merged in Alex-Galarce/cnc_hotfix_effective_date_utilspy-edited-1732659983687 (pull request #301)
Remove "license" from effective date smart chunking
* Remove "license" from effective date smart chunking
* Merged bugfix/all_cnc_hotfixes into Alex-Galarce/cnc_hotfix_effective_date_utilspy-edited-1732659983687
Approved-by: Katon Minhas
* Merged in hotfix/irs_batch2_feedback (pull request #300)
IRS fix for feedback on batch2
* IRS fix for feedback on batch2
* Merged bugfix/all_cnc_hotfixes into hotfix/irs_batch2_feedback
Approved-by: Katon Minhas
* Merged main into bugfix/all_cnc_hotfixes
Approved-by: Katon Minhas
grouped keywords and IRS regex chunking
* abandon old branch and rebuild
* removed print for field names, prompts
* pushing as part of leftovers
* IRS regex chunking
* Merged in feature/chunk_term_clean (pull request #262)
chunk_log_added
* chunk_log_added
* Merge remote-tracking branch 'remotes/origin/feature_ac_chunking_clean' into feature/chunk_term_clean
Approved-by: Alex Galarce
* added function for regex based IRS chunking
* shifted regex_match_chunk function to utils
* added helper functions for regex based chunking
* replaced tin_regex function to ac_funcs
* removed prompt for irs_others
* added regex chunking for irs
* removed irs group from smart chunking
* testing functionality for regex based irs fields
* updated return N/A condition
* Merged main into feature_ac_chunking_clean
* file_processing.py edited online with Bitbucket
* added back conditional prompts
* minor fixed for PR
* remove install types
* mering clean branch
* with passed test casses
* added flag for regex based tin execution
Approved-by: Michael McGuinness
Praneel Panchigar