Merged in dev (pull request #1001)
* Merged in feature/fixPlaceholder2 (pull request #979)
Updated feature> dev gate to print variables of echo statements
* Updated feature> dev gate to print variables of echo statement
* Reverted placeholder changes
Approved-by: Sujit Deokar
* Merged in bugfix/prov_info_json_fixes (pull request #981)
fix: fall back to PROVIDER_NAME in PROV_INFO_JSON when no TIN/NPI extracted
* fix: fall back to PROVIDER_NAME in PROV_INFO_JSON when no TIN/NPI extracted
When get_prov_info_json short-circuits due to no TIN/NPI regex matches,
PROV_INFO_JSON was left as [] even when PROVIDER_NAME was successfully
extracted via the one-to-one pipeline. This caused inconsistent output
across contracts with the same provider — some files produced a NAME-only
entry (via a false-positive regex hit triggering the LLM), others produced [].
Reconcile at add_group_and_other, the first point where both extraction
streams' results are available. When PROV_INFO_JSON is empty but
PROVIDER_NAME is known, synthesize a NAME-only entry with IS_GROUP:"Y" and
populate PROV_GROUP_NAME_FULL directly — skipping the provider_name_match_check
LLM call since the match is tautological by construction.
Adds 5 unit tests covering the str, list, already-populated, empty-name,
and all-empty-lis…
* Merged in feature/DAIP2-pacificsource-reimbursements-issues (pull request #982)
Feature/DAIP2 pacificsource reimbursements issues
* Tighten PREMIUM_TERM and DISCOUNT_TERM classifier prompts
CARVEOUT_CHECK was misrouting table rate rows into special-case fields,
dropping them from the reimbursement output:
- "110% of CMS allowed" (base fee-schedule rates) was being classified as
PREMIUM_TERM because the prompt treated "above 100% of reference" as an
implicit premium. Seen on PacificSource Medicare_Attachment_A1 and A2
Facility contracts where Inpatient/Outpatient rows were missing (2556)
or silently fell back to 100% fee-schedule (2557).
- Per-service discount rates like "Progressive Lenses: 15% discount",
"Contact Lenses: 2% discount", "Frame: 20% discount" were being
classified as DISCOUNT_TERM because the prompt only required the word
"discount" to appear. Seen on PacificSource Commercial_Attachment_A2
and A5 Professional contracts (2558, 2559).
Prompts now require the literal keyword …
* Merged in bugfix/filter-docusign-lines (pull request #983)
remove docusign lines
* remove docusign lines
* add unit tests for clean_header_footer docusign/deleted_lines changes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* black forrmatting
Approved-by: Katon Minhas
* Merged in feature/PC_logic_cleanup_output (pull request #984)
Feature/PC logic cleanup output
* Few tweaks PC_logics
* Fixed orphan_ranking
* black formatting
* Changes on output field and ranking method
* updated few hotfixes
* black format fix
Approved-by: Katon Minhas
* Merged in hotfix/provider_name_group_fix (pull request #988)
Hotfix/provider name group fix
* Fixes done in GROUP column
* black format fix
Approved-by: Katon Minhas
* Merged in bugfix/DAIP2-2524-carveout-code-optimization (pull request #980)
CARVEOUT_CD issue fixed
* CARVEOUT_CD issue fixed
* pipeline error fixed
* Merged dev into bugfix/DAIP2-2524-carveout-code-optimization
* Merged dev into bugfix/DAIP2-2524-carveout-code-optimization
* Merged dev into bugfix/DAIP2-2524-carveout-code-optimization
* trigger cap issue fixed
* trigger cap prompt updated
* Merged dev into bugfix/DAIP2-2524-carveout-code-optimization
Approved-by: Katon Minhas
* Merged in bugfix/exhibit-smart-chunking-cost-improvements (pull request #985)
Bugfix/exhibit smart chunking cost improvements
* Add opt-in instrumentation for per-call token and row-count tracing
Introduce src/utils/instrumentation.py (thread-safe CSV logger) and
src/utils/instrumentation_context.py (ContextVar scope plus
submit_with_context / map_with_context helpers for propagating context
into ThreadPoolExecutor workers).
Emit events at every Bedrock call in llm_utils.invoke_claude, including
in-memory claude_cache hits, with full input/output/cache-read/cache-write
token breakdown. Emit row-count events at each row-mutating stage in the
one-to-N pipeline (clean_reimbursement_primary,
filter_services_without_reimbursements, methodology_breakout,
split_service_terms, carveout, dynamic_code_assignment,
lesser_of_distribution, dynamic_assignment) and chunking / retrieval
events in exhibit smart chunking (chunking_done, retrieval_done) plus
exhibit lifecycle events (exhibit_start, exhibit_gate_skip,
stage_…
* Merged in hotfix/fileextension_issue (pull request #989)
Hotfix/fileextension issue
* fixed strip_ext issue
* black format
Approved-by: Katon Minhas
* Merged in hotfix/exhibit-header-in-tables (pull request #987)
Hotfix/exhibit header in tables
* Merged in feature/FixplaceholderIssue (pull request #977)
Remove curly braces from echo statements in dev->stg
* Remove curly brances from echo statements in dev->stg
* Removed curly braces in echo statements in feature-> dev gate
Approved-by: Sujit Deokar
* Merged in feature/standardized-services (pull request #958)
Feature/standardized services
* service term standardization
* prompt update
* prompt updates for standardization
* only service standardization
* new file
* add supporting files and test scripts for standardization work
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* remove old files
* Merge remote-tracking branch 'origin/dev' into feature/standardized-services
* final fixes
* Merged dev into feature/standardized-services
* additional features
* removed unwanted files
* remove unwanted files
* Merge branch 'dev' into feature/standardized-services
* Merge remote-tr…
* Merged in hotfix/postprocess_csv (pull request #990)
Hotfix/postprocess csv
* date issue_fix
* black_format
* Merged dev into hotfix/postprocess_csv
Approved-by: Katon Minhas
* Merged in bugfix/molina_ut_dynamic_primary (pull request #991)
Bugfix/molina ut dynamic primary
* Prompt changes for dynamic primary
* Route cover-sheet-only files to ERRORS.csv instead of leaking phantom rows
When every page of a contract was filtered out as a cover sheet / quick-review
form, process_file silently returned a FILE_NAME-only DataFrame. Because the
runner routes by checking for an "error" column, that file landed in
RESULTS.csv as a near-empty row and no ERRORS.csv was generated for the run.
- saas/file_processing.py: raise ValueError when text_dict is empty after
cover-sheet filtering, so safe_process_file produces a proper error row.
- runner.py: add _is_phantom_result defense-in-depth — promote any result
with no extracted fields beyond FILE_NAME to error_results with
error_type=PhantomSuccess.
* Merged dev into bugfix/molina_ut_dynamic_primary
* Tighten PRODUCT prompt: restrict to valid_values, prune LOB/PROGRAM examples
* Merge branch 'bugfix/molina_ut_dynamic_primary' of bit…
* Merged in bugfix/black-format (pull request #994)
Black format for pipeline pass
* Black format for pipeline pass
* Merged in bugfix/DAIP2-2679-fix-nebraska-issues (pull request #995)
incorrect inclusion of CPT4_PROC_CD fixed
* incorrect inclusion of CPT4_PROC_CD fixed
Approved-by: Katon Minhas
* Merged in feature/DAIP2-2314-DAIP2-1687-hybrid (pull request #993)
Feature/DAIP2-2314 DAIP2 1687 hybrid
* remove -files from s3 prefix requirements
* Resolve input paths
* fix: VendorProcessor.process_file returns (df, None) tuple
runner.safe_process_file unpacks the result as (cc_df, dashboard_df), so
returning a single DataFrame caused every vendor/generic file to fail with
"too many values to unpack (expected 2)" — Python iterates DataFrame columns
during unpacking. Vendor pipelines have no dashboard variant; second slot is
None and the existing `dashboard_result is not None` guard in runner.py
already handles it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* DAIP2-2314 + DAIP2-1687: pad DYNAMIC_PRIMARY + DYNAMIC_PRIMARY_ENTITY_CLASSIFICATION over 1024-token cache floor
- Pad DYNAMIC_PRIMARY_INSTRUCTION with three new sections: [SCOPE BOUNDARIES], [SOURCE TEXT INTERPRETATION], [REASONING DISCIPLINE], plus a [WORKED EXAMPLES] block. Estimated tokens: 447 -> 1117 (Sonnet 4.5 …
* Merged in bugfix/postprocessing_date_fix (pull request #996)
Date formatting changes
* Date formatting changes
* Merged dev into bugfix/postprocessing_date_fix
Approved-by: Katon Minhas
* Merged in bugfix/parent-child-rank-orphan-uniqueness (pull request #999)
PC_logic bugfix
* PC_logic bugfix
Approved-by: Katon Minhas
* Merged in feature/document-index (pull request #1005)
Feature/document index
* Add Document Index preprocessing — Layers 1, 2, and 3 wiring
Parse the Textract-emitted Document Index block at the top of each contract
with a single cached LLM call (prompt_document_index) instead of one per-page
call per page. Layer 2 verifies parsed entries via literal string match and
structural regex sweep, escalating suspect pages back to the existing per-page
path. Layer 1+2 failure triggers a full fallback to today's per-page flow.
New symbols:
- preprocessing_funcs.extract_document_index_block — regex slice of index prefix
- preprocessing_funcs.verify_index_against_pages — structural verifier (plain dict return)
- prompt_templates.DOCUMENT_INDEX_INSTRUCTION / DOCUMENT_INDEX — cached prompt pair
- prompt_calls.prompt_document_index — LLM wrapper (usage_label DOCUMENT_INDEX_PARSE)
- config: DOCUMENT_INDEX_PARSE_ENABLED and three threshold flags
- instrumentation: DOCUMENT_INDEX_PARSE mapped to preprocessing segment
one…
* Merged in feature/active-rates (pull request #1004)
Feature/active rates
* initial commit
* Merged in feature/FixplaceholderIssue (pull request #977)
Remove curly braces from echo statements in dev->stg
* Remove curly brances from echo statements in dev->stg
* Removed curly braces in echo statements in feature-> dev gate
Approved-by: Sujit Deokar
* Merged in feature/standardized-services (pull request #958)
Feature/standardized services
* service term standardization
* prompt update
* prompt updates for standardization
* only service standardization
* new file
* add supporting files and test scripts for standardization work
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* remove old files
* Merge remote-tracking branch 'origin/dev' into feature/standardized-services
* final fixes
* Merged dev into feature/standardized-services
* additional features
* removed unwanted files
* remove unwanted files
* Merge branch 'dev' into feature/standardized-services
* Merge remote-track…
* Merged in feature/one-to-one-confidence-scoring (pull request #1002)
Feature/one to one confidence scoring
* T1 plumbing: capture per-field confidence + retrieved-chunk metadata for 1:1 HSC fields
Prep work for the 1:1 confidence-scoring stage. No scoring logic yet — this
just collects the inputs the next ticket (rule-based scorer) will consume.
- ONE_TO_ONE_SINGLE_FIELD_TEMPLATE: ask the LLM for confidence (0.0-1.0),
verdict (correct/uncertain/not_found), and supporting_snippet alongside
the field value. Existing field parser passes the extra keys through
unchanged.
- prompt_hsc_single_field: now returns a 4-tuple (name, value, field,
metadata) where metadata holds the confidence/verdict/snippet plus a
lightweight summary of which chunks the LLM saw (count + ids).
_extract_hsc_metadata is defensive: clamps out-of-range confidences,
defaults a missing/garbage verdict, caps the snippet at 500 chars,
returns _empty_hsc_metadata() on every bail-out path.
- run_hybrid_smart_chunked_fields: opt…
* Merged in bugfix/confidence-flagged-missing-field-col (pull request #1006)
Fix KeyError in compute_flagged when a *_CONF column has no value sibling
* Fix KeyError in compute_flagged when a *_CONF column has no value sibling
Production hit a hard crash at the end of every run:
KeyError: "['DYNAMIC_PRIMARY_ENTITIES'] not in index"
src/qc_qa/confidence/summary.py:143
compute_flagged was iterating over *_CONF columns and unconditionally
indexing the dataframe with both the FILE_NAME column and the stripped
value column. That assumed every <FIELD>_CONF column has a sibling
<FIELD> value column in final_df. That isn't always true: dynamic-primary
features carry only the _CONF side (their value side is dropped by
reorder_columns since it isn't in FIELD_FORMAT_MAPPING but its _CONF
suffix matches the explicit _CONF carve-out). When the model produced a
below-threshold score for one of these and the value column was absent,
pandas .loc raised KeyError and the runner crashed.
Fix:
- Build the .loc colu…
* Merged in feature/active-rates (pull request #1007)
Feature/active rates
* amendment intent tag
* prompt update
* Merge remote-tracking branch 'origin/dev' into feature/active-rates
* Merge remote-tracking branch 'origin/dev' into feature/active-rates
* Merge branch 'dev' into feature/active-rates
* exhibit standardization updates
* use AARETE_DERIVED_EXHIBIT_TITLE for intent
* active rates stuff
* Merge branch 'dev' into feature/active-rates
* prompt update
* amendment intent types
* active rates logic update
* unit tests
* Merged dev into feature/active-rates
* logic updates
* Merge remote-tracking branch 'origin/dev' into feature/active-rates
* Merge remote-tracking branch 'origin/dev' into feature/active-rates
* updated instrumentation cost logs
* cost loging
* caching updates
* Merge remote-tracking branch 'origin/dev' into feature/active-rates
* Merge remote-tracking branch 'origin/dev' into feature/active-rates
* logging fix
* null check issue fixes
* caching fix
Approved-by…
* Merged in bugfix/stg-to-main-prep (pull request #1012)
Bugfix/stg to main prep
* Merged in dev (pull request #1001)
Dev
* Revert premature merge of bugfix/retire_stale_client_file_processing
PR #959 was merged into dev without approval. This reverts commits
5e143c10, 63f32c41, 849aa626, and 927abcae to restore dev to its
pre-merge state. The changes will be re-submitted via a new PR
after proper review.
* Merged in bugfix/retire_stale_client_file_processing (pull request #961)
Return None for dashboard output when dashboard postprocessing is off
* Return None for dashboard output when dashboard postprocessing is off
FINAL_RESULT_DF_DASHBOARD was initialized as an empty DataFrame even
when RUN_DASHBOARD_POSTPROCESSING was False, causing downstream code
to needlessly process it (reorder_columns, etc). Now returns None
when dashboard is not requested, matching the postprocess() contract.
* Merged dev into bugfix/retire_stale_client_file_processing
* Merge dev (with revert) into feature branch
* Re-ap…
* Merged in bugfix/sync-stg-into-dev-20260518 (pull request #1015)
Merged in dev (pull request #1001)
* Merged in dev (pull request #1001)
Dev
* Revert premature merge of bugfix/retire_stale_client_file_processing
PR #959 was merged into dev without approval. This reverts commits
5e143c10, 63f32c41, 849aa626, and 927abcae to restore dev to its
pre-merge state. The changes will be re-submitted via a new PR
after proper review.
* Merged in bugfix/retire_stale_client_file_processing (pull request #961)
Return None for dashboard output when dashboard postprocessing is off
* Return None for dashboard output when dashboard postprocessing is off
FINAL_RESULT_DF_DASHBOARD was initialized as an empty DataFrame even
when RUN_DASHBOARD_POSTPROCESSING was False, causing downstream code
to needlessly process it (reorder_columns, etc). Now returns None
when dashboard is not requested, matching the postprocess() contract.
* Merged dev into bugfix/retire_stale_client_file_processing
* Merge dev (with revert) into fe…
* Merged in dev (pull request #1016)
Dev
* Merged in bugfix/DAIP2-2823-generic-issue-fixes-one-to-one (pull request #1011)
Bugfix/DAIP2-2823 generic issue fixes one to one
* updated auto renewal ind prompt
* updated CONTRACT_AMENDMENT_NUM prompt
* Merged dev into bugfix/DAIP2-2823-generic-issue-fixes-one-to-one
Approved-by: Praneel Panchigar
Approved-by: Siddhant Medar
* Merged in feature/DAIP2-2698-phase-3-program-product-lob-mapping (pull request #1000)
Feature/DAIP2-2698 phase 3 program product lob mapping
* added missing phase 2 modifications
* added phase 3 modifications
* Fixed acronym issues
* black format fix
* Merged dev into feature/DAIP2-2698-phase-3-program-product-lob-mapping
* standardization fixes
* Merged dev into feature/DAIP2-2698-phase-3-program-product-lob-mapping
* added hyphenated suffix names fix
* black format fix
* standardization fix
* dynamic primary fix
* Merge Dev into feature/DAIP2-2698-phase-3-program-product-lob-mapping
* updated Program Product Standardiza…
Approved-by: Praneel Panchigar
DEV
* Apply formatting fix
* Merged in DAIP2-1898-one-to-one-issues-filename-tin-auto-renewal-and-contract-title (pull request #888)
DAIP2-1898 one to one issues filename tin auto renewal and contract title
* contract title fixes
* Add instruction to strip hyphens from TIN and NPI
* prompt update
* Merge branch 'bugfix/auto-renewal-term' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* removed print statements
* pipeline error fixed
* signed_ind logic changed
* signatory_ind logic updated
* prompt update
* prompt update
* pipeline error fixed
* Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* requested changes applied
* test case added
* prompt update
* contract title prompt update
* prompt update
* Merge branch 'DEV' into DAIP2-1898-one-to-one-issues-filename-tin-auto-renewal-and-contract-title
Approved-by: Siddhant Medar
* Merged in feature/DAIP2-1803-add-aarete-derived-provider-name (pull request #884)
Feature/DAIP2-1803 add aarete derived provider name
* Merged DEV into feature/DAIP2-1562-add-aarete_derived_payer_name
* state logic added in clustering
* removed print statements
* pull request updates
* black format fix
* Merged DEV into feature/DAIP2-1562-add-aarete_derived_payer_name
* added scalability feature and optimization
* black format
* Pull Request Changes
* generalised funcs from aarete derived payer name
* updated remove states and updated docstring for build similarity matrix
* derived payer name functionality added
* config and main file changes
* sync dev into ADD AARETE DERIVED PROVIDER GROUP NAME FULL
* updated derived_provider name
* pipeline fixes
* Merged DEV into feature/DAIP2-1803-add-aarete-derived-provider-name
* config changes
* field name updated to AARETE_DERIVED_PROVIDER_NAME
* Merged DEV into feature/DAIP2-1803-add-aarete-derived-provider-name
* black format fix
* Fix docstr…
* Merged in improve-logging (pull request #886)
Improve logging
* Add timing blocks for comprehensive pipeline logging
- Add timing_utils.timed_block() to key extraction functions in one_to_n_funcs.py:
- exhibit_level sub-functions (prompt_exhibit_level, dynamic_primary, etc.)
- reimbursement_level extraction and cleaning
- breakout functions (methodology_breakout, special_case_breakout)
- carveout_and_special_case parallel processing
- one_to_n_cleaning sub-steps (crosswalk, lob_relationship, split_reimb_dates)
- Add timing to dynamic_funcs.py:
- dynamic_assignment parallel processing
- Add timing to postprocess.py:
- standard_postprocess, contract_config_postprocess, dashboard_postprocess
- Add timing to preprocess.py:
- exhibit_chunking sub-steps (get_exhibit_pages, link_exhibit_pages, chunk_by_exhibit)
- Add info-level logging statements for completed operations with row/item counts
* Standardize logging levels and remove deprecated code
DAIP2-1798: Ensure logging statements are at …
* Merged in bugfix/DAIP2-1913-lesser-of-indicator-issues (pull request #889)
Bugfix/DAIP2-1913 lesser of indicator issues
* contract title fixes
* Add instruction to strip hyphens from TIN and NPI
* prompt update
* Merge branch 'bugfix/auto-renewal-term' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* removed print statements
* pipeline error fixed
* signed_ind logic changed
* signatory_ind logic updated
* prompt update
* prompt update
* pipeline error fixed
* Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* requested changes applied
* test case added
* prompt update
* contract title prompt update
* prompt update
* Merge branch 'DEV' into DAIP2-1898-one-to-one-issues-filename-tin-auto-renewal-and-contract-title
* handle nested cases of lesser_of and greater_of
* Merge branch 'DEV' into bugfix/DAIP2-1913-lesser-of-indicator-issues
* pi…
* Merged in bugfix/reimb_primary (pull request #890)
Update reimbursement prompts for consistent PMPM handling, time-unit dedup, and validation improvements
* Update reimbursement prompts for consistent PMPM handling, time-unit dedup, and validation improvements
- REIMBURSEMENT_PRIMARY: append unit of measure (PMPM, per visit, etc.) from table headers into REIMB_TERM
- METHODOLOGY_BREAKOUT: collapse mathematically equivalent time-unit rates into single entry
- VALIDATE_REIMBURSEMENTS: add umbrella terms as valid service terms, accept CPT/HCPCS percentage as concrete rate, narrow COB disqualifier to primary subject only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Approved-by: Siddhant Medar
* Merged in feature/update-postprocessing-columns (pull request #893)
move column order to final step
* move column order to final step
* Black
Approved-by: Siddhant Medar
* Merged in bugfix/update-postprocessing-paths (pull request #896)
Update postprocessing paths
* Update postprocessing paths
Approved-by: Siddhant Medar
* Merged in feature/TIN_stats_update (pull request #898)
Feature/TIN stats update
* Tin_stats_report
* black formatting
* minor changes
* black format applied
* Addressed the katons comments
* Merged DEV into feature/TIN_stats_update
* black reformat
Approved-by: Katon Minhas
* Merged main into DEV
* Merged in feature/exhibit-smart-chunking (pull request #883)
Feature/exhibit smart chunking
* exhibit processing per page
* header dict deduplication
* dedup prompt refinment
* refinment for header extraction proecess
* Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking
* merge updates
* minor fix
* prompt fix for reimb type
* COB defenition for clear understanding
* black formatting
* remove quit statement
* pipiline test
* pipeline test
* black formatting
* Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking
* black formating
* Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking
* typo
* PR comment fixes
* exhibit funcs refactored
* black formatting
* Refactor exhibit chunking config into dedicated class
Created ExhibitChunkingConfig class to centralize exhibit smart chunking
configuration parameters (DEFAULT_SUBCHUNK_SIZE, MIN_PARENT_CHUNK_SIZE,
CHUNK_RELEVANCE_THRESHOLD). This improves code organization …
* Merged in bugfix/DAIP2-1870-dynamic-issues (pull request #892)
Bugfix/DAIP2-1870 dynamic issues
* updated list format appending
* fill bill type from claim type cd
* Fixed remaining format fixes
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* changed crosswalk mapping from PRODUCT to AARETE_DERIVED_PRODUCT
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* updating LOB field values as well from PRODUCT AND PROGRAM
* updated apply crosswalk for reverse mapping
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* added QHP and BHP
* removed lob reverse mapping
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* Remove Bill Type/Claim Type fill
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* Black format
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
Approved-by: Katon Minhas
* Merged in bugfix/prov_info_json_fixes (pull request #899)
Bugfix/prov info json fixes
* fix: robust PROV_INFO_JSON sanitization and TIN backfill logic
json_utils:
- Add sanitize_prov_info_json with layered parsing (JSON, literal_eval,
empty-value-after-colon fix, best-effort dict extraction).
- Add _normalize_prov_entries and _prov_value_to_str for uniform
str-valued output; flatten list values, strip TIN hyphens.
- format_prov_info_json now delegates to sanitize_prov_info_json.
postprocessing_funcs:
- Add fill_prov_info_tin_from_filename_tin for TIN backfill.
- Add validate_and_reformat_date (pipe-wrapped, datetime strings).
- Add format_as_json_list (pipe-delimited, comma-separated, quote
stripping).
postprocess:
- Integrate new postprocessing helpers into pipeline flow.
postprocess_existing_output:
- Support CSV and Excel input, configurable paths, fillna for CSV.
tests:
- Add test_json_parsers.py for PROV_INFO_JSON parsing coverage.
- Add test_postprocess.py for date/list formatting and defau…
* Merged in bugfix/reimb_primary_issues (pull request #900)
Bugfix/reimb primary issues
* Update lesser of distribution prompt to preserve reimb term langauge
* Added prompt change to extract correct service when there are multiple sub-rates; Updated validate reimb term to prevent filtering out erm with valid rate
* Make reimb primary prompt change
* Revert "Make reimb primary prompt change"
This reverts commit 55f7d73b9cb523097c96ea50b6614c1db53b46d2.
Approved-by: Katon Minhas
* Merged in dtc_report (pull request #902)
Dtc report
* dtc_report_added
* lint format fixed
* Changed regex location call
Approved-by: Katon Minhas
* Merged in feature/historical-cost-analysis (pull request #901)
Feature/historical cost analysis
* Initial commit
* Finalize cost_analysis
* Black format
* Merged DEV into feature/historical-cost-analysis
* Refactor
* Black
* Merged DEV into feature/historical-cost-analysis
Approved-by: Siddhant Medar
* Merged in feature/ai-pr-review-agent (pull request #906)
Add AI code review agent to pull request pipeline
* Add AI code review agent to pull request pipeline
Adds a pull-requests pipeline that runs CI checks (lint, type check,
unit tests) followed by an AI code review step. The AI review uses
OIDC to authenticate with AWS Bedrock - no hardcoded AWS keys. It
clones the code-review-agent repo and posts a review comment on the PR.
* Fix AI code review step: install git in python:3.12-slim image
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Remove debug token decode line
* Update guide: all checklist items complete, agent working
* Remove guide file from repo
* Trigger pipeline
Approved-by: Katon Minhas
* Merged in DAIP2-1980-aarete-derived-effective-dt-issues (pull request #904)
DAIP2-1980 aarete derived effective dt issues
* fixed earlier of a and b
* prompt update
* prompt shortened
* test changes reverted
* Merge branch 'DEV' into DAIP2-1980-aarete-derived-effective-dt-issues
* test changes reverted
* Merged DEV into DAIP2-1980-aarete-derived-effective-dt-issues
* Fix effective date prompt
* Replace em-dashes with readable version
Approved-by: Katon Minhas
* Merged in feature/DAIP2-2023-eliminate-full-context-processing (pull request #905)
Feature/DAIP2-2023 eliminate full context processing
* testing full context fields
* remove full context processing
* merge Dev with DAIP2-2-23
* full context removal in client codes
* AARETE_DERIVED_PROVIDER_NAME field changes
* Merged DEV into feature/DAIP2-2023-eliminate-full-context-processing
* optimized provider name
* black format fix
* contract title fixes
* PAYER NAME AUTO RENEWAL IND fixes
* Merged DEV into feature/DAIP2-2023-eliminate-full-context-processing
* Merge branch 'DEV' into feature/DAIP2-2023-eliminate-full-context-processing
* Remove prints
Approved-by: Katon Minhas
* Merged in feature/code_optimization (pull request #903)
Feature/code optimization
* Ran Black
* made a small change in code_last_check, fixed so it returns string and not single char
* Made changes to make sure that default_ind postprocess only happens to the cc output and not dashboard
* Merge bugfix/default_ind_postprocess into feature/code_optimization
- Parent-child: child_rank column init and cols_to_keep filter
- Postprocess: default_ind only on cc output
- code_last_check: parser str/list return handling + exception logging
- Prompt: FIELD ASSIGNMENT for code extraction (no CRITICAL prefix)
* More prompt changes and debugging print statements
* Added system to group same service term + bill type cd + claim type cd and then parallelize the code extraction (will improve consistency)
* Revert one-time PROV_INFO_JSON ad hoc logic; retain json_utils and output format
- Remove temporary postprocess step that filled empty TIN in PROV_INFO_JSON from
FILENAME_TIN (fill_prov_info_tin_from_filename_ti…
* Merged in DAIP2-2121-prov-info-fields-issue-fixes (pull request #908)
empty prov_info_json fixed
* empty prov_info_json fixed
* Merged DEV into DAIP2-2121-prov-info-fields-issue-fixes
* format of PROV_OTHER_TIN fixed
* Revert "format of PROV_OTHER_TIN fixed"
This reverts commit 580847de03265520b1b4fad473bbf38ef1573753.
* formatting and deduplication
* pipeline error fixed
* test cases added
* pipeline error fixed
Approved-by: Katon Minhas
* Merged in feature/context-caching (pull request #909)
Feature/context caching
* Initial commit - context caching for DYNAMIC_PRIMARY
* implement context caching for all relevant prompts
* Remove option to not context cache
* IndentationError fixed
* Merge branch 'DEV' into feature/context-caching
* Merge and format
* Move documentation
* Merged DEV into feature/context-caching
* Update unit tests
* Merged DEV into feature/context-caching
* Update signatures
* Fix test coverage gap
Approved-by: Praneel Panchigar
Approved-by: Karan Desai
* Merged in bugfix/DAIP2-2138-reimbursement-primary-testing (pull request #910)
Bugfix/DAIP2-2138 reimbursement primary testing
* updated table instructions for reimb term extraction
* updated split tables to include header rows
* issue fix
* Merged DEV into bugfix/DAIP2-2138-reimbursement-primary-testing
* black format fix
Approved-by: Katon Minhas
* Merged in feature/doczy_reports (pull request #911)
Feature/doczy reports
* Added post_doczy_reports
* black format fix
* Merged DEV into feature/doczy_reports
* fixed code review
Approved-by: Katon Minhas
* Merged in feature/median-cost-analysis (pull request #912)
Add median
* Add median
Approved-by: Siddhant Medar
Approved-by: Siddhant Medar
Feature/lesser table caching refactor hybrid
* chore: Remove unused duplicate main.py from shared pipeline
* fix: Correct crosswalk paths in aarete_derived.py
* chore: Remove unused documentation files from fieldExtraction
* docs: Add documentation files to documentation folder
* docs: Update README with uv setup, expanded project structure, and branching conventions
* docs: Add uv installation steps with Ubuntu/WSL emphasis
* Enable prompt caching for all remaining LLM calls
- Add _INSTRUCTION() functions for: EXHIBIT_HEADER, EXHIBIT_LINKAGE,
EXHIBIT_TITLE_MATCH, DATE_FIX, DERIVED_TERM_DATE, CHECK_PROVIDER_NAME_MATCH,
SPECIAL_CASE_ASSIGNMENT
- Update all invoke_claude() calls in saas and clover pipelines to use
cache=True with corresponding _INSTRUCTION() functions
- Add new instructions to get_cacheable_instructions() for cache warming
- Update tests for new instruction functions
Functions now using caching:
- prompt_exhibit_level
- prompt_exhibit_lesser (EXHIBIT_LEVEL_LESSER_OF)
- prompt_fee_schedule_breakout
- prompt_grouper_breakout
- prompt_special_case_assignment
- prompt_exhibit_linkage
- prompt_exhibit_header
- prompt_smart_chunked (ONE_TO_ONE templates)
- prompt_date_fix
- prompt_derived_term_date
- prompt_exhibit_title_match
- provider_name_match_check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Reorder
* feat: Add bcbs_promise client pipeline with OFFSET_TERM extraction
- Add new bcbs_promise client with HSC-based OFFSET_TERM field extraction
- Extract full paragraph text of offset/recoupment provisions from contracts
- Derive OFFSET_INDICATOR (Y/N) from OFFSET_TERM presence
- Fix reorder_columns to preserve extra columns not in COLUMN_ORDER
- Update QC/QA output path to outputs/qc_qa/
* fix: Update dev deps and test assertions for QC/QA output path
- Add pytest/pytest-mock to dev dependencies for mypy type checking
- Update test assertions to expect outputs/qc_qa instead of qa_qc_output
* style: Apply black formatting to prompt_templates.py
* Merge main, move scripts
* Archive some scripts
* update py version
* remove .py version file
* Remove ASCII characters
* Restore testbed code
* restore tracking
* Update testbed metrics
* Enable prompt caching for CODE_LAST_CHECK, FILL_BILL_TYPE, DUAL_LOB_CHECK, and GROUPER_BREAKOUT
- Add CODE_LAST_CHECK_INSTRUCTION() for service specificity classification
- Add FILL_BILL_TYPE_INSTRUCTION() for bill type code determination
- Add DUAL_LOB_CHECK_INSTRUCTION() for Medicare/Medicaid classification
- Update code_funcs.py to use caching for CODE_LAST_CHECK, FILL_BILL_TYPE, GROUPER_BREAKOUT
- Update postprocessing_funcs.py to use caching for DUAL_LOB_CHECK
- Add new instructions to get_cacheable_instructions() for cache warming
- Add unit tests for new instruction functions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix postprocessing_funcs to remove invalid columns
* Merge branch 'main' into feature/lesser-table-caching-refactor-hybrid
* Revert prompt caching changes from aed1b73c
* update formatting
* Update imports
Approved-by: Sha Brown
Approved-by: Praneel Panchigar
Katon Minhas