reorder_columns() was only keeping columns listed in FIELD_FORMAT_MAPPING,
silently discarding any extra columns like BCBS OFFSET_TERM/OFFSET_INDICATOR
and Clover's 12 full_context fields. The function's docstring documented
appending extra columns but the implementation was missing that step.
1. Tag format validation in release-prod (MEDIUM, defensive).
Before parsing major/minor/patch from the latest tag, assert it
matches ^v[0-9]+\.[0-9]+\.[0-9]+$. Empirically tested: catches
v1, v1.2, v1.2.3.4, v1.0.0-rc1, v1.2.3-beta+build, and still
accepts v0.0.0 (the first-release fallback).
Without this, bash arithmetic silently mangles non-semver tags
into wrong results via its 'treat empty/non-numeric as 0' rule.
Worst case: tag 'v1' → cut -d. -f2/-f3 both return '1' →
minor bump produces v1.1.2 instead of v1.0.1. No visible error.
2. Rewrite the AI review comment to match actual behavior.
The old comment said 'advisory, not a gate' but also admitted
git clone / apt-get are hard failures, contradicting itself.
The new comment makes the runtime vs. infrastructure distinction
explicit: runtime failures (agent crash, STS errors, Python
exceptions) become warnings via || echo; infrastructure failures
(missing token, clone failure, apt-get failure) stay hard.
This is a doc fix, not a code change. Deliberately preserving
the hard-fail behavior on config errors because silent
degradation of AI review is the worst outcome — feature
disappears from CI with no signal.
Explicitly rejected from the second review (empirically verified):
- Gate logic 'fragile' claim: tested in bash, correct for all
our hardcoded ALLOWED values.
- ROLLBACK_TAG -z check 'passes empty': tested, -z correctly
catches empty strings. Reviewer has the semantics wrong.
- chmod 600 + set +x: our comment already documents chmod 600
as cosmetic/scanner-silencing; set +x wouldn't help because
Bitbucket's line-echo is runner-level, not bash set -x.
- cd/python refactor (previous round): already rebutted in
commit 4836e36f via empirical bash AND-OR list tests.
- Option B for git clone wrapping: silent degradation of
config errors is worse than the current loud-fail behavior.
1. Replace 'uv sync --frozen' with 'uv sync --locked'. --frozen silently
uses a stale lockfile if pyproject.toml is updated without regenerating
uv.lock, masking missed dependencies. --locked fails loudly with a clear
error when the lockfile is out of sync. Verified empirically with uv
0.9.14: --frozen exits 0 on mismatch, --locked exits 1.
2. Add 'chmod 600' on the OIDC token file after writing it. The container
is already single-user root so this is defensive/cosmetic, but it
silences security scanners and signals intent.
3. Add a comment explaining why gate steps use atlassian/default-image:4
instead of python:3.12.7 (gates run bash only, no Python toolchain
needed — lighter image, faster pull).
Explicitly rejected from the PR review:
- Token-masking sed mitigation (delimiter collision with / in real
Bitbucket clone tokens; cmd | sed || exit 1 swallows git failures
without set -o pipefail). Bitbucket's built-in Secured variable
masking is the correct mitigation and is already documented as a
setup requirement.
- SSH key alternative for git clone (architecturally worse — more
secrets to manage; HTTPS+Secured is the Bitbucket-recommended pattern).
- cd/python refactor ('fragile logic bug'). Verified empirically that
'cd X && python Y || echo Z' with set -e correctly catches both
cd and python failures via the ||. The step exits 0 as intended
by the 'advisory, not a gate' design. The suggested refactor would
introduce a hard-fail regression.
The unquoted ': ' (colon + space) in 'WARNING: AI review step failed'
was being interpreted as a YAML mapping key-value separator, causing
the entire script item to be parsed as a dict instead of a command
string. Bitbucket then rejected it with 'Missing or empty command
string' error at pull-requests > feature/* > 2 > step > script > 9.
Replaced the colon with a hyphen. Validated with yaml.safe_load that
all 10 script items in the ai-code-review step now parse as strings.
Bitbucket's YAML parser was interpreting the deeper-indented comment
after the printf line as a phantom empty list item, causing a
'Missing or empty command string' error at script item 9.
BCBS: change strict == "YES" to "YES" in final_answer. The strict
equality was rejecting borderline LLM responses (e.g. "YES, ..."),
dropping row counts from ~7 to 1.
Clover: remove .strip().upper() which crashed with AttributeError
because the parser returns a list, not a string. Every validation
call failed, dropping row counts from 106 to 1.
Both overrides now use the same "YES" in final_answer logic as SaaS.
E2E verified: BCBS 5 rows (within LLM variance of baseline 7),
Clover 106 rows (exact match to baseline).
Delete BCBS/Clover/CHC client file_processing.py files (1090 lines of
96% stale drift with 3 crash points). Recover the 4% real business
logic (BCBS OFFSET_TERM, Clover 12 full_context fields) into the
shared pipeline via config-driven field loading.
Make runner.py the canonical pipeline entry point by merging all
production features from main.py (timing, duplicate detection, aarete
derived fields, TIN statistics, column reordering). Reduce main.py to
a thin wrapper that delegates to runner.main().
Fix the root cause of all client overrides being dead in production:
set_active_client is now called with the correct client name instead
of being hardcoded to "saas". Both resolver shims (file_processing
and prompt_calls) now route correctly.
Includes temporary [DEBUG ROUTING] print statements at 6 routing
decision points for E2E verification. Cherry-pick this commit to
restore debug instrumentation for future testing.
E2E verified: BCBS Promise and Clover both pass with correct routing.
Bugfix/generic issue fixes
* Patch for llm responses
* Merged dev into bugfix/generic-issue-fixes
* Update Exhibit-Level instruction for Claim Type and Bill Type
* Update Service-Level instruction for Claim Type and Bill Type
* Update Bill Type code to prompt for DESC field
* Black format
* Remove test
Approved-by: Siddhant Medar
Bugfix/unify prompt exhibit header
* Remove dead prompt_exhibit_header overrides and legacy get_exhibit_pages
Investigation (AST call tree analysis) proved both were dead code:
1. get_exhibit_pages() in preprocessing_funcs.py had ZERO callers in the
entire codebase. It was superseded by get_exhibit_pages_new() which is
the only live path (called by preprocess.one_to_n_exhibit_chunking).
2. Client prompt_exhibit_header overrides (2-arg) in bcbs_promise and
clover prompt_calls.py were dead for two independent reasons:
a) Their only consumer (get_exhibit_pages) has zero callers
b) The live consumer (get_exhibit_pages_new) calls with 3 args —
if the resolver returned the client 2-arg function, Python would
raise TypeError. The SaaS 3-arg version was always running.
Verification:
- Programmatic AST call tree (local_scripts/call_tree_analysis.py)
confirms zero callers and signature mismatch
- Resolver test confirms all clients resolve to SaaS 3-arg implementation
- Full unit suite: 1077 passed, 0 failures
- E2E: BCBS P…
* Merged dev into bugfix/unify_prompt_exhibit_header
* Add dev dependencies for local code index
Add tree-sitter, lancedb, and tiktoken as dev dependencies for a local
codebase indexing system. The index provides hybrid search (symbol table,
call graph, semantic embeddings, BM25) for token-efficient code exploration.
The index lives in a gitignored .index/ folder — each developer builds
their own locally.
Approved-by: Katon Minhas
Feature/pre doczy status column
* pre-doczy changes
* format
* Merged dev into feature/pre-doczy-status-column
* function moved to utils
Approved-by: Katon Minhas
Bugfix/implicit cpt update
* DAIP2-2310: enrich service term before implicit code extraction
- Add SERVICE_ENRICHMENT prompt + enrich_service_for_codes() that runs
after explicit extraction and before the implicit pipeline, stripping
provider/facility/contractual noise while preserving billing codes.
- Tighten CODE_IMPLICIT_ARBITRATION instructions to require strict
synonymy and prefer no_match over best-effort guesses.
- Remove single-candidate auto-accept bypass so all implicit candidates
go through LLM arbitration.
- Update unit tests for new arbitration path and enrichment mock.
* DAIP2-2310: refine service enrichment and track implicit mapping source
- Service enrichment prompt: bare 'Inpatient'/'Outpatient' place-of-service
rows now collapse to 'Covered Services' to prevent spurious Level 1 matches.
When paired with a real procedure (e.g. 'Outpatient Surgery'), the place
modifier is preserved.
- code_implicit_rag: record per-match mapping origin (CPT_LEVEL1,
HCPCS_LEVEL1, CPT_LEVEL2, HCPCS_LEVEL2) on the code answer dict for
debugging which mapping produced each implicit match.
- constants: add TODO next to HCPCS_LEVEL1_MAPPING load flagging the
upcoming DME code range change.
* Merged dev into DAIP2-2310-implicit-cpt-update
* DAIP2-2310: fix 3-digit revenue code extraction and enrichment exclusion rule
Two related fixes surfaced during short-circuit analysis of the NICU rows:
1. 3-digit revenue codes were being dropped by validate_explicit_codes,
causing the explicit path to return empty and the row to fall through
to implicit RAG (which would then semantic-match against CPT/HCPCS
and produce 'Specific - No Match'). Root causes:
- _revenue_code_format_valid required exactly 4 digits, rejecting the
common contract form like 'Revenue Code 173'.
- REV_MAPPING keys are loaded as ints by pandas (leading zero stripped
from all-numeric CSV columns), so even after accepting 3-digit form
the mapping lookup missed.
Fixes in src/codes/code_funcs.py:
- _revenue_code_format_valid now accepts 3 or 4 digits.
- New _normalize_revenue_code helper returns canonical 4-digit form.
- validate_explicit_codes stores the normalized form in REVENUE_CD.
- _has_unmatched_codes normalizes before membership chec…
* Merged dev into bugfix/implicit-cpt-update
* DAIP2-2310: accept revenue code wildcards and ranges; add unit tests
Extends the earlier 3-digit revenue code fix to cover two more forms the
CODE_EXPLICIT prompt instructs the LLM to return:
1. X-suffix wildcards (e.g. '17X', '173X') - standard UB-04 shorthand
meaning any digit across a revenue family. The prompt explicitly says
'may end in X' but the validator was rejecting them, causing the same
silent drop + implicit-RAG fall-through as the 3-digit bug.
2. Ranges (e.g. '173-174', '0170-0179') - the CODE_EXPLICIT prompt
instructs ranges in 'LOW-HIGH' form. Procedure codes support ranges
but revenue codes did not. The Gulf Coast 'Rev Codes 173-174' rows
were landing on Specific - No Match because of this.
New helpers in src/codes/code_funcs.py:
- _revenue_wildcard_format_valid / _revenue_range_format_valid /
_revenue_code_any_format_valid: targeted format checks.
- _normalize_revenue_any: canonical form for any valid kind
(single code -> '0173', wildcard -> '017X', range -> '0173-0…
* DAIP2-2310: prevent enrichment from hallucinating or dropping descriptors
Two related prompt changes to SERVICE_ENRICHMENT_INSTRUCTION based on
review feedback:
1. Add a 'no information not in input' rule. The enrichment is a
faithful rewrite that strips noise — not a knowledge-augmented
expansion. The LLM must NOT look up what a code means and inject
the description, infer category names from codes, or translate
medical shorthand into clinical terms not present in the input.
2. Add a 'preserve descriptive labels' rule. Things like 'Level 1',
'Level 2-5', 'Tier A', 'Class III', 'low complexity' etc. are
meaningful service descriptors written by the contract author and
must remain in the output even when codes are also present. The
prior ER few-shot example showed the LLM stripping 'Level 1'..
'Level 5' labels — that's information loss.
Updated few-shot examples to match the new rules:
- Emergency Room example output now preserves 'Level 1 - 99281,
Level 2 - 99282, ...' verbati…
Approved-by: Katon Minhas
DAIP2-2346 update client prompt calls
* DAIP2-2346: retire accidental client prompt_calls drift
Delete 14 client prompt_calls overrides in bcbs_promise and clover that were
stale forks of old SaaS implementations. Shared callers all expect SaaS return
shapes; client shims were either redundant with parser-side normalization or
outright broken dead code (prompt_dynamic_assignment unpack crash,
prompt_lesser_of_check double-parse). Resolver shim at
src/pipelines/shared/prompts/prompt_calls.py now falls through to SaaS.
Preserved with annotations:
- validate_reimbursements_for_llm: real business rule (strict YES equality)
- prompt_exhibit_header: required by 2-arg call site at preprocessing_funcs.py:192
* DAIP2-2346: also retire prompt_exhibit_level and prompt_dynamic overrides
Follow-up to the Bucket A cleanup. Both client overrides only drifted by
omitting the optional field_names kwarg on prompt_templates.EXHIBIT_LEVEL.
SaaS passes field_names and gets the format-aware parser
(_create_json_dict_parser); falling through to SaaS is a strict improvement
in normalization. Client files now contain only the two truly
client-specific overrides.
* Changes to gitignore
* Merged dev into DAIP2-2346-update-client-prompt-calls
Approved-by: Katon Minhas
Bugfix/DAIP2-2287 payer state fix
* updated payer name prompt
* Merged dev into bugfix/DAIP2-2287-payer-state-fix
* Merged dev into bugfix/DAIP2-2287-payer-state-fix
* added client state field
Approved-by: Katon Minhas
Feature/dynamic codes final
* Move Claim Type to dynamic code group
* Update CLaim Type and Bill TYpe prompts
* Prompt optimization
* Prompt tuning
* streamline
* Format
* Merged dev into feature/dynamic-codes
* test claim and bill type
* Merged dev into DAIP2-2186-service-claim-type-testing
* Merged dev into feature/dynamic-codes
* prompt updated for claim type
* Merge branch 'feature/dynamic-codes' into DAIP2-2186-service-claim-type-testing
* Merged dev into DAIP2-2186-service-claim-type-testing
* Remove print
* Switch to map
* Merged dev into feature/dynamic-codes-final
* prompt inconsistency fixed
Approved-by: Praneel Panchigar
Approved-by: Siddhant Medar
Feature/DAIP2-1615 refactor maximize code reusability
* added reusability
* client resolver added
* Merged DEV into feature/DAIP2-1615-refactor-maximize-code-reusability
* Refactor client prompt reuse and harden resolver test coverage.
Remove duplicated client prompt functions so unresolved names fall back to SaaS, add resolver contract tests with runtime-context isolation, and clean resolver-path test imports to prevent cross-test client leakage.
* Merge origin/dev into feature/DAIP2-1615-refactor-maximize-code-reusability.
Bring in latest vendor pipeline framework changes from dev and resolve runner routing conflict by preserving vendor processor dispatch while keeping shared function-level client fallback behavior.
* Merged dev into feature/DAIP2-1615-refactor-maximize-code-reusability
* Remove tracked drift review report from documentation.
Keep drift/alignment notes as local-only artifacts under ignored docs/ and stop tracking documentation/function_level_override_drift_review.md in the branch.
* Merged dev into feature/DAIP2-1615-refactor-maximize-code-reusability
Approved-by: Katon Minhas
Feature/DAIP2-2301 flag and remove identical contracts
* initiated flaging duplicate contracts
* Merged dev into feature/DAIP2-2301-flag-and-remove-identical-contracts
* Merged dev into feature/DAIP2-2301-flag-and-remove-identical-contracts
* updated duplicate detection
* pipeline fixes
* remove print statements
* Merged dev into feature/DAIP2-2301-flag-and-remove-identical-contracts
Approved-by: Katon Minhas
Feature/DAIP2-1575 set up dev prod uat branches
* Add branch promotion gates and release pipelines
* Add resolve_source_branch.sh script and update .gitignore; fix pipeline step name casing 'dev'
* Add AI code review step and refactor branch resolution in pipelines
* Merged dev into feature/DAIP2-1575-set-up-dev-prod-uat-branches-
* Merged dev into feature/DAIP2-1575-set-up-dev-prod-uat-branches-
* Merged dev into feature/DAIP2-1575-set-up-dev-prod-uat-branches-
Approved-by: Katon Minhas
removed 2 signature related fields
* removed 2 signature related fields
* Merged dev into DAIP2-1595-optimize-signature-fields
* Merged dev into DAIP2-1595-optimize-signature-fields
Approved-by: Katon Minhas
Bugfix/DAIP2-2164 reimbursement primary issue fixes
* sample calculations reimbursements removed
* split service term added
* Merged DEV into bugfix/DAIP2-2164-reimbursement-primary-issue-fixes
* updated prompt and unit test added
* added prompt for codes listed below
* Merged DEV into bugfix/DAIP2-2164-reimbursement-primary-issue-fixes
* Output format at end
* Genericize prompt
* Merged DEV into bugfix/DAIP2-2164-reimbursement-primary-issue-fixes
* context missing issue fixed
* updated service term split prompt for program varints
* Merged DEV into bugfix/DAIP2-2164-reimbursement-primary-issue-fixes
* Merged DEV into bugfix/DAIP2-2164-reimbursement-primary-issue-fixes
* Merged DEV into bugfix/DAIP2-2164-reimbursement-primary-issue-fixes
* Merged DEV into bugfix/DAIP2-2164-reimbursement-primary-issue-fixes
* updated split service term prompt
* optimized and minimized split service term prompt
* updated service term split prompt
* Merged DEV into bugfix/DAIP2-2164-reimbursement-primary-issue-fixes
* Merged dev into bugfix/DAIP2-2164-reimbursement-primary-issue-fixes
* Updated Prompt
* grouped instructions
* removed print statements
Approved-by: Katon Minhas
Feature/vendor generic
* initial commit for generic vendor logic
* updated generic field list
* combined special fields
* joint fields template
* payment term and amount extraction
* model update
* model update
* added constants
* care-source fields added
* care-source constants
* fields update for bcbs-az
* prompt update for term type
* prompt update for payment type
* post process additions for generic
* Merge remote-tracking branch 'origin/DEV' into feature/vendor-generic
* Merged DEV into feature/vendor-generic
* Merged DEV into feature/vendor-generic
* Black format
* Merge remote-tracking branch 'origin/DEV' into feature/vendor-generic
* Merge branch 'DEV' into feature/vendor-generic
* PR comments
* deleted redundant file
* Merged dev into feature/vendor-generic
Approved-by: Katon Minhas
Bugfix/DAIP2-2230 mcs issues effective date new
* updated scope and fixes
* optimized prompt
* shortened prompt
* remove redundant prompt line
* Merged DEV into bugfix/DAIP2-2230-mcs-issues-effective-date-new
Approved-by: Katon Minhas
Feature/DAIP2-2112 implement pre commit hooks
* Add pre-commit hooks for local code quality checks
Adds .pre-commit-config.yaml with file hygiene hooks (trailing-whitespace,
end-of-file-fixer, check-yaml/json, check-merge-conflict, check-added-large-files)
plus local hooks for black and mypy on pre-commit and pytest on pre-push.
Includes setup documentation and adds pre-commit to dev dependencies.
Auto-fixed trailing whitespace and missing newlines caught by the new hooks.
* Add sample pre-commit hook for testing purposes
* Add test file for pre-commit hook validation
* Remove test file for pre-commit hook validation
* Merged DEV into feature/DAIP2-2112-implement-pre-commit-hooks---
* Merged DEV into feature/DAIP2-2112-implement-pre-commit-hooks---
* Move to documentation
Approved-by: Katon Minhas
DAIP2-2178 dynamic primary issue fixes LOBs incorrectly added
* LOB addition from product and program fixed
* Revert "LOB addition from product and program fixed"
This reverts commit fb0cb48a2e74eff6ab66340d07bcf080eb92708c.
* PROGRAM prompt updated
Approved-by: Katon Minhas
DEV
* Apply formatting fix
* Merged in DAIP2-1898-one-to-one-issues-filename-tin-auto-renewal-and-contract-title (pull request #888)
DAIP2-1898 one to one issues filename tin auto renewal and contract title
* contract title fixes
* Add instruction to strip hyphens from TIN and NPI
* prompt update
* Merge branch 'bugfix/auto-renewal-term' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* removed print statements
* pipeline error fixed
* signed_ind logic changed
* signatory_ind logic updated
* prompt update
* prompt update
* pipeline error fixed
* Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* requested changes applied
* test case added
* prompt update
* contract title prompt update
* prompt update
* Merge branch 'DEV' into DAIP2-1898-one-to-one-issues-filename-tin-auto-renewal-and-contract-title
Approved-by: Siddhant Medar
* Merged in feature/DAIP2-1803-add-aarete-derived-provider-name (pull request #884)
Feature/DAIP2-1803 add aarete derived provider name
* Merged DEV into feature/DAIP2-1562-add-aarete_derived_payer_name
* state logic added in clustering
* removed print statements
* pull request updates
* black format fix
* Merged DEV into feature/DAIP2-1562-add-aarete_derived_payer_name
* added scalability feature and optimization
* black format
* Pull Request Changes
* generalised funcs from aarete derived payer name
* updated remove states and updated docstring for build similarity matrix
* derived payer name functionality added
* config and main file changes
* sync dev into ADD AARETE DERIVED PROVIDER GROUP NAME FULL
* updated derived_provider name
* pipeline fixes
* Merged DEV into feature/DAIP2-1803-add-aarete-derived-provider-name
* config changes
* field name updated to AARETE_DERIVED_PROVIDER_NAME
* Merged DEV into feature/DAIP2-1803-add-aarete-derived-provider-name
* black format fix
* Fix docstr…
* Merged in improve-logging (pull request #886)
Improve logging
* Add timing blocks for comprehensive pipeline logging
- Add timing_utils.timed_block() to key extraction functions in one_to_n_funcs.py:
- exhibit_level sub-functions (prompt_exhibit_level, dynamic_primary, etc.)
- reimbursement_level extraction and cleaning
- breakout functions (methodology_breakout, special_case_breakout)
- carveout_and_special_case parallel processing
- one_to_n_cleaning sub-steps (crosswalk, lob_relationship, split_reimb_dates)
- Add timing to dynamic_funcs.py:
- dynamic_assignment parallel processing
- Add timing to postprocess.py:
- standard_postprocess, contract_config_postprocess, dashboard_postprocess
- Add timing to preprocess.py:
- exhibit_chunking sub-steps (get_exhibit_pages, link_exhibit_pages, chunk_by_exhibit)
- Add info-level logging statements for completed operations with row/item counts
* Standardize logging levels and remove deprecated code
DAIP2-1798: Ensure logging statements are at …
* Merged in bugfix/DAIP2-1913-lesser-of-indicator-issues (pull request #889)
Bugfix/DAIP2-1913 lesser of indicator issues
* contract title fixes
* Add instruction to strip hyphens from TIN and NPI
* prompt update
* Merge branch 'bugfix/auto-renewal-term' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* removed print statements
* pipeline error fixed
* signed_ind logic changed
* signatory_ind logic updated
* prompt update
* prompt update
* pipeline error fixed
* Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info
* requested changes applied
* test case added
* prompt update
* contract title prompt update
* prompt update
* Merge branch 'DEV' into DAIP2-1898-one-to-one-issues-filename-tin-auto-renewal-and-contract-title
* handle nested cases of lesser_of and greater_of
* Merge branch 'DEV' into bugfix/DAIP2-1913-lesser-of-indicator-issues
* pi…
* Merged in bugfix/reimb_primary (pull request #890)
Update reimbursement prompts for consistent PMPM handling, time-unit dedup, and validation improvements
* Update reimbursement prompts for consistent PMPM handling, time-unit dedup, and validation improvements
- REIMBURSEMENT_PRIMARY: append unit of measure (PMPM, per visit, etc.) from table headers into REIMB_TERM
- METHODOLOGY_BREAKOUT: collapse mathematically equivalent time-unit rates into single entry
- VALIDATE_REIMBURSEMENTS: add umbrella terms as valid service terms, accept CPT/HCPCS percentage as concrete rate, narrow COB disqualifier to primary subject only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Approved-by: Siddhant Medar
* Merged in feature/update-postprocessing-columns (pull request #893)
move column order to final step
* move column order to final step
* Black
Approved-by: Siddhant Medar
* Merged in bugfix/update-postprocessing-paths (pull request #896)
Update postprocessing paths
* Update postprocessing paths
Approved-by: Siddhant Medar
* Merged in feature/TIN_stats_update (pull request #898)
Feature/TIN stats update
* Tin_stats_report
* black formatting
* minor changes
* black format applied
* Addressed the katons comments
* Merged DEV into feature/TIN_stats_update
* black reformat
Approved-by: Katon Minhas
* Merged main into DEV
* Merged in feature/exhibit-smart-chunking (pull request #883)
Feature/exhibit smart chunking
* exhibit processing per page
* header dict deduplication
* dedup prompt refinment
* refinment for header extraction proecess
* Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking
* merge updates
* minor fix
* prompt fix for reimb type
* COB defenition for clear understanding
* black formatting
* remove quit statement
* pipiline test
* pipeline test
* black formatting
* Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking
* black formating
* Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking
* typo
* PR comment fixes
* exhibit funcs refactored
* black formatting
* Refactor exhibit chunking config into dedicated class
Created ExhibitChunkingConfig class to centralize exhibit smart chunking
configuration parameters (DEFAULT_SUBCHUNK_SIZE, MIN_PARENT_CHUNK_SIZE,
CHUNK_RELEVANCE_THRESHOLD). This improves code organization …
* Merged in bugfix/DAIP2-1870-dynamic-issues (pull request #892)
Bugfix/DAIP2-1870 dynamic issues
* updated list format appending
* fill bill type from claim type cd
* Fixed remaining format fixes
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* changed crosswalk mapping from PRODUCT to AARETE_DERIVED_PRODUCT
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* updating LOB field values as well from PRODUCT AND PROGRAM
* updated apply crosswalk for reverse mapping
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* added QHP and BHP
* removed lob reverse mapping
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* Remove Bill Type/Claim Type fill
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
* Black format
* Merged DEV into bugfix/DAIP2-1870-dynamic-issues
Approved-by: Katon Minhas
* Merged in bugfix/prov_info_json_fixes (pull request #899)
Bugfix/prov info json fixes
* fix: robust PROV_INFO_JSON sanitization and TIN backfill logic
json_utils:
- Add sanitize_prov_info_json with layered parsing (JSON, literal_eval,
empty-value-after-colon fix, best-effort dict extraction).
- Add _normalize_prov_entries and _prov_value_to_str for uniform
str-valued output; flatten list values, strip TIN hyphens.
- format_prov_info_json now delegates to sanitize_prov_info_json.
postprocessing_funcs:
- Add fill_prov_info_tin_from_filename_tin for TIN backfill.
- Add validate_and_reformat_date (pipe-wrapped, datetime strings).
- Add format_as_json_list (pipe-delimited, comma-separated, quote
stripping).
postprocess:
- Integrate new postprocessing helpers into pipeline flow.
postprocess_existing_output:
- Support CSV and Excel input, configurable paths, fillna for CSV.
tests:
- Add test_json_parsers.py for PROV_INFO_JSON parsing coverage.
- Add test_postprocess.py for date/list formatting and defau…
* Merged in bugfix/reimb_primary_issues (pull request #900)
Bugfix/reimb primary issues
* Update lesser of distribution prompt to preserve reimb term langauge
* Added prompt change to extract correct service when there are multiple sub-rates; Updated validate reimb term to prevent filtering out erm with valid rate
* Make reimb primary prompt change
* Revert "Make reimb primary prompt change"
This reverts commit 55f7d73b9cb523097c96ea50b6614c1db53b46d2.
Approved-by: Katon Minhas
* Merged in dtc_report (pull request #902)
Dtc report
* dtc_report_added
* lint format fixed
* Changed regex location call
Approved-by: Katon Minhas
* Merged in feature/historical-cost-analysis (pull request #901)
Feature/historical cost analysis
* Initial commit
* Finalize cost_analysis
* Black format
* Merged DEV into feature/historical-cost-analysis
* Refactor
* Black
* Merged DEV into feature/historical-cost-analysis
Approved-by: Siddhant Medar
* Merged in feature/ai-pr-review-agent (pull request #906)
Add AI code review agent to pull request pipeline
* Add AI code review agent to pull request pipeline
Adds a pull-requests pipeline that runs CI checks (lint, type check,
unit tests) followed by an AI code review step. The AI review uses
OIDC to authenticate with AWS Bedrock - no hardcoded AWS keys. It
clones the code-review-agent repo and posts a review comment on the PR.
* Fix AI code review step: install git in python:3.12-slim image
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Trigger pipeline
* Remove debug token decode line
* Update guide: all checklist items complete, agent working
* Remove guide file from repo
* Trigger pipeline
Approved-by: Katon Minhas
* Merged in DAIP2-1980-aarete-derived-effective-dt-issues (pull request #904)
DAIP2-1980 aarete derived effective dt issues
* fixed earlier of a and b
* prompt update
* prompt shortened
* test changes reverted
* Merge branch 'DEV' into DAIP2-1980-aarete-derived-effective-dt-issues
* test changes reverted
* Merged DEV into DAIP2-1980-aarete-derived-effective-dt-issues
* Fix effective date prompt
* Replace em-dashes with readable version
Approved-by: Katon Minhas
* Merged in feature/DAIP2-2023-eliminate-full-context-processing (pull request #905)
Feature/DAIP2-2023 eliminate full context processing
* testing full context fields
* remove full context processing
* merge Dev with DAIP2-2-23
* full context removal in client codes
* AARETE_DERIVED_PROVIDER_NAME field changes
* Merged DEV into feature/DAIP2-2023-eliminate-full-context-processing
* optimized provider name
* black format fix
* contract title fixes
* PAYER NAME AUTO RENEWAL IND fixes
* Merged DEV into feature/DAIP2-2023-eliminate-full-context-processing
* Merge branch 'DEV' into feature/DAIP2-2023-eliminate-full-context-processing
* Remove prints
Approved-by: Katon Minhas
* Merged in feature/code_optimization (pull request #903)
Feature/code optimization
* Ran Black
* made a small change in code_last_check, fixed so it returns string and not single char
* Made changes to make sure that default_ind postprocess only happens to the cc output and not dashboard
* Merge bugfix/default_ind_postprocess into feature/code_optimization
- Parent-child: child_rank column init and cols_to_keep filter
- Postprocess: default_ind only on cc output
- code_last_check: parser str/list return handling + exception logging
- Prompt: FIELD ASSIGNMENT for code extraction (no CRITICAL prefix)
* More prompt changes and debugging print statements
* Added system to group same service term + bill type cd + claim type cd and then parallelize the code extraction (will improve consistency)
* Revert one-time PROV_INFO_JSON ad hoc logic; retain json_utils and output format
- Remove temporary postprocess step that filled empty TIN in PROV_INFO_JSON from
FILENAME_TIN (fill_prov_info_tin_from_filename_ti…
* Merged in DAIP2-2121-prov-info-fields-issue-fixes (pull request #908)
empty prov_info_json fixed
* empty prov_info_json fixed
* Merged DEV into DAIP2-2121-prov-info-fields-issue-fixes
* format of PROV_OTHER_TIN fixed
* Revert "format of PROV_OTHER_TIN fixed"
This reverts commit 580847de03265520b1b4fad473bbf38ef1573753.
* formatting and deduplication
* pipeline error fixed
* test cases added
* pipeline error fixed
Approved-by: Katon Minhas
* Merged in feature/context-caching (pull request #909)
Feature/context caching
* Initial commit - context caching for DYNAMIC_PRIMARY
* implement context caching for all relevant prompts
* Remove option to not context cache
* IndentationError fixed
* Merge branch 'DEV' into feature/context-caching
* Merge and format
* Move documentation
* Merged DEV into feature/context-caching
* Update unit tests
* Merged DEV into feature/context-caching
* Update signatures
* Fix test coverage gap
Approved-by: Praneel Panchigar
Approved-by: Karan Desai
* Merged in bugfix/DAIP2-2138-reimbursement-primary-testing (pull request #910)
Bugfix/DAIP2-2138 reimbursement primary testing
* updated table instructions for reimb term extraction
* updated split tables to include header rows
* issue fix
* Merged DEV into bugfix/DAIP2-2138-reimbursement-primary-testing
* black format fix
Approved-by: Katon Minhas
* Merged in feature/doczy_reports (pull request #911)
Feature/doczy reports
* Added post_doczy_reports
* black format fix
* Merged DEV into feature/doczy_reports
* fixed code review
Approved-by: Katon Minhas
* Merged in feature/median-cost-analysis (pull request #912)
Add median
* Add median
Approved-by: Siddhant Medar
Approved-by: Siddhant Medar
Rahul Ailaboina