0d00d581b111feb3f5446dab766a45c358e518fd
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Katon Minhas
|
0d00d581b1 |
Merged in DEV (pull request #913)
DEV * Apply formatting fix * Merged in DAIP2-1898-one-to-one-issues-filename-tin-auto-renewal-and-contract-title (pull request #888) DAIP2-1898 one to one issues filename tin auto renewal and contract title * contract title fixes * Add instruction to strip hyphens from TIN and NPI * prompt update * Merge branch 'bugfix/auto-renewal-term' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info * removed print statements * pipeline error fixed * signed_ind logic changed * signatory_ind logic updated * prompt update * prompt update * pipeline error fixed * Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info * Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info * requested changes applied * test case added * prompt update * contract title prompt update * prompt update * Merge branch 'DEV' into DAIP2-1898-one-to-one-issues-filename-tin-auto-renewal-and-contract-title Approved-by: Siddhant Medar * Merged in feature/DAIP2-1803-add-aarete-derived-provider-name (pull request #884) Feature/DAIP2-1803 add aarete derived provider name * Merged DEV into feature/DAIP2-1562-add-aarete_derived_payer_name * state logic added in clustering * removed print statements * pull request updates * black format fix * Merged DEV into feature/DAIP2-1562-add-aarete_derived_payer_name * added scalability feature and optimization * black format * Pull Request Changes * generalised funcs from aarete derived payer name * updated remove states and updated docstring for build similarity matrix * derived payer name functionality added * config and main file changes * sync dev into ADD AARETE DERIVED PROVIDER GROUP NAME FULL * updated derived_provider name * pipeline fixes * Merged DEV into feature/DAIP2-1803-add-aarete-derived-provider-name * config changes * field name updated to AARETE_DERIVED_PROVIDER_NAME * Merged DEV into feature/DAIP2-1803-add-aarete-derived-provider-name * black format fix * Fix docstr… * Merged in improve-logging (pull request #886) Improve logging * Add timing blocks for comprehensive pipeline logging - Add timing_utils.timed_block() to key extraction functions in one_to_n_funcs.py: - exhibit_level sub-functions (prompt_exhibit_level, dynamic_primary, etc.) - reimbursement_level extraction and cleaning - breakout functions (methodology_breakout, special_case_breakout) - carveout_and_special_case parallel processing - one_to_n_cleaning sub-steps (crosswalk, lob_relationship, split_reimb_dates) - Add timing to dynamic_funcs.py: - dynamic_assignment parallel processing - Add timing to postprocess.py: - standard_postprocess, contract_config_postprocess, dashboard_postprocess - Add timing to preprocess.py: - exhibit_chunking sub-steps (get_exhibit_pages, link_exhibit_pages, chunk_by_exhibit) - Add info-level logging statements for completed operations with row/item counts * Standardize logging levels and remove deprecated code DAIP2-1798: Ensure logging statements are at … * Merged in bugfix/DAIP2-1913-lesser-of-indicator-issues (pull request #889) Bugfix/DAIP2-1913 lesser of indicator issues * contract title fixes * Add instruction to strip hyphens from TIN and NPI * prompt update * Merge branch 'bugfix/auto-renewal-term' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info * removed print statements * pipeline error fixed * signed_ind logic changed * signatory_ind logic updated * prompt update * prompt update * pipeline error fixed * Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info * Merge branch 'DEV' into bugfix/DAIP2-1699-issue-tracker-fixes-one-to-one-dates-prov-info * requested changes applied * test case added * prompt update * contract title prompt update * prompt update * Merge branch 'DEV' into DAIP2-1898-one-to-one-issues-filename-tin-auto-renewal-and-contract-title * handle nested cases of lesser_of and greater_of * Merge branch 'DEV' into bugfix/DAIP2-1913-lesser-of-indicator-issues * pi… * Merged in bugfix/reimb_primary (pull request #890) Update reimbursement prompts for consistent PMPM handling, time-unit dedup, and validation improvements * Update reimbursement prompts for consistent PMPM handling, time-unit dedup, and validation improvements - REIMBURSEMENT_PRIMARY: append unit of measure (PMPM, per visit, etc.) from table headers into REIMB_TERM - METHODOLOGY_BREAKOUT: collapse mathematically equivalent time-unit rates into single entry - VALIDATE_REIMBURSEMENTS: add umbrella terms as valid service terms, accept CPT/HCPCS percentage as concrete rate, narrow COB disqualifier to primary subject only Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Approved-by: Siddhant Medar * Merged in feature/update-postprocessing-columns (pull request #893) move column order to final step * move column order to final step * Black Approved-by: Siddhant Medar * Merged in bugfix/update-postprocessing-paths (pull request #896) Update postprocessing paths * Update postprocessing paths Approved-by: Siddhant Medar * Merged in feature/TIN_stats_update (pull request #898) Feature/TIN stats update * Tin_stats_report * black formatting * minor changes * black format applied * Addressed the katons comments * Merged DEV into feature/TIN_stats_update * black reformat Approved-by: Katon Minhas * Merged main into DEV * Merged in feature/exhibit-smart-chunking (pull request #883) Feature/exhibit smart chunking * exhibit processing per page * header dict deduplication * dedup prompt refinment * refinment for header extraction proecess * Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking * merge updates * minor fix * prompt fix for reimb type * COB defenition for clear understanding * black formatting * remove quit statement * pipiline test * pipeline test * black formatting * Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking * black formating * Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking * typo * PR comment fixes * exhibit funcs refactored * black formatting * Refactor exhibit chunking config into dedicated class Created ExhibitChunkingConfig class to centralize exhibit smart chunking configuration parameters (DEFAULT_SUBCHUNK_SIZE, MIN_PARENT_CHUNK_SIZE, CHUNK_RELEVANCE_THRESHOLD). This improves code organization … * Merged in bugfix/DAIP2-1870-dynamic-issues (pull request #892) Bugfix/DAIP2-1870 dynamic issues * updated list format appending * fill bill type from claim type cd * Fixed remaining format fixes * Merged DEV into bugfix/DAIP2-1870-dynamic-issues * changed crosswalk mapping from PRODUCT to AARETE_DERIVED_PRODUCT * Merged DEV into bugfix/DAIP2-1870-dynamic-issues * updating LOB field values as well from PRODUCT AND PROGRAM * updated apply crosswalk for reverse mapping * Merged DEV into bugfix/DAIP2-1870-dynamic-issues * added QHP and BHP * removed lob reverse mapping * Merged DEV into bugfix/DAIP2-1870-dynamic-issues * Remove Bill Type/Claim Type fill * Merged DEV into bugfix/DAIP2-1870-dynamic-issues * Black format * Merged DEV into bugfix/DAIP2-1870-dynamic-issues Approved-by: Katon Minhas * Merged in bugfix/prov_info_json_fixes (pull request #899) Bugfix/prov info json fixes * fix: robust PROV_INFO_JSON sanitization and TIN backfill logic json_utils: - Add sanitize_prov_info_json with layered parsing (JSON, literal_eval, empty-value-after-colon fix, best-effort dict extraction). - Add _normalize_prov_entries and _prov_value_to_str for uniform str-valued output; flatten list values, strip TIN hyphens. - format_prov_info_json now delegates to sanitize_prov_info_json. postprocessing_funcs: - Add fill_prov_info_tin_from_filename_tin for TIN backfill. - Add validate_and_reformat_date (pipe-wrapped, datetime strings). - Add format_as_json_list (pipe-delimited, comma-separated, quote stripping). postprocess: - Integrate new postprocessing helpers into pipeline flow. postprocess_existing_output: - Support CSV and Excel input, configurable paths, fillna for CSV. tests: - Add test_json_parsers.py for PROV_INFO_JSON parsing coverage. - Add test_postprocess.py for date/list formatting and defau… * Merged in bugfix/reimb_primary_issues (pull request #900) Bugfix/reimb primary issues * Update lesser of distribution prompt to preserve reimb term langauge * Added prompt change to extract correct service when there are multiple sub-rates; Updated validate reimb term to prevent filtering out erm with valid rate * Make reimb primary prompt change * Revert "Make reimb primary prompt change" This reverts commit 55f7d73b9cb523097c96ea50b6614c1db53b46d2. Approved-by: Katon Minhas * Merged in dtc_report (pull request #902) Dtc report * dtc_report_added * lint format fixed * Changed regex location call Approved-by: Katon Minhas * Merged in feature/historical-cost-analysis (pull request #901) Feature/historical cost analysis * Initial commit * Finalize cost_analysis * Black format * Merged DEV into feature/historical-cost-analysis * Refactor * Black * Merged DEV into feature/historical-cost-analysis Approved-by: Siddhant Medar * Merged in feature/ai-pr-review-agent (pull request #906) Add AI code review agent to pull request pipeline * Add AI code review agent to pull request pipeline Adds a pull-requests pipeline that runs CI checks (lint, type check, unit tests) followed by an AI code review step. The AI review uses OIDC to authenticate with AWS Bedrock - no hardcoded AWS keys. It clones the code-review-agent repo and posts a review comment on the PR. * Fix AI code review step: install git in python:3.12-slim image * Trigger pipeline * Trigger pipeline * Trigger pipeline * Trigger pipeline * Trigger pipeline * Trigger pipeline * Trigger pipeline * Remove debug token decode line * Update guide: all checklist items complete, agent working * Remove guide file from repo * Trigger pipeline Approved-by: Katon Minhas * Merged in DAIP2-1980-aarete-derived-effective-dt-issues (pull request #904) DAIP2-1980 aarete derived effective dt issues * fixed earlier of a and b * prompt update * prompt shortened * test changes reverted * Merge branch 'DEV' into DAIP2-1980-aarete-derived-effective-dt-issues * test changes reverted * Merged DEV into DAIP2-1980-aarete-derived-effective-dt-issues * Fix effective date prompt * Replace em-dashes with readable version Approved-by: Katon Minhas * Merged in feature/DAIP2-2023-eliminate-full-context-processing (pull request #905) Feature/DAIP2-2023 eliminate full context processing * testing full context fields * remove full context processing * merge Dev with DAIP2-2-23 * full context removal in client codes * AARETE_DERIVED_PROVIDER_NAME field changes * Merged DEV into feature/DAIP2-2023-eliminate-full-context-processing * optimized provider name * black format fix * contract title fixes * PAYER NAME AUTO RENEWAL IND fixes * Merged DEV into feature/DAIP2-2023-eliminate-full-context-processing * Merge branch 'DEV' into feature/DAIP2-2023-eliminate-full-context-processing * Remove prints Approved-by: Katon Minhas * Merged in feature/code_optimization (pull request #903) Feature/code optimization * Ran Black * made a small change in code_last_check, fixed so it returns string and not single char * Made changes to make sure that default_ind postprocess only happens to the cc output and not dashboard * Merge bugfix/default_ind_postprocess into feature/code_optimization - Parent-child: child_rank column init and cols_to_keep filter - Postprocess: default_ind only on cc output - code_last_check: parser str/list return handling + exception logging - Prompt: FIELD ASSIGNMENT for code extraction (no CRITICAL prefix) * More prompt changes and debugging print statements * Added system to group same service term + bill type cd + claim type cd and then parallelize the code extraction (will improve consistency) * Revert one-time PROV_INFO_JSON ad hoc logic; retain json_utils and output format - Remove temporary postprocess step that filled empty TIN in PROV_INFO_JSON from FILENAME_TIN (fill_prov_info_tin_from_filename_ti… * Merged in DAIP2-2121-prov-info-fields-issue-fixes (pull request #908) empty prov_info_json fixed * empty prov_info_json fixed * Merged DEV into DAIP2-2121-prov-info-fields-issue-fixes * format of PROV_OTHER_TIN fixed * Revert "format of PROV_OTHER_TIN fixed" This reverts commit 580847de03265520b1b4fad473bbf38ef1573753. * formatting and deduplication * pipeline error fixed * test cases added * pipeline error fixed Approved-by: Katon Minhas * Merged in feature/context-caching (pull request #909) Feature/context caching * Initial commit - context caching for DYNAMIC_PRIMARY * implement context caching for all relevant prompts * Remove option to not context cache * IndentationError fixed * Merge branch 'DEV' into feature/context-caching * Merge and format * Move documentation * Merged DEV into feature/context-caching * Update unit tests * Merged DEV into feature/context-caching * Update signatures * Fix test coverage gap Approved-by: Praneel Panchigar Approved-by: Karan Desai * Merged in bugfix/DAIP2-2138-reimbursement-primary-testing (pull request #910) Bugfix/DAIP2-2138 reimbursement primary testing * updated table instructions for reimb term extraction * updated split tables to include header rows * issue fix * Merged DEV into bugfix/DAIP2-2138-reimbursement-primary-testing * black format fix Approved-by: Katon Minhas * Merged in feature/doczy_reports (pull request #911) Feature/doczy reports * Added post_doczy_reports * black format fix * Merged DEV into feature/doczy_reports * fixed code review Approved-by: Katon Minhas * Merged in feature/median-cost-analysis (pull request #912) Add median * Add median Approved-by: Siddhant Medar Approved-by: Siddhant Medar |
||
|
Katon Minhas
|
afb6d5185d |
Merged in feature/lesser-table-caching-refactor-hybrid (pull request #847)
Feature/lesser table caching refactor hybrid * chore: Remove unused duplicate main.py from shared pipeline * fix: Correct crosswalk paths in aarete_derived.py * chore: Remove unused documentation files from fieldExtraction * docs: Add documentation files to documentation folder * docs: Update README with uv setup, expanded project structure, and branching conventions * docs: Add uv installation steps with Ubuntu/WSL emphasis * Enable prompt caching for all remaining LLM calls - Add _INSTRUCTION() functions for: EXHIBIT_HEADER, EXHIBIT_LINKAGE, EXHIBIT_TITLE_MATCH, DATE_FIX, DERIVED_TERM_DATE, CHECK_PROVIDER_NAME_MATCH, SPECIAL_CASE_ASSIGNMENT - Update all invoke_claude() calls in saas and clover pipelines to use cache=True with corresponding _INSTRUCTION() functions - Add new instructions to get_cacheable_instructions() for cache warming - Update tests for new instruction functions Functions now using caching: - prompt_exhibit_level - prompt_exhibit_lesser (EXHIBIT_LEVEL_LESSER_OF) - prompt_fee_schedule_breakout - prompt_grouper_breakout - prompt_special_case_assignment - prompt_exhibit_linkage - prompt_exhibit_header - prompt_smart_chunked (ONE_TO_ONE templates) - prompt_date_fix - prompt_derived_term_date - prompt_exhibit_title_match - provider_name_match_check 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Reorder * feat: Add bcbs_promise client pipeline with OFFSET_TERM extraction - Add new bcbs_promise client with HSC-based OFFSET_TERM field extraction - Extract full paragraph text of offset/recoupment provisions from contracts - Derive OFFSET_INDICATOR (Y/N) from OFFSET_TERM presence - Fix reorder_columns to preserve extra columns not in COLUMN_ORDER - Update QC/QA output path to outputs/qc_qa/ * fix: Update dev deps and test assertions for QC/QA output path - Add pytest/pytest-mock to dev dependencies for mypy type checking - Update test assertions to expect outputs/qc_qa instead of qa_qc_output * style: Apply black formatting to prompt_templates.py * Merge main, move scripts * Archive some scripts * update py version * remove .py version file * Remove ASCII characters * Restore testbed code * restore tracking * Update testbed metrics * Enable prompt caching for CODE_LAST_CHECK, FILL_BILL_TYPE, DUAL_LOB_CHECK, and GROUPER_BREAKOUT - Add CODE_LAST_CHECK_INSTRUCTION() for service specificity classification - Add FILL_BILL_TYPE_INSTRUCTION() for bill type code determination - Add DUAL_LOB_CHECK_INSTRUCTION() for Medicare/Medicaid classification - Update code_funcs.py to use caching for CODE_LAST_CHECK, FILL_BILL_TYPE, GROUPER_BREAKOUT - Update postprocessing_funcs.py to use caching for DUAL_LOB_CHECK - Add new instructions to get_cacheable_instructions() for cache warming - Add unit tests for new instruction functions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix postprocessing_funcs to remove invalid columns * Merge branch 'main' into feature/lesser-table-caching-refactor-hybrid * Revert prompt caching changes from aed1b73c * update formatting * Update imports Approved-by: Sha Brown Approved-by: Praneel Panchigar |