Katon Minhas
afb6d5185d
Feature/lesser table caching refactor hybrid * chore: Remove unused duplicate main.py from shared pipeline * fix: Correct crosswalk paths in aarete_derived.py * chore: Remove unused documentation files from fieldExtraction * docs: Add documentation files to documentation folder * docs: Update README with uv setup, expanded project structure, and branching conventions * docs: Add uv installation steps with Ubuntu/WSL emphasis * Enable prompt caching for all remaining LLM calls - Add _INSTRUCTION() functions for: EXHIBIT_HEADER, EXHIBIT_LINKAGE, EXHIBIT_TITLE_MATCH, DATE_FIX, DERIVED_TERM_DATE, CHECK_PROVIDER_NAME_MATCH, SPECIAL_CASE_ASSIGNMENT - Update all invoke_claude() calls in saas and clover pipelines to use cache=True with corresponding _INSTRUCTION() functions - Add new instructions to get_cacheable_instructions() for cache warming - Update tests for new instruction functions Functions now using caching: - prompt_exhibit_level - prompt_exhibit_lesser (EXHIBIT_LEVEL_LESSER_OF) - prompt_fee_schedule_breakout - prompt_grouper_breakout - prompt_special_case_assignment - prompt_exhibit_linkage - prompt_exhibit_header - prompt_smart_chunked (ONE_TO_ONE templates) - prompt_date_fix - prompt_derived_term_date - prompt_exhibit_title_match - provider_name_match_check 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Reorder * feat: Add bcbs_promise client pipeline with OFFSET_TERM extraction - Add new bcbs_promise client with HSC-based OFFSET_TERM field extraction - Extract full paragraph text of offset/recoupment provisions from contracts - Derive OFFSET_INDICATOR (Y/N) from OFFSET_TERM presence - Fix reorder_columns to preserve extra columns not in COLUMN_ORDER - Update QC/QA output path to outputs/qc_qa/ * fix: Update dev deps and test assertions for QC/QA output path - Add pytest/pytest-mock to dev dependencies for mypy type checking - Update test assertions to expect outputs/qc_qa instead of qa_qc_output * style: Apply black formatting to prompt_templates.py * Merge main, move scripts * Archive some scripts * update py version * remove .py version file * Remove ASCII characters * Restore testbed code * restore tracking * Update testbed metrics * Enable prompt caching for CODE_LAST_CHECK, FILL_BILL_TYPE, DUAL_LOB_CHECK, and GROUPER_BREAKOUT - Add CODE_LAST_CHECK_INSTRUCTION() for service specificity classification - Add FILL_BILL_TYPE_INSTRUCTION() for bill type code determination - Add DUAL_LOB_CHECK_INSTRUCTION() for Medicare/Medicaid classification - Update code_funcs.py to use caching for CODE_LAST_CHECK, FILL_BILL_TYPE, GROUPER_BREAKOUT - Update postprocessing_funcs.py to use caching for DUAL_LOB_CHECK - Add new instructions to get_cacheable_instructions() for cache warming - Add unit tests for new instruction functions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix postprocessing_funcs to remove invalid columns * Merge branch 'main' into feature/lesser-table-caching-refactor-hybrid * Revert prompt caching changes from aed1b73c * update formatting * Update imports Approved-by: Sha Brown Approved-by: Praneel Panchigar
65 lines
2.5 KiB
Python
65 lines
2.5 KiB
Python
|
|
import pandas as pd
|
|
|
|
|
|
# # Nonprompt
|
|
abc_nonprompt = pd.read_excel('hotfix_output/CNC-1A-ABC-NonPrompt-HotfixFull.xlsx')
|
|
print("NonPrompt:", abc_nonprompt.shape)
|
|
print(len(abc_nonprompt['Contract Name'].unique()))
|
|
|
|
# Agreement Name
|
|
abc_agreement = pd.read_csv('hotfix_output/CNC-1A-AgreementName-All.csv')
|
|
print("Agreement Name:", abc_agreement.shape)
|
|
print(len(abc_agreement['Contract Name'].unique()))
|
|
|
|
# Effective Date
|
|
abc_date = pd.read_csv('hotfix_output/CNC-1A-ContractEffectiveDate-All.csv')
|
|
abc_date = abc_date[['Contract Name', 'Contract Effective Date_corrected']]
|
|
abc_date.drop_duplicates(inplace=True)
|
|
print("Effective Date:", abc_date.shape)
|
|
print(len(abc_date['Contract Name'].unique()))
|
|
|
|
# Term Group
|
|
abc_term = pd.read_csv('hotfix_output/CNC-1A-TermGroup-All.csv')
|
|
abc_term.columns = ['Contract Name', 'Term Clause_corrected', 'Contract Auto-Renewal Indicator_corrected', 'Termination Date_corrected']
|
|
print("Term Group:", abc_term.shape)
|
|
print(len(abc_term['Contract Name'].unique()))
|
|
|
|
# Rate
|
|
rate_full = pd.read_csv('hotfix_output/CNC-1A-Rate.csv')
|
|
rate_468 = pd.read_excel('hotfix_output/CNC-1A-Rate-468.xlsx')
|
|
mask = ~rate_full['Contract Name'].isin(rate_468['Contract Name'])
|
|
rate_full_filtered = rate_full[mask]
|
|
abc_rate = pd.concat([rate_full_filtered, rate_468], ignore_index=True)
|
|
abc_rate = abc_rate[['Contract Name', 'Attachment/Exhibit', 'Reimb. Methodology', 'FULL_METHODOLOGY_fixed', 'LESSER_fixed', 'RATE_STANDARD_fixed', 'RATE_SHORT_fixed', 'FLAT_FEE_STANDARD_fixed', 'LESSER_RATE_fixed', 'NOT_TO_EXCEED_fixed', 'SHORT_METHODOLOGY_fixed']]
|
|
abc_rate.drop_duplicates(inplace=True)
|
|
print("Rate:", abc_rate.shape)
|
|
print(len(abc_rate['Contract Name'].unique()))
|
|
|
|
# Merge Effective Date
|
|
abc_final = pd.merge(abc_nonprompt, abc_date, how='left', on='Contract Name')
|
|
print("After Date:", abc_final.shape)
|
|
print(len(abc_final['Contract Name'].unique()))
|
|
|
|
# Merge Rate
|
|
abc_final = pd.merge(abc_final, abc_rate, how='left', on=['Contract Name', 'Attachment/Exhibit', 'Reimb. Methodology'])
|
|
print("After Rate:", abc_final.shape)
|
|
print(len(abc_final['Contract Name'].unique()))
|
|
|
|
# Merge Agreement Name
|
|
abc_final = pd.merge(abc_final, abc_agreement, how='left', on='Contract Name')
|
|
print("After Agreement:", abc_final.shape)
|
|
print(len(abc_final['Contract Name'].unique()))
|
|
|
|
# Merge Term Group
|
|
abc_final = pd.merge(abc_final, abc_term, how='left', on='Contract Name')
|
|
print("After Term (Final):", abc_final.shape)
|
|
print(len(abc_final['Contract Name'].unique()))
|
|
|
|
print(list(abc_final.columns))
|
|
|
|
abc_final.to_excel('CNC-1A-Hotfix-Final.xlsx')
|
|
|
|
|
|
|