jacob-mathison/doczyai-pipelines
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

Merged in feature/update-caching (pull request #856)

Browse Source 
Feature/update caching

* Enable prompt caching for CODE_LAST_CHECK, FILL_BILL_TYPE, DUAL_LOB_CHECK, and GROUPER_BREAKOUT

- Add CODE_LAST_CHECK_INSTRUCTION() for service specificity classification
- Add FILL_BILL_TYPE_INSTRUCTION() for bill type code determination
- Add DUAL_LOB_CHECK_INSTRUCTION() for Medicare/Medicaid classification
- Update code_funcs.py to use caching for CODE_LAST_CHECK, FILL_BILL_TYPE, GROUPER_BREAKOUT
- Update postprocessing_funcs.py to use caching for DUAL_LOB_CHECK
- Add new instructions to get_cacheable_instructions() for cache warming
- Add unit tests for new instruction functions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix postprocessing_funcs to remove invalid columns

* Merge branch 'main' into feature/lesser-table-caching-refactor-hybrid

* Apply black formatting to 11 files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix import paths in testbed files

* Merge main into feature/update-caching

* Add SPLIT_REIMB_DATES_INSTRUCTION for prompt caching

* Merged main into feature/update-caching

* Merge remote-tracking branch 'origin/main' into feature/update-caching

* Merge branch 'feature/update-caching' of https://bitbucket.org/aarete/doczy.ai into feature/update-caching

* Add CODE_* instruction functions for prompt caching

* Merged main into feature/update-caching

* Merged main into feature/update-caching

* Maximize prompt caching by moving static content to instructions

- Refactored 12+ prompt functions to move static rules/examples to _INSTRUCTION()
- VALIDATE_REIMBURSEMENTS: Moved 45+ lines of validation rules to instruction
- DATE_FIX: Moved conversion rules and examples to instruction
- EXHIBIT_LEVEL, EXHIBIT_TITLE_MATCH, EXHIBIT_HEADER, EXHIBIT_LINKAGE: Moved rules
- FEE_SCHEDULE_BREAKOUT, GROUPER_BREAKOUT: Moved extraction rules
- CHECK_PROVIDER_NAME_MATCH: Moved matching rules to instruction
- LOB_RELATIONSHIP: Consolidated split instructions
- DERIVED_TERM_DATE: Moved calculation logic to instruction
- DYNAMIC_PRIMARY: Created separate HEADER and TEXT instructions
- Created new REIMB_DATES_ASSIGNMENT_INSTRUCTION with all date rules
- Updated get_cacheable_instructions() with new instruction functions
- Updated all callers in saas, clover, bcbs_promise pipelines

This enables ~90% cost savings via Claude API prompt caching by ensuring
static instruction text is cached and only dynamic content varies.

* Remove Dynamic Primary Header

* Standardize prompt caching across all LLM prompts for ~90% cost savings

Move static content to _INSTRUCTION() functions for caching:
- METHODOLOGY_BREAKOUT: 8 field definitions with resolved valid_values
- FEE_SCHEDULE_BREAKOUT: 3 field definitions
- GROUPER_BREAKOUT: 11 field definitions
- OUTLIER_BREAKOUT: 8 field definitions
- CODE_EXPLICIT: 8 field definitions
- EXHIBIT_HEADER: static header markers list
- CARVEOUT_CHECK: carveout and special case definitions
- SPECIAL_CASE_ASSIGNMENT: task rules and output format

Update callers to use cache=True, instruction, and usage_label params.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Apply black formatting to prompt_templates.py

* Update unit tests for prompt caching changes

- Fix function signatures in tests (METHODOLOGY_BREAKOUT, FEE_SCHEDULE_BREAKOUT, GROUPER_BREAKOUT, CODE_EXPLICIT)
- Add tests for CARVEOUT_CHECK and EXHIBIT_HEADER prompts
- Add TestInstructionFieldContent class to verify instruction functions contain field definitions
- Add tests for CARVEOUT_CHECK_INSTRUCTION and CODE_EXPLICIT_INSTRUCTION

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Refactor instructions to load fields from investment_prompts.json and Constants

Address PR review feedback:
- Add _get_constants() helper for lazy Constants instantiation
- Add _get_fields_text() helper to load fields from investment_prompts.json
- METHODOLOGY_BREAKOUT_INSTRUCTION: Load fields via FieldSet
- FEE_SCHEDULE_BREAKOUT_INSTRUCTION: Load fields via FieldSet
- GROUPER_BREAKOUT_INSTRUCTION: Load fields via FieldSet
- OUTLIER_BREAKOUT_INSTRUCTION: Load fields via FieldSet
- EXHIBIT_HEADER_INSTRUCTION: Load markers from Constants.EXHIBIT_HEADER_MARKERS
- CARVEOUT_CHECK_INSTRUCTION: Load carveouts from Constants.VALID_CARVEOUTS,
  special cases from investment_prompts.json
- CODE_EXPLICIT_INSTRUCTION: Keep hardcoded (no code_explicit field_type in JSON)

Update tests to match actual field names from investment_prompts.json.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Merge main into feature/update-caching

* Remove deprecated dynamic_primary_header

* Update CODE_EXPLICIT_INSTRUCTION to load fields from investment_prompts.json

Use field_type=code_primary_breakout to load code fields dynamically.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Merged main into feature/update-caching

* Merge main into feature/update-caching

* Merge branch 'feature/update-caching' of https://bitbucket.org/aarete/doczy.ai into feature/update-caching


Approved-by: Katon Minhas
This commit is contained in:
Faizan Mohiuddin
2026-01-30 23:23:37 +00:00
committed by Katon Minhas
parent ff8ec57ed0
commit 90bed0d5f6
10 changed files with 887 additions and 558 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/codes/code_funcs.py
+28 -8
View File
@@ -105,14 +105,14 @@ def code_explicit(service, methodology, filename):
Returns an empty dictionary if the service string is empty.
"""
# Prompt
code_primary_questions = FieldSet(
file_path=config.FIELD_JSON_PATH, field_type="code_primary_breakout"
).print_prompt_dict()
# Prompt - field definitions are now included in CODE_EXPLICIT_INSTRUCTION() for caching
claude_answer_raw = llm_utils.invoke_claude(
prompt_templates.CODE_EXPLICIT(service, methodology, code_primary_questions),
prompt_templates.CODE_EXPLICIT(service, methodology),
"sonnet_latest",
filename,
cache=True,
instruction=prompt_templates.CODE_EXPLICIT_INSTRUCTION(),
usage_label="CODE_EXPLICIT",
)
code_answer_dict = string_utils.universal_json_load(claude_answer_raw)
return code_answer_dict
@@ -143,6 +143,8 @@ def code_category(service, proc_category, hcpcs_level2_mapping, filename):
prompt_templates.CODE_CATEGORY(service, matching_values),
"sonnet_latest",
filename,
cache=True,
instruction=prompt_templates.CODE_CATEGORY_INSTRUCTION(),
)
claude_answer_final = string_utils.universal_json_load(
claude_answer_raw
@@ -176,7 +178,11 @@ def code_implicit_special(service, filename):
"""
claude_answer_raw = llm_utils.invoke_claude(
prompt_templates.CODE_IMPLICIT_SPECIAL(service), "sonnet_latest", filename
prompt_templates.CODE_IMPLICIT_SPECIAL(service),
"sonnet_latest",
filename,
cache=True,
instruction=prompt_templates.CODE_IMPLICIT_SPECIAL_INSTRUCTION(),
)
claude_answer_final = string_utils.extract_text_from_delimiters(
claude_answer_raw, Delimiter.PIPE
@@ -350,6 +356,8 @@ def code_implicit_rag(service, implicit_run_dict, filename, constants):
prompt_templates.CODE_IMPLICIT(service, level_dict["match_list"]),
"sonnet_latest",
filename,
cache=True,
instruction=prompt_templates.CODE_IMPLICIT_INSTRUCTION(),
)
try:
@@ -430,7 +438,11 @@ def code_last_check(service, filename):
"""
claude_answer_raw = llm_utils.invoke_claude(
prompt_templates.CODE_LAST_CHECK(service), "sonnet_latest", filename
prompt_templates.CODE_LAST_CHECK(service),
"sonnet_latest",
filename,
cache=True,
instruction=prompt_templates.CODE_LAST_CHECK_INSTRUCTION(),
)
claude_answer_final = string_utils.extract_text_from_delimiters(
claude_answer_raw, Delimiter.PIPE
@@ -470,6 +482,8 @@ def fill_bill_type(
),
"sonnet_latest",
"",
cache=True,
instruction=prompt_templates.FILL_BILL_TYPE_INSTRUCTION(),
)
claude_answer_final = string_utils.universal_json_load(claude_answer_raw)
@@ -484,6 +498,8 @@ def fill_bill_type(
),
"sonnet_latest",
"",
cache=True,
instruction=prompt_templates.FILL_BILL_TYPE_INSTRUCTION(),
)
claude_answer_final = string_utils.universal_json_load(claude_answer_raw)
@@ -843,7 +859,11 @@ def grouper_breakout(results_with_code: pd.DataFrame):
GROUPER_QUESTIONS,
)
claude_answer_raw = llm_utils.invoke_claude(
grouper_breakout_prompt, "sonnet_latest", ""
grouper_breakout_prompt,
"sonnet_latest",
"",
cache=True,
instruction=prompt_templates.GROUPER_BREAKOUT_INSTRUCTION(),
)
try:
grouper_answer = string_utils.universal_json_load(claude_answer_raw)
src/pipelines/clients/bcbs_promise/prompts/prompt_calls.py
+51 -23
View File
@@ -132,14 +132,15 @@ def prompt_reimbursement_primary(
def prompt_methodology_breakout(
service_term: str,
reimb_term: str,
methodology_breakout_questions: FieldSet,
constants: Constants,
filename: str,
):
"""
Call METHODOLOGY_BREAKOUT prompt with cached instruction.
Note: Field definitions are now included in METHODOLOGY_BREAKOUT_INSTRUCTION() for caching.
"""
prompt = prompt_templates.METHODOLOGY_BREAKOUT(
service_term,
reimb_term,
methodology_breakout_questions.print_prompt_dict(constants),
)
logging.debug(f"Methodology Breakout Prompt for {filename}: {prompt}")
llm_response = llm_utils.invoke_claude(
@@ -148,6 +149,7 @@ def prompt_methodology_breakout(
filename,
cache=True,
instruction=prompt_templates.METHODOLOGY_BREAKOUT_INSTRUCTION(),
usage_label="METHODOLOGY_BREAKOUT",
)
logging.debug(f"LLM Response for {filename}: {llm_response}")
try:
@@ -161,20 +163,24 @@ def prompt_methodology_breakout(
def prompt_fee_schedule_breakout(
methodology_breakout_dict: dict,
reimbursement_method: str,
fs_breakout_questions: FieldSet,
constants: Constants,
filename: str,
):
"""
Call FEE_SCHEDULE_BREAKOUT prompt with cached instruction.
Note: Field definitions are now included in FEE_SCHEDULE_BREAKOUT_INSTRUCTION() for caching.
"""
prompt = prompt_templates.FEE_SCHEDULE_BREAKOUT(
reimbursement_method,
methodology_breakout_dict.get("FEE_SCHEDULE"),
fs_breakout_questions.print_prompt_dict(constants),
)
logging.debug(f"Fee Schedule Breakout Prompt for {filename}: {prompt}")
llm_answer_raw = llm_utils.invoke_claude(
prompt,
"sonnet_latest",
filename,
cache=True,
instruction=prompt_templates.FEE_SCHEDULE_BREAKOUT_INSTRUCTION(),
usage_label="FEE_SCHEDULE_BREAKOUT",
)
logging.debug(f"LLM Response for {filename}: {llm_answer_raw}")
try:
@@ -187,20 +193,24 @@ def prompt_fee_schedule_breakout(
def prompt_grouper_breakout(
service: str,
reimbursement_method: str,
grouper_breakout_questions: FieldSet,
constants: Constants,
filename: str,
):
"""
Call GROUPER_BREAKOUT prompt with cached instruction.
Note: Field definitions are now included in GROUPER_BREAKOUT_INSTRUCTION() for caching.
"""
prompt = prompt_templates.GROUPER_BREAKOUT(
service,
reimbursement_method,
grouper_breakout_questions.print_prompt_dict(constants),
)
logging.debug(f"Grouper Breakout Prompt for {filename}: {prompt}")
llm_answer_raw = llm_utils.invoke_claude(
prompt,
"sonnet_latest",
filename,
cache=True,
instruction=prompt_templates.GROUPER_BREAKOUT_INSTRUCTION(),
usage_label="GROUPER_BREAKOUT",
)
logging.debug(f"LLM Response for {filename}: {llm_answer_raw}")
try:
@@ -214,13 +224,13 @@ def prompt_grouper_breakout(
def prompt_carveout_check(
service_term: str,
reimb_term: str,
carveout_definitions,
special_case_definitions,
filename: str,
):
carveout_prompt = prompt_templates.CARVEOUT_CHECK(
service_term, reimb_term, carveout_definitions, special_case_definitions
)
"""
Call CARVEOUT_CHECK prompt with cached instruction.
Note: Carveout and special case definitions are now included in CARVEOUT_CHECK_INSTRUCTION() for caching.
"""
carveout_prompt = prompt_templates.CARVEOUT_CHECK(service_term, reimb_term)
logging.debug(
f"Running carveout check for {filename} with prompt:\n{carveout_prompt}"
)
@@ -327,6 +337,9 @@ def prompt_special_case_assignment(
),
"sonnet_latest",
filename,
cache=True,
instruction=prompt_templates.SPECIAL_CASE_ASSIGNMENT_INSTRUCTION(),
usage_label="SPECIAL_CASE_ASSIGNMENT",
)
index_answer = string_utils.extract_text_from_delimiters(
llm_answer_raw, Delimiter.PIPE
@@ -468,25 +481,29 @@ def prompt_exhibit_linkage(page_header, previous_header, filename):
return exhibit_is_different
def prompt_exhibit_header(page_content, EXHIBIT_HEADER_MARKERS, filename):
def prompt_exhibit_header(page_content, filename):
"""Extract exhibit header identifier from page content using pattern matching.
Uses LLM to identify exhibit headers in page content based on predefined markers
and extract the exhibit identifier.
Note: Header markers are now included in EXHIBIT_HEADER_INSTRUCTION() for caching.
Args:
page_content (str): First 400 characters of page content to analyze.
EXHIBIT_HEADER_MARKERS (list): List of exhibit header pattern markers.
filename (str): Source filename for logging and LLM attribution.
Returns:
str: Extracted exhibit header identifier or marker indicating no header found.
"""
prompt = prompt_templates.EXHIBIT_HEADER(
page_content[0:400], EXHIBIT_HEADER_MARKERS
)
prompt = prompt_templates.EXHIBIT_HEADER(page_content[0:400])
llm_answer_raw = llm_utils.invoke_claude(
prompt, "sonnet_latest", filename, max_tokens=300
prompt,
"sonnet_latest",
filename,
max_tokens=300,
cache=True,
instruction=prompt_templates.EXHIBIT_HEADER_INSTRUCTION(),
usage_label="EXHIBIT_HEADER",
)
llm_answer_extracted = string_utils.extract_text_from_delimiters(
llm_answer_raw, Delimiter.PIPE
@@ -506,7 +523,14 @@ def prompt_date_fix(date: str) -> str:
if string_utils.is_empty(date):
return "N/A"
prompt = prompt_templates.DATE_FIX_PROMPT(date)
response = llm_utils.invoke_claude(prompt, "haiku_latest", "date_fix")
response = llm_utils.invoke_claude(
prompt,
"haiku_latest",
"date_fix",
cache=True,
instruction=prompt_templates.DATE_FIX_INSTRUCTION(),
usage_label="DATE_FIX",
)
return string_utils.extract_text_from_delimiters(response, Delimiter.PIPE)
@@ -550,6 +574,8 @@ def prompt_dynamic_assignment(
prompt = prompt_templates.REIMB_DATES_ASSIGNMENT(
service_term, reimb_term, field_prompt, exhibit_text_simplified, page_num
)
instruction = prompt_templates.REIMB_DATES_ASSIGNMENT_INSTRUCTION()
usage_label = "REIMB_DATES_ASSIGNMENT"
else:
# Use generic DYNAMIC_ASSIGNMENT for other fields
prompt = prompt_templates.DYNAMIC_ASSIGNMENT(
@@ -560,14 +586,16 @@ def prompt_dynamic_assignment(
exhibit_text_simplified,
page_num,
)
instruction = prompt_templates.DYNAMIC_ASSIGNMENT_INSTRUCTION()
usage_label = "DYNAMIC_ASSIGNMENT"
llm_answer_raw = llm_utils.invoke_claude(
prompt,
model_id="sonnet_latest",
filename=filename,
cache=True,
instruction=prompt_templates.DYNAMIC_ASSIGNMENT_INSTRUCTION(),
usage_label="DYNAMIC_ASSIGNMENT",
instruction=instruction,
usage_label=usage_label,
)
try:
src/pipelines/clients/clover/prompts/prompt_calls.py
+27 -21
View File
@@ -138,14 +138,15 @@ def prompt_reimbursement_primary(
def prompt_methodology_breakout(
service_term: str,
reimb_term: str,
methodology_breakout_questions: FieldSet,
constants: Constants,
filename: str,
):
"""
Call METHODOLOGY_BREAKOUT prompt with cached instruction.
Note: Field definitions are now included in METHODOLOGY_BREAKOUT_INSTRUCTION() for caching.
"""
prompt = prompt_templates.METHODOLOGY_BREAKOUT(
service_term,
reimb_term,
methodology_breakout_questions.print_prompt_dict(constants),
)
logging.debug(f"Methodology Breakout Prompt for {filename}: {prompt}")
llm_response = llm_utils.invoke_claude(
@@ -154,6 +155,7 @@ def prompt_methodology_breakout(
filename,
cache=True,
instruction=prompt_templates.METHODOLOGY_BREAKOUT_INSTRUCTION(),
usage_label="METHODOLOGY_BREAKOUT",
)
logging.debug(f"LLM Response for {filename}: {llm_response}")
try:
@@ -167,14 +169,15 @@ def prompt_methodology_breakout(
def prompt_fee_schedule_breakout(
methodology_breakout_dict: dict,
reimbursement_method: str,
fs_breakout_questions: FieldSet,
constants: Constants,
filename: str,
):
"""
Call FEE_SCHEDULE_BREAKOUT prompt with cached instruction.
Note: Field definitions are now included in FEE_SCHEDULE_BREAKOUT_INSTRUCTION() for caching.
"""
prompt = prompt_templates.FEE_SCHEDULE_BREAKOUT(
reimbursement_method,
methodology_breakout_dict.get("FEE_SCHEDULE"),
fs_breakout_questions.print_prompt_dict(constants),
)
logging.debug(f"Fee Schedule Breakout Prompt for {filename}: {prompt}")
llm_answer_raw = llm_utils.invoke_claude(
@@ -196,14 +199,15 @@ def prompt_fee_schedule_breakout(
def prompt_grouper_breakout(
service: str,
reimbursement_method: str,
grouper_breakout_questions: FieldSet,
constants: Constants,
filename: str,
):
"""
Call GROUPER_BREAKOUT prompt with cached instruction.
Note: Field definitions are now included in GROUPER_BREAKOUT_INSTRUCTION() for caching.
"""
prompt = prompt_templates.GROUPER_BREAKOUT(
service,
reimbursement_method,
grouper_breakout_questions.print_prompt_dict(constants),
)
logging.debug(f"Grouper Breakout Prompt for {filename}: {prompt}")
llm_answer_raw = llm_utils.invoke_claude(
@@ -226,13 +230,13 @@ def prompt_grouper_breakout(
def prompt_carveout_check(
service_term: str,
reimb_term: str,
carveout_definitions,
special_case_definitions,
filename: str,
):
carveout_prompt = prompt_templates.CARVEOUT_CHECK(
service_term, reimb_term, carveout_definitions, special_case_definitions
)
"""
Call CARVEOUT_CHECK prompt with cached instruction.
Note: Case definitions are now included in CARVEOUT_CHECK_INSTRUCTION() for caching.
"""
carveout_prompt = prompt_templates.CARVEOUT_CHECK(service_term, reimb_term)
logging.debug(
f"Running carveout check for {filename} with prompt:\n{carveout_prompt}"
)
@@ -509,23 +513,21 @@ def prompt_exhibit_linkage(page_header, previous_header, filename):
return exhibit_is_different
def prompt_exhibit_header(page_content, EXHIBIT_HEADER_MARKERS, filename):
def prompt_exhibit_header(page_content, filename):
"""Extract exhibit header identifier from page content using pattern matching.
Uses LLM to identify exhibit headers in page content based on predefined markers
and extract the exhibit identifier.
Note: Header markers are now included in EXHIBIT_HEADER_INSTRUCTION() for caching.
Args:
page_content (str): First 400 characters of page content to analyze.
EXHIBIT_HEADER_MARKERS (list): List of exhibit header pattern markers.
filename (str): Source filename for logging and LLM attribution.
Returns:
str: Extracted exhibit header identifier or marker indicating no header found.
"""
prompt = prompt_templates.EXHIBIT_HEADER(
page_content[0:400], EXHIBIT_HEADER_MARKERS
)
prompt = prompt_templates.EXHIBIT_HEADER(page_content[0:400])
llm_answer_raw = llm_utils.invoke_claude(
prompt,
"sonnet_latest",
@@ -611,6 +613,8 @@ def prompt_dynamic_assignment(
prompt = prompt_templates.REIMB_DATES_ASSIGNMENT(
service_term, reimb_term, field_prompt, exhibit_text_simplified, page_num
)
instruction = prompt_templates.REIMB_DATES_ASSIGNMENT_INSTRUCTION()
usage_label = "REIMB_DATES_ASSIGNMENT"
else:
# Use generic DYNAMIC_ASSIGNMENT for other fields
prompt = prompt_templates.DYNAMIC_ASSIGNMENT(
@@ -621,14 +625,16 @@ def prompt_dynamic_assignment(
exhibit_text_simplified,
page_num,
)
instruction = prompt_templates.DYNAMIC_ASSIGNMENT_INSTRUCTION()
usage_label = "DYNAMIC_ASSIGNMENT"
llm_answer_raw = llm_utils.invoke_claude(
prompt,
model_id="sonnet_latest",
filename=filename,
cache=True,
instruction=prompt_templates.DYNAMIC_ASSIGNMENT_INSTRUCTION(),
usage_label="DYNAMIC_ASSIGNMENT",
instruction=instruction,
usage_label=usage_label,
)
try:
src/pipelines/saas/prompts/prompt_calls.py
+27 -21
View File
@@ -137,14 +137,15 @@ def prompt_reimbursement_primary(
def prompt_methodology_breakout(
service_term: str,
reimb_term: str,
methodology_breakout_questions: FieldSet,
constants: Constants,
filename: str,
):
"""
Call METHODOLOGY_BREAKOUT prompt with cached instruction.
Note: Field definitions are now included in METHODOLOGY_BREAKOUT_INSTRUCTION() for caching.
"""
prompt = prompt_templates.METHODOLOGY_BREAKOUT(
service_term,
reimb_term,
methodology_breakout_questions.print_prompt_dict(constants),
)
logging.debug(f"Methodology Breakout Prompt for {filename}: {prompt}")
llm_response = llm_utils.invoke_claude(
@@ -153,6 +154,7 @@ def prompt_methodology_breakout(
filename,
cache=True,
instruction=prompt_templates.METHODOLOGY_BREAKOUT_INSTRUCTION(),
usage_label="METHODOLOGY_BREAKOUT",
)
logging.debug(f"LLM Response for {filename}: {llm_response}")
try:
@@ -166,14 +168,15 @@ def prompt_methodology_breakout(
def prompt_fee_schedule_breakout(
methodology_breakout_dict: dict,
reimbursement_method: str,
fs_breakout_questions: FieldSet,
constants: Constants,
filename: str,
):
"""
Call FEE_SCHEDULE_BREAKOUT prompt with cached instruction.
Note: Field definitions are now included in FEE_SCHEDULE_BREAKOUT_INSTRUCTION() for caching.
"""
prompt = prompt_templates.FEE_SCHEDULE_BREAKOUT(
reimbursement_method,
methodology_breakout_dict.get("FEE_SCHEDULE"),
fs_breakout_questions.print_prompt_dict(constants),
)
logging.debug(f"Fee Schedule Breakout Prompt for {filename}: {prompt}")
llm_answer_raw = llm_utils.invoke_claude(
@@ -195,14 +198,15 @@ def prompt_fee_schedule_breakout(
def prompt_grouper_breakout(
service: str,
reimbursement_method: str,
grouper_breakout_questions: FieldSet,
constants: Constants,
filename: str,
):
"""
Call GROUPER_BREAKOUT prompt with cached instruction.
Note: Field definitions are now included in GROUPER_BREAKOUT_INSTRUCTION() for caching.
"""
prompt = prompt_templates.GROUPER_BREAKOUT(
service,
reimbursement_method,
grouper_breakout_questions.print_prompt_dict(constants),
)
logging.debug(f"Grouper Breakout Prompt for {filename}: {prompt}")
llm_answer_raw = llm_utils.invoke_claude(
@@ -225,13 +229,13 @@ def prompt_grouper_breakout(
def prompt_carveout_check(
service_term: str,
reimb_term: str,
carveout_definitions,
special_case_definitions,
filename: str,
):
carveout_prompt = prompt_templates.CARVEOUT_CHECK(
service_term, reimb_term, carveout_definitions, special_case_definitions
)
"""
Call CARVEOUT_CHECK prompt with cached instruction.
Note: Case definitions are now included in CARVEOUT_CHECK_INSTRUCTION() for caching.
"""
carveout_prompt = prompt_templates.CARVEOUT_CHECK(service_term, reimb_term)
logging.debug(
f"Running carveout check for {filename} with prompt:\n{carveout_prompt}"
)
@@ -489,23 +493,21 @@ def prompt_exhibit_linkage(page_header, previous_header, filename):
return exhibit_is_different
def prompt_exhibit_header(page_content, EXHIBIT_HEADER_MARKERS, filename):
def prompt_exhibit_header(page_content, filename):
"""Extract exhibit header identifier from page content using pattern matching.
Uses LLM to identify exhibit headers in page content based on predefined markers
and extract the exhibit identifier.
Note: Header markers are now included in EXHIBIT_HEADER_INSTRUCTION() for caching.
Args:
page_content (str): First 400 characters of page content to analyze.
EXHIBIT_HEADER_MARKERS (list): List of exhibit header pattern markers.
filename (str): Source filename for logging and LLM attribution.
Returns:
str: Extracted exhibit header identifier or marker indicating no header found.
"""
prompt = prompt_templates.EXHIBIT_HEADER(
page_content[0:400], EXHIBIT_HEADER_MARKERS
)
prompt = prompt_templates.EXHIBIT_HEADER(page_content[0:400])
llm_answer_raw = llm_utils.invoke_claude(
prompt,
"sonnet_latest",
@@ -591,6 +593,8 @@ def prompt_dynamic_assignment(
prompt = prompt_templates.REIMB_DATES_ASSIGNMENT(
service_term, reimb_term, field_prompt, exhibit_text_simplified, page_num
)
instruction = prompt_templates.REIMB_DATES_ASSIGNMENT_INSTRUCTION()
usage_label = "REIMB_DATES_ASSIGNMENT"
else:
# Use generic DYNAMIC_ASSIGNMENT for other fields
prompt = prompt_templates.DYNAMIC_ASSIGNMENT(
@@ -601,14 +605,16 @@ def prompt_dynamic_assignment(
exhibit_text_simplified,
page_num,
)
instruction = prompt_templates.DYNAMIC_ASSIGNMENT_INSTRUCTION()
usage_label = "DYNAMIC_ASSIGNMENT"
llm_answer_raw = llm_utils.invoke_claude(
prompt,
model_id="sonnet_latest",
filename=filename,
cache=True,
instruction=prompt_templates.DYNAMIC_ASSIGNMENT_INSTRUCTION(),
usage_label="DYNAMIC_ASSIGNMENT",
instruction=instruction,
usage_label=usage_label,
)
try:
src/pipelines/shared/extraction/one_to_n_funcs.py
+12 -9
View File
@@ -166,7 +166,12 @@ def breakout(
def process_single_carveout(
answer_dict, carveout_definitions, special_case_definitions, filename
):
"""Process a single carveout/special case in parallel"""
"""Process a single carveout/special case in parallel.
Note: carveout_definitions and special_case_definitions are still passed to this function
for the categorization logic below, but are no longer passed to prompt_carveout_check
since they are now included in CARVEOUT_CHECK_INSTRUCTION() for caching.
"""
service_term, reimb_term = answer_dict.get("SERVICE_TERM"), answer_dict.get(
"REIMB_TERM"
)
@@ -174,8 +179,6 @@ def process_single_carveout(
reimbursement_categorization = prompt_calls.prompt_carveout_check(
service_term,
reimb_term,
carveout_definitions,
special_case_definitions,
filename,
)
@@ -341,12 +344,11 @@ def methodology_breakout_single_row(
)
# Methodology Breakout: AARETE_DERIVED_REIMB_METHOD, REIMB_FEE_RATE, REIMB_PCT_RATE, LESSER_OF_IND, GREATER_OF_IND, AARETE_DERIVED_PROV_TYPE
# Note: Field definitions are now included in METHODOLOGY_BREAKOUT_INSTRUCTION() for caching
if reimb_term:
methodology_breakout_answers = prompt_calls.prompt_methodology_breakout(
service_term,
reimb_term,
methodology_breakout_questions,
constants,
filename,
)
if isinstance(
@@ -420,13 +422,12 @@ def methodology_breakout_secondary(
"AARETE_DERIVED_REIMB_METHOD", ""
)
# Fee Schedule Breakout
# Note: Field definitions are now included in FEE_SCHEDULE_BREAKOUT_INSTRUCTION() for caching
if AARETE_DERIVED_REIMB_METHOD == "Fee Schedule":
# Prompt for FS breakout, fill grouper with N/A
fs_breakout_dict = prompt_calls.prompt_fee_schedule_breakout(
methodology_breakout_dict,
reimb_term,
fs_breakout_questions,
constants,
filename,
)
fs_breakout_dict.update(
@@ -437,12 +438,11 @@ def methodology_breakout_secondary(
)
return fs_breakout_dict
# Grouper Breakout
# Note: Field definitions are now included in GROUPER_BREAKOUT_INSTRUCTION() for caching
elif AARETE_DERIVED_REIMB_METHOD == "Grouper":
grouper_breakout_dict = prompt_calls.prompt_grouper_breakout(
service_term,
reimb_term,
grouper_breakout_questions,
constants,
filename,
)
grouper_breakout_dict.update(
@@ -681,6 +681,9 @@ def split_reimb_dates(one_to_n_results: list, filename: str) -> list:
model_id="sonnet_latest",
filename=filename,
max_tokens=200,
cache=True,
instruction=prompt_templates.SPLIT_REIMB_DATES_INSTRUCTION(),
usage_label="SPLIT_REIMB_DATES",
)
# Extract the effective and termination dates from the LLM output
src/pipelines/shared/postprocessing/postprocessing_funcs.py
+6 -1
View File
@@ -543,7 +543,12 @@ def update_lob_for_duals(answer_dicts):
prompt = prompt_templates.DUAL_LOB_CHECK(service_term)
logging.debug(f"Prompt for dual LOB check: {prompt}")
claude_answer_raw = llm_utils.invoke_claude(
prompt, model_id="legacy_sonnet", filename="", max_tokens=200
prompt,
model_id="legacy_sonnet",
filename="",
max_tokens=200,
cache=True,
instruction=prompt_templates.DUAL_LOB_CHECK_INSTRUCTION(),
)
logging.debug(f"Claude response for dual LOB check: {claude_answer_raw}")
claude_answer_extracted = string_utils.extract_text_from_delimiters(
src/pipelines/shared/preprocessing/preprocessing_funcs.py
+2 -3
View File
@@ -187,9 +187,8 @@ def get_exhibit_pages(
page_content = page_dict[page_num]
if "." not in page_num or ".0" in page_num:
exhibit_header = prompt_calls.prompt_exhibit_header(
page_content, EXHIBIT_HEADER_MARKERS, filename
)
# Note: Header markers are now included in EXHIBIT_HEADER_INSTRUCTION() for caching
exhibit_header = prompt_calls.prompt_exhibit_header(page_content, filename)
if "N/A" in exhibit_header:
is_exhibit = False
else:
src/prompts/prompt_templates.py
+583 -461
View File
File diff suppressed because it is too large Load Diff
src/scripts/qa_qc.py
-1
View File
@@ -15,7 +15,6 @@ import config
from qa_qc_helpers import perform_qc_qa, save_qcqa_wb
from consolidate_output import consolidate_output
file_name_column = config.FILE_NAME_COLUMN
"""python scripts/qa_qc.py input_dir=______ output_dir=_______ ac_df=________ b_df=______ read_mode=____ df_read_mode=________
src/tests/test_prompt_caching.py
+151 -10
View File
@@ -20,10 +20,10 @@ class TestPromptTemplatesReturnStrings(unittest.TestCase):
self.assertIn("test context", result)
def test_methodology_breakout_returns_string(self):
"""Test METHODOLOGY_BREAKOUT returns a prompt string."""
result = prompt_templates.METHODOLOGY_BREAKOUT(
"service term", "reimb term", "questions"
)
"""Test METHODOLOGY_BREAKOUT returns a prompt string.
Note: Field definitions are now in METHODOLOGY_BREAKOUT_INSTRUCTION() for caching.
"""
result = prompt_templates.METHODOLOGY_BREAKOUT("service term", "reimb term")
self.assertIsInstance(result, str)
self.assertIn("service term", result)
@@ -61,18 +61,20 @@ class TestPromptTemplatesReturnStrings(unittest.TestCase):
self.assertIn("reimb term", result)
def test_fee_schedule_breakout_returns_string(self):
"""Test FEE_SCHEDULE_BREAKOUT returns a prompt string."""
result = prompt_templates.FEE_SCHEDULE_BREAKOUT(
"methodology", "fee schedule", "questions"
)
"""Test FEE_SCHEDULE_BREAKOUT returns a prompt string.
Note: Field definitions are now in FEE_SCHEDULE_BREAKOUT_INSTRUCTION() for caching.
"""
result = prompt_templates.FEE_SCHEDULE_BREAKOUT("methodology", "fee schedule")
self.assertIsInstance(result, str)
self.assertIn("methodology", result)
self.assertIn("fee schedule", result)
def test_grouper_breakout_returns_string(self):
"""Test GROUPER_BREAKOUT returns a prompt string."""
result = prompt_templates.GROUPER_BREAKOUT("service", "term", "questions")
"""Test GROUPER_BREAKOUT returns a prompt string.
Note: Field definitions are now in GROUPER_BREAKOUT_INSTRUCTION() for caching.
"""
result = prompt_templates.GROUPER_BREAKOUT("service", "term")
self.assertIsInstance(result, str)
self.assertIn("service", result)
@@ -105,6 +107,35 @@ class TestPromptTemplatesReturnStrings(unittest.TestCase):
self.assertIn("Payer Name", result)
self.assertIn("contract context", result)
def test_carveout_check_returns_string(self):
"""Test CARVEOUT_CHECK returns a prompt string.
Note: Case definitions are now in CARVEOUT_CHECK_INSTRUCTION() for caching.
"""
result = prompt_templates.CARVEOUT_CHECK("service term", "reimb term")
self.assertIsInstance(result, str)
self.assertIn("service term", result)
self.assertIn("reimb term", result)
def test_exhibit_header_returns_string(self):
"""Test EXHIBIT_HEADER returns a prompt string.
Note: Header markers are now in EXHIBIT_HEADER_INSTRUCTION() for caching.
"""
result = prompt_templates.EXHIBIT_HEADER("page context text")
self.assertIsInstance(result, str)
self.assertIn("page context text", result)
def test_code_explicit_returns_string(self):
"""Test CODE_EXPLICIT returns a prompt string.
Note: Field definitions are now in CODE_EXPLICIT_INSTRUCTION() for caching.
"""
result = prompt_templates.CODE_EXPLICIT("Lab Services", "Medicare Fee Schedule")
self.assertIsInstance(result, str)
self.assertIn("Lab Services", result)
self.assertIn("Medicare Fee Schedule", result)
class TestInstructionFunctions(unittest.TestCase):
"""Test that _INSTRUCTION() functions return static instruction text."""
@@ -231,6 +262,116 @@ class TestInstructionFunctions(unittest.TestCase):
self.assertIsInstance(result, str)
self.assertIn("OBJECTIVE", result)
def test_carveout_check_instruction_returns_string(self):
"""Test CARVEOUT_CHECK_INSTRUCTION returns a string with case definitions."""
result = prompt_templates.CARVEOUT_CHECK_INSTRUCTION()
self.assertIsInstance(result, str)
self.assertIn("OBJECTIVE", result)
# Should contain static carveout definitions
self.assertIn("BASE_COVERED_SERVICES", result)
self.assertIn("PAYMENT_CARVEOUT", result)
# Should contain static special case definitions
self.assertIn("OUTLIER_TERM", result)
self.assertIn("STOP_LOSS_TERM", result)
def test_code_explicit_instruction_returns_string(self):
"""Test CODE_EXPLICIT_INSTRUCTION returns a string with field definitions.
Field definitions are loaded from investment_prompts.json (field_type=code_primary_breakout).
"""
result = prompt_templates.CODE_EXPLICIT_INSTRUCTION()
self.assertIsInstance(result, str)
self.assertIn("OBJECTIVE", result)
# Should contain field definitions from investment_prompts.json
self.assertIn("PROCEDURE_CD", result)
self.assertIn("REVENUE_CD", result)
self.assertIn("DIAG_CD", result)
class TestInstructionFieldContent(unittest.TestCase):
"""Test that _INSTRUCTION() functions contain expected static field definitions."""
def test_methodology_breakout_instruction_contains_fields(self):
"""Test METHODOLOGY_BREAKOUT_INSTRUCTION contains field definitions."""
result = prompt_templates.METHODOLOGY_BREAKOUT_INSTRUCTION()
self.assertIsInstance(result, str)
# Should contain field definitions
self.assertIn("LESSER_OF_IND", result)
self.assertIn("AARETE_DERIVED_REIMB_METHOD", result)
self.assertIn("REIMB_PCT_RATE", result)
self.assertIn("REIMB_FEE_RATE", result)
# Should contain valid values
self.assertIn("Fee Schedule", result)
self.assertIn("Grouper", result)
self.assertIn("Per Diem", result)
def test_fee_schedule_breakout_instruction_contains_fields(self):
"""Test FEE_SCHEDULE_BREAKOUT_INSTRUCTION contains field definitions from investment_prompts.json."""
result = prompt_templates.FEE_SCHEDULE_BREAKOUT_INSTRUCTION()
self.assertIsInstance(result, str)
# Should contain field definitions (from investment_prompts.json)
self.assertIn("AARETE_DERIVED_FEE_SCHEDULE", result)
self.assertIn("FEE_SCHEDULE_VERSION", result)
self.assertIn("AARETE_DERIVED_FEE_SCHEDULE_VERSION", result)
# Should contain valid values (from Constants)
self.assertIn("CMS", result)
self.assertIn("State", result)
def test_grouper_breakout_instruction_contains_fields(self):
"""Test GROUPER_BREAKOUT_INSTRUCTION contains field definitions from investment_prompts.json."""
result = prompt_templates.GROUPER_BREAKOUT_INSTRUCTION()
self.assertIsInstance(result, str)
# Should contain field definitions (from investment_prompts.json)
self.assertIn("GROUPER_TYPE", result)
self.assertIn("GROUPER_VERSION", result)
self.assertIn("AARETE_DERIVED_GROUPER_VERSION", result)
# Should contain grouper-related terms from field prompts
self.assertIn("MS-DRG", result)
self.assertIn("APR-DRG", result)
def test_outlier_breakout_instruction_contains_fields(self):
"""Test OUTLIER_BREAKOUT_INSTRUCTION contains field definitions from investment_prompts.json."""
result = prompt_templates.OUTLIER_BREAKOUT_INSTRUCTION()
self.assertIsInstance(result, str)
# Should contain field definitions (from investment_prompts.json)
self.assertIn("OUTLIER_FIRST_DOLLAR_IND", result)
self.assertIn("OUTLIER_FIXED_LOSS_THRESHOLD", result)
self.assertIn("OUTLIER_PCT_RATE", result)
def test_exhibit_header_instruction_contains_markers(self):
"""Test EXHIBIT_HEADER_INSTRUCTION contains static header markers."""
result = prompt_templates.EXHIBIT_HEADER_INSTRUCTION()
self.assertIsInstance(result, str)
# Should contain header markers
self.assertIn("EXHIBIT", result)
self.assertIn("ATTACHMENT", result)
self.assertIn("SCHEDULE", result)
self.assertIn("ADDENDUM", result)
def test_carveout_check_instruction_contains_all_definitions(self):
"""Test CARVEOUT_CHECK_INSTRUCTION contains all carveout and special case definitions."""
result = prompt_templates.CARVEOUT_CHECK_INSTRUCTION()
# Carveout definitions
carveout_types = [
"BASE_COVERED_SERVICES",
"PAYMENT_CARVEOUT",
"TRIGGER_CAP",
"ADDITION",
"DEFAULT_TERM",
"UNLISTED_CODE",
]
for carveout in carveout_types:
self.assertIn(carveout, result)
# Special case definitions
special_cases = [
"OUTLIER_TERM",
"STOP_LOSS_TERM",
"DISCOUNT_TERM",
"PREMIUM_TERM",
]
for case in special_cases:
self.assertIn(case, result)
class TestGetCacheableInstructions(unittest.TestCase):
"""Test get_cacheable_instructions function returns expected instructions."""