jacob-mathison/doczyai-pipelines
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

Merged in bugfix/bill_type_code_fixes (pull request #819)

Browse Source 
Bugfix/bill type code fixes

* Fix LOB_PROGRAM_RELATIONSHIP extraction and preserve debugging code

- Add format requirements to LOB_RELATIONSHIP_INSTRUCTION for proper caching
- Strengthen LOB_RELATIONSHIP prompt with explicit pipe delimiter requirements
- Preserve all DEBUG blocks and print statements in dynamic_funcs.py and file_processing.py
- Update crosswalk mappings and gitignore
- Ensure format instructions are cached at API level to prevent missing pipe delimiters

* Merge main into bugfix/dynamic_issuefixes

Resolved merge conflicts in:
- fieldExtraction/src/investment/dynamic_funcs.py (merged debug print statements)
- fieldExtraction/src/investment/file_processing.py (merged debug statements and exhibit inheritance code)

All conflicts resolved successfully.

* Update FILL_BILL_TYPE prompt with place-of-service priority and optional exhibit_text

- Add optional exhibit_text parameter to FILL_BILL_TYPE function
- Update instructions to prioritize place-of-service over service-only terminology
- Add new examples: 'home dialysis' and 'dialysis' while keeping original examples
- Add conditional exhibit context section when exhibit_text is provided
- Maintains backward compatibility (exhibit_text defaults to None)

Ref: PLAN_BILL_TYPE_CD_IMPROVEMENTS.md

* Implement two-step bill type CD determination with exhibit text fallback

- Add EXHIBIT_TEXT to answer_dict in combine_one_to_n() for contextual analysis
- Update fill_bill_type() to use two-step approach:
  1. First attempt: service term only (no exhibit text)
  2. Second attempt: service term + exhibit text (only if first fails)
- Update extract_codes_from_service() to pass exhibit_text to fill_bill_type()
- Add debug print when full exhibit text context is used
- More efficient: only makes second LLM call when service term alone fails

Ref: PLAN_BILL_TYPE_CD_IMPROVEMENTS.md

* Remove debug print statement and merge main into bill_type_code_fixes

- Remove debug print statement from fill_bill_type() in code_funcs.py
- Merge latest changes from main branch
- Includes updates to prompt_templates.py (payer_name parameter)
- Includes updates to file_processing.py (provider info ordering)
- Includes updates to aarete_derived.py, crosswalk mappings, and tin_npi_funcs

* Remove remaining debug print statements from dynamic_funcs.py and file_processing.py


Approved-by: Katon Minhas
This commit is contained in:
Praneel Panchigar
2025-12-26 18:24:57 +00:00
committed by Katon Minhas
parent b64e1fe4cc
commit 85f2c8cecc
5 changed files with 43 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fieldExtraction/src/codes/code_funcs.py
+21 -2
View File
@@ -408,24 +408,41 @@ def code_last_check(service, filename):
return claude_answer_final
def fill_bill_type(service, answer_dict, BILL_TYPE_MAPPING, BILL_TYPE_REVERSE_MAPPING):
def fill_bill_type(service, answer_dict, BILL_TYPE_MAPPING, BILL_TYPE_REVERSE_MAPPING, exhibit_text=None):
"""
Fills the BILL_TYPE_CD and BILL_TYPE_CD_DESC fields in the answer dictionary.
Args:
service (str): The service term.
answer_dict (dict): The answer dictionary to update.
BILL_TYPE_MAPPING (dict): Mapping from codes to descriptions.
BILL_TYPE_REVERSE_MAPPING (dict): Mapping from descriptions to codes.
exhibit_text (str, optional): Full exhibit text for contextual analysis.
Returns:
dict: The updated answer dictionary.
"""
valid_bill_type = sorted(list(set(BILL_TYPE_MAPPING.values())))
# First attempt: Service term only
claude_answer_raw = llm_utils.invoke_claude(
prompt_templates.FILL_BILL_TYPE(service, valid_bill_type), "sonnet_latest", ""
prompt_templates.FILL_BILL_TYPE(service, valid_bill_type, exhibit_text=None),
"sonnet_latest",
""
)
claude_answer_final = string_utils.universal_json_load(claude_answer_raw)
# Second attempt: Service term + exhibit context (if first attempt failed and context available)
if not claude_answer_final and exhibit_text:
claude_answer_raw = llm_utils.invoke_claude(
prompt_templates.FILL_BILL_TYPE(service, valid_bill_type, exhibit_text=exhibit_text),
"sonnet_latest",
""
)
claude_answer_final = string_utils.universal_json_load(claude_answer_raw)
# Keep legacy behavior: return answer_dict unchanged if no result found
if not claude_answer_final:
return answer_dict
@@ -581,11 +598,13 @@ def extract_codes_from_service(answer_dict, constants: Constants):
# Fill Bill Type if not filled
if string_utils.is_empty(bill_type):
exhibit_text = answer_dict.get("EXHIBIT_TEXT", None)
answer_dict = fill_bill_type(
service,
answer_dict,
constants.BILL_TYPE_MAPPING,
constants.BILL_TYPE_REVERSE_MAPPING,
exhibit_text=exhibit_text,
)
# Get list of codes we want implicit - based on claim type
fieldExtraction/src/investment/dynamic_funcs.py
+2 -1
View File
@@ -35,7 +35,7 @@ def dynamic_primary(
dynamic_reimbursement_fields = FieldSet()
exhibit_level_answer_dict = {}
for field in dynamic_primary_fields.fields:
exhibit_text_answer = prompt_calls.prompt_dynamic_primary(
exhibit_text,
@@ -242,6 +242,7 @@ def get_dynamic_one_to_one_fields(answer_dicts: list[dict[str, str]], constants:
one_to_one_fields, field_to_add, answer_dicts, constants
)
return one_to_one_fields
fieldExtraction/src/investment/file_processing.py
+4
View File
@@ -60,6 +60,7 @@ def process_file(file_object, constants: Constants, run_timestamp):
# ONE TO ONE PROCESSING
if process_one_to_one:
one_to_one_results = run_one_to_one_prompts(
filename,
contract_text,
@@ -244,6 +245,7 @@ def run_one_to_n_prompts(text_dict: dict[str, str],
################################ Add to Total ###############################
one_to_n_results += all_exhibit_rows
################################ Store Current Exhibit for Next Iteration ###############################
# Store original (non-combined) values for next exhibit to potentially inherit from
@@ -263,6 +265,7 @@ def run_one_to_n_prompts(text_dict: dict[str, str],
dynamic_one_to_one_fields = dynamic_funcs.get_dynamic_one_to_one_fields(
one_to_n_results, constants
)
################## CONVERT TO DF ##################
one_to_n_df = pd.DataFrame(one_to_n_results)
@@ -295,6 +298,7 @@ def process_page_one_to_n(text_dict: dict[str, str], exhibit_page_nums: list[str
reimbursement_level_answers, special_case_answers = one_to_n_funcs.carveout_and_special_case(
reimbursement_level_answers, constants, filename
) # Breaks the reimbursement answers into reimbursments (regular lines) and special cases (distributed across exhibit)
############################### Dynamic Assignment ###############################
reimbursement_level_answers = dynamic_funcs.dynamic_assignment(reimbursement_level_answers, dynamic_reimbursement_fields, exhibit_text_simplified, page_num, constants, filename)
fieldExtraction/src/investment/row_funcs.py
+2
View File
@@ -26,6 +26,8 @@ def combine_one_to_n(
for key, value in exhibit_level_answers.items():
if key not in combined_dict or string_utils.is_empty(combined_dict.get(key)):
combined_dict[key] = value
# Add exhibit text for contextual analysis (e.g., for bill type code determination)
combined_dict["EXHIBIT_TEXT"] = exhibit_text
combined_answers.append(combined_dict)
# Initialize final_combined_answers with the current combined list so that it has value even if no special cases are present
fieldExtraction/src/prompts/prompt_templates.py
+14 -2
View File
@@ -814,7 +814,16 @@ Briefly explain your answer, then put your final answer in |pipes|.
"""
def FILL_BILL_TYPE(service, choices):
def FILL_BILL_TYPE(service, choices, exhibit_text=None):
# Build exhibit context section conditionally
exhibit_context_section = ""
if exhibit_text:
exhibit_context_section = f"""
[EXHIBIT CONTEXT]
If the Service Term alone does not provide sufficient information, examine the following exhibit text for additional context about where the service is performed:
{exhibit_text.replace('"', "'")}
"""
return f"""Analyze a given medical Service Term.
What Bill Type Code Description does the Service Term fall under?
@@ -824,16 +833,19 @@ Here are the descriptions to choose from:
{choices}
[INSTRUCTIONS]
- While determining the bill type code, first prioritize the place where the service is taking place, that will always take precedence over the service only terminology. For example in home dialysis, we can recognize home as place of service and only need to identify that for code determination. However if place of service is not available fall back on the service term as long as it is clearly and unambiguously referred to by the service and is present in [VALID DESCRIPTIONS].
- Answer with the Description or Descriptions from [VALID DESCRIPTIONS] only if it is clearly and unambiguously referred to by the Service. Do not attempt much inference, the answer will be very clear if it is present.
- It is highly likely that none of the Descriptions will be a clear and unambiguous match. When this is the case, return an empty list: []
- Here are some examples:
- "Ambulatory Surgery Procedures" --> ["Ambulatory Surgery Center"]
- "Hospital Services" --> ["Inpatient Hospital", "Outpatient Hospital"]
- "home dialysis" --> ["Home"]
- "dialysis" --> ["Dialysis Center"]
[CONTEXT]
Here is the Service Term to analyze:
{service.replace('"', "'")}
{exhibit_context_section}
[OUTPUT FORMAT]
Briefly explain your answer, then return your final answer in JSON list format.
"""