Katon Minhas
f8b8a580ff
fields/additional initializations * Set up test file * New fields + new field names * Add initial prompts to json * Column renaming - remove CONTRACT_ prefix * update tests * testing * Fix regex pattern in json.loads * Merge branch 'hotfix/file-errors' into refactor/new-field-names * add UNIT_OF_MEASURE * initialize new values * Add rate change interval * Merge branch 'main' into fields/additional-initializations * remove test * replace field names * update REIMB_ID * update date columns * update reimbursement effective dt * Revert adhoc scripts * Revert client files * Update postprocessing_funcs * revert regex_utils * Fix valid issue * UNIT_OF_MEASURE * REIMB_TERM * Revert postprocessing funcs tests Approved-by: Alex Galarce
355 lines
12 KiB
Plaintext
355 lines
12 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Manual Test Bed\n",
|
|
"Runs contracts through from `input_data`, but cannot yet automatically check them against the answer key (that workbook is unreliable in its answers)\n",
|
|
"\n",
|
|
"To make this work: in `config.py`, change READ_MODE and RUN_MODE to 'local'.\n",
|
|
"Create an .env in the base directory of your project according to these instructions:\n",
|
|
"https://aarete.atlassian.net/wiki/spaces/DoczyAI/pages/1283751954/Using+.env+for+AWS+keys\n",
|
|
"\n",
|
|
"Feel free to comment out parts of `file_processing.py` that do not apply to the fields you are optimizing. For instance, if you're optimizing a smart chunked field, comment out the `one_to_n_fields` block and feed just `one_to_n_results` into `postprocess.postprocess`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"import pandas as pd"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"os.chdir(\"../../\") # go to FieldExtraction folder\n",
|
|
"TEST_DATA_INPUT_DIR = \"scripts/investment_field_testing/input_data\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"For now, we are no longer doing anything with the test bed"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# # Load target data from `scripts/investment_field_testing/input_data` directory\n",
|
|
"\n",
|
|
"# df_targets = pd.read_excel(TEST_DATA_INPUT_DIR + \"/test_bed_investment.xlsx\", sheet_name=\"test_bed_investment\")\n",
|
|
"\n",
|
|
"# print(df_targets.shape)\n",
|
|
"\n",
|
|
"# df_targets.head()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Also change config.RUN_MODE and config.READ_MODE to 'local'\n",
|
|
"test_params = {}\n",
|
|
"# test_params['local_input_dir'] = TEST_DATA_INPUT_DIR + \"/investment_228\"\n",
|
|
"test_params['local_input_dir'] = TEST_DATA_INPUT_DIR + \"/mix_all_358\"\n",
|
|
"test_params['max_workers'] = 1\n",
|
|
"test_params['input_files'] = [\"2001-11-01 California Emergency Physicians PSA_MU.txt\"] # you can specify specific files to run here"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"doczy-output exists\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from src.investment.main import main as investment_main"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Total Input Files : 1\n",
|
|
"Processing 2001-11-01 California Emergency Physicians PSA_MU.txt...\n",
|
|
"Preprocessing Complete - 2001-11-01 California Emergency Physicians PSA_MU.txt\n",
|
|
"Running regex fields...\n",
|
|
"Running smart chunked fields...\n",
|
|
"\n",
|
|
"fields: [<src.prompts.investment_prompts.Field object at 0x7f20a7569400>, <src.prompts.investment_prompts.Field object at 0x7f20a7569340>, <src.prompts.investment_prompts.Field object at 0x7f20a7569160>, <src.prompts.investment_prompts.Field object at 0x7f20a7569370>, <src.prompts.investment_prompts.Field object at 0x7f20a75693d0>]\n",
|
|
"field_group_answer_raw: I'll analyze the contract and answer each question, then provide the final answers in JSON format.\n",
|
|
"\n",
|
|
"CONTRACT_AUTO_RENEWAL_TERM:\n",
|
|
"The contract states in section 5.1: \"this agreement shall automatically renew for successive one year periods, unless terminated as set forth herein.\" Therefore, the auto-renewal term is one year.\n",
|
|
"\n",
|
|
"CONTRACT_EFFECTIVE_DT:\n",
|
|
"The effective date is found on the signature page, which states: \"11/10/ 10-30-01\". This appears to be two dates, but following the instructions to preserve the exact format as it appears in the document, I'll use this as is.\n",
|
|
"\n",
|
|
"CONTRACT_TITLE:\n",
|
|
"The title of the contract is found at the beginning of the document: \"CALIFORNIA ER PHYSICIANS PHYSICIAN GROUP SERVICES AGREEMENT\".\n",
|
|
"\n",
|
|
"CONTRACT_TERMINATION_DT:\n",
|
|
"The contract does not specify a specific termination date. Section 5.1 states that the initial term is 12 months after the effective date, and it automatically renews for successive one-year periods unless terminated. Therefore, it's effectively \"year to year\".\n",
|
|
"\n",
|
|
"CONTRACT_AUTO_RENEWAL_IND:\n",
|
|
"The contract does automatically renew according to section 5.1, so the answer is Y.\n",
|
|
"\n",
|
|
"Here's the final answer in JSON format:\n",
|
|
"\n",
|
|
"{\n",
|
|
" \"CONTRACT_AUTO_RENEWAL_TERM\": \"one year\",\n",
|
|
" \"CONTRACT_EFFECTIVE_DT\": \"11/10/ 10-30-01\",\n",
|
|
" \"CONTRACT_TITLE\": \"CALIFORNIA ER PHYSICIANS PHYSICIAN GROUP SERVICES AGREEMENT\",\n",
|
|
" \"CONTRACT_TERMINATION_DT\": \"year to year\",\n",
|
|
" \"CONTRACT_AUTO_RENEWAL_IND\": \"Y\"\n",
|
|
"}\n",
|
|
"Running full context fields...\n",
|
|
"One to One Complete - 2001-11-01 California Emergency Physicians PSA_MU.txt\n",
|
|
"Postprocessing Complete - 2001-11-01 California Emergency Physicians PSA_MU.txt\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"df_out = investment_main(testing=True, test_params=test_params)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>CONTRACT_AUTO_RENEWAL_TERM</th>\n",
|
|
" <th>CONTRACT_EFFECTIVE_DT</th>\n",
|
|
" <th>CONTRACT_TITLE</th>\n",
|
|
" <th>CONTRACT_TERMINATION_DT</th>\n",
|
|
" <th>CONTRACT_AUTO_RENEWAL_IND</th>\n",
|
|
" <th>CONTRACT_FILE_NAME</th>\n",
|
|
" <th>CLIENT_NAME</th>\n",
|
|
" <th>AARETE_DERIVED_TERMINATION_DATE</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>one year</td>\n",
|
|
" <td>10/30/2001</td>\n",
|
|
" <td>CALIFORNIA ER PHYSICIANS PHYSICIAN GROUP SERVI...</td>\n",
|
|
" <td>year to year</td>\n",
|
|
" <td>Y</td>\n",
|
|
" <td>2001-11-01 California Emergency Physicians PSA...</td>\n",
|
|
" <td>Test-Client</td>\n",
|
|
" <td>N/A</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>"
|
|
],
|
|
"text/plain": [
|
|
" CONTRACT_AUTO_RENEWAL_TERM CONTRACT_EFFECTIVE_DT \\\n",
|
|
"0 one year 10/30/2001 \n",
|
|
"\n",
|
|
" CONTRACT_TITLE CONTRACT_TERMINATION_DT \\\n",
|
|
"0 CALIFORNIA ER PHYSICIANS PHYSICIAN GROUP SERVI... year to year \n",
|
|
"\n",
|
|
" CONTRACT_AUTO_RENEWAL_IND \\\n",
|
|
"0 Y \n",
|
|
"\n",
|
|
" CONTRACT_FILE_NAME CLIENT_NAME \\\n",
|
|
"0 2001-11-01 California Emergency Physicians PSA... Test-Client \n",
|
|
"\n",
|
|
" AARETE_DERIVED_TERMINATION_DATE \n",
|
|
"0 N/A "
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"df_out.head()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# The following is not functional because we first need to clean the test bed"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# one_to_one_fields = []\n",
|
|
"# for col in df_out.columns:\n",
|
|
"# if df_out[col].nunique() == 1:\n",
|
|
"# # print(col, df_out[col].nunique())\n",
|
|
"# one_to_one_fields.append(col)\n",
|
|
"\n",
|
|
"# exclude_list = ['CARVEOUT_IND', # 1:n\n",
|
|
"# \"GREATER_OF_IND\", # 1:n\n",
|
|
"# \"REIMB_FEE_RATE\", # 1:n\n",
|
|
"# \"NETWORK\", # dynamic\n",
|
|
"# \"CLAIM_TYPE_CD\", # not found in the source spreadsheet\n",
|
|
"# \"CONTRACT_AMENDMENT_NUM\", # not found in the source spreadsheet\n",
|
|
"# \"PROV_GROUP_NAME_FULL\", # not found in the source spreadsheet\n",
|
|
"# \"PROV_OTHER_NAME_FULL\", # This is called PROV_GROUP_NAME in the source spreadsheet\n",
|
|
"# \"CONTRACT_FILE_NAME\", # not found in the source spreadsheet\n",
|
|
"# \"CARVEOUT_CD\", # 1:n, also not found in the source spreadsheet\n",
|
|
"# 'PAYER_NAME',\n",
|
|
"# 'CLIENT_NAME',\n",
|
|
" \n",
|
|
"# ]\n",
|
|
"\n",
|
|
"# one_to_one_fields = [item for item in one_to_one_fields \n",
|
|
"# if item not in exclude_list]\n",
|
|
"\n",
|
|
"# one_to_one_fields"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# mapping_main_to_testbed = {\"AUTO_RENEWAL_TERM\" : \"AUTO_RENEWAL_TERM_LENGTH\", \"AARETE_DERIVED_TERMINATION_DT\" : \"TERMINATION_DT\"}\n",
|
|
"\n",
|
|
"# def apply_name_mapping(col: str):\n",
|
|
"# if col in mapping_main_to_testbed:\n",
|
|
"# return mapping_main_to_testbed[col]\n",
|
|
"# return col"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# exclude_list = [\"CLAIM_TYPE_CD\"]\n",
|
|
"\n",
|
|
"# for field in one_to_one_fields:\n",
|
|
"# field = apply_name_mapping(field)\n",
|
|
"# # if field == \"AUTO_RENEWAL_TERM\":\n",
|
|
"# # field = \"AUTO_RENEWAL_TERM_LENGTH\"\n",
|
|
"# # elif field == \"AARETE_DERIVED_TERMINATION_DT\":\n",
|
|
"# # field = \"TERMINATION_DT\"\n",
|
|
"# # elif field == \"CLAIM_TYPE_CD\":\n",
|
|
"# print(df_targets[field].head())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# row = df_out[one_to_one_fields].iloc[0]\n",
|
|
"# row"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# filter_target = df_targets[df_targets['Contract Name'] == 'Adena Health System National Template Agreement C19460229AA.txt']\n",
|
|
"\n",
|
|
"# for field, answer in row.items():\n",
|
|
"# print(field, f\"LLM answer: {answer}\")\n",
|
|
"# print(filter_target[apply_name_mapping(field)].iloc[0],\"\\n\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"\n",
|
|
"# filter_target"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "doczy-smart-chunking-blZ6pB_m-py3.12",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.12.7"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|