jacob-mathison/doczyai-pipelines
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

Merged in field/reconfigure-prov-type (pull request #414)

Browse Source 
Field/reconfigure prov type

* Run One-To-N before One-To-One

* Switch PROV_TYPE to Exhibit-Level

* Modify PROV_TYPE prompt

* Merge branch 'main' into field/reconfigure-prov-type


Approved-by: Alex Galarce
This commit is contained in:
Katon Minhas
2025-02-25 17:11:02 +00:00
committed by Alex Galarce
parent ac6f28bc44
commit ba6420102b
2 changed files with 14 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fieldExtraction/src/investment/file_processing.py
+12 -9
View File
@@ -32,21 +32,24 @@ def process_file(file_object, all_dataset, run_timestamp):
exhibit_pages, exhibit_chunk_mapping = preprocess.one_to_n_exhibit_chunking(text_dict, filename)
print(f"{datetime_str()} Preprocessing Complete - {filename}")
################## ONE TO N ##################
if string_utils.contains_reimbursement(contract_text):
one_to_n_results = run_one_to_n_prompts(filename, text_dict, exhibit_pages, exhibit_chunk_mapping, all_dataset) # Return df
one_to_n_results['CONTRACT_FILE_NAME'] = filename
print(f"{datetime_str()} One to N Complete - {filename}")
else:
one_to_n_results = {'CONTRACT_FILE_NAME' : filename}
print(f"{datetime_str()} No One to N Found, Skipping - {filename}")
################## ONE TO ONE ##################
one_to_one_results = run_one_to_one_prompts(filename, contract_text, text_dict, top_sheet_dict) # Return df
one_to_one_results['CONTRACT_FILE_NAME'] = filename
print(f"{datetime_str()} One to One Complete - {filename}")
# one_to_one_results = {"CONTRACT_FILE_NAME" : filename}
################## ONE TO N ##################
if string_utils.contains_reimbursement(contract_text):
one_to_n_results = run_one_to_n_prompts(filename, text_dict, exhibit_pages, exhibit_chunk_mapping, all_dataset) # Return df
one_to_n_results['CONTRACT_FILE_NAME'] = filename
contract_results = pd.merge(one_to_one_results, one_to_n_results, how='outer', on='CONTRACT_FILE_NAME')
else:
contract_results = one_to_one_results
print(f"{datetime_str()} One to N Complete - {filename}")
################## MERGE ##################
contract_results = pd.merge(one_to_one_results, one_to_n_results, how='outer', on='CONTRACT_FILE_NAME')
################## AARETE-DERIVED ##################
final_results = aarete_derived.get_crosswalk_and_derived(contract_results, filename)
print(f"{datetime_str()} AArete-Derived Complete - {filename}")
fieldExtraction/src/prompts/investment_prompts.json
+2 -2
View File
@@ -322,8 +322,8 @@
{
"field_name": "AARETE_DERIVED_PROV_TYPE",
"relationship": "one_to_n",
"field_type": "service_breakout",
"prompt": "What Provider Type does the Service and Methodology apply to? Choose ONLY from the following: {valid_values}.",
"field_type": "exhibit_level",
"prompt": "What Provider Type does the Exhibit apply to? Choose ONLY from the following: {valid_values}. If there is no obvious answer, write 'N/A'.",
"valid_values": "VALID_PROV_TYPE"
},
{