jacob-mathison/doczyai-pipelines
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

Merged in bugfix/dynamic-primary-and-pyproject (pull request #683)

Browse Source 
pyproject requirements, prep for logging fixes

* remove chroma

* update project name to "doczy-field-extraction" in pyproject.toml

* update lock file

* replace print statements with logging for better traceability in process_file function

* replace print statements with logging for improved error tracking and information logging

* replace print statements with logging for improved error handling and traceability

* replace print statements with logging for improved error handling in llm_utils and string_utils

* remove unused table_utils module; supplanted by table_funcs and unreferenced

* replace capsys with caplog in test_invalid_input_type for improved logging of invalid input


Approved-by: Katon Minhas
This commit is contained in:
Alex Galarce
2025-08-27 16:37:48 +00:00
parent 90bdc672d5
commit 39a07f63c9
14 changed files with 1319 additions and 3074 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fieldExtraction/src/utils/string_utils.py
+8 -7
View File
@@ -1,4 +1,5 @@
import json
import logging
import re
import traceback
import warnings
@@ -206,9 +207,9 @@ def universal_json_load(string_dict: str):
try:
return json.loads(matched_str)
except json.JSONDecodeError as e:
print(f"Failed to parse JSON: {e}")
print(f"Original string: {string_dict}")
print(f"Traceback: {traceback.format_exc()}")
logging.error(f"Failed to parse JSON: {e}")
logging.error(f"Original string: {string_dict}")
logging.error(f"Traceback: {traceback.format_exc()}")
# Use placeholders to preserve JSON structure characters for CSV compatibility
csv_safe_string = (
string_dict.replace(",", "[COMMA]")
@@ -230,9 +231,9 @@ def universal_json_load(string_dict: str):
) # Replace single quotes with double quotes
return json.loads(cleaned_str)
except json.JSONDecodeError as e:
print(f"Failed to parse list: {e}")
print(f"Original string: {string_dict}")
print(f"Traceback: {traceback.format_exc()}")
logging.error(f"Failed to parse list: {e}")
logging.error(f"Original string: {string_dict}")
logging.error(f"Traceback: {traceback.format_exc()}")
# Use placeholders to preserve JSON structure characters for CSV compatibility
csv_safe_string = (
string_dict.replace(",", "[COMMA]")
@@ -275,7 +276,7 @@ def contains_reimbursement(text, page="1", method="keyword"):
elif isinstance(text, str):
text_to_check = text.lower()
else:
print("contains_reimbursement - Invalid data type")
logging.warning("contains_reimbursement - Invalid data type")
return False
return any(keyword in text_to_check for keyword in reimbursement_strings)
elif method == "regex":