jacob-mathison/doczyai-pipelines
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

Merged in bugfix/table-preprocessing (pull request #560)

Browse Source 
Bugfix/table preprocessing

* restructure get_exhibit_dict

* exhibit header

* update exhibit header handling

* update docstring

* bugfix

* remove test - update docstring

* Merged main into bugfix/table-preprocessing

* refactor: update page_key_sort to return float for numeric keys

* refactor: enhance get_exhibit_dict to improve chunking logic for exhibits

* refactor: update page_key_sort to return float for numeric keys

* test: add unit tests for get_exhibit_dict function

* test: fix test case 7

* Add final, difficult test

* refactor: enhance get_exhibit_dict to handle single decimal page exhibits and improve chunking logic

* pulled `one_to_n_test.py` from dynamic primary branch

* fix: update common_file_names logic to remove specific entry and log changes

* docs: update get_exhibit_dict docstring to clarify input and output formats

* test: add test for exhibit handling with decimal pages in get_exhibit_dict

* feat: add debug logging for exhibit chunk mapping and exhibit dictionary keys in one_to_n_exhibit_chunking

* Merge remote-tracking branch 'origin/main' into bugfix/table-preprocessing

* feat: enhance spatial processing rules and consistency checks for reimbursement terms

* feat: update global scan methodology to enhance context extraction for reimbursement constraints

* Merge remote-tracking branch 'origin/main' into bugfix/table-preprocessing


Approved-by: Katon Minhas
This commit is contained in:
Alex Galarce
2025-06-09 21:49:45 +00:00
parent 32d431ec4e
commit f84023d742
8 changed files with 663 additions and 111 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fieldExtraction/src/utils/string_utils.py
+5 -5
View File
@@ -71,18 +71,18 @@ def extract_text_from_delimiters(
return matches[match_index]
def page_key_sort(page_key: str) -> tuple[int, int|str]:
def page_key_sort(page_key: str) -> tuple[int, float|str]:
"""Sorts page keys, prioritizing numeric keys first.
Args:
page_key (str): The page key to sort.
Returns:
tuple[int, str]: A tuple where the first element is 0 for numeric keys and 1 for non-numeric keys, and the second element is the page key.
tuple[int, float|str]: A tuple where the first element is 0 for numeric keys and 1 for non-numeric keys, and the second element is the numeric value for numeric keys or the original string for non-numeric keys.
Example:
>>> page_key_sort("1")
(0, 1)
(0, 1.0)
>>> page_key_sort("A")
(1, "A")
@@ -95,7 +95,7 @@ def page_key_sort(page_key: str) -> tuple[int, int|str]:
"""
try:
return (0, int(page_key)) # Numeric pages first, in numerical order
return (0, float(page_key)) # Numeric pages first, in numerical order
except ValueError:
return (1, str(page_key)) # Non-numeric pages after, in alphabetical order