Merged in feature/simplify-b-field-chunking (pull request #348)

feature/simplify b field chunking

* Add chunk_by_exhibit function and unit tests

* Refactor preprocess

* Updated to run BU primary on entire Exhibit

* Makes BU secondary compatible with exhibit chunk mapping

* Added unit tests for get_exhibit_chunk

* Trimmed TD funcs

* Downstream changes to run_top_down

* Removed conditional funcs

* Merged main into feature/simplify-b-field-chunking

* removed conditional funcs import

* Merged main into feature/simplify-b-field-chunking

* Fixed run_bottom_up_primary arguments

* Fixed TD primary input

* Fixed grouped keyword mappings

* Remove filename tracking until postprocessing

* Removed postprocessing for client columns

* Removed col-based postprocessing

* Merged main into feature/simplify-b-field-chunking

* Fixed NoneType

* Add docstrings

* Updated unit tests for removal of 'Filename'

* Updated comments


Approved-by: Alex Galarce

fieldExtraction/src/utils/string_utils.py

@@ -219,7 +219,6 @@ def primary_string_to_dict(string_dict, filename):
         dict_list = [secondary_string_to_dict(d, filename) for d in dict_list]
         for dict_ in dict_list:
             dict_["page_num"] = page_num
-            dict_["Filename"] = filename
             data.append(dict_)
     return data
 
@@ -250,4 +249,30 @@ def is_empty(value): # string_funcs.py
         return True
     else:
         empty_values = [None, "", "N/A", "NA", "null", "none", "NaN", np.nan, "nan"]
-        return value in empty_values
+        return value in empty_values
+    
+
+def get_exhibit_chunk(text_dict: dict, 
+                      exhibit_chunk_mapping: dict, 
+                      exhibit_page: str) -> str:
+    """
+    Compiles text from a specific exhibit based on its starting page number.
+
+    This function aggregates the text of all pages that are mapped to a specific exhibit
+    start page in a document. It uses the exhibit chunk mapping to determine which pages
+    belong to the exhibit defined by the given start page number, and concatenates their
+    texts into a single string.
+
+    Parameters:
+        text_dict (dict): A dictionary where keys are page numbers and values are the text on those pages.
+        exhibit_chunk_mapping (dict): A dictionary mapping each page number to the starting page number of the exhibit it belongs to.
+        exhibit_page (str): The page number that marks the beginning of the exhibit to compile text for.
+
+    Returns:
+        str: A string that combines all the text from the pages belonging to the specified exhibit, separated by newlines.
+
+    Notes:
+        - The function assumes that `exhibit_chunk_mapping` correctly maps all page numbers in `text_dict` to their respective exhibit starting pages.
+        - It is important that `exhibit_page` exists as a key in the `exhibit_chunk_mapping` dictionary and corresponds to the starting page of an exhibit.
+    """
+    return '\n'.join([page_text for page_num, page_text in text_dict.items() if exhibit_chunk_mapping[page_num] == exhibit_page])