Merged in feature/simplify-b-field-chunking (pull request #348)

feature/simplify b field chunking * Add chunk_by_exhibit function and unit tests * Refactor preprocess * Updated to run BU primary on entire Exhibit * Makes BU secondary compatible with exhibit chunk mapping * Added unit tests for get_exhibit_chunk * Trimmed TD funcs * Downstream changes to run_top_down * Removed conditional funcs * Merged main into feature/simplify-b-field-chunking * removed conditional funcs import * Merged main into feature/simplify-b-field-chunking * Fixed run_bottom_up_primary arguments * Fixed TD primary input * Fixed grouped keyword mappings * Remove filename tracking until postprocessing * Removed postprocessing for client columns * Removed col-based postprocessing * Merged main into feature/simplify-b-field-chunking * Fixed NoneType * Add docstrings * Updated unit tests for removal of 'Filename' * Updated comments Approved-by: Alex Galarce
2025-01-10 17:24:44 +00:00
parent b2c4bef039
commit 3bb6abbcef
10 changed files with 278 additions and 391 deletions
@@ -307,3 +307,39 @@ def get_exhibit_pages(text_dict, filename):
            exhibit_pages.append(page_num)
    return exhibit_pages

+def chunk_by_exhibit(text_dict: dict, 
+                     exhibit_pages: list
+                     ) -> dict:
+    """
+    Organizes pages into groups based on their association with specific exhibits.
+
+    This function assigns each page number from the `text_dict` dictionary to an exhibit
+    based on the `exhibit_pages` list. Pages are grouped under the nearest preceding 
+    page number in `exhibit_pages`. If a page number in `text_dict` is itself in 
+    `exhibit_pages`, it starts a new exhibit group.
+
+    Parameters:
+        text_dict (dict): A dictionary where keys are page numbers and values are page text
+        exhibit_pages (list): A list of page numbers that mark the beginning of a new
+                              exhibit.
+
+    Returns:
+        dict: A dictionary mapping each page number in `text_dict` to its corresponding
+              exhibit identifier. The exhibit identifier is the page number of the first
+              page in that exhibit as listed in `exhibit_pages`. If there are pages before
+              the first `exhibit_page`, they are grouped under the exhibit identifier "0".
+    """
+    if len(exhibit_pages) == 0:
+        return {key : key for key in text_dict.keys()}
+    
+    exhibit_dict = {}
+    current_exhibit = "0"
+    for page_num in text_dict.keys():
+        if page_num in exhibit_pages:
+            current_exhibit = page_num
+            exhibit_dict[page_num] = current_exhibit
+        else:
+            exhibit_dict[page_num] = current_exhibit
+    return exhibit_dict
+
+