Merged in feature/exhibit-smart-chunking (pull request #883)

Feature/exhibit smart chunking

* exhibit processing per page

* header dict deduplication

* dedup prompt refinment

* refinment for header extraction proecess

* Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking

* merge updates

* minor fix

* prompt fix for reimb type

* COB defenition for clear understanding

* black formatting

* remove quit statement

* pipiline test

* pipeline test

* black formatting

* Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking

* black formating

* Merge remote-tracking branch 'origin/DEV' into feature/exhibit-smart-chunking

* typo

* PR comment fixes

* exhibit funcs refactored

* black formatting

* Refactor exhibit chunking config into dedicated class

Created ExhibitChunkingConfig class to centralize exhibit smart chunking
configuration parameters (DEFAULT_SUBCHUNK_SIZE, MIN_PARENT_CHUNK_SIZE,
CHUNK_RELEVANCE_THRESHOLD). This improves code organization by consolidating
related constants and makes configuration more maintainable.

Changes:
- Created ExhibitChunkingConfig class with ESC_CONFIG instance
- Moved CHUNK_RELEVANCE_THRESHOLD from config.py to ExhibitChunkingConfig
- Updated all constant references to use ESC_CONFIG prefix
- Added missing EXHIBIT_HEADER_MARKERS parameter documentation

* Fix logging levels and refactor imports for exhibit chunking

- Upgrade logging from WARNING to ERROR for embedding and semantic search failures
- Remove unused constant imports from exhibit_funcs.py
- Update test imports to use ESC_CONFIG pattern for configuration constants
- Add warning when no exhibit headers found during deduplication
- Expand mypy type checking by removing s3_utilities from exclude list

* Merged DEV into feature/exhibit-smart-chunking

* Merged DEV into feature/exhibit-smart-chunking


Approved-by: Siddhant Medar

src/utils/string_utils.py

@@ -1092,3 +1092,21 @@ def token_signature(text: str) -> tuple:
 
     # Sort tokens to ensure order-independent comparison
     return tuple(sorted(tokens))
+
+
+def keyword_match(text: str, keywords: list) -> bool:
+    """
+    Check if text contains any of the keywords (case-insensitive).
+
+    Args:
+        text: Text to search in
+        keywords: List of keywords to look for
+
+    Returns:
+        True if any keyword is found in text
+    """
+    text_lower = text.lower()
+    for keyword in keywords:
+        if keyword.lower() in text_lower:
+            return True
+    return False