Merged in bugfix/parser-downstream-improvements (pull request #875)

Bugfix/parser downstream improvements

* Refactor: Implement field-aware JSON parsers with centralized normalization

This refactor introduces a robust system for normalizing LLM output based on
field format mappings, ensuring consistent data types throughout the pipeline.

Key Changes:
- Add FIELD_FORMAT_MAPPING constant defining expected formats for all fields
- Create format_normalization.py utility for type-aware normalization
- Update json_utils.py parsers to accept field_names/field_name parameters
- Refactor prompt_templates.py to use parser factories (_create_json_dict_parser,
  _create_json_list_parser) that bind field metadata for automatic normalization
- Update prompt_calls.py to pass field names to parsers, eliminating redundant
  normalization logic
- Remove parse_json_dict_or_list (unused, ambiguous function)
- Simplify METHODOLOGY_BREAKOUT and REIMBURSEMENT_PRIMARY to use helper functions
- Add comprehensive integration tests verifying normalization works end-to-end

Benefits:
- Single source of truth for field formats (FIELD_FORMAT_…
* refactor: normalize helper prompt outputs at prompt_calls level

- Update CARVEOUT_CHECK to use field-aware parser for CARVEOUT_CD normalization
- Update LOB_RELATIONSHIP to normalize to string format in prompt_calls.py
- Update SPLIT_REIMB_DATES to normalize date values in prompt_calls.py
- Remove defensive normalization from one_to_n_funcs.py for LOB relationships
- Remove manual normalization from split_reimb_dates() - values now normalized upstream
- All helper prompts that populate fields now normalize at prompt_calls.py level
- Downstream functions receive correctly formatted values without additional processing

* refactor: remove band-aid normalization functions and migrate HSC to field-aware parsers

- Update ONE_TO_ONE_SINGLE_FIELD_TEMPLATE to use field-aware parser with field_name parameter
- Remove list wrapping logic in hybrid_smart_chunking_funcs (field-aware parser handles normalization)
- Remove normalize_one_to_one_field_value and normalize_one_to_one_answers_dict from string_utils.py
- Remove all debug print statements from HSC processing
- Remove commented-out normalization calls from client-specific files (clover, bcbs_promise)
- All normalization now handled exclusively through FIELD_FORMAT_MAPPING via field-aware parsers

* Ran Black for formatting

* Print Statements removed, more cleaning

* Merge branch 'DEV' into bugfix/parser-downstream-improvements

* refactor: combine FIELD_FORMAT_MAPPING into investment_columns.py

- Merged field_format_mapping.py into investment_columns.py to create single source of truth
- FIELD_FORMAT_MAPPING now ordered by COLUMN_ORDER (161 fields)
- Added 5 missing fields from COLUMN_ORDER with default format types
- Updated all imports across codebase to use investment_columns
- Python dict preserves insertion order (3.7+), maintaining COLUMN_ORDER sequence
- All tests passing (38 field-aware tests verified)

* Deprecate COLUMN_ORDER, rely on Mapping only

* Merged DEV into bugfix/parser-downstream-improvements


Approved-by: Katon Minhas

src/pipelines/shared/preprocessing/hybrid_smart_chunking_funcs.py

@@ -284,10 +284,6 @@ def run_hybrid_smart_chunked_fields(
             answers_dict, field_name="PAYER_STATE"
         )
 
-        # Normalize all 1:1 fields: convert single-element lists to strings
-        # Preserves legitimate multi-value lists (e.g., LOB, PROGRAM when passed to 1:1)
-        answers_dict = string_utils.normalize_one_to_one_answers_dict(answers_dict)
-
         return answers_dict
 
     except Exception as e:
@@ -372,7 +368,7 @@ def prompt_hsc_single_field(
             constants
         )  # Returns dict of {field_name : prompt}
         prompt, _parser = prompt_templates.ONE_TO_ONE_SINGLE_FIELD_TEMPLATE(
-            context, llm_prompt
+            context, llm_prompt, field_name=field.field_name
         )
 
         logging.debug(f"Field: {field.field_name}")
@@ -388,12 +384,19 @@ def prompt_hsc_single_field(
             )
         try:
             llm_answer_final = _parser(llm_answer_raw)  # returns dict
-
             field_value = llm_answer_final.get(field.field_name)
-            if not isinstance(field_value, list):
-                field_value = [field_value]
 
-            if field_value[0] == "N/A":
+            # Check for N/A - handle both string and list formats
+            field_value_str = (
+                field_value
+                if isinstance(field_value, str)
+                else (
+                    field_value[0]
+                    if isinstance(field_value, list) and field_value
+                    else "N/A"
+                )
+            )
+            if field_value_str == "N/A":
                 field.field_type = "full_context"
                 if field.field_name == "PAYER_NAME":
                     field.prompt = (