Merged in bugfix/hmk-issues (pull request #824)

Bugfix/hmk issues

* fix for missing AARETE_DERIVED_LOB

* fix for AARETE_DERIVED_SID

* bugfix in universal_json_load

* bugfix for normalize_state_field

*  reimb eff date assignmnet prompt update

* prompt update for missing reimb terms

* Merged main into bugfix/hmk-issues

* test case for universal_json_loads

* reveresed the prompt update for missing reimb

* prompt fix for reimb dates


Approved-by: Katon Minhas

fieldExtraction/src/utils/string_utils.py

@@ -197,8 +197,19 @@ def universal_json_load(s: str):
         if s[i] in '{[':
             start = i
             stack = [s[i]]
-            for j in range(i+1, len(s)):
-                if s[j] in '{[':
+            j = i + 1
+            while j < len(s):
+                # Skip over string literals to avoid counting brackets inside strings
+                if s[j] == '"':
+                    j += 1
+                    while j < len(s):
+                        if s[j] == '\\' and j + 1 < len(s):
+                            j += 2  # Skip escaped character
+                            continue
+                        if s[j] == '"':
+                            break
+                        j += 1
+                elif s[j] in '{[':
                     stack.append(s[j])
                 elif s[j] in '}]':
                     if not stack:
@@ -218,6 +229,7 @@ def universal_json_load(s: str):
                             pass
                         i = j  # Move past this top-level object
                         break
+                j += 1
             else:
                 pass
         i += 1
@@ -675,8 +687,8 @@ def parse_raw_state_value(raw_value):
             except (ValueError, SyntaxError):
                 pass  # Not valid Python literal, continue with string parsing
         
-        # Regular string splitting on delimiters
-        states = re.split(r'[|,;\s]+', stripped)
+        # Regular string splitting on delimiters (not whitespace - state names can have spaces like "West Virginia")
+        states = re.split(r'[|,;]+', stripped)
         return [s.strip() for s in states if s.strip()]
     
     if isinstance(raw_value, list):