Feature/DAIP2-2314 DAIP2 1687 hybrid
* remove -files from s3 prefix requirements
* Resolve input paths
* fix: VendorProcessor.process_file returns (df, None) tuple
runner.safe_process_file unpacks the result as (cc_df, dashboard_df), so
returning a single DataFrame caused every vendor/generic file to fail with
"too many values to unpack (expected 2)" — Python iterates DataFrame columns
during unpacking. Vendor pipelines have no dashboard variant; second slot is
None and the existing `dashboard_result is not None` guard in runner.py
already handles it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* DAIP2-2314 + DAIP2-1687: pad DYNAMIC_PRIMARY + DYNAMIC_PRIMARY_ENTITY_CLASSIFICATION over 1024-token cache floor
- Pad DYNAMIC_PRIMARY_INSTRUCTION with three new sections: [SCOPE BOUNDARIES], [SOURCE TEXT INTERPRETATION], [REASONING DISCIPLINE], plus a [WORKED EXAMPLES] block. Estimated tokens: 447 -> 1117 (Sonnet 4.5 1024-min, +93 margin). All additions reinforce existing rules (sibling-field separation, alias mapping, pricing-vs-LOB distinction, contrastive-clause exclusion, exhibit-header binding) — no new directives that could bias extraction.
- Pad DYNAMIC_PRIMARY_ENTITY_CLASSIFICATION_INSTRUCTION with a [FINAL CHECKLIST BEFORE OUTPUT] block. Estimated tokens: 956 -> 1101 (Sonnet 4.5 1024-min, +77 margin). Reinforces the existing 4-step anti-duplication protocol and JSON shape requirements.
- Register both new entries in cache_registry: DYNAMIC_PRIMARY_ENTITY_CLASSIFICATION as INSTRUCTION_PLUS_CONTEXT (caches at warm-up), DYNAMIC_PRIMARY_ENTITIES as CONTEXT (instruction is intentionally short; CONTEXT c…
* black format fix
* Merged dev into feature/DAIP2-2314-DAIP2-1687-hybrid
* fixed raw lob values in base lob field mapping and composite entities fix
* black format fix
* fixed LOB Program output issues
* issue fixes
* remove debugging code
* Updated prompts
* updated additional instructions
* Update Program-->LOB
* LLM-based AD-Program/Product mapping to LOB even when there is a crosswalk
* black format fix
* Merged dev into feature/DAIP2-2314-DAIP2-1687-hybrid
* added logging in prompt call tracking
* added updated logging in prompt call tracking
* aaded min cache token per usage label
* added cache registry for dynamic primary mapping prompt calls
* reolved mapping prompts ambiguities
* black format fix
* Phase 2 modifications added
* reverted phase 2 modifications
Approved-by: Katon Minhas
Venkatakrishna Reddy Avula