Bugfix/exhibit smart chunking cost improvements
* Add opt-in instrumentation for per-call token and row-count tracing
Introduce src/utils/instrumentation.py (thread-safe CSV logger) and
src/utils/instrumentation_context.py (ContextVar scope plus
submit_with_context / map_with_context helpers for propagating context
into ThreadPoolExecutor workers).
Emit events at every Bedrock call in llm_utils.invoke_claude, including
in-memory claude_cache hits, with full input/output/cache-read/cache-write
token breakdown. Emit row-count events at each row-mutating stage in the
one-to-N pipeline (clean_reimbursement_primary,
filter_services_without_reimbursements, methodology_breakout,
split_service_terms, carveout, dynamic_code_assignment,
lesser_of_distribution, dynamic_assignment) and chunking / retrieval
events in exhibit smart chunking (chunking_done, retrieval_done) plus
exhibit lifecycle events (exhibit_start, exhibit_gate_skip,
stage_transition).
All hooks are no-ops unless DOCZY_INSTRUMENTATION=1; production defaults
unchanged. Runner and inspector scripts in …
* Extend instrumentation: segment, cache_miss_reason, retry/error events, runtime hookup, analyzer
Schema expansion and new event types:
- Add segment column to every llm_call / llm_call_inmem_hit, resolved from
USAGE_LABEL_TO_SEGMENT (authoritative) with scope fallback. Mapping built
from grep + smoke-run ground truth; earlier guessed entries removed.
- Add cache_miss_reason classifier: hit / first_call / ttl_expired /
under_min_tokens / silent_miss / not_attempted. Uses a per-process
_cache_key_seen dict guarded by its own lock.
- Emit llm_retry on each retry attempt (attempt, error_class, error_msg,
backoff_sec) and llm_error on exhausted retries in ec2_claude_3_and_up
(rotation and non-rotation branches) plus local_claude_3_and_up.
- Add six new CSV columns: segment, cache_miss_reason, attempt,
error_class, error_msg, backoff_sec. Total schema now 34 columns.
Defensive kwarg hygiene:
- _ctx_minus_explicit_keys filter on all emit sites to prevent
segment / filename kwarg collisions between …
* Merged dev into bugfix/exhibit-smart-chunking-cost-improvements
* Merged dev into bugfix/exhibit-smart-chunking-cost-improvements
* Make instrumentation tracking on by default
Flip the gate: instrumentation is now enabled unless DOCZY_INSTRUMENTATION
is explicitly set to a falsy value (0/false/no/off). Replaces the prior
opt-in behaviour where it was off unless DOCZY_INSTRUMENTATION=1 was set.
* Fix missing WRITE_TO_S3 patch in upload_instrumentation_csv test
* Merged dev into bugfix/exhibit-smart-chunking-cost-improvements
Approved-by: Katon Minhas
Feature/DAIP2-2023 eliminate full context processing
* testing full context fields
* remove full context processing
* merge Dev with DAIP2-2-23
* full context removal in client codes
* AARETE_DERIVED_PROVIDER_NAME field changes
* Merged DEV into feature/DAIP2-2023-eliminate-full-context-processing
* optimized provider name
* black format fix
* contract title fixes
* PAYER NAME AUTO RENEWAL IND fixes
* Merged DEV into feature/DAIP2-2023-eliminate-full-context-processing
* Merge branch 'DEV' into feature/DAIP2-2023-eliminate-full-context-processing
* Remove prints
Approved-by: Katon Minhas
Feature/DAIP2-1803 add aarete derived provider name
* Merged DEV into feature/DAIP2-1562-add-aarete_derived_payer_name
* state logic added in clustering
* removed print statements
* pull request updates
* black format fix
* Merged DEV into feature/DAIP2-1562-add-aarete_derived_payer_name
* added scalability feature and optimization
* black format
* Pull Request Changes
* generalised funcs from aarete derived payer name
* updated remove states and updated docstring for build similarity matrix
* derived payer name functionality added
* config and main file changes
* sync dev into ADD AARETE DERIVED PROVIDER GROUP NAME FULL
* updated derived_provider name
* pipeline fixes
* Merged DEV into feature/DAIP2-1803-add-aarete-derived-provider-name
* config changes
* field name updated to AARETE_DERIVED_PROVIDER_NAME
* Merged DEV into feature/DAIP2-1803-add-aarete-derived-provider-name
* black format fix
* Fix docstring and add empty column when not found
* Rename test file and add missing column test case
- Renamed test_derived_payer_name.py to test_derived_names.py since
file now covers both payer and provider name functionality
- Added test_derived_provider_names_column_missing to verify empty
column is added when AARETE_DERIVED_PROVIDER_NAME is not present
* Remove trailing whitespace
* Merge remote-tracking branch 'origin/DEV' into feature/DAIP2-1803-add-aarete-derived-provider-name
Approved-by: Siddhant Medar
Bugfix/parser downstream improvements
* Refactor: Implement field-aware JSON parsers with centralized normalization
This refactor introduces a robust system for normalizing LLM output based on
field format mappings, ensuring consistent data types throughout the pipeline.
Key Changes:
- Add FIELD_FORMAT_MAPPING constant defining expected formats for all fields
- Create format_normalization.py utility for type-aware normalization
- Update json_utils.py parsers to accept field_names/field_name parameters
- Refactor prompt_templates.py to use parser factories (_create_json_dict_parser,
_create_json_list_parser) that bind field metadata for automatic normalization
- Update prompt_calls.py to pass field names to parsers, eliminating redundant
normalization logic
- Remove parse_json_dict_or_list (unused, ambiguous function)
- Simplify METHODOLOGY_BREAKOUT and REIMBURSEMENT_PRIMARY to use helper functions
- Add comprehensive integration tests verifying normalization works end-to-end
Benefits:
- Single source of truth for field formats (FIELD_FORMAT_…
* refactor: normalize helper prompt outputs at prompt_calls level
- Update CARVEOUT_CHECK to use field-aware parser for CARVEOUT_CD normalization
- Update LOB_RELATIONSHIP to normalize to string format in prompt_calls.py
- Update SPLIT_REIMB_DATES to normalize date values in prompt_calls.py
- Remove defensive normalization from one_to_n_funcs.py for LOB relationships
- Remove manual normalization from split_reimb_dates() - values now normalized upstream
- All helper prompts that populate fields now normalize at prompt_calls.py level
- Downstream functions receive correctly formatted values without additional processing
* refactor: remove band-aid normalization functions and migrate HSC to field-aware parsers
- Update ONE_TO_ONE_SINGLE_FIELD_TEMPLATE to use field-aware parser with field_name parameter
- Remove list wrapping logic in hybrid_smart_chunking_funcs (field-aware parser handles normalization)
- Remove normalize_one_to_one_field_value and normalize_one_to_one_answers_dict from string_utils.py
- Remove all debug print statements from HSC processing
- Remove commented-out normalization calls from client-specific files (clover, bcbs_promise)
- All normalization now handled exclusively through FIELD_FORMAT_MAPPING via field-aware parsers
* Ran Black for formatting
* Print Statements removed, more cleaning
* Merge branch 'DEV' into bugfix/parser-downstream-improvements
* refactor: combine FIELD_FORMAT_MAPPING into investment_columns.py
- Merged field_format_mapping.py into investment_columns.py to create single source of truth
- FIELD_FORMAT_MAPPING now ordered by COLUMN_ORDER (161 fields)
- Added 5 missing fields from COLUMN_ORDER with default format types
- Updated all imports across codebase to use investment_columns
- Python dict preserves insertion order (3.7+), maintaining COLUMN_ORDER sequence
- All tests passing (38 field-aware tests verified)
* Deprecate COLUMN_ORDER, rely on Mapping only
* Merged DEV into bugfix/parser-downstream-improvements
Approved-by: Katon Minhas
Bugfix/daip2 1440 contract amendment num to main
* update full context additional instruction for amendment num
* uv formatting done
* remove print statements
* Merged main into bugfix/daip2-1440-contract-amendment-num
* added filename amendment num
* updated pipelines and removed print statements
* Merged main into bugfix/daip2-1440-contract-amendment-num
* code moved to hybrid smart chunking
* Merged main into bugfix/daip2-1440-contract-amendment-num
* Merged main into bugfix/daip2-1440-contract-amendment-num
* removed categorical values from amendment number
Approved-by: Katon Minhas
Feature/lesser table caching refactor hybrid
* chore: Remove unused duplicate main.py from shared pipeline
* fix: Correct crosswalk paths in aarete_derived.py
* chore: Remove unused documentation files from fieldExtraction
* docs: Add documentation files to documentation folder
* docs: Update README with uv setup, expanded project structure, and branching conventions
* docs: Add uv installation steps with Ubuntu/WSL emphasis
* Enable prompt caching for all remaining LLM calls
- Add _INSTRUCTION() functions for: EXHIBIT_HEADER, EXHIBIT_LINKAGE,
EXHIBIT_TITLE_MATCH, DATE_FIX, DERIVED_TERM_DATE, CHECK_PROVIDER_NAME_MATCH,
SPECIAL_CASE_ASSIGNMENT
- Update all invoke_claude() calls in saas and clover pipelines to use
cache=True with corresponding _INSTRUCTION() functions
- Add new instructions to get_cacheable_instructions() for cache warming
- Update tests for new instruction functions
Functions now using caching:
- prompt_exhibit_level
- prompt_exhibit_lesser (EXHIBIT_LEVEL_LESSER_OF)
- prompt_fee_schedule_breakout
- prompt_grouper_breakout
- prompt_special_case_assignment
- prompt_exhibit_linkage
- prompt_exhibit_header
- prompt_smart_chunked (ONE_TO_ONE templates)
- prompt_date_fix
- prompt_derived_term_date
- prompt_exhibit_title_match
- provider_name_match_check
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Reorder
* feat: Add bcbs_promise client pipeline with OFFSET_TERM extraction
- Add new bcbs_promise client with HSC-based OFFSET_TERM field extraction
- Extract full paragraph text of offset/recoupment provisions from contracts
- Derive OFFSET_INDICATOR (Y/N) from OFFSET_TERM presence
- Fix reorder_columns to preserve extra columns not in COLUMN_ORDER
- Update QC/QA output path to outputs/qc_qa/
* fix: Update dev deps and test assertions for QC/QA output path
- Add pytest/pytest-mock to dev dependencies for mypy type checking
- Update test assertions to expect outputs/qc_qa instead of qa_qc_output
* style: Apply black formatting to prompt_templates.py
* Merge main, move scripts
* Archive some scripts
* update py version
* remove .py version file
* Remove ASCII characters
* Restore testbed code
* restore tracking
* Update testbed metrics
* Enable prompt caching for CODE_LAST_CHECK, FILL_BILL_TYPE, DUAL_LOB_CHECK, and GROUPER_BREAKOUT
- Add CODE_LAST_CHECK_INSTRUCTION() for service specificity classification
- Add FILL_BILL_TYPE_INSTRUCTION() for bill type code determination
- Add DUAL_LOB_CHECK_INSTRUCTION() for Medicare/Medicaid classification
- Update code_funcs.py to use caching for CODE_LAST_CHECK, FILL_BILL_TYPE, GROUPER_BREAKOUT
- Update postprocessing_funcs.py to use caching for DUAL_LOB_CHECK
- Add new instructions to get_cacheable_instructions() for cache warming
- Add unit tests for new instruction functions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix postprocessing_funcs to remove invalid columns
* Merge branch 'main' into feature/lesser-table-caching-refactor-hybrid
* Revert prompt caching changes from aed1b73c
* update formatting
* Update imports
Approved-by: Sha Brown
Approved-by: Praneel Panchigar
Praneel Panchigar