Files
doczyai-pipelines/src/prompts/cache_registry.py
T
Praneel Panchigar 53a0c76d8e Merged in bugfix/stg-to-main-prep (pull request #1012)
Bugfix/stg to main prep

* Merged in dev (pull request #1001)

Dev

* Revert premature merge of bugfix/retire_stale_client_file_processing

PR #959 was merged into dev without approval. This reverts commits
5e143c10, 63f32c41, 849aa626, and 927abcae to restore dev to its
pre-merge state. The changes will be re-submitted via a new PR
after proper review.

* Merged in bugfix/retire_stale_client_file_processing (pull request #961)

Return None for dashboard output when dashboard postprocessing is off

* Return None for dashboard output when dashboard postprocessing is off

FINAL_RESULT_DF_DASHBOARD was initialized as an empty DataFrame even
when RUN_DASHBOARD_POSTPROCESSING was False, causing downstream code
to needlessly process it (reorder_columns, etc). Now returns None
when dashboard is not requested, matching the postprocess() contract.

* Merged dev into bugfix/retire_stale_client_file_processing

* Merge dev (with revert) into feature branch

* Re-apply retire stale client file_processing changes

Revert of the revert (4f53b528) to…
* fix grouping_key casing in service term standardization

qc.py uppercases all PC_Details columns to GROUPING_KEY, but the
service-term reader looked up lowercase 'grouping_key'. The lookup
silently failed and degraded per-group standardization to global mode.

* enable prompt caching on service_term_standardization LLM calls

cache=True was missing; the 2.5k-token instruction was being re-billed
on every chunk and mapping-batch call instead of reading from the
shared instruction cache.

* register SERVICE_ENRICHMENT_INSTRUCTION in cache registry

The runtime call at code_funcs.py:75 already passes cache=True with
usage_label="SERVICE_ENRICHMENT", but the registry entry was missing,
so the prompt was never warmed and every first call paid create-tokens.

* remove stray debug prints from document_classification.generate_pre_doczy_report

Three print() calls were dumping the Program/Product mapping to stdout
on every run.

* demote code_explicit raw-answer log from INFO to DEBUG

Full raw LLM responses were spamming INFO-level logs on every call.

* flag vendor prompt_templates convention drift for follow-up

Vendor pipeline still uses legacy hand-rolled JSON wording and has one
stale 'Return result in pipes' reference. Adopting the
JSON_*_FORMAT_INSTRUCTIONS macros from the main pipeline is deferred to
a dedicated follow-up; header comment points to the review tracker.

* skip healthcare finalizer for vendor pipelines

runner.main applied add_aarete_derived_payer_name/provider_name and
the FIELD_FORMAT_MAPPING reorder unconditionally to every client. The
mapping is healthcare-only (src/constants/investment_columns.py), so
reorder_columns dropped every vendor-specific column (TITLE,
VENDOR_NAME, AMOUNT, SLA/KPI, ...) from the combined cc_results_full
output. Per-file individual outputs survived because they are written
inside generic_processor before the finalizer runs.

Guard the entire healthcare finalizer block with
registry.is_vendor(client) so vendor results keep their own flat
schema.

* move DTC short-circuit above duplicate_detection in runner

In DTC mode, dedup ran twice: once in runner before the short-circuit
(immediately discarded at return), and again inside dtc_main.main on
the same input_dict. Reorder so DTC exits before the runner-scope
dedup is computed. dtc_main keeps its own internal dedup pass.

Inline comment notes the cleaner follow-up — have dtc_main.main take
(original_files_to_process, file_status_map) so both call sites share
a single computation.

* remove dead code from saas/main.py

This module is a thin delegator to runner.main; the actual
safe_process_file lives in runner.py:101. The local copy at saas/main.py
referenced logging, traceback, pd, and file_processing — none of which
were imported, so any caller would have raised NameError immediately.

Drops the function and the four unused module-level imports
(parent_child_main, dtc_main, standardize_service_terms,
qc_qa.pipeline.*). File shrinks from 102 to 28 lines.

* Merge origin/dev into bugfix/stg-to-main-prep

Reconciles the 9 stg-to-main prep fixes with 197 commits dev gained
since 2026-03-16. Resolved 14 conflicts:

Group A (preserve our 9 fixes):
- saas/main.py: dead-code cleanup preserved
- code_funcs.py: code_explicit log demoted to DEBUG preserved
- document_classification/main.py: 3 stray prints removed preserved
- vendors/prompt_templates.py: NOTE flagging legacy JSON wording preserved
- cache_registry.py: SERVICE_ENRICHMENT registry entry preserved
- service_term_standardization.py: GROUPING_KEY casing + cache=True preserved
- runner.py: vendor finalizer guard + DTC short-circuit reorder preserved

Group B (dev prevails; our fixes don't touch these):
- saas/file_processing.py: dev's per-field confidence scoring (DAIP2-2692)
- postprocessing_funcs.py: dev's add_amendment_intent + _CONF carve-out
- preprocess.py: dev's removal of stale fallback comment
- preprocessing_funcs.py: dev's _find_header_in_text import (needed by body)
- prompts/prompt_templates.py: dev…

Approved-by: Siddhant Medar
2026-05-19 14:37:23 +00:00

629 lines
25 KiB
Python

"""Central prompt cache registry.
Single source of truth for which prompts are eligible for prompt caching,
which model they actually run on at runtime, and which cache class they
use. This is consumed by the cache-warming layer in
`src.pipelines.runner` and by reporting / contract tests.
Why this exists
---------------
Before the registry, cache warming and runtime call sites were split:
- `prompt_templates.get_cacheable_instructions()` produced the warmed list
- runtime code in `pipelines/.../prompt_calls.py` chose its own model id
The two could (and did) drift. For example `SPLIT_SERVICE_TERM` was warmed
on `sonnet_latest` but invoked on `haiku_latest`, so warm-up tokens were
spent against a cache key the runtime never read.
Cache classes
-------------
- INSTRUCTION: the system instruction block is cached; cache reads
are expected on every call after warm-up.
- CONTEXT: a dynamic `context_for_caching` block is cached;
reads are only expected when the same context
repeats (e.g. the same exhibit text across fields).
- INSTRUCTION_PLUS_CONTEXT: both blocks are cached.
- NONE: explicitly opted out; either the runtime model
does not support caching in this Bedrock setup,
or the static instruction is too small to qualify
and is not yet padded.
Reporting should evaluate INSTRUCTION and CONTEXT cache classes
separately so per-context variability does not make instruction caching
look broken.
"""
from __future__ import annotations
import logging
import re
from dataclasses import dataclass
from typing import Callable, Iterable, Iterator
import src.config as config
import src.prompts.prompt_templates as prompt_templates
# Default minimum tokens for a cache_control block to be honored.
# Per-model overrides live in `llm_utils._MODEL_MIN_CACHE_TOKENS`. This
# constant is the conservative fallback used when an entry has no
# explicit min_instruction_tokens set and the resolved model is unknown.
MIN_CACHE_TOKENS = 1024
# Same heuristic used by `prompt_call_tracking._estimate_tokens` so warm-up
# audits agree with runtime token reporting.
_WORDS_TO_TOKENS_MULTIPLIER = 1.3
# ---------------------------------------------------------------------------
# Cache classes
# ---------------------------------------------------------------------------
class CacheClass:
INSTRUCTION = "instruction"
CONTEXT = "context"
INSTRUCTION_PLUS_CONTEXT = "instruction+context"
NONE = "none"
_VALID_CACHE_CLASSES = {
CacheClass.INSTRUCTION,
CacheClass.CONTEXT,
CacheClass.INSTRUCTION_PLUS_CONTEXT,
CacheClass.NONE,
}
@dataclass(frozen=True)
class CachePromptEntry:
"""One cacheable prompt declaration.
Attributes:
usage_label: Stable label passed to `invoke_claude(usage_label=...)`
and emitted in runtime logs. Must match what runtime call sites
pass.
instruction_func: Zero-arg callable returning the static
instruction text. May be None if the prompt only uses context
caching (cache_class == CONTEXT) and has no static instruction
we want to warm.
runtime_model: Model alias or full id this prompt is invoked with
at runtime (e.g. "sonnet_latest"). The warming layer resolves
this through `config.resolve_model_id`.
cache_class: One of the CacheClass values.
min_instruction_tokens: Optional explicit minimum size for the
instruction block. When None (the default) the warming layer
uses the per-model threshold from
`llm_utils._min_cache_tokens(resolved_model_id)` — 1024 for
Sonnet 4.5, 4096 for Haiku 4.5, etc. Set this only when an
entry needs a stricter floor than the model defaults.
notes: Free text — usually the reason this entry deviates from the
default (e.g. why something is NONE).
"""
usage_label: str
instruction_func: Callable[[], str] | None
runtime_model: str
cache_class: str
min_instruction_tokens: int | None = None
notes: str = ""
def __post_init__(self) -> None:
if self.cache_class not in _VALID_CACHE_CLASSES:
raise ValueError(
f"Unknown cache_class={self.cache_class!r} on {self.usage_label}. "
f"Expected one of {sorted(_VALID_CACHE_CLASSES)}."
)
if (
self.cache_class
in (
CacheClass.INSTRUCTION,
CacheClass.INSTRUCTION_PLUS_CONTEXT,
)
and self.instruction_func is None
):
raise ValueError(
f"{self.usage_label}: instruction-cached entries require "
f"an instruction_func."
)
# ---------------------------------------------------------------------------
# Token estimation
# ---------------------------------------------------------------------------
def estimate_instruction_tokens(text: str | None) -> int:
"""Estimate token count for an instruction string.
Uses the same words*1.3 heuristic as runtime prompt-call tracking so
warm-up audits and runtime reports stay aligned.
"""
if not text:
return 0
words = len(re.findall(r"\S+", text))
return int(round(words * _WORDS_TO_TOKENS_MULTIPLIER))
# ---------------------------------------------------------------------------
# Registry
# ---------------------------------------------------------------------------
# Conventions:
# - usage_label MUST match what call sites pass to `invoke_claude(...)`.
# - runtime_model is the alias the call site passes, not the resolved id.
# - Instruction-only entries that are below 1024 tokens after the latest
# pad expansion are still listed as INSTRUCTION; the warming layer logs
# a below-threshold warning so the gap is visible without silently
# hiding the prompt.
_REGISTRY: tuple[CachePromptEntry, ...] = (
# ---------- one_to_n primary / breakout family ----------
CachePromptEntry(
usage_label="REIMBURSEMENT_PRIMARY",
instruction_func=prompt_templates.REIMBURSEMENT_PRIMARY_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="METHODOLOGY_BREAKOUT",
instruction_func=prompt_templates.METHODOLOGY_BREAKOUT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="DYNAMIC_CODE_ASSIGNMENT",
instruction_func=prompt_templates.DYNAMIC_CODE_ASSIGNMENT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="FEE_SCHEDULE_BREAKOUT",
instruction_func=prompt_templates.FEE_SCHEDULE_BREAKOUT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="GROUPER_BREAKOUT",
instruction_func=prompt_templates.GROUPER_BREAKOUT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="OUTLIER_BREAKOUT",
instruction_func=prompt_templates.OUTLIER_BREAKOUT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="CARVEOUT_CHECK",
instruction_func=prompt_templates.CARVEOUT_CHECK_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="DYNAMIC_ASSIGNMENT",
instruction_func=prompt_templates.DYNAMIC_ASSIGNMENT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="REIMB_DATES_ASSIGNMENT",
instruction_func=prompt_templates.REIMB_DATES_ASSIGNMENT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="LESSER_OF_DISTRIBUTION",
instruction_func=prompt_templates.LESSER_OF_DISTRIBUTION_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="LESSER_OF_CHECK",
instruction_func=prompt_templates.LESSER_OF_CHECK_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="VALIDATE_REIMBURSEMENTS",
instruction_func=prompt_templates.VALIDATE_REIMBURSEMENTS_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="LOB_RELATIONSHIP",
instruction_func=prompt_templates.LOB_RELATIONSHIP_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
notes=(
"Runtime label is `prompt_lob_relationship` (see alias). "
"Currently below 1024-token threshold; warming layer logs a "
"below-threshold warning until the instruction is expanded."
),
),
# ---------- exhibit-level (instruction + context cache) ----------
CachePromptEntry(
usage_label="EXHIBIT_LEVEL",
instruction_func=prompt_templates.EXHIBIT_LEVEL_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION_PLUS_CONTEXT,
notes=(
"Calls also pass context_for_caching for the exhibit text; "
"instruction reads are stable, context reads only repeat per "
"exhibit."
),
),
CachePromptEntry(
usage_label="DYNAMIC_PRIMARY",
instruction_func=prompt_templates.DYNAMIC_PRIMARY_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION_PLUS_CONTEXT,
notes="Exhibit text is sent as context_for_caching across fields.",
),
CachePromptEntry(
usage_label="DYNAMIC_PRIMARY_ENTITIES",
instruction_func=prompt_templates.DYNAMIC_PRIMARY_ENTITIES_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.CONTEXT,
notes=(
"Static instruction is intentionally short (~180 tokens) and "
"below the 1024-token Sonnet 4.5 cache minimum; padding it "
"would dwarf the actual prompt. Caching here relies on the "
"context_for_caching block (the exhibit text) repeating "
"within a run. CONTEXT class skips instruction warm-up to "
"avoid spending warm-tokens on a block that cannot cache."
),
),
CachePromptEntry(
usage_label="DYNAMIC_PRIMARY_ENTITY_CLASSIFICATION",
instruction_func=prompt_templates.DYNAMIC_PRIMARY_ENTITY_CLASSIFICATION_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION_PLUS_CONTEXT,
notes=(
"Classification step that runs after entity extraction. "
"Instruction expanded over the 1024-token Sonnet 4.5 cache "
"minimum so the static block caches; the dynamic entity list "
"in context_for_caching is too small to cache reliably and "
"is expected to miss on the context block."
),
),
CachePromptEntry(
usage_label="DYNAMIC_PRIMARY_ENTITIES_ASSIGNMENT",
instruction_func=prompt_templates.DYNAMIC_PRIMARY_ENTITIES_ASSIGNMENT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION_PLUS_CONTEXT,
notes="Field-level assignment step for DYNAMIC_PRIMARY_ENTITIES; exhibit text passed as context_for_caching.",
),
# ---------- LOB / program / product mapping ----------
CachePromptEntry(
usage_label="MAP_LOB_TO_VALID",
instruction_func=prompt_templates.MAP_LOB_TO_VALID_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION_PLUS_CONTEXT,
notes="Maps raw LOB values to valid crosswalk keys; valid-LOB list passed as context_for_caching.",
),
CachePromptEntry(
usage_label="MAP_NETWORK_TO_VALID",
instruction_func=prompt_templates.MAP_NETWORK_TO_VALID_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION_PLUS_CONTEXT,
notes="Maps raw NETWORK values to valid crosswalk keys; valid-network list passed as context_for_caching.",
),
CachePromptEntry(
usage_label="AARETE_DERIVED_PROGRAM",
instruction_func=prompt_templates.AARETE_DERIVED_PROGRAM_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION_PLUS_CONTEXT,
notes="Maps extracted PROGRAM names to AARETE_DERIVED_PROGRAM values; valid-values list passed as context_for_caching.",
),
CachePromptEntry(
usage_label="AARETE_DERIVED_PRODUCT",
instruction_func=prompt_templates.AARETE_DERIVED_PRODUCT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION_PLUS_CONTEXT,
notes="Maps extracted PRODUCT names to AARETE_DERIVED_PRODUCT values; valid-values list passed as context_for_caching.",
),
CachePromptEntry(
usage_label="PROGRAM_TO_LOB",
instruction_func=prompt_templates.PROGRAM_TO_LOB_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
notes="Derives LOB values from raw PROGRAM + AARETE_DERIVED_PROGRAM; no context_for_caching at runtime.",
),
CachePromptEntry(
usage_label="PRODUCT_TO_LOB",
instruction_func=prompt_templates.PRODUCT_TO_LOB_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
notes="Derives LOB values from raw PRODUCT + AARETE_DERIVED_PRODUCT; no context_for_caching at runtime.",
),
# ---------- service term split (Sonnet 4.5 — Haiku 4.5 won't cache) ----------
CachePromptEntry(
usage_label="SPLIT_SERVICE_TERM",
instruction_func=prompt_templates.SPLIT_SERVICE_TERM_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
notes=(
"Moved off haiku_latest because Haiku 4.5's 4096-token "
"minimum cache size means the ~1600-token SPLIT_SERVICE_TERM "
"instruction was never cached. Sonnet 4.5's 1024-token "
"minimum makes it cache cleanly. With heavy repetition of "
"service terms across exhibits, cache reads on Sonnet beat "
"uncached Haiku on cost despite the higher per-token rate."
),
),
# ---------- preprocessing ----------
CachePromptEntry(
usage_label="EXHIBIT_HEADER",
instruction_func=prompt_templates.EXHIBIT_HEADER_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="EXHIBIT_LINKAGE",
instruction_func=prompt_templates.EXHIBIT_LINKAGE_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="EXHIBIT_TITLE_MATCH",
instruction_func=prompt_templates.EXHIBIT_TITLE_MATCH_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
# ---------- utility ----------
CachePromptEntry(
usage_label="DATE_FIX",
instruction_func=prompt_templates.DATE_FIX_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="DERIVED_TERM_DATE",
instruction_func=prompt_templates.DERIVED_TERM_DATE_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="CHECK_PROVIDER_NAME_MATCH",
instruction_func=prompt_templates.CHECK_PROVIDER_NAME_MATCH_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="SPECIAL_CASE_ASSIGNMENT",
instruction_func=prompt_templates.SPECIAL_CASE_ASSIGNMENT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="SPLIT_REIMB_DATES",
instruction_func=prompt_templates.SPLIT_REIMB_DATES_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
# ---------- code extraction family ----------
CachePromptEntry(
usage_label="SERVICE_ENRICHMENT",
instruction_func=prompt_templates.SERVICE_ENRICHMENT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="CODE_EXPLICIT",
instruction_func=prompt_templates.CODE_EXPLICIT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="CODE_CATEGORY",
instruction_func=prompt_templates.CODE_CATEGORY_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="CODE_IMPLICIT_SPECIAL",
instruction_func=prompt_templates.CODE_IMPLICIT_SPECIAL_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="CODE_IMPLICIT",
instruction_func=prompt_templates.CODE_IMPLICIT_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="CODE_IMPLICIT_ARBITRATION",
instruction_func=prompt_templates.CODE_IMPLICIT_ARBITRATION_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="CODE_LAST_CHECK",
instruction_func=prompt_templates.CODE_LAST_CHECK_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="FILL_BILL_TYPE",
instruction_func=prompt_templates.FILL_BILL_TYPE_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
# ---------- one_to_one ----------
CachePromptEntry(
usage_label="TIN_NPI_TEMPLATE",
instruction_func=prompt_templates.TIN_NPI_TEMPLATE_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
CachePromptEntry(
usage_label="ONE_TO_ONE_SINGLE_FIELD",
instruction_func=prompt_templates.ONE_TO_ONE_SINGLE_FIELD_INSTRUCTION,
runtime_model="sonnet_latest",
cache_class=CacheClass.INSTRUCTION,
),
)
# Aliases let runtime usage_labels that don't match the canonical
# instruction-function name still resolve to one registry entry.
# Callers should prefer canonical labels; this is a compatibility shim.
_USAGE_LABEL_ALIASES: dict[str, str] = {
"code_implicit_rag": "CODE_IMPLICIT",
"prompt_provider_info": "TIN_NPI_TEMPLATE",
"validate_reimbursements_for_llm": "VALIDATE_REIMBURSEMENTS",
"fill_bill_type": "FILL_BILL_TYPE",
"prompt_lob_relationship": "LOB_RELATIONSHIP",
}
def _build_index() -> dict[str, CachePromptEntry]:
index: dict[str, CachePromptEntry] = {}
for entry in _REGISTRY:
if entry.usage_label in index:
raise ValueError(
f"Duplicate registry entry for usage_label={entry.usage_label}"
)
index[entry.usage_label] = entry
return index
_INDEX: dict[str, CachePromptEntry] = _build_index()
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def all_entries() -> tuple[CachePromptEntry, ...]:
"""Return every registered entry (including NONE / opted-out ones)."""
return _REGISTRY
def get_entry(usage_label: str) -> CachePromptEntry | None:
"""Look up an entry by usage_label. Resolves aliases."""
if usage_label in _INDEX:
return _INDEX[usage_label]
canonical = _USAGE_LABEL_ALIASES.get(usage_label)
if canonical is not None:
return _INDEX.get(canonical)
return None
def _resolve_min_tokens(entry: CachePromptEntry, resolved_model_id: str) -> int:
"""Pick the threshold to enforce: explicit on the entry > per-model > default."""
if entry.min_instruction_tokens is not None:
return entry.min_instruction_tokens
# Imported lazily so this module stays importable without llm_utils.
from src.utils.llm_utils import _min_cache_tokens
return _min_cache_tokens(resolved_model_id)
def iter_warming_targets() -> Iterator[tuple[str, str, str]]:
"""Yield (usage_label, resolved_model_id, instruction_text) tuples for
every entry whose instruction block is supposed to be cached.
Skips:
- cache_class == NONE
- cache_class == CONTEXT (no static instruction to warm)
- entries whose resolved model does not support prompt caching
- entries whose instruction function fails to render
"""
# Imported here to avoid a circular import: llm_utils → cache_registry
# is allowed, but cache_registry → llm_utils only at call time.
from src.utils.llm_utils import _supports_prompt_cache
for entry in _REGISTRY:
if entry.cache_class in (CacheClass.NONE, CacheClass.CONTEXT):
continue
if entry.instruction_func is None:
continue
resolved_model_id = config.resolve_model_id(entry.runtime_model)
if not _supports_prompt_cache(resolved_model_id):
logging.info(
"cache_registry: skipping warm-up for %s — model %s "
"(resolved %s) does not support prompt caching.",
entry.usage_label,
entry.runtime_model,
resolved_model_id,
)
continue
try:
instruction_text = entry.instruction_func()
except Exception as exc: # field bundles can be unavailable in CLI/tests
logging.debug(
"cache_registry: skipping %s — instruction function raised: %s",
entry.usage_label,
exc,
)
continue
token_estimate = estimate_instruction_tokens(instruction_text)
threshold = _resolve_min_tokens(entry, resolved_model_id)
if token_estimate < threshold:
# Warn but still warm — the call still tracks
# cache_creation_input_tokens in the response, and the runtime
# cache_miss_reason makes the gap visible.
logging.warning(
"cache_registry: %s instruction is ~%d tokens, below the "
"%d-token minimum for %s; cache_control block may not be "
"honored.",
entry.usage_label,
token_estimate,
threshold,
resolved_model_id,
)
yield entry.usage_label, resolved_model_id, instruction_text
def audit_below_threshold() -> list[tuple[str, int, int]]:
"""Return (usage_label, estimated_tokens, min_required) for every
INSTRUCTION-class entry whose instruction is below threshold. The
threshold is resolved per-model: Sonnet 4.5 = 1024, Haiku 4.5 = 4096,
etc. Useful for a startup audit / contract test.
"""
findings: list[tuple[str, int, int]] = []
for entry in _REGISTRY:
if entry.cache_class not in (
CacheClass.INSTRUCTION,
CacheClass.INSTRUCTION_PLUS_CONTEXT,
):
continue
if entry.instruction_func is None:
continue
try:
text = entry.instruction_func()
except Exception:
continue
resolved_model_id = config.resolve_model_id(entry.runtime_model)
threshold = _resolve_min_tokens(entry, resolved_model_id)
tokens = estimate_instruction_tokens(text)
if tokens < threshold:
findings.append((entry.usage_label, tokens, threshold))
return findings
def instruction_cache_entries() -> Iterable[CachePromptEntry]:
"""Entries where the instruction block is expected to be cached."""
return tuple(
e
for e in _REGISTRY
if e.cache_class
in (CacheClass.INSTRUCTION, CacheClass.INSTRUCTION_PLUS_CONTEXT)
)
def context_cache_entries() -> Iterable[CachePromptEntry]:
"""Entries where a dynamic context block is cached separately."""
return tuple(
e
for e in _REGISTRY
if e.cache_class in (CacheClass.CONTEXT, CacheClass.INSTRUCTION_PLUS_CONTEXT)
)