Bugfix/stg to main prep
* Merged in dev (pull request #1001)
Dev
* Revert premature merge of bugfix/retire_stale_client_file_processing
PR #959 was merged into dev without approval. This reverts commits
5e143c10, 63f32c41, 849aa626, and 927abcae to restore dev to its
pre-merge state. The changes will be re-submitted via a new PR
after proper review.
* Merged in bugfix/retire_stale_client_file_processing (pull request #961)
Return None for dashboard output when dashboard postprocessing is off
* Return None for dashboard output when dashboard postprocessing is off
FINAL_RESULT_DF_DASHBOARD was initialized as an empty DataFrame even
when RUN_DASHBOARD_POSTPROCESSING was False, causing downstream code
to needlessly process it (reorder_columns, etc). Now returns None
when dashboard is not requested, matching the postprocess() contract.
* Merged dev into bugfix/retire_stale_client_file_processing
* Merge dev (with revert) into feature branch
* Re-apply retire stale client file_processing changes
Revert of the revert (4f53b528) to…
* fix grouping_key casing in service term standardization
qc.py uppercases all PC_Details columns to GROUPING_KEY, but the
service-term reader looked up lowercase 'grouping_key'. The lookup
silently failed and degraded per-group standardization to global mode.
* enable prompt caching on service_term_standardization LLM calls
cache=True was missing; the 2.5k-token instruction was being re-billed
on every chunk and mapping-batch call instead of reading from the
shared instruction cache.
* register SERVICE_ENRICHMENT_INSTRUCTION in cache registry
The runtime call at code_funcs.py:75 already passes cache=True with
usage_label="SERVICE_ENRICHMENT", but the registry entry was missing,
so the prompt was never warmed and every first call paid create-tokens.
* remove stray debug prints from document_classification.generate_pre_doczy_report
Three print() calls were dumping the Program/Product mapping to stdout
on every run.
* demote code_explicit raw-answer log from INFO to DEBUG
Full raw LLM responses were spamming INFO-level logs on every call.
* flag vendor prompt_templates convention drift for follow-up
Vendor pipeline still uses legacy hand-rolled JSON wording and has one
stale 'Return result in pipes' reference. Adopting the
JSON_*_FORMAT_INSTRUCTIONS macros from the main pipeline is deferred to
a dedicated follow-up; header comment points to the review tracker.
* skip healthcare finalizer for vendor pipelines
runner.main applied add_aarete_derived_payer_name/provider_name and
the FIELD_FORMAT_MAPPING reorder unconditionally to every client. The
mapping is healthcare-only (src/constants/investment_columns.py), so
reorder_columns dropped every vendor-specific column (TITLE,
VENDOR_NAME, AMOUNT, SLA/KPI, ...) from the combined cc_results_full
output. Per-file individual outputs survived because they are written
inside generic_processor before the finalizer runs.
Guard the entire healthcare finalizer block with
registry.is_vendor(client) so vendor results keep their own flat
schema.
* move DTC short-circuit above duplicate_detection in runner
In DTC mode, dedup ran twice: once in runner before the short-circuit
(immediately discarded at return), and again inside dtc_main.main on
the same input_dict. Reorder so DTC exits before the runner-scope
dedup is computed. dtc_main keeps its own internal dedup pass.
Inline comment notes the cleaner follow-up — have dtc_main.main take
(original_files_to_process, file_status_map) so both call sites share
a single computation.
* remove dead code from saas/main.py
This module is a thin delegator to runner.main; the actual
safe_process_file lives in runner.py:101. The local copy at saas/main.py
referenced logging, traceback, pd, and file_processing — none of which
were imported, so any caller would have raised NameError immediately.
Drops the function and the four unused module-level imports
(parent_child_main, dtc_main, standardize_service_terms,
qc_qa.pipeline.*). File shrinks from 102 to 28 lines.
* Merge origin/dev into bugfix/stg-to-main-prep
Reconciles the 9 stg-to-main prep fixes with 197 commits dev gained
since 2026-03-16. Resolved 14 conflicts:
Group A (preserve our 9 fixes):
- saas/main.py: dead-code cleanup preserved
- code_funcs.py: code_explicit log demoted to DEBUG preserved
- document_classification/main.py: 3 stray prints removed preserved
- vendors/prompt_templates.py: NOTE flagging legacy JSON wording preserved
- cache_registry.py: SERVICE_ENRICHMENT registry entry preserved
- service_term_standardization.py: GROUPING_KEY casing + cache=True preserved
- runner.py: vendor finalizer guard + DTC short-circuit reorder preserved
Group B (dev prevails; our fixes don't touch these):
- saas/file_processing.py: dev's per-field confidence scoring (DAIP2-2692)
- postprocessing_funcs.py: dev's add_amendment_intent + _CONF carve-out
- preprocess.py: dev's removal of stale fallback comment
- preprocessing_funcs.py: dev's _find_header_in_text import (needed by body)
- prompts/prompt_templates.py: dev…
Approved-by: Siddhant Medar
Praneel Panchigar
53a0c76d8e