623799f2a6
Merged in dev (pull request #1001) * Merged in feature/fixPlaceholder2 (pull request #979) Updated feature> dev gate to print variables of echo statements * Updated feature> dev gate to print variables of echo statement * Reverted placeholder changes Approved-by: Sujit Deokar * Merged in bugfix/prov_info_json_fixes (pull request #981) fix: fall back to PROVIDER_NAME in PROV_INFO_JSON when no TIN/NPI extracted * fix: fall back to PROVIDER_NAME in PROV_INFO_JSON when no TIN/NPI extracted When get_prov_info_json short-circuits due to no TIN/NPI regex matches, PROV_INFO_JSON was left as [] even when PROVIDER_NAME was successfully extracted via the one-to-one pipeline. This caused inconsistent output across contracts with the same provider — some files produced a NAME-only entry (via a false-positive regex hit triggering the LLM), others produced []. Reconcile at add_group_and_other, the first point where both extraction streams' results are available. When PROV_INFO_JSON is empty but PROVIDER_NAME is known, synthesize a NAME-only entry with IS_GROUP:"Y" and populate PROV_GROUP_NAME_FULL directly — skipping the provider_name_match_check LLM call since the match is tautological by construction. Adds 5 unit tests covering the str, list, already-populated, empty-name, and all-empty-lis… * Merged in feature/DAIP2-pacificsource-reimbursements-issues (pull request #982) Feature/DAIP2 pacificsource reimbursements issues * Tighten PREMIUM_TERM and DISCOUNT_TERM classifier prompts CARVEOUT_CHECK was misrouting table rate rows into special-case fields, dropping them from the reimbursement output: - "110% of CMS allowed" (base fee-schedule rates) was being classified as PREMIUM_TERM because the prompt treated "above 100% of reference" as an implicit premium. Seen on PacificSource Medicare_Attachment_A1 and A2 Facility contracts where Inpatient/Outpatient rows were missing (2556) or silently fell back to 100% fee-schedule (2557). - Per-service discount rates like "Progressive Lenses: 15% discount", "Contact Lenses: 2% discount", "Frame: 20% discount" were being classified as DISCOUNT_TERM because the prompt only required the word "discount" to appear. Seen on PacificSource Commercial_Attachment_A2 and A5 Professional contracts (2558, 2559). Prompts now require the literal keyword … * Merged in bugfix/filter-docusign-lines (pull request #983) remove docusign lines * remove docusign lines * add unit tests for clean_header_footer docusign/deleted_lines changes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * black forrmatting Approved-by: Katon Minhas * Merged in feature/PC_logic_cleanup_output (pull request #984) Feature/PC logic cleanup output * Few tweaks PC_logics * Fixed orphan_ranking * black formatting * Changes on output field and ranking method * updated few hotfixes * black format fix Approved-by: Katon Minhas * Merged in hotfix/provider_name_group_fix (pull request #988) Hotfix/provider name group fix * Fixes done in GROUP column * black format fix Approved-by: Katon Minhas * Merged in bugfix/DAIP2-2524-carveout-code-optimization (pull request #980) CARVEOUT_CD issue fixed * CARVEOUT_CD issue fixed * pipeline error fixed * Merged dev into bugfix/DAIP2-2524-carveout-code-optimization * Merged dev into bugfix/DAIP2-2524-carveout-code-optimization * Merged dev into bugfix/DAIP2-2524-carveout-code-optimization * trigger cap issue fixed * trigger cap prompt updated * Merged dev into bugfix/DAIP2-2524-carveout-code-optimization Approved-by: Katon Minhas * Merged in bugfix/exhibit-smart-chunking-cost-improvements (pull request #985) Bugfix/exhibit smart chunking cost improvements * Add opt-in instrumentation for per-call token and row-count tracing Introduce src/utils/instrumentation.py (thread-safe CSV logger) and src/utils/instrumentation_context.py (ContextVar scope plus submit_with_context / map_with_context helpers for propagating context into ThreadPoolExecutor workers). Emit events at every Bedrock call in llm_utils.invoke_claude, including in-memory claude_cache hits, with full input/output/cache-read/cache-write token breakdown. Emit row-count events at each row-mutating stage in the one-to-N pipeline (clean_reimbursement_primary, filter_services_without_reimbursements, methodology_breakout, split_service_terms, carveout, dynamic_code_assignment, lesser_of_distribution, dynamic_assignment) and chunking / retrieval events in exhibit smart chunking (chunking_done, retrieval_done) plus exhibit lifecycle events (exhibit_start, exhibit_gate_skip, stage_… * Merged in hotfix/fileextension_issue (pull request #989) Hotfix/fileextension issue * fixed strip_ext issue * black format Approved-by: Katon Minhas * Merged in hotfix/exhibit-header-in-tables (pull request #987) Hotfix/exhibit header in tables * Merged in feature/FixplaceholderIssue (pull request #977) Remove curly braces from echo statements in dev->stg * Remove curly brances from echo statements in dev->stg * Removed curly braces in echo statements in feature-> dev gate Approved-by: Sujit Deokar * Merged in feature/standardized-services (pull request #958) Feature/standardized services * service term standardization * prompt update * prompt updates for standardization * only service standardization * new file * add supporting files and test scripts for standardization work Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * remove old files * Merge remote-tracking branch 'origin/dev' into feature/standardized-services * final fixes * Merged dev into feature/standardized-services * additional features * removed unwanted files * remove unwanted files * Merge branch 'dev' into feature/standardized-services * Merge remote-tr… * Merged in hotfix/postprocess_csv (pull request #990) Hotfix/postprocess csv * date issue_fix * black_format * Merged dev into hotfix/postprocess_csv Approved-by: Katon Minhas * Merged in bugfix/molina_ut_dynamic_primary (pull request #991) Bugfix/molina ut dynamic primary * Prompt changes for dynamic primary * Route cover-sheet-only files to ERRORS.csv instead of leaking phantom rows When every page of a contract was filtered out as a cover sheet / quick-review form, process_file silently returned a FILE_NAME-only DataFrame. Because the runner routes by checking for an "error" column, that file landed in RESULTS.csv as a near-empty row and no ERRORS.csv was generated for the run. - saas/file_processing.py: raise ValueError when text_dict is empty after cover-sheet filtering, so safe_process_file produces a proper error row. - runner.py: add _is_phantom_result defense-in-depth — promote any result with no extracted fields beyond FILE_NAME to error_results with error_type=PhantomSuccess. * Merged dev into bugfix/molina_ut_dynamic_primary * Tighten PRODUCT prompt: restrict to valid_values, prune LOB/PROGRAM examples * Merge branch 'bugfix/molina_ut_dynamic_primary' of bit… * Merged in bugfix/black-format (pull request #994) Black format for pipeline pass * Black format for pipeline pass * Merged in bugfix/DAIP2-2679-fix-nebraska-issues (pull request #995) incorrect inclusion of CPT4_PROC_CD fixed * incorrect inclusion of CPT4_PROC_CD fixed Approved-by: Katon Minhas * Merged in feature/DAIP2-2314-DAIP2-1687-hybrid (pull request #993) Feature/DAIP2-2314 DAIP2 1687 hybrid * remove -files from s3 prefix requirements * Resolve input paths * fix: VendorProcessor.process_file returns (df, None) tuple runner.safe_process_file unpacks the result as (cc_df, dashboard_df), so returning a single DataFrame caused every vendor/generic file to fail with "too many values to unpack (expected 2)" — Python iterates DataFrame columns during unpacking. Vendor pipelines have no dashboard variant; second slot is None and the existing `dashboard_result is not None` guard in runner.py already handles it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * DAIP2-2314 + DAIP2-1687: pad DYNAMIC_PRIMARY + DYNAMIC_PRIMARY_ENTITY_CLASSIFICATION over 1024-token cache floor - Pad DYNAMIC_PRIMARY_INSTRUCTION with three new sections: [SCOPE BOUNDARIES], [SOURCE TEXT INTERPRETATION], [REASONING DISCIPLINE], plus a [WORKED EXAMPLES] block. Estimated tokens: 447 -> 1117 (Sonnet 4.5 … * Merged in bugfix/postprocessing_date_fix (pull request #996) Date formatting changes * Date formatting changes * Merged dev into bugfix/postprocessing_date_fix Approved-by: Katon Minhas * Merged in bugfix/parent-child-rank-orphan-uniqueness (pull request #999) PC_logic bugfix * PC_logic bugfix Approved-by: Katon Minhas * Merged in feature/document-index (pull request #1005) Feature/document index * Add Document Index preprocessing — Layers 1, 2, and 3 wiring Parse the Textract-emitted Document Index block at the top of each contract with a single cached LLM call (prompt_document_index) instead of one per-page call per page. Layer 2 verifies parsed entries via literal string match and structural regex sweep, escalating suspect pages back to the existing per-page path. Layer 1+2 failure triggers a full fallback to today's per-page flow. New symbols: - preprocessing_funcs.extract_document_index_block — regex slice of index prefix - preprocessing_funcs.verify_index_against_pages — structural verifier (plain dict return) - prompt_templates.DOCUMENT_INDEX_INSTRUCTION / DOCUMENT_INDEX — cached prompt pair - prompt_calls.prompt_document_index — LLM wrapper (usage_label DOCUMENT_INDEX_PARSE) - config: DOCUMENT_INDEX_PARSE_ENABLED and three threshold flags - instrumentation: DOCUMENT_INDEX_PARSE mapped to preprocessing segment one… * Merged in feature/active-rates (pull request #1004) Feature/active rates * initial commit * Merged in feature/FixplaceholderIssue (pull request #977) Remove curly braces from echo statements in dev->stg * Remove curly brances from echo statements in dev->stg * Removed curly braces in echo statements in feature-> dev gate Approved-by: Sujit Deokar * Merged in feature/standardized-services (pull request #958) Feature/standardized services * service term standardization * prompt update * prompt updates for standardization * only service standardization * new file * add supporting files and test scripts for standardization work Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * remove old files * Merge remote-tracking branch 'origin/dev' into feature/standardized-services * final fixes * Merged dev into feature/standardized-services * additional features * removed unwanted files * remove unwanted files * Merge branch 'dev' into feature/standardized-services * Merge remote-track… * Merged in feature/one-to-one-confidence-scoring (pull request #1002) Feature/one to one confidence scoring * T1 plumbing: capture per-field confidence + retrieved-chunk metadata for 1:1 HSC fields Prep work for the 1:1 confidence-scoring stage. No scoring logic yet — this just collects the inputs the next ticket (rule-based scorer) will consume. - ONE_TO_ONE_SINGLE_FIELD_TEMPLATE: ask the LLM for confidence (0.0-1.0), verdict (correct/uncertain/not_found), and supporting_snippet alongside the field value. Existing field parser passes the extra keys through unchanged. - prompt_hsc_single_field: now returns a 4-tuple (name, value, field, metadata) where metadata holds the confidence/verdict/snippet plus a lightweight summary of which chunks the LLM saw (count + ids). _extract_hsc_metadata is defensive: clamps out-of-range confidences, defaults a missing/garbage verdict, caps the snippet at 500 chars, returns _empty_hsc_metadata() on every bail-out path. - run_hybrid_smart_chunked_fields: opt… * Merged in bugfix/confidence-flagged-missing-field-col (pull request #1006) Fix KeyError in compute_flagged when a *_CONF column has no value sibling * Fix KeyError in compute_flagged when a *_CONF column has no value sibling Production hit a hard crash at the end of every run: KeyError: "['DYNAMIC_PRIMARY_ENTITIES'] not in index" src/qc_qa/confidence/summary.py:143 compute_flagged was iterating over *_CONF columns and unconditionally indexing the dataframe with both the FILE_NAME column and the stripped value column. That assumed every <FIELD>_CONF column has a sibling <FIELD> value column in final_df. That isn't always true: dynamic-primary features carry only the _CONF side (their value side is dropped by reorder_columns since it isn't in FIELD_FORMAT_MAPPING but its _CONF suffix matches the explicit _CONF carve-out). When the model produced a below-threshold score for one of these and the value column was absent, pandas .loc raised KeyError and the runner crashed. Fix: - Build the .loc colu… * Merged in feature/active-rates (pull request #1007) Feature/active rates * amendment intent tag * prompt update * Merge remote-tracking branch 'origin/dev' into feature/active-rates * Merge remote-tracking branch 'origin/dev' into feature/active-rates * Merge branch 'dev' into feature/active-rates * exhibit standardization updates * use AARETE_DERIVED_EXHIBIT_TITLE for intent * active rates stuff * Merge branch 'dev' into feature/active-rates * prompt update * amendment intent types * active rates logic update * unit tests * Merged dev into feature/active-rates * logic updates * Merge remote-tracking branch 'origin/dev' into feature/active-rates * Merge remote-tracking branch 'origin/dev' into feature/active-rates * updated instrumentation cost logs * cost loging * caching updates * Merge remote-tracking branch 'origin/dev' into feature/active-rates * Merge remote-tracking branch 'origin/dev' into feature/active-rates * logging fix * null check issue fixes * caching fix Approved-by… * Merged in bugfix/stg-to-main-prep (pull request #1012) Bugfix/stg to main prep * Merged in dev (pull request #1001) Dev * Revert premature merge of bugfix/retire_stale_client_file_processing PR #959 was merged into dev without approval. This reverts commits5e143c10,63f32c41,849aa626, and927abcaeto restore dev to its pre-merge state. The changes will be re-submitted via a new PR after proper review. * Merged in bugfix/retire_stale_client_file_processing (pull request #961) Return None for dashboard output when dashboard postprocessing is off * Return None for dashboard output when dashboard postprocessing is off FINAL_RESULT_DF_DASHBOARD was initialized as an empty DataFrame even when RUN_DASHBOARD_POSTPROCESSING was False, causing downstream code to needlessly process it (reorder_columns, etc). Now returns None when dashboard is not requested, matching the postprocess() contract. * Merged dev into bugfix/retire_stale_client_file_processing * Merge dev (with revert) into feature branch * Re-ap… * Merged in bugfix/sync-stg-into-dev-20260518 (pull request #1015) Merged in dev (pull request #1001) * Merged in dev (pull request #1001) Dev * Revert premature merge of bugfix/retire_stale_client_file_processing PR #959 was merged into dev without approval. This reverts commits5e143c10,63f32c41,849aa626, and927abcaeto restore dev to its pre-merge state. The changes will be re-submitted via a new PR after proper review. * Merged in bugfix/retire_stale_client_file_processing (pull request #961) Return None for dashboard output when dashboard postprocessing is off * Return None for dashboard output when dashboard postprocessing is off FINAL_RESULT_DF_DASHBOARD was initialized as an empty DataFrame even when RUN_DASHBOARD_POSTPROCESSING was False, causing downstream code to needlessly process it (reorder_columns, etc). Now returns None when dashboard is not requested, matching the postprocess() contract. * Merged dev into bugfix/retire_stale_client_file_processing * Merge dev (with revert) into fe… * Merged in dev (pull request #1016) Dev * Merged in bugfix/DAIP2-2823-generic-issue-fixes-one-to-one (pull request #1011) Bugfix/DAIP2-2823 generic issue fixes one to one * updated auto renewal ind prompt * updated CONTRACT_AMENDMENT_NUM prompt * Merged dev into bugfix/DAIP2-2823-generic-issue-fixes-one-to-one Approved-by: Praneel Panchigar Approved-by: Siddhant Medar * Merged in feature/DAIP2-2698-phase-3-program-product-lob-mapping (pull request #1000) Feature/DAIP2-2698 phase 3 program product lob mapping * added missing phase 2 modifications * added phase 3 modifications * Fixed acronym issues * black format fix * Merged dev into feature/DAIP2-2698-phase-3-program-product-lob-mapping * standardization fixes * Merged dev into feature/DAIP2-2698-phase-3-program-product-lob-mapping * added hyphenated suffix names fix * black format fix * standardization fix * dynamic primary fix * Merge Dev into feature/DAIP2-2698-phase-3-program-product-lob-mapping * updated Program Product Standardiza… Approved-by: Praneel Panchigar
587 lines
28 KiB
YAML
587 lines
28 KiB
YAML
# =============================================================================
|
|
# Bitbucket Pipelines — CI/CD Configuration
|
|
# =============================================================================
|
|
#
|
|
# Promotion flow (the path code takes from dev to production):
|
|
#
|
|
# feature/* ──┐
|
|
# bugfix/* ──┼──→ dev ──→ stg ──→ main ──→ [release tag] ──→ production
|
|
# hotfix/* ──┼──→ main (fast-track for urgent fixes)
|
|
# └──→ dev (to keep dev in sync with the hotfix)
|
|
#
|
|
# Three pipeline triggers:
|
|
# default → Runs on every push/commit when NO PR is open
|
|
# pull-requests → Runs when a PR is opened or updated
|
|
# custom → Runs manually from the Bitbucket UI (release / rollback)
|
|
#
|
|
# Branch name policy:
|
|
# Only feature/*, bugfix/*, hotfix/*, dev, stg have PR pipelines defined.
|
|
# Any other branch name simply gets NO PR pipeline → checks won't pass
|
|
# → PR cannot be merged. No catch-all '**' pattern (that caused double runs).
|
|
#
|
|
# SETUP REQUIRED: enforce these gates by enabling "Require a successful
|
|
# build to merge" for dev, stg, and main in:
|
|
# Bitbucket → Repository settings → Branch permissions
|
|
# Without this, the gates are advisory only — PRs can be merged regardless
|
|
# of whether the gate, lint, tests, or AI review pass or fail.
|
|
# =============================================================================
|
|
|
|
|
|
# =============================================================================
|
|
# DEFINITIONS — Reusable building blocks
|
|
# =============================================================================
|
|
definitions:
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Custom cache: uv stores packages in ~/.cache/uv, NOT ~/.cache/pip.
|
|
# pip cache is intentionally absent — uv fully replaces pip for dependency
|
|
# management, so caching pip's download dir would waste space and never hit.
|
|
# Without this uv cache, every pipeline run re-downloads all dependencies.
|
|
# ---------------------------------------------------------------------------
|
|
caches:
|
|
uv: ~/.cache/uv
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Reusable script fragments (YAML anchors — defined with & , used with * )
|
|
# These are single strings that get injected into a step's script list.
|
|
# ---------------------------------------------------------------------------
|
|
scripts:
|
|
|
|
# Installs the uv package manager (pinned to avoid surprise breaks),
|
|
# then syncs both dev and test dependency groups from pyproject.toml.
|
|
# --locked (NOT --frozen) asserts uv.lock matches pyproject.toml and
|
|
# fails loudly on mismatch. --frozen would silently use a stale lockfile.
|
|
# allows patch security updates, blocks breaking minor/major bumps — review uv releases quarterly
|
|
- script: &install |
|
|
pip install "uv>=0.11.5,<0.12";
|
|
uv sync --locked --group dev --group test;
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Reusable step definitions
|
|
# Each step is a self-contained CI job with its own Docker container.
|
|
# ---------------------------------------------------------------------------
|
|
steps:
|
|
|
|
# -- Lint: checks that all Python files are formatted with Black ----------
|
|
# Fails if any file would be reformatted (--check = read-only mode)
|
|
- step: &lint
|
|
name: Lint (black)
|
|
image: python:3.12.7
|
|
script:
|
|
- *install # install uv + sync dependencies
|
|
- uv run black --check src/ # check formatting (no changes made)
|
|
caches:
|
|
- uv # cache uv packages between runs
|
|
|
|
# -- Type check: static analysis with mypy --------------------------------
|
|
# Catches type errors at CI time instead of at runtime
|
|
- step: &typecheck
|
|
name: Type Check (mypy)
|
|
image: python:3.12.7
|
|
script:
|
|
- *install # install uv + sync dependencies
|
|
- uv run mypy src/ # run type checker on source code
|
|
caches:
|
|
- uv
|
|
|
|
# -- Unit tests: runs the test suite with pytest --------------------------
|
|
- step: &unit-tests
|
|
name: Unit Tests (pytest)
|
|
image: python:3.12.7
|
|
script:
|
|
- *install # install uv + sync dependencies
|
|
- uv run pytest src/tests/ # run all tests under src/tests/
|
|
caches:
|
|
- uv
|
|
|
|
# -- AI Code Review: automated review via DoczyAI -------------------------
|
|
# Uses AWS OIDC auth (no stored secrets) to call the review agent.
|
|
# The agent clones itself from a separate repo, analyzes the PR diff,
|
|
# and posts review comments directly on the pull request.
|
|
- step: &ai-code-review
|
|
name: AI Code Review
|
|
image: python:3.12.7-slim # slim is fine here — no uv sync
|
|
# SETUP REQUIRED: configure OIDC provider in
|
|
# Bitbucket → Workspace settings → Security → OpenID Connect
|
|
# and add the matching trust policy to the IAM role below.
|
|
# Without it, the step fails at startup before any script runs.
|
|
oidc: true # enables OIDC token injection
|
|
script:
|
|
# Slim image doesn't include git — install it + AWS/HTTP libs
|
|
# --no-install-recommends skips git-man/less/openssh-client etc.
|
|
- apt-get update && apt-get install -y --no-install-recommends git && rm -rf /var/lib/apt/lists/*
|
|
- pip install "boto3==1.35.*" "requests==2.32.*"
|
|
|
|
# --- AWS OIDC authentication setup ---
|
|
# How it works:
|
|
# 1. Bitbucket generates a short-lived OIDC token for this step
|
|
# 2. We write that token to a file
|
|
# 3. AWS SDK reads the file + role ARN to assume the IAM role
|
|
# 4. No long-lived AWS keys needed — token expires after the step
|
|
- export AWS_REGION=us-east-1
|
|
- export AWS_ROLE_ARN="arn:aws:iam::975049960860:role/DoczyAI-Bitbucket-OIDC"
|
|
- export AWS_WEB_IDENTITY_TOKEN_FILE="$(pwd)/web-identity-token"
|
|
|
|
# Clone the review agent repo (private — needs BITBUCKET_CLONE_TOKEN)
|
|
# SETUP REQUIRED: set BITBUCKET_CLONE_TOKEN (marked "Secured") in
|
|
# Bitbucket → Repository settings → Repository variables
|
|
# Guard against missing token so failure is self-diagnosing
|
|
- |
|
|
if [ -z "${BITBUCKET_CLONE_TOKEN}" ]; then
|
|
echo "BLOCKED: BITBUCKET_CLONE_TOKEN not set."
|
|
echo "Set it in Repository settings → Repository variables (Secured)."
|
|
exit 1
|
|
fi
|
|
|
|
# Disable debug output (set +x) during token/credential handling to prevent
|
|
# token exposure in process lists or build logs. Wrap all credential operations
|
|
# in this block. Re-enable debug (set -x) after.
|
|
- set +x
|
|
# Create OIDC token file with secure permissions
|
|
- install -m 600 /dev/null "$(pwd)/web-identity-token"
|
|
- printf '%s' "$BITBUCKET_STEP_OIDC_TOKEN" > "$(pwd)/web-identity-token"
|
|
# Configure git credential helper and store credentials
|
|
- git config --global credential.helper store
|
|
- printf 'https://x-token-auth:%s@bitbucket.org\n' "${BITBUCKET_CLONE_TOKEN}" > ~/.git-credentials
|
|
- chmod 600 ~/.git-credentials
|
|
- set -x
|
|
|
|
# Clone the review agent repo
|
|
- git clone -q "https://bitbucket.org/${BITBUCKET_WORKSPACE}/code-review-agent.git"
|
|
|
|
# Build the PR URL and run the agent.
|
|
# AI review is advisory for *runtime* failures: agent crashes,
|
|
# AWS STS errors, and Python exceptions become warnings via the
|
|
# || echo below (which only wraps the python main.py call).
|
|
# *Infrastructure/configuration* failures (missing BITBUCKET_CLONE_TOKEN,
|
|
# git clone failures, apt-get failures) remain HARD failures by
|
|
# design — they indicate the pipeline setup itself is broken and
|
|
# need to be loud, not silently degraded into a stale warning.
|
|
- export PR_URL="https://bitbucket.org/${BITBUCKET_WORKSPACE}/${BITBUCKET_REPO_SLUG}/pull-requests/${BITBUCKET_PR_ID}"
|
|
- cd code-review-agent
|
|
- python main.py "$PR_URL" || echo "WARNING - AI review step failed (non-blocking)"
|
|
|
|
|
|
# =============================================================================
|
|
# PIPELINES
|
|
# =============================================================================
|
|
pipelines:
|
|
|
|
# ===========================================================================
|
|
# DEFAULT — runs on every push when there is NO open PR for the branch
|
|
# ===========================================================================
|
|
# Basic safety net. When a developer pushes commits to their branch
|
|
# before opening a PR, these checks run to catch issues early.
|
|
# No gate checks — this is just "did you break anything?"
|
|
default:
|
|
- parallel: # all three run at the same time
|
|
- step: *lint
|
|
- step: *typecheck
|
|
- step: *unit-tests
|
|
|
|
# ===========================================================================
|
|
# PULL REQUESTS — runs when a PR is opened or updated
|
|
# ===========================================================================
|
|
# Each entry matches a SOURCE branch pattern (the branch the PR comes
|
|
# FROM, not the branch it targets). The gate step inside validates
|
|
# the DESTINATION branch.
|
|
#
|
|
# Flow for every PR:
|
|
# Step 1: Gate → is this PR targeting the correct branch?
|
|
# Step 2: Quality → lint + typecheck + tests (in parallel)
|
|
# Step 3: AI review → automated code review comments on the PR
|
|
#
|
|
# If the gate fails (step 1), steps 2 and 3 never run.
|
|
# ===========================================================================
|
|
pull-requests:
|
|
|
|
# -------------------------------------------------------------------------
|
|
# feature/* → dev only
|
|
# -------------------------------------------------------------------------
|
|
# Feature branches represent new functionality. They must go through
|
|
# dev first for integration testing before promotion to stg/main.
|
|
# Note: Code quality checks (lint, typecheck, unit-tests) run in the
|
|
# default pipeline on every push. This PR pipeline validates branch
|
|
# routing and performs AI code review only.
|
|
"feature/*":
|
|
# Gate steps below use atlassian/default-image:4 (not python:3.12.7)
|
|
# because gates only run bash/shell — no Python needed. The default
|
|
# image is lighter and avoids pulling a full Python toolchain.
|
|
- step:
|
|
name: "Gate: feature → dev"
|
|
image: atlassian/default-image:4
|
|
script:
|
|
- |
|
|
# Resolve source branch (fallback to BITBUCKET_BRANCH if PR variable unavailable)
|
|
SOURCE_BRANCH="${BITBUCKET_PR_SOURCE_BRANCH:-$BITBUCKET_BRANCH}"
|
|
TARGET_BRANCH="${BITBUCKET_PR_DESTINATION_BRANCH}"
|
|
# Gate: feature branches can only target dev
|
|
ALLOWED="dev"
|
|
echo "Source: $SOURCE_BRANCH"
|
|
echo "Target: $TARGET_BRANCH (allowed: $ALLOWED)"
|
|
IFS=',' read -ra targets <<< "$ALLOWED"
|
|
for t in "${targets[@]}"; do
|
|
[ "${TARGET_BRANCH}" = "$t" ] && exit 0
|
|
done
|
|
echo "BLOCKED: target must be one of: $ALLOWED"; exit 1
|
|
- step: *ai-code-review
|
|
|
|
# -------------------------------------------------------------------------
|
|
# bugfix/* → dev only
|
|
# -------------------------------------------------------------------------
|
|
# Bug fixes follow the same path as features — must land in dev first.
|
|
# Note: Code quality checks (lint, typecheck, unit-tests) run in the
|
|
# default pipeline on every push. This PR pipeline validates branch
|
|
# routing and performs AI code review only.
|
|
"bugfix/*":
|
|
- step:
|
|
name: "Gate: bugfix → dev"
|
|
image: atlassian/default-image:4
|
|
script:
|
|
- |
|
|
# Resolve source branch (fallback to BITBUCKET_BRANCH if PR variable unavailable)
|
|
SOURCE_BRANCH="${BITBUCKET_PR_SOURCE_BRANCH:-$BITBUCKET_BRANCH}"
|
|
TARGET_BRANCH="${BITBUCKET_PR_DESTINATION_BRANCH}"
|
|
# Gate: bugfix branches can only target dev
|
|
ALLOWED="dev"
|
|
echo "Source: ${SOURCE_BRANCH}"
|
|
echo "Target: ${TARGET_BRANCH} (allowed: ${ALLOWED})"
|
|
IFS=',' read -ra targets <<< "$ALLOWED"
|
|
for t in "${targets[@]}"; do
|
|
[ "${TARGET_BRANCH}" = "$t" ] && exit 0
|
|
done
|
|
echo "BLOCKED: target must be one of: ${ALLOWED}"; exit 1
|
|
- step: *ai-code-review
|
|
|
|
# -------------------------------------------------------------------------
|
|
# hotfix/* → main OR dev
|
|
# -------------------------------------------------------------------------
|
|
# Hotfixes are urgent production fixes. They can go directly to main
|
|
# (fast-track) or to dev (to keep the dev branch in sync).
|
|
# Note: Code quality checks (lint, typecheck, unit-tests) run in the
|
|
# default pipeline on every push. This PR pipeline validates branch
|
|
# routing and performs AI code review only.
|
|
"hotfix/*":
|
|
- step:
|
|
name: "Gate: hotfix → main or dev"
|
|
image: atlassian/default-image:4
|
|
script:
|
|
- |
|
|
# Resolve source branch (fallback to BITBUCKET_BRANCH if PR variable unavailable)
|
|
SOURCE_BRANCH="${BITBUCKET_PR_SOURCE_BRANCH:-$BITBUCKET_BRANCH}"
|
|
TARGET_BRANCH="${BITBUCKET_PR_DESTINATION_BRANCH}"
|
|
# Gate: hotfix branches can target main (fast-track) or dev
|
|
ALLOWED="main,dev"
|
|
echo "Source: ${SOURCE_BRANCH}"
|
|
echo "Target: ${TARGET_BRANCH} (allowed: ${ALLOWED})"
|
|
IFS=',' read -ra targets <<< "$ALLOWED"
|
|
for t in "${targets[@]}"; do
|
|
[ "${TARGET_BRANCH}" = "$t" ] && exit 0
|
|
done
|
|
echo "BLOCKED: target must be one of: ${ALLOWED}"; exit 1
|
|
- step: *ai-code-review
|
|
|
|
# -------------------------------------------------------------------------
|
|
# dev → stg only
|
|
# -------------------------------------------------------------------------
|
|
# Once features/fixes are integrated in dev, a PR from dev to stg
|
|
# promotes the code to staging for final validation.
|
|
# Note: Code quality checks (lint, typecheck, unit-tests) run in the
|
|
# default pipeline on every push. This PR pipeline validates branch
|
|
# routing and performs AI code review only.
|
|
dev:
|
|
- step:
|
|
name: "Gate: dev → stg"
|
|
image: atlassian/default-image:4
|
|
script:
|
|
- |
|
|
# Resolve source branch (fallback to BITBUCKET_BRANCH if PR variable unavailable)
|
|
SOURCE_BRANCH="${BITBUCKET_PR_SOURCE_BRANCH:-$BITBUCKET_BRANCH}"
|
|
TARGET_BRANCH="${BITBUCKET_PR_DESTINATION_BRANCH}"
|
|
# Gate: dev can only be promoted to stg
|
|
ALLOWED="stg"
|
|
echo "Source: $SOURCE_BRANCH"
|
|
echo "Target: ${TARGET_BRANCH} (allowed: ${ALLOWED})"
|
|
IFS=',' read -ra targets <<< "$ALLOWED"
|
|
for t in "${targets[@]}"; do
|
|
[ "${TARGET_BRANCH}" = "$t" ] && exit 0
|
|
done
|
|
echo "BLOCKED: target must be one of: ${ALLOWED}"; exit 1
|
|
- step: *ai-code-review
|
|
|
|
# -------------------------------------------------------------------------
|
|
# stg → main only
|
|
# -------------------------------------------------------------------------
|
|
# Final promotion: staging to main. After this merges, the code is
|
|
# ready for a production release via the custom release-prod pipeline.
|
|
# Note: Code quality checks (lint, typecheck, unit-tests) run in the
|
|
# default pipeline on every push. This PR pipeline validates branch
|
|
# routing and performs AI code review only.
|
|
stg:
|
|
- step:
|
|
name: "Gate: stg → main"
|
|
image: atlassian/default-image:4
|
|
script:
|
|
- |
|
|
# Resolve source branch (fallback to BITBUCKET_BRANCH if PR variable unavailable)
|
|
SOURCE_BRANCH="${BITBUCKET_PR_SOURCE_BRANCH:-$BITBUCKET_BRANCH}"
|
|
TARGET_BRANCH="${BITBUCKET_PR_DESTINATION_BRANCH}"
|
|
# Gate: stg can only be promoted to main
|
|
ALLOWED="main"
|
|
echo "Source: ${SOURCE_BRANCH}"
|
|
echo "Target: ${TARGET_BRANCH} (allowed: ${ALLOWED})"
|
|
IFS=',' read -ra targets <<< "$ALLOWED"
|
|
for t in "${targets[@]}"; do
|
|
[ "${TARGET_BRANCH}" = "$t" ] && exit 0
|
|
done
|
|
echo "BLOCKED: target must be one of: ${ALLOWED}"; exit 1
|
|
- step: *ai-code-review
|
|
|
|
# -------------------------------------------------------------------------
|
|
# Catch-all: Enforce branch naming standards
|
|
# -------------------------------------------------------------------------
|
|
# Any branch that doesn't match the patterns above (feature/*, bugfix/*,
|
|
# hotfix/*, dev, stg) will match this catch-all. It fails immediately
|
|
# to enforce branch naming conventions.
|
|
#
|
|
# Allowed branch prefixes:
|
|
# • feature/* (new features)
|
|
# • bugfix/* (bug fixes)
|
|
# • hotfix/* (urgent production fixes)
|
|
# • dev (development integration)
|
|
# • stg (staging promotion)
|
|
# -------------------------------------------------------------------------
|
|
"**":
|
|
- step:
|
|
name: "Validate branch naming"
|
|
image: atlassian/default-image:4
|
|
script:
|
|
- |
|
|
# Resolve source branch (fallback to BITBUCKET_BRANCH if PR variable unavailable)
|
|
SOURCE_BRANCH="${BITBUCKET_PR_SOURCE_BRANCH:-$BITBUCKET_BRANCH}"
|
|
echo "============================================================"
|
|
echo "BRANCH NAMING STANDARD VIOLATION"
|
|
echo "============================================================"
|
|
echo "Branch: ${SOURCE_BRANCH}"
|
|
echo ""
|
|
echo "This branch does not match allowed naming conventions."
|
|
echo ""
|
|
echo "Allowed prefixes:"
|
|
echo " • feature/* — for new features"
|
|
echo " • bugfix/* — for bug fixes"
|
|
echo " • hotfix/* — for urgent production fixes"
|
|
echo ""
|
|
echo "Examples of valid branch names:"
|
|
echo " • feature/user-authentication"
|
|
echo " • bugfix/login-page-crash"
|
|
echo " • hotfix/security-patch"
|
|
echo ""
|
|
echo "Please rename your branch to match the standard and try again."
|
|
echo "============================================================"
|
|
exit 1
|
|
|
|
# ===========================================================================
|
|
# CUSTOM — manually triggered from the Bitbucket UI
|
|
# ===========================================================================
|
|
# These don't run automatically. Go to:
|
|
# Pipelines → Run pipeline → select the pipeline → fill in variables
|
|
custom:
|
|
|
|
# -------------------------------------------------------------------------
|
|
# release-prod: Create a version tag and deploy to production
|
|
# -------------------------------------------------------------------------
|
|
# Steps:
|
|
# 1. Verify we're on main (releases only come from main)
|
|
# 2. Find the latest existing tag (e.g., v1.2.0)
|
|
# 3. Bump the version based on RELEASE_TYPE (major / minor / patch)
|
|
# 4. Create + push the new git tag
|
|
# 5. Deploy using the DEPLOY_COMMAND repository variable
|
|
#
|
|
# Version bump examples from v1.2.3:
|
|
# major → v2.0.0 (breaking changes)
|
|
# minor → v1.3.0 (new features, backwards compatible)
|
|
# patch → v1.2.4 (bug fixes only)
|
|
# -------------------------------------------------------------------------
|
|
release-prod:
|
|
- variables:
|
|
- name: RELEASE_TYPE
|
|
default: minor
|
|
allowed-values:
|
|
- major
|
|
- minor
|
|
- patch # supports hotfix releases (v1.2.1)
|
|
|
|
# Step 1: Create the release tag
|
|
- step:
|
|
name: "Create release tag"
|
|
image: atlassian/default-image:4
|
|
script:
|
|
# Verify we're running from main — only main can be released
|
|
- |
|
|
SOURCE_BRANCH="${BITBUCKET_PR_SOURCE_BRANCH:-$BITBUCKET_BRANCH}"
|
|
if [ "$SOURCE_BRANCH" != "main" ]; then
|
|
echo "BLOCKED: releases can only be created from main."
|
|
exit 1
|
|
fi
|
|
|
|
# Configure git identity for the tag commit
|
|
- git config user.email "bitbucket-pipelines@local"
|
|
- git config user.name "Bitbucket Pipelines"
|
|
|
|
# Fetch all existing tags to find the latest version
|
|
- git fetch --tags --force
|
|
|
|
# Find the latest tag matching v* (e.g., v1.2.0)
|
|
# sort -V = version-aware sorting so v1.9 < v1.10
|
|
# If no tags exist yet, start from v0.0.0
|
|
- |
|
|
latest_tag=$(git tag -l "v*" | sort -V | tail -n1)
|
|
if [ -z "$latest_tag" ]; then latest_tag="v0.0.0"; fi
|
|
echo "Latest tag: ${latest_tag}"
|
|
|
|
# Parse the tag into its three version components.
|
|
# Validate strict semver first — bash arithmetic silently mangles
|
|
# non-semver tags (v1, v1.2, v1.2.3-rc1 etc.) into wrong results.
|
|
# Fail loudly with a clear error so nobody ships a bad release.
|
|
- |
|
|
version=$(echo "$latest_tag" | sed 's/^v//')
|
|
# Reject leading zeros (e.g. v01.02.03) — bash arithmetic treats
|
|
# them as octal, which silently produces wrong version numbers.
|
|
if ! [[ "$version" =~ ^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$ ]]; then
|
|
echo "BLOCKED: latest tag '$latest_tag' is not strict semver (expected v#.#.#, no leading zeros)."
|
|
echo "Fix by creating a properly-formatted tag: git tag -a vX.Y.Z -m 'Release vX.Y.Z'"
|
|
exit 1
|
|
fi
|
|
major=$(echo "$version" | cut -d. -f1)
|
|
minor=$(echo "$version" | cut -d. -f2)
|
|
patch=$(echo "$version" | cut -d. -f3)
|
|
# Validate all version components were extracted
|
|
if [ -z "$major" ] || [ -z "$minor" ] || [ -z "$patch" ]; then
|
|
echo "BLOCKED: failed to parse version components from $latest_tag"
|
|
exit 1
|
|
fi
|
|
|
|
# Bump the correct component, reset everything below it
|
|
- |
|
|
if [ "$RELEASE_TYPE" = "major" ]; then
|
|
major=$((major + 1)); minor=0; patch=0
|
|
elif [ "$RELEASE_TYPE" = "minor" ]; then
|
|
minor=$((minor + 1)); patch=0
|
|
else
|
|
patch=$((patch + 1))
|
|
fi
|
|
|
|
# Create the annotated tag and push it to the remote.
|
|
# SETUP REQUIRED: enable "Repository write access" in
|
|
# Bitbucket → Repository settings → Pipelines
|
|
# Without it, git push fails with 403 and releases don't work.
|
|
- |
|
|
new_tag="v${major}.${minor}.${patch}"
|
|
echo "Creating tag: ${new_tag}"
|
|
git tag -a "$new_tag" -m "Release ${new_tag}"
|
|
git push origin "$new_tag"
|
|
|
|
# Save the tag name as an artifact for the deploy step
|
|
- echo "$new_tag" > release_tag.txt
|
|
artifacts:
|
|
- release_tag.txt # passed to the deploy step below
|
|
|
|
# Step 2: Deploy to production
|
|
- step:
|
|
name: "Deploy to production"
|
|
deployment: production # Bitbucket deployment environment
|
|
image: atlassian/default-image:4
|
|
script:
|
|
# Read the tag that was created in step 1.
|
|
# Validate the artifact exists first — a missing file means the
|
|
# tag-creation step failed partway and should not silently deploy.
|
|
- |
|
|
if [ ! -f release_tag.txt ]; then
|
|
echo "BLOCKED: release_tag.txt not found."
|
|
echo "The tag creation step may have failed before writing the artifact."
|
|
exit 1
|
|
fi
|
|
- export RELEASE_TAG="$(cat release_tag.txt)"
|
|
# Validate tag format before deployment — reject corrupted/empty tags.
|
|
# Regex: v followed by three version numbers (no leading zeros).
|
|
# Escaped dots: \. matches literal dots, not any character.
|
|
- |
|
|
if ! [[ "$RELEASE_TAG" =~ ^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$ ]]; then
|
|
echo "BLOCKED: invalid release tag format: $RELEASE_TAG"
|
|
echo "Expected format: v#.#.# (e.g., v1.2.3, no leading zeros)"
|
|
exit 1
|
|
fi
|
|
- echo "Deploying ${RELEASE_TAG}"
|
|
|
|
# DEPLOY_COMMAND is a repository variable configured in
|
|
# Bitbucket → Repository settings → Repository variables.
|
|
# It can reference $RELEASE_TAG. Example:
|
|
# kubectl set image deployment/app container=myrepo/app:$RELEASE_TAG
|
|
- |
|
|
if [ -z "${DEPLOY_COMMAND}" ]; then
|
|
echo "BLOCKED: set repository variable DEPLOY_COMMAND."
|
|
echo "Use \$RELEASE_TAG inside the command to reference the version."
|
|
exit 1
|
|
fi
|
|
- sh -c "$DEPLOY_COMMAND"
|
|
|
|
# -------------------------------------------------------------------------
|
|
# rollback-prod: Roll back production to a previous release tag
|
|
# -------------------------------------------------------------------------
|
|
# Use when a release goes wrong and you need to revert quickly.
|
|
# Provide the tag to roll back to (e.g., v1.1.0) and it redeploys
|
|
# that version using the same DEPLOY_COMMAND.
|
|
# -------------------------------------------------------------------------
|
|
rollback-prod:
|
|
- variables:
|
|
- name: ROLLBACK_TAG
|
|
default: "" # user MUST provide a real tag (e.g. v1.2.0)
|
|
|
|
# Step 1: Validate the tag exists
|
|
- step:
|
|
name: "Validate rollback tag"
|
|
image: atlassian/default-image:4
|
|
script:
|
|
# Verify we're running from main
|
|
- |
|
|
SOURCE_BRANCH="${BITBUCKET_PR_SOURCE_BRANCH:-$BITBUCKET_BRANCH}"
|
|
if [ "$SOURCE_BRANCH" != "main" ]; then
|
|
echo "BLOCKED: rollback can only run from main."
|
|
exit 1
|
|
fi
|
|
|
|
# Make sure a tag was actually provided
|
|
- |
|
|
if [ -z "${ROLLBACK_TAG}" ]; then
|
|
echo "BLOCKED: ROLLBACK_TAG is required."
|
|
exit 1
|
|
fi
|
|
|
|
# Verify the ref exists AND is specifically a tag (not a branch,
|
|
# HEAD, or commit hash). git rev-parse would accept any ref;
|
|
# `git tag -l` + exact-match grep ensures we only accept tags.
|
|
- git fetch --tags --force
|
|
- |
|
|
if ! git tag -l "${ROLLBACK_TAG}" | grep -qxF "${ROLLBACK_TAG}"; then
|
|
echo "BLOCKED: ${ROLLBACK_TAG} is not a tag."
|
|
echo "Rollback requires a release tag (e.g. v1.2.0), not a branch/commit."
|
|
exit 1
|
|
fi
|
|
- echo "Tag ${ROLLBACK_TAG} found — proceeding with rollback."
|
|
|
|
# Step 2: Redeploy using the rollback tag
|
|
- step:
|
|
name: "Rollback deploy to production"
|
|
deployment: production
|
|
image: atlassian/default-image:4
|
|
script:
|
|
- export RELEASE_TAG="${ROLLBACK_TAG}"
|
|
- echo "Rolling back to ${RELEASE_TAG}"
|
|
- |
|
|
if [ -z "${DEPLOY_COMMAND}" ]; then
|
|
echo "BLOCKED: set repository variable DEPLOY_COMMAND."
|
|
echo "Use \$RELEASE_TAG inside the command to reference the version."
|
|
exit 1
|
|
fi
|
|
- sh -c "$DEPLOY_COMMAND"
|