Merged in feature/lesser-table-caching-refactor-hybrid (pull request #847)
Feature/lesser table caching refactor hybrid * chore: Remove unused duplicate main.py from shared pipeline * fix: Correct crosswalk paths in aarete_derived.py * chore: Remove unused documentation files from fieldExtraction * docs: Add documentation files to documentation folder * docs: Update README with uv setup, expanded project structure, and branching conventions * docs: Add uv installation steps with Ubuntu/WSL emphasis * Enable prompt caching for all remaining LLM calls - Add _INSTRUCTION() functions for: EXHIBIT_HEADER, EXHIBIT_LINKAGE, EXHIBIT_TITLE_MATCH, DATE_FIX, DERIVED_TERM_DATE, CHECK_PROVIDER_NAME_MATCH, SPECIAL_CASE_ASSIGNMENT - Update all invoke_claude() calls in saas and clover pipelines to use cache=True with corresponding _INSTRUCTION() functions - Add new instructions to get_cacheable_instructions() for cache warming - Update tests for new instruction functions Functions now using caching: - prompt_exhibit_level - prompt_exhibit_lesser (EXHIBIT_LEVEL_LESSER_OF) - prompt_fee_schedule_breakout - prompt_grouper_breakout - prompt_special_case_assignment - prompt_exhibit_linkage - prompt_exhibit_header - prompt_smart_chunked (ONE_TO_ONE templates) - prompt_date_fix - prompt_derived_term_date - prompt_exhibit_title_match - provider_name_match_check 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Reorder * feat: Add bcbs_promise client pipeline with OFFSET_TERM extraction - Add new bcbs_promise client with HSC-based OFFSET_TERM field extraction - Extract full paragraph text of offset/recoupment provisions from contracts - Derive OFFSET_INDICATOR (Y/N) from OFFSET_TERM presence - Fix reorder_columns to preserve extra columns not in COLUMN_ORDER - Update QC/QA output path to outputs/qc_qa/ * fix: Update dev deps and test assertions for QC/QA output path - Add pytest/pytest-mock to dev dependencies for mypy type checking - Update test assertions to expect outputs/qc_qa instead of qa_qc_output * style: Apply black formatting to prompt_templates.py * Merge main, move scripts * Archive some scripts * update py version * remove .py version file * Remove ASCII characters * Restore testbed code * restore tracking * Update testbed metrics * Enable prompt caching for CODE_LAST_CHECK, FILL_BILL_TYPE, DUAL_LOB_CHECK, and GROUPER_BREAKOUT - Add CODE_LAST_CHECK_INSTRUCTION() for service specificity classification - Add FILL_BILL_TYPE_INSTRUCTION() for bill type code determination - Add DUAL_LOB_CHECK_INSTRUCTION() for Medicare/Medicaid classification - Update code_funcs.py to use caching for CODE_LAST_CHECK, FILL_BILL_TYPE, GROUPER_BREAKOUT - Update postprocessing_funcs.py to use caching for DUAL_LOB_CHECK - Add new instructions to get_cacheable_instructions() for cache warming - Add unit tests for new instruction functions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix postprocessing_funcs to remove invalid columns * Merge branch 'main' into feature/lesser-table-caching-refactor-hybrid * Revert prompt caching changes from aed1b73c * update formatting * Update imports Approved-by: Sha Brown Approved-by: Praneel Panchigar
This commit is contained in:
Katon Minhas
+15
-937
@@ -1,962 +1,40 @@
|
||||
definitions:
|
||||
scripts:
|
||||
- script: &prep-dir
|
||||
DIR=${DIR:=.};
|
||||
cd $DIR;
|
||||
pip install poetry;
|
||||
poetry install;
|
||||
- script: &lint
|
||||
poetry run black .
|
||||
- script: &validate-static
|
||||
poetry run mypy .
|
||||
- script: &unit-test
|
||||
poetry run pytest
|
||||
- script: &aws-context
|
||||
apt-get update;
|
||||
apt-get install -y jq;
|
||||
pip install awscli;
|
||||
export AWS_ROLE_ARN=arn:aws:iam::$AWS_ACCOUNT_NO:role/$OIDC_ROLE;
|
||||
export AWS_ROLE_SESSION_NAME=$OIDC_ROLE;
|
||||
export AWS_WEB_IDENTITY_TOKEN_FILE=$(pwd)/web-identity-token;
|
||||
echo $BITBUCKET_STEP_OIDC_TOKEN > $(pwd)/web-identity-token;
|
||||
export AWS_ACCESS_KEY_ID=$(echo $STS_OUTPUT | jq -r '.Credentials.AccessKeyId');
|
||||
export AWS_SECRET_ACCESS_KEY=$(echo $STS_OUTPUT | jq -r '.Credentials.SecretAccessKey');
|
||||
export AWS_SECURITY_TOKEN=$(echo $STS_OUTPUT | jq -r '.Credentials.SessionToken');
|
||||
export AWS_REGION='us-east-2';
|
||||
aws sts get-caller-identity;
|
||||
export IS_DEPLOY_ALL=true;
|
||||
steps: &deployment-plan-steps
|
||||
- step:
|
||||
name: Plan devops-pipeline
|
||||
image: hugree/bitbucket-aws-python38-tf154:latest
|
||||
oidc: true
|
||||
script:
|
||||
- *prep-dir
|
||||
- *aws-context
|
||||
- python terraform.py plan --environment=$ENVIRONMENT --module=devops-pipeline/other-resources
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
name: Plan textract-pipeline
|
||||
image: hugree/terraform-with-snowsql:latest
|
||||
oidc: true
|
||||
script:
|
||||
- *prep-dir
|
||||
- *aws-context
|
||||
- python terraform.py plan --environment=$ENVIRONMENT --module=textract-pipeline/terraform
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
name: Plan streamlit-server
|
||||
image: hugree/bitbucket-aws-python38-tf154:latest
|
||||
oidc: true
|
||||
script:
|
||||
- *prep-dir
|
||||
- *aws-context
|
||||
- python terraform.py plan --environment=$ENVIRONMENT --module=streamlit-server
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
name: Plan Snowflake Changes
|
||||
image: python:3.7
|
||||
size: 2x
|
||||
script:
|
||||
- pip install snowflake-connector-python
|
||||
- pip install schemachange --upgrade
|
||||
- |
|
||||
schemachange -a $SNOWFLAKE_ACCOUNT -u $SNOWFLAKE_USER \
|
||||
-f snowflake/ \
|
||||
-r $SNOWFLAKE_ROLE -w $SNOWFLAKE_WAREHOUSE \
|
||||
-d $SNOWFLAKE_DATABASE -c $SNOWFLAKE_DATABASE.SCHEMACHANGE.CHANGE_HISTORY \
|
||||
--create-change-history-table -v \
|
||||
--dry-run
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
image: hugree/terraform-with-snowsql:latest
|
||||
name: Plan Snowsql
|
||||
oidc: true
|
||||
script:
|
||||
- *prep-dir
|
||||
- *aws-context
|
||||
- python terraform.py plan --environment=$ENVIRONMENT --module=textract-pipeline/terraform --apply-extra-args="-target=null_resource.snowsql_client_create"
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
image: python:3.8
|
||||
name: Plan streamlit to EC2
|
||||
oidc: true
|
||||
script:
|
||||
- *aws-context
|
||||
- apt-get install -y git
|
||||
- >-
|
||||
aws ssm send-command --document-name "AWS-RunShellScript" --instance-ids $INSTANCE_ID_TF --region $AWS_DEFAULT_REGION --parameters commands='[
|
||||
"sudo -u ubuntu -i <<EOF",
|
||||
"export HOME=/home/ubuntu",
|
||||
"git config --global --add safe.directory /home/ubuntu/doczy.ai",
|
||||
"if [ -d /home/ubuntu/doczy.ai ]; then cd /home/ubuntu/doczy.ai && git fetch && git status; else cd /home/ubuntu && echo \"Doczy.AI does not exist\"; fi",
|
||||
"EOF"
|
||||
]'
|
||||
caches:
|
||||
- pip
|
||||
steps: &deployment-steps
|
||||
- step:
|
||||
name: Apply devops-pipeline
|
||||
image: hugree/bitbucket-aws-python38-tf154:latest
|
||||
oidc: true
|
||||
script:
|
||||
- *prep-dir
|
||||
- *aws-context
|
||||
- python terraform.py apply --environment=$ENVIRONMENT --module=devops-pipeline/other-resources
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
name: Apply textract-pipeline
|
||||
image: hugree/terraform-with-snowsql:latest
|
||||
oidc: true
|
||||
script:
|
||||
- *prep-dir
|
||||
- *aws-context
|
||||
- python terraform.py apply --environment=$ENVIRONMENT --module=textract-pipeline/terraform
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
name: Apply streamlit-server
|
||||
image: hugree/bitbucket-aws-python38-tf154:latest
|
||||
oidc: true
|
||||
script:
|
||||
- *prep-dir
|
||||
- *aws-context
|
||||
- python terraform.py apply --environment=$ENVIRONMENT --module=streamlit-server
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
name: Apply Snowflake Changes
|
||||
image: python:3.7
|
||||
size: 2x
|
||||
script:
|
||||
- pip install snowflake-connector-python
|
||||
- pip install schemachange --upgrade
|
||||
- |
|
||||
schemachange -a $SNOWFLAKE_ACCOUNT -u $SNOWFLAKE_USER \
|
||||
-f snowflake/ \
|
||||
-r $SNOWFLAKE_ROLE -w $SNOWFLAKE_WAREHOUSE \
|
||||
-d $SNOWFLAKE_DATABASE -c $SNOWFLAKE_DATABASE.SCHEMACHANGE.CHANGE_HISTORY \
|
||||
--create-change-history-table -v
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
image: hugree/terraform-with-snowsql:latest
|
||||
name: Apply Snowsql
|
||||
oidc: true
|
||||
script:
|
||||
- *prep-dir
|
||||
- *aws-context
|
||||
- python terraform.py apply --environment=$ENVIRONMENT --module=textract-pipeline/terraform --apply-extra-args="-target=null_resource.snowsql_client_create"
|
||||
caches:
|
||||
- pip
|
||||
- step:
|
||||
image: python:3.8
|
||||
name: Apply streamlit to EC2
|
||||
oidc: true
|
||||
script:
|
||||
- *aws-context
|
||||
- apt-get install -y git
|
||||
- >-
|
||||
aws ssm send-command --document-name "AWS-RunShellScript" --instance-ids $INSTANCE_ID_TF --region $AWS_DEFAULT_REGION --parameters commands='[
|
||||
"sudo -u ubuntu -i <<EOF",
|
||||
"export HOME=/home/ubuntu",
|
||||
"git config --global --add safe.directory /home/ubuntu/doczy.ai",
|
||||
"if [ -d /home/ubuntu/doczy.ai ]; then cd /home/ubuntu/doczy.ai && git fetch && git pull; else cd /home/ubuntu && git clone git@bitbucket.org:aarete/doczy.ai.git; fi",
|
||||
"EOF"
|
||||
]'
|
||||
caches:
|
||||
- pip
|
||||
- script: &install
|
||||
pip install uv;
|
||||
uv sync --group dev --group test;
|
||||
|
||||
pipelines:
|
||||
default:
|
||||
- step:
|
||||
image: python:3.12
|
||||
name: Install Pip
|
||||
name: Install Dependencies
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- python -m pip install --upgrade pip
|
||||
- *install
|
||||
caches:
|
||||
- pip
|
||||
- parallel:
|
||||
- step:
|
||||
name: Lint check tiff-to-pdf
|
||||
name: Lint (black)
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/tiff-to-pdf
|
||||
- *prep-dir
|
||||
- *lint
|
||||
- *install
|
||||
- uv run black --check src/
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/tiff-to-pdf/**
|
||||
- step:
|
||||
name: Lint check batch-creation
|
||||
name: Type Check (mypy)
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/batch-creation
|
||||
- *prep-dir
|
||||
- *lint
|
||||
- *install
|
||||
- uv run mypy src/
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/batch-creation/**
|
||||
- step:
|
||||
name: Lint check call-bedrock
|
||||
name: Unit Tests (pytest)
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/call-bedrock
|
||||
- *prep-dir
|
||||
- *lint
|
||||
- *install
|
||||
- uv run pytest src/tests/
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/call-bedrock/**
|
||||
- step:
|
||||
name: Lint check database-interface
|
||||
image: python:3.11.10
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/database-interface
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/database-interface/**
|
||||
- step:
|
||||
name: Lint check database-interface-get
|
||||
image: python:3.11.10
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/database-interface-get
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/database-interface-get/**
|
||||
- step:
|
||||
name: Lint check docx-to-pdf
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/docx-to-pdf
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/docx-to-pdf/**
|
||||
- step:
|
||||
name: Lint check insert-record
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/insert-record
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/insert-record/**
|
||||
- step:
|
||||
name: Lint check pdf-validation
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/pdf-validation
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/pdf-validation/**
|
||||
- step:
|
||||
name: Lint check prompt-orchestrator
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/prompt-processor
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/prompt-processor/**
|
||||
- step:
|
||||
name: Lint check s3-folder-details
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/s3-folder-details
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/s3-folder-details/**
|
||||
- step:
|
||||
name: Lint check text-creation
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/text-creation
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/text-creation/**
|
||||
- step:
|
||||
name: Lint check textract-receiver
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/textract-receiver
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/textract-receiver/**
|
||||
- step:
|
||||
name: Lint check textract-sender
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/textract-sender
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/textract-sender/**
|
||||
- step:
|
||||
name: Lint check trigger-pipeline
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/trigger-pipeline
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/trigger-pipeline/**
|
||||
- step:
|
||||
name: Lint check streamlit
|
||||
image: python:3.11.10
|
||||
size: 4x
|
||||
script:
|
||||
- DIR=streamlit
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- streamlit/**
|
||||
- step:
|
||||
name: Lint check streamlit_multipage
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=streamlit_multipage
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- streamlit_multipage/**
|
||||
- step:
|
||||
name: Lint check fieldExtraction
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=fieldExtraction
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- fieldExtraction/**
|
||||
- step:
|
||||
name: Lint check Doczy.AI_Automation
|
||||
image: python:3.11.10
|
||||
script:
|
||||
- DIR=Doczy.AI_Automation
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- Doczy.AI_Automation/**
|
||||
- step:
|
||||
name: Lint check airflow
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=airflow
|
||||
- *prep-dir
|
||||
- *lint
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- airflow/**
|
||||
- parallel:
|
||||
- step:
|
||||
name: Static check tiff-to-pdf
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/tiff-to-pdf
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/tiff-to-pdf/**
|
||||
- step:
|
||||
name: Static check batch-creation
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/batch-creation
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/batch-creation/**
|
||||
- step:
|
||||
name: Static check call-bedrock
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/call-bedrock
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/call-bedrock/**
|
||||
- step:
|
||||
name: Static check database-interface
|
||||
image: python:3.11.10
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/database-interface
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/database-interface/**
|
||||
- step:
|
||||
name: Static check database-interface-get
|
||||
image: python:3.11.10
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/database-interface-get
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/database-interface-get/**
|
||||
- step:
|
||||
name: Static check docx-to-pdf
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/docx-to-pdf
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/docx-to-pdf/**
|
||||
- step:
|
||||
name: Static check insert-record
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/insert-record
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/insert-record/**
|
||||
- step:
|
||||
name: Static check pdf-validation
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/pdf-validation
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/pdf-validation/**
|
||||
- step:
|
||||
name: Static check prompt-orchestrator
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/prompt-processor
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/prompt-processor/**
|
||||
- step:
|
||||
name: Static check s3-folder-details
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/s3-folder-details
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/s3-folder-details/**
|
||||
- step:
|
||||
name: Static check text-creation
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/text-creation
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/text-creation/**
|
||||
- step:
|
||||
name: Static check textract-receiver
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/textract-receiver
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/textract-receiver/**
|
||||
- step:
|
||||
name: Static check textract-sender
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/textract-sender
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/textract-sender/**
|
||||
- step:
|
||||
name: Static check trigger-pipeline
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/trigger-pipeline
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/trigger-pipeline/**
|
||||
- step:
|
||||
name: Static check streamlit
|
||||
image: python:3.11.10
|
||||
size: 4x
|
||||
script:
|
||||
- DIR=streamlit
|
||||
- echo Must be in usage
|
||||
- exit 0
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- streamlit/**
|
||||
- step:
|
||||
name: Static check streamlit_multipage
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=streamlit_multipage
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- streamlit_multipage/**
|
||||
- step:
|
||||
name: Static check fieldExtraction
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=fieldExtraction
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- fieldExtraction/**
|
||||
- step:
|
||||
name: Static check Doczy.AI_Automation
|
||||
image: python:3.11.10
|
||||
script:
|
||||
- DIR=Doczy.AI_Automation
|
||||
- echo Must be in usage
|
||||
- exit 0
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- Doczy.AI_Automation/**
|
||||
- step:
|
||||
name: Static check airflow
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=airflow
|
||||
- *prep-dir
|
||||
- *validate-static
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- airflow/**
|
||||
- parallel:
|
||||
- step:
|
||||
name: Unit Test check tiff-to-pdf
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/tiff-to-pdf
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/tiff-to-pdf/**
|
||||
- step:
|
||||
name: Unit Test check batch-creation
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/batch-creation
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/batch-creation/**
|
||||
- step:
|
||||
name: Unit Test check call-bedrock
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/call-bedrock
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/call-bedrock/**
|
||||
- step:
|
||||
name: Unit Test check database-interface
|
||||
image: python:3.11.10
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/database-interface
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/database-interface/**
|
||||
- step:
|
||||
name: Unit Test check database-interface-get
|
||||
image: python:3.11.10
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/database-interface-get
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/database-interface-get/**
|
||||
- step:
|
||||
name: Unit Test check docx-to-pdf
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/docx-to-pdf
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/docx-to-pdf/**
|
||||
- step:
|
||||
name: Unit Test check insert-record
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/insert-record
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/insert-record/**
|
||||
- step:
|
||||
name: Unit Test check pdf-validation
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/pdf-validation
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/pdf-validation/**
|
||||
- step:
|
||||
name: Unit Test check prompt-orchestrator
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/prompt-processor
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/prompt-processor/**
|
||||
- step:
|
||||
name: Unit Test check s3-folder-details
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/s3-folder-details
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/s3-folder-details/**
|
||||
- step:
|
||||
name: Unit Test check text-creation
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/text-creation
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/text-creation/**
|
||||
- step:
|
||||
name: Unit Test check textract-receiver
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/textract-receiver
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/textract-receiver/**
|
||||
- step:
|
||||
name: Unit Test check textract-sender
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/textract-sender
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/textract-sender/**
|
||||
- step:
|
||||
name: Unit Test check trigger-pipeline
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=textract-pipeline/src/lambda/trigger-pipeline
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- textract-pipeline/src/lambda/trigger-pipeline/**
|
||||
- step:
|
||||
name: Unit Test check streamlit
|
||||
image: python:3.11.10
|
||||
size: 4x
|
||||
script:
|
||||
- DIR=streamlit
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- streamlit/**
|
||||
- step:
|
||||
name: Unit Test check streamlit_multipage
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=streamlit_multipage
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- streamlit_multipage/**
|
||||
- step:
|
||||
name: Unit Test check fieldExtraction
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=fieldExtraction
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- fieldExtraction/**
|
||||
- step:
|
||||
name: Unit Test check Doczy.AI_Automation
|
||||
image: python:3.11.10
|
||||
script:
|
||||
- DIR=Doczy.AI_Automation
|
||||
- echo Must be in usage
|
||||
- exit 0
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- Doczy.AI_Automation/**
|
||||
- step:
|
||||
name: Unit Test check airflow
|
||||
image: python:3.12.7
|
||||
script:
|
||||
- DIR=airflow
|
||||
- *prep-dir
|
||||
- *unit-test
|
||||
caches:
|
||||
- pip
|
||||
condition:
|
||||
changesets:
|
||||
includePaths:
|
||||
- airflow/**
|
||||
#branches:
|
||||
# main:
|
||||
# - step:
|
||||
# name: Build and Test
|
||||
# script:
|
||||
# - echo Build and test pipeline will go here
|
||||
# - stage:
|
||||
# name: deployment-dev
|
||||
# deployment: development
|
||||
# steps: *deployment-steps
|
||||
# - stage:
|
||||
# name: deployment-plan-uat
|
||||
# deployment: uat-plan
|
||||
# trigger: manual
|
||||
# steps: *deployment-plan-steps
|
||||
# - stage:
|
||||
# name: deployment-uat
|
||||
# deployment: uat
|
||||
# trigger: manual
|
||||
# steps: *deployment-steps
|
||||
# - stage:
|
||||
# name: deployment-prod
|
||||
# deployment: production
|
||||
# trigger: manual
|
||||
# steps: *deployment-steps
|
||||
Reference in New Issue
Block a user