From 6c209e5a8188cb7c7071f5e1ab91a3fae5281cac Mon Sep 17 00:00:00 2001 From: Jay Brown Date: Mon, 15 Sep 2025 18:05:54 +0000 Subject: [PATCH] Merged in feature/config-docs (pull request #181) update docs to reflect all environment variables required and optional * list all env vars --- README.md | 130 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 129 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 31056ac8..68924dcf 100644 --- a/README.md +++ b/README.md @@ -111,12 +111,140 @@ For regular usage, please use `task` scripts in order run the most appropriate c ## Environment Variables -When using environment variables, there is a hiearchy that will be respected. (First will override the next) +### Configuration Hierarchy + +When using environment variables, there is a hierarchy that will be respected. (First will override the next) 1. Within the devbox.json file, a variable added to "env" 2. Within the devbox.json file, a variable exported in "init_hook" 3. A variable added to .env +### Required Environment Variables + +The following environment variables are required for the application to function properly: + +#### Core Application +```bash +LOG_LEVEL=INFO # Logging level: DEBUG|INFO|WARN|ERROR +``` + +#### Database (Required) +```bash +PGHOST=localhost # Database host +PGPORT=5432 # Database port +PGDATABASE=query_orchestration # Database name +PGUSER=postgres # Database username +PGPASSWORD=your_password # Database password +``` + +#### AWS Configuration (Required) +```bash +AWS_REGION=us-east-1 # AWS region +AWS_ACCESS_KEY_ID=your_access_key # AWS access key ID +AWS_SECRET_ACCESS_KEY=your_secret_key # AWS secret access key +AWS_SESSION_TOKEN=your_session_token # AWS session token (if using temporary credentials) +``` + +#### S3 Storage (Required) +```bash +BUCKET=your-s3-bucket-name # S3 bucket for document storage +``` + +#### SQS Queue URLs (All Required) +```bash +QUEUE_URL=https://sqs.region.amazonaws.com/account/queue-name # Main query processing queue +DOCUMENT_INIT_URL=https://sqs.region.amazonaws.com/account/doc-init # Document initialization queue +DOCUMENT_SYNC_URL=https://sqs.region.amazonaws.com/account/doc-sync # Document synchronization queue +DOCUMENT_CLEAN_URL=https://sqs.region.amazonaws.com/account/doc-clean # Document cleaning queue +DOCUMENT_TEXT_URL=https://sqs.region.amazonaws.com/account/doc-text # Document text extraction queue +QUERY_SYNC_URL=https://sqs.region.amazonaws.com/account/query-sync # Query synchronization queue +STORE_EVENT_URL=https://sqs.region.amazonaws.com/account/store-event # Store event queue +CLIENT_SYNC_URL=https://sqs.region.amazonaws.com/account/client-sync # Client synchronization queue +QUERY_VERSION_SYNC_URL=https://sqs.region.amazonaws.com/account/query-version-sync # Query version sync queue +``` + +#### Authentication (Required for Production) +```bash +COGNITO_USER_POOL_ID=us-east-1_xxxxxxx # AWS Cognito User Pool ID +COGNITO_CLIENT_ID=your_client_id # AWS Cognito App Client ID +COGNITO_CLIENT_SECRET=your_secret # AWS Cognito App Client Secret +COGNITO_DOMAIN=your-cognito-domain # AWS Cognito domain name +``` + +#### Authorization (Required for RBAC) +```bash +PERMIT_IO_API_KEY=permit_key_xxxxxxx # Permit.io API key for RBAC +PERMIT_IO_PDP_URL=http://localhost:7766 # Permit.io Policy Decision Point URL - can use cloud version optionally +``` + +### Optional Environment Variables + +#### Development & Testing +```bash +DEBUG=true # Enable debug mode (development only) +DISABLE_AUTH=true # Bypass authentication (development/testing only) +DB_NOSSL=true # Disable SSL for database connections (development only) +``` + +#### AWS LocalStack (Development) +```bash +AWS_ENDPOINT_URL=http://localhost:4566 # Override AWS endpoints for LocalStack +AWS_S3_USE_PATH_STYLE=true # Use path-style S3 URLs for LocalStack +``` + +#### Server Configuration +```bash +PORT=8080 # HTTP server port (default: 8080) +HTTP_HOST=0.0.0.0 # HTTP server host (default: 0.0.0.0) +``` + +#### Observability +```bash +ENABLE_OTEL=false # Enable OpenTelemetry (default: false) +``` + +### Example .env File for Development + +```bash +# Development overrides +LOG_LEVEL=DEBUG +DEBUG=true +DISABLE_AUTH=true +DB_NOSSL=true + +# Local services +PGHOST=localhost +PGPORT=5432 +PGDATABASE=query_orchestration +PGUSER=postgres +PGPASSWORD=pass + +# LocalStack for AWS services +AWS_ENDPOINT_URL=http://localhost:4566 +AWS_S3_USE_PATH_STYLE=true +AWS_REGION=us-east-1 +AWS_ACCESS_KEY_ID=test +AWS_SECRET_ACCESS_KEY=test + +# S3 bucket +BUCKET=test-bucket + +# LocalStack SQS queues +QUEUE_URL=http://localhost:4566/queue/us-east-1/000000000000/query_runner +DOCUMENT_INIT_URL=http://localhost:4566/queue/us-east-1/000000000000/document_init +DOCUMENT_SYNC_URL=http://localhost:4566/queue/us-east-1/000000000000/document_sync +DOCUMENT_CLEAN_URL=http://localhost:4566/queue/us-east-1/000000000000/document_clean +DOCUMENT_TEXT_URL=http://localhost:4566/queue/us-east-1/000000000000/document_text +QUERY_SYNC_URL=http://localhost:4566/queue/us-east-1/000000000000/query_sync +STORE_EVENT_URL=http://localhost:4566/queue/us-east-1/000000000000/store_event +CLIENT_SYNC_URL=http://localhost:4566/queue/us-east-1/000000000000/client_sync +QUERY_VERSION_SYNC_URL=http://localhost:4566/queue/us-east-1/000000000000/query_version_sync + +# Development auth (optional) +PERMIT_IO_API_KEY=test-key +PERMIT_IO_PDP_URL=http://localhost:7766 +``` + ## Testing For testing endtoends such as queries against a database, or interactions with an SQS queue, this repository employs the use of docker testcontainers.