Merged in feature/aws-config-fix3 (pull request #184)
Make AWS variables optional * comment * aws vars optional
This commit is contained in:
Jay Brown
@@ -1,10 +1,13 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
## General rules DO NOT VIOLATE
|
||||
NEVER code additional features that I did not explicitly ask for without asking permission.
|
||||
|
||||
## General rules DO NOT VIOLATE
|
||||
|
||||
NEVER code additional features that I did not explicitly ask for without asking permission.
|
||||
NEVER create mock data or simplified components unless explicitly told to do so
|
||||
NEVER replace existing complex components with simplified versions - always fix the actual problem or the feature that was requested.
|
||||
Never just bypass tests that have regressed. Stop and find out why they are failing and fix them.
|
||||
ALWAYS work with the existing codebase - do not create new simplified alternatives
|
||||
ALWAYS find and fix the root cause of issues instead of creating workarounds
|
||||
When debugging issues, focus on fixing the existing implementation, not replacing it
|
||||
@@ -13,11 +16,12 @@ Don’t build things I am not specifically asking for unless its required to bui
|
||||
When asked to make unit tests avoid mocks and use the actual thing that is being tested when reasonable.
|
||||
For commands that read such as grep or cat there is no need to ask for permission.
|
||||
|
||||
## IDE related
|
||||
## IDE related
|
||||
|
||||
Always use ide diagnostics to validate code changes when running with ide integration (goland, vscode)
|
||||
|
||||
|
||||
## linting all edits after every change
|
||||
|
||||
Always run `task lint` once you think you are done with edits to verify your changes have not caused other issues.
|
||||
When running `task lint` output should always be clean. 0 errors.
|
||||
Warnings about variable not set are ok.
|
||||
@@ -28,12 +32,14 @@ Use `task go:lint -- --fix` to fix go (file not formmatted) types of errors from
|
||||
## Essential Commands
|
||||
|
||||
### Development Setup
|
||||
- `devbox shell` - Enter development environment (Nix-based) I will start you already in the devbox shell for the project if there is one.
|
||||
|
||||
- `devbox shell` - Enter development environment (Nix-based) I will start you already in the devbox shell for the project if there is one.
|
||||
- `touch .env` - Create environment variables file
|
||||
- `task fullsuite:ci` - Complete development workflow (generate, build, lint, test) This must pass clean or its not working. If its output has the word `FAIL` then investigate what the issue is before calling the change complete.
|
||||
- when running fullsuite:ci make sure that you have stopped the containers that run from a previous `task compose:refresh' if this applies. (use task compose:down to clean those up first)
|
||||
- `task fullsuite:ci` - Complete development workflow (generate, build, lint, test) This must pass clean or its not working. If its output has the word `FAIL` then investigate what the issue is before calling the change complete.
|
||||
- when running fullsuite:ci make sure that you have stopped the containers that run from a previous `task compose:refresh' if this applies. (use task compose:down to clean those up first)
|
||||
|
||||
### Core Development Tasks
|
||||
|
||||
- `task generate` - Generate all code (DB queries via SQLC, OpenAPI clients/servers, mocks, docs)
|
||||
- `task build` - Build Docker image
|
||||
- `task lint` - Run all linting (Go, YAML, JSON, Docker, OpenAPI)
|
||||
@@ -41,9 +47,8 @@ Use `task go:lint -- --fix` to fix go (file not formmatted) types of errors from
|
||||
- `task test:functional` - Run full test suite with 80% coverage threshold
|
||||
- `task precommit` - Pre-commit checks for CI/CD readiness
|
||||
|
||||
|
||||
|
||||
### Testing Commands
|
||||
|
||||
- `task test:unit:short` - Quick unit tests
|
||||
- `task test:race` - Race condition detection
|
||||
- `task test:perf` - Performance analysis with slowest test reporting
|
||||
@@ -51,11 +56,12 @@ Use `task go:lint -- --fix` to fix go (file not formmatted) types of errors from
|
||||
- `task test:bench` - Benchmark execution
|
||||
- full suite of tests is `task fullsuit:ci` and it must have this statement at the end of its output 'All coverage checks passed!'
|
||||
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### System Design
|
||||
|
||||
This is a **document processing and query orchestration platform** with microservices architecture using:
|
||||
|
||||
- **Event-driven processing** via AWS SQS queues
|
||||
- **RESTful API** (queryAPI on port 8080) for external interactions
|
||||
- **Runner services** (ports 8081-8090) as queue consumers
|
||||
@@ -63,6 +69,7 @@ This is a **document processing and query orchestration platform** with microser
|
||||
- **AWS services** (S3, SQS, Textract, Cognito) for cloud functionality
|
||||
|
||||
### Document Processing Pipeline
|
||||
|
||||
1. **Upload** → S3 storage → storeEventRunner
|
||||
2. **Init** → Validation/deduplication → docInitRunner
|
||||
3. **Sync** → Client eligibility → docSyncRunner
|
||||
@@ -72,6 +79,7 @@ This is a **document processing and query orchestration platform** with microser
|
||||
7. **Export** → Result generation
|
||||
|
||||
### Core Entities
|
||||
|
||||
- **Client**: Customer configurations with document sets and permissions
|
||||
- **Collector**: Query collection specifications per client
|
||||
- **Query**: Data processing blocks with dependency chains
|
||||
@@ -79,6 +87,7 @@ This is a **document processing and query orchestration platform** with microser
|
||||
- **Export**: Manual result output processes
|
||||
|
||||
### Service Architecture
|
||||
|
||||
- **queryAPI** (8080): Main REST API with OpenAPI-generated handlers
|
||||
- **Runners** (8081-8090): Queue-based processing services implementing `Controller` interface
|
||||
- Each runner consumes from specific SQS queues
|
||||
@@ -88,20 +97,25 @@ This is a **document processing and query orchestration platform** with microser
|
||||
## Development Patterns
|
||||
|
||||
### Code Generation
|
||||
|
||||
All code generation happens via `task generate`:
|
||||
|
||||
- **SQLC**: Database queries from `internal/database/queries/*.sql`
|
||||
- **OpenAPI**: API clients/servers from `serviceAPIs/*.yaml`
|
||||
- **Mockery**: Test mocks for interfaces
|
||||
- **gomarkdoc**: Documentation generation
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
- **Testcontainers**: Docker-based integration testing for database/queue interactions
|
||||
- **80% coverage threshold** (60% function coverage) enforced
|
||||
- **Private test files**: Use `*private_test.go` for internal testing
|
||||
- **Mock interfaces**: Auto-generated in `mocks/` directory
|
||||
|
||||
### Project Structure
|
||||
|
||||
Follows Go Standard Project Layout:
|
||||
|
||||
- `cmd/`: Application entry points (main.go files)
|
||||
- `internal/`: Private application code
|
||||
- `api/`: Generated API server implementations
|
||||
@@ -109,18 +123,22 @@ Follows Go Standard Project Layout:
|
||||
- `mocks/`: Auto-generated test mocks
|
||||
|
||||
### Naming Conventions
|
||||
|
||||
- **Service**: Commands deployed as APIs (e.g., queryAPI)
|
||||
- **Runner**: Commands deployed as queue consumers (e.g., docTextRunner)
|
||||
- **API naming**: `<functionality>API` format
|
||||
- **Runner naming**: `<functionality>Runner` format
|
||||
|
||||
### Environment Configuration
|
||||
|
||||
Environment variable hierarchy (first overrides subsequent):
|
||||
|
||||
1. `devbox.json` "env" section
|
||||
2. `devbox.json` "init_hook" exports
|
||||
3. `.env` file variables
|
||||
|
||||
### Authentication & Authorization
|
||||
|
||||
- **AWS Cognito**: JWT-based authentication
|
||||
- **Permit.io**: RBAC integration for fine-grained permissions
|
||||
- Middleware handles token validation and permission checking
|
||||
@@ -128,6 +146,7 @@ Environment variable hierarchy (first overrides subsequent):
|
||||
## Key Dependencies
|
||||
|
||||
### Core Technologies
|
||||
|
||||
- **Go 1.24**: Primary language
|
||||
- **PostgreSQL**: Database with pgx driver
|
||||
- **Echo v4**: HTTP framework with middleware
|
||||
@@ -135,6 +154,7 @@ Environment variable hierarchy (first overrides subsequent):
|
||||
- **AWS SDK v2**: Cloud service integration
|
||||
|
||||
### Code Quality Tools
|
||||
|
||||
- **golangci-lint**: Go linting
|
||||
- **yamllint**: YAML validation
|
||||
- **Task**: Build automation
|
||||
@@ -143,18 +163,21 @@ Environment variable hierarchy (first overrides subsequent):
|
||||
## Quick Start for New Features
|
||||
|
||||
### Adding New API Endpoint
|
||||
|
||||
1. Modify `serviceAPIs/queryAPI.yaml` OpenAPI specification
|
||||
2. Run `task generate` to regenerate API code
|
||||
3. Implement handlers in `api/queryAPI/`
|
||||
4. Add tests following existing patterns
|
||||
|
||||
### Adding New Queue Consumer
|
||||
|
||||
1. Create `api/<runnerName>/runner.go` implementing `Controller` interface
|
||||
2. Create `cmd/<runnerName>/main.go` entry point
|
||||
3. Add queue configuration in `internal/serviceconfig/queue/`
|
||||
4. Add to docker-compose and deployment configurations
|
||||
|
||||
### Database Changes
|
||||
|
||||
1. Add migration files in `internal/database/migrations/`
|
||||
2. Add queries in `internal/database/queries/`
|
||||
3. Run `task db:generate` to regenerate Go code
|
||||
|
||||
Reference in New Issue
Block a user