query-orchestration/testing.md

Testing Documentation

This document describes all available test targets in the Task build system and their specific purposes.

Primary Test Suites

task test:functional

Default comprehensive test suite with intelligent incremental testing

• Purpose: Main test runner that adapts behavior based on context
• Unique Features:
  • Incremental Mode (default): Only tests packages affected by changes since origin/main
  • Coverage Merging: Maintains cumulative coverage across incremental runs
  • Full Fallback: Automatically runs full suite if no baseline exists or git branch issues
• Coverage: 80% total threshold, 60% function threshold
• Parallelism: Minimal settings to avoid resource contention (GOMAXPROCS=2, -p 1, -parallel 2)
• Use Case: Development workflow, pre-commit checks

task test:functional:full

Complete test suite with coverage reset

• Purpose: Run all tests from scratch, ignoring previous coverage
• Unique Features:
  • Deletes existing coverage file before running
  • Forces full test execution regardless of git changes
• Use Case: Clean slate testing, debugging coverage issues, release validation

task fullsuite:ci

CI/CD optimized complete workflow

• Purpose: Full build, lint, and test pipeline for continuous integration
• Unique Features:
  • Forces INCREMENTAL=false to ensure all tests run
  • Includes generation, building, linting, and testing
  • No shortcuts or incremental optimizations
• Use Case: CI/CD pipelines, release builds, comprehensive validation

Specialized Test Types

task test:race

Race condition detection

• Purpose: Detect data races and concurrency issues
• Unique Features: Uses Go's -race flag to detect race conditions
• Use Case: Debugging concurrency issues, ensuring thread safety

task test:unit:short

Fast unit tests only

• Purpose: Quick feedback loop for development
• Unique Features:
  • Uses -short flag to skip long-running tests
  • Focuses on ./internal/... and ./api/... packages only
  • Generates mocks automatically before running
• Use Case: Rapid development feedback, pre-commit quick checks

task test:permitio

Permit.io integration tests

• Purpose: Run tests that require local Permit.io PDP service
• Unique Features:
  • Uses -tags=permitio to include only tagged integration tests
  • Requires PERMIT_IO_API_KEY environment variable
  • Requires local PDP running on port 7766
  • Provides clear error messages about requirements
• Use Case: Testing authorization integration, local development with full Permit.io stack

task test:bench

Benchmark testing

• Purpose: Performance testing and optimization
• Unique Features:
  • Runs only benchmark functions (-run ^Benchmark)
  • Uses benchstat for statistical analysis of results
  • Outputs performance metrics to out/bench.out
• Use Case: Performance optimization, regression testing for speed

Performance Analysis

task test:perf

Performance profiling and slowest test identification

• Purpose: Identify performance bottlenecks in the test suite
• Unique Features:
  • Reports top 5 slowest packages
  • Reports top 5 slowest individual tests
  • Deep-dives into slowest package for detailed analysis
  • Uses JSON output for precise timing analysis
• Use Case: Test suite optimization, CI performance tuning

task test:mem

Memory usage analysis

• Purpose: Monitor memory consumption across test packages
• Unique Features:
  • Runs tests with memory profiling for each package
  • Ranks packages by memory usage (top 5)
  • Provides total memory consumption summary
  • Individual timeout protection (120s per package)
• Use Case: Memory optimization, resource planning, leak detection

task test:timeline

Visual test execution timeline

• Purpose: Visualize test execution flow and timing
• Unique Features:
  • Uses JSON output piped to visual timeline generator
  • Shows parallel execution patterns
  • Helps identify bottlenecks and optimization opportunities
• Use Case: Understanding test execution patterns, optimization planning

Test Infrastructure

task test:mocks:generate

Mock generation for testing

• Purpose: Generate test mocks from interfaces
• Unique Features:
  • Uses mockery tool to auto-generate mocks
  • Clears existing mocks before regeneration
  • Runs only once per task execution cycle
• Use Case: Maintaining test isolation, dependency mocking

task test:wait

Test environment preparation

• Purpose: Ensure clean test environment before running tests
• Unique Features:
  • Waits for testcontainers cleanup (Ryuk containers)
  • Prunes Docker containers and volumes
  • Prevents resource conflicts between test runs
• Use Case: Test environment hygiene, CI stability

task test:prune

Docker resource cleanup

• Purpose: Clean up Docker resources left by tests
• Unique Features:
  • Force removes unused containers and volumes
  • Prevents disk space issues from accumulated test artifacts
• Use Case: Resource management, CI environment maintenance

Visualization and Analysis

task test:graph

Resource usage visualization during testing

• Purpose: Monitor system resources during test execution
• Unique Features:
  • Uses psrecord to track CPU/memory during test:functional
  • Generates visual plots of resource usage
  • Includes child processes in monitoring
• Use Case: Performance analysis, resource optimization

task fullsuite:graph

Complete build and test resource monitoring

• Purpose: Monitor resources during entire build and test cycle
• Unique Features: Same as test:graph but for the complete fullsuite workflow
• Use Case: End-to-end performance analysis, CI optimization

Test Execution Patterns

Incremental Testing Logic

The test:functional target implements sophisticated incremental testing:

1. Full Mode Triggers:

   • INCREMENTAL=false explicitly set
   • No existing coverage file found
   • Cannot find origin/main branch reference

2. Incremental Mode:

   • Compares against origin/main branch
   • Only tests packages with changed Go files
   • Merges coverage with existing baseline
   • Skips testing if no Go files changed

3. Coverage Merging:

   • Preserves existing coverage for unchanged packages
   • Adds new coverage for changed packages
   • Maintains comprehensive coverage metrics

Parallelism Settings

All test targets use conservative parallelism to avoid resource contention:

• GOMAXPROCS: 2 (fixed minimal value)
• Package Parallelism: 1 (single package at a time)
• Test Parallelism: 2 (minimal test parallelism)

This ensures stable execution in resource-constrained environments like CI.

Usage Recommendations

• Development: Use task test:functional for normal development workflow
• Pre-commit: Use task test:unit:short for quick feedback
• CI/CD: Use task fullsuite:ci for comprehensive validation
• Performance: Use task test:perf and task test:mem for optimization
• Debugging: Use task test:race for concurrency issues
• Clean Slate: Use task test:functional:full when coverage gets corrupted
• Permit.io Integration: Use task test:permitio for testing authorization features with local PDP

Integration with Build Tags

Some tests may use build tags to control execution:

• Tests requiring external services (like Permit.io PDP) may be tagged
• Use -tags=tagname with go test commands to include tagged tests
• CI typically excludes integration tests that require external dependencies

Test User Header Injection (DISABLE_AUTH Mode)

When running the service with DISABLE_AUTH=true, a special middleware (TestUserMiddleware) activates that allows injecting user identity via HTTP headers. This enables testing of user-authenticated endpoints without requiring real JWT authentication.

How It Works

The TestUserMiddleware (located in internal/cognitoauth/test_middleware.go):

1. Only activates when DISABLE_AUTH=true environment variable is set
2. Checks for the X-Test-User-Subject header on each request
3. If present, injects user identity into the request context
4. Handlers can then retrieve user info via GetUserSubject() and GetUserInfo()

Headers

Header	Required	Description
X-Test-User-Subject	Yes	The user's subject ID (Cognito sub claim)
X-Test-User-Email	No	The user's email (defaults to <subject>@test.local)

Example Usage

# Start the service with DISABLE_AUTH=true (default for local compose)
task compose:refresh

# Call user-authenticated endpoint with test user headers
curl -X GET http://localhost:8080/eula/status \
  -H "X-Test-User-Subject: test-user-123" \
  -H "X-Test-User-Email: testuser@example.com"

# User agrees to EULA
curl -X POST http://localhost:8080/eula/agree \
  -H "X-Test-User-Subject: test-user-123" \
  -H "X-Test-User-Email: testuser@example.com"

Integration Test Scripts

Three integration test scripts are available for EULA functionality:

1. Complete EULA Tests (test/text_eula_all_integration_test.sh) (Recommended)

   • Comprehensive test suite combining admin and user EULA testing
   • Tests full EULA lifecycle: version management, user agreements, version upgrades, compliance
   • 24 test steps across 9 phases
   • Uses both admin endpoints (system user) and user endpoints (header injection)

2. Admin EULA Tests (test/text_eula_admin_integration_test.sh)

   • Tests admin-only endpoints: create, activate, update, list EULA versions
   • Uses system user (no headers needed with DISABLE_AUTH=true)

3. User EULA Tests (test/text_eula_user_integration_test.sh)

   • Tests user-authenticated endpoints: /eula/status, /eula/agree
   • Uses X-Test-User-Subject header to inject test user identity
   • Tests multiple users and idempotency

# Run complete EULA integration tests (recommended)
./test/text_eula_all_integration_test.sh http://localhost:8080

# Run individual test suites
./test/text_eula_admin_integration_test.sh http://localhost:8080
./test/text_eula_user_integration_test.sh http://localhost:8080

Security Note

The TestUserMiddleware is only active when DISABLE_AUTH=true. In production environments where DISABLE_AUTH is not set or is false, this middleware does nothing and the headers are ignored. The real authentication middleware handles all requests normally.