query-orchestration/docs/ai.generated/06-code-organization.md

Code Organization

Project Structure

The project follows Go Standard Project Layout with enhanced organization for batch processing, observability, and background task management.

The codebase follows the Standard Go Project Layout with additional conventions specific to the domain requirements.

Directory Overview

query-orchestration.2/
├── cmd/                    # Main applications and entry points
├── internal/               # Private application code  
├── api/                    # Generated API server implementations
├── pkg/                    # Public library code
├── deployments/            # Deployment configurations
├── docs/                   # Documentation
├── scripts/                # Build and automation scripts
├── test/                   # Integration and e2e tests
├── assets/                 # Test data and sample files
└── vendor/                 # Vendored dependencies

Package Architecture

Command Layer (cmd/)

Entry points for all deployable services, following the pattern:

cmd/
├── [serviceName]/
│   └── main.go            # Service bootstrap and configuration

Naming Conventions:

• APIs: [functionality]API (e.g., queryAPI)
• Runners: [functionality]Runner (e.g., docInitRunner)
• Utilities: Descriptive names (e.g., healthcheck)

Internal Packages (internal/)

Core business logic and infrastructure components, organized by domain:

New Components (Since June 2025):

• internal/backgroundtask/: Background task execution framework with periodic scheduling
• internal/document/batch/: Batch upload processing and ZIP archive handling
• internal/serviceconfig/observability/prometheus/: Comprehensive Prometheus metrics framework

Internal Packages (internal/)

Core business logic organized by domain boundaries:

Domain Services

internal/
├── client/                # Client management domain
├── document/              # Document processing domain
├── fieldextraction/       # Field extraction domain
├── collector/             # Client-specific configurations
└── export/                # Data export operations

Infrastructure Services

internal/
├── database/              # Data persistence layer
├── server/                # HTTP and queue server infrastructure
├── serviceconfig/         # Configuration management
├── cognitoauth/           # Authentication middleware (includes environment access control)
├── usermanagement/        # Cognito and Permit.io user operations, environment filtering
├── eula/                  # EULA version management and compliance
├── test/                  # Testing utilities
└── validation/            # Input validation

API Layer (api/)

Generated server implementations from OpenAPI specifications:

api/
├── queryAPI/              # REST API implementation
│   ├── api.gen.go         # Generated types and interfaces
│   ├── controllers.go     # Business logic implementation
│   ├── authHandlers.go    # Authentication endpoints
│   └── *.go              # Domain-specific handlers
└── [runnerName]/          # Queue-based service implementations
    └── runner.go          # Queue message processing

Design Patterns and Principles

Service Layer Pattern

Each domain follows a consistent service layer structure:

// Service struct with injected dependencies
type Service struct {
    cfg ConfigProvider  // Configuration interface
}

// Constructor with interface-based dependency injection
func New(cfg ConfigProvider) *Service {
    return &Service{cfg: cfg}
}

// Business operations
func (s *Service) CreateEntity(ctx context.Context, req CreateRequest) (*Entity, error) {
    // Business logic implementation
}

Repository Pattern

Database access is abstracted through repositories:

// Generated by SQLC from SQL queries
type Repository interface {
    CreateClient(ctx context.Context, arg CreateClientParams) (Client, error)
    GetClient(ctx context.Context, clientID string) (Client, error)
    // ... other operations
}

// Service uses repository interface
type Service struct {
    repo Repository
}

Configuration Provider Pattern

Services receive configuration through composed interfaces:

// Domain-specific configuration interface
type ConfigProvider interface {
    serviceconfig.ConfigProvider      // Base configuration
    database.ConfigProvider          // Database access
    objectstore.ConfigProvider       // S3 access
    // ... other provider interfaces
}

// Concrete configuration embeds all providers
type Config struct {
    serviceconfig.BaseConfig
    database.Config
    objectstore.Config
}

Dependency Injection

Services are composed through constructor injection:

// In main.go
cfg := &Config{
    BaseConfig:     baseConfig,
    DatabaseConfig: dbConfig,
    // ... other configs
}

// Create service with injected dependencies
svc := service.New(cfg)

// Create controller with service dependency
controller := api.NewController(svc)

Naming Conventions

Package Names

• Domain packages: Singular nouns (client, document, fieldextraction)
• Utility packages: Descriptive (serviceconfig, cognitoauth)
• Generated packages: Match source (queryAPI from queryAPI.yaml)

File Names

• Service files: service.go (main business logic)
• Test files: *_test.go (alongside source)
• Private tests: *private_test.go (package-private testing)
• Interface files: Match domain (client.go, document.go)

Function and Method Names

• Public functions: PascalCase (CreateClient, GetDocument)
• Private functions: camelCase (validateInput, processResult)
• Constructors: New or New[Type] (New, NewService)
• Getters: No Get prefix for simple accessors (ID(), not GetID())

Variable Names

• Short scope: Single letters (i, j, c for client)
• Medium scope: Abbreviations (ctx, req, resp)
• Long scope: Descriptive names (clientRepository, configProvider)
• Constants: PascalCase (MaxRetryAttempts, DefaultTimeout)

Type Names

• Structs: PascalCase nouns (Client, DocumentProcessor)
• Interfaces: PascalCase with optional "er" suffix (ConfigProvider, Processor)
• Enums: PascalCase with type suffix (QueryType, ProcessingStatus)

Code Style Guidelines

Import Organization

package main

import (
    // Standard library
    "context"
    "fmt"
    
    // Third-party packages  
    "github.com/labstack/echo/v4"
    "github.com/google/uuid"
    
    // Local packages
    "queryorchestration/internal/client"
    "queryorchestration/internal/database"
)

Error Handling

// Wrap errors with context
func (s *Service) ProcessDocument(ctx context.Context, id uuid.UUID) error {
    doc, err := s.repo.GetDocument(ctx, id)
    if err != nil {
        return fmt.Errorf("failed to get document %s: %w", id, err)
    }
    
    if err := s.processContent(doc); err != nil {
        return fmt.Errorf("failed to process document content: %w", err)
    }
    
    return nil
}

Interface Design

// Prefer small, focused interfaces
type ClientReader interface {
    GetClient(ctx context.Context, id string) (*Client, error)
}

type ClientWriter interface {
    CreateClient(ctx context.Context, client *Client) error
    UpdateClient(ctx context.Context, client *Client) error
}

// Compose when needed
type ClientRepository interface {
    ClientReader
    ClientWriter
}

Struct Design

// Group related fields
type Client struct {
    // Identity
    ID   string
    Name string
    
    // Configuration
    CanSync bool
    
    // Metadata
    CreatedAt time.Time
    UpdatedAt time.Time
}

// Use embedding for composition
type APIClient struct {
    Client
    AuthToken string
}

Testing Patterns

Table-Driven Tests

func TestService_CreateClient(t *testing.T) {
    tests := []struct {
        name    string
        input   CreateClientRequest
        want    *Client
        wantErr bool
    }{
        {
            name: "valid client",
            input: CreateClientRequest{
                ID:   "test-client",
                Name: "Test Client",
            },
            want: &Client{
                ID:   "test-client", 
                Name: "Test Client",
            },
            wantErr: false,
        },
        {
            name: "empty ID",
            input: CreateClientRequest{
                Name: "Test Client",
            },
            want:    nil,
            wantErr: true,
        },
    }
    
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            // Test implementation
        })
    }
}

Mock Usage

// Generate mocks for interfaces
//go:generate mockery --name=ClientRepository --output=mocks

func TestService_WithMocks(t *testing.T) {
    // Create mock
    mockRepo := &mocks.ClientRepository{}
    
    // Set expectations
    mockRepo.On("GetClient", mock.Anything, "test-id").
        Return(&Client{ID: "test-id"}, nil)
    
    // Create service with mock
    svc := &Service{repo: mockRepo}
    
    // Test and verify
    result, err := svc.GetClient(context.Background(), "test-id")
    assert.NoError(t, err)
    assert.Equal(t, "test-id", result.ID)
    
    mockRepo.AssertExpectations(t)
}

Integration Test Structure

func TestIntegration_DocumentProcessing(t *testing.T) {
    // Setup test environment
    ctx := context.Background()
    testDB := test.NewDatabase(t)
    testS3 := test.NewObjectStore(t)
    
    // Create service with real dependencies
    cfg := &Config{
        Database:    testDB,
        ObjectStore: testS3,
    }
    svc := New(cfg)
    
    // Run test scenario
    client := test.CreateTestClient(t, svc)
    document := test.UploadTestDocument(t, svc, client.ID)
    
    // Verify results
    result := test.ProcessDocument(t, svc, document.ID)
    assert.NotNil(t, result)
    
    // Cleanup handled by test utilities
}

Architecture Decisions

Configuration Management

• Environment-first: All configuration from environment variables
• Type safety: Strong typing with validation
• Composition: Interface-based configuration providers
• Testing: Override-friendly for test scenarios

Error Handling

• Wrap errors: Add context at each layer
• Structured logging: Use slog for consistent error logging
• Graceful degradation: Continue operation when possible
• Client-friendly: Translate internal errors to appropriate HTTP status

Database Access

• Type safety: SQLC generates type-safe database code
• Migrations: Version-controlled schema evolution
• Connection pooling: Shared connections across services
• Transaction management: Explicit transaction boundaries

API Design

• Contract-first: OpenAPI specification drives implementation
• Version compatibility: Backward-compatible changes
• Validation: Automatic request/response validation
• Documentation: Auto-generated from specification

This organization enables maintainable, testable, and scalable code while following Go best practices and domain-driven design principles.