Merged in feature/ai-generated-docs (pull request #167)

ai generated docs and related tools for publishing new versions * ai generated docs and tooling * tools * add mermaid links * add task docs:ai:generate * Merged main into feature/ai-generated-docs Approved-by: Michael McGuinness
2025-06-24 19:56:56 +00:00
parent 079aec770d
commit 45b5f2fd50
16 changed files with 3717 additions and 0 deletions
@@ -0,0 +1,389 @@
+# Code Organization
+
+## Project Structure
+
+The codebase follows the [Standard Go Project Layout](https://github.com/golang-standards/project-layout) with additional conventions specific to the domain requirements.
+
+### Directory Overview
+
+```
+query-orchestration.2/
+├── cmd/                    # Main applications and entry points
+├── internal/               # Private application code  
+├── api/                    # Generated API server implementations
+├── pkg/                    # Public library code
+├── deployments/            # Deployment configurations
+├── docs/                   # Documentation
+├── scripts/                # Build and automation scripts
+├── test/                   # Integration and e2e tests
+├── mocks/                  # Generated mock implementations
+├── assets/                 # Test data and sample files
+└── vendor/                 # Vendored dependencies
+```
+
+## Package Architecture
+
+### Command Layer (`cmd/`)
+
+Entry points for all deployable services, following the pattern:
+```
+cmd/
+├── [serviceName]/
+│   └── main.go            # Service bootstrap and configuration
+```
+
+**Naming Conventions**:
+- **APIs**: `[functionality]API` (e.g., `queryAPI`)
+- **Runners**: `[functionality]Runner` (e.g., `docInitRunner`)
+- **Utilities**: Descriptive names (e.g., `healthcheck`)
+
+### Internal Packages (`internal/`)
+
+Core business logic organized by domain boundaries:
+
+#### Domain Services
+```
+internal/
+├── client/                # Client management domain
+├── document/              # Document processing domain  
+├── query/                 # Query definition and execution
+├── collector/             # Client-specific configurations
+└── export/                # Data export operations
+```
+
+#### Infrastructure Services  
+```
+internal/
+├── database/              # Data persistence layer
+├── server/                # HTTP and queue server infrastructure
+├── serviceconfig/         # Configuration management
+├── cognitoauth/          # Authentication middleware
+├── test/                  # Testing utilities
+└── validation/            # Input validation
+```
+
+### API Layer (`api/`)
+
+Generated server implementations from OpenAPI specifications:
+```
+api/
+├── queryAPI/              # REST API implementation
+│   ├── api.gen.go         # Generated types and interfaces
+│   ├── controllers.go     # Business logic implementation
+│   ├── authHandlers.go    # Authentication endpoints
+│   └── *.go              # Domain-specific handlers
+└── [runnerName]/          # Queue-based service implementations
+    └── runner.go          # Queue message processing
+```
+
+## Design Patterns and Principles
+
+### Service Layer Pattern
+
+Each domain follows a consistent service layer structure:
+
+```go
+// Service struct with injected dependencies
+type Service struct {
+    cfg ConfigProvider  // Configuration interface
+}
+
+// Constructor with interface-based dependency injection
+func New(cfg ConfigProvider) *Service {
+    return &Service{cfg: cfg}
+}
+
+// Business operations
+func (s *Service) CreateEntity(ctx context.Context, req CreateRequest) (*Entity, error) {
+    // Business logic implementation
+}
+```
+
+### Repository Pattern
+
+Database access is abstracted through repositories:
+
+```go
+// Generated by SQLC from SQL queries
+type Repository interface {
+    CreateClient(ctx context.Context, arg CreateClientParams) (Client, error)
+    GetClient(ctx context.Context, clientID string) (Client, error)
+    // ... other operations
+}
+
+// Service uses repository interface
+type Service struct {
+    repo Repository
+}
+```
+
+### Configuration Provider Pattern
+
+Services receive configuration through composed interfaces:
+
+```go
+// Domain-specific configuration interface
+type ConfigProvider interface {
+    serviceconfig.ConfigProvider      // Base configuration
+    database.ConfigProvider          // Database access
+    objectstore.ConfigProvider       // S3 access
+    // ... other provider interfaces
+}
+
+// Concrete configuration embeds all providers
+type Config struct {
+    serviceconfig.BaseConfig
+    database.Config
+    objectstore.Config
+}
+```
+
+### Dependency Injection
+
+Services are composed through constructor injection:
+
+```go
+// In main.go
+cfg := &Config{
+    BaseConfig:     baseConfig,
+    DatabaseConfig: dbConfig,
+    // ... other configs
+}
+
+// Create service with injected dependencies
+svc := service.New(cfg)
+
+// Create controller with service dependency
+controller := api.NewController(svc)
+```
+
+## Naming Conventions
+
+### Package Names
+- **Domain packages**: Singular nouns (`client`, `document`, `query`)
+- **Utility packages**: Descriptive (`serviceconfig`, `cognitoauth`)
+- **Generated packages**: Match source (`queryAPI` from `queryAPI.yaml`)
+
+### File Names
+- **Service files**: `service.go` (main business logic)
+- **Test files**: `*_test.go` (alongside source)
+- **Private tests**: `*private_test.go` (package-private testing)
+- **Interface files**: Match domain (`client.go`, `document.go`)
+
+### Function and Method Names
+- **Public functions**: PascalCase (`CreateClient`, `GetDocument`)
+- **Private functions**: camelCase (`validateInput`, `processResult`)
+- **Constructors**: `New` or `New[Type]` (`New`, `NewService`)
+- **Getters**: No `Get` prefix for simple accessors (`ID()`, not `GetID()`)
+
+### Variable Names
+- **Short scope**: Single letters (`i`, `j`, `c` for client)
+- **Medium scope**: Abbreviations (`ctx`, `req`, `resp`)
+- **Long scope**: Descriptive names (`clientRepository`, `configProvider`)
+- **Constants**: PascalCase (`MaxRetryAttempts`, `DefaultTimeout`)
+
+### Type Names
+- **Structs**: PascalCase nouns (`Client`, `DocumentProcessor`)
+- **Interfaces**: PascalCase with optional "er" suffix (`ConfigProvider`, `Processor`)
+- **Enums**: PascalCase with type suffix (`QueryType`, `ProcessingStatus`)
+
+## Code Style Guidelines
+
+### Import Organization
+```go
+package main
+
+import (
+    // Standard library
+    "context"
+    "fmt"
+    
+    // Third-party packages  
+    "github.com/labstack/echo/v4"
+    "github.com/google/uuid"
+    
+    // Local packages
+    "queryorchestration/internal/client"
+    "queryorchestration/internal/database"
+)
+```
+
+### Error Handling
+```go
+// Wrap errors with context
+func (s *Service) ProcessDocument(ctx context.Context, id uuid.UUID) error {
+    doc, err := s.repo.GetDocument(ctx, id)
+    if err != nil {
+        return fmt.Errorf("failed to get document %s: %w", id, err)
+    }
+    
+    if err := s.processContent(doc); err != nil {
+        return fmt.Errorf("failed to process document content: %w", err)
+    }
+    
+    return nil
+}
+```
+
+### Interface Design
+```go
+// Prefer small, focused interfaces
+type ClientReader interface {
+    GetClient(ctx context.Context, id string) (*Client, error)
+}
+
+type ClientWriter interface {
+    CreateClient(ctx context.Context, client *Client) error
+    UpdateClient(ctx context.Context, client *Client) error
+}
+
+// Compose when needed
+type ClientRepository interface {
+    ClientReader
+    ClientWriter
+}
+```
+
+### Struct Design
+```go
+// Group related fields
+type Client struct {
+    // Identity
+    ID   string
+    Name string
+    
+    // Configuration
+    CanSync bool
+    
+    // Metadata
+    CreatedAt time.Time
+    UpdatedAt time.Time
+}
+
+// Use embedding for composition
+type APIClient struct {
+    Client
+    AuthToken string
+}
+```
+
+## Testing Patterns
+
+### Table-Driven Tests
+```go
+func TestService_CreateClient(t *testing.T) {
+    tests := []struct {
+        name    string
+        input   CreateClientRequest
+        want    *Client
+        wantErr bool
+    }{
+        {
+            name: "valid client",
+            input: CreateClientRequest{
+                ID:   "test-client",
+                Name: "Test Client",
+            },
+            want: &Client{
+                ID:   "test-client", 
+                Name: "Test Client",
+            },
+            wantErr: false,
+        },
+        {
+            name: "empty ID",
+            input: CreateClientRequest{
+                Name: "Test Client",
+            },
+            want:    nil,
+            wantErr: true,
+        },
+    }
+    
+    for _, tt := range tests {
+        t.Run(tt.name, func(t *testing.T) {
+            // Test implementation
+        })
+    }
+}
+```
+
+### Mock Usage
+```go
+// Generate mocks for interfaces
+//go:generate mockery --name=ClientRepository --output=mocks
+
+func TestService_WithMocks(t *testing.T) {
+    // Create mock
+    mockRepo := &mocks.ClientRepository{}
+    
+    // Set expectations
+    mockRepo.On("GetClient", mock.Anything, "test-id").
+        Return(&Client{ID: "test-id"}, nil)
+    
+    // Create service with mock
+    svc := &Service{repo: mockRepo}
+    
+    // Test and verify
+    result, err := svc.GetClient(context.Background(), "test-id")
+    assert.NoError(t, err)
+    assert.Equal(t, "test-id", result.ID)
+    
+    mockRepo.AssertExpectations(t)
+}
+```
+
+### Integration Test Structure
+```go
+func TestIntegration_DocumentProcessing(t *testing.T) {
+    // Setup test environment
+    ctx := context.Background()
+    testDB := test.NewDatabase(t)
+    testS3 := test.NewObjectStore(t)
+    
+    // Create service with real dependencies
+    cfg := &Config{
+        Database:    testDB,
+        ObjectStore: testS3,
+    }
+    svc := New(cfg)
+    
+    // Run test scenario
+    client := test.CreateTestClient(t, svc)
+    document := test.UploadTestDocument(t, svc, client.ID)
+    
+    // Verify results
+    result := test.ProcessDocument(t, svc, document.ID)
+    assert.NotNil(t, result)
+    
+    // Cleanup handled by test utilities
+}
+```
+
+## Architecture Decisions
+
+### Configuration Management
+- **Environment-first**: All configuration from environment variables
+- **Type safety**: Strong typing with validation
+- **Composition**: Interface-based configuration providers
+- **Testing**: Override-friendly for test scenarios
+
+### Error Handling
+- **Wrap errors**: Add context at each layer
+- **Structured logging**: Use slog for consistent error logging
+- **Graceful degradation**: Continue operation when possible
+- **Client-friendly**: Translate internal errors to appropriate HTTP status
+
+### Database Access
+- **Type safety**: SQLC generates type-safe database code
+- **Migrations**: Version-controlled schema evolution
+- **Connection pooling**: Shared connections across services
+- **Transaction management**: Explicit transaction boundaries
+
+### API Design
+- **Contract-first**: OpenAPI specification drives implementation
+- **Version compatibility**: Backward-compatible changes
+- **Validation**: Automatic request/response validation
+- **Documentation**: Auto-generated from specification
+
+This organization enables maintainable, testable, and scalable code while following Go best practices and domain-driven design principles.