# QueryAPI Software Architecture Guide

This document provides a comprehensive guide to the QueryAPI software architecture for new engineers joining the project. It explains the purpose of each architectural layer, how components interact, and what logic belongs where when adding new features.

## Table of Contents

- [Architecture Overview](#architecture-overview)
- [Component Layers](#component-layers)
- [Request Flow](#request-flow)
- [Adding a New API Endpoint](#adding-a-new-api-endpoint)
- [Key Patterns](#key-patterns)
- [Component Diagrams](#component-diagrams)

## Architecture Overview

QueryAPI follows a layered architecture pattern with clear separation of concerns:

```mermaid
flowchart TB
    subgraph External["External Layer"]
        Client[HTTP Client]
        OpenAPI[OpenAPI Spec<br/>serviceAPIs/queryAPI.yaml]
    end

    subgraph Entry["Entry Point"]
        Main[cmd/queryAPI/main.go]
    end

    subgraph Server["Server Layer"]
        Listener[internal/server/api/listener.go]
        Echo[Echo HTTP Router]
    end

    subgraph Middleware["Middleware Layer"]
        RateLimit[Rate Limiting]
        JWT[JWT Auth Middleware]
        TokenValidation[Token Validation]
        PermitIO[Permit.io Authorization]
        OAPIValidation[OpenAPI Request Validation]
    end

    subgraph Generated["Generated Layer"]
        APIGen[api/queryAPI/api.gen.go<br/>ServerInterface + Types]
        RegisterHandlers[RegisterHandlers]
    end

    subgraph Controllers["Controller Layer"]
        Controllers_impl[api/queryAPI/*.go<br/>Controllers struct]
    end

    subgraph Services["Service Layer"]
        ServiceImpl[internal/&lt;domain&gt;/service.go<br/>Business Logic]
    end

    subgraph Data["Data Access Layer"]
        Repository[internal/database/repository/*.sql.go<br/>SQLC Generated]
        Queries[internal/database/queries/*.sql<br/>SQL Definitions]
        Migrations[internal/database/migrations/*.sql<br/>Schema Changes]
    end

    subgraph Infrastructure["Infrastructure"]
        DB[(PostgreSQL)]
        S3[(AWS S3)]
        SQS[(AWS SQS)]
    end

    Client --> Main
    OpenAPI -.->|generates| APIGen
    Main --> Listener
    Listener --> Echo
    Echo --> RateLimit --> JWT --> TokenValidation --> PermitIO --> OAPIValidation
    OAPIValidation --> RegisterHandlers
    RegisterHandlers --> Controllers_impl
    Controllers_impl --> ServiceImpl
    ServiceImpl --> Repository
    Queries -.->|generates| Repository
    Repository --> DB
    ServiceImpl --> S3
    ServiceImpl --> SQS
```

## Component Layers

### 1. Entry Point (`cmd/queryAPI/main.go`)

**Purpose**: Application bootstrap and dependency wiring.

**Responsibilities**:
- Print version information
- Initialize configuration from environment variables
- Create and wire all service dependencies
- Register handlers with the router
- Start the HTTP server

**Key Code Pattern**:
```go
// Create services
cli := client.New(cfg)
doc := document.New(cfg)

// Wire services into Controllers
services := &queryapi.Services{
    Client:   cli,
    Document: doc,
}
cons := queryapi.NewControllers(services, cfg)

// Register routes
queryapi.RegisterHandlers(cfg.Router, cons)
```

**When to modify**: When adding a new domain service that needs to be injected into the API layer.

---

### 2. Server Layer (`internal/server/api/listener.go`)

**Purpose**: HTTP server configuration, middleware setup, and lifecycle management.

**Responsibilities**:
- Create and configure the Echo HTTP router
- Register middleware in the correct order
- Set up health and metrics endpoints
- Initialize background task runners
- Handle graceful shutdown

**Key Components**:
| Component | Purpose |
|-----------|---------|
| `Server` struct | Holds router, port, and cleanup functions |
| `New()` function | Creates server with all middleware configured |
| `Listen()` method | Starts the HTTP server |
| Middleware chain | Rate limiting -> Auth -> Validation |

**Middleware Order** (applied in sequence):
1. Config injection middleware
2. Prometheus metrics
3. Request logging (slog)
4. Recovery (panic handling)
5. CORS
6. Debug logging (when DEBUG=true)
7. Rate limiting
8. Authentication routes registration
9. Test user injection (when DISABLE_AUTH=true)
10. Handler registration
11. OpenAPI request validation

---

### 3. Middleware Layer (`internal/cognitoauth/`)

**Purpose**: Authentication and authorization enforcement.

**Key Files**:
| File | Purpose |
|------|---------|
| `middleware.go` | Main middleware functions (`TokenValidationMiddleware`, `JWTAuthMiddleware`) |
| `handler.go` | OAuth callback and login flow handlers |
| `permitio.go` | Permit.io authorization client |
| `token.go` | JWT token operations (refresh, validation) |
| `validation.go` | Token signature and claims validation |

**Authentication Flow**:
```mermaid
sequenceDiagram
    participant C as Client
    participant M as Middleware
    participant J as JWKS
    participant P as Permit.io

    C->>M: Request with JWT
    M->>M: Check if public path
    alt Public Path
        M->>C: Allow (skip auth)
    else Protected Path
        M->>M: Extract token from header/cookie
        M->>J: Verify token signature
        J-->>M: Token valid
        M->>M: Extract user claims
        M->>P: Check permission(user, action, resource)
        P-->>M: Permitted/Denied
        alt Permitted
            M->>C: Allow request
        else Denied
            M->>C: 403 Forbidden
        end
    end
```

**Path Types**:
- **Public paths**: `/health`, `/swagger/*`, `/metrics`, `/eula` (GET only)
- **Auth-only paths**: `/identity`, `/eula/status`, `/eula/agree` (authenticated but no specific permissions)
- **Protected paths**: All other routes (require both authentication and Permit.io authorization)

---

### 4. Generated API Layer (`api/queryAPI/api.gen.go`)

**Purpose**: Type-safe API interface generated from OpenAPI specification.

**Generated by**: `oapi-codegen` from `serviceAPIs/queryAPI.yaml`

**Key Generated Components**:
| Component | Purpose |
|-----------|---------|
| `ServerInterface` | Interface that controllers must implement |
| Request/Response types | Strongly-typed DTOs (e.g., `ClientCreate`, `DocClient`) |
| `RegisterHandlers()` | Connects routes to handler methods |
| `ServerInterfaceWrapper` | Wraps handlers with parameter extraction |
| Parameter types | Path/query parameter types (e.g., `ClientID`) |

**Example Interface Method**:
```go
// From ServerInterface
CreateClient(ctx echo.Context) error
GetClient(ctx echo.Context, id ClientID) error
```

**Important**: Never edit `api.gen.go` directly. Modify `serviceAPIs/queryAPI.yaml` and run `task generate`.

---

### 5. Controller Layer (`api/queryAPI/*.go`)

**Purpose**: HTTP request handling and response formatting.

**Key Files**:
| File | Domain |
|------|--------|
| `controllers.go` | `Controllers` struct and constructor |
| `client.go` | Client endpoints |
| `documents.go` | Document endpoints |
| `folders.go` | Folder endpoints |
| `labels.go` | Label endpoints |
| `collector.go` | Collector endpoints |
| `export.go` | Export endpoints |
| `authHandlers.go` | Authentication-related endpoints |
| `adminHandlers.go` | User administration endpoints |

**Responsibilities**:
- Bind and validate request bodies
- Call appropriate service methods
- Transform service results to API response types
- Return appropriate HTTP status codes
- Handle errors with proper HTTP error responses

**Controller Pattern**:
```go
func (s *Controllers) GetClient(ctx echo.Context, id ClientID) error {
    // 1. Call service layer
    client, err := s.svc.Client.Get(ctx.Request().Context(), id)
    if err != nil {
        // 2. Handle error
        return echo.NewHTTPError(http.StatusBadRequest,
            fmt.Sprintf("Unable to get client: %s", err))
    }

    // 3. Transform to API type and return
    return ctx.JSON(http.StatusOK, DocClient{
        Id:      client.ID,
        Name:    client.Name,
        CanSync: client.CanSync,
    })
}
```

**What belongs here**:
- Request parsing and binding
- Input validation (beyond OpenAPI validation)
- Calling service methods
- Response transformation
- HTTP status code decisions

**What does NOT belong here**:
- Business logic
- Database operations
- External service calls
- Complex data transformations

---

### 6. Service Layer (`internal/<domain>/`)

**Purpose**: Business logic and domain operations.

**Structure per domain**:
```
internal/client/
├── service.go      # Service struct and constructor
├── create.go       # Create operation
├── get.go          # Read operations
├── list.go         # List operations
├── status.go       # Status checks
└── *_test.go       # Unit tests
```

**Service Pattern**:
```go
// Service struct with config dependency
type Service struct {
    cfg serviceconfig.ConfigProvider
}

// Constructor
func New(cfg serviceconfig.ConfigProvider) *Service {
    return &Service{cfg}
}

// Business operation
func (s *Service) Get(ctx context.Context, id string) (*Client, error) {
    // Access database through config
    client, err := s.cfg.GetDBQueries().GetClient(ctx, id)
    if err != nil {
        return nil, err
    }
    return parseFullClient(client), nil
}
```

**What belongs here**:
- Business logic and rules
- Input normalization and validation
- Orchestration of multiple repository calls
- Transaction management
- Domain-specific transformations
- Business error handling

**Accessing Infrastructure**:
```go
// Database queries
s.cfg.GetDBQueries().GetClient(ctx, id)

// Database transactions
s.cfg.ExecuteDBTransaction(ctx, func(ctx context.Context, q *repository.Queries) error {
    // Multiple operations in a transaction
    return nil
})

// Object storage
s.cfg.GetStoreClient()   // S3 client
s.cfg.GetBucket()        // Bucket name

// Queue operations
s.cfg.GetQueueClient()
```

---

### 7. Repository Layer (`internal/database/repository/`)

**Purpose**: Type-safe database access generated from SQL.

**Generated by**: SQLC from `internal/database/queries/*.sql`

**Key Files**:
| File | Purpose |
|------|---------|
| `db.go` | `Queries` struct and `DBTX` interface |
| `models.go` | Generated Go types from database schema |
| `client.sql.go` | Generated client query methods |
| `document.sql.go` | Generated document query methods |
| etc. | One file per SQL source file |

**SQL Query Definition** (`internal/database/queries/client.sql`):
```sql
-- name: GetClient :one
SELECT * FROM fullClients WHERE clientId = $1;

-- name: CreateClient :exec
INSERT INTO clients (clientId, name) VALUES ($1, $2);

-- name: ListClients :many
SELECT * FROM fullClients ORDER BY name;
```

**Generated Go Code**:
```go
func (q *Queries) GetClient(ctx context.Context, clientid string) (*Fullclient, error)
func (q *Queries) CreateClient(ctx context.Context, arg *CreateClientParams) error
func (q *Queries) ListClients(ctx context.Context) ([]*Fullclient, error)
```

**Transaction Support**:
```go
// Create queries with transaction
q.WithTx(tx)
```

---

### 8. Database Migrations (`internal/database/migrations/`)

**Purpose**: Version-controlled schema evolution.

**Naming Convention**: `YYYYMMDDHHMMSS_description.up.sql` / `.down.sql`

**Examples**:
- `00000000000003_clients.up.sql` - Create clients table
- `00000000000109_create_folders_table.up.sql` - Add folders feature

**Migration Tool**: golang-migrate

---

### 9. Configuration Layer (`internal/serviceconfig/`)

**Purpose**: Environment-based configuration with dependency injection.

**Key Interface** (`ConfigProvider`):
```go
type ConfigProvider interface {
    // Database access
    GetDBQueries() *repository.Queries
    GetDBPool() *pgxpool.Pool
    ExecuteDBTransaction(ctx, func) error

    // Logging
    GetLogger() *slog.Logger

    // AWS services
    GetStoreClient() objectstore.Client  // S3
    GetQueueClient() queue.Client        // SQS
    GetBucket() string

    // Auth
    GetAuthConfig() auth.ConfigProvider
}
```

**Configuration Composition**:
```go
type QueryAPIConfig struct {
    api.BaseConfig                    // Server config
    clientsync.ClientSyncConfig       // Queue config
    objectstore.ObjectStoreConfig     // S3 config
    BackgroundRunner *backgroundtask.Runner
}
```

---

## Request Flow

Here's how a request flows through the system:

```mermaid
sequenceDiagram
    participant C as HTTP Client
    participant E as Echo Router
    participant MW as Middleware Chain
    participant V as OpenAPI Validator
    participant W as ServerInterfaceWrapper
    participant H as Controller Handler
    participant S as Service
    participant R as Repository
    participant DB as PostgreSQL

    C->>E: POST /client
    E->>MW: Apply middleware
    MW->>MW: Rate limit check
    MW->>MW: JWT validation
    MW->>MW: Permit.io authorization
    MW->>V: Validate request body
    V->>W: Route to wrapper
    W->>W: Extract path/query params
    W->>H: CreateClient(ctx)
    H->>H: ctx.Bind(&req)
    H->>S: client.Create(ctx, params)
    S->>S: Normalize input
    S->>R: CreateClient(ctx, dbParams)
    R->>DB: INSERT INTO clients...
    DB-->>R: Success
    R-->>S: nil error
    S-->>H: clientID, nil
    H->>H: Build response
    H-->>C: 201 Created + JSON body
```

---

## Adding a New API Endpoint

Follow these steps to add a new endpoint (e.g., `GET /client/{id}/summary`):

### Step 1: Define the API Contract

Edit `serviceAPIs/queryAPI.yaml`:

```yaml
paths:
  /client/{id}/summary:
    parameters:
      - $ref: "#/components/parameters/ClientID"
    get:
      operationId: getClientSummary
      tags:
        - ClientService
      summary: Get client summary
      security:
        - jwtAuth: []
      responses:
        "200":
          description: Client summary
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/ClientSummary"

components:
  schemas:
    ClientSummary:
      type: object
      required:
        - id
        - documentCount
      properties:
        id:
          type: string
        documentCount:
          type: integer
```

### Step 2: Generate Code

```bash
task generate
```

This updates:
- `api/queryAPI/api.gen.go` - Adds `GetClientSummary` to `ServerInterface` and `ClientSummary` type

### Step 3: Add Database Query (if needed)

Edit `internal/database/queries/client.sql`:

```sql
-- name: GetClientDocumentCount :one
SELECT COUNT(*) as count FROM documents WHERE clientId = $1;
```

Run: `task generate`

### Step 4: Add Service Method

Create or edit `internal/client/summary.go`:

```go
package client

import "context"

type Summary struct {
    ID            string
    DocumentCount int64
}

func (s *Service) GetSummary(ctx context.Context, id string) (*Summary, error) {
    // Validate client exists
    client, err := s.Get(ctx, id)
    if err != nil {
        return nil, err
    }

    // Get document count
    count, err := s.cfg.GetDBQueries().GetClientDocumentCount(ctx, id)
    if err != nil {
        return nil, fmt.Errorf("failed to get document count: %w", err)
    }

    return &Summary{
        ID:            client.ID,
        DocumentCount: count,
    }, nil
}
```

### Step 5: Implement Handler

Edit `api/queryAPI/client.go`:

```go
func (s *Controllers) GetClientSummary(ctx echo.Context, id ClientID) error {
    summary, err := s.svc.Client.GetSummary(ctx.Request().Context(), id)
    if err != nil {
        return echo.NewHTTPError(http.StatusBadRequest,
            fmt.Sprintf("Unable to get client summary: %s", err))
    }

    return ctx.JSON(http.StatusOK, ClientSummary{
        Id:            summary.ID,
        DocumentCount: int32(summary.DocumentCount),
    })
}
```

### Step 6: Add Tests

Create `internal/client/summary_test.go`:

```go
func TestService_GetSummary(t *testing.T) {
    // Use testcontainers for real database testing
    cfg := test.NewTestConfig(t)
    svc := client.New(cfg)

    // Create test client
    clientID, err := svc.Create(ctx, client.CreateParams{...})
    require.NoError(t, err)

    // Test summary
    summary, err := svc.GetSummary(ctx, clientID)
    require.NoError(t, err)
    assert.Equal(t, clientID, summary.ID)
    assert.Equal(t, int64(0), summary.DocumentCount)
}
```

### Step 7: Verify

```bash
task lint          # Check for issues
task test:functional  # Run test suite
task fullsuite:ci  # Run full test suite
```

---

## Key Patterns

### Dependency Injection via ConfigProvider

Services receive all dependencies through the `ConfigProvider` interface:

```go
type Service struct {
    cfg serviceconfig.ConfigProvider
}

func (s *Service) DoSomething(ctx context.Context) error {
    // Database
    s.cfg.GetDBQueries().SomeQuery(ctx, ...)

    // Logging
    s.cfg.GetLogger().Info("Operation completed")

    // S3
    s.cfg.GetStoreClient().PutObject(...)

    return nil
}
```

### Transaction Handling

Use `ExecuteDBTransaction` for operations requiring atomicity:

```go
err := s.cfg.ExecuteDBTransaction(ctx, func(ctx context.Context, q *repository.Queries) error {
    if err := q.CreateClient(ctx, params); err != nil {
        return err
    }
    if err := q.CreateRootFolder(ctx, folderParams); err != nil {
        return err
    }
    return nil
})
```

### Error Handling

Services return errors, controllers convert to HTTP errors:

```go
// Service - return domain errors
if client == nil {
    return nil, errors.New("client not found")
}

// Controller - convert to HTTP error
if err != nil {
    return echo.NewHTTPError(http.StatusNotFound, err.Error())
}
```

### Input Normalization

Normalize and validate inputs in the service layer:

```go
func (s *Service) normalizeCreate(params *CreateParams) error {
    // Trim whitespace
    params.Name = strings.TrimSpace(params.Name)

    // Validate format
    if !isValidID(params.ID) {
        return errors.New("invalid ID format")
    }
    return nil
}
```

---

## Component Diagrams

### Domain Package Structure

```mermaid
flowchart LR
    subgraph "internal/client/"
        S[service.go<br/>Service struct]
        C[create.go<br/>Create operation]
        G[get.go<br/>Read operations]
        L[list.go<br/>List operations]
        ST[status.go<br/>Status checks]
    end

    subgraph "Repository"
        R[repository/client.sql.go]
    end

    S --> C
    S --> G
    S --> L
    S --> ST
    C --> R
    G --> R
    L --> R
    ST --> R
```

### Configuration Composition

```mermaid
classDiagram
    class ConfigProvider {
        <<interface>>
        GetDBQueries()
        GetLogger()
        GetStoreClient()
    }

    class BaseConfig {
        Port int
        BaseURL string
        CognitoConfig
        DBConfig
        AWSConfig
    }

    class QueryAPIConfig {
        BaseConfig
        ClientSyncConfig
        ObjectStoreConfig
    }

    ConfigProvider <|.. BaseConfig
    BaseConfig <|-- QueryAPIConfig
```

### Service Layer Interactions

```mermaid
flowchart TB
    subgraph Controllers
        CH[Client Handler]
        DH[Document Handler]
    end

    subgraph Services
        CS[Client Service]
        DS[Document Service]
        US[Upload Service]
    end

    subgraph Repository
        CR[Client Queries]
        DR[Document Queries]
    end

    subgraph Infrastructure
        DB[(PostgreSQL)]
        S3[(S3)]
    end

    CH --> CS
    DH --> DS
    DH --> US
    CS --> CR
    DS --> DR
    US --> DR
    US --> S3
    CR --> DB
    DR --> DB
```

---

## Quick Reference

| Layer | Location | Responsibility |
|-------|----------|----------------|
| Entry Point | `cmd/queryAPI/main.go` | Bootstrap, wire dependencies |
| Server | `internal/server/api/listener.go` | HTTP server, middleware setup |
| Middleware | `internal/cognitoauth/` | Auth, authorization |
| Generated | `api/queryAPI/api.gen.go` | Interface, types (don't edit) |
| Controllers | `api/queryAPI/*.go` | Request handling, response formatting |
| Services | `internal/<domain>/` | Business logic |
| Repository | `internal/database/repository/` | Database access (generated) |
| Queries | `internal/database/queries/*.sql` | SQL definitions |
| Migrations | `internal/database/migrations/` | Schema changes |
| Config | `internal/serviceconfig/` | Environment config, DI |

---

## See Also

- [02-service-architecture.md](./02-service-architecture.md) - Overall system architecture
- [06-code-organization.md](./06-code-organization.md) - Project structure details
- [03-api-documentation.md](./03-api-documentation.md) - API reference
- [04-data-architecture.md](./04-data-architecture.md) - Database schema