query-orchestration/docs/ai.generated/03-api-documentation.md

# API Documentation

## API Overview

The DoczyAI Query Orchestration Platform exposes a comprehensive REST API following OpenAPI 3.0.3 specification. The API is automatically generated from the specification file `serviceAPIs/queryAPI.yaml` using oapi-codegen.
also see [query api summary](./queryapi.summary.md)

### Base Configuration

- **Base URL**: `https://doczy.com/v1` (production), `http://localhost:8080` (development)
- **API Version**: 0.0.1
- **Content Type**: `application/json`
- **Documentation**: Auto-generated Swagger UI at `/swagger/index.html`

### API Services

The API is organized into the following service groups:

| Service                | Description                                                 |
| ---------------------- | ----------------------------------------------------------- |
| ClientService          | Operations related to clients                               |
| CollectorService       | Operations related to collectors                            |
| DocumentsService       | Operations related to documents                             |
| FolderService          | Operations related to document folders and organization     |
| LabelService           | Operations related to document labels and workflow tracking |
| FieldExtractionService | Operations related to document field extractions            |
| ExportService          | Operations related to exports                               |
| AuthService            | Operations related to authentication                        |
| AdminService           | Operations related to user management and administration    |
| EulaService            | Operations related to End User License Agreement management and tracking |
| UISettingsService      | Operations related to UI-scoped key-value settings (per user, client, or global) |
| SuperAdminSchemaService | Operations related to custom schema management (super_admin only)              |
| CustomMetadataService  | Operations related to per-document custom metadata (client_user and above)       |
| BotService             | Chatbot session/scope/turn endpoints plus read-only document schema and all-metadata bundles for `client_user`. See [Chatbot Guide](12-chatbot-guide.md) for the complete reference. |
| SuperAdminBotService   | Super-admin chatbot session list/get/soft-delete across all users in a client. See [Chatbot Guide](12-chatbot-guide.md). |

## Chatbot Endpoints

The chatbot endpoints expose a synchronous facade over the Aaria FastAPI
agent service. Plan §3 mandates the singular `/client/...` form so the
existing Permit.io `client_user` policy applies. See
[Chatbot Guide](12-chatbot-guide.md) for the operator runbook,
configuration reference, AddTurn algorithm, and HTTP status mapping.

| Method | Path                                                                  | OperationId                       | Auth                  |
| ------ | --------------------------------------------------------------------- | --------------------------------- | --------------------- |
| POST   | `/client/{clientId}/bot/sessions`                                     | `createBotSession`                | client_user           |
| GET    | `/client/{clientId}/bot/sessions`                                     | `listBotSessions`                 | client_user           |
| GET    | `/client/{clientId}/bot/sessions/{sessionId}`                         | `getBotSession`                   | client_user (owner)   |
| PATCH  | `/client/{clientId}/bot/sessions/{sessionId}/scope`                   | `patchBotSessionScope`            | client_user (owner)   |
| POST   | `/client/{clientId}/bot/sessions/{sessionId}/turns`                   | `createBotTurn`                   | client_user (owner)   |
| GET    | `/client/{clientId}/bot/sessions/{sessionId}/turns`                   | `listBotTurns`                    | client_user (owner)   |
| GET    | `/client/{clientId}/documents/{documentId}/schema`                    | `getDocumentSchema`               | client_user           |
| GET    | `/client/{clientId}/documents/{documentId}/all-metadata`              | `getDocumentAllMetadata`          | client_user           |
| GET    | `/super-admin/bot/sessions?clientId=...`                              | `listBotSessionsAsSuperAdmin`     | super_admin           |
| GET    | `/super-admin/bot/sessions/{sessionId}?clientId=...`                  | `getBotSessionAsSuperAdmin`       | super_admin           |
| DELETE | `/super-admin/bot/sessions/{sessionId}?clientId=...`                  | `deleteBotSessionAsSuperAdmin`    | super_admin           |

`BotSessionResponse.lastTurn` is the derived `last_terminal_ordinal`
from `bot_turns` (highest ordinal whose status is not `in_flight`),
NOT the highest seen ordinal. `last_seen_ordinal` is computed but used
only internally by `AddTurn` for ordinal classification.

`DocumentSchemaResponse.schema` is omitted from the JSON wire response
when the document has no `custom_schema_id` binding (the field is
declared `omitempty`); the document still exists, only the optional
binding is absent. Plan §10 M5: HTTP 200 in this case, NOT 404.

The all-metadata endpoint emits `BotLabelRecord` (plain-string
`appliedBy`) instead of the existing `LabelRecord` (email-validated)
for the embedded labels list. Both wrap the same
`documentLabels.appliedBy varchar` column; the chatbot variant tolerates
non-email values that pre-date the email-typing convention. Long-term
cleanup is to relax `LabelRecord.AppliedBy` to plain string.

## Authentication and Authorization

### Security Schemes

#### JWT Authentication

- **Type**: Bearer token authentication
- **Header**: `Authorization: Bearer <token>`
- **Token Source**: JWT cookie set via OAuth flow

#### OAuth2 Flow

- **Type**: OAuth2 Authorization Code Flow
- **Provider**: AWS Cognito
- **Scopes**: `openid`, `email`, `profile`

### Authentication Endpoints

#### `GET /login`

Initiates OAuth2 authentication flow with AWS Cognito.

**Security**: Public (no authentication required)
**Response**: `302` Redirect to Cognito login page

#### `GET /login-callback`

OAuth2 callback handler that processes authentication response.

**Security**: Public
**Parameters**:

- `code` (query, required): Authorization code from Cognito (max 2048 chars)
- `state` (query, required): CSRF protection parameter (max 1024 chars)

**Response**: `302` Sets JWT cookie and redirects to application

#### `GET /logout`

Terminates user session and clears authentication cookies.

**Security**: Public (no authentication required)
**Response**: `302` Redirect to Cognito logout page or application home

#### `GET /home`

Returns the HTML menu page.

**Security**: Public (no authentication required)
**Response**: `200` HTML content

#### `GET /identity`

Returns the authenticated user's assigned roles from Permit.io, including the permissions granted by each role. Any authenticated user can call this endpoint to discover their own roles. This endpoint requires JWT authentication but does not require specific Permit.io permissions.

**Security**: Requires JWT authentication (auth-only, no Permit.io check)

**Response** (`IdentityResponse`):

```json
{
  "roles": [
    {
      "key": "auditor",
      "name": "Auditor",
      "description": "Read-only access to view documents and exports",
      "permissions": ["document:read", "document:list", "export:read"]
    }
  ]
}
```

**Responses**:

- `200`: User roles and permissions retrieved successfully
- `401`: Authentication required
- `429`: Rate limit exceeded
- `502`: Failed to retrieve roles from Permit.io

## Rate Limiting

The queryAPI service implements dual-layer rate limiting to protect against DDoS attacks and ensure fair resource allocation.

### Rate Limit Architecture

**Dual-Layer Protection:**

- **Layer 1: Global/Aggregate Rate Limiting** - Protects total system capacity across all IP addresses
- **Layer 2: Per-IP Rate Limiting** - Prevents individual client abuse

Both layers must pass for a request to proceed. The global limiter checks first for faster failure paths under DDoS conditions.

### Default Rate Limits

By default, all endpoints are limited to:

- **Global**: 500 requests per second total (1000 burst)
- **Per-IP**: 5 requests per second per IP (10 burst)
- **Cleanup Interval**: 3 minutes for inactive IP limiters

**Important:** No endpoints are exempt from rate limiting. This is a security requirement to prevent DDoS attacks.

### Configuration

Rate limits are configured via environment variables:

```bash
# Global/Aggregate rate limiting (protects total system capacity)
RATE_LIMIT_GLOBAL_RATE=500         # total req/s across ALL IPs
RATE_LIMIT_GLOBAL_BURST=1000        # total burst capacity across ALL IPs

# Per-IP rate limiting (protects against individual abusers)
RATE_LIMIT_DEFAULT_RATE=5          # req/s per IP
RATE_LIMIT_DEFAULT_BURST=10         # burst per IP
RATE_LIMIT_EXPIRES_IN=180           # seconds (3 minutes)

# Per-endpoint overrides (JSON format).
# Plan §10 M6 + §11: setting an override here NOW throttles the actual
# per-(IP, route) request rate (Allow() decision), not just the
# RateLimit / Retry-After response headers. Operators upgrading from
# pre-M6 deployments should review existing overrides for unintended
# tightening.
#
# Keys use Echo's path template form (`:clientId`, `:sessionId`), NOT
# the OpenAPI `{clientId}` form. The middleware uses c.Path() which
# returns Echo's registered path string.
#
# Example for the chatbot turn endpoint (illustrative — adjust before
# production rollout):
#   RATE_LIMIT_ENDPOINT_OVERRIDES='{"POST /client/:clientId/bot/sessions/:sessionId/turns":{"global_rate":50,"global_burst":100,"rate":2,"burst":4}}'
RATE_LIMIT_ENDPOINT_OVERRIDES={}
```

### Rate Limit Headers

All API responses include a `RateLimit` header indicating the current limit:

```
RateLimit: 20;window=2
```

**Format:** `{burst};window={seconds}`

### Rate Limit Exceeded Response

**HTTP Status:** `429 Too Many Requests`

**Headers:**

- `Retry-After: {seconds}` - Number of seconds before retrying
- `RateLimit: {burst};window={seconds}` - Current rate limit

**Response Body:**

```json
{
  "message": "rate limit exceeded"
}
```

---

## Client Management (ClientService)

### `GET /clients`

Lists all clients in the system.

**Security**: Requires JWT authentication

**Response** (`ClientListResponse`):

```json
{
  "clients": [
    {
      "id": "AAA",
      "name": "ACME Corporation",
      "can_sync": true
    }
  ],
  "totalCount": 1
}
```

### `POST /client`

Creates a new client in the system.

**Security**: Requires JWT authentication

**Request Body** (`ClientCreate`):

```json
{
  "id": "string (max 36 chars, required)",
  "name": "string (max 256 chars, required)"
}
```

**Responses**:

- `201`: Client created successfully
  ```json
  {
    "id": "AAA"
  }
  ```
- `400`: Invalid request data
- `401`: Authentication required
- `429`: Rate limit exceeded
- `500`: Internal server error

### `GET /client/{id}`

Retrieves client details by ID.

**Security**: Requires JWT authentication

**Path Parameters**:

- `id`: Client identifier (string, max 36 chars)

**Response** (`DocClient`):

```json
{
  "id": "AAA",
  "name": "ACME Corporation",
  "can_sync": true
}
```

### `PATCH /client/{id}`

Updates client information.

**Security**: Requires JWT authentication

**Request Body** (`ClientUpdate`):

```json
{
  "name": "string (max 256 chars, optional)",
  "can_sync": "boolean (optional)"
}
```

**Response**: `200` Client updated successfully

### `DELETE /client/{id}`

Permanently deletes a client and all associated data: documents (with all dependent rows), folders, collector configurations, batch uploads, and clientCanSync records. Source documents in S3 are NOT deleted; their paths are logged and optionally returned. This operation is irreversible.

**Security**: Requires JWT authentication. Restricted to `super_admin` role only.

**Query Parameters**:

- `confirm` (required, boolean): Must be `true` to proceed. Returns `400 Bad Request` if absent or false.
- `verbose` (optional, boolean, default: false): When `true`, returns S3 paths of orphaned source documents.

**Response**:

- `204`: Client deleted successfully (when `verbose=false`)
- `200`: Client deleted successfully with S3 paths (when `verbose=true`):

```json
{
  "deletedDocuments": [
    {
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "s3Path": "s3://bucket-name/key/path/to/file"
    }
  ]
}
```

- `400`: Missing `confirm=true` query parameter
- `404`: Client not found

### `GET /client/{id}/status`

Returns client synchronization status.

**Security**: Requires JWT authentication

**Response** (`ClientStatusBody`):

```json
{
  "status": "IN_SYNC"
}
```

**Status Values**: `IN_SYNC`, `NOT_SYNCED`, `NOT_SYNCING`

---

## Document Operations (DocumentsService)

### `POST /client/{id}/document`

Uploads a document for a specific client.

**Security**: Requires JWT authentication
**Content Type**: `multipart/form-data`
**Max File Size**: 100GB

**Form Parameters**:

- `file` (required): Document file. Accepted types: PDF, TIFF, JPEG, PNG, BMP, DOCX, XLSX, PPTX, TXT, EML
- `filename` (optional): Custom filename (max 4096 chars)

**Response**: `200` Document uploaded

### `GET /client/{id}/document`

Lists documents for a client.

**Security**: Requires JWT authentication

**Response** (`ListDocuments`):

```json
[
  {
    "id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
    "hash": "sha256:abc123..."
  }
]
```

### `GET /document/{id}`

Retrieves enriched document details by ID, including metadata such as folder, filename, labels, and optionally the full text extraction record.

**Security**: Requires JWT authentication

**Path Parameters**:

- `id`: Document UUID (max 36 chars)

**Query Parameters**:

- `textRecord` (optional, default: false): When true, includes the full text extraction record in the response

**Response** (`DocumentEnriched`):

```json
{
  "id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "client_id": "AAA",
  "hash": "sha256:abc123...",
  "hasTextRecord": true,
  "folderId": "019580df-ef65-7676-8de9-94435a93337a",
  "filename": "contract_2024.pdf",
  "originalPath": "/documents/2024/contract_2024.pdf",
  "labels": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337b",
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "label": "OCR_Processed",
      "appliedBy": "user@example.com",
      "appliedAt": "2025-10-16T14:27:15Z"
    }
  ],
  "fileSizeBytes": 1048576
}
```

### `GET /document/{id}/download`

Streams the original uploaded document file directly from S3. The response Content-Type is auto-detected from the stored filename extension.

**Security**: Requires JWT authentication

**Path Parameters**:

- `id`: Document UUID (max 36 chars)

**Response** (`200`): Raw file bytes (streamed)

- `Content-Type`: Auto-detected from filename (e.g., `application/pdf`, `image/png`, `application/vnd.openxmlformats-officedocument.wordprocessingml.document`)
- `Content-Disposition`: `attachment; filename="<original_filename>"`
- `Content-Length`: File size in bytes

**Error Responses**:

- `400`: Invalid document ID format
- `401`: Missing or invalid JWT token
- `404`: Document not found or no S3 entry for the document
- `500`: S3 retrieval failure

**Example**:

```bash
curl -H "Authorization: Bearer <token>" \
     -o report.pdf \
     http://localhost:8080/document/019580df-ef65-7676-8de9-94435a93337a/download
```

**Notes**:

- No query parameters needed -- file type is automatically determined from the stored filename
- Response streams directly from S3, suitable for large files
- Returns 404 if the document has no stored file (e.g., legacy documents without S3 entries)
- Supports any file type without code changes (PDF, PNG, DOCX, CSV, etc.)

### `DELETE /document/{id}`

Permanently deletes a document and all dependent data: document entries, cleans (and their entries), text extractions (and their entries), field extractions (versions and array fields), labels, results, and result dependencies. Source documents in S3 are NOT deleted; their paths are logged and optionally returned.

**Security**: Requires JWT authentication. Restricted to `super_admin` role only.

**Path Parameters**:

- `id`: Document UUID (max 36 chars)

**Query Parameters**:

- `verbose` (optional, boolean, default: false): When `true`, returns S3 paths of orphaned source documents.

**Response**:

- `204`: Document deleted successfully (when `verbose=false`)
- `200`: Document deleted successfully with S3 paths (when `verbose=true`):

```json
{
  "deletedDocuments": [
    {
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "s3Path": "s3://bucket-name/key/path/to/file"
    }
  ]
}
```

- `404`: Document not found

---

## Batch Document Operations (DocumentsService)

### `POST /client/{id}/document/batch`

Uploads a ZIP archive containing multiple documents for asynchronous batch processing.

**Security**: Requires JWT authentication
**Content Type**: `multipart/form-data`
**Max Archive Size**: 1GB
**Supported Archive Type**: ZIP only

**Accepted File Extensions**: `.pdf`, `.tiff`, `.tif`, `.jpeg`, `.jpg`, `.png`, `.bmp`, `.docx`, `.xlsx`, `.pptx`, `.txt`, `.eml`

Note: Macro-enabled Office formats (`.docm`, `.xlsm`, `.pptm`) are intentionally excluded because documents containing macros are rejected during validation.

**Form Parameters**:

- `archive` (required): ZIP file containing documents

**Response** (`202 Accepted`):

```json
{
  "batch_id": "019580df-ef65-7676-8de9-94435a93337a",
  "status": "processing",
  "status_url": "/client/AAA/document/batch/019580df-ef65-7676-8de9-94435a93337a"
}
```

### `GET /client/{id}/document/batch`

Lists batch uploads for a client with pagination support.

**Security**: Requires JWT authentication

**Query Parameters**:

- `limit`: Maximum results (integer, 1-100, default: 20)
- `offset`: Pagination offset (integer, default: 0)

**Response** (`BatchUploadList`):

```json
{
  "batches": [
    {
      "batch_id": "019580df-ef65-7676-8de9-94435a93337a",
      "client_id": "AAA",
      "original_filename": "documents.zip",
      "status": "completed",
      "total_documents": 100,
      "processed_documents": 100,
      "failed_documents": 2,
      "invalid_type_documents": 1,
      "progress_percent": 100,
      "created_at": "2024-01-01T00:00:00Z",
      "completed_at": "2024-01-01T00:05:00Z"
    }
  ],
  "total_count": 25
}
```

**Batch Status Values**: `processing`, `completed`, `failed`, `cancelled`

### `GET /client/{id}/document/batch/{batch_id}`

Retrieves detailed status information for a specific batch upload, including per-file outcome tracking through the document processing pipeline.

**Security**: Requires JWT authentication

**Path Parameters**:

- `batch_id`: Batch upload identifier (UUID)

**Response** (`BatchUploadDetails`):

```json
{
  "batch_id": "019580df-ef65-7676-8de9-94435a93337a",
  "client_id": "AAA",
  "original_filename": "documents.zip",
  "status": "completed",
  "total_documents": 5,
  "processed_documents": 3,
  "failed_documents": 1,
  "invalid_type_documents": 1,
  "progress_percent": 100,
  "created_at": "2024-01-01T00:00:00Z",
  "completed_at": "2024-01-01T00:05:00Z",
  "failed_filenames": ["corrupt.pdf"],
  "document_outcomes": [
    {
      "filename": "invoice.pdf",
      "outcome": "clean_passed",
      "document_id": "019580e0-1111-7676-8de9-000000000001",
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:04:50Z"
    },
    {
      "filename": "receipt.pdf",
      "outcome": "clean_passed",
      "document_id": "019580e0-2222-7676-8de9-000000000002",
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:04:52Z"
    },
    {
      "filename": "duplicate_invoice.pdf",
      "outcome": "duplicate",
      "document_id": "019580e0-1111-7676-8de9-000000000001",
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:00:01Z"
    },
    {
      "filename": "corrupt.pdf",
      "outcome": "clean_failed",
      "error_detail": "PDF validation failed",
      "document_id": "019580e0-3333-7676-8de9-000000000003",
      "clean_fail_reason": "invalid PDF structure: missing xref table",
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:04:55Z"
    },
    {
      "filename": "macro_enabled.xlsm",
      "outcome": "invalid_type",
      "error_detail": null,
      "document_id": null,
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:00:01Z"
    }
  ]
}
```

The `document_outcomes` array provides per-file tracking through the entire processing pipeline. Each file in the ZIP archive gets one outcome entry, starting at extraction time and updating as the file progresses through the document runners (init, sync, clean). When all outcomes reach a terminal state, the batch is fully processed.

**Document Outcome Values**:

| Outcome            | Meaning                                                | Terminal? |
| ------------------ | ------------------------------------------------------ | --------- |
| `submitted`        | File extracted and uploaded, waiting for processing    | No        |
| `duplicate`        | File hash matches an existing document for this client | Yes       |
| `failed_open`      | Could not open file from ZIP archive                   | Yes       |
| `failed_read`      | Could not read file content from ZIP archive           | Yes       |
| `failed_s3_upload` | Could not upload extracted file to S3                  | Yes       |
| `failed_upload`    | Upload handler returned an error                       | Yes       |
| `invalid_type`     | File type is not in the accepted extensions list       | Yes       |
| `init_complete`    | Document record created, forwarded to sync             | No        |
| `init_duplicate`   | Duplicate detected during init processing              | Yes       |
| `sync_complete`    | Sync check passed, forwarded to clean                  | No        |
| `sync_skipped`     | Client sync is disabled                                | Yes       |
| `clean_passed`     | Document passed all validation checks                  | Yes       |
| `clean_failed`     | Document failed validation (see `clean_fail_reason`)   | Yes       |

**Outcome Fields**:

| Field               | Type           | Description                                                         |
| ------------------- | -------------- | ------------------------------------------------------------------- |
| `filename`          | string         | Original filename from the ZIP archive                              |
| `outcome`           | string         | Current processing status (see table above)                         |
| `error_detail`      | string or null | Human-readable error message for failure outcomes                   |
| `document_id`       | uuid or null   | Assigned document ID (null for files that never became documents)   |
| `clean_fail_reason` | string or null | Specific validation failure reason when `outcome` is `clean_failed` |
| `created_at`        | datetime       | When the file was first extracted from the ZIP                      |
| `updated_at`        | datetime       | When the outcome was last updated by a pipeline stage               |

---

## Folder Operations (FolderService)

### Folder Path Constraints

All folder paths must comply with the following constraints when creating or renaming folders:

- **Must start with `/`**: All paths must begin with a forward slash
- **Maximum path length**: 1000 characters
- **Maximum segment length**: 255 characters per path segment
- **Maximum depth**: 20 levels of nesting
- **Valid characters only**: Alphanumeric (`a-z`, `A-Z`, `0-9`), hyphen (`-`), underscore (`_`), period (`.`), and space (` `)
- **Segment must start with alphanumeric**: Each path segment must begin with a letter or number
- **No empty segments**: Double slashes (`//`) or trailing slashes are not allowed
- **No path traversal**: Sequences like `..` or `./` are blocked
- **No hidden folders**: Segments cannot start with a period (e.g., `/.hidden`)
- **No leading/trailing whitespace**: Spaces within segment names are allowed, but not at the start or end
- **No control characters**: ASCII control characters (0x00-0x1F, 0x7F) are blocked
- **No Windows reserved names**: Names like `CON`, `PRN`, `AUX`, `NUL`, `COM1`-`COM9`, `LPT1`-`LPT9` are blocked
- **Root folder restrictions**: The root path `/` can only exist once per client and cannot be renamed
- **Path uniqueness**: Each path must be unique within a client (enforced by database)
- **Unicode normalization**: Paths are normalized to NFC form

**Valid path examples**:

- `/documents`
- `/documents/2024/Q1`
- `/client-data/reports_2024`
- `/My Documents/Archive`

**Invalid path examples**:

- `documents` (missing leading `/`)
- `/documents/` (trailing slash)
- `/foo//bar` (empty segment)
- `/foo/../bar` (path traversal)
- `/.hidden` (dot prefix)
- `/foo@bar` (invalid character)
- `/CON` (Windows reserved name)

---

### `GET /clients/{clientId}/folders`

Lists all folders for a specific client, including the folder hierarchy.

**Security**: Requires JWT authentication

**Path Parameters**:

- `clientId`: Client external ID (string, max 36 chars)

**Query Parameters**:

- `metrics` (optional, default: false): When true, includes processing metrics for each folder (document counts by label). Adds overhead so should only be used when metrics are needed.

**Response** (`200 OK`):

```json
{
  "folders": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "path": "/",
      "parentId": null,
      "clientId": "AAA",
      "createdAt": "2025-10-16T14:27:15Z",
      "createdBy": "system"
    },
    {
      "id": "019580df-ef65-7676-8de9-94435a93337b",
      "path": "/documents",
      "parentId": "019580df-ef65-7676-8de9-94435a93337a",
      "clientId": "AAA",
      "createdAt": "2025-10-16T14:30:00Z",
      "createdBy": "user@example.com"
    },
    {
      "id": "019580df-ef65-7676-8de9-94435a93337c",
      "path": "/documents/2024",
      "parentId": "019580df-ef65-7676-8de9-94435a93337b",
      "clientId": "AAA",
      "createdAt": "2025-10-16T14:35:00Z",
      "createdBy": "user@example.com"
    }
  ]
}
```

**Notes**:

- Returns all folders for the client, including nested folders
- Folders are returned in a flat list; use `parentId` to reconstruct the hierarchy
- The root folder (path="/") is included if it exists

### `POST /folders`

Creates a new folder for organizing documents.

**Security**: Requires JWT authentication

**Request Body** (`FolderCreate`):

```json
{
  "path": "/documents/2024",
  "clientId": "AAA",
  "createdBy": "user@example.com",
  "parentId": "019580df-ef65-7676-8de9-94435a93337a"
}
```

| Field     | Type   | Required | Description                                       |
| --------- | ------ | -------- | ------------------------------------------------- |
| path      | string | Yes      | Folder path, must start with "/" (max 1000 chars) |
| clientId  | string | Yes      | Client external ID                                |
| createdBy | string | Yes      | Email of user creating the folder                 |
| parentId  | uuid   | No       | Parent folder ID for nested folders               |

**Response** (`201 Created`):

```json
{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "path": "/documents/2024",
  "parentId": null,
  "clientId": "AAA",
  "createdAt": "2025-10-16T14:27:15Z",
  "createdBy": "user@example.com"
}
```

### `PATCH /folders/{folderId}`

Renames an existing folder.

**Security**: Requires JWT authentication

**Path Parameters**:

- `folderId`: Folder UUID

**Request Body** (`FolderRename`):

```json
{
  "path": "/documents/2024-renamed"
}
```

**Response** (`200 OK`): Updated folder object

### `DELETE /folders/{folderId}`

Permanently deletes a folder and all child folders recursively. The root folder (`/`) cannot be deleted. If the folder tree contains documents, the caller must either delete the documents first or pass `include_documents=true`.

**Security**: Requires JWT authentication. Restricted to `super_admin` role only.

**Path Parameters**:

- `folderId`: Folder UUID

**Query Parameters**:

- `include_documents` (optional, boolean, default: false): When `true`, deletes all documents in the folder tree along with the folders. When `false` and the folder tree contains documents, returns `409 Conflict`.
- `verbose` (optional, boolean, default: false): When `true`, returns S3 paths of orphaned source documents.

**Response**:

- `204`: Folder tree deleted successfully (when `verbose=false`)
- `200`: Folder tree deleted successfully with S3 paths (when `verbose=true`):

```json
{
  "deletedDocuments": [
    {
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "s3Path": "s3://bucket-name/key/path/to/file"
    }
  ]
}
```

- `404`: Folder not found
- `409`: Cannot delete root folder, or folder tree contains documents and `include_documents` is not `true`

### `GET /folders/{folderId}/documents`

Retrieves all documents within a specific folder, including filename and labels. Returns a lighter-weight format than `GET /document/{id}` (excludes `client_id`).

**Security**: Requires JWT authentication

**Query Parameters**:

- `textRecord` (optional, default: false): When true, includes the full text extraction record for each document

**Response**:

```json
{
  "documents": [
    {
      "id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "hash": "sha256:abc123...",
      "hasTextRecord": true,
      "folderId": "019580df-ef65-7676-8de9-94435a93337a",
      "filename": "contract_2024.pdf",
      "originalPath": "/documents/2024/contract_2024.pdf",
      "labels": [],
      "fileSizeBytes": 1048576
    }
  ]
}
```

### `GET /folders/{folderId}/metrics`

Retrieves processing progress metrics for documents in a folder.

**Security**: Requires JWT authentication

**Response** (`FolderMetrics`):

```json
{
  "folderId": "019580df-ef65-7676-8de9-94435a93337a",
  "totalDocuments": 150,
  "byLabel": {
    "Ingested": 150,
    "OCR_Processed": 120,
    "Dashboard_Ready": 100
  }
}
```

---

## Label Operations (LabelService)

### `POST /documents/{documentId}/labels`

Applies a workflow label to a document for tracking processing state.

**Security**: Requires JWT authentication

**Path Parameters**:

- `documentId`: Document UUID

**Request Body** (`LabelApplication`):

```json
{
  "label": "OCR_Processed",
  "appliedBy": "user@example.com"
}
```

| Field     | Type   | Required | Description                                                  |
| --------- | ------ | -------- | ------------------------------------------------------------ |
| label     | string | Yes      | Label name (alphanumeric and underscore only, max 100 chars) |
| appliedBy | string | Yes      | Email of user applying the label                             |

**Response** (`201 Created`):

```json
{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "label": "OCR_Processed",
  "appliedBy": "user@example.com",
  "appliedAt": "2025-10-16T14:27:15Z"
}
```

### `GET /documents/{documentId}/labels`

Retrieves all labels that have been applied to a document, ordered by most recent first.

**Security**: Requires JWT authentication

**Response**:

```json
{
  "labels": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "label": "OCR_Processed",
      "appliedBy": "user@example.com",
      "appliedAt": "2025-10-16T14:27:15Z"
    }
  ]
}
```

### `GET /labels/{labelName}/documents`

Retrieves all documents that have been tagged with a specific label.

**Security**: Requires JWT authentication

**Path Parameters**:

- `labelName`: Label name (alphanumeric and underscore only, max 100 chars)

**Query Parameters**:

- `clientId` (required): Client ID to filter documents (string, max 36 chars)

**Response**:

```json
{
  "documents": [
    {
      "id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "hash": "sha256:abc123..."
    }
  ]
}
```

---

## Field Extraction Operations (FieldExtractionService)

### `POST /field-extractions`

Creates a new field extraction with single-value and array fields for a document.

**Security**: Requires JWT authentication

**Request Body** (`FieldExtractionRequest`):

```json
{
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "singleFields": {
    "fileName": "contract_2024.pdf",
    "contractTitle": "Service Agreement",
    "clientName": "ACME Corp",
    "payerName": "Insurance Co",
    "payerState": "CA",
    "providerState": "NY",
    "aareteDerivedEffectiveDt": "2024-01-01",
    "aareteDerivedTerminationDt": "2025-12-31",
    "autoRenewalInd": true
  },
  "arrayFields": [
    {
      "exhibitTitle": "Exhibit A",
      "exhibitPage": "15",
      "reimbProvTin": "123456789",
      "reimbProvNpi": "1234567890",
      "reimbProvName": "Provider Name",
      "reimbEffectiveDt": "2024-01-01",
      "aareteDerivedClaimTypeCd": "INPATIENT",
      "aareteDerivedReimbMethod": "Per Diem",
      "reimbPctRate": 85.5,
      "reimbFeeRate": 150.0
    }
  ],
  "createdBy": "user@example.com"
}
```

**Single Fields** (1:1 relationship with document):

- `fileName`, `contractTitle`, `clientName`, `payerName`
- `payerState`, `providerState` (2-letter state codes)
- `filenameTin`, `provGroupTin`, `provGroupNpi`, `provGroupNameFull`
- `provOtherTin`, `provOtherNpi`, `provOtherNameFull`
- `aareteDerivedEffectiveDt`, `aareteDerivedTerminationDt` (dates)
- `aareteDerivedAmendmentNum` (integer)
- `autoRenewalInd` (boolean), `autoRenewalTerm`

**Array Fields** (1:N relationship - multiple rows per document):

- Provider info: `reimbProvTin`, `reimbProvNpi`, `reimbProvName`
- Dates: `reimbEffectiveDt`, `reimbTerminationDt`
- Classifications: `aareteDerivedClaimTypeCd`, `aareteDerivedProduct`, `aareteDerivedLob`, `aareteDerivedProgram`, `aareteDerivedNetwork`, `aareteDerivedProvType`
- Codes: `provTaxonomyCd`, `provSpecialtyCd`, `placeOfServiceCd`, `billTypeCd`, `cpt4ProcCd`, `revenueCd`, `diagCd`, `ndcCd`
- Rates: `reimbPctRate`, `reimbFeeRate`, `reimbConversionFactor`, `grouperPctRate`, `grouperBaseRate`
- Indicators: `carveoutInd`, `lesserOfInd`, `greaterOfInd`, `defaultInd`

**Response** (`201 Created`):

```json
{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "version": 1,
  "singleFields": { ... },
  "arrayFields": [ ... ],
  "createdAt": "2025-10-16T14:27:15Z",
  "createdBy": "user@example.com"
}
```

### `GET /field-extractions`

Retrieves the most recent field extraction for a document.

**Security**: Requires JWT authentication

**Query Parameters**:

- `documentId` (required): Document UUID

**Response**: `FieldExtractionResponse` (same as POST response)

### `GET /field-extractions/history`

Retrieves all versions of field extractions for a document, ordered by most recent first.

**Security**: Requires JWT authentication

**Query Parameters**:

- `documentId` (required): Document UUID

**Response**:

```json
{
  "versions": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "version": 2,
      "createdAt": "2025-10-17T10:00:00Z",
      "createdBy": "user@example.com"
    },
    {
      "id": "019580df-ef65-7676-8de9-94435a93337b",
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "version": 1,
      "createdAt": "2025-10-16T14:27:15Z",
      "createdBy": "user@example.com"
    }
  ]
}
```

### `GET /field-extractions/version`

Retrieves a specific version of a field extraction for a document by version number.

**Security**: Requires JWT authentication

**Query Parameters**:

- `documentId` (required): Document UUID
- `version` (required): Version number (integer, minimum 1)

**Response** (`200 OK`):

```json
{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "version": 1,
  "singleFields": {
    "fileName": "contract_2024.pdf",
    "contractTitle": "Service Agreement",
    "clientName": "ACME Corp"
  },
  "arrayFields": [],
  "createdAt": "2025-10-16T14:27:15Z",
  "createdBy": "user@example.com"
}
```

**Error Responses**:

- `400`: Missing required query parameters
- `404`: Version not found for the specified document

---

## Collector Configuration (CollectorService)

### `GET /client/{id}/collector`

Retrieves collector configuration for a client.

**Security**: Requires JWT authentication

**Response** (`Collector`):

```json
{
  "client_id": "AAA",
  "active_version": 1,
  "latest_version": 2,
  "minimum_cleaner_version": 1,
  "minimum_text_version": 1,
  "fields": [
    {
      "name": "contract_date",
      "query_id": "019580df-ef65-7676-8de9-94435a93337a"
    }
  ]
}
```

### `PATCH /client/{id}/collector`

Updates collector configuration.

**Security**: Requires JWT authentication

**Request Body** (`CollectorSet`):

```json
{
  "minimum_cleaner_version": 2,
  "minimum_text_version": 2,
  "active_version": 1,
  "fields": [
    {
      "name": "contract_date",
      "query_id": "019580df-ef65-7676-8de9-94435a93337a"
    }
  ]
}
```

**Response**: `204` Collector set successfully

---

## Export Operations (ExportService)

### `POST /client/{id}/export`

Triggers data export for a client.

**Security**: Requires JWT authentication

**Request Body** (`ExportTrigger`):

```json
{
  "ingestion_filters": {
    "start_date": "2024-01-01T00:00:00Z",
    "end_date": "2024-12-31T23:59:59Z"
  },
  "field_filters": [
    {
      "field_name": "contract_type",
      "condition": "include",
      "values": ["SERVICE", "PRODUCT"]
    }
  ]
}
```

**Filter Conditions**: `less_than`, `greater_than`, `closed_interval`, `open_interval`, `left_closed_interval`, `right_closed_interval`, `include`, `exclude`

**Response** (`201 Created`):

```json
{
  "id": "019580de-4d51-713c-98ee-464e83811f13"
}
```

### `GET /export/{id}`

Checks export status and retrieves download URL.

**Security**: Requires JWT authentication

**Response** (`ExportDetails`):

```json
{
  "client_id": "AAA",
  "status": "completed",
  "output_location": "s3://bucket/AAA/export/20250213/uuid.csv"
}
```

**Export Status Values**: `completed`, `in_progress`, `failed`

---

## User Administration (AdminService)

### `POST /admin/users`

Creates a new user in both AWS Cognito and Permit.io systems with optional role assignments.

**Security**: Requires JWT authentication

**Team Management Permissions**: Super Admin can create any user. User Admin and Client Admin can create only non-AArete users with `client_user` and/or `user_admin` roles. Auditor is read-only and cannot create users.

**Environment Scoping**: When `COGNITO_ENVIRONMENT_ID` is configured on the server, newly created users are automatically tagged with the `custom:environment_id` attribute matching the server's value. This ensures users are scoped to the correct environment.

**Creation Flow**:

1. User is created in AWS Cognito (primary authentication system) with the server's environment ID (if configured)
2. If Cognito creation fails, operation fails with HTTP 502
3. User is created in Permit.io (authorization/role system) with the environment ID as an attribute
4. If Permit.io creation fails, user still exists in Cognito but `permit_synced=false`
5. Roles are assigned in Permit.io (best-effort, continues on failures)

**Request Body** (`AdminUserCreate`):

```json
{
  "email": "john.doe@example.com",
  "first_name": "John",
  "last_name": "Doe",
  "roles": ["user_admin", "auditor"]
}
```

| Field      | Type   | Required | Description                              |
| ---------- | ------ | -------- | ---------------------------------------- |
| email      | string | Yes      | User email address (max 320 chars)       |
| first_name | string | Yes      | User's given name (max 100 chars)        |
| last_name  | string | Yes      | User's family name (max 100 chars)       |
| roles      | array  | Yes      | List of role keys to assign (1-50 roles) |

**Response** (`201 Created` or `200 OK` if user exists):

```json
{
  "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "email": "john.doe@example.com",
  "first_name": "John",
  "last_name": "Doe",
  "status": "FORCE_CHANGE_PASSWORD",
  "enabled": true,
  "permit_synced": true,
  "roles_assigned": ["user_admin", "auditor"],
  "created_at": "2025-10-16T14:23:45Z"
}
```

**Error Responses**:

- `400`: Invalid request data
- `401`: Authentication required
- `403`: Insufficient permissions
- `409`: User already exists with different attributes
- `429`: Rate limit exceeded
- `502`: External service (Cognito) error

### `GET /admin/users`

Lists users from AWS Cognito with pagination, filtering, and sorting support.

**Security**: Requires JWT authentication

**Environment Filtering**: When `COGNITO_ENVIRONMENT_ID` is configured, the response is filtered to include only users whose `custom:environment_id` matches the server's value or users with no environment tag (legacy/untagged users). Users belonging to other environments are excluded from the results.

**Team Management Visibility**: User Admin and Client Admin receive only non-AArete users that do not have `auditor` or `super_admin` roles. Super Admin and Auditor receive all environment-visible users. The `roles` field is populated from Permit.io for list responses.

**Query Parameters**:
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| page | integer | 1 | Page number (1-10000) |
| page_size | integer | 50 | Results per page (1-60) |
| search | string | - | Search term for email/name filtering |
| status | string | all | Filter by status: `enabled`, `disabled`, `all` |
| sort_by | string | email | Sort field: `email`, `created_at`, `last_name` |
| sort_order | string | asc | Sort direction: `asc`, `desc` |

**Response** (`AdminUserList`):

```json
{
  "users": [
    {
      "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "email": "john.doe@example.com",
      "first_name": "John",
      "last_name": "Doe",
      "status": "CONFIRMED",
      "enabled": true,
      "roles": ["admin", "viewer"],
      "created_at": "2025-10-15T09:15:30Z",
      "updated_at": "2025-10-16T14:20:10Z"
    }
  ],
  "total": 147,
  "page": 1,
  "page_size": 50,
  "has_more": true
}
```

### `GET /admin/users/{email}`

Retrieves user details from both AWS Cognito and Permit.io by email address.

**Security**: Requires JWT authentication

**Environment Ownership**: When `COGNITO_ENVIRONMENT_ID` is configured, returns `404 Not Found` for users belonging to a different environment to prevent information leakage. Untagged users and users matching the server's environment are accessible.

**Team Management Visibility**: Hidden users return `404 Not Found` so User Admin and Client Admin callers cannot enumerate AArete, Auditor, or Super Admin users.

**Path Parameters**:

- `email`: URL-encoded user email address (max 320 chars)

**Response** (`AdminUserDetails`): Same structure as user list item

### `HEAD /admin/users/{email}`

Checks if a user exists in Cognito and/or Permit.io without returning full details.

**Security**: Requires JWT authentication

**Environment Ownership**: When `COGNITO_ENVIRONMENT_ID` is configured, returns `404` for users belonging to a different environment.

**Team Management Visibility**: Hidden users return `404 Not Found` so User Admin and Client Admin callers cannot enumerate AArete, Auditor, or Super Admin users.

**Response Headers**:

- `X-User-Exists-Cognito`: boolean
- `X-User-Exists-PermitIO`: boolean

**Response**:

- `200`: User exists (at least in one system)
- `404`: User does not exist in either system

### `PATCH /admin/users/{email}`

Updates user attributes (first_name, last_name) in Cognito and manages role assignments in Permit.io.

**Security**: Requires JWT authentication

**Environment Ownership**: When `COGNITO_ENVIRONMENT_ID` is configured, returns `404 Not Found` for users belonging to a different environment.

**Team Management Permissions**: Super Admin can update any environment-visible user. User Admin and Client Admin can update only non-AArete client-level users and can assign only `client_user` and `user_admin` roles. Auditor is read-only.

**Role Management (REPLACE Strategy)**:

- Providing a `roles` array REPLACES all existing roles (not additive)
- To add a role: include ALL current roles PLUS the new role
- To remove a role: include only the roles you want to keep
- To remove all roles: send an empty array `[]`
- Omitting the roles field leaves role assignments unchanged

**Request Body** (`AdminUserUpdate`):

```json
{
  "first_name": "Jonathan",
  "last_name": "Doe-Smith",
  "roles": ["user_admin", "auditor"]
}
```

**Response** (`200 OK`): Updated `AdminUserDetails`

### `DELETE /admin/users/{email}`

Permanently deletes a user from both AWS Cognito and Permit.io. This operation is irreversible.

**Security**: Requires JWT authentication

**Environment Ownership**: When `COGNITO_ENVIRONMENT_ID` is configured, returns `404 Not Found` for users belonging to a different environment.

**Team Management Permissions**: Super Admin can delete any environment-visible user. User Admin and Client Admin can delete only non-AArete client-level users. Auditor is read-only.

**Query Parameters**:

- `confirm` (required): Must be `true` to confirm deletion

**Response** (`200 OK` or `207 Multi-Status`):

```json
{
  "success": true,
  "email": "john.doe@example.com",
  "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "deleted_from": {
    "cognito": true,
    "permit": true
  },
  "timestamp": "2025-10-16T14:27:15Z"
}
```

### `POST /admin/users/{email}/disable`

Disables a user in AWS Cognito, preventing authentication. User data and roles are preserved. Idempotent.

**Security**: Requires JWT authentication

**Environment Ownership**: When `COGNITO_ENVIRONMENT_ID` is configured, returns `404 Not Found` for users belonging to a different environment.

**Team Management Permissions**: Super Admin can disable any environment-visible user. User Admin and Client Admin can disable only non-AArete client-level users. Auditor is read-only.

**Response** (`AdminUserActionResponse`):

```json
{
  "success": true,
  "email": "john.doe@example.com",
  "action": "disable",
  "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "timestamp": "2025-10-16T14:25:33Z"
}
```

### `POST /admin/users/{email}/enable`

Re-enables a previously disabled user in AWS Cognito. Restores authentication capability. Idempotent.

**Security**: Requires JWT authentication

**Environment Ownership**: When `COGNITO_ENVIRONMENT_ID` is configured, returns `404 Not Found` for users belonging to a different environment.

**Team Management Permissions**: Super Admin can enable any environment-visible user. User Admin and Client Admin can enable only non-AArete client-level users. Auditor is read-only.

**Response**: Same as disable endpoint with `"action": "enable"`

---

## EULA Operations (EulaService)

The EULA (End User License Agreement) service manages EULA versions and tracks user consent. It provides both public endpoints for retrieving and agreeing to EULAs, and administrative endpoints for managing EULA versions and viewing compliance reports.

### Public Endpoints

#### `GET /eula`

Returns the current active EULA version. This endpoint is public and does not require authentication.

**Security**: Public (no authentication required)

**Response** (`EulaPublicResponse`):

```json
{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "version": "1.0",
  "title": "Terms of Service",
  "content": "# Terms of Service\n\nThis is the full EULA content in Markdown format...",
  "effective_date": "2025-01-01T00:00:00Z"
}
```

**Responses**:

- `200`: Current EULA version retrieved successfully
- `404`: No current EULA version configured
- `500`: Internal server error

#### `GET /eula/status`

Returns the current user's agreement status for the current EULA version. Used by the frontend to determine if the user needs to be prompted to agree.

**Security**: Requires JWT authentication

**Response** (`EulaStatusResponse`):

```json
{
  "has_agreed": true,
  "current_version": "1.0",
  "current_version_id": "019580df-ef65-7676-8de9-94435a93337a",
  "agreed_at": "2025-01-15T10:30:00Z",
  "agreed_version": "1.0"
}
```

**Responses**:

- `200`: EULA status retrieved successfully
- `401`: Authentication required
- `404`: No current EULA version configured
- `500`: Internal server error

#### `POST /eula/agree`

Records the user's agreement to the current EULA version. This operation is idempotent - calling it multiple times will not create duplicate records. The IP address is automatically captured from the request.

**Security**: Requires JWT authentication

**Response** (`EulaAgreementResponse`):

```json
{
  "id": "019580df-ef65-7676-8de9-94435a93337b",
  "eula_version_id": "019580df-ef65-7676-8de9-94435a93337a",
  "eula_version": "1.0",
  "agreed_at": "2025-01-15T10:30:00Z"
}
```

**Responses**:

- `201`: Agreement recorded successfully (first agreement)
- `200`: User already agreed to current version (returns existing agreement)
- `400`: No current EULA version configured
- `401`: Authentication required
- `500`: Internal server error

### Administrative Endpoints

#### `GET /admin/eula`

Returns a paginated list of all EULA versions.

**Security**: Requires JWT authentication

**Query Parameters**:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| page | integer | 1 | Page number (1-10000) |
| page_size | integer | 20 | Results per page (1-100) |

**Response** (`EulaVersionList`):

```json
{
  "versions": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "version": "1.0",
      "title": "Terms of Service",
      "content": "# Terms of Service...",
      "effective_date": "2025-01-01T00:00:00Z",
      "is_current": true,
      "created_at": "2024-12-01T00:00:00Z",
      "created_by": "admin@example.com",
      "activated_at": "2025-01-01T00:00:00Z",
      "activated_by": "admin@example.com"
    }
  ],
  "total": 3,
  "has_more": false
}
```

#### `POST /admin/eula`

Creates a new EULA version. The content should be in Markdown format.

**Security**: Requires JWT authentication

**Request Body** (`EulaVersionCreate`):

```json
{
  "version": "2.0",
  "title": "Updated Terms of Service",
  "content": "# Updated Terms of Service\n\nThis is the new EULA content...",
  "effective_date": "2025-06-01T00:00:00Z"
}
```

| Field | Type | Required | Description |
| ----- | ---- | -------- | ----------- |
| version | string | Yes | Unique version identifier (max 50 chars) |
| title | string | Yes | Human-readable title |
| content | string | Yes | Full EULA text in Markdown format |
| effective_date | datetime | Yes | When this version becomes effective |

**Response** (`201 Created`): `EulaVersion` object

**Error Responses**:

- `400`: Invalid request data
- `401`: Authentication required
- `403`: Insufficient permissions
- `409`: Version string already exists

#### `GET /admin/eula/{version_id}`

Retrieves a specific EULA version by its ID.

**Security**: Requires JWT authentication

**Path Parameters**:

- `version_id`: EULA version UUID

**Response** (`EulaVersion`): Full EULA version object

#### `PATCH /admin/eula/{version_id}`

Updates only the metadata (title and effective date) of an EULA version. Content cannot be modified to maintain audit integrity.

**Security**: Requires JWT authentication

**Request Body** (`EulaVersionUpdate`):

```json
{
  "title": "Updated Title",
  "effective_date": "2025-07-01T00:00:00Z"
}
```

**Response** (`200 OK`): Updated `EulaVersion` object

#### `POST /admin/eula/{version_id}/activate`

Sets this version as the current active EULA. Uses a database transaction to atomically clear the previous current flag and set the new one.

**Security**: Requires JWT authentication

**Path Parameters**:

- `version_id`: EULA version UUID to activate

**Response** (`200 OK`): Activated `EulaVersion` object with `is_current=true`

**Error Responses**:

- `404`: Version not found
- `409`: Concurrent activation conflict (another admin activated a different version)

#### `GET /admin/eula/agreements`

Returns a paginated list of user agreements. Use filters to view agreements for specific EULA versions or users.

**Security**: Requires JWT authentication

**Query Parameters**:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| page | integer | 1 | Page number (1-10000) |
| page_size | integer | 20 | Results per page (1-100) |
| version_id | uuid | - | Filter by specific EULA version |
| user_id | string | - | Filter by Cognito subject ID |

**Response** (`EulaAgreementList`):

```json
{
  "agreements": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337b",
      "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "user_email": "user@example.com",
      "eula_version_id": "019580df-ef65-7676-8de9-94435a93337a",
      "eula_version": "1.0",
      "agreed_at": "2025-01-15T10:30:00Z",
      "agreed_from_ip": "192.168.1.1"
    }
  ],
  "total": 150,
  "has_more": true
}
```

#### `GET /admin/eula/compliance`

Returns a comprehensive compliance report showing which users have and have not agreed to a specific EULA version. Defaults to the current active version if no version_id is provided.

**Security**: Requires JWT authentication

**Environment Filtering**: When `COGNITO_ENVIRONMENT_ID` is configured, the compliance report only includes users matching the server's environment or untagged legacy users. Users from other environments are excluded from the total counts and user list.

**Query Parameters**:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| page | integer | 1 | Page number (1-10000) |
| page_size | integer | 50 | Results per page (1-100) |
| version_id | uuid | - | Specific EULA version (defaults to current) |
| agreed | boolean | - | Filter by agreement status |

**Response** (`EulaComplianceReport`):

```json
{
  "version_id": "019580df-ef65-7676-8de9-94435a93337a",
  "version": "1.0",
  "total_users": 200,
  "agreed_count": 150,
  "not_agreed_count": 50,
  "compliance_percentage": 75.0,
  "users": [
    {
      "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "email": "user@example.com",
      "first_name": "John",
      "last_name": "Doe",
      "has_agreed": true,
      "agreed_at": "2025-01-15T10:30:00Z"
    }
  ],
  "page": 1,
  "page_size": 50,
  "has_more": true
}
```

---

## UI Settings Operations (UISettingsService)

The UI Settings service provides a generic key-value store scoped by namespace. Use it for user preferences, feature flags, saved filters, or other UI state. Namespaces scope data (e.g. `demo-dashboard:{userId}`, `feature-flags:global`, `saved-filters:{clientId}`). Records use soft delete (`isDeleted`); list/get exclude soft-deleted rows.

**Security**: All UI settings endpoints require JWT authentication (auth-only, no Permit.io resource check).

#### `GET /ui-settings`

Returns settings where `isDeleted` is false. Optional query param `namespace` filters by namespace.

**Query Parameters**:

| Parameter | Type   | Required | Description                                      |
| --------- | ------ | -------- | ------------------------------------------------ |
| namespace | string | No       | Filter by namespace (omit to list all). Max 255 chars. |

**Response** (`UISettingListResponse`): `{ settings: UISetting[], totalCount: number }`

**Responses**: `200` OK, `400` Bad Request, `401` Unauthorized, `429` Too Many Requests, `500` Internal Server Error

#### `POST /ui-settings`

Creates a new UI setting. Returns 201 with the created resource.

**Request Body** (`UISettingCreate`):

| Field     | Type   | Required | Description                                                                 |
| --------- | ------ | -------- | --------------------------------------------------------------------------- |
| namespace | string | Yes      | Scope for the setting (e.g. demo-dashboard:{userId}, feature-flags:global). Max 255. |
| key       | string | Yes      | Setting key within the namespace. Max 255.                                  |
| value     | object | Yes      | JSON value (object) for the setting.                                        |

**Response** (`UISetting`): Created setting with `id`, `namespace`, `key`, `value`, `isDeleted`, `createdAt`, `updatedAt`

**Responses**: `201` Created, `400` Bad Request, `401` Unauthorized, `429` Too Many Requests, `500` Internal Server Error

#### `GET /ui-settings/{id}`

Returns a single setting by id. Returns 404 if not found or soft-deleted.

**Path Parameters**: `id` (uuid) – UI setting ID

**Response** (`UISetting`): Full setting object

**Responses**: `200` OK, `400` Bad Request, `401` Unauthorized, `404` Not Found, `429` Too Many Requests, `500` Internal Server Error

#### `PATCH /ui-settings/{id}`

Updates the value of an existing setting. Returns 404 if not found or soft-deleted.

**Path Parameters**: `id` (uuid) – UI setting ID

**Request Body** (`UISettingUpdate`): `{ value: object }` – New JSON value for the setting

**Response** (`UISetting`): Updated setting object

**Responses**: `200` OK, `400` Bad Request, `401` Unauthorized, `404` Not Found, `429` Too Many Requests, `500` Internal Server Error

#### `DELETE /ui-settings/{id}`

Soft-deletes a UI setting (sets `isDeleted` to true). Record is retained. Returns 204 on success, 404 if not found.

**Path Parameters**: `id` (uuid) – UI setting ID

**Responses**: `204` No Content, `400` Bad Request, `401` Unauthorized, `404` Not Found, `429` Too Many Requests, `500` Internal Server Error

---

## Schema Management (SuperAdminSchemaService)

All endpoints in this group require the `super_admin` role. Any other role receives `403 Forbidden`. The Permit.io resource is `super-admin`; auditor is granted `get` only.

### `POST /super-admin/custom-schemas`

Creates a new custom schema for a client. The schema definition must be a valid JSON Schema draft 2020-12 object with `additionalProperties` explicitly declared and a root `type` of `object`. Maximum definition size is 64 KB.

**Security**: Requires JWT authentication. Requires `super_admin` role.

**Request Body** (`CustomSchemaRequest`):

```json
{
  "clientId": "AAA",
  "name": "aircraft-engineering",
  "description": "Schema for aircraft inspection documents",
  "schemaDef": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "aircraft_type": { "type": "string", "maxLength": 100 },
      "inspection_date": { "type": "string", "format": "date" }
    },
    "required": ["aircraft_type", "inspection_date"]
  }
}
```

**Response** (`201 Created`, `CustomSchemaResponse`):

```json
{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "clientId": "AAA",
  "name": "aircraft-engineering",
  "description": "Schema for aircraft inspection documents",
  "version": 1,
  "status": "active",
  "schemaDef": { ... },
  "createdAt": "2026-04-14T10:00:00Z",
  "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}
```

**Notes**:
- `createdBy` is populated from the JWT `sub` claim (a Cognito subject UUID), not accepted from the request body.
- `schemaDef` must have `type: "object"` at the root.
- `additionalProperties` must be explicitly `true` or `false`.

**Error Responses**:
- `400`: Invalid or oversized schema definition
- `401`: Authentication required
- `403`: Insufficient permissions
- `409`: A schema with this name already exists for the client (at version 1)

### `GET /super-admin/custom-schemas`

Lists schemas for a client with optional filtering.

**Security**: Requires JWT authentication. Requires `super_admin` role (`auditor` may use GET).

**Query Parameters**:

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| clientId | string | Yes | — | Client ID to filter schemas |
| name | string | No | — | Filter by schema name |
| status | string | No | `active` | Filter by status: `active`, `superseded`, `retired`, `any` |
| includeAllVersions | boolean | No | `false` | When `true`, returns all versions per name; otherwise returns only the latest active version |
| limit | integer | No | 20 | Pagination page size (1–100) |
| offset | integer | No | 0 | Pagination offset |

**Response** (`200 OK`, `CustomSchemaListResponse`):

```json
{
  "schemas": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "clientId": "AAA",
      "name": "aircraft-engineering",
      "version": 2,
      "status": "active",
      "documentCount": 14,
      "canDelete": false,
      "createdAt": "2026-04-14T10:00:00Z",
      "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
    }
  ],
  "totalCount": 1
}
```

**Notes**:
- Default list (no `status`, no `includeAllVersions`) returns only the latest active version per name.
- `canDelete` is `false` when any document has `custom_schema_id` pointing to this schema.
- `documentCount` is the count of documents currently bound to this specific schema version.

### `GET /super-admin/custom-schemas/{schemaId}`

Retrieves a single schema by ID, including the full `schemaDef`.

**Security**: Requires JWT authentication. Requires `super_admin` role (`auditor` may use GET).

**Path Parameters**:

- `schemaId`: Schema UUID

**Response** (`200 OK`, `CustomSchemaResponse`): Full schema object including `schemaDef`.

**Error Responses**:
- `404`: Schema not found

### `POST /super-admin/custom-schemas/{schemaId}/versions`

Creates a new version of an existing schema. The parent schema must be `active`. On success: a new row is inserted with an incremented `version` number and `status=active`; the parent row's `status` is set to `superseded`. Both changes are applied in a single transaction.

**Security**: Requires JWT authentication. Requires `super_admin` role.

**Path Parameters**:

- `schemaId`: UUID of the current active schema (the parent version)

**Request Body** (`CustomSchemaVersionRequest`): Same shape as `CustomSchemaRequest` body (name, description, schemaDef — no `createdBy`).

**Response** (`201 Created`, `CustomSchemaResponse`): The new schema version row.

**Notes**:
- This is a `POST`, not a `PUT` — it creates a new row with a new ID. The old version is preserved and queryable.
- Passing a `schemaId` that is not `active` (e.g. `superseded` or `retired`) returns `409`.
- `createdBy` is derived from JWT, not the request body.

**Error Responses**:
- `400`: Invalid schema definition
- `404`: Parent schema not found
- `409`: Parent schema is not active (already superseded or retired)

### `DELETE /super-admin/custom-schemas/{schemaId}`

Retires a schema. Sets `status` to `retired`. This operation is reversible only via a new version.

**Security**: Requires JWT authentication. Requires `super_admin` role.

**Path Parameters**:

- `schemaId`: Schema UUID

**Response**: `204 No Content`

**Error Responses**:
- `404`: Schema not found
- `409`: One or more documents are currently bound to this schema (`documentCount > 0`)

### `PATCH /super-admin/documents/{id}/schema`

Assigns or clears a custom schema on a document. Once a document has custom metadata written to it, the schema binding is frozen — this endpoint will return `409` if metadata exists. To upgrade a schema, first use `POST .../reset-metadata`.

**Security**: Requires JWT authentication. Requires `super_admin` role.

**Path Parameters**:

- `id`: Document UUID

**Request Body** (`DocumentSchemaAssignRequest`):

```json
{
  "schemaId": "019580df-ef65-7676-8de9-94435a93337a"
}
```

Pass `"schemaId": null` to clear an existing schema binding (allowed only when no metadata exists).

**Response** (`200 OK`, `DocumentSchemaAssignResponse`):

```json
{
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "customSchemaId": "019580df-ef65-7676-8de9-94435a93337a",
  "schemaName": "aircraft-engineering",
  "schemaVersion": 2
}
```

**Error Responses**:
- `400`: Schema is not active or document already has legacy field extractions
- `404`: Document or schema not found
- `409`: Document already has custom metadata (schema binding is frozen)

### `POST /super-admin/documents/{id}/reset-metadata`

Atomically deletes all custom metadata rows for a document and nullifies `custom_schema_id`. This is the required first step when upgrading a document to a newer schema version. The operation is irreversible — metadata history is lost.

**Security**: Requires JWT authentication. Requires `super_admin` role.

**Path Parameters**:

- `id`: Document UUID

**Request Body**: None.

**Response** (`200 OK`, `DocumentMetadataResetResponse`):

```json
{
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "previousSchemaId": "019580df-ef65-7676-8de9-94435a93337a",
  "previousSchemaName": "aircraft-engineering",
  "previousSchemaVersion": 1,
  "metadataVersionsDeleted": 3,
  "resetAt": "2026-04-15T09:30:00Z",
  "resetBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}
```

This endpoint is idempotent. If the document has no schema binding and no metadata, `metadataVersionsDeleted` is `0` and all `previous*` fields are `null`.

**Error Responses**:
- `404`: Document not found
- `409`: Document has legacy field extractions (mutual exclusivity — cannot reset a legacy document)

---

## Custom Metadata Operations (CustomMetadataService)

These endpoints are available to `client_user`, `user_admin`, and `super_admin` for both read and write. `auditor` has read-only access (GET endpoints only). The Permit.io resource is `custom-metadata`.

### `POST /custom-metadata`

Writes a new metadata version for a document. The document must already have a custom schema assigned. The metadata payload is validated against the bound schema. Each call appends a new version; the latest version is what `GET /custom-metadata` returns.

**Security**: Requires JWT authentication. Requires `client_user`, `user_admin`, or `super_admin` role.

**Request Body** (`CustomMetadataRequest`):

```json
{
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "metadata": {
    "aircraft_type": "Boeing 737",
    "inspection_date": "2026-04-01",
    "flight_hours": 12450.5
  }
}
```

Maximum payload size is 1 MB.

**Response** (`201 Created`, `CustomMetadataResponse`):

```json
{
  "id": "019580df-ef65-7676-8de9-94435a93337c",
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "schemaId": "019580df-ef65-7676-8de9-94435a93337a",
  "schemaName": "aircraft-engineering",
  "schemaVersion": 2,
  "version": 4,
  "metadata": { ... },
  "createdAt": "2026-04-15T10:00:00Z",
  "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}
```

**Notes**:
- `createdBy` is derived from JWT, not the request body.
- Returns `400` with validation errors if the payload does not conform to the schema.

**Error Responses**:
- `400`: Document has no schema assigned, payload exceeds 1 MB, or payload fails schema validation
- `409`: Document has legacy field extractions (mutually exclusive with custom metadata)

### `GET /custom-metadata`

Returns the current (latest) metadata version for a document.

**Security**: Requires JWT authentication. Requires `client_user`, `user_admin`, `super_admin`, or `auditor` role.

**Query Parameters**:

- `documentId` (required): Document UUID

**Response** (`200 OK`, `CustomMetadataResponse`): The latest version row.

**Error Responses**:
- `404`: Document not found or no metadata exists for the document

### `GET /custom-metadata/version`

Returns a specific historical metadata version for a document.

**Security**: Requires JWT authentication. Requires `client_user`, `user_admin`, `super_admin`, or `auditor` role.

**Query Parameters**:

| Parameter | Type | Required | Description |
|---|---|---|---|
| documentId | uuid | Yes | Document UUID |
| version | integer | Yes | Version number (>= 1) |

**Response** (`200 OK`, `CustomMetadataResponse`): The requested version row.

**Error Responses**:
- `400`: `version` is less than 1
- `404`: Document not found or the specified version does not exist

### `GET /custom-metadata/history`

Returns a paginated list of metadata version summaries for a document, ordered by version descending (newest first). Summaries include `version`, `createdAt`, and `createdBy` but not the full metadata payload.

**Security**: Requires JWT authentication. Requires `client_user`, `user_admin`, `super_admin`, or `auditor` role.

**Query Parameters**:

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| documentId | uuid | Yes | — | Document UUID |
| limit | integer | No | 20 | Number of versions to return (1–200) |
| offset | integer | No | 0 | Pagination offset (>= 0) |

**Response** (`200 OK`, `CustomMetadataHistoryResponse`):

```json
{
  "versions": [
    {
      "version": 3,
      "createdAt": "2026-04-15T12:00:00Z",
      "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
    },
    {
      "version": 2,
      "createdAt": "2026-04-14T15:00:00Z",
      "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
    }
  ]
}
```

**Error Responses**:
- `400`: `limit` is out of range (must be 1–200) or `offset` is negative
- `404`: Document not found

**Notes**:
- If the document exists but has no metadata, returns `200` with an empty `versions` array.
- `limit=201` is clamped to 200.

---

## GET /document/{id} — Custom Metadata Enrichment

`GET /document/{id}` (documented under [Document Operations](#document-operations-documentsservice)) now returns two additional fields on the `DocumentEnriched` response:

| Field | Type | Description |
|---|---|---|
| `customSchemaId` | uuid or null | UUID of the custom schema currently bound to the document. Null if no schema has been assigned. |
| `hasCustomMetadata` | boolean | `true` if at least one custom metadata version exists for the document. |

These fields are present on every `GET /document/{id}` response regardless of whether the caller has any custom metadata role grants. A document that has never been assigned a schema returns `customSchemaId: null, hasCustomMetadata: false`.

---

## Request/Response Schemas

### Standard Response Headers

All API responses include:

- `RateLimit`: Current rate limit status (format: `100;window=60`)
- `Content-Type`: `application/json` (except file downloads and HTML pages)

### Error Response Format

```json
{
  "message": "Error description (max 256 chars)"
}
```

### Common HTTP Status Codes

| Code | Description                                  |
| ---- | -------------------------------------------- |
| 200  | Success                                      |
| 201  | Created                                      |
| 202  | Accepted (async processing)                  |
| 204  | No Content (success with no body)            |
| 207  | Multi-Status (partial success)               |
| 302  | Redirect                                     |
| 400  | Bad Request (validation errors)              |
| 401  | Unauthorized (authentication required)       |
| 403  | Forbidden (insufficient permissions)         |
| 404  | Not Found                                    |
| 409  | Conflict (resource exists or state conflict) |
| 429  | Too Many Requests (rate limited)             |
| 500  | Internal Server Error                        |
| 502  | Bad Gateway (external service error)         |

---

## API Versioning

### Current Strategy

- **URL-based versioning**: `/v1` prefix
- **Backward Compatibility**: Maintained within major versions
- **OpenAPI Version**: 0.0.1 (development phase)

### Future Considerations

- Semantic versioning for production releases
- Deprecation notices for breaking changes
- Version sunset policies

---

## Authorization Reference (Permit.io Integration)

The API uses Permit.io for fine-grained authorization. This section documents what data is sent to Permit.io for each endpoint, enabling system administrators to configure the correct resources, actions, and roles in Permit.io.

### Authorization Flow

1. **Authentication**: User authenticates via AWS Cognito, receiving a JWT token
2. **User Identification**: The `sub` (subject) claim from the JWT is extracted as the user identifier
3. **Resource Mapping**: The first path segment of the URL becomes the resource (e.g., `/client/{id}/document` → `client`)
4. **Action Mapping**: The HTTP method is converted to lowercase to become the action (e.g., `GET` → `get`)
5. **Permission Check**: Permit.io checks if the user has permission to perform the action on the resource

### Permit.io Data Mapping Logic

**Resource Extraction** (from `internal/cognitoauth/mapping.go`):

- The resource is derived from the **first segment** of the URL path
- Leading slashes are removed
- Example: `/client/{id}/document/batch/{batch_id}` → resource: `client`

**Action Extraction**:

- HTTP method is converted to lowercase
- `GET` → `get`, `POST` → `post`, `PATCH` → `patch`, `DELETE` → `delete`, `HEAD` → `head`

### Endpoints Exempt from Authorization

The following endpoints skip authentication and Permit.io authorization entirely:

| Endpoint         | Purpose                           |
| ---------------- | --------------------------------- |
| `GET /home`      | Public home/menu page             |
| `GET /logout`    | Session termination               |
| `GET /health`    | Health check endpoint             |
| `GET /swagger/*` | Swagger UI documentation          |
| `GET /metrics`   | Prometheus metrics                |
| `GET /eula`      | Public EULA version retrieval     |

### Special Authentication Endpoints

These endpoints handle authentication flows and bypass normal authorization:

| Endpoint              | Purpose                                       |
| --------------------- | --------------------------------------------- |
| `GET /login`          | Initiates OAuth2/PKCE login flow with Cognito |
| `GET /login-callback` | Handles OAuth2 callback from Cognito          |

### Auth-Only Endpoints (JWT required, no Permit.io check)

The following endpoints require a valid JWT token but do **not** check Permit.io permissions. Any authenticated user can access them:

| Endpoint           | Purpose                                          |
| ------------------ | ------------------------------------------------ |
| `GET /identity`    | Users need to discover their roles               |
| `GET /eula/status` | Users need to check their EULA agreement status  |
| `POST /eula/agree` | Users need to be able to agree to the EULA       |
| `GET /ui-settings` | List UI settings (optional namespace filter)      |
| `POST /ui-settings` | Create UI setting                                |
| `GET /ui-settings/{id}` | Get UI setting by ID                         |
| `PATCH /ui-settings/{id}` | Update UI setting value                     |
| `DELETE /ui-settings/{id}` | Soft-delete UI setting                      |

### Authorization Reference Table

The following table shows what data is passed to Permit.io for each protected endpoint:

| HTTP Method                     | Endpoint                                 | Resource            | Action   | Notes                   |
| ------------------------------- | ---------------------------------------- | ------------------- | -------- | ----------------------- |
| **Client Management**           |
| `POST`                          | `/client`                                | `client`            | `post`   | Create new client       |
| `GET`                           | `/clients`                               | `clients`           | `get`    | List all clients        |
| `GET`                           | `/client/{id}`                           | `client`            | `get`    | Get client details      |
| `PATCH`                         | `/client/{id}`                           | `client`            | `patch`  | Update client           |
| `DELETE`                        | `/client/{id}`                           | `client`            | `delete` | Hard-delete client (super_admin only) |
| `GET`                           | `/client/{id}/status`                    | `client`            | `get`    | Get client sync status  |
| **Document Operations**         |
| `POST`                          | `/client/{id}/document`                  | `client`            | `post`   | Upload document         |
| `GET`                           | `/client/{id}/document`                  | `client`            | `get`    | List client documents   |
| `GET`                           | `/document/{id}`                         | `document`          | `get`    | Get document details    |
| `GET`                           | `/document/{id}/download`                | `document`          | `get`    | Download document file  |
| `DELETE`                        | `/document/{id}`                         | `document`          | `delete` | Hard-delete document (super_admin only) |
| **Batch Document Operations**   |
| `POST`                          | `/client/{id}/document/batch`            | `client`            | `post`   | Upload batch (ZIP)      |
| `GET`                           | `/client/{id}/document/batch`            | `client`            | `get`    | List batch uploads      |
| `GET`                           | `/client/{id}/document/batch/{batch_id}` | `client`            | `get`    | Get batch status        |
| **Folder Operations**           |
| `GET`                           | `/client/{id}/folders`                   | `client`            | `get`    | List client folders     |
| `POST`                          | `/folders`                               | `folders`           | `post`   | Create folder           |
| `PATCH`                         | `/folders/{folderId}`                    | `folders`           | `patch`  | Rename folder           |
| `DELETE`                        | `/folders/{folderId}`                    | `folders`           | `delete` | Hard-delete folder tree (super_admin only) |
| `GET`                           | `/folders/{folderId}/documents`          | `folders`           | `get`    | List folder documents   |
| `GET`                           | `/folders/{folderId}/metrics`            | `folders`           | `get`    | Get folder metrics      |
| **Label Operations**            |
| `POST`                          | `/documents/{documentId}/labels`         | `documents`         | `post`   | Apply label to document |
| `GET`                           | `/documents/{documentId}/labels`         | `documents`         | `get`    | Get document labels     |
| `GET`                           | `/labels/{labelName}/documents`          | `labels`            | `get`    | Get documents by label  |
| **Field Extraction Operations** |
| `POST`                          | `/field-extractions`                     | `field-extractions` | `post`   | Create field extraction |
| `GET`                           | `/field-extractions`                     | `field-extractions` | `get`    | Get latest extraction   |
| `GET`                           | `/field-extractions/version`             | `field-extractions` | `get`    | Get specific version    |
| `GET`                           | `/field-extractions/history`             | `field-extractions` | `get`    | Get extraction history  |
| **Collector Configuration**     |
| `GET`                           | `/client/{id}/collector`                 | `client`            | `get`    | Get collector config    |
| `PATCH`                         | `/client/{id}/collector`                 | `client`            | `patch`  | Update collector config |
| **Export Operations**           |
| `POST`                          | `/client/{id}/export`                    | `client`            | `post`   | Trigger export          |
| `GET`                           | `/export/{id}`                           | `export`            | `get`    | Get export status       |
| **User Administration**         |
| `POST`                          | `/admin/users`                           | `admin`             | `post`   | Create user             |
| `GET`                           | `/admin/users`                           | `admin`             | `get`    | List users              |
| `GET`                           | `/admin/users/{email}`                   | `admin`             | `get`    | Get user details        |
| `HEAD`                          | `/admin/users/{email}`                   | `admin`             | `head`   | Check user existence    |
| `PATCH`                         | `/admin/users/{email}`                   | `admin`             | `patch`  | Update user             |
| `DELETE`                        | `/admin/users/{email}`                   | `admin`             | `delete` | Delete user             |
| `POST`                          | `/admin/users/{email}/disable`           | `admin`             | `post`   | Disable user            |
| `POST`                          | `/admin/users/{email}/enable`            | `admin`             | `post`   | Enable user             |
| **EULA Operations**             |
| `GET`                           | `/eula`                                  | -                   | -        | Public (no auth)        |
| `GET`                           | `/eula/status`                           | -                   | -        | Auth-only (no Permit.io)|
| `POST`                          | `/eula/agree`                            | -                   | -        | Auth-only (no Permit.io)|
| `GET`                           | `/admin/eula`                            | `admin`             | `get`    | List EULA versions      |
| `POST`                          | `/admin/eula`                            | `admin`             | `post`   | Create EULA version     |
| `GET`                           | `/admin/eula/{version_id}`               | `admin`             | `get`    | Get EULA version        |
| `PATCH`                         | `/admin/eula/{version_id}`               | `admin`             | `patch`  | Update EULA metadata    |
| `POST`                          | `/admin/eula/{version_id}/activate`      | `admin`             | `post`   | Activate EULA version   |
| `GET`                           | `/admin/eula/agreements`                 | `admin`             | `get`    | List EULA agreements    |
| `GET`                           | `/admin/eula/compliance`                 | `admin`             | `get`    | Get compliance report   |
| **UI Settings Operations**     |
| `GET`                           | `/ui-settings`                           | -                   | -        | Auth-only (no Permit.io)|
| `POST`                          | `/ui-settings`                           | -                   | -        | Auth-only (no Permit.io)|
| `GET`                           | `/ui-settings/{id}`                      | -                   | -        | Auth-only (no Permit.io)|
| `PATCH`                         | `/ui-settings/{id}`                      | -                   | -        | Auth-only (no Permit.io)|
| `DELETE`                        | `/ui-settings/{id}`                      | -                   | -        | Auth-only (no Permit.io)|
| **Schema Management (SuperAdminSchemaService)** |
| `POST`                          | `/super-admin/custom-schemas`                     | `super-admin` | `post`   | Create new schema (super_admin only) |
| `GET`                           | `/super-admin/custom-schemas`                     | `super-admin` | `get`    | List schemas with filters            |
| `GET`                           | `/super-admin/custom-schemas/{schemaId}`           | `super-admin` | `get`    | Get a schema by ID                   |
| `POST`                          | `/super-admin/custom-schemas/{schemaId}/versions`  | `super-admin` | `post`   | Create new version (supersedes parent) |
| `DELETE`                        | `/super-admin/custom-schemas/{schemaId}`           | `super-admin` | `delete` | Retire schema (409 if documents bound) |
| **Document Schema Assignment (SuperAdminSchemaService)** |
| `PATCH`                         | `/super-admin/documents/{id}/schema`               | `super-admin` | `patch`  | Assign (or clear) custom schema for a document |
| `POST`                          | `/super-admin/documents/{id}/reset-metadata`       | `super-admin` | `post`   | Wipe custom metadata history and clear schema binding |
| **Custom Metadata Operations (CustomMetadataService)** |
| `GET`                           | `/custom-metadata`                                 | `custom-metadata` | `get`  | Get current (latest) metadata version for a document |
| `POST`                          | `/custom-metadata`                                 | `custom-metadata` | `post` | Write new validated metadata version for a document |
| `GET`                           | `/custom-metadata/version`                         | `custom-metadata` | `get`  | Get a specific historical metadata version |
| `GET`                           | `/custom-metadata/history`                         | `custom-metadata` | `get`  | List metadata version summaries (paged, descending) |

### Permit.io Configuration Requirements

To configure Permit.io to work with this API, administrators must create:

#### Resources

| Resource Key        | Description                                    |
| ------------------- | ---------------------------------------------- |
| `client`            | Client management and client-scoped operations |
| `clients`           | Client listing (plural form)                   |
| `document`          | Individual document operations                 |
| `documents`         | Document collection and label operations       |
| `folders`           | Folder management                              |
| `labels`            | Label-based document queries                   |
| `field-extractions` | Field extraction operations                    |
| `export`            | Export operations                              |
| `admin`             | User administration and EULA admin operations  |
| `super-admin`       | Schema management — gates all `/super-admin/custom-schemas` routes (super_admin CRUD; auditor read-only) |
| `custom-metadata`   | Custom document metadata operations (all roles except auditor can read and write; auditor read-only) |

#### Actions

| Action Key | HTTP Method | Description              |
| ---------- | ----------- | ------------------------ |
| `get`      | GET         | Read/retrieve resources  |
| `post`     | POST        | Create resources         |
| `patch`    | PATCH       | Update resources         |
| `delete`   | DELETE      | Delete resources         |
| `head`     | HEAD        | Check resource existence |

#### Role Configuration

The following roles are defined in the Permit.io configuration (`cmd/auth_related/permit.setup/permit_policies.yaml`):

| Role           | Resources                                                                              | Actions                          | Use Case                                |
| -------------- | -------------------------------------------------------------------------------------- | -------------------------------- | --------------------------------------- |
| `super_admin`  | All resources                                                                          | All actions including `delete`   | Full administrative access              |
| `user_admin`   | `admin`, `documents`, `field-extractions`, `folders`, `custom-metadata`                | `get`, `post`, `patch`, `delete` (admin only); `get`, `post` (custom-metadata) | User management |
| `auditor`      | `admin`, `client`, `clients`, `document`, `export`, `documents`, `field-extractions`, `folders`, `labels`, `super-admin`, `custom-metadata` | `get` (and `head` for admin) | Read-only access for auditing |
| `client_user`  | `client`, `clients`, `document`, `export`, `documents`, `field-extractions`, `folders`, `labels`, `custom-metadata` | `get`, `post`, `patch` (varies by resource); `get`, `post` (custom-metadata) | Client-specific access |

**Delete permissions**: Only `super_admin` has `delete` access to `client`, `document`, and `folders` resources. The `client_user` role does NOT have `client:delete` or any other delete permissions on these resources.

**Schema management permissions** (`super-admin` and `custom-metadata` resources):

| Role          | `super-admin` resource              | `custom-metadata` resource |
| ------------- | ----------------------------------- | -------------------------- |
| `super_admin` | `get`, `post`, `patch`, `delete`    | `get`, `post`              |
| `user_admin`  | none                                | `get`, `post`              |
| `auditor`     | `get`                               | `get`                      |
| `client_user` | none                                | `get`, `post`              |

The `super-admin` resource gates all `/super-admin/custom-schemas` CRUD endpoints. A `user_admin` has no `super-admin` grant and receives 403 on every schema management route.

### Authorization Error Responses

When authorization fails, the API returns:

**HTTP 401 Unauthorized** - No valid JWT token:

```json
{
  "error": "Authorization required",
  "message": "This endpoint requires authentication. Please obtain a valid JWT token.",
  "login_url": "/login"
}
```

**HTTP 403 Forbidden** - Insufficient permissions:

```json
{
  "error": "Insufficient permissions",
  "message": "Access denied by authorization service",
  "user": "<cognito-subject-id>",
  "resource": "<resource-name>",
  "action": "<action-name>"
}
```

---

## Mutable Metadata v4 — Design Decisions

The following decisions shaped the Milestone 1 schema CRUD implementation and apply to all subsequent milestones.

- **`POST .../versions` instead of `PUT`**: A `PUT` route implies in-place replacement, which would overwrite the existing row and lose the ability to audit what the previous definition was. `POST /super-admin/custom-schemas/{schemaId}/versions` creates a new row with a new `id`, increments the `version` counter, and marks the parent row as `superseded` — all in a single transaction. The old row is preserved permanently and remains queryable.

- **`createdBy` is not in request bodies**: The actor identity comes from the JWT `sub` claim (the Cognito subject UUID), extracted server-side by the handler. Accepting `createdBy` in the request body would allow any caller to impersonate a different actor. The `sub` claim is cryptographically bound to the token and cannot be spoofed by the client.

- **Composite FK replaces the v3 trigger**: v3 used a trigger (`trg_validate_schema_client_match`) to prevent a document from being bound to a schema owned by a different client. v4 replaces this with a composite foreign key `fk_documents_custom_schema_same_client` on `(custom_schema_id, clientId)` referencing `client_metadata_schemas(id, client_id)`. Structural enforcement fires before any application code runs, catches the violation even on direct SQL updates, and requires no ongoing maintenance.

- **`additionalProperties` must be explicitly declared**: JSON Schema's default behavior when `additionalProperties` is absent is to allow all additional properties. Requiring an explicit declaration forces the schema author to make a deliberate choice and prevents accidental data loss due to schema drift.

- **`documentFieldExtractions` and custom metadata are mutually exclusive**: A document may either have legacy field extractions or custom schema metadata — never both. This is enforced at three layers: service-layer `FOR UPDATE` parent-row locks, a Postgres trigger (`trg_prevent_schema_reassignment`), and a second trigger (`trg_prevent_legacy_extraction_on_custom_document`). Defense-in-depth ensures the invariant holds even if one layer is bypassed.

---

## Code Generation

### OpenAPI Integration

The API is generated from the OpenAPI specification using:

```bash
oapi-codegen -generate "types,server" -o api.gen.go serviceAPIs/queryAPI.yaml
```

### Generated Artifacts

- Type-safe Go structs for requests/responses
- Echo framework route handlers
- OpenAPI validation middleware
- Swagger documentation

### Validation

- Automatic request validation against OpenAPI schema
- Response validation in development mode
- Parameter binding with type safety