Files
query-orchestration/docs/ai.generated/03-api-documentation.md
T
Jacob Mathison e3d9047143 Merged in jmathison/DEVOPS-722 (pull request #226)
DEVOPS-722 enforce team management visibility

* DEVOPS-722 enforce team management visibility

* DEVOPS-722 reduce admin user list complexity

* DEVOPS-722 cover admin access helpers

* DEVOPS-722 address team management PR feedback

* DEVOPS-722 fix admin access tests in CI

* DEVOPS-722 fix remaining admin PR feedback
2026-05-12 16:49:12 +00:00

91 KiB
Raw Blame History

API Documentation

API Overview

The DoczyAI Query Orchestration Platform exposes a comprehensive REST API following OpenAPI 3.0.3 specification. The API is automatically generated from the specification file serviceAPIs/queryAPI.yaml using oapi-codegen. also see query api summary

Base Configuration

  • Base URL: https://doczy.com/v1 (production), http://localhost:8080 (development)
  • API Version: 0.0.1
  • Content Type: application/json
  • Documentation: Auto-generated Swagger UI at /swagger/index.html

API Services

The API is organized into the following service groups:

Service Description
ClientService Operations related to clients
CollectorService Operations related to collectors
DocumentsService Operations related to documents
FolderService Operations related to document folders and organization
LabelService Operations related to document labels and workflow tracking
FieldExtractionService Operations related to document field extractions
ExportService Operations related to exports
AuthService Operations related to authentication
AdminService Operations related to user management and administration
EulaService Operations related to End User License Agreement management and tracking
UISettingsService Operations related to UI-scoped key-value settings (per user, client, or global)
SuperAdminSchemaService Operations related to custom schema management (super_admin only)
CustomMetadataService Operations related to per-document custom metadata (client_user and above)
BotService Chatbot session/scope/turn endpoints plus read-only document schema and all-metadata bundles for client_user. See Chatbot Guide for the complete reference.
SuperAdminBotService Super-admin chatbot session list/get/soft-delete across all users in a client. See Chatbot Guide.

Chatbot Endpoints

The chatbot endpoints expose a synchronous facade over the Aaria FastAPI agent service. Plan §3 mandates the singular /client/... form so the existing Permit.io client_user policy applies. See Chatbot Guide for the operator runbook, configuration reference, AddTurn algorithm, and HTTP status mapping.

Method Path OperationId Auth
POST /client/{clientId}/bot/sessions createBotSession client_user
GET /client/{clientId}/bot/sessions listBotSessions client_user
GET /client/{clientId}/bot/sessions/{sessionId} getBotSession client_user (owner)
PATCH /client/{clientId}/bot/sessions/{sessionId}/scope patchBotSessionScope client_user (owner)
POST /client/{clientId}/bot/sessions/{sessionId}/turns createBotTurn client_user (owner)
GET /client/{clientId}/bot/sessions/{sessionId}/turns listBotTurns client_user (owner)
GET /client/{clientId}/documents/{documentId}/schema getDocumentSchema client_user
GET /client/{clientId}/documents/{documentId}/all-metadata getDocumentAllMetadata client_user
GET /super-admin/bot/sessions?clientId=... listBotSessionsAsSuperAdmin super_admin
GET /super-admin/bot/sessions/{sessionId}?clientId=... getBotSessionAsSuperAdmin super_admin
DELETE /super-admin/bot/sessions/{sessionId}?clientId=... deleteBotSessionAsSuperAdmin super_admin

BotSessionResponse.lastTurn is the derived last_terminal_ordinal from bot_turns (highest ordinal whose status is not in_flight), NOT the highest seen ordinal. last_seen_ordinal is computed but used only internally by AddTurn for ordinal classification.

DocumentSchemaResponse.schema is omitted from the JSON wire response when the document has no custom_schema_id binding (the field is declared omitempty); the document still exists, only the optional binding is absent. Plan §10 M5: HTTP 200 in this case, NOT 404.

The all-metadata endpoint emits BotLabelRecord (plain-string appliedBy) instead of the existing LabelRecord (email-validated) for the embedded labels list. Both wrap the same documentLabels.appliedBy varchar column; the chatbot variant tolerates non-email values that pre-date the email-typing convention. Long-term cleanup is to relax LabelRecord.AppliedBy to plain string.

Authentication and Authorization

Security Schemes

JWT Authentication

  • Type: Bearer token authentication
  • Header: Authorization: Bearer <token>
  • Token Source: JWT cookie set via OAuth flow

OAuth2 Flow

  • Type: OAuth2 Authorization Code Flow
  • Provider: AWS Cognito
  • Scopes: openid, email, profile

Authentication Endpoints

GET /login

Initiates OAuth2 authentication flow with AWS Cognito.

Security: Public (no authentication required) Response: 302 Redirect to Cognito login page

GET /login-callback

OAuth2 callback handler that processes authentication response.

Security: Public Parameters:

  • code (query, required): Authorization code from Cognito (max 2048 chars)
  • state (query, required): CSRF protection parameter (max 1024 chars)

Response: 302 Sets JWT cookie and redirects to application

GET /logout

Terminates user session and clears authentication cookies.

Security: Public (no authentication required) Response: 302 Redirect to Cognito logout page or application home

GET /home

Returns the HTML menu page.

Security: Public (no authentication required) Response: 200 HTML content

GET /identity

Returns the authenticated user's assigned roles from Permit.io, including the permissions granted by each role. Any authenticated user can call this endpoint to discover their own roles. This endpoint requires JWT authentication but does not require specific Permit.io permissions.

Security: Requires JWT authentication (auth-only, no Permit.io check)

Response (IdentityResponse):

{
  "roles": [
    {
      "key": "auditor",
      "name": "Auditor",
      "description": "Read-only access to view documents and exports",
      "permissions": ["document:read", "document:list", "export:read"]
    }
  ]
}

Responses:

  • 200: User roles and permissions retrieved successfully
  • 401: Authentication required
  • 429: Rate limit exceeded
  • 502: Failed to retrieve roles from Permit.io

Rate Limiting

The queryAPI service implements dual-layer rate limiting to protect against DDoS attacks and ensure fair resource allocation.

Rate Limit Architecture

Dual-Layer Protection:

  • Layer 1: Global/Aggregate Rate Limiting - Protects total system capacity across all IP addresses
  • Layer 2: Per-IP Rate Limiting - Prevents individual client abuse

Both layers must pass for a request to proceed. The global limiter checks first for faster failure paths under DDoS conditions.

Default Rate Limits

By default, all endpoints are limited to:

  • Global: 500 requests per second total (1000 burst)
  • Per-IP: 5 requests per second per IP (10 burst)
  • Cleanup Interval: 3 minutes for inactive IP limiters

Important: No endpoints are exempt from rate limiting. This is a security requirement to prevent DDoS attacks.

Configuration

Rate limits are configured via environment variables:

# Global/Aggregate rate limiting (protects total system capacity)
RATE_LIMIT_GLOBAL_RATE=500         # total req/s across ALL IPs
RATE_LIMIT_GLOBAL_BURST=1000        # total burst capacity across ALL IPs

# Per-IP rate limiting (protects against individual abusers)
RATE_LIMIT_DEFAULT_RATE=5          # req/s per IP
RATE_LIMIT_DEFAULT_BURST=10         # burst per IP
RATE_LIMIT_EXPIRES_IN=180           # seconds (3 minutes)

# Per-endpoint overrides (JSON format).
# Plan §10 M6 + §11: setting an override here NOW throttles the actual
# per-(IP, route) request rate (Allow() decision), not just the
# RateLimit / Retry-After response headers. Operators upgrading from
# pre-M6 deployments should review existing overrides for unintended
# tightening.
#
# Keys use Echo's path template form (`:clientId`, `:sessionId`), NOT
# the OpenAPI `{clientId}` form. The middleware uses c.Path() which
# returns Echo's registered path string.
#
# Example for the chatbot turn endpoint (illustrative — adjust before
# production rollout):
#   RATE_LIMIT_ENDPOINT_OVERRIDES='{"POST /client/:clientId/bot/sessions/:sessionId/turns":{"global_rate":50,"global_burst":100,"rate":2,"burst":4}}'
RATE_LIMIT_ENDPOINT_OVERRIDES={}

Rate Limit Headers

All API responses include a RateLimit header indicating the current limit:

RateLimit: 20;window=2

Format: {burst};window={seconds}

Rate Limit Exceeded Response

HTTP Status: 429 Too Many Requests

Headers:

  • Retry-After: {seconds} - Number of seconds before retrying
  • RateLimit: {burst};window={seconds} - Current rate limit

Response Body:

{
  "message": "rate limit exceeded"
}

Client Management (ClientService)

GET /clients

Lists all clients in the system.

Security: Requires JWT authentication

Response (ClientListResponse):

{
  "clients": [
    {
      "id": "AAA",
      "name": "ACME Corporation",
      "can_sync": true
    }
  ],
  "totalCount": 1
}

POST /client

Creates a new client in the system.

Security: Requires JWT authentication

Request Body (ClientCreate):

{
  "id": "string (max 36 chars, required)",
  "name": "string (max 256 chars, required)"
}

Responses:

  • 201: Client created successfully
    {
      "id": "AAA"
    }
    
  • 400: Invalid request data
  • 401: Authentication required
  • 429: Rate limit exceeded
  • 500: Internal server error

GET /client/{id}

Retrieves client details by ID.

Security: Requires JWT authentication

Path Parameters:

  • id: Client identifier (string, max 36 chars)

Response (DocClient):

{
  "id": "AAA",
  "name": "ACME Corporation",
  "can_sync": true
}

PATCH /client/{id}

Updates client information.

Security: Requires JWT authentication

Request Body (ClientUpdate):

{
  "name": "string (max 256 chars, optional)",
  "can_sync": "boolean (optional)"
}

Response: 200 Client updated successfully

DELETE /client/{id}

Permanently deletes a client and all associated data: documents (with all dependent rows), folders, collector configurations, batch uploads, and clientCanSync records. Source documents in S3 are NOT deleted; their paths are logged and optionally returned. This operation is irreversible.

Security: Requires JWT authentication. Restricted to super_admin role only.

Query Parameters:

  • confirm (required, boolean): Must be true to proceed. Returns 400 Bad Request if absent or false.
  • verbose (optional, boolean, default: false): When true, returns S3 paths of orphaned source documents.

Response:

  • 204: Client deleted successfully (when verbose=false)
  • 200: Client deleted successfully with S3 paths (when verbose=true):
{
  "deletedDocuments": [
    {
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "s3Path": "s3://bucket-name/key/path/to/file"
    }
  ]
}
  • 400: Missing confirm=true query parameter
  • 404: Client not found

GET /client/{id}/status

Returns client synchronization status.

Security: Requires JWT authentication

Response (ClientStatusBody):

{
  "status": "IN_SYNC"
}

Status Values: IN_SYNC, NOT_SYNCED, NOT_SYNCING


Document Operations (DocumentsService)

POST /client/{id}/document

Uploads a document for a specific client.

Security: Requires JWT authentication Content Type: multipart/form-data Max File Size: 100GB

Form Parameters:

  • file (required): Document file. Accepted types: PDF, TIFF, JPEG, PNG, BMP, DOCX, XLSX, PPTX, TXT, EML
  • filename (optional): Custom filename (max 4096 chars)

Response: 200 Document uploaded

GET /client/{id}/document

Lists documents for a client.

Security: Requires JWT authentication

Response (ListDocuments):

[
  {
    "id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
    "hash": "sha256:abc123..."
  }
]

GET /document/{id}

Retrieves enriched document details by ID, including metadata such as folder, filename, labels, and optionally the full text extraction record.

Security: Requires JWT authentication

Path Parameters:

  • id: Document UUID (max 36 chars)

Query Parameters:

  • textRecord (optional, default: false): When true, includes the full text extraction record in the response

Response (DocumentEnriched):

{
  "id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "client_id": "AAA",
  "hash": "sha256:abc123...",
  "hasTextRecord": true,
  "folderId": "019580df-ef65-7676-8de9-94435a93337a",
  "filename": "contract_2024.pdf",
  "originalPath": "/documents/2024/contract_2024.pdf",
  "labels": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337b",
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "label": "OCR_Processed",
      "appliedBy": "user@example.com",
      "appliedAt": "2025-10-16T14:27:15Z"
    }
  ],
  "fileSizeBytes": 1048576
}

GET /document/{id}/download

Streams the original uploaded document file directly from S3. The response Content-Type is auto-detected from the stored filename extension.

Security: Requires JWT authentication

Path Parameters:

  • id: Document UUID (max 36 chars)

Response (200): Raw file bytes (streamed)

  • Content-Type: Auto-detected from filename (e.g., application/pdf, image/png, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
  • Content-Disposition: attachment; filename="<original_filename>"
  • Content-Length: File size in bytes

Error Responses:

  • 400: Invalid document ID format
  • 401: Missing or invalid JWT token
  • 404: Document not found or no S3 entry for the document
  • 500: S3 retrieval failure

Example:

curl -H "Authorization: Bearer <token>" \
     -o report.pdf \
     http://localhost:8080/document/019580df-ef65-7676-8de9-94435a93337a/download

Notes:

  • No query parameters needed -- file type is automatically determined from the stored filename
  • Response streams directly from S3, suitable for large files
  • Returns 404 if the document has no stored file (e.g., legacy documents without S3 entries)
  • Supports any file type without code changes (PDF, PNG, DOCX, CSV, etc.)

DELETE /document/{id}

Permanently deletes a document and all dependent data: document entries, cleans (and their entries), text extractions (and their entries), field extractions (versions and array fields), labels, results, and result dependencies. Source documents in S3 are NOT deleted; their paths are logged and optionally returned.

Security: Requires JWT authentication. Restricted to super_admin role only.

Path Parameters:

  • id: Document UUID (max 36 chars)

Query Parameters:

  • verbose (optional, boolean, default: false): When true, returns S3 paths of orphaned source documents.

Response:

  • 204: Document deleted successfully (when verbose=false)
  • 200: Document deleted successfully with S3 paths (when verbose=true):
{
  "deletedDocuments": [
    {
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "s3Path": "s3://bucket-name/key/path/to/file"
    }
  ]
}
  • 404: Document not found

Batch Document Operations (DocumentsService)

POST /client/{id}/document/batch

Uploads a ZIP archive containing multiple documents for asynchronous batch processing.

Security: Requires JWT authentication Content Type: multipart/form-data Max Archive Size: 1GB Supported Archive Type: ZIP only

Accepted File Extensions: .pdf, .tiff, .tif, .jpeg, .jpg, .png, .bmp, .docx, .xlsx, .pptx, .txt, .eml

Note: Macro-enabled Office formats (.docm, .xlsm, .pptm) are intentionally excluded because documents containing macros are rejected during validation.

Form Parameters:

  • archive (required): ZIP file containing documents

Response (202 Accepted):

{
  "batch_id": "019580df-ef65-7676-8de9-94435a93337a",
  "status": "processing",
  "status_url": "/client/AAA/document/batch/019580df-ef65-7676-8de9-94435a93337a"
}

GET /client/{id}/document/batch

Lists batch uploads for a client with pagination support.

Security: Requires JWT authentication

Query Parameters:

  • limit: Maximum results (integer, 1-100, default: 20)
  • offset: Pagination offset (integer, default: 0)

Response (BatchUploadList):

{
  "batches": [
    {
      "batch_id": "019580df-ef65-7676-8de9-94435a93337a",
      "client_id": "AAA",
      "original_filename": "documents.zip",
      "status": "completed",
      "total_documents": 100,
      "processed_documents": 100,
      "failed_documents": 2,
      "invalid_type_documents": 1,
      "progress_percent": 100,
      "created_at": "2024-01-01T00:00:00Z",
      "completed_at": "2024-01-01T00:05:00Z"
    }
  ],
  "total_count": 25
}

Batch Status Values: processing, completed, failed, cancelled

GET /client/{id}/document/batch/{batch_id}

Retrieves detailed status information for a specific batch upload, including per-file outcome tracking through the document processing pipeline.

Security: Requires JWT authentication

Path Parameters:

  • batch_id: Batch upload identifier (UUID)

Response (BatchUploadDetails):

{
  "batch_id": "019580df-ef65-7676-8de9-94435a93337a",
  "client_id": "AAA",
  "original_filename": "documents.zip",
  "status": "completed",
  "total_documents": 5,
  "processed_documents": 3,
  "failed_documents": 1,
  "invalid_type_documents": 1,
  "progress_percent": 100,
  "created_at": "2024-01-01T00:00:00Z",
  "completed_at": "2024-01-01T00:05:00Z",
  "failed_filenames": ["corrupt.pdf"],
  "document_outcomes": [
    {
      "filename": "invoice.pdf",
      "outcome": "clean_passed",
      "document_id": "019580e0-1111-7676-8de9-000000000001",
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:04:50Z"
    },
    {
      "filename": "receipt.pdf",
      "outcome": "clean_passed",
      "document_id": "019580e0-2222-7676-8de9-000000000002",
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:04:52Z"
    },
    {
      "filename": "duplicate_invoice.pdf",
      "outcome": "duplicate",
      "document_id": "019580e0-1111-7676-8de9-000000000001",
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:00:01Z"
    },
    {
      "filename": "corrupt.pdf",
      "outcome": "clean_failed",
      "error_detail": "PDF validation failed",
      "document_id": "019580e0-3333-7676-8de9-000000000003",
      "clean_fail_reason": "invalid PDF structure: missing xref table",
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:04:55Z"
    },
    {
      "filename": "macro_enabled.xlsm",
      "outcome": "invalid_type",
      "error_detail": null,
      "document_id": null,
      "created_at": "2024-01-01T00:00:01Z",
      "updated_at": "2024-01-01T00:00:01Z"
    }
  ]
}

The document_outcomes array provides per-file tracking through the entire processing pipeline. Each file in the ZIP archive gets one outcome entry, starting at extraction time and updating as the file progresses through the document runners (init, sync, clean). When all outcomes reach a terminal state, the batch is fully processed.

Document Outcome Values:

Outcome Meaning Terminal?
submitted File extracted and uploaded, waiting for processing No
duplicate File hash matches an existing document for this client Yes
failed_open Could not open file from ZIP archive Yes
failed_read Could not read file content from ZIP archive Yes
failed_s3_upload Could not upload extracted file to S3 Yes
failed_upload Upload handler returned an error Yes
invalid_type File type is not in the accepted extensions list Yes
init_complete Document record created, forwarded to sync No
init_duplicate Duplicate detected during init processing Yes
sync_complete Sync check passed, forwarded to clean No
sync_skipped Client sync is disabled Yes
clean_passed Document passed all validation checks Yes
clean_failed Document failed validation (see clean_fail_reason) Yes

Outcome Fields:

Field Type Description
filename string Original filename from the ZIP archive
outcome string Current processing status (see table above)
error_detail string or null Human-readable error message for failure outcomes
document_id uuid or null Assigned document ID (null for files that never became documents)
clean_fail_reason string or null Specific validation failure reason when outcome is clean_failed
created_at datetime When the file was first extracted from the ZIP
updated_at datetime When the outcome was last updated by a pipeline stage

Folder Operations (FolderService)

Folder Path Constraints

All folder paths must comply with the following constraints when creating or renaming folders:

  • Must start with /: All paths must begin with a forward slash
  • Maximum path length: 1000 characters
  • Maximum segment length: 255 characters per path segment
  • Maximum depth: 20 levels of nesting
  • Valid characters only: Alphanumeric (a-z, A-Z, 0-9), hyphen (-), underscore (_), period (.), and space ( )
  • Segment must start with alphanumeric: Each path segment must begin with a letter or number
  • No empty segments: Double slashes (//) or trailing slashes are not allowed
  • No path traversal: Sequences like .. or ./ are blocked
  • No hidden folders: Segments cannot start with a period (e.g., /.hidden)
  • No leading/trailing whitespace: Spaces within segment names are allowed, but not at the start or end
  • No control characters: ASCII control characters (0x00-0x1F, 0x7F) are blocked
  • No Windows reserved names: Names like CON, PRN, AUX, NUL, COM1-COM9, LPT1-LPT9 are blocked
  • Root folder restrictions: The root path / can only exist once per client and cannot be renamed
  • Path uniqueness: Each path must be unique within a client (enforced by database)
  • Unicode normalization: Paths are normalized to NFC form

Valid path examples:

  • /documents
  • /documents/2024/Q1
  • /client-data/reports_2024
  • /My Documents/Archive

Invalid path examples:

  • documents (missing leading /)
  • /documents/ (trailing slash)
  • /foo//bar (empty segment)
  • /foo/../bar (path traversal)
  • /.hidden (dot prefix)
  • /foo@bar (invalid character)
  • /CON (Windows reserved name)

GET /clients/{clientId}/folders

Lists all folders for a specific client, including the folder hierarchy.

Security: Requires JWT authentication

Path Parameters:

  • clientId: Client external ID (string, max 36 chars)

Query Parameters:

  • metrics (optional, default: false): When true, includes processing metrics for each folder (document counts by label). Adds overhead so should only be used when metrics are needed.

Response (200 OK):

{
  "folders": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "path": "/",
      "parentId": null,
      "clientId": "AAA",
      "createdAt": "2025-10-16T14:27:15Z",
      "createdBy": "system"
    },
    {
      "id": "019580df-ef65-7676-8de9-94435a93337b",
      "path": "/documents",
      "parentId": "019580df-ef65-7676-8de9-94435a93337a",
      "clientId": "AAA",
      "createdAt": "2025-10-16T14:30:00Z",
      "createdBy": "user@example.com"
    },
    {
      "id": "019580df-ef65-7676-8de9-94435a93337c",
      "path": "/documents/2024",
      "parentId": "019580df-ef65-7676-8de9-94435a93337b",
      "clientId": "AAA",
      "createdAt": "2025-10-16T14:35:00Z",
      "createdBy": "user@example.com"
    }
  ]
}

Notes:

  • Returns all folders for the client, including nested folders
  • Folders are returned in a flat list; use parentId to reconstruct the hierarchy
  • The root folder (path="/") is included if it exists

POST /folders

Creates a new folder for organizing documents.

Security: Requires JWT authentication

Request Body (FolderCreate):

{
  "path": "/documents/2024",
  "clientId": "AAA",
  "createdBy": "user@example.com",
  "parentId": "019580df-ef65-7676-8de9-94435a93337a"
}
Field Type Required Description
path string Yes Folder path, must start with "/" (max 1000 chars)
clientId string Yes Client external ID
createdBy string Yes Email of user creating the folder
parentId uuid No Parent folder ID for nested folders

Response (201 Created):

{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "path": "/documents/2024",
  "parentId": null,
  "clientId": "AAA",
  "createdAt": "2025-10-16T14:27:15Z",
  "createdBy": "user@example.com"
}

PATCH /folders/{folderId}

Renames an existing folder.

Security: Requires JWT authentication

Path Parameters:

  • folderId: Folder UUID

Request Body (FolderRename):

{
  "path": "/documents/2024-renamed"
}

Response (200 OK): Updated folder object

DELETE /folders/{folderId}

Permanently deletes a folder and all child folders recursively. The root folder (/) cannot be deleted. If the folder tree contains documents, the caller must either delete the documents first or pass include_documents=true.

Security: Requires JWT authentication. Restricted to super_admin role only.

Path Parameters:

  • folderId: Folder UUID

Query Parameters:

  • include_documents (optional, boolean, default: false): When true, deletes all documents in the folder tree along with the folders. When false and the folder tree contains documents, returns 409 Conflict.
  • verbose (optional, boolean, default: false): When true, returns S3 paths of orphaned source documents.

Response:

  • 204: Folder tree deleted successfully (when verbose=false)
  • 200: Folder tree deleted successfully with S3 paths (when verbose=true):
{
  "deletedDocuments": [
    {
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "s3Path": "s3://bucket-name/key/path/to/file"
    }
  ]
}
  • 404: Folder not found
  • 409: Cannot delete root folder, or folder tree contains documents and include_documents is not true

GET /folders/{folderId}/documents

Retrieves all documents within a specific folder, including filename and labels. Returns a lighter-weight format than GET /document/{id} (excludes client_id).

Security: Requires JWT authentication

Query Parameters:

  • textRecord (optional, default: false): When true, includes the full text extraction record for each document

Response:

{
  "documents": [
    {
      "id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "hash": "sha256:abc123...",
      "hasTextRecord": true,
      "folderId": "019580df-ef65-7676-8de9-94435a93337a",
      "filename": "contract_2024.pdf",
      "originalPath": "/documents/2024/contract_2024.pdf",
      "labels": [],
      "fileSizeBytes": 1048576
    }
  ]
}

GET /folders/{folderId}/metrics

Retrieves processing progress metrics for documents in a folder.

Security: Requires JWT authentication

Response (FolderMetrics):

{
  "folderId": "019580df-ef65-7676-8de9-94435a93337a",
  "totalDocuments": 150,
  "byLabel": {
    "Ingested": 150,
    "OCR_Processed": 120,
    "Dashboard_Ready": 100
  }
}

Label Operations (LabelService)

POST /documents/{documentId}/labels

Applies a workflow label to a document for tracking processing state.

Security: Requires JWT authentication

Path Parameters:

  • documentId: Document UUID

Request Body (LabelApplication):

{
  "label": "OCR_Processed",
  "appliedBy": "user@example.com"
}
Field Type Required Description
label string Yes Label name (alphanumeric and underscore only, max 100 chars)
appliedBy string Yes Email of user applying the label

Response (201 Created):

{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "label": "OCR_Processed",
  "appliedBy": "user@example.com",
  "appliedAt": "2025-10-16T14:27:15Z"
}

GET /documents/{documentId}/labels

Retrieves all labels that have been applied to a document, ordered by most recent first.

Security: Requires JWT authentication

Response:

{
  "labels": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "label": "OCR_Processed",
      "appliedBy": "user@example.com",
      "appliedAt": "2025-10-16T14:27:15Z"
    }
  ]
}

GET /labels/{labelName}/documents

Retrieves all documents that have been tagged with a specific label.

Security: Requires JWT authentication

Path Parameters:

  • labelName: Label name (alphanumeric and underscore only, max 100 chars)

Query Parameters:

  • clientId (required): Client ID to filter documents (string, max 36 chars)

Response:

{
  "documents": [
    {
      "id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "hash": "sha256:abc123..."
    }
  ]
}

Field Extraction Operations (FieldExtractionService)

POST /field-extractions

Creates a new field extraction with single-value and array fields for a document.

Security: Requires JWT authentication

Request Body (FieldExtractionRequest):

{
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "singleFields": {
    "fileName": "contract_2024.pdf",
    "contractTitle": "Service Agreement",
    "clientName": "ACME Corp",
    "payerName": "Insurance Co",
    "payerState": "CA",
    "providerState": "NY",
    "aareteDerivedEffectiveDt": "2024-01-01",
    "aareteDerivedTerminationDt": "2025-12-31",
    "autoRenewalInd": true
  },
  "arrayFields": [
    {
      "exhibitTitle": "Exhibit A",
      "exhibitPage": "15",
      "reimbProvTin": "123456789",
      "reimbProvNpi": "1234567890",
      "reimbProvName": "Provider Name",
      "reimbEffectiveDt": "2024-01-01",
      "aareteDerivedClaimTypeCd": "INPATIENT",
      "aareteDerivedReimbMethod": "Per Diem",
      "reimbPctRate": 85.5,
      "reimbFeeRate": 150.0
    }
  ],
  "createdBy": "user@example.com"
}

Single Fields (1:1 relationship with document):

  • fileName, contractTitle, clientName, payerName
  • payerState, providerState (2-letter state codes)
  • filenameTin, provGroupTin, provGroupNpi, provGroupNameFull
  • provOtherTin, provOtherNpi, provOtherNameFull
  • aareteDerivedEffectiveDt, aareteDerivedTerminationDt (dates)
  • aareteDerivedAmendmentNum (integer)
  • autoRenewalInd (boolean), autoRenewalTerm

Array Fields (1:N relationship - multiple rows per document):

  • Provider info: reimbProvTin, reimbProvNpi, reimbProvName
  • Dates: reimbEffectiveDt, reimbTerminationDt
  • Classifications: aareteDerivedClaimTypeCd, aareteDerivedProduct, aareteDerivedLob, aareteDerivedProgram, aareteDerivedNetwork, aareteDerivedProvType
  • Codes: provTaxonomyCd, provSpecialtyCd, placeOfServiceCd, billTypeCd, cpt4ProcCd, revenueCd, diagCd, ndcCd
  • Rates: reimbPctRate, reimbFeeRate, reimbConversionFactor, grouperPctRate, grouperBaseRate
  • Indicators: carveoutInd, lesserOfInd, greaterOfInd, defaultInd

Response (201 Created):

{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "version": 1,
  "singleFields": { ... },
  "arrayFields": [ ... ],
  "createdAt": "2025-10-16T14:27:15Z",
  "createdBy": "user@example.com"
}

GET /field-extractions

Retrieves the most recent field extraction for a document.

Security: Requires JWT authentication

Query Parameters:

  • documentId (required): Document UUID

Response: FieldExtractionResponse (same as POST response)

GET /field-extractions/history

Retrieves all versions of field extractions for a document, ordered by most recent first.

Security: Requires JWT authentication

Query Parameters:

  • documentId (required): Document UUID

Response:

{
  "versions": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "version": 2,
      "createdAt": "2025-10-17T10:00:00Z",
      "createdBy": "user@example.com"
    },
    {
      "id": "019580df-ef65-7676-8de9-94435a93337b",
      "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
      "version": 1,
      "createdAt": "2025-10-16T14:27:15Z",
      "createdBy": "user@example.com"
    }
  ]
}

GET /field-extractions/version

Retrieves a specific version of a field extraction for a document by version number.

Security: Requires JWT authentication

Query Parameters:

  • documentId (required): Document UUID
  • version (required): Version number (integer, minimum 1)

Response (200 OK):

{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "version": 1,
  "singleFields": {
    "fileName": "contract_2024.pdf",
    "contractTitle": "Service Agreement",
    "clientName": "ACME Corp"
  },
  "arrayFields": [],
  "createdAt": "2025-10-16T14:27:15Z",
  "createdBy": "user@example.com"
}

Error Responses:

  • 400: Missing required query parameters
  • 404: Version not found for the specified document

Collector Configuration (CollectorService)

GET /client/{id}/collector

Retrieves collector configuration for a client.

Security: Requires JWT authentication

Response (Collector):

{
  "client_id": "AAA",
  "active_version": 1,
  "latest_version": 2,
  "minimum_cleaner_version": 1,
  "minimum_text_version": 1,
  "fields": [
    {
      "name": "contract_date",
      "query_id": "019580df-ef65-7676-8de9-94435a93337a"
    }
  ]
}

PATCH /client/{id}/collector

Updates collector configuration.

Security: Requires JWT authentication

Request Body (CollectorSet):

{
  "minimum_cleaner_version": 2,
  "minimum_text_version": 2,
  "active_version": 1,
  "fields": [
    {
      "name": "contract_date",
      "query_id": "019580df-ef65-7676-8de9-94435a93337a"
    }
  ]
}

Response: 204 Collector set successfully


Export Operations (ExportService)

POST /client/{id}/export

Triggers data export for a client.

Security: Requires JWT authentication

Request Body (ExportTrigger):

{
  "ingestion_filters": {
    "start_date": "2024-01-01T00:00:00Z",
    "end_date": "2024-12-31T23:59:59Z"
  },
  "field_filters": [
    {
      "field_name": "contract_type",
      "condition": "include",
      "values": ["SERVICE", "PRODUCT"]
    }
  ]
}

Filter Conditions: less_than, greater_than, closed_interval, open_interval, left_closed_interval, right_closed_interval, include, exclude

Response (201 Created):

{
  "id": "019580de-4d51-713c-98ee-464e83811f13"
}

GET /export/{id}

Checks export status and retrieves download URL.

Security: Requires JWT authentication

Response (ExportDetails):

{
  "client_id": "AAA",
  "status": "completed",
  "output_location": "s3://bucket/AAA/export/20250213/uuid.csv"
}

Export Status Values: completed, in_progress, failed


User Administration (AdminService)

POST /admin/users

Creates a new user in both AWS Cognito and Permit.io systems with optional role assignments.

Security: Requires JWT authentication

Team Management Permissions: Super Admin can create any user. User Admin and Client Admin can create only non-AArete users with client_user and/or user_admin roles. Auditor is read-only and cannot create users.

Environment Scoping: When COGNITO_ENVIRONMENT_ID is configured on the server, newly created users are automatically tagged with the custom:environment_id attribute matching the server's value. This ensures users are scoped to the correct environment.

Creation Flow:

  1. User is created in AWS Cognito (primary authentication system) with the server's environment ID (if configured)
  2. If Cognito creation fails, operation fails with HTTP 502
  3. User is created in Permit.io (authorization/role system) with the environment ID as an attribute
  4. If Permit.io creation fails, user still exists in Cognito but permit_synced=false
  5. Roles are assigned in Permit.io (best-effort, continues on failures)

Request Body (AdminUserCreate):

{
  "email": "john.doe@example.com",
  "first_name": "John",
  "last_name": "Doe",
  "roles": ["user_admin", "auditor"]
}
Field Type Required Description
email string Yes User email address (max 320 chars)
first_name string Yes User's given name (max 100 chars)
last_name string Yes User's family name (max 100 chars)
roles array Yes List of role keys to assign (1-50 roles)

Response (201 Created or 200 OK if user exists):

{
  "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "email": "john.doe@example.com",
  "first_name": "John",
  "last_name": "Doe",
  "status": "FORCE_CHANGE_PASSWORD",
  "enabled": true,
  "permit_synced": true,
  "roles_assigned": ["user_admin", "auditor"],
  "created_at": "2025-10-16T14:23:45Z"
}

Error Responses:

  • 400: Invalid request data
  • 401: Authentication required
  • 403: Insufficient permissions
  • 409: User already exists with different attributes
  • 429: Rate limit exceeded
  • 502: External service (Cognito) error

GET /admin/users

Lists users from AWS Cognito with pagination, filtering, and sorting support.

Security: Requires JWT authentication

Environment Filtering: When COGNITO_ENVIRONMENT_ID is configured, the response is filtered to include only users whose custom:environment_id matches the server's value or users with no environment tag (legacy/untagged users). Users belonging to other environments are excluded from the results.

Team Management Visibility: User Admin and Client Admin receive only non-AArete users that do not have auditor or super_admin roles. Super Admin and Auditor receive all environment-visible users. The roles field is populated from Permit.io for list responses.

Query Parameters:

Parameter Type Default Description
page integer 1 Page number (1-10000)
page_size integer 50 Results per page (1-60)
search string - Search term for email/name filtering
status string all Filter by status: enabled, disabled, all
sort_by string email Sort field: email, created_at, last_name
sort_order string asc Sort direction: asc, desc

Response (AdminUserList):

{
  "users": [
    {
      "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "email": "john.doe@example.com",
      "first_name": "John",
      "last_name": "Doe",
      "status": "CONFIRMED",
      "enabled": true,
      "roles": ["admin", "viewer"],
      "created_at": "2025-10-15T09:15:30Z",
      "updated_at": "2025-10-16T14:20:10Z"
    }
  ],
  "total": 147,
  "page": 1,
  "page_size": 50,
  "has_more": true
}

GET /admin/users/{email}

Retrieves user details from both AWS Cognito and Permit.io by email address.

Security: Requires JWT authentication

Environment Ownership: When COGNITO_ENVIRONMENT_ID is configured, returns 404 Not Found for users belonging to a different environment to prevent information leakage. Untagged users and users matching the server's environment are accessible.

Team Management Visibility: Hidden users return 404 Not Found so User Admin and Client Admin callers cannot enumerate AArete, Auditor, or Super Admin users.

Path Parameters:

  • email: URL-encoded user email address (max 320 chars)

Response (AdminUserDetails): Same structure as user list item

HEAD /admin/users/{email}

Checks if a user exists in Cognito and/or Permit.io without returning full details.

Security: Requires JWT authentication

Environment Ownership: When COGNITO_ENVIRONMENT_ID is configured, returns 404 for users belonging to a different environment.

Team Management Visibility: Hidden users return 404 Not Found so User Admin and Client Admin callers cannot enumerate AArete, Auditor, or Super Admin users.

Response Headers:

  • X-User-Exists-Cognito: boolean
  • X-User-Exists-PermitIO: boolean

Response:

  • 200: User exists (at least in one system)
  • 404: User does not exist in either system

PATCH /admin/users/{email}

Updates user attributes (first_name, last_name) in Cognito and manages role assignments in Permit.io.

Security: Requires JWT authentication

Environment Ownership: When COGNITO_ENVIRONMENT_ID is configured, returns 404 Not Found for users belonging to a different environment.

Team Management Permissions: Super Admin can update any environment-visible user. User Admin and Client Admin can update only non-AArete client-level users and can assign only client_user and user_admin roles. Auditor is read-only.

Role Management (REPLACE Strategy):

  • Providing a roles array REPLACES all existing roles (not additive)
  • To add a role: include ALL current roles PLUS the new role
  • To remove a role: include only the roles you want to keep
  • To remove all roles: send an empty array []
  • Omitting the roles field leaves role assignments unchanged

Request Body (AdminUserUpdate):

{
  "first_name": "Jonathan",
  "last_name": "Doe-Smith",
  "roles": ["user_admin", "auditor"]
}

Response (200 OK): Updated AdminUserDetails

DELETE /admin/users/{email}

Permanently deletes a user from both AWS Cognito and Permit.io. This operation is irreversible.

Security: Requires JWT authentication

Environment Ownership: When COGNITO_ENVIRONMENT_ID is configured, returns 404 Not Found for users belonging to a different environment.

Team Management Permissions: Super Admin can delete any environment-visible user. User Admin and Client Admin can delete only non-AArete client-level users. Auditor is read-only.

Query Parameters:

  • confirm (required): Must be true to confirm deletion

Response (200 OK or 207 Multi-Status):

{
  "success": true,
  "email": "john.doe@example.com",
  "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "deleted_from": {
    "cognito": true,
    "permit": true
  },
  "timestamp": "2025-10-16T14:27:15Z"
}

POST /admin/users/{email}/disable

Disables a user in AWS Cognito, preventing authentication. User data and roles are preserved. Idempotent.

Security: Requires JWT authentication

Environment Ownership: When COGNITO_ENVIRONMENT_ID is configured, returns 404 Not Found for users belonging to a different environment.

Team Management Permissions: Super Admin can disable any environment-visible user. User Admin and Client Admin can disable only non-AArete client-level users. Auditor is read-only.

Response (AdminUserActionResponse):

{
  "success": true,
  "email": "john.doe@example.com",
  "action": "disable",
  "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "timestamp": "2025-10-16T14:25:33Z"
}

POST /admin/users/{email}/enable

Re-enables a previously disabled user in AWS Cognito. Restores authentication capability. Idempotent.

Security: Requires JWT authentication

Environment Ownership: When COGNITO_ENVIRONMENT_ID is configured, returns 404 Not Found for users belonging to a different environment.

Team Management Permissions: Super Admin can enable any environment-visible user. User Admin and Client Admin can enable only non-AArete client-level users. Auditor is read-only.

Response: Same as disable endpoint with "action": "enable"


EULA Operations (EulaService)

The EULA (End User License Agreement) service manages EULA versions and tracks user consent. It provides both public endpoints for retrieving and agreeing to EULAs, and administrative endpoints for managing EULA versions and viewing compliance reports.

Public Endpoints

GET /eula

Returns the current active EULA version. This endpoint is public and does not require authentication.

Security: Public (no authentication required)

Response (EulaPublicResponse):

{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "version": "1.0",
  "title": "Terms of Service",
  "content": "# Terms of Service\n\nThis is the full EULA content in Markdown format...",
  "effective_date": "2025-01-01T00:00:00Z"
}

Responses:

  • 200: Current EULA version retrieved successfully
  • 404: No current EULA version configured
  • 500: Internal server error

GET /eula/status

Returns the current user's agreement status for the current EULA version. Used by the frontend to determine if the user needs to be prompted to agree.

Security: Requires JWT authentication

Response (EulaStatusResponse):

{
  "has_agreed": true,
  "current_version": "1.0",
  "current_version_id": "019580df-ef65-7676-8de9-94435a93337a",
  "agreed_at": "2025-01-15T10:30:00Z",
  "agreed_version": "1.0"
}

Responses:

  • 200: EULA status retrieved successfully
  • 401: Authentication required
  • 404: No current EULA version configured
  • 500: Internal server error

POST /eula/agree

Records the user's agreement to the current EULA version. This operation is idempotent - calling it multiple times will not create duplicate records. The IP address is automatically captured from the request.

Security: Requires JWT authentication

Response (EulaAgreementResponse):

{
  "id": "019580df-ef65-7676-8de9-94435a93337b",
  "eula_version_id": "019580df-ef65-7676-8de9-94435a93337a",
  "eula_version": "1.0",
  "agreed_at": "2025-01-15T10:30:00Z"
}

Responses:

  • 201: Agreement recorded successfully (first agreement)
  • 200: User already agreed to current version (returns existing agreement)
  • 400: No current EULA version configured
  • 401: Authentication required
  • 500: Internal server error

Administrative Endpoints

GET /admin/eula

Returns a paginated list of all EULA versions.

Security: Requires JWT authentication

Query Parameters:

Parameter Type Default Description
page integer 1 Page number (1-10000)
page_size integer 20 Results per page (1-100)

Response (EulaVersionList):

{
  "versions": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "version": "1.0",
      "title": "Terms of Service",
      "content": "# Terms of Service...",
      "effective_date": "2025-01-01T00:00:00Z",
      "is_current": true,
      "created_at": "2024-12-01T00:00:00Z",
      "created_by": "admin@example.com",
      "activated_at": "2025-01-01T00:00:00Z",
      "activated_by": "admin@example.com"
    }
  ],
  "total": 3,
  "has_more": false
}

POST /admin/eula

Creates a new EULA version. The content should be in Markdown format.

Security: Requires JWT authentication

Request Body (EulaVersionCreate):

{
  "version": "2.0",
  "title": "Updated Terms of Service",
  "content": "# Updated Terms of Service\n\nThis is the new EULA content...",
  "effective_date": "2025-06-01T00:00:00Z"
}
Field Type Required Description
version string Yes Unique version identifier (max 50 chars)
title string Yes Human-readable title
content string Yes Full EULA text in Markdown format
effective_date datetime Yes When this version becomes effective

Response (201 Created): EulaVersion object

Error Responses:

  • 400: Invalid request data
  • 401: Authentication required
  • 403: Insufficient permissions
  • 409: Version string already exists

GET /admin/eula/{version_id}

Retrieves a specific EULA version by its ID.

Security: Requires JWT authentication

Path Parameters:

  • version_id: EULA version UUID

Response (EulaVersion): Full EULA version object

PATCH /admin/eula/{version_id}

Updates only the metadata (title and effective date) of an EULA version. Content cannot be modified to maintain audit integrity.

Security: Requires JWT authentication

Request Body (EulaVersionUpdate):

{
  "title": "Updated Title",
  "effective_date": "2025-07-01T00:00:00Z"
}

Response (200 OK): Updated EulaVersion object

POST /admin/eula/{version_id}/activate

Sets this version as the current active EULA. Uses a database transaction to atomically clear the previous current flag and set the new one.

Security: Requires JWT authentication

Path Parameters:

  • version_id: EULA version UUID to activate

Response (200 OK): Activated EulaVersion object with is_current=true

Error Responses:

  • 404: Version not found
  • 409: Concurrent activation conflict (another admin activated a different version)

GET /admin/eula/agreements

Returns a paginated list of user agreements. Use filters to view agreements for specific EULA versions or users.

Security: Requires JWT authentication

Query Parameters:

Parameter Type Default Description
page integer 1 Page number (1-10000)
page_size integer 20 Results per page (1-100)
version_id uuid - Filter by specific EULA version
user_id string - Filter by Cognito subject ID

Response (EulaAgreementList):

{
  "agreements": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337b",
      "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "user_email": "user@example.com",
      "eula_version_id": "019580df-ef65-7676-8de9-94435a93337a",
      "eula_version": "1.0",
      "agreed_at": "2025-01-15T10:30:00Z",
      "agreed_from_ip": "192.168.1.1"
    }
  ],
  "total": 150,
  "has_more": true
}

GET /admin/eula/compliance

Returns a comprehensive compliance report showing which users have and have not agreed to a specific EULA version. Defaults to the current active version if no version_id is provided.

Security: Requires JWT authentication

Environment Filtering: When COGNITO_ENVIRONMENT_ID is configured, the compliance report only includes users matching the server's environment or untagged legacy users. Users from other environments are excluded from the total counts and user list.

Query Parameters:

Parameter Type Default Description
page integer 1 Page number (1-10000)
page_size integer 50 Results per page (1-100)
version_id uuid - Specific EULA version (defaults to current)
agreed boolean - Filter by agreement status

Response (EulaComplianceReport):

{
  "version_id": "019580df-ef65-7676-8de9-94435a93337a",
  "version": "1.0",
  "total_users": 200,
  "agreed_count": 150,
  "not_agreed_count": 50,
  "compliance_percentage": 75.0,
  "users": [
    {
      "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "email": "user@example.com",
      "first_name": "John",
      "last_name": "Doe",
      "has_agreed": true,
      "agreed_at": "2025-01-15T10:30:00Z"
    }
  ],
  "page": 1,
  "page_size": 50,
  "has_more": true
}

UI Settings Operations (UISettingsService)

The UI Settings service provides a generic key-value store scoped by namespace. Use it for user preferences, feature flags, saved filters, or other UI state. Namespaces scope data (e.g. demo-dashboard:{userId}, feature-flags:global, saved-filters:{clientId}). Records use soft delete (isDeleted); list/get exclude soft-deleted rows.

Security: All UI settings endpoints require JWT authentication (auth-only, no Permit.io resource check).

GET /ui-settings

Returns settings where isDeleted is false. Optional query param namespace filters by namespace.

Query Parameters:

Parameter Type Required Description
namespace string No Filter by namespace (omit to list all). Max 255 chars.

Response (UISettingListResponse): { settings: UISetting[], totalCount: number }

Responses: 200 OK, 400 Bad Request, 401 Unauthorized, 429 Too Many Requests, 500 Internal Server Error

POST /ui-settings

Creates a new UI setting. Returns 201 with the created resource.

Request Body (UISettingCreate):

Field Type Required Description
namespace string Yes Scope for the setting (e.g. demo-dashboard:{userId}, feature-flags:global). Max 255.
key string Yes Setting key within the namespace. Max 255.
value object Yes JSON value (object) for the setting.

Response (UISetting): Created setting with id, namespace, key, value, isDeleted, createdAt, updatedAt

Responses: 201 Created, 400 Bad Request, 401 Unauthorized, 429 Too Many Requests, 500 Internal Server Error

GET /ui-settings/{id}

Returns a single setting by id. Returns 404 if not found or soft-deleted.

Path Parameters: id (uuid) UI setting ID

Response (UISetting): Full setting object

Responses: 200 OK, 400 Bad Request, 401 Unauthorized, 404 Not Found, 429 Too Many Requests, 500 Internal Server Error

PATCH /ui-settings/{id}

Updates the value of an existing setting. Returns 404 if not found or soft-deleted.

Path Parameters: id (uuid) UI setting ID

Request Body (UISettingUpdate): { value: object } New JSON value for the setting

Response (UISetting): Updated setting object

Responses: 200 OK, 400 Bad Request, 401 Unauthorized, 404 Not Found, 429 Too Many Requests, 500 Internal Server Error

DELETE /ui-settings/{id}

Soft-deletes a UI setting (sets isDeleted to true). Record is retained. Returns 204 on success, 404 if not found.

Path Parameters: id (uuid) UI setting ID

Responses: 204 No Content, 400 Bad Request, 401 Unauthorized, 404 Not Found, 429 Too Many Requests, 500 Internal Server Error


Schema Management (SuperAdminSchemaService)

All endpoints in this group require the super_admin role. Any other role receives 403 Forbidden. The Permit.io resource is super-admin; auditor is granted get only.

POST /super-admin/custom-schemas

Creates a new custom schema for a client. The schema definition must be a valid JSON Schema draft 2020-12 object with additionalProperties explicitly declared and a root type of object. Maximum definition size is 64 KB.

Security: Requires JWT authentication. Requires super_admin role.

Request Body (CustomSchemaRequest):

{
  "clientId": "AAA",
  "name": "aircraft-engineering",
  "description": "Schema for aircraft inspection documents",
  "schemaDef": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "aircraft_type": { "type": "string", "maxLength": 100 },
      "inspection_date": { "type": "string", "format": "date" }
    },
    "required": ["aircraft_type", "inspection_date"]
  }
}

Response (201 Created, CustomSchemaResponse):

{
  "id": "019580df-ef65-7676-8de9-94435a93337a",
  "clientId": "AAA",
  "name": "aircraft-engineering",
  "description": "Schema for aircraft inspection documents",
  "version": 1,
  "status": "active",
  "schemaDef": { ... },
  "createdAt": "2026-04-14T10:00:00Z",
  "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}

Notes:

  • createdBy is populated from the JWT sub claim (a Cognito subject UUID), not accepted from the request body.
  • schemaDef must have type: "object" at the root.
  • additionalProperties must be explicitly true or false.

Error Responses:

  • 400: Invalid or oversized schema definition
  • 401: Authentication required
  • 403: Insufficient permissions
  • 409: A schema with this name already exists for the client (at version 1)

GET /super-admin/custom-schemas

Lists schemas for a client with optional filtering.

Security: Requires JWT authentication. Requires super_admin role (auditor may use GET).

Query Parameters:

Parameter Type Required Default Description
clientId string Yes Client ID to filter schemas
name string No Filter by schema name
status string No active Filter by status: active, superseded, retired, any
includeAllVersions boolean No false When true, returns all versions per name; otherwise returns only the latest active version
limit integer No 20 Pagination page size (1100)
offset integer No 0 Pagination offset

Response (200 OK, CustomSchemaListResponse):

{
  "schemas": [
    {
      "id": "019580df-ef65-7676-8de9-94435a93337a",
      "clientId": "AAA",
      "name": "aircraft-engineering",
      "version": 2,
      "status": "active",
      "documentCount": 14,
      "canDelete": false,
      "createdAt": "2026-04-14T10:00:00Z",
      "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
    }
  ],
  "totalCount": 1
}

Notes:

  • Default list (no status, no includeAllVersions) returns only the latest active version per name.
  • canDelete is false when any document has custom_schema_id pointing to this schema.
  • documentCount is the count of documents currently bound to this specific schema version.

GET /super-admin/custom-schemas/{schemaId}

Retrieves a single schema by ID, including the full schemaDef.

Security: Requires JWT authentication. Requires super_admin role (auditor may use GET).

Path Parameters:

  • schemaId: Schema UUID

Response (200 OK, CustomSchemaResponse): Full schema object including schemaDef.

Error Responses:

  • 404: Schema not found

POST /super-admin/custom-schemas/{schemaId}/versions

Creates a new version of an existing schema. The parent schema must be active. On success: a new row is inserted with an incremented version number and status=active; the parent row's status is set to superseded. Both changes are applied in a single transaction.

Security: Requires JWT authentication. Requires super_admin role.

Path Parameters:

  • schemaId: UUID of the current active schema (the parent version)

Request Body (CustomSchemaVersionRequest): Same shape as CustomSchemaRequest body (name, description, schemaDef — no createdBy).

Response (201 Created, CustomSchemaResponse): The new schema version row.

Notes:

  • This is a POST, not a PUT — it creates a new row with a new ID. The old version is preserved and queryable.
  • Passing a schemaId that is not active (e.g. superseded or retired) returns 409.
  • createdBy is derived from JWT, not the request body.

Error Responses:

  • 400: Invalid schema definition
  • 404: Parent schema not found
  • 409: Parent schema is not active (already superseded or retired)

DELETE /super-admin/custom-schemas/{schemaId}

Retires a schema. Sets status to retired. This operation is reversible only via a new version.

Security: Requires JWT authentication. Requires super_admin role.

Path Parameters:

  • schemaId: Schema UUID

Response: 204 No Content

Error Responses:

  • 404: Schema not found
  • 409: One or more documents are currently bound to this schema (documentCount > 0)

PATCH /super-admin/documents/{id}/schema

Assigns or clears a custom schema on a document. Once a document has custom metadata written to it, the schema binding is frozen — this endpoint will return 409 if metadata exists. To upgrade a schema, first use POST .../reset-metadata.

Security: Requires JWT authentication. Requires super_admin role.

Path Parameters:

  • id: Document UUID

Request Body (DocumentSchemaAssignRequest):

{
  "schemaId": "019580df-ef65-7676-8de9-94435a93337a"
}

Pass "schemaId": null to clear an existing schema binding (allowed only when no metadata exists).

Response (200 OK, DocumentSchemaAssignResponse):

{
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "customSchemaId": "019580df-ef65-7676-8de9-94435a93337a",
  "schemaName": "aircraft-engineering",
  "schemaVersion": 2
}

Error Responses:

  • 400: Schema is not active or document already has legacy field extractions
  • 404: Document or schema not found
  • 409: Document already has custom metadata (schema binding is frozen)

POST /super-admin/documents/{id}/reset-metadata

Atomically deletes all custom metadata rows for a document and nullifies custom_schema_id. This is the required first step when upgrading a document to a newer schema version. The operation is irreversible — metadata history is lost.

Security: Requires JWT authentication. Requires super_admin role.

Path Parameters:

  • id: Document UUID

Request Body: None.

Response (200 OK, DocumentMetadataResetResponse):

{
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "previousSchemaId": "019580df-ef65-7676-8de9-94435a93337a",
  "previousSchemaName": "aircraft-engineering",
  "previousSchemaVersion": 1,
  "metadataVersionsDeleted": 3,
  "resetAt": "2026-04-15T09:30:00Z",
  "resetBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}

This endpoint is idempotent. If the document has no schema binding and no metadata, metadataVersionsDeleted is 0 and all previous* fields are null.

Error Responses:

  • 404: Document not found
  • 409: Document has legacy field extractions (mutual exclusivity — cannot reset a legacy document)

Custom Metadata Operations (CustomMetadataService)

These endpoints are available to client_user, user_admin, and super_admin for both read and write. auditor has read-only access (GET endpoints only). The Permit.io resource is custom-metadata.

POST /custom-metadata

Writes a new metadata version for a document. The document must already have a custom schema assigned. The metadata payload is validated against the bound schema. Each call appends a new version; the latest version is what GET /custom-metadata returns.

Security: Requires JWT authentication. Requires client_user, user_admin, or super_admin role.

Request Body (CustomMetadataRequest):

{
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "metadata": {
    "aircraft_type": "Boeing 737",
    "inspection_date": "2026-04-01",
    "flight_hours": 12450.5
  }
}

Maximum payload size is 1 MB.

Response (201 Created, CustomMetadataResponse):

{
  "id": "019580df-ef65-7676-8de9-94435a93337c",
  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
  "schemaId": "019580df-ef65-7676-8de9-94435a93337a",
  "schemaName": "aircraft-engineering",
  "schemaVersion": 2,
  "version": 4,
  "metadata": { ... },
  "createdAt": "2026-04-15T10:00:00Z",
  "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}

Notes:

  • createdBy is derived from JWT, not the request body.
  • Returns 400 with validation errors if the payload does not conform to the schema.

Error Responses:

  • 400: Document has no schema assigned, payload exceeds 1 MB, or payload fails schema validation
  • 409: Document has legacy field extractions (mutually exclusive with custom metadata)

GET /custom-metadata

Returns the current (latest) metadata version for a document.

Security: Requires JWT authentication. Requires client_user, user_admin, super_admin, or auditor role.

Query Parameters:

  • documentId (required): Document UUID

Response (200 OK, CustomMetadataResponse): The latest version row.

Error Responses:

  • 404: Document not found or no metadata exists for the document

GET /custom-metadata/version

Returns a specific historical metadata version for a document.

Security: Requires JWT authentication. Requires client_user, user_admin, super_admin, or auditor role.

Query Parameters:

Parameter Type Required Description
documentId uuid Yes Document UUID
version integer Yes Version number (>= 1)

Response (200 OK, CustomMetadataResponse): The requested version row.

Error Responses:

  • 400: version is less than 1
  • 404: Document not found or the specified version does not exist

GET /custom-metadata/history

Returns a paginated list of metadata version summaries for a document, ordered by version descending (newest first). Summaries include version, createdAt, and createdBy but not the full metadata payload.

Security: Requires JWT authentication. Requires client_user, user_admin, super_admin, or auditor role.

Query Parameters:

Parameter Type Required Default Description
documentId uuid Yes Document UUID
limit integer No 20 Number of versions to return (1200)
offset integer No 0 Pagination offset (>= 0)

Response (200 OK, CustomMetadataHistoryResponse):

{
  "versions": [
    {
      "version": 3,
      "createdAt": "2026-04-15T12:00:00Z",
      "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
    },
    {
      "version": 2,
      "createdAt": "2026-04-14T15:00:00Z",
      "createdBy": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
    }
  ]
}

Error Responses:

  • 400: limit is out of range (must be 1200) or offset is negative
  • 404: Document not found

Notes:

  • If the document exists but has no metadata, returns 200 with an empty versions array.
  • limit=201 is clamped to 200.

GET /document/{id} — Custom Metadata Enrichment

GET /document/{id} (documented under Document Operations) now returns two additional fields on the DocumentEnriched response:

Field Type Description
customSchemaId uuid or null UUID of the custom schema currently bound to the document. Null if no schema has been assigned.
hasCustomMetadata boolean true if at least one custom metadata version exists for the document.

These fields are present on every GET /document/{id} response regardless of whether the caller has any custom metadata role grants. A document that has never been assigned a schema returns customSchemaId: null, hasCustomMetadata: false.


Request/Response Schemas

Standard Response Headers

All API responses include:

  • RateLimit: Current rate limit status (format: 100;window=60)
  • Content-Type: application/json (except file downloads and HTML pages)

Error Response Format

{
  "message": "Error description (max 256 chars)"
}

Common HTTP Status Codes

Code Description
200 Success
201 Created
202 Accepted (async processing)
204 No Content (success with no body)
207 Multi-Status (partial success)
302 Redirect
400 Bad Request (validation errors)
401 Unauthorized (authentication required)
403 Forbidden (insufficient permissions)
404 Not Found
409 Conflict (resource exists or state conflict)
429 Too Many Requests (rate limited)
500 Internal Server Error
502 Bad Gateway (external service error)

API Versioning

Current Strategy

  • URL-based versioning: /v1 prefix
  • Backward Compatibility: Maintained within major versions
  • OpenAPI Version: 0.0.1 (development phase)

Future Considerations

  • Semantic versioning for production releases
  • Deprecation notices for breaking changes
  • Version sunset policies

Authorization Reference (Permit.io Integration)

The API uses Permit.io for fine-grained authorization. This section documents what data is sent to Permit.io for each endpoint, enabling system administrators to configure the correct resources, actions, and roles in Permit.io.

Authorization Flow

  1. Authentication: User authenticates via AWS Cognito, receiving a JWT token
  2. User Identification: The sub (subject) claim from the JWT is extracted as the user identifier
  3. Resource Mapping: The first path segment of the URL becomes the resource (e.g., /client/{id}/documentclient)
  4. Action Mapping: The HTTP method is converted to lowercase to become the action (e.g., GETget)
  5. Permission Check: Permit.io checks if the user has permission to perform the action on the resource

Permit.io Data Mapping Logic

Resource Extraction (from internal/cognitoauth/mapping.go):

  • The resource is derived from the first segment of the URL path
  • Leading slashes are removed
  • Example: /client/{id}/document/batch/{batch_id} → resource: client

Action Extraction:

  • HTTP method is converted to lowercase
  • GETget, POSTpost, PATCHpatch, DELETEdelete, HEADhead

Endpoints Exempt from Authorization

The following endpoints skip authentication and Permit.io authorization entirely:

Endpoint Purpose
GET /home Public home/menu page
GET /logout Session termination
GET /health Health check endpoint
GET /swagger/* Swagger UI documentation
GET /metrics Prometheus metrics
GET /eula Public EULA version retrieval

Special Authentication Endpoints

These endpoints handle authentication flows and bypass normal authorization:

Endpoint Purpose
GET /login Initiates OAuth2/PKCE login flow with Cognito
GET /login-callback Handles OAuth2 callback from Cognito

Auth-Only Endpoints (JWT required, no Permit.io check)

The following endpoints require a valid JWT token but do not check Permit.io permissions. Any authenticated user can access them:

Endpoint Purpose
GET /identity Users need to discover their roles
GET /eula/status Users need to check their EULA agreement status
POST /eula/agree Users need to be able to agree to the EULA
GET /ui-settings List UI settings (optional namespace filter)
POST /ui-settings Create UI setting
GET /ui-settings/{id} Get UI setting by ID
PATCH /ui-settings/{id} Update UI setting value
DELETE /ui-settings/{id} Soft-delete UI setting

Authorization Reference Table

The following table shows what data is passed to Permit.io for each protected endpoint:

HTTP Method Endpoint Resource Action Notes
Client Management
POST /client client post Create new client
GET /clients clients get List all clients
GET /client/{id} client get Get client details
PATCH /client/{id} client patch Update client
DELETE /client/{id} client delete Hard-delete client (super_admin only)
GET /client/{id}/status client get Get client sync status
Document Operations
POST /client/{id}/document client post Upload document
GET /client/{id}/document client get List client documents
GET /document/{id} document get Get document details
GET /document/{id}/download document get Download document file
DELETE /document/{id} document delete Hard-delete document (super_admin only)
Batch Document Operations
POST /client/{id}/document/batch client post Upload batch (ZIP)
GET /client/{id}/document/batch client get List batch uploads
GET /client/{id}/document/batch/{batch_id} client get Get batch status
Folder Operations
GET /client/{id}/folders client get List client folders
POST /folders folders post Create folder
PATCH /folders/{folderId} folders patch Rename folder
DELETE /folders/{folderId} folders delete Hard-delete folder tree (super_admin only)
GET /folders/{folderId}/documents folders get List folder documents
GET /folders/{folderId}/metrics folders get Get folder metrics
Label Operations
POST /documents/{documentId}/labels documents post Apply label to document
GET /documents/{documentId}/labels documents get Get document labels
GET /labels/{labelName}/documents labels get Get documents by label
Field Extraction Operations
POST /field-extractions field-extractions post Create field extraction
GET /field-extractions field-extractions get Get latest extraction
GET /field-extractions/version field-extractions get Get specific version
GET /field-extractions/history field-extractions get Get extraction history
Collector Configuration
GET /client/{id}/collector client get Get collector config
PATCH /client/{id}/collector client patch Update collector config
Export Operations
POST /client/{id}/export client post Trigger export
GET /export/{id} export get Get export status
User Administration
POST /admin/users admin post Create user
GET /admin/users admin get List users
GET /admin/users/{email} admin get Get user details
HEAD /admin/users/{email} admin head Check user existence
PATCH /admin/users/{email} admin patch Update user
DELETE /admin/users/{email} admin delete Delete user
POST /admin/users/{email}/disable admin post Disable user
POST /admin/users/{email}/enable admin post Enable user
EULA Operations
GET /eula - - Public (no auth)
GET /eula/status - - Auth-only (no Permit.io)
POST /eula/agree - - Auth-only (no Permit.io)
GET /admin/eula admin get List EULA versions
POST /admin/eula admin post Create EULA version
GET /admin/eula/{version_id} admin get Get EULA version
PATCH /admin/eula/{version_id} admin patch Update EULA metadata
POST /admin/eula/{version_id}/activate admin post Activate EULA version
GET /admin/eula/agreements admin get List EULA agreements
GET /admin/eula/compliance admin get Get compliance report
UI Settings Operations
GET /ui-settings - - Auth-only (no Permit.io)
POST /ui-settings - - Auth-only (no Permit.io)
GET /ui-settings/{id} - - Auth-only (no Permit.io)
PATCH /ui-settings/{id} - - Auth-only (no Permit.io)
DELETE /ui-settings/{id} - - Auth-only (no Permit.io)
Schema Management (SuperAdminSchemaService)
POST /super-admin/custom-schemas super-admin post Create new schema (super_admin only)
GET /super-admin/custom-schemas super-admin get List schemas with filters
GET /super-admin/custom-schemas/{schemaId} super-admin get Get a schema by ID
POST /super-admin/custom-schemas/{schemaId}/versions super-admin post Create new version (supersedes parent)
DELETE /super-admin/custom-schemas/{schemaId} super-admin delete Retire schema (409 if documents bound)
Document Schema Assignment (SuperAdminSchemaService)
PATCH /super-admin/documents/{id}/schema super-admin patch Assign (or clear) custom schema for a document
POST /super-admin/documents/{id}/reset-metadata super-admin post Wipe custom metadata history and clear schema binding
Custom Metadata Operations (CustomMetadataService)
GET /custom-metadata custom-metadata get Get current (latest) metadata version for a document
POST /custom-metadata custom-metadata post Write new validated metadata version for a document
GET /custom-metadata/version custom-metadata get Get a specific historical metadata version
GET /custom-metadata/history custom-metadata get List metadata version summaries (paged, descending)

Permit.io Configuration Requirements

To configure Permit.io to work with this API, administrators must create:

Resources

Resource Key Description
client Client management and client-scoped operations
clients Client listing (plural form)
document Individual document operations
documents Document collection and label operations
folders Folder management
labels Label-based document queries
field-extractions Field extraction operations
export Export operations
admin User administration and EULA admin operations
super-admin Schema management — gates all /super-admin/custom-schemas routes (super_admin CRUD; auditor read-only)
custom-metadata Custom document metadata operations (all roles except auditor can read and write; auditor read-only)

Actions

Action Key HTTP Method Description
get GET Read/retrieve resources
post POST Create resources
patch PATCH Update resources
delete DELETE Delete resources
head HEAD Check resource existence

Role Configuration

The following roles are defined in the Permit.io configuration (cmd/auth_related/permit.setup/permit_policies.yaml):

Role Resources Actions Use Case
super_admin All resources All actions including delete Full administrative access
user_admin admin, documents, field-extractions, folders, custom-metadata get, post, patch, delete (admin only); get, post (custom-metadata) User management
auditor admin, client, clients, document, export, documents, field-extractions, folders, labels, super-admin, custom-metadata get (and head for admin) Read-only access for auditing
client_user client, clients, document, export, documents, field-extractions, folders, labels, custom-metadata get, post, patch (varies by resource); get, post (custom-metadata) Client-specific access

Delete permissions: Only super_admin has delete access to client, document, and folders resources. The client_user role does NOT have client:delete or any other delete permissions on these resources.

Schema management permissions (super-admin and custom-metadata resources):

Role super-admin resource custom-metadata resource
super_admin get, post, patch, delete get, post
user_admin none get, post
auditor get get
client_user none get, post

The super-admin resource gates all /super-admin/custom-schemas CRUD endpoints. A user_admin has no super-admin grant and receives 403 on every schema management route.

Authorization Error Responses

When authorization fails, the API returns:

HTTP 401 Unauthorized - No valid JWT token:

{
  "error": "Authorization required",
  "message": "This endpoint requires authentication. Please obtain a valid JWT token.",
  "login_url": "/login"
}

HTTP 403 Forbidden - Insufficient permissions:

{
  "error": "Insufficient permissions",
  "message": "Access denied by authorization service",
  "user": "<cognito-subject-id>",
  "resource": "<resource-name>",
  "action": "<action-name>"
}

Mutable Metadata v4 — Design Decisions

The following decisions shaped the Milestone 1 schema CRUD implementation and apply to all subsequent milestones.

  • POST .../versions instead of PUT: A PUT route implies in-place replacement, which would overwrite the existing row and lose the ability to audit what the previous definition was. POST /super-admin/custom-schemas/{schemaId}/versions creates a new row with a new id, increments the version counter, and marks the parent row as superseded — all in a single transaction. The old row is preserved permanently and remains queryable.

  • createdBy is not in request bodies: The actor identity comes from the JWT sub claim (the Cognito subject UUID), extracted server-side by the handler. Accepting createdBy in the request body would allow any caller to impersonate a different actor. The sub claim is cryptographically bound to the token and cannot be spoofed by the client.

  • Composite FK replaces the v3 trigger: v3 used a trigger (trg_validate_schema_client_match) to prevent a document from being bound to a schema owned by a different client. v4 replaces this with a composite foreign key fk_documents_custom_schema_same_client on (custom_schema_id, clientId) referencing client_metadata_schemas(id, client_id). Structural enforcement fires before any application code runs, catches the violation even on direct SQL updates, and requires no ongoing maintenance.

  • additionalProperties must be explicitly declared: JSON Schema's default behavior when additionalProperties is absent is to allow all additional properties. Requiring an explicit declaration forces the schema author to make a deliberate choice and prevents accidental data loss due to schema drift.

  • documentFieldExtractions and custom metadata are mutually exclusive: A document may either have legacy field extractions or custom schema metadata — never both. This is enforced at three layers: service-layer FOR UPDATE parent-row locks, a Postgres trigger (trg_prevent_schema_reassignment), and a second trigger (trg_prevent_legacy_extraction_on_custom_document). Defense-in-depth ensures the invariant holds even if one layer is bypassed.


Code Generation

OpenAPI Integration

The API is generated from the OpenAPI specification using:

oapi-codegen -generate "types,server" -o api.gen.go serviceAPIs/queryAPI.yaml

Generated Artifacts

  • Type-safe Go structs for requests/responses
  • Echo framework route handlers
  • OpenAPI validation middleware
  • Swagger documentation

Validation

  • Automatic request validation against OpenAPI schema
  • Response validation in development mode
  • Parameter binding with type safety