From 4ad8168f35d3a40ebedecbd8bb11a7d5ac4011b7 Mon Sep 17 00:00:00 2001 From: Jay Brown Date: Wed, 7 Jan 2026 19:35:53 +0000 Subject: [PATCH] Merged in feature/setup_docs (pull request #199) docs for new env setup * docs for new env setup * lint fix * docs --- .../permit.setup/permit_policies.yaml | 20 +- .../user.creation.tool/build.linux.sh | 19 + cmd/cognito_test/user.creation.tool/readme.md | 82 ++ docs/ai.generated/03-api-documentation.md | 400 +++++--- .../07-configuration-management.md | 139 ++- docs/ai.generated/09-authentication-guide.md | 945 ++++++++++++++++++ .../ai.generated/queryapi.summary.md | 119 ++- 7 files changed, 1588 insertions(+), 136 deletions(-) create mode 100755 cmd/cognito_test/user.creation.tool/build.linux.sh create mode 100644 docs/ai.generated/09-authentication-guide.md rename queryapi.summary.md => docs/ai.generated/queryapi.summary.md (60%) diff --git a/cmd/cognito_test/permit.setup/permit_policies.yaml b/cmd/cognito_test/permit.setup/permit_policies.yaml index 4404071e..9a2d55a7 100644 --- a/cmd/cognito_test/permit.setup/permit_policies.yaml +++ b/cmd/cognito_test/permit.setup/permit_policies.yaml @@ -15,7 +15,15 @@ resources: - head - name: client - description: Client management + description: Single client operations + actions: + - get + - post + - patch + - delete + + - name: clients + description: Clients administration actions: - get - post @@ -63,6 +71,11 @@ roles: - post - patch - delete + clients: + - get + - post + - patch + - delete query: - get - post @@ -97,6 +110,8 @@ roles: - head client: - get + clients: + - get query: - get document: @@ -127,7 +142,8 @@ roles: Future implementation will require application-level filtering based on client_id attribute. # Documentation only from here down. -# All of the secions that follow are for documentation only and can be removed. +# +# All of the sections that follow are for documentation only and can be removed. # They are not used by the permit setup tool. # Policy mappings - maps HTTP endpoints to resource:action combinations diff --git a/cmd/cognito_test/user.creation.tool/build.linux.sh b/cmd/cognito_test/user.creation.tool/build.linux.sh new file mode 100755 index 00000000..b291cc60 --- /dev/null +++ b/cmd/cognito_test/user.creation.tool/build.linux.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Build Linux AMD64 binary for AWS CloudShell +# Usage: ./build.linux.sh +# +# Output: dist/user.creation.tool + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" +cd "$SCRIPT_DIR" + +mkdir -p dist + +echo "Building Linux AMD64 binary for AWS CloudShell..." +echo " GOOS=linux GOARCH=amd64 CGO_ENABLED=0" + +GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o dist/user.creation.tool . + +echo "Built: dist/user.creation.tool" diff --git a/cmd/cognito_test/user.creation.tool/readme.md b/cmd/cognito_test/user.creation.tool/readme.md index f4831d8e..b5fa90ed 100644 --- a/cmd/cognito_test/user.creation.tool/readme.md +++ b/cmd/cognito_test/user.creation.tool/readme.md @@ -92,6 +92,7 @@ Re-enables previously disabled users in AWS Cognito: 2. **Permit.io Account** with: - A project and environment set up + - All roles that will be assigned to users pre-defined (see [Permit.io Prerequisites](#permitio-prerequisites)) - API key with user creation permissions (organization, project, or environment-level key) ## Configuration @@ -321,6 +322,87 @@ Works with all operation modes: - Perfect for testing and validation - **Recommended**: Always test with `--dry-run` first, especially for disable/enable operations +## Permit.io Integration Details + +This section documents the specific Permit.io API operations performed by the tool and what must be configured before running. + +### Permit.io Prerequisites + +Before running this tool, the following must already exist in your Permit.io environment: + +| Requirement | Description | +|-------------|-------------| +| **Project** | A Permit.io project must already be created | +| **Environment** | An environment must exist within the project | +| **Roles** | All roles specified in the CSV must be pre-defined in Permit.io. The tool validates this upfront and **fails fast** if any role is missing. | +| **"default" Tenant** | The tool assigns all roles to a hardcoded `"default"` tenant. This tenant must exist (Permit.io creates it automatically for new environments). | +| **API Key** | An API key with appropriate permissions (organization, project, or environment-scoped) | + +### What the Tool Creates in Permit.io + +| Resource | API Endpoint | Details | +|----------|--------------|---------| +| **Users** | `POST /v2/facts/{project}/{env}/users` | Creates a user with the Cognito Subject ID as the `key`, plus email, first/last name, and custom attributes | +| **Role Assignments** | `POST /v2/facts/{project}/{env}/role_assignments` | Assigns roles to users on the `"default"` tenant | + +**User object created:** +```json +{ + "key": "", + "email": "user@example.com", + "first_name": "John", + "last_name": "Doe", + "attributes": { + "cognito_username": "user@example.com", + "cognito_status": "CONFIRMED" + } +} +``` + +### What the Tool Deletes from Permit.io + +| Resource | API Endpoint | Details | +|----------|--------------|---------| +| **Users** | `DELETE /v2/facts/{project}/{env}/users/{user_key}` | Removes user by their Cognito Subject ID | +| **Role Assignments** | `DELETE /v2/facts/{project}/{env}/role_assignments` | Unassigns roles that are no longer specified in the CSV | + +### What the Tool Reads from Permit.io (Validation/Lookup) + +| Resource | API Endpoint | Purpose | +|----------|--------------|---------| +| Existing roles | `GET /v2/schema/{project}/{env}/roles` | Validates all CSV roles exist before processing any users | +| Users by email | `GET /v2/facts/{project}/{env}/users?search={email}` | Checks if user already exists | +| User's role assignments | `GET /v2/facts/{project}/{env}/role_assignments?user={key}` | Gets current roles for intelligent sync | +| API key scope | `GET /v2/api-key/scope` | Auto-detects project/env when using `-project use-key` | +| Projects/Environments | `GET /v2/projects`, `GET /v2/projects/{id}/envs` | Lists available options when using `-project list` | + +### Key Implementation Details + +| Detail | Description | +|--------|-------------| +| **User Key** | Uses Cognito `Subject ID` (the `sub` claim from JWT) as the Permit.io user key - **not** the email address | +| **Tenant** | All role assignments use a hardcoded `"default"` tenant. Multi-tenant support is not currently implemented. | +| **Role Sync** | Idempotent - removes roles no longer in CSV, adds missing ones, skips unchanged roles | +| **User Attributes** | Stores `cognito_username` and `cognito_status` as custom attributes on the Permit.io user | +| **Role Validation** | Validates all roles exist **before** processing any users. If any role is missing, the tool exits with an error listing the missing roles. | + +### Creating Required Roles in Permit.io + +If you receive an error about missing roles, you must create them in the Permit.io dashboard before running the tool: + +1. Go to the [Permit.io Dashboard](https://app.permit.io) +2. Navigate to your **Project** → **Environment** +3. Go to **Policy** → **Roles** +4. Create each role that will be used in your CSV file + +Alternatively, use the Permit.io API: +```bash +curl -X POST "https://api.permit.io/v2/schema/{project_id}/{env_id}/roles" \ + -H "Authorization: Bearer $PERMIT_KEY" \ + -H "Content-Type: application/json" \ + -d '{"key": "admin", "name": "Admin", "description": "Administrator role"}' +``` + ## Audit Logging ### Audit Files diff --git a/docs/ai.generated/03-api-documentation.md b/docs/ai.generated/03-api-documentation.md index b961c6a4..e2ce1f3b 100644 --- a/docs/ai.generated/03-api-documentation.md +++ b/docs/ai.generated/03-api-documentation.md @@ -3,8 +3,10 @@ ## API Overview The DoczyAI Query Orchestration Platform exposes a comprehensive REST API following OpenAPI 3.0.3 specification. The API is automatically generated from the specification file `serviceAPIs/queryAPI.yaml` using oapi-codegen. +also see [query api summary](./queryapi.summary.md) ### Base Configuration + - **Base URL**: `https://doczy.com/v1` (production), `http://localhost:8080` (development) - **API Version**: 0.0.1 - **Content Type**: `application/json` @@ -14,29 +16,31 @@ The DoczyAI Query Orchestration Platform exposes a comprehensive REST API follow The API is organized into the following service groups: -| Service | Description | -|---------|-------------| -| ClientService | Operations related to clients | -| CollectorService | Operations related to collectors | -| DocumentsService | Operations related to documents | -| FolderService | Operations related to document folders and organization | -| LabelService | Operations related to document labels and workflow tracking | -| FieldExtractionService | Operations related to document field extractions | -| QueryService | Operations related to queries | -| ExportService | Operations related to exports | -| AuthService | Operations related to authentication | -| AdminService | Operations related to user management and administration | +| Service | Description | +| ---------------------- | ----------------------------------------------------------- | +| ClientService | Operations related to clients | +| CollectorService | Operations related to collectors | +| DocumentsService | Operations related to documents | +| FolderService | Operations related to document folders and organization | +| LabelService | Operations related to document labels and workflow tracking | +| FieldExtractionService | Operations related to document field extractions | +| QueryService | Operations related to queries | +| ExportService | Operations related to exports | +| AuthService | Operations related to authentication | +| AdminService | Operations related to user management and administration | ## Authentication and Authorization ### Security Schemes #### JWT Authentication + - **Type**: Bearer token authentication - **Header**: `Authorization: Bearer ` - **Token Source**: JWT cookie set via OAuth flow #### OAuth2 Flow + - **Type**: OAuth2 Authorization Code Flow - **Provider**: AWS Cognito - **Scopes**: `openid`, `email`, `profile` @@ -44,28 +48,33 @@ The API is organized into the following service groups: ### Authentication Endpoints #### `GET /login` + Initiates OAuth2 authentication flow with AWS Cognito. **Security**: Public (no authentication required) **Response**: `302` Redirect to Cognito login page #### `GET /login-callback` + OAuth2 callback handler that processes authentication response. **Security**: Public **Parameters**: + - `code` (query, required): Authorization code from Cognito (max 2048 chars) - `state` (query, required): CSRF protection parameter (max 1024 chars) **Response**: `302` Sets JWT cookie and redirects to application #### `GET /logout` + Terminates user session and clears authentication cookies. **Security**: Requires JWT authentication **Response**: `302` Redirect to Cognito logout page or application home #### `GET /home` + Returns the HTML menu page for authenticated users. **Security**: Requires JWT authentication @@ -78,6 +87,7 @@ The queryAPI service implements dual-layer rate limiting to protect against DDoS ### Rate Limit Architecture **Dual-Layer Protection:** + - **Layer 1: Global/Aggregate Rate Limiting** - Protects total system capacity across all IP addresses - **Layer 2: Per-IP Rate Limiting** - Prevents individual client abuse @@ -86,6 +96,7 @@ Both layers must pass for a request to proceed. The global limiter checks first ### Default Rate Limits By default, all endpoints are limited to: + - **Global**: 500 requests per second total (1000 burst) - **Per-IP**: 5 requests per second per IP (10 burst) - **Cleanup Interval**: 3 minutes for inactive IP limiters @@ -125,10 +136,12 @@ RateLimit: 20;window=2 **HTTP Status:** `429 Too Many Requests` **Headers:** + - `Retry-After: {seconds}` - Number of seconds before retrying - `RateLimit: {burst};window={seconds}` - Current rate limit **Response Body:** + ```json { "message": "rate limit exceeded" @@ -140,11 +153,13 @@ RateLimit: 20;window=2 ## Client Management (ClientService) ### `POST /client` + Creates a new client in the system. **Security**: Requires JWT authentication **Request Body** (`ClientCreate`): + ```json { "id": "string (max 36 chars, required)", @@ -153,6 +168,7 @@ Creates a new client in the system. ``` **Responses**: + - `201`: Client created successfully ```json { @@ -165,14 +181,17 @@ Creates a new client in the system. - `500`: Internal server error ### `GET /client/{id}` + Retrieves client details by ID. **Security**: Requires JWT authentication **Path Parameters**: + - `id`: Client identifier (string, max 36 chars) **Response** (`DocClient`): + ```json { "id": "AAA", @@ -182,11 +201,13 @@ Retrieves client details by ID. ``` ### `PATCH /client/{id}` + Updates client information. **Security**: Requires JWT authentication **Request Body** (`ClientUpdate`): + ```json { "name": "string (max 256 chars, optional)", @@ -197,11 +218,13 @@ Updates client information. **Response**: `200` Client updated successfully ### `GET /client/{id}/status` + Returns client synchronization status. **Security**: Requires JWT authentication **Response** (`ClientStatusBody`): + ```json { "status": "IN_SYNC" @@ -215,6 +238,7 @@ Returns client synchronization status. ## Document Operations (DocumentsService) ### `POST /client/{id}/document` + Uploads a document for a specific client. **Security**: Requires JWT authentication @@ -222,17 +246,20 @@ Uploads a document for a specific client. **Max File Size**: 100GB **Form Parameters**: + - `file` (required): Document file (PDF) - `filename` (optional): Custom filename (max 4096 chars) **Response**: `200` Document uploaded ### `GET /client/{id}/document` + Lists documents for a client. **Security**: Requires JWT authentication **Response** (`ListDocuments`): + ```json [ { @@ -243,14 +270,17 @@ Lists documents for a client. ``` ### `GET /document/{id}` + Retrieves document details by ID. **Security**: Requires JWT authentication **Path Parameters**: + - `id`: Document UUID (max 36 chars) **Response** (`Document`): + ```json { "id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7", @@ -268,6 +298,7 @@ Retrieves document details by ID. ## Batch Document Operations (DocumentsService) ### `POST /client/{id}/document/batch` + Uploads a ZIP archive containing multiple PDF documents for asynchronous batch processing. **Security**: Requires JWT authentication @@ -276,9 +307,11 @@ Uploads a ZIP archive containing multiple PDF documents for asynchronous batch p **Supported Archive Type**: ZIP only **Form Parameters**: + - `archive` (required): ZIP file containing PDF documents **Response** (`202 Accepted`): + ```json { "batch_id": "019580df-ef65-7676-8de9-94435a93337a", @@ -288,15 +321,18 @@ Uploads a ZIP archive containing multiple PDF documents for asynchronous batch p ``` ### `GET /client/{id}/document/batch` + Lists batch uploads for a client with pagination support. **Security**: Requires JWT authentication **Query Parameters**: + - `limit`: Maximum results (integer, 1-100, default: 20) - `offset`: Pagination offset (integer, default: 0) **Response** (`BatchUploadList`): + ```json { "batches": [ @@ -321,14 +357,17 @@ Lists batch uploads for a client with pagination support. **Batch Status Values**: `processing`, `completed`, `failed`, `cancelled` ### `GET /client/{id}/document/batch/{batch_id}` + Retrieves detailed status information for a specific batch upload. **Security**: Requires JWT authentication **Path Parameters**: + - `batch_id`: Batch upload identifier (UUID) **Response** (`BatchUploadDetails`): + ```json { "batch_id": "019580df-ef65-7676-8de9-94435a93337a", @@ -347,11 +386,13 @@ Retrieves detailed status information for a specific batch upload. ``` ### `DELETE /client/{id}/document/batch/{batch_id}` + Cancels a batch upload that is currently processing. **Security**: Requires JWT authentication **Response**: + - `204`: Batch upload cancelled successfully - `404`: Batch not found @@ -380,12 +421,14 @@ All folder paths must comply with the following constraints when creating or ren - **Unicode normalization**: Paths are normalized to NFC form **Valid path examples**: + - `/documents` - `/documents/2024/Q1` - `/client-data/reports_2024` - `/My Documents/Archive` **Invalid path examples**: + - `documents` (missing leading `/`) - `/documents/` (trailing slash) - `/foo//bar` (empty segment) @@ -397,14 +440,17 @@ All folder paths must comply with the following constraints when creating or ren --- ### `GET /clients/{clientId}/folders` + Lists all folders for a specific client, including the folder hierarchy. **Security**: Requires JWT authentication **Path Parameters**: + - `clientId`: Client external ID (string, max 36 chars) **Response** (`200 OK`): + ```json { "folders": [ @@ -437,16 +483,19 @@ Lists all folders for a specific client, including the folder hierarchy. ``` **Notes**: + - Returns all folders for the client, including nested folders - Folders are returned in a flat list; use `parentId` to reconstruct the hierarchy - The root folder (path="/") is included if it exists ### `POST /folders` + Creates a new folder for organizing documents. **Security**: Requires JWT authentication **Request Body** (`FolderCreate`): + ```json { "path": "/documents/2024", @@ -456,14 +505,15 @@ Creates a new folder for organizing documents. } ``` -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| path | string | Yes | Folder path, must start with "/" (max 1000 chars) | -| clientId | string | Yes | Client external ID | -| createdBy | string | Yes | Email of user creating the folder | -| parentId | uuid | No | Parent folder ID for nested folders | +| Field | Type | Required | Description | +| --------- | ------ | -------- | ------------------------------------------------- | +| path | string | Yes | Folder path, must start with "/" (max 1000 chars) | +| clientId | string | Yes | Client external ID | +| createdBy | string | Yes | Email of user creating the folder | +| parentId | uuid | No | Parent folder ID for nested folders | **Response** (`201 Created`): + ```json { "id": "019580df-ef65-7676-8de9-94435a93337a", @@ -476,14 +526,17 @@ Creates a new folder for organizing documents. ``` ### `PATCH /folders/{folderId}` + Renames an existing folder. **Security**: Requires JWT authentication **Path Parameters**: + - `folderId`: Folder UUID **Request Body** (`FolderRename`): + ```json { "path": "/documents/2024-renamed" @@ -493,11 +546,13 @@ Renames an existing folder. **Response** (`200 OK`): Updated folder object ### `GET /folders/{folderId}/documents` + Retrieves all documents within a specific folder. **Security**: Requires JWT authentication **Response**: + ```json { "documents": [ @@ -512,11 +567,13 @@ Retrieves all documents within a specific folder. ``` ### `GET /folders/{folderId}/metrics` + Retrieves processing progress metrics for documents in a folder. **Security**: Requires JWT authentication **Response** (`FolderMetrics`): + ```json { "folderId": "019580df-ef65-7676-8de9-94435a93337a", @@ -534,14 +591,17 @@ Retrieves processing progress metrics for documents in a folder. ## Label Operations (LabelService) ### `POST /documents/{documentId}/labels` + Applies a workflow label to a document for tracking processing state. **Security**: Requires JWT authentication **Path Parameters**: + - `documentId`: Document UUID **Request Body** (`LabelApplication`): + ```json { "label": "OCR_Processed", @@ -549,12 +609,13 @@ Applies a workflow label to a document for tracking processing state. } ``` -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| label | string | Yes | Label name (alphanumeric and underscore only, max 100 chars) | -| appliedBy | string | Yes | Email of user applying the label | +| Field | Type | Required | Description | +| --------- | ------ | -------- | ------------------------------------------------------------ | +| label | string | Yes | Label name (alphanumeric and underscore only, max 100 chars) | +| appliedBy | string | Yes | Email of user applying the label | **Response** (`201 Created`): + ```json { "id": "019580df-ef65-7676-8de9-94435a93337a", @@ -566,11 +627,13 @@ Applies a workflow label to a document for tracking processing state. ``` ### `GET /documents/{documentId}/labels` + Retrieves all labels that have been applied to a document, ordered by most recent first. **Security**: Requires JWT authentication **Response**: + ```json { "labels": [ @@ -586,17 +649,21 @@ Retrieves all labels that have been applied to a document, ordered by most recen ``` ### `GET /labels/{labelName}/documents` + Retrieves all documents that have been tagged with a specific label. **Security**: Requires JWT authentication **Path Parameters**: + - `labelName`: Label name (alphanumeric and underscore only, max 100 chars) **Query Parameters**: + - `clientId` (required): Client ID to filter documents (UUID) **Response**: + ```json { "documents": [ @@ -613,11 +680,13 @@ Retrieves all documents that have been tagged with a specific label. ## Field Extraction Operations (FieldExtractionService) ### `POST /field-extractions` + Creates a new field extraction with single-value and array fields for a document. **Security**: Requires JWT authentication **Request Body** (`FieldExtractionRequest`): + ```json { "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7", @@ -643,7 +712,7 @@ Creates a new field extraction with single-value and array fields for a document "aareteDerivedClaimTypeCd": "INPATIENT", "aareteDerivedReimbMethod": "Per Diem", "reimbPctRate": 85.5, - "reimbFeeRate": 150.00 + "reimbFeeRate": 150.0 } ], "createdBy": "user@example.com" @@ -651,6 +720,7 @@ Creates a new field extraction with single-value and array fields for a document ``` **Single Fields** (1:1 relationship with document): + - `fileName`, `contractTitle`, `clientName`, `payerName` - `payerState`, `providerState` (2-letter state codes) - `filenameTin`, `provGroupTin`, `provGroupNpi`, `provGroupNameFull` @@ -660,6 +730,7 @@ Creates a new field extraction with single-value and array fields for a document - `autoRenewalInd` (boolean), `autoRenewalTerm` **Array Fields** (1:N relationship - multiple rows per document): + - Provider info: `reimbProvTin`, `reimbProvNpi`, `reimbProvName` - Dates: `reimbEffectiveDt`, `reimbTerminationDt` - Classifications: `aareteDerivedClaimTypeCd`, `aareteDerivedProduct`, `aareteDerivedLob`, `aareteDerivedProgram`, `aareteDerivedNetwork`, `aareteDerivedProvType` @@ -668,6 +739,7 @@ Creates a new field extraction with single-value and array fields for a document - Indicators: `carveoutInd`, `lesserOfInd`, `greaterOfInd`, `defaultInd` **Response** (`201 Created`): + ```json { "id": "019580df-ef65-7676-8de9-94435a93337a", @@ -681,24 +753,29 @@ Creates a new field extraction with single-value and array fields for a document ``` ### `GET /field-extractions` + Retrieves the most recent field extraction for a document. **Security**: Requires JWT authentication **Query Parameters**: + - `documentId` (required): Document UUID **Response**: `FieldExtractionResponse` (same as POST response) ### `GET /field-extractions/history` + Retrieves all versions of field extractions for a document, ordered by most recent first. **Security**: Requires JWT authentication **Query Parameters**: + - `documentId` (required): Document UUID **Response**: + ```json { "versions": [ @@ -721,15 +798,18 @@ Retrieves all versions of field extractions for a document, ordered by most rece ``` ### `GET /field-extractions/version` + Retrieves a specific version of a field extraction for a document by version number. **Security**: Requires JWT authentication **Query Parameters**: + - `documentId` (required): Document UUID - `version` (required): Version number (integer, minimum 1) **Response** (`200 OK`): + ```json { "id": "019580df-ef65-7676-8de9-94435a93337a", @@ -747,6 +827,7 @@ Retrieves a specific version of a field extraction for a document by version num ``` **Error Responses**: + - `400`: Missing required query parameters - `404`: Version not found for the specified document @@ -755,11 +836,13 @@ Retrieves a specific version of a field extraction for a document by version num ## Query Management (QueryService) ### `GET /query` + Lists all available queries in the system. **Security**: Requires JWT authentication **Response** (`ListQueries`): + ```json { "queries": [ @@ -778,11 +861,13 @@ Lists all available queries in the system. **Query Types**: `JSON_EXTRACTOR`, `CONTEXT_FULL` ### `POST /query` + Creates a new query definition. **Security**: Requires JWT authentication **Request Body** (`QueryCreate`): + ```json { "type": "JSON_EXTRACTOR", @@ -792,6 +877,7 @@ Creates a new query definition. ``` **Response** (`201 Created`): + ```json { "id": "019580df-ef65-7676-8de9-94435a93337a" @@ -799,6 +885,7 @@ Creates a new query definition. ``` ### `GET /query/{id}` + Retrieves query details including configuration. **Security**: Requires JWT authentication @@ -806,11 +893,13 @@ Retrieves query details including configuration. **Response**: `Query` object ### `PATCH /query/{id}` + Updates query definition. **Security**: Requires JWT authentication **Request Body** (`QueryUpdate`): + ```json { "config": "{\"updated\": true}", @@ -820,11 +909,13 @@ Updates query definition. ``` ### `POST /query/{id}/test` + Tests query execution against a document. **Security**: Requires JWT authentication **Request Body** (`QueryTestRequest`): + ```json { "document_id": "019580df-b3f8-7348-9ab1-1e55b3f18ed7", @@ -833,6 +924,7 @@ Tests query execution against a document. ``` **Response** (`QueryTestResponse`): + ```json { "value": "extracted result" @@ -844,11 +936,13 @@ Tests query execution against a document. ## Collector Configuration (CollectorService) ### `GET /client/{id}/collector` + Retrieves collector configuration for a client. **Security**: Requires JWT authentication **Response** (`Collector`): + ```json { "client_id": "AAA", @@ -866,11 +960,13 @@ Retrieves collector configuration for a client. ``` ### `PATCH /client/{id}/collector` + Updates collector configuration. **Security**: Requires JWT authentication **Request Body** (`CollectorSet`): + ```json { "minimum_cleaner_version": 2, @@ -892,11 +988,13 @@ Updates collector configuration. ## Export Operations (ExportService) ### `POST /client/{id}/export` + Triggers data export for a client. **Security**: Requires JWT authentication **Request Body** (`ExportTrigger`): + ```json { "ingestion_filters": { @@ -916,6 +1014,7 @@ Triggers data export for a client. **Filter Conditions**: `less_than`, `greater_than`, `closed_interval`, `open_interval`, `left_closed_interval`, `right_closed_interval`, `include`, `exclude` **Response** (`201 Created`): + ```json { "id": "019580de-4d51-713c-98ee-464e83811f13" @@ -923,11 +1022,13 @@ Triggers data export for a client. ``` ### `GET /export/{id}` + Checks export status and retrieves download URL. **Security**: Requires JWT authentication **Response** (`ExportDetails`): + ```json { "client_id": "AAA", @@ -943,11 +1044,13 @@ Checks export status and retrieves download URL. ## User Administration (AdminService) ### `POST /admin/users` + Creates a new user in both AWS Cognito and Permit.io systems with optional role assignments. **Security**: Requires JWT authentication **Creation Flow**: + 1. User is created in AWS Cognito (primary authentication system) 2. If Cognito creation fails, operation fails with HTTP 502 3. User is created in Permit.io (authorization/role system) @@ -955,6 +1058,7 @@ Creates a new user in both AWS Cognito and Permit.io systems with optional role 5. Roles are assigned in Permit.io (best-effort, continues on failures) **Request Body** (`AdminUserCreate`): + ```json { "email": "john.doe@example.com", @@ -964,14 +1068,15 @@ Creates a new user in both AWS Cognito and Permit.io systems with optional role } ``` -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| email | string | Yes | User email address (max 320 chars) | -| first_name | string | Yes | User's given name (max 100 chars) | -| last_name | string | Yes | User's family name (max 100 chars) | -| roles | array | Yes | List of role keys to assign (1-50 roles) | +| Field | Type | Required | Description | +| ---------- | ------ | -------- | ---------------------------------------- | +| email | string | Yes | User email address (max 320 chars) | +| first_name | string | Yes | User's given name (max 100 chars) | +| last_name | string | Yes | User's family name (max 100 chars) | +| roles | array | Yes | List of role keys to assign (1-50 roles) | **Response** (`201 Created` or `200 OK` if user exists): + ```json { "cognito_subject_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", @@ -987,6 +1092,7 @@ Creates a new user in both AWS Cognito and Permit.io systems with optional role ``` **Error Responses**: + - `400`: Invalid request data - `401`: Authentication required - `403`: Insufficient permissions @@ -995,6 +1101,7 @@ Creates a new user in both AWS Cognito and Permit.io systems with optional role - `502`: External service (Cognito) error ### `GET /admin/users` + Lists users from AWS Cognito with pagination, filtering, and sorting support. **Security**: Requires JWT authentication @@ -1010,6 +1117,7 @@ Lists users from AWS Cognito with pagination, filtering, and sorting support. | sort_order | string | asc | Sort direction: `asc`, `desc` | **Response** (`AdminUserList`): + ```json { "users": [ @@ -1033,34 +1141,41 @@ Lists users from AWS Cognito with pagination, filtering, and sorting support. ``` ### `GET /admin/users/{email}` + Retrieves user details from both AWS Cognito and Permit.io by email address. **Security**: Requires JWT authentication **Path Parameters**: + - `email`: URL-encoded user email address (max 320 chars) **Response** (`AdminUserDetails`): Same structure as user list item ### `HEAD /admin/users/{email}` + Checks if a user exists in Cognito and/or Permit.io without returning full details. **Security**: Requires JWT authentication **Response Headers**: + - `X-User-Exists-Cognito`: boolean - `X-User-Exists-PermitIO`: boolean **Response**: + - `200`: User exists (at least in one system) - `404`: User does not exist in either system ### `PATCH /admin/users/{email}` + Updates user attributes (first_name, last_name) in Cognito and manages role assignments in Permit.io. **Security**: Requires JWT authentication **Role Management (REPLACE Strategy)**: + - Providing a `roles` array REPLACES all existing roles (not additive) - To add a role: include ALL current roles PLUS the new role - To remove a role: include only the roles you want to keep @@ -1068,6 +1183,7 @@ Updates user attributes (first_name, last_name) in Cognito and manages role assi - Omitting the roles field leaves role assignments unchanged **Request Body** (`AdminUserUpdate`): + ```json { "first_name": "Jonathan", @@ -1079,14 +1195,17 @@ Updates user attributes (first_name, last_name) in Cognito and manages role assi **Response** (`200 OK`): Updated `AdminUserDetails` ### `DELETE /admin/users/{email}` + Permanently deletes a user from both AWS Cognito and Permit.io. This operation is irreversible. **Security**: Requires JWT authentication **Query Parameters**: + - `confirm` (required): Must be `true` to confirm deletion **Response** (`200 OK` or `207 Multi-Status`): + ```json { "success": true, @@ -1101,11 +1220,13 @@ Permanently deletes a user from both AWS Cognito and Permit.io. This operation i ``` ### `POST /admin/users/{email}/disable` + Disables a user in AWS Cognito, preventing authentication. User data and roles are preserved. Idempotent. **Security**: Requires JWT authentication **Response** (`AdminUserActionResponse`): + ```json { "success": true, @@ -1117,6 +1238,7 @@ Disables a user in AWS Cognito, preventing authentication. User data and roles a ``` ### `POST /admin/users/{email}/enable` + Re-enables a previously disabled user in AWS Cognito. Restores authentication capability. Idempotent. **Security**: Requires JWT authentication @@ -1128,11 +1250,14 @@ Re-enables a previously disabled user in AWS Cognito. Restores authentication ca ## Request/Response Schemas ### Standard Response Headers + All API responses include: + - `RateLimit`: Current rate limit status (format: `100;window=60`) - `Content-Type`: `application/json` (except file downloads and HTML pages) ### Error Response Format + ```json { "message": "Error description (max 256 chars)" @@ -1140,33 +1265,36 @@ All API responses include: ``` ### Common HTTP Status Codes -| Code | Description | -|------|-------------| -| 200 | Success | -| 201 | Created | -| 202 | Accepted (async processing) | -| 204 | No Content (success with no body) | -| 207 | Multi-Status (partial success) | -| 302 | Redirect | -| 400 | Bad Request (validation errors) | -| 401 | Unauthorized (authentication required) | -| 403 | Forbidden (insufficient permissions) | -| 404 | Not Found | -| 409 | Conflict (resource exists or state conflict) | -| 429 | Too Many Requests (rate limited) | -| 500 | Internal Server Error | -| 502 | Bad Gateway (external service error) | + +| Code | Description | +| ---- | -------------------------------------------- | +| 200 | Success | +| 201 | Created | +| 202 | Accepted (async processing) | +| 204 | No Content (success with no body) | +| 207 | Multi-Status (partial success) | +| 302 | Redirect | +| 400 | Bad Request (validation errors) | +| 401 | Unauthorized (authentication required) | +| 403 | Forbidden (insufficient permissions) | +| 404 | Not Found | +| 409 | Conflict (resource exists or state conflict) | +| 429 | Too Many Requests (rate limited) | +| 500 | Internal Server Error | +| 502 | Bad Gateway (external service error) | --- ## API Versioning ### Current Strategy + - **URL-based versioning**: `/v1` prefix - **Backward Compatibility**: Maintained within major versions - **OpenAPI Version**: 0.0.1 (development phase) ### Future Considerations + - Semantic versioning for production releases - Deprecation notices for breaking changes - Version sunset policies @@ -1188,11 +1316,13 @@ The API uses Permit.io for fine-grained authorization. This section documents wh ### Permit.io Data Mapping Logic **Resource Extraction** (from `internal/cognitoauth/mapping.go`): + - The resource is derived from the **first segment** of the URL path - Leading slashes are removed - Example: `/client/{id}/document/batch/{batch_id}` → resource: `client` **Action Extraction**: + - HTTP method is converted to lowercase - `GET` → `get`, `POST` → `post`, `PATCH` → `patch`, `DELETE` → `delete`, `HEAD` → `head` @@ -1200,80 +1330,80 @@ The API uses Permit.io for fine-grained authorization. This section documents wh The following endpoints skip authentication and Permit.io authorization entirely: -| Endpoint | Purpose | -|----------|---------| -| `GET /home` | Public home/menu page | -| `GET /logout` | Session termination | -| `GET /health` | Health check endpoint | +| Endpoint | Purpose | +| ---------------- | ------------------------ | +| `GET /home` | Public home/menu page | +| `GET /logout` | Session termination | +| `GET /health` | Health check endpoint | | `GET /swagger/*` | Swagger UI documentation | -| `GET /metrics` | Prometheus metrics | +| `GET /metrics` | Prometheus metrics | ### Special Authentication Endpoints These endpoints handle authentication flows and bypass normal authorization: -| Endpoint | Purpose | -|----------|---------| -| `GET /login` | Initiates OAuth2/PKCE login flow with Cognito | -| `GET /login-callback` | Handles OAuth2 callback from Cognito | +| Endpoint | Purpose | +| --------------------- | --------------------------------------------- | +| `GET /login` | Initiates OAuth2/PKCE login flow with Cognito | +| `GET /login-callback` | Handles OAuth2 callback from Cognito | ### Authorization Reference Table The following table shows what data is passed to Permit.io for each protected endpoint: -| HTTP Method | Endpoint | Resource | Action | Notes | -|-------------|----------|----------|--------|-------| -| **Client Management** | -| `POST` | `/client` | `client` | `post` | Create new client | -| `GET` | `/clients` | `clients` | `get` | List all clients | -| `GET` | `/client/{id}` | `client` | `get` | Get client details | -| `PATCH` | `/client/{id}` | `client` | `patch` | Update client | -| `GET` | `/client/{id}/status` | `client` | `get` | Get client sync status | -| **Document Operations** | -| `POST` | `/client/{id}/document` | `client` | `post` | Upload document | -| `GET` | `/client/{id}/document` | `client` | `get` | List client documents | -| `GET` | `/document/{id}` | `document` | `get` | Get document details | -| **Batch Document Operations** | -| `POST` | `/client/{id}/document/batch` | `client` | `post` | Upload batch (ZIP) | -| `GET` | `/client/{id}/document/batch` | `client` | `get` | List batch uploads | -| `GET` | `/client/{id}/document/batch/{batch_id}` | `client` | `get` | Get batch status | -| `DELETE` | `/client/{id}/document/batch/{batch_id}` | `client` | `delete` | Cancel batch upload | -| **Folder Operations** | -| `GET` | `/client/{id}/folders` | `client` | `get` | List client folders | -| `POST` | `/folders` | `folders` | `post` | Create folder | -| `PATCH` | `/folders/{folderId}` | `folders` | `patch` | Rename folder | -| `GET` | `/folders/{folderId}/documents` | `folders` | `get` | List folder documents | -| `GET` | `/folders/{folderId}/metrics` | `folders` | `get` | Get folder metrics | -| **Label Operations** | -| `POST` | `/documents/{documentId}/labels` | `documents` | `post` | Apply label to document | -| `GET` | `/documents/{documentId}/labels` | `documents` | `get` | Get document labels | -| `GET` | `/labels/{labelName}/documents` | `labels` | `get` | Get documents by label | +| HTTP Method | Endpoint | Resource | Action | Notes | +| ------------------------------- | ---------------------------------------- | ------------------- | -------- | ----------------------- | +| **Client Management** | +| `POST` | `/client` | `client` | `post` | Create new client | +| `GET` | `/clients` | `clients` | `get` | List all clients | +| `GET` | `/client/{id}` | `client` | `get` | Get client details | +| `PATCH` | `/client/{id}` | `client` | `patch` | Update client | +| `GET` | `/client/{id}/status` | `client` | `get` | Get client sync status | +| **Document Operations** | +| `POST` | `/client/{id}/document` | `client` | `post` | Upload document | +| `GET` | `/client/{id}/document` | `client` | `get` | List client documents | +| `GET` | `/document/{id}` | `document` | `get` | Get document details | +| **Batch Document Operations** | +| `POST` | `/client/{id}/document/batch` | `client` | `post` | Upload batch (ZIP) | +| `GET` | `/client/{id}/document/batch` | `client` | `get` | List batch uploads | +| `GET` | `/client/{id}/document/batch/{batch_id}` | `client` | `get` | Get batch status | +| `DELETE` | `/client/{id}/document/batch/{batch_id}` | `client` | `delete` | Cancel batch upload | +| **Folder Operations** | +| `GET` | `/client/{id}/folders` | `client` | `get` | List client folders | +| `POST` | `/folders` | `folders` | `post` | Create folder | +| `PATCH` | `/folders/{folderId}` | `folders` | `patch` | Rename folder | +| `GET` | `/folders/{folderId}/documents` | `folders` | `get` | List folder documents | +| `GET` | `/folders/{folderId}/metrics` | `folders` | `get` | Get folder metrics | +| **Label Operations** | +| `POST` | `/documents/{documentId}/labels` | `documents` | `post` | Apply label to document | +| `GET` | `/documents/{documentId}/labels` | `documents` | `get` | Get document labels | +| `GET` | `/labels/{labelName}/documents` | `labels` | `get` | Get documents by label | | **Field Extraction Operations** | -| `POST` | `/field-extractions` | `field-extractions` | `post` | Create field extraction | -| `GET` | `/field-extractions` | `field-extractions` | `get` | Get latest extraction | -| `GET` | `/field-extractions/version` | `field-extractions` | `get` | Get specific version | -| `GET` | `/field-extractions/history` | `field-extractions` | `get` | Get extraction history | -| **Query Management** | -| `GET` | `/query` | `query` | `get` | List queries | -| `POST` | `/query` | `query` | `post` | Create query | -| `GET` | `/query/{id}` | `query` | `get` | Get query details | -| `PATCH` | `/query/{id}` | `query` | `patch` | Update query | -| `POST` | `/query/{id}/test` | `query` | `post` | Test query execution | -| **Collector Configuration** | -| `GET` | `/client/{id}/collector` | `client` | `get` | Get collector config | -| `PATCH` | `/client/{id}/collector` | `client` | `patch` | Update collector config | -| **Export Operations** | -| `POST` | `/client/{id}/export` | `client` | `post` | Trigger export | -| `GET` | `/export/{id}` | `export` | `get` | Get export status | -| **User Administration** | -| `POST` | `/admin/users` | `admin` | `post` | Create user | -| `GET` | `/admin/users` | `admin` | `get` | List users | -| `GET` | `/admin/users/{email}` | `admin` | `get` | Get user details | -| `HEAD` | `/admin/users/{email}` | `admin` | `head` | Check user existence | -| `PATCH` | `/admin/users/{email}` | `admin` | `patch` | Update user | -| `DELETE` | `/admin/users/{email}` | `admin` | `delete` | Delete user | -| `POST` | `/admin/users/{email}/disable` | `admin` | `post` | Disable user | -| `POST` | `/admin/users/{email}/enable` | `admin` | `post` | Enable user | +| `POST` | `/field-extractions` | `field-extractions` | `post` | Create field extraction | +| `GET` | `/field-extractions` | `field-extractions` | `get` | Get latest extraction | +| `GET` | `/field-extractions/version` | `field-extractions` | `get` | Get specific version | +| `GET` | `/field-extractions/history` | `field-extractions` | `get` | Get extraction history | +| **Query Management** | +| `GET` | `/query` | `query` | `get` | List queries | +| `POST` | `/query` | `query` | `post` | Create query | +| `GET` | `/query/{id}` | `query` | `get` | Get query details | +| `PATCH` | `/query/{id}` | `query` | `patch` | Update query | +| `POST` | `/query/{id}/test` | `query` | `post` | Test query execution | +| **Collector Configuration** | +| `GET` | `/client/{id}/collector` | `client` | `get` | Get collector config | +| `PATCH` | `/client/{id}/collector` | `client` | `patch` | Update collector config | +| **Export Operations** | +| `POST` | `/client/{id}/export` | `client` | `post` | Trigger export | +| `GET` | `/export/{id}` | `export` | `get` | Get export status | +| **User Administration** | +| `POST` | `/admin/users` | `admin` | `post` | Create user | +| `GET` | `/admin/users` | `admin` | `get` | List users | +| `GET` | `/admin/users/{email}` | `admin` | `get` | Get user details | +| `HEAD` | `/admin/users/{email}` | `admin` | `head` | Check user existence | +| `PATCH` | `/admin/users/{email}` | `admin` | `patch` | Update user | +| `DELETE` | `/admin/users/{email}` | `admin` | `delete` | Delete user | +| `POST` | `/admin/users/{email}/disable` | `admin` | `post` | Disable user | +| `POST` | `/admin/users/{email}/enable` | `admin` | `post` | Enable user | ### Permit.io Configuration Requirements @@ -1281,45 +1411,46 @@ To configure Permit.io to work with this API, administrators must create: #### Resources -| Resource Key | Description | -|--------------|-------------| -| `client` | Client management and client-scoped operations | -| `clients` | Client listing (plural form) | -| `document` | Individual document operations | -| `documents` | Document collection and label operations | -| `folders` | Folder management | -| `labels` | Label-based document queries | -| `field-extractions` | Field extraction operations | -| `query` | Query definition management | -| `export` | Export operations | -| `admin` | User administration | +| Resource Key | Description | +| ------------------- | ---------------------------------------------- | +| `client` | Client management and client-scoped operations | +| `clients` | Client listing (plural form) | +| `document` | Individual document operations | +| `documents` | Document collection and label operations | +| `folders` | Folder management | +| `labels` | Label-based document queries | +| `field-extractions` | Field extraction operations | +| `query` | Query definition management | +| `export` | Export operations | +| `admin` | User administration | #### Actions -| Action Key | HTTP Method | Description | -|------------|-------------|-------------| -| `get` | GET | Read/retrieve resources | -| `post` | POST | Create resources | -| `patch` | PATCH | Update resources | -| `delete` | DELETE | Delete resources | -| `head` | HEAD | Check resource existence | +| Action Key | HTTP Method | Description | +| ---------- | ----------- | ------------------------ | +| `get` | GET | Read/retrieve resources | +| `post` | POST | Create resources | +| `patch` | PATCH | Update resources | +| `delete` | DELETE | Delete resources | +| `head` | HEAD | Check resource existence | #### Example Role Configuration Typical role assignments for common personas: -| Role | Resources | Actions | Use Case | -|------|-----------|---------|----------| -| `viewer` | `client`, `clients`, `document`, `documents`, `folders`, `query`, `export` | `get` | Read-only access | -| `editor` | `client`, `document`, `documents`, `folders`, `field-extractions`, `query` | `get`, `post`, `patch` | Content management | -| `admin` | All resources | All actions | Full administrative access | -| `user_admin` | `admin` | All actions | User management only | +| Role | Resources | Actions | Use Case | +| ------------ | -------------------------------------------------------------------------- | ---------------------- | -------------------------- | +| `viewer` | `client`, `clients`, `document`, `documents`, `folders`, `query`, `export` | `get` | Read-only access | +| `editor` | `client`, `document`, `documents`, `folders`, `field-extractions`, `query` | `get`, `post`, `patch` | Content management | +| `admin` | All resources | All actions | Full administrative access | +| `user_admin` | `admin` | All actions | User management only | ### Authorization Error Responses When authorization fails, the API returns: **HTTP 401 Unauthorized** - No valid JWT token: + ```json { "error": "Authorization required", @@ -1329,6 +1460,7 @@ When authorization fails, the API returns: ``` **HTTP 403 Forbidden** - Insufficient permissions: + ```json { "error": "Insufficient permissions", @@ -1344,18 +1476,22 @@ When authorization fails, the API returns: ## Code Generation ### OpenAPI Integration + The API is generated from the OpenAPI specification using: + ```bash oapi-codegen -generate "types,server" -o api.gen.go serviceAPIs/queryAPI.yaml ``` ### Generated Artifacts + - Type-safe Go structs for requests/responses - Echo framework route handlers - OpenAPI validation middleware - Swagger documentation ### Validation + - Automatic request validation against OpenAPI schema - Response validation in development mode - Parameter binding with type safety diff --git a/docs/ai.generated/07-configuration-management.md b/docs/ai.generated/07-configuration-management.md index ee73b7bc..ef5ab892 100644 --- a/docs/ai.generated/07-configuration-management.md +++ b/docs/ai.generated/07-configuration-management.md @@ -420,4 +420,141 @@ AWS_ENDPOINT_URL=http://test-localstack:4566 - **Override-friendly**: Allow easy configuration overrides for tests - **Isolation**: Each test should have independent configuration - **Realistic defaults**: Test configuration should mirror production patterns -- **Environment separation**: Clear separation between dev/test/prod configs \ No newline at end of file +- **Environment separation**: Clear separation between dev/test/prod configs + +## Setting Up Authentication for a New Environment + +This section covers the initial setup of authentication infrastructure when deploying to a new environment (e.g., dev, UAT, production). This includes configuring Permit.io for authorization and creating the initial bootstrap users in both AWS Cognito and Permit.io. + +### Prerequisites + +Before setting up authentication for a new environment, ensure you have: + +1. **AWS Cognito User Pool** already created for the environment +2. **AWS credentials** configured with permissions to create Cognito users +3. **Access to Permit.io** organization account + +### Step 1: Create Permit.io Environment and Get API Key + +1. Log in to the [Permit.io Dashboard](https://app.permit.io) +2. Navigate to your project (or create a new one for the environment) +3. Create a new **Environment** for your deployment (e.g., `Development`, `UAT`, `Production`) +4. Go to **Settings → API Keys** and create a new API key scoped to this environment +5. Copy the API key - you will need it for the following steps + +**Important**: Each environment (dev, UAT, prod) requires its own separate Permit.io environment and API key. + +### Step 2: Clean Up Default Permit.io Roles + +When you create a new environment in Permit.io, it automatically creates default `admin`, `viewer`, and `editor` roles. These must be deleted before running the setup tool because they conflict with the role names used by this system. + +1. In the Permit.io Dashboard, navigate to **Policy → Roles** +2. Delete the default `admin`, `viewer`, and `editor` roles +3. Verify no roles remain before proceeding + +### Step 3: Run the Permit.io Setup Tool + +The permit setup tool configures all required resources and roles in Permit.io based on the `permit_policies.yaml` configuration file. + +```bash +cd cmd/cognito_test/permit.setup + +# Set the API key for your target environment +export PERMIT_API_KEY="permit_key_your_environment_key_here" + +# Verify connection (list current configuration) +go run setup_permit.go -list + +# Dry run to preview changes +go run setup_permit.go -config permit_policies.yaml -project default -env -dry-run + +# Apply the configuration +go run setup_permit.go -config permit_policies.yaml -project default -env +``` + +This creates: +- All required **resources** (admin, client, document, folders, etc.) with their actions +- All required **roles** (super_admin, user_admin, auditor, editor, viewer, etc.) with appropriate permissions + +See `cmd/cognito_test/permit.setup/README.md` for detailed documentation. + +### Step 4: Create Bootstrap Users + +After Permit.io is configured, use the user creation tool to create the initial users in both AWS Cognito and Permit.io. The tool ensures that users are created with matching Subject IDs in both systems, which is critical for authentication to work correctly. + +**Important**: Always use the user creation tool to create users. Do not manually create users in Cognito and Permit.io separately, as this will result in mismatched Subject IDs and authentication failures. + +```bash +cd cmd/cognito_test/user.creation.tool + +# Configure environment variables +export PERMIT_KEY="permit_key_your_environment_key_here" +export COGNITO_USER_POOL_ID="us-east-1_xxxxxxxxx" +export AWS_REGION="us-east-1" +# Either use AWS credentials or an SSO profile +export AWS_PROFILE="your-sso-profile" # if using SSO + +# Create a CSV file with your bootstrap users +# Format: email,first_name,last_name,role1,role2,... +cat > bootstrap_users.csv << 'EOF' +email,first_name,last_name,super_admin,user_admin +admin@yourcompany.com,Admin,User,super_admin, +useradmin@yourcompany.com,User,Admin,,user_admin +EOF + +# Dry run to preview +go run main.go -project use-key -csv bootstrap_users.csv --dry-run + +# Create the users +go run main.go -project use-key -csv bootstrap_users.csv +``` + +The tool will: +1. Create each user in AWS Cognito (with email as username) +2. Create corresponding user in Permit.io using the Cognito Subject ID as the key +3. Assign the specified roles in Permit.io + +See `cmd/cognito_test/user.creation.tool/readme.md` for detailed documentation including: +- Full CSV format specification +- Delete, disable, and enable operations +- Audit logging and compliance features +- Troubleshooting guide + +### Step 5: Verify Setup + +After creating bootstrap users: + +1. **Test Cognito login**: Verify users can authenticate through the Cognito hosted UI or your application +2. **Test API access**: Make an authenticated request to verify authorization works: + ```bash + # Get a token (via Cognito login flow) + # Then test the identity endpoint + curl -H "Authorization: Bearer " https://your-api/identity + ``` +3. **Check Permit.io Dashboard**: Verify users appear with correct role assignments + +### Environment-Specific Scripts + +For convenience, you can create environment-specific run scripts: + +```bash +# Example: run.tool.prod.sh +#!/bin/bash +export PERMIT_KEY="permit_key_production_key_here" +export COGNITO_USER_POOL_ID="us-east-1_prodPoolId" +export AWS_REGION="us-east-1" +export AWS_PROFILE="production-profile" + +go run main.go -project use-key -csv "$@" +``` + +### Quick Reference: New Environment Checklist + +| Step | Action | Tool/Location | +|------|--------|---------------| +| 1 | Create Permit.io environment | Permit.io Dashboard | +| 2 | Get environment API key | Permit.io Dashboard → Settings → API Keys | +| 3 | Delete default roles | Permit.io Dashboard → Policy → Roles | +| 4 | Run permit setup | `cmd/cognito_test/permit.setup/` | +| 5 | Create bootstrap users | `cmd/cognito_test/user.creation.tool/` | +| 6 | Verify authentication | Test login and API access | \ No newline at end of file diff --git a/docs/ai.generated/09-authentication-guide.md b/docs/ai.generated/09-authentication-guide.md new file mode 100644 index 00000000..ea01f47f --- /dev/null +++ b/docs/ai.generated/09-authentication-guide.md @@ -0,0 +1,945 @@ +# UI Client Authentication Guide + +This document provides technical instructions for frontend (UI) developers integrating with the Query Orchestration API. It covers how to authenticate requests, discover user permissions, handle errors, and configure CORS for cross-origin deployments. + +## Table of Contents + +1. [Authentication Overview](#authentication-overview) +2. [Sending Authenticated Requests](#sending-authenticated-requests) +3. [Discovering User Permissions](#discovering-user-permissions) +4. [Error Handling](#error-handling) +5. [Token Refresh Strategy](#token-refresh-strategy) +6. [CORS Configuration](#cors-configuration) +7. [Security Best Practices](#security-best-practices) +8. [Quick Reference](#quick-reference) + +--- + +## Authentication Overview + +The Query Orchestration API uses **JWT (JSON Web Token) authentication** with tokens issued by **AWS Cognito**. The authentication flow assumes that the UI application handles user login directly with Cognito and obtains the necessary tokens. + +### Token Types + +When a user authenticates with Cognito, your UI receives three tokens: + +| Token | Purpose | Typical Lifetime | +|-------|---------|------------------| +| `access_token` | **Used for API requests** - Contains user identity and scopes | 1 hour | +| `id_token` | Contains user profile claims (email, name, etc.) | 1 hour | +| `refresh_token` | Used to obtain new access/id tokens without re-authentication | 30 days | + +**Important**: The Query Orchestration API validates the `access_token`. Always use this token (not the `id_token`) in the `Authorization` header. + +### Authentication Flow + +``` +┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ ┌─────────────┐ +│ UI App │ │ Cognito │ │ Query Orchestration│ │ Permit.io │ +│ (Frontend) │ │ (IDP) │ │ API │ │ (AuthZ) │ +└──────┬──────┘ └──────┬──────┘ └──────────┬──────────┘ └──────┬──────┘ + │ │ │ │ + │ 1. Login request │ │ │ + │───────────────────>│ │ │ + │ │ │ │ + │ 2. Tokens │ │ │ + │<───────────────────│ │ │ + │ │ │ │ + │ 3. API Request + Bearer Token │ │ + │────────────────────────────────────────────>│ │ + │ │ │ │ + │ │ │ 4. Check Permission │ + │ │ │───────────────────────>│ + │ │ │ │ + │ │ │ 5. Allowed/Denied │ + │ │ │<───────────────────────│ + │ │ │ │ + │ 6. API Response │ │ + │<────────────────────────────────────────────│ │ +``` + +--- + +## Sending Authenticated Requests + +### Authorization Header Format + +All authenticated API requests must include the JWT access token in the `Authorization` header using the Bearer scheme: + +``` +Authorization: Bearer +``` + +### Accepted Token Locations + +The API accepts tokens from exactly two locations: + +| Location | Format | Notes | +|----------|--------|-------| +| `Authorization` header | `Bearer ` | **Recommended for cross-origin requests** | +| `auth_token` cookie | JWT string | Used by cookie-based auth flow | + +**Not supported**: +- Query parameters (e.g., `?token=...`) +- Other header names +- Request body + +### Cookie-Based Authentication (Alternative) + +If your deployment uses cookie-based authentication, the server uses these HTTP-only cookies: + +| Cookie Name | Purpose | +|-------------|---------| +| `auth_token` | Access token (JWT) | +| `refresh_token` | Refresh token for obtaining new access tokens | + +When cookies are present, the server middleware automatically copies the `auth_token` cookie value to the `Authorization` header internally. The server can also automatically refresh expired tokens using the `refresh_token` cookie. + +**Note**: Cookie-based auth requires CORS configuration for cross-origin deployments. See [CORS Configuration](#cors-configuration). + +### No CSRF Token Required + +The API does not require CSRF tokens. JWT bearer token authentication provides protection against CSRF attacks because: +- The token must be explicitly included in the `Authorization` header +- Browsers do not automatically attach custom headers to cross-origin requests + +### JavaScript/TypeScript Examples + +#### Using Fetch API + +```javascript +const API_BASE_URL = 'https://api.yourdomain.com'; + +async function makeAuthenticatedRequest(endpoint, options = {}) { + const accessToken = getAccessToken(); // Retrieve from your token storage + + const response = await fetch(`${API_BASE_URL}${endpoint}`, { + ...options, + headers: { + 'Authorization': `Bearer ${accessToken}`, + 'Content-Type': 'application/json', + ...options.headers, + }, + }); + + if (!response.ok) { + await handleErrorResponse(response); + } + + return response.json(); +} + +// Example: Get client details +async function getClient(clientId) { + return makeAuthenticatedRequest(`/client/${clientId}`); +} + +// Example: Upload a document +async function uploadDocument(clientId, file, folderId = null) { + const accessToken = getAccessToken(); + const formData = new FormData(); + formData.append('file', file); + if (folderId) { + formData.append('folderId', folderId); + } + + const response = await fetch(`${API_BASE_URL}/client/${clientId}/document`, { + method: 'POST', + headers: { + 'Authorization': `Bearer ${accessToken}`, + // Note: Do NOT set Content-Type for FormData - browser sets it automatically + }, + body: formData, + }); + + if (!response.ok) { + await handleErrorResponse(response); + } + + return response.json(); +} +``` + +#### Using Axios + +```javascript +import axios from 'axios'; + +const apiClient = axios.create({ + baseURL: 'https://api.yourdomain.com', +}); + +// Add auth token to every request +apiClient.interceptors.request.use((config) => { + const accessToken = getAccessToken(); + if (accessToken) { + config.headers.Authorization = `Bearer ${accessToken}`; + } + return config; +}); + +// Handle 401 errors globally +apiClient.interceptors.response.use( + (response) => response, + async (error) => { + if (error.response?.status === 401) { + // Token expired or invalid - attempt refresh or redirect to login + const refreshed = await attemptTokenRefresh(); + if (refreshed) { + // Retry the original request with new token + error.config.headers.Authorization = `Bearer ${getAccessToken()}`; + return apiClient.request(error.config); + } + // Refresh failed - redirect to login + redirectToLogin(); + } + return Promise.reject(error); + } +); + +// Example usage +const client = await apiClient.get('/client/AAA'); +const documents = await apiClient.get('/client/AAA/document'); +``` + +#### Using React Query + +```typescript +import { useQuery, useMutation, QueryClient } from '@tanstack/react-query'; + +const queryClient = new QueryClient(); + +// Fetch wrapper with auth +async function authFetch(endpoint: string, options?: RequestInit): Promise { + const accessToken = getAccessToken(); + + const response = await fetch(`${API_BASE_URL}${endpoint}`, { + ...options, + headers: { + 'Authorization': `Bearer ${accessToken}`, + 'Content-Type': 'application/json', + ...options?.headers, + }, + }); + + if (response.status === 401) { + // Trigger re-authentication + throw new AuthenticationError('Token expired'); + } + + if (!response.ok) { + throw new ApiError(response.status, await response.json()); + } + + return response.json(); +} + +// Hook for fetching client data +function useClient(clientId: string) { + return useQuery({ + queryKey: ['client', clientId], + queryFn: () => authFetch(`/client/${clientId}`), + retry: (failureCount, error) => { + // Don't retry on auth errors + if (error instanceof AuthenticationError) return false; + return failureCount < 3; + }, + }); +} + +// Hook for fetching user identity/permissions +function useIdentity() { + return useQuery({ + queryKey: ['identity'], + queryFn: () => authFetch('/identity'), + staleTime: 5 * 60 * 1000, // Cache for 5 minutes + }); +} +``` + +--- + +## Discovering User Permissions + +### The `/identity` Endpoint + +After authentication, use the `/identity` endpoint to discover what the current user is authorized to do. This endpoint returns the user's roles and associated permissions from Permit.io. + +**Endpoint**: `GET /identity` + +**Authentication**: Required (Bearer token) + +**Response** (200 OK): + +```json +{ + "roles": [ + { + "key": "auditor", + "name": "Auditor", + "description": "Read-only access to view documents and exports", + "permissions": [ + "client:get", + "document:get", + "folders:get", + "export:get" + ] + }, + { + "key": "editor", + "name": "Editor", + "description": "Can create and modify documents", + "permissions": [ + "client:get", + "client:post", + "document:get", + "document:post", + "folders:get", + "folders:post", + "folders:patch" + ] + } + ] +} +``` + +### Permission Format + +Permissions follow the format `resource:action`: + +| Resource | Actions | Description | +|----------|---------|-------------| +| `client` | `get`, `post`, `patch`, `delete` | Client management | +| `clients` | `get` | List all clients | +| `document` | `get`, `post`, `patch`, `delete` | Document operations | +| `folders` | `get`, `post`, `patch`, `delete` | Folder management | +| `query` | `get`, `post`, `patch` | Query definitions | +| `export` | `get`, `post` | Export operations | +| `admin` | `get`, `post`, `patch`, `delete` | User administration | + +### Using Permissions in UI + +```typescript +interface IdentityRole { + key: string; + name: string; + description?: string; + permissions: string[]; +} + +interface IdentityResponse { + roles: IdentityRole[]; +} + +// Fetch and cache user permissions +let cachedPermissions: Set | null = null; + +async function loadUserPermissions(): Promise> { + if (cachedPermissions) { + return cachedPermissions; + } + + const response = await authFetch('/identity'); + + // Flatten all permissions from all roles + const permissions = new Set(); + for (const role of response.roles) { + for (const permission of role.permissions) { + permissions.add(permission); + } + } + + cachedPermissions = permissions; + return permissions; +} + +// Check if user has a specific permission +async function hasPermission(resource: string, action: string): Promise { + const permissions = await loadUserPermissions(); + return permissions.has(`${resource}:${action}`); +} + +// Check if user can perform an action +async function canUploadDocuments(): Promise { + return hasPermission('client', 'post'); // Upload uses POST /client/{id}/document +} + +async function canManageUsers(): Promise { + return hasPermission('admin', 'post'); +} + +// React hook example +function usePermission(resource: string, action: string) { + const { data: identity, isLoading } = useIdentity(); + + const hasPermission = useMemo(() => { + if (!identity) return false; + const permission = `${resource}:${action}`; + return identity.roles.some(role => + role.permissions.includes(permission) + ); + }, [identity, resource, action]); + + return { hasPermission, isLoading }; +} + +// Usage in component +function UploadButton() { + const { hasPermission, isLoading } = usePermission('client', 'post'); + + if (isLoading) return ; + if (!hasPermission) return null; // Hide button if not authorized + + return ; +} +``` + +### Role-Based UI Rendering + +```typescript +// Define UI feature permissions +const UI_FEATURES = { + viewDocuments: ['client:get', 'document:get'], + uploadDocuments: ['client:post'], + manageClients: ['client:post', 'client:patch'], + viewExports: ['export:get'], + createExports: ['export:post'], + adminPanel: ['admin:get'], +} as const; + +function canAccessFeature( + userPermissions: Set, + feature: keyof typeof UI_FEATURES +): boolean { + const requiredPermissions = UI_FEATURES[feature]; + return requiredPermissions.every(p => userPermissions.has(p)); +} +``` + +--- + +## Error Handling + +### HTTP Status Codes + +| Status | Meaning | UI Action | +|--------|---------|-----------| +| `200` | Success | Process response | +| `201` | Created | Resource created successfully | +| `204` | No Content | Operation successful (no body) | +| `400` | Bad Request | Show validation error to user | +| `401` | Unauthorized | Token invalid/expired - refresh or re-login | +| `403` | Forbidden | User lacks permission - show access denied | +| `404` | Not Found | Resource doesn't exist | +| `429` | Too Many Requests | Rate limited - implement backoff | +| `500` | Server Error | Show generic error, retry later | +| `502` | Bad Gateway | Authorization service unavailable | + +### Error Response Format + +Error responses typically include a `message` field, but the structure varies by endpoint and error type: + +```json +// Standard error format +{ + "message": "Description of what went wrong" +} + +// Authentication error format (401) +{ + "error": "Authorization required", + "message": "This endpoint requires authentication. Please obtain a valid JWT token.", + "login_url": "/login" +} + +// Authorization error format (403) +{ + "error": "Access denied" +} +``` + +**Recommended error handling**: Check for both `error` and `message` fields when parsing error responses. + +### Handling Authentication Errors (401) + +```typescript +// Helper to extract error message from varying response formats +function getErrorMessage(body: any, fallback: string): string { + return body.message || body.error || fallback; +} + +async function handleErrorResponse(response: Response) { + const body = await response.json().catch(() => ({})); + + switch (response.status) { + case 401: + // Token expired or invalid + console.log('Authentication required:', getErrorMessage(body, 'Authentication required')); + + // Attempt to refresh the token (only if using header-based auth) + const refreshed = await attemptTokenRefresh(); + if (!refreshed) { + // Clear stored tokens + clearTokens(); + // Redirect to login + redirectToLogin(); + } + throw new AuthenticationError(getErrorMessage(body, 'Authentication required')); + + case 403: + // User authenticated but not authorized + console.log('Access denied:', getErrorMessage(body, 'Access denied')); + throw new AuthorizationError(getErrorMessage(body, 'Access denied')); + + case 429: + // Rate limited - check RateLimit header + const rateLimitInfo = response.headers.get('RateLimit'); + console.log('Rate limited:', rateLimitInfo); + throw new RateLimitError(getErrorMessage(body, 'Rate limit exceeded'), rateLimitInfo); + + default: + throw new ApiError(response.status, getErrorMessage(body, 'Request failed')); + } +} +``` + +### Rate Limiting + +The API includes rate limiting. Monitor the `RateLimit` response header: + +``` +RateLimit: 100;window=60 +``` + +This indicates 100 requests allowed per 60-second window. + +```typescript +// Implement exponential backoff for rate limits +async function fetchWithRetry(url: string, options: RequestInit, maxRetries = 3) { + for (let attempt = 0; attempt < maxRetries; attempt++) { + const response = await fetch(url, options); + + if (response.status === 429) { + const retryAfter = response.headers.get('Retry-After'); + const waitTime = retryAfter + ? parseInt(retryAfter) * 1000 + : Math.pow(2, attempt) * 1000; + + console.log(`Rate limited. Retrying in ${waitTime}ms`); + await sleep(waitTime); + continue; + } + + return response; + } + + throw new Error('Max retries exceeded'); +} +``` + +--- + +## Token Refresh Strategy + +### Server-Side vs Client-Side Refresh + +The refresh strategy depends on how your UI sends authentication: + +| Authentication Method | Who Handles Refresh | Notes | +|-----------------------|---------------------|-------| +| `Authorization` header | **UI must refresh** | Server has no access to refresh token | +| Cookie-based | **Server auto-refreshes** | Server reads `refresh_token` cookie | + +**If using `Authorization: Bearer` header** (recommended for cross-origin): +- The UI must store and manage the refresh token +- The UI must call Cognito directly to refresh tokens +- The server cannot refresh for you (it doesn't have your refresh token) + +**If using cookie-based authentication**: +- The server automatically refreshes expired tokens +- New tokens are returned as updated cookies +- No action required from the UI + +### When to Refresh (Header-Based Auth) + +The access token has a limited lifetime (typically 1 hour). Implement proactive refresh to avoid authentication errors: + +1. **Proactive refresh**: Check token expiration before each request +2. **Reactive refresh**: Refresh when receiving a 401 response + +### Checking Token Expiration + +JWT tokens contain an `exp` claim with the expiration timestamp: + +```typescript +function isTokenExpired(token: string, bufferSeconds = 60): boolean { + try { + // Decode JWT payload (base64) + const payload = JSON.parse(atob(token.split('.')[1])); + const expiresAt = payload.exp * 1000; // Convert to milliseconds + const now = Date.now(); + + // Consider expired if within buffer period + return now >= (expiresAt - bufferSeconds * 1000); + } catch { + return true; // Treat decode errors as expired + } +} + +function getAccessToken(): string | null { + const token = localStorage.getItem('access_token'); + + if (!token || isTokenExpired(token)) { + return null; + } + + return token; +} +``` + +### Refreshing Tokens with Cognito + +Use the Cognito SDK or direct API call to refresh tokens: + +```typescript +import { + CognitoIdentityProviderClient, + InitiateAuthCommand +} from '@aws-sdk/client-cognito-identity-provider'; + +const cognitoClient = new CognitoIdentityProviderClient({ + region: 'us-east-2', // Your Cognito region +}); + +async function refreshTokens(): Promise { + const refreshToken = localStorage.getItem('refresh_token'); + + if (!refreshToken) { + return false; + } + + try { + const command = new InitiateAuthCommand({ + AuthFlow: 'REFRESH_TOKEN_AUTH', + ClientId: 'your-cognito-client-id', + AuthParameters: { + REFRESH_TOKEN: refreshToken, + }, + }); + + const response = await cognitoClient.send(command); + + if (response.AuthenticationResult) { + // Store new tokens + localStorage.setItem('access_token', response.AuthenticationResult.AccessToken!); + localStorage.setItem('id_token', response.AuthenticationResult.IdToken!); + // Note: Refresh token may not be returned - keep existing one + + return true; + } + + return false; + } catch (error) { + console.error('Token refresh failed:', error); + return false; + } +} +``` + +### Request Interceptor with Auto-Refresh + +```typescript +async function authFetchWithRefresh( + endpoint: string, + options?: RequestInit +): Promise { + // Check if token needs refresh before request + let accessToken = localStorage.getItem('access_token'); + + if (!accessToken || isTokenExpired(accessToken)) { + const refreshed = await refreshTokens(); + if (!refreshed) { + throw new AuthenticationError('Session expired. Please log in again.'); + } + accessToken = localStorage.getItem('access_token'); + } + + const response = await fetch(`${API_BASE_URL}${endpoint}`, { + ...options, + headers: { + 'Authorization': `Bearer ${accessToken}`, + 'Content-Type': 'application/json', + ...options?.headers, + }, + }); + + // Handle 401 in case token expired between check and request + if (response.status === 401) { + const refreshed = await refreshTokens(); + if (refreshed) { + // Retry with new token + return authFetchWithRefresh(endpoint, options); + } + throw new AuthenticationError('Session expired. Please log in again.'); + } + + if (!response.ok) { + throw new ApiError(response.status, await response.json()); + } + + return response.json(); +} +``` + +--- + +## CORS Configuration + +### Overview + +Since the UI and API are hosted on different domains, Cross-Origin Resource Sharing (CORS) must be configured. The API uses Echo framework's CORS middleware with the following **default** configuration: + +- **Allowed Origins**: `*` (all origins) +- **Allowed Methods**: `GET, HEAD, PUT, PATCH, POST, DELETE` +- **Allowed Headers**: All headers +- **Credentials**: **NOT enabled by default** (`AllowCredentials: false`) + +**Important**: The default CORS configuration does **not** support credentials (cookies). This means: +- Cross-origin requests with `credentials: 'include'` will fail +- Cookie-based authentication will not work cross-origin without server configuration changes +- **Use `Authorization: Bearer` header for cross-origin authentication** (recommended) + +### UI Configuration for Cross-Origin Requests + +For cross-origin requests, use the `Authorization` header (not cookies): + +```typescript +// Recommended: Use Authorization header for cross-origin requests +const response = await fetch(`${API_BASE_URL}/client/AAA`, { + method: 'GET', + headers: { + 'Authorization': `Bearer ${accessToken}`, + 'Content-Type': 'application/json', + }, +}); +``` + +**Cookie-based authentication cross-origin** requires server-side CORS configuration changes: +1. `AllowCredentials: true` must be set on the server +2. `AllowOrigins` must list specific origins (cannot use wildcard `*`) +3. UI must use `credentials: 'include'` in fetch calls + +```typescript +// Cookie-based auth (ONLY works if server is configured for credentials) +const response = await fetch(`${API_BASE_URL}/client/AAA`, { + method: 'GET', + credentials: 'include', // Sends cookies cross-origin + headers: { + 'Content-Type': 'application/json', + // Authorization header optional if using cookies + }, +}); +``` + +### Preflight Requests + +For non-simple requests (e.g., with custom headers like `Authorization`), the browser sends a preflight `OPTIONS` request. The API automatically handles these. + +**Preflight triggers include**: +- Custom headers (e.g., `Authorization`) +- Content-Type other than `application/x-www-form-urlencoded`, `multipart/form-data`, or `text/plain` +- HTTP methods other than `GET`, `HEAD`, `POST` + +### Production CORS Configuration + +For production deployments with specific UI domains, the API CORS configuration can be customized. Contact the backend team to configure: + +```go +// Example production CORS config (server-side) +e.Use(middleware.CORSWithConfig(middleware.CORSConfig{ + AllowOrigins: []string{ + "https://app.yourdomain.com", + "https://admin.yourdomain.com", + }, + AllowMethods: []string{ + http.MethodGet, + http.MethodHead, + http.MethodPut, + http.MethodPatch, + http.MethodPost, + http.MethodDelete, + }, + AllowHeaders: []string{ + "Authorization", + "Content-Type", + "Accept", + "X-Requested-With", + }, + AllowCredentials: true, + MaxAge: 86400, // Preflight cache: 24 hours +})) +``` + +### Troubleshooting CORS Issues + +| Error | Cause | Solution | +|-------|-------|----------| +| `No 'Access-Control-Allow-Origin' header` | Origin not allowed | Add UI origin to server config | +| `Preflight response is not successful` | OPTIONS blocked | Ensure server handles OPTIONS | +| `Credentials flag is true, but...` | Wildcard origin with credentials | Use specific origins, not `*` | +| `Request header not allowed` | Custom header blocked | Add header to `AllowHeaders` | + +--- + +## Security Best Practices + +### Token Storage + +| Storage Option | Security | Recommendation | +|----------------|----------|----------------| +| `localStorage` | Vulnerable to XSS | Use with caution | +| `sessionStorage` | Cleared on tab close, vulnerable to XSS | Better than localStorage | +| Memory (variable) | Lost on refresh, most secure | Best for high-security apps | +| HTTP-only cookies | Protected from XSS | Requires server-side support | + +**Recommended approach for SPAs**: + +```typescript +// Store tokens in memory with sessionStorage backup +class TokenManager { + private accessToken: string | null = null; + private idToken: string | null = null; + private refreshToken: string | null = null; + + constructor() { + // Restore from sessionStorage on page load + this.accessToken = sessionStorage.getItem('access_token'); + this.idToken = sessionStorage.getItem('id_token'); + this.refreshToken = sessionStorage.getItem('refresh_token'); + } + + setTokens(tokens: { accessToken: string; idToken: string; refreshToken: string }) { + this.accessToken = tokens.accessToken; + this.idToken = tokens.idToken; + this.refreshToken = tokens.refreshToken; + + // Backup to sessionStorage (cleared on tab close) + sessionStorage.setItem('access_token', tokens.accessToken); + sessionStorage.setItem('id_token', tokens.idToken); + sessionStorage.setItem('refresh_token', tokens.refreshToken); + } + + getAccessToken(): string | null { + return this.accessToken; + } + + clearTokens() { + this.accessToken = null; + this.idToken = null; + this.refreshToken = null; + sessionStorage.clear(); + } +} + +export const tokenManager = new TokenManager(); +``` + +### Content Security Policy + +Configure CSP headers in your UI application to prevent XSS: + +```html + +``` + +### Additional Security Measures + +1. **Always use HTTPS** for both UI and API +2. **Validate token on the client** before making requests +3. **Implement logout** that clears all stored tokens +4. **Set reasonable token lifetimes** in Cognito configuration +5. **Monitor for suspicious activity** via API logs + +--- + +## Quick Reference + +### Required Headers + +``` +Authorization: Bearer +Content-Type: application/json +``` + +### Common Endpoints + +| Endpoint | Method | Description | +|----------|--------|-------------| +| `/identity` | GET | Get user roles and permissions | +| `/health` | GET | API health check (no auth required) | +| `/clients` | GET | List all clients | +| `/client/{id}` | GET | Get client details | +| `/client/{id}/document` | GET | List client documents | +| `/client/{id}/document` | POST | Upload document (multipart/form-data) | +| `/client/{id}/document/batch` | POST | Upload ZIP batch | +| `/document/{id}` | GET | Get document details | +| `/clients/{clientId}/folders` | GET | List folders | + +### Public Endpoints (No Authentication Required) + +| Endpoint | Description | +|----------|-------------| +| `/health` | Health check and version info | +| `/swagger/*` | API documentation | +| `/metrics` | Prometheus metrics | + +### TypeScript Type Definitions + +```typescript +// Identity types +interface IdentityRole { + key: string; + name: string; + description?: string; + permissions: string[]; +} + +interface IdentityResponse { + roles: IdentityRole[]; +} + +// Error types +interface ErrorResponse { + message: string; +} + +// Client types +interface Client { + client_id: string; + name: string; + can_sync: boolean; +} + +// Document types +interface Document { + id: string; + client_id: string; + hash: string; + filename?: string; + folder_id?: string; + fields?: Record; +} +``` + +--- + +## Support + +For API issues or questions: +- Check the Swagger documentation at `/swagger/index.html` +- Review API logs for detailed error information +- Contact the backend team for CORS or authentication configuration changes diff --git a/queryapi.summary.md b/docs/ai.generated/queryapi.summary.md similarity index 60% rename from queryapi.summary.md rename to docs/ai.generated/queryapi.summary.md index caba46d3..b21e29b1 100644 --- a/queryapi.summary.md +++ b/docs/ai.generated/queryapi.summary.md @@ -2,6 +2,46 @@ This document provides a comprehensive summary of all REST API endpoints available in the Query Orchestration Platform. +## Quick Reference (All Endpoints) + +| Path | Methods | Service | +|----------------------------------------|--------------------------|----------------------| +| /login | GET | AuthService | +| /login-callback | GET | AuthService | +| /logout | GET | AuthService | +| /home | GET | AuthService | +| /identity | GET | AuthService | +| /client | POST | ClientService | +| /clients | GET | ClientService | +| /client/{id} | GET, PATCH | ClientService | +| /client/{id}/status | GET | ClientService | +| /client/{id}/collector | GET, PATCH | CollectorService | +| /client/{id}/folders | GET | FolderService | +| /folders | POST | FolderService | +| /folders/{folderId} | PATCH | FolderService | +| /folders/{folderId}/documents | GET | FolderService | +| /folders/{folderId}/metrics | GET | FolderService | +| /client/{id}/document | GET, POST | DocumentsService | +| /client/{id}/document/batch | GET, POST | DocumentsService | +| /client/{id}/document/batch/{batch_id} | GET, DELETE | DocumentsService | +| /document/{id} | GET | DocumentsService | +| /documents/{documentId}/labels | GET, POST | LabelService | +| /labels/{labelName}/documents | GET | LabelService | +| /field-extractions | GET, POST | FieldExtractionService | +| /field-extractions/version | GET | FieldExtractionService | +| /field-extractions/history | GET | FieldExtractionService | +| /query | GET, POST | QueryService | +| /query/{id} | GET, PATCH | QueryService | +| /query/{id}/test | POST | QueryService | +| /client/{id}/export | POST | ExportService | +| /export/{id} | GET | ExportService | +| /admin/users | GET, POST | AdminService | +| /admin/users/{email} | GET, HEAD, PATCH, DELETE | AdminService | +| /admin/users/{email}/disable | POST | AdminService | +| /admin/users/{email}/enable | POST | AdminService | + +--- + ## Authentication Service ### OAuth2 Authentication Flow @@ -23,6 +63,11 @@ This document provides a comprehensive summary of all REST API endpoints availab - Requires authentication - Returns HTML content for authenticated user menu +- **GET /identity** - Get current user identity and roles + - Requires authentication + - Returns user's assigned roles from Permit.io with permissions + - Returns: `{user_id, email, roles[]}` + ## Client Service ### Client Management @@ -30,6 +75,9 @@ This document provides a comprehensive summary of all REST API endpoints availab - Request Body: `{id, name}` - Returns: `{id}` (201 Created) +- **GET /clients** - List all clients in the system + - Returns: `{clients[]}` where each client contains `{id, name}` + - **GET /client/{id}** - Get a specific client by its ID - Returns: `{id, name, can_sync}` (200 OK) @@ -50,6 +98,31 @@ This document provides a comprehensive summary of all REST API endpoints availab - Request Body: `{minimum_cleaner_version?, minimum_text_version?, active_version?, fields[]?}` - Returns: 204 No Content on success +## Folder Service + +### Folder Management +- **GET /client/{id}/folders** - List all folders for a client + - Query Parameters: + - `metrics` (optional, default: false) - When true, includes processing metrics for each folder + - Returns: `{folders[]}` where each folder contains `{id, path, parent_id, metrics?}` + - Root-level folders have null parentId + +- **POST /folders** - Create a new folder + - Request Body: `{client_id, path, parent_id?}` + - Returns: `{id, client_id, path, parent_id}` (201 Created) + +- **PATCH /folders/{folderId}** - Rename a folder + - Request Body: `{path}` + - Returns: `{id, client_id, path, parent_id}` (200 OK) + +- **GET /folders/{folderId}/documents** - Get documents in a folder + - Query Parameters: + - `textRecord` (optional, default: false) - When true, includes full text extraction record + - Returns: `{documents[]}` with document details including filename and labels + +- **GET /folders/{folderId}/metrics** - Get folder processing metrics + - Returns: `{total_documents, processed_documents, label_counts{}}` + ## Documents Service ### Document Management @@ -64,7 +137,9 @@ This document provides a comprehensive summary of all REST API endpoints availab - Returns: 200 OK on successful upload - **GET /document/{id}** - Get document details by its ID - - Returns: `{id, client_id, hash, fields{}}` + - Query Parameters: + - `textRecord` (optional, default: false) - When true, includes full text extraction record + - Returns: `{id, client_id, hash, folder_id, filename, labels[], fields{}, text_record?}` ### Batch Document Processing - **POST /client/{id}/document/batch** - Upload multiple PDFs as ZIP archive for batch processing @@ -85,6 +160,48 @@ This document provides a comprehensive summary of all REST API endpoints availab - **DELETE /client/{id}/document/batch/{batch_id}** - Cancel a batch upload currently processing - Returns: 204 No Content on successful cancellation +## Label Service + +### Document Label Management +- **POST /documents/{documentId}/labels** - Apply a label to a document + - Request Body: `{label_name}` + - Labels track document processing state/workflow + - Returns: `{id, document_id, label_name, applied_at}` (201 Created) + +- **GET /documents/{documentId}/labels** - Get all labels for a document + - Returns: `{labels[]}` ordered by most recent first + - Each label contains: `{id, document_id, label_name, applied_at}` + +- **GET /labels/{labelName}/documents** - Get all documents with a specific label + - Query Parameters: + - `clientId` (required) - Filter by client ID + - Returns: `{documents[]}` with document summaries + +## Field Extraction Service + +### Document Field Extraction Management +- **POST /field-extractions** - Create a new field extraction + - Request Body: `{document_id, single_fields{}, array_fields{}}` + - Creates versioned field extraction record + - Returns: `{id, document_id, version, single_fields{}, array_fields{}, created_at}` (201 Created) + +- **GET /field-extractions** - Get current (most recent) field extraction for a document + - Query Parameters: + - `documentId` (required) - The document ID + - Returns: `{id, document_id, version, single_fields{}, array_fields{}, created_at}` + +- **GET /field-extractions/version** - Get field extraction by specific version + - Query Parameters: + - `documentId` (required) - The document ID + - `version` (required) - The version number to retrieve + - Returns: `{id, document_id, version, single_fields{}, array_fields{}, created_at}` + +- **GET /field-extractions/history** - Get field extraction version history + - Query Parameters: + - `documentId` (required) - The document ID + - Returns: `{versions[]}` ordered by most recent first + - Each version contains: `{version, created_at}` + ## Query Service ### Query Definition and Execution