Merged in feature/fix_test_timeouts (pull request #194)

text extractions tests * increase timeout for slow ci/cd systems * add tests
2025-12-04 22:45:15 +00:00
parent 3bdc27f4de
commit a94637ab13
14 changed files with 2071 additions and 270 deletions
@@ -359,6 +359,88 @@ Cancels a batch upload that is currently processing.

 ## Folder Operations (FolderService)

+### Folder Path Constraints
+
+All folder paths must comply with the following constraints when creating or renaming folders:
+
+- **Must start with `/`**: All paths must begin with a forward slash
+- **Maximum path length**: 1000 characters
+- **Maximum segment length**: 255 characters per path segment
+- **Maximum depth**: 20 levels of nesting
+- **Valid characters only**: Alphanumeric (`a-z`, `A-Z`, `0-9`), hyphen (`-`), underscore (`_`), period (`.`), and space (` `)
+- **Segment must start with alphanumeric**: Each path segment must begin with a letter or number
+- **No empty segments**: Double slashes (`//`) or trailing slashes are not allowed
+- **No path traversal**: Sequences like `..` or `./` are blocked
+- **No hidden folders**: Segments cannot start with a period (e.g., `/.hidden`)
+- **No leading/trailing whitespace**: Spaces within segment names are allowed, but not at the start or end
+- **No control characters**: ASCII control characters (0x00-0x1F, 0x7F) are blocked
+- **No Windows reserved names**: Names like `CON`, `PRN`, `AUX`, `NUL`, `COM1`-`COM9`, `LPT1`-`LPT9` are blocked
+- **Root folder restrictions**: The root path `/` can only exist once per client and cannot be renamed
+- **Path uniqueness**: Each path must be unique within a client (enforced by database)
+- **Unicode normalization**: Paths are normalized to NFC form
+
+**Valid path examples**:
+- `/documents`
+- `/documents/2024/Q1`
+- `/client-data/reports_2024`
+- `/My Documents/Archive`
+
+**Invalid path examples**:
+- `documents` (missing leading `/`)
+- `/documents/` (trailing slash)
+- `/foo//bar` (empty segment)
+- `/foo/../bar` (path traversal)
+- `/.hidden` (dot prefix)
+- `/foo@bar` (invalid character)
+- `/CON` (Windows reserved name)
+
+---
+
+### `GET /clients/{clientId}/folders`
+Lists all folders for a specific client, including the folder hierarchy.
+
+**Security**: Requires JWT authentication
+
+**Path Parameters**:
+- `clientId`: Client external ID (string, max 36 chars)
+
+**Response** (`200 OK`):
+```json
+{
+  "folders": [
+    {
+      "id": "019580df-ef65-7676-8de9-94435a93337a",
+      "path": "/",
+      "parentId": null,
+      "clientId": "AAA",
+      "createdAt": "2025-10-16T14:27:15Z",
+      "createdBy": "system"
+    },
+    {
+      "id": "019580df-ef65-7676-8de9-94435a93337b",
+      "path": "/documents",
+      "parentId": "019580df-ef65-7676-8de9-94435a93337a",
+      "clientId": "AAA",
+      "createdAt": "2025-10-16T14:30:00Z",
+      "createdBy": "user@example.com"
+    },
+    {
+      "id": "019580df-ef65-7676-8de9-94435a93337c",
+      "path": "/documents/2024",
+      "parentId": "019580df-ef65-7676-8de9-94435a93337b",
+      "clientId": "AAA",
+      "createdAt": "2025-10-16T14:35:00Z",
+      "createdBy": "user@example.com"
+    }
+  ]
+}
+```
+
+**Notes**:
+- Returns all folders for the client, including nested folders
+- Folders are returned in a flat list; use `parentId` to reconstruct the hierarchy
+- The root folder (path="/") is included if it exists
+
 ### `POST /folders`
 Creates a new folder for organizing documents.

@@ -638,6 +720,36 @@ Retrieves all versions of field extractions for a document, ordered by most rece
 }
 ```

+### `GET /field-extractions/version`
+Retrieves a specific version of a field extraction for a document by version number.
+
+**Security**: Requires JWT authentication
+
+**Query Parameters**:
+- `documentId` (required): Document UUID
+- `version` (required): Version number (integer, minimum 1)
+
+**Response** (`200 OK`):
+```json
+{
+  "id": "019580df-ef65-7676-8de9-94435a93337a",
+  "documentId": "019580df-b3f8-7348-9ab1-1e55b3f18ed7",
+  "version": 1,
+  "singleFields": {
+    "fileName": "contract_2024.pdf",
+    "contractTitle": "Service Agreement",
+    "clientName": "ACME Corp"
+  },
+  "arrayFields": [],
+  "createdAt": "2025-10-16T14:27:15Z",
+  "createdBy": "user@example.com"
+}
+```
+
+**Error Responses**:
+- `400`: Missing required query parameters
+- `404`: Version not found for the specified document
+
 ---

 ## Query Management (QueryService)
@@ -1061,6 +1173,174 @@ All API responses include:

 ---

+## Authorization Reference (Permit.io Integration)
+
+The API uses Permit.io for fine-grained authorization. This section documents what data is sent to Permit.io for each endpoint, enabling system administrators to configure the correct resources, actions, and roles in Permit.io.
+
+### Authorization Flow
+
+1. **Authentication**: User authenticates via AWS Cognito, receiving a JWT token
+2. **User Identification**: The `sub` (subject) claim from the JWT is extracted as the user identifier
+3. **Resource Mapping**: The first path segment of the URL becomes the resource (e.g., `/client/{id}/document` → `client`)
+4. **Action Mapping**: The HTTP method is converted to lowercase to become the action (e.g., `GET` → `get`)
+5. **Permission Check**: Permit.io checks if the user has permission to perform the action on the resource
+
+### Permit.io Data Mapping Logic
+
+**Resource Extraction** (from `internal/cognitoauth/mapping.go`):
+- The resource is derived from the **first segment** of the URL path
+- Leading slashes are removed
+- Example: `/client/{id}/document/batch/{batch_id}` → resource: `client`
+
+**Action Extraction**:
+- HTTP method is converted to lowercase
+- `GET` → `get`, `POST` → `post`, `PATCH` → `patch`, `DELETE` → `delete`, `HEAD` → `head`
+
+### Endpoints Exempt from Authorization
+
+The following endpoints skip authentication and Permit.io authorization entirely:
+
+| Endpoint | Purpose |
+|----------|---------|
+| `GET /home` | Public home/menu page |
+| `GET /logout` | Session termination |
+| `GET /health` | Health check endpoint |
+| `GET /swagger/*` | Swagger UI documentation |
+| `GET /metrics` | Prometheus metrics |
+
+### Special Authentication Endpoints
+
+These endpoints handle authentication flows and bypass normal authorization:
+
+| Endpoint | Purpose |
+|----------|---------|
+| `GET /login` | Initiates OAuth2/PKCE login flow with Cognito |
+| `GET /login-callback` | Handles OAuth2 callback from Cognito |
+
+### Authorization Reference Table
+
+The following table shows what data is passed to Permit.io for each protected endpoint:
+
+| HTTP Method | Endpoint | Resource | Action | Notes |
+|-------------|----------|----------|--------|-------|
+| **Client Management** |
+| `POST` | `/client` | `client` | `post` | Create new client |
+| `GET` | `/clients` | `clients` | `get` | List all clients |
+| `GET` | `/client/{id}` | `client` | `get` | Get client details |
+| `PATCH` | `/client/{id}` | `client` | `patch` | Update client |
+| `GET` | `/client/{id}/status` | `client` | `get` | Get client sync status |
+| **Document Operations** |
+| `POST` | `/client/{id}/document` | `client` | `post` | Upload document |
+| `GET` | `/client/{id}/document` | `client` | `get` | List client documents |
+| `GET` | `/document/{id}` | `document` | `get` | Get document details |
+| **Batch Document Operations** |
+| `POST` | `/client/{id}/document/batch` | `client` | `post` | Upload batch (ZIP) |
+| `GET` | `/client/{id}/document/batch` | `client` | `get` | List batch uploads |
+| `GET` | `/client/{id}/document/batch/{batch_id}` | `client` | `get` | Get batch status |
+| `DELETE` | `/client/{id}/document/batch/{batch_id}` | `client` | `delete` | Cancel batch upload |
+| **Folder Operations** |
+| `GET` | `/client/{id}/folders` | `client` | `get` | List client folders |
+| `POST` | `/folders` | `folders` | `post` | Create folder |
+| `PATCH` | `/folders/{folderId}` | `folders` | `patch` | Rename folder |
+| `GET` | `/folders/{folderId}/documents` | `folders` | `get` | List folder documents |
+| `GET` | `/folders/{folderId}/metrics` | `folders` | `get` | Get folder metrics |
+| **Label Operations** |
+| `POST` | `/documents/{documentId}/labels` | `documents` | `post` | Apply label to document |
+| `GET` | `/documents/{documentId}/labels` | `documents` | `get` | Get document labels |
+| `GET` | `/labels/{labelName}/documents` | `labels` | `get` | Get documents by label |
+| **Field Extraction Operations** |
+| `POST` | `/field-extractions` | `field-extractions` | `post` | Create field extraction |
+| `GET` | `/field-extractions` | `field-extractions` | `get` | Get latest extraction |
+| `GET` | `/field-extractions/version` | `field-extractions` | `get` | Get specific version |
+| `GET` | `/field-extractions/history` | `field-extractions` | `get` | Get extraction history |
+| **Query Management** |
+| `GET` | `/query` | `query` | `get` | List queries |
+| `POST` | `/query` | `query` | `post` | Create query |
+| `GET` | `/query/{id}` | `query` | `get` | Get query details |
+| `PATCH` | `/query/{id}` | `query` | `patch` | Update query |
+| `POST` | `/query/{id}/test` | `query` | `post` | Test query execution |
+| **Collector Configuration** |
+| `GET` | `/client/{id}/collector` | `client` | `get` | Get collector config |
+| `PATCH` | `/client/{id}/collector` | `client` | `patch` | Update collector config |
+| **Export Operations** |
+| `POST` | `/client/{id}/export` | `client` | `post` | Trigger export |
+| `GET` | `/export/{id}` | `export` | `get` | Get export status |
+| **User Administration** |
+| `POST` | `/admin/users` | `admin` | `post` | Create user |
+| `GET` | `/admin/users` | `admin` | `get` | List users |
+| `GET` | `/admin/users/{email}` | `admin` | `get` | Get user details |
+| `HEAD` | `/admin/users/{email}` | `admin` | `head` | Check user existence |
+| `PATCH` | `/admin/users/{email}` | `admin` | `patch` | Update user |
+| `DELETE` | `/admin/users/{email}` | `admin` | `delete` | Delete user |
+| `POST` | `/admin/users/{email}/disable` | `admin` | `post` | Disable user |
+| `POST` | `/admin/users/{email}/enable` | `admin` | `post` | Enable user |
+
+### Permit.io Configuration Requirements
+
+To configure Permit.io to work with this API, administrators must create:
+
+#### Resources
+
+| Resource Key | Description |
+|--------------|-------------|
+| `client` | Client management and client-scoped operations |
+| `clients` | Client listing (plural form) |
+| `document` | Individual document operations |
+| `documents` | Document collection and label operations |
+| `folders` | Folder management |
+| `labels` | Label-based document queries |
+| `field-extractions` | Field extraction operations |
+| `query` | Query definition management |
+| `export` | Export operations |
+| `admin` | User administration |
+
+#### Actions
+
+| Action Key | HTTP Method | Description |
+|------------|-------------|-------------|
+| `get` | GET | Read/retrieve resources |
+| `post` | POST | Create resources |
+| `patch` | PATCH | Update resources |
+| `delete` | DELETE | Delete resources |
+| `head` | HEAD | Check resource existence |
+
+#### Example Role Configuration
+
+Typical role assignments for common personas:
+
+| Role | Resources | Actions | Use Case |
+|------|-----------|---------|----------|
+| `viewer` | `client`, `clients`, `document`, `documents`, `folders`, `query`, `export` | `get` | Read-only access |
+| `editor` | `client`, `document`, `documents`, `folders`, `field-extractions`, `query` | `get`, `post`, `patch` | Content management |
+| `admin` | All resources | All actions | Full administrative access |
+| `user_admin` | `admin` | All actions | User management only |
+
+### Authorization Error Responses
+
+When authorization fails, the API returns:
+
+**HTTP 401 Unauthorized** - No valid JWT token:
+```json
+{
+  "error": "Authorization required",
+  "message": "This endpoint requires authentication. Please obtain a valid JWT token.",
+  "login_url": "/login"
+}
+```
+
+**HTTP 403 Forbidden** - Insufficient permissions:
+```json
+{
+  "error": "Insufficient permissions",
+  "message": "Access denied by authorization service",
+  "user": "<cognito-subject-id>",
+  "resource": "<resource-name>",
+  "action": "<action-name>"
+}
+```
+
+---
+
 ## Code Generation

 ### OpenAPI Integration