Merged in feature/setup_docs (pull request #199)
docs for new env setup * docs for new env setup * lint fix * docs
This commit is contained in:
Jay Brown
File diff suppressed because it is too large
Load Diff
@@ -420,4 +420,141 @@ AWS_ENDPOINT_URL=http://test-localstack:4566
|
||||
- **Override-friendly**: Allow easy configuration overrides for tests
|
||||
- **Isolation**: Each test should have independent configuration
|
||||
- **Realistic defaults**: Test configuration should mirror production patterns
|
||||
- **Environment separation**: Clear separation between dev/test/prod configs
|
||||
- **Environment separation**: Clear separation between dev/test/prod configs
|
||||
|
||||
## Setting Up Authentication for a New Environment
|
||||
|
||||
This section covers the initial setup of authentication infrastructure when deploying to a new environment (e.g., dev, UAT, production). This includes configuring Permit.io for authorization and creating the initial bootstrap users in both AWS Cognito and Permit.io.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
Before setting up authentication for a new environment, ensure you have:
|
||||
|
||||
1. **AWS Cognito User Pool** already created for the environment
|
||||
2. **AWS credentials** configured with permissions to create Cognito users
|
||||
3. **Access to Permit.io** organization account
|
||||
|
||||
### Step 1: Create Permit.io Environment and Get API Key
|
||||
|
||||
1. Log in to the [Permit.io Dashboard](https://app.permit.io)
|
||||
2. Navigate to your project (or create a new one for the environment)
|
||||
3. Create a new **Environment** for your deployment (e.g., `Development`, `UAT`, `Production`)
|
||||
4. Go to **Settings → API Keys** and create a new API key scoped to this environment
|
||||
5. Copy the API key - you will need it for the following steps
|
||||
|
||||
**Important**: Each environment (dev, UAT, prod) requires its own separate Permit.io environment and API key.
|
||||
|
||||
### Step 2: Clean Up Default Permit.io Roles
|
||||
|
||||
When you create a new environment in Permit.io, it automatically creates default `admin`, `viewer`, and `editor` roles. These must be deleted before running the setup tool because they conflict with the role names used by this system.
|
||||
|
||||
1. In the Permit.io Dashboard, navigate to **Policy → Roles**
|
||||
2. Delete the default `admin`, `viewer`, and `editor` roles
|
||||
3. Verify no roles remain before proceeding
|
||||
|
||||
### Step 3: Run the Permit.io Setup Tool
|
||||
|
||||
The permit setup tool configures all required resources and roles in Permit.io based on the `permit_policies.yaml` configuration file.
|
||||
|
||||
```bash
|
||||
cd cmd/cognito_test/permit.setup
|
||||
|
||||
# Set the API key for your target environment
|
||||
export PERMIT_API_KEY="permit_key_your_environment_key_here"
|
||||
|
||||
# Verify connection (list current configuration)
|
||||
go run setup_permit.go -list
|
||||
|
||||
# Dry run to preview changes
|
||||
go run setup_permit.go -config permit_policies.yaml -project default -env <your-env> -dry-run
|
||||
|
||||
# Apply the configuration
|
||||
go run setup_permit.go -config permit_policies.yaml -project default -env <your-env>
|
||||
```
|
||||
|
||||
This creates:
|
||||
- All required **resources** (admin, client, document, folders, etc.) with their actions
|
||||
- All required **roles** (super_admin, user_admin, auditor, editor, viewer, etc.) with appropriate permissions
|
||||
|
||||
See `cmd/cognito_test/permit.setup/README.md` for detailed documentation.
|
||||
|
||||
### Step 4: Create Bootstrap Users
|
||||
|
||||
After Permit.io is configured, use the user creation tool to create the initial users in both AWS Cognito and Permit.io. The tool ensures that users are created with matching Subject IDs in both systems, which is critical for authentication to work correctly.
|
||||
|
||||
**Important**: Always use the user creation tool to create users. Do not manually create users in Cognito and Permit.io separately, as this will result in mismatched Subject IDs and authentication failures.
|
||||
|
||||
```bash
|
||||
cd cmd/cognito_test/user.creation.tool
|
||||
|
||||
# Configure environment variables
|
||||
export PERMIT_KEY="permit_key_your_environment_key_here"
|
||||
export COGNITO_USER_POOL_ID="us-east-1_xxxxxxxxx"
|
||||
export AWS_REGION="us-east-1"
|
||||
# Either use AWS credentials or an SSO profile
|
||||
export AWS_PROFILE="your-sso-profile" # if using SSO
|
||||
|
||||
# Create a CSV file with your bootstrap users
|
||||
# Format: email,first_name,last_name,role1,role2,...
|
||||
cat > bootstrap_users.csv << 'EOF'
|
||||
email,first_name,last_name,super_admin,user_admin
|
||||
admin@yourcompany.com,Admin,User,super_admin,
|
||||
useradmin@yourcompany.com,User,Admin,,user_admin
|
||||
EOF
|
||||
|
||||
# Dry run to preview
|
||||
go run main.go -project use-key -csv bootstrap_users.csv --dry-run
|
||||
|
||||
# Create the users
|
||||
go run main.go -project use-key -csv bootstrap_users.csv
|
||||
```
|
||||
|
||||
The tool will:
|
||||
1. Create each user in AWS Cognito (with email as username)
|
||||
2. Create corresponding user in Permit.io using the Cognito Subject ID as the key
|
||||
3. Assign the specified roles in Permit.io
|
||||
|
||||
See `cmd/cognito_test/user.creation.tool/readme.md` for detailed documentation including:
|
||||
- Full CSV format specification
|
||||
- Delete, disable, and enable operations
|
||||
- Audit logging and compliance features
|
||||
- Troubleshooting guide
|
||||
|
||||
### Step 5: Verify Setup
|
||||
|
||||
After creating bootstrap users:
|
||||
|
||||
1. **Test Cognito login**: Verify users can authenticate through the Cognito hosted UI or your application
|
||||
2. **Test API access**: Make an authenticated request to verify authorization works:
|
||||
```bash
|
||||
# Get a token (via Cognito login flow)
|
||||
# Then test the identity endpoint
|
||||
curl -H "Authorization: Bearer <access_token>" https://your-api/identity
|
||||
```
|
||||
3. **Check Permit.io Dashboard**: Verify users appear with correct role assignments
|
||||
|
||||
### Environment-Specific Scripts
|
||||
|
||||
For convenience, you can create environment-specific run scripts:
|
||||
|
||||
```bash
|
||||
# Example: run.tool.prod.sh
|
||||
#!/bin/bash
|
||||
export PERMIT_KEY="permit_key_production_key_here"
|
||||
export COGNITO_USER_POOL_ID="us-east-1_prodPoolId"
|
||||
export AWS_REGION="us-east-1"
|
||||
export AWS_PROFILE="production-profile"
|
||||
|
||||
go run main.go -project use-key -csv "$@"
|
||||
```
|
||||
|
||||
### Quick Reference: New Environment Checklist
|
||||
|
||||
| Step | Action | Tool/Location |
|
||||
|------|--------|---------------|
|
||||
| 1 | Create Permit.io environment | Permit.io Dashboard |
|
||||
| 2 | Get environment API key | Permit.io Dashboard → Settings → API Keys |
|
||||
| 3 | Delete default roles | Permit.io Dashboard → Policy → Roles |
|
||||
| 4 | Run permit setup | `cmd/cognito_test/permit.setup/` |
|
||||
| 5 | Create bootstrap users | `cmd/cognito_test/user.creation.tool/` |
|
||||
| 6 | Verify authentication | Test login and API access |
|
||||
@@ -0,0 +1,945 @@
|
||||
# UI Client Authentication Guide
|
||||
|
||||
This document provides technical instructions for frontend (UI) developers integrating with the Query Orchestration API. It covers how to authenticate requests, discover user permissions, handle errors, and configure CORS for cross-origin deployments.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Authentication Overview](#authentication-overview)
|
||||
2. [Sending Authenticated Requests](#sending-authenticated-requests)
|
||||
3. [Discovering User Permissions](#discovering-user-permissions)
|
||||
4. [Error Handling](#error-handling)
|
||||
5. [Token Refresh Strategy](#token-refresh-strategy)
|
||||
6. [CORS Configuration](#cors-configuration)
|
||||
7. [Security Best Practices](#security-best-practices)
|
||||
8. [Quick Reference](#quick-reference)
|
||||
|
||||
---
|
||||
|
||||
## Authentication Overview
|
||||
|
||||
The Query Orchestration API uses **JWT (JSON Web Token) authentication** with tokens issued by **AWS Cognito**. The authentication flow assumes that the UI application handles user login directly with Cognito and obtains the necessary tokens.
|
||||
|
||||
### Token Types
|
||||
|
||||
When a user authenticates with Cognito, your UI receives three tokens:
|
||||
|
||||
| Token | Purpose | Typical Lifetime |
|
||||
|-------|---------|------------------|
|
||||
| `access_token` | **Used for API requests** - Contains user identity and scopes | 1 hour |
|
||||
| `id_token` | Contains user profile claims (email, name, etc.) | 1 hour |
|
||||
| `refresh_token` | Used to obtain new access/id tokens without re-authentication | 30 days |
|
||||
|
||||
**Important**: The Query Orchestration API validates the `access_token`. Always use this token (not the `id_token`) in the `Authorization` header.
|
||||
|
||||
### Authentication Flow
|
||||
|
||||
```
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ ┌─────────────┐
|
||||
│ UI App │ │ Cognito │ │ Query Orchestration│ │ Permit.io │
|
||||
│ (Frontend) │ │ (IDP) │ │ API │ │ (AuthZ) │
|
||||
└──────┬──────┘ └──────┬──────┘ └──────────┬──────────┘ └──────┬──────┘
|
||||
│ │ │ │
|
||||
│ 1. Login request │ │ │
|
||||
│───────────────────>│ │ │
|
||||
│ │ │ │
|
||||
│ 2. Tokens │ │ │
|
||||
│<───────────────────│ │ │
|
||||
│ │ │ │
|
||||
│ 3. API Request + Bearer Token │ │
|
||||
│────────────────────────────────────────────>│ │
|
||||
│ │ │ │
|
||||
│ │ │ 4. Check Permission │
|
||||
│ │ │───────────────────────>│
|
||||
│ │ │ │
|
||||
│ │ │ 5. Allowed/Denied │
|
||||
│ │ │<───────────────────────│
|
||||
│ │ │ │
|
||||
│ 6. API Response │ │
|
||||
│<────────────────────────────────────────────│ │
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sending Authenticated Requests
|
||||
|
||||
### Authorization Header Format
|
||||
|
||||
All authenticated API requests must include the JWT access token in the `Authorization` header using the Bearer scheme:
|
||||
|
||||
```
|
||||
Authorization: Bearer <access_token>
|
||||
```
|
||||
|
||||
### Accepted Token Locations
|
||||
|
||||
The API accepts tokens from exactly two locations:
|
||||
|
||||
| Location | Format | Notes |
|
||||
|----------|--------|-------|
|
||||
| `Authorization` header | `Bearer <token>` | **Recommended for cross-origin requests** |
|
||||
| `auth_token` cookie | JWT string | Used by cookie-based auth flow |
|
||||
|
||||
**Not supported**:
|
||||
- Query parameters (e.g., `?token=...`)
|
||||
- Other header names
|
||||
- Request body
|
||||
|
||||
### Cookie-Based Authentication (Alternative)
|
||||
|
||||
If your deployment uses cookie-based authentication, the server uses these HTTP-only cookies:
|
||||
|
||||
| Cookie Name | Purpose |
|
||||
|-------------|---------|
|
||||
| `auth_token` | Access token (JWT) |
|
||||
| `refresh_token` | Refresh token for obtaining new access tokens |
|
||||
|
||||
When cookies are present, the server middleware automatically copies the `auth_token` cookie value to the `Authorization` header internally. The server can also automatically refresh expired tokens using the `refresh_token` cookie.
|
||||
|
||||
**Note**: Cookie-based auth requires CORS configuration for cross-origin deployments. See [CORS Configuration](#cors-configuration).
|
||||
|
||||
### No CSRF Token Required
|
||||
|
||||
The API does not require CSRF tokens. JWT bearer token authentication provides protection against CSRF attacks because:
|
||||
- The token must be explicitly included in the `Authorization` header
|
||||
- Browsers do not automatically attach custom headers to cross-origin requests
|
||||
|
||||
### JavaScript/TypeScript Examples
|
||||
|
||||
#### Using Fetch API
|
||||
|
||||
```javascript
|
||||
const API_BASE_URL = 'https://api.yourdomain.com';
|
||||
|
||||
async function makeAuthenticatedRequest(endpoint, options = {}) {
|
||||
const accessToken = getAccessToken(); // Retrieve from your token storage
|
||||
|
||||
const response = await fetch(`${API_BASE_URL}${endpoint}`, {
|
||||
...options,
|
||||
headers: {
|
||||
'Authorization': `Bearer ${accessToken}`,
|
||||
'Content-Type': 'application/json',
|
||||
...options.headers,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
await handleErrorResponse(response);
|
||||
}
|
||||
|
||||
return response.json();
|
||||
}
|
||||
|
||||
// Example: Get client details
|
||||
async function getClient(clientId) {
|
||||
return makeAuthenticatedRequest(`/client/${clientId}`);
|
||||
}
|
||||
|
||||
// Example: Upload a document
|
||||
async function uploadDocument(clientId, file, folderId = null) {
|
||||
const accessToken = getAccessToken();
|
||||
const formData = new FormData();
|
||||
formData.append('file', file);
|
||||
if (folderId) {
|
||||
formData.append('folderId', folderId);
|
||||
}
|
||||
|
||||
const response = await fetch(`${API_BASE_URL}/client/${clientId}/document`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${accessToken}`,
|
||||
// Note: Do NOT set Content-Type for FormData - browser sets it automatically
|
||||
},
|
||||
body: formData,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
await handleErrorResponse(response);
|
||||
}
|
||||
|
||||
return response.json();
|
||||
}
|
||||
```
|
||||
|
||||
#### Using Axios
|
||||
|
||||
```javascript
|
||||
import axios from 'axios';
|
||||
|
||||
const apiClient = axios.create({
|
||||
baseURL: 'https://api.yourdomain.com',
|
||||
});
|
||||
|
||||
// Add auth token to every request
|
||||
apiClient.interceptors.request.use((config) => {
|
||||
const accessToken = getAccessToken();
|
||||
if (accessToken) {
|
||||
config.headers.Authorization = `Bearer ${accessToken}`;
|
||||
}
|
||||
return config;
|
||||
});
|
||||
|
||||
// Handle 401 errors globally
|
||||
apiClient.interceptors.response.use(
|
||||
(response) => response,
|
||||
async (error) => {
|
||||
if (error.response?.status === 401) {
|
||||
// Token expired or invalid - attempt refresh or redirect to login
|
||||
const refreshed = await attemptTokenRefresh();
|
||||
if (refreshed) {
|
||||
// Retry the original request with new token
|
||||
error.config.headers.Authorization = `Bearer ${getAccessToken()}`;
|
||||
return apiClient.request(error.config);
|
||||
}
|
||||
// Refresh failed - redirect to login
|
||||
redirectToLogin();
|
||||
}
|
||||
return Promise.reject(error);
|
||||
}
|
||||
);
|
||||
|
||||
// Example usage
|
||||
const client = await apiClient.get('/client/AAA');
|
||||
const documents = await apiClient.get('/client/AAA/document');
|
||||
```
|
||||
|
||||
#### Using React Query
|
||||
|
||||
```typescript
|
||||
import { useQuery, useMutation, QueryClient } from '@tanstack/react-query';
|
||||
|
||||
const queryClient = new QueryClient();
|
||||
|
||||
// Fetch wrapper with auth
|
||||
async function authFetch<T>(endpoint: string, options?: RequestInit): Promise<T> {
|
||||
const accessToken = getAccessToken();
|
||||
|
||||
const response = await fetch(`${API_BASE_URL}${endpoint}`, {
|
||||
...options,
|
||||
headers: {
|
||||
'Authorization': `Bearer ${accessToken}`,
|
||||
'Content-Type': 'application/json',
|
||||
...options?.headers,
|
||||
},
|
||||
});
|
||||
|
||||
if (response.status === 401) {
|
||||
// Trigger re-authentication
|
||||
throw new AuthenticationError('Token expired');
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
throw new ApiError(response.status, await response.json());
|
||||
}
|
||||
|
||||
return response.json();
|
||||
}
|
||||
|
||||
// Hook for fetching client data
|
||||
function useClient(clientId: string) {
|
||||
return useQuery({
|
||||
queryKey: ['client', clientId],
|
||||
queryFn: () => authFetch<Client>(`/client/${clientId}`),
|
||||
retry: (failureCount, error) => {
|
||||
// Don't retry on auth errors
|
||||
if (error instanceof AuthenticationError) return false;
|
||||
return failureCount < 3;
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// Hook for fetching user identity/permissions
|
||||
function useIdentity() {
|
||||
return useQuery({
|
||||
queryKey: ['identity'],
|
||||
queryFn: () => authFetch<IdentityResponse>('/identity'),
|
||||
staleTime: 5 * 60 * 1000, // Cache for 5 minutes
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Discovering User Permissions
|
||||
|
||||
### The `/identity` Endpoint
|
||||
|
||||
After authentication, use the `/identity` endpoint to discover what the current user is authorized to do. This endpoint returns the user's roles and associated permissions from Permit.io.
|
||||
|
||||
**Endpoint**: `GET /identity`
|
||||
|
||||
**Authentication**: Required (Bearer token)
|
||||
|
||||
**Response** (200 OK):
|
||||
|
||||
```json
|
||||
{
|
||||
"roles": [
|
||||
{
|
||||
"key": "auditor",
|
||||
"name": "Auditor",
|
||||
"description": "Read-only access to view documents and exports",
|
||||
"permissions": [
|
||||
"client:get",
|
||||
"document:get",
|
||||
"folders:get",
|
||||
"export:get"
|
||||
]
|
||||
},
|
||||
{
|
||||
"key": "editor",
|
||||
"name": "Editor",
|
||||
"description": "Can create and modify documents",
|
||||
"permissions": [
|
||||
"client:get",
|
||||
"client:post",
|
||||
"document:get",
|
||||
"document:post",
|
||||
"folders:get",
|
||||
"folders:post",
|
||||
"folders:patch"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Permission Format
|
||||
|
||||
Permissions follow the format `resource:action`:
|
||||
|
||||
| Resource | Actions | Description |
|
||||
|----------|---------|-------------|
|
||||
| `client` | `get`, `post`, `patch`, `delete` | Client management |
|
||||
| `clients` | `get` | List all clients |
|
||||
| `document` | `get`, `post`, `patch`, `delete` | Document operations |
|
||||
| `folders` | `get`, `post`, `patch`, `delete` | Folder management |
|
||||
| `query` | `get`, `post`, `patch` | Query definitions |
|
||||
| `export` | `get`, `post` | Export operations |
|
||||
| `admin` | `get`, `post`, `patch`, `delete` | User administration |
|
||||
|
||||
### Using Permissions in UI
|
||||
|
||||
```typescript
|
||||
interface IdentityRole {
|
||||
key: string;
|
||||
name: string;
|
||||
description?: string;
|
||||
permissions: string[];
|
||||
}
|
||||
|
||||
interface IdentityResponse {
|
||||
roles: IdentityRole[];
|
||||
}
|
||||
|
||||
// Fetch and cache user permissions
|
||||
let cachedPermissions: Set<string> | null = null;
|
||||
|
||||
async function loadUserPermissions(): Promise<Set<string>> {
|
||||
if (cachedPermissions) {
|
||||
return cachedPermissions;
|
||||
}
|
||||
|
||||
const response = await authFetch<IdentityResponse>('/identity');
|
||||
|
||||
// Flatten all permissions from all roles
|
||||
const permissions = new Set<string>();
|
||||
for (const role of response.roles) {
|
||||
for (const permission of role.permissions) {
|
||||
permissions.add(permission);
|
||||
}
|
||||
}
|
||||
|
||||
cachedPermissions = permissions;
|
||||
return permissions;
|
||||
}
|
||||
|
||||
// Check if user has a specific permission
|
||||
async function hasPermission(resource: string, action: string): Promise<boolean> {
|
||||
const permissions = await loadUserPermissions();
|
||||
return permissions.has(`${resource}:${action}`);
|
||||
}
|
||||
|
||||
// Check if user can perform an action
|
||||
async function canUploadDocuments(): Promise<boolean> {
|
||||
return hasPermission('client', 'post'); // Upload uses POST /client/{id}/document
|
||||
}
|
||||
|
||||
async function canManageUsers(): Promise<boolean> {
|
||||
return hasPermission('admin', 'post');
|
||||
}
|
||||
|
||||
// React hook example
|
||||
function usePermission(resource: string, action: string) {
|
||||
const { data: identity, isLoading } = useIdentity();
|
||||
|
||||
const hasPermission = useMemo(() => {
|
||||
if (!identity) return false;
|
||||
const permission = `${resource}:${action}`;
|
||||
return identity.roles.some(role =>
|
||||
role.permissions.includes(permission)
|
||||
);
|
||||
}, [identity, resource, action]);
|
||||
|
||||
return { hasPermission, isLoading };
|
||||
}
|
||||
|
||||
// Usage in component
|
||||
function UploadButton() {
|
||||
const { hasPermission, isLoading } = usePermission('client', 'post');
|
||||
|
||||
if (isLoading) return <Spinner />;
|
||||
if (!hasPermission) return null; // Hide button if not authorized
|
||||
|
||||
return <Button onClick={handleUpload}>Upload Document</Button>;
|
||||
}
|
||||
```
|
||||
|
||||
### Role-Based UI Rendering
|
||||
|
||||
```typescript
|
||||
// Define UI feature permissions
|
||||
const UI_FEATURES = {
|
||||
viewDocuments: ['client:get', 'document:get'],
|
||||
uploadDocuments: ['client:post'],
|
||||
manageClients: ['client:post', 'client:patch'],
|
||||
viewExports: ['export:get'],
|
||||
createExports: ['export:post'],
|
||||
adminPanel: ['admin:get'],
|
||||
} as const;
|
||||
|
||||
function canAccessFeature(
|
||||
userPermissions: Set<string>,
|
||||
feature: keyof typeof UI_FEATURES
|
||||
): boolean {
|
||||
const requiredPermissions = UI_FEATURES[feature];
|
||||
return requiredPermissions.every(p => userPermissions.has(p));
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### HTTP Status Codes
|
||||
|
||||
| Status | Meaning | UI Action |
|
||||
|--------|---------|-----------|
|
||||
| `200` | Success | Process response |
|
||||
| `201` | Created | Resource created successfully |
|
||||
| `204` | No Content | Operation successful (no body) |
|
||||
| `400` | Bad Request | Show validation error to user |
|
||||
| `401` | Unauthorized | Token invalid/expired - refresh or re-login |
|
||||
| `403` | Forbidden | User lacks permission - show access denied |
|
||||
| `404` | Not Found | Resource doesn't exist |
|
||||
| `429` | Too Many Requests | Rate limited - implement backoff |
|
||||
| `500` | Server Error | Show generic error, retry later |
|
||||
| `502` | Bad Gateway | Authorization service unavailable |
|
||||
|
||||
### Error Response Format
|
||||
|
||||
Error responses typically include a `message` field, but the structure varies by endpoint and error type:
|
||||
|
||||
```json
|
||||
// Standard error format
|
||||
{
|
||||
"message": "Description of what went wrong"
|
||||
}
|
||||
|
||||
// Authentication error format (401)
|
||||
{
|
||||
"error": "Authorization required",
|
||||
"message": "This endpoint requires authentication. Please obtain a valid JWT token.",
|
||||
"login_url": "/login"
|
||||
}
|
||||
|
||||
// Authorization error format (403)
|
||||
{
|
||||
"error": "Access denied"
|
||||
}
|
||||
```
|
||||
|
||||
**Recommended error handling**: Check for both `error` and `message` fields when parsing error responses.
|
||||
|
||||
### Handling Authentication Errors (401)
|
||||
|
||||
```typescript
|
||||
// Helper to extract error message from varying response formats
|
||||
function getErrorMessage(body: any, fallback: string): string {
|
||||
return body.message || body.error || fallback;
|
||||
}
|
||||
|
||||
async function handleErrorResponse(response: Response) {
|
||||
const body = await response.json().catch(() => ({}));
|
||||
|
||||
switch (response.status) {
|
||||
case 401:
|
||||
// Token expired or invalid
|
||||
console.log('Authentication required:', getErrorMessage(body, 'Authentication required'));
|
||||
|
||||
// Attempt to refresh the token (only if using header-based auth)
|
||||
const refreshed = await attemptTokenRefresh();
|
||||
if (!refreshed) {
|
||||
// Clear stored tokens
|
||||
clearTokens();
|
||||
// Redirect to login
|
||||
redirectToLogin();
|
||||
}
|
||||
throw new AuthenticationError(getErrorMessage(body, 'Authentication required'));
|
||||
|
||||
case 403:
|
||||
// User authenticated but not authorized
|
||||
console.log('Access denied:', getErrorMessage(body, 'Access denied'));
|
||||
throw new AuthorizationError(getErrorMessage(body, 'Access denied'));
|
||||
|
||||
case 429:
|
||||
// Rate limited - check RateLimit header
|
||||
const rateLimitInfo = response.headers.get('RateLimit');
|
||||
console.log('Rate limited:', rateLimitInfo);
|
||||
throw new RateLimitError(getErrorMessage(body, 'Rate limit exceeded'), rateLimitInfo);
|
||||
|
||||
default:
|
||||
throw new ApiError(response.status, getErrorMessage(body, 'Request failed'));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
The API includes rate limiting. Monitor the `RateLimit` response header:
|
||||
|
||||
```
|
||||
RateLimit: 100;window=60
|
||||
```
|
||||
|
||||
This indicates 100 requests allowed per 60-second window.
|
||||
|
||||
```typescript
|
||||
// Implement exponential backoff for rate limits
|
||||
async function fetchWithRetry(url: string, options: RequestInit, maxRetries = 3) {
|
||||
for (let attempt = 0; attempt < maxRetries; attempt++) {
|
||||
const response = await fetch(url, options);
|
||||
|
||||
if (response.status === 429) {
|
||||
const retryAfter = response.headers.get('Retry-After');
|
||||
const waitTime = retryAfter
|
||||
? parseInt(retryAfter) * 1000
|
||||
: Math.pow(2, attempt) * 1000;
|
||||
|
||||
console.log(`Rate limited. Retrying in ${waitTime}ms`);
|
||||
await sleep(waitTime);
|
||||
continue;
|
||||
}
|
||||
|
||||
return response;
|
||||
}
|
||||
|
||||
throw new Error('Max retries exceeded');
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Token Refresh Strategy
|
||||
|
||||
### Server-Side vs Client-Side Refresh
|
||||
|
||||
The refresh strategy depends on how your UI sends authentication:
|
||||
|
||||
| Authentication Method | Who Handles Refresh | Notes |
|
||||
|-----------------------|---------------------|-------|
|
||||
| `Authorization` header | **UI must refresh** | Server has no access to refresh token |
|
||||
| Cookie-based | **Server auto-refreshes** | Server reads `refresh_token` cookie |
|
||||
|
||||
**If using `Authorization: Bearer` header** (recommended for cross-origin):
|
||||
- The UI must store and manage the refresh token
|
||||
- The UI must call Cognito directly to refresh tokens
|
||||
- The server cannot refresh for you (it doesn't have your refresh token)
|
||||
|
||||
**If using cookie-based authentication**:
|
||||
- The server automatically refreshes expired tokens
|
||||
- New tokens are returned as updated cookies
|
||||
- No action required from the UI
|
||||
|
||||
### When to Refresh (Header-Based Auth)
|
||||
|
||||
The access token has a limited lifetime (typically 1 hour). Implement proactive refresh to avoid authentication errors:
|
||||
|
||||
1. **Proactive refresh**: Check token expiration before each request
|
||||
2. **Reactive refresh**: Refresh when receiving a 401 response
|
||||
|
||||
### Checking Token Expiration
|
||||
|
||||
JWT tokens contain an `exp` claim with the expiration timestamp:
|
||||
|
||||
```typescript
|
||||
function isTokenExpired(token: string, bufferSeconds = 60): boolean {
|
||||
try {
|
||||
// Decode JWT payload (base64)
|
||||
const payload = JSON.parse(atob(token.split('.')[1]));
|
||||
const expiresAt = payload.exp * 1000; // Convert to milliseconds
|
||||
const now = Date.now();
|
||||
|
||||
// Consider expired if within buffer period
|
||||
return now >= (expiresAt - bufferSeconds * 1000);
|
||||
} catch {
|
||||
return true; // Treat decode errors as expired
|
||||
}
|
||||
}
|
||||
|
||||
function getAccessToken(): string | null {
|
||||
const token = localStorage.getItem('access_token');
|
||||
|
||||
if (!token || isTokenExpired(token)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return token;
|
||||
}
|
||||
```
|
||||
|
||||
### Refreshing Tokens with Cognito
|
||||
|
||||
Use the Cognito SDK or direct API call to refresh tokens:
|
||||
|
||||
```typescript
|
||||
import {
|
||||
CognitoIdentityProviderClient,
|
||||
InitiateAuthCommand
|
||||
} from '@aws-sdk/client-cognito-identity-provider';
|
||||
|
||||
const cognitoClient = new CognitoIdentityProviderClient({
|
||||
region: 'us-east-2', // Your Cognito region
|
||||
});
|
||||
|
||||
async function refreshTokens(): Promise<boolean> {
|
||||
const refreshToken = localStorage.getItem('refresh_token');
|
||||
|
||||
if (!refreshToken) {
|
||||
return false;
|
||||
}
|
||||
|
||||
try {
|
||||
const command = new InitiateAuthCommand({
|
||||
AuthFlow: 'REFRESH_TOKEN_AUTH',
|
||||
ClientId: 'your-cognito-client-id',
|
||||
AuthParameters: {
|
||||
REFRESH_TOKEN: refreshToken,
|
||||
},
|
||||
});
|
||||
|
||||
const response = await cognitoClient.send(command);
|
||||
|
||||
if (response.AuthenticationResult) {
|
||||
// Store new tokens
|
||||
localStorage.setItem('access_token', response.AuthenticationResult.AccessToken!);
|
||||
localStorage.setItem('id_token', response.AuthenticationResult.IdToken!);
|
||||
// Note: Refresh token may not be returned - keep existing one
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
} catch (error) {
|
||||
console.error('Token refresh failed:', error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Request Interceptor with Auto-Refresh
|
||||
|
||||
```typescript
|
||||
async function authFetchWithRefresh<T>(
|
||||
endpoint: string,
|
||||
options?: RequestInit
|
||||
): Promise<T> {
|
||||
// Check if token needs refresh before request
|
||||
let accessToken = localStorage.getItem('access_token');
|
||||
|
||||
if (!accessToken || isTokenExpired(accessToken)) {
|
||||
const refreshed = await refreshTokens();
|
||||
if (!refreshed) {
|
||||
throw new AuthenticationError('Session expired. Please log in again.');
|
||||
}
|
||||
accessToken = localStorage.getItem('access_token');
|
||||
}
|
||||
|
||||
const response = await fetch(`${API_BASE_URL}${endpoint}`, {
|
||||
...options,
|
||||
headers: {
|
||||
'Authorization': `Bearer ${accessToken}`,
|
||||
'Content-Type': 'application/json',
|
||||
...options?.headers,
|
||||
},
|
||||
});
|
||||
|
||||
// Handle 401 in case token expired between check and request
|
||||
if (response.status === 401) {
|
||||
const refreshed = await refreshTokens();
|
||||
if (refreshed) {
|
||||
// Retry with new token
|
||||
return authFetchWithRefresh(endpoint, options);
|
||||
}
|
||||
throw new AuthenticationError('Session expired. Please log in again.');
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
throw new ApiError(response.status, await response.json());
|
||||
}
|
||||
|
||||
return response.json();
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CORS Configuration
|
||||
|
||||
### Overview
|
||||
|
||||
Since the UI and API are hosted on different domains, Cross-Origin Resource Sharing (CORS) must be configured. The API uses Echo framework's CORS middleware with the following **default** configuration:
|
||||
|
||||
- **Allowed Origins**: `*` (all origins)
|
||||
- **Allowed Methods**: `GET, HEAD, PUT, PATCH, POST, DELETE`
|
||||
- **Allowed Headers**: All headers
|
||||
- **Credentials**: **NOT enabled by default** (`AllowCredentials: false`)
|
||||
|
||||
**Important**: The default CORS configuration does **not** support credentials (cookies). This means:
|
||||
- Cross-origin requests with `credentials: 'include'` will fail
|
||||
- Cookie-based authentication will not work cross-origin without server configuration changes
|
||||
- **Use `Authorization: Bearer` header for cross-origin authentication** (recommended)
|
||||
|
||||
### UI Configuration for Cross-Origin Requests
|
||||
|
||||
For cross-origin requests, use the `Authorization` header (not cookies):
|
||||
|
||||
```typescript
|
||||
// Recommended: Use Authorization header for cross-origin requests
|
||||
const response = await fetch(`${API_BASE_URL}/client/AAA`, {
|
||||
method: 'GET',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${accessToken}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
**Cookie-based authentication cross-origin** requires server-side CORS configuration changes:
|
||||
1. `AllowCredentials: true` must be set on the server
|
||||
2. `AllowOrigins` must list specific origins (cannot use wildcard `*`)
|
||||
3. UI must use `credentials: 'include'` in fetch calls
|
||||
|
||||
```typescript
|
||||
// Cookie-based auth (ONLY works if server is configured for credentials)
|
||||
const response = await fetch(`${API_BASE_URL}/client/AAA`, {
|
||||
method: 'GET',
|
||||
credentials: 'include', // Sends cookies cross-origin
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
// Authorization header optional if using cookies
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Preflight Requests
|
||||
|
||||
For non-simple requests (e.g., with custom headers like `Authorization`), the browser sends a preflight `OPTIONS` request. The API automatically handles these.
|
||||
|
||||
**Preflight triggers include**:
|
||||
- Custom headers (e.g., `Authorization`)
|
||||
- Content-Type other than `application/x-www-form-urlencoded`, `multipart/form-data`, or `text/plain`
|
||||
- HTTP methods other than `GET`, `HEAD`, `POST`
|
||||
|
||||
### Production CORS Configuration
|
||||
|
||||
For production deployments with specific UI domains, the API CORS configuration can be customized. Contact the backend team to configure:
|
||||
|
||||
```go
|
||||
// Example production CORS config (server-side)
|
||||
e.Use(middleware.CORSWithConfig(middleware.CORSConfig{
|
||||
AllowOrigins: []string{
|
||||
"https://app.yourdomain.com",
|
||||
"https://admin.yourdomain.com",
|
||||
},
|
||||
AllowMethods: []string{
|
||||
http.MethodGet,
|
||||
http.MethodHead,
|
||||
http.MethodPut,
|
||||
http.MethodPatch,
|
||||
http.MethodPost,
|
||||
http.MethodDelete,
|
||||
},
|
||||
AllowHeaders: []string{
|
||||
"Authorization",
|
||||
"Content-Type",
|
||||
"Accept",
|
||||
"X-Requested-With",
|
||||
},
|
||||
AllowCredentials: true,
|
||||
MaxAge: 86400, // Preflight cache: 24 hours
|
||||
}))
|
||||
```
|
||||
|
||||
### Troubleshooting CORS Issues
|
||||
|
||||
| Error | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| `No 'Access-Control-Allow-Origin' header` | Origin not allowed | Add UI origin to server config |
|
||||
| `Preflight response is not successful` | OPTIONS blocked | Ensure server handles OPTIONS |
|
||||
| `Credentials flag is true, but...` | Wildcard origin with credentials | Use specific origins, not `*` |
|
||||
| `Request header not allowed` | Custom header blocked | Add header to `AllowHeaders` |
|
||||
|
||||
---
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
### Token Storage
|
||||
|
||||
| Storage Option | Security | Recommendation |
|
||||
|----------------|----------|----------------|
|
||||
| `localStorage` | Vulnerable to XSS | Use with caution |
|
||||
| `sessionStorage` | Cleared on tab close, vulnerable to XSS | Better than localStorage |
|
||||
| Memory (variable) | Lost on refresh, most secure | Best for high-security apps |
|
||||
| HTTP-only cookies | Protected from XSS | Requires server-side support |
|
||||
|
||||
**Recommended approach for SPAs**:
|
||||
|
||||
```typescript
|
||||
// Store tokens in memory with sessionStorage backup
|
||||
class TokenManager {
|
||||
private accessToken: string | null = null;
|
||||
private idToken: string | null = null;
|
||||
private refreshToken: string | null = null;
|
||||
|
||||
constructor() {
|
||||
// Restore from sessionStorage on page load
|
||||
this.accessToken = sessionStorage.getItem('access_token');
|
||||
this.idToken = sessionStorage.getItem('id_token');
|
||||
this.refreshToken = sessionStorage.getItem('refresh_token');
|
||||
}
|
||||
|
||||
setTokens(tokens: { accessToken: string; idToken: string; refreshToken: string }) {
|
||||
this.accessToken = tokens.accessToken;
|
||||
this.idToken = tokens.idToken;
|
||||
this.refreshToken = tokens.refreshToken;
|
||||
|
||||
// Backup to sessionStorage (cleared on tab close)
|
||||
sessionStorage.setItem('access_token', tokens.accessToken);
|
||||
sessionStorage.setItem('id_token', tokens.idToken);
|
||||
sessionStorage.setItem('refresh_token', tokens.refreshToken);
|
||||
}
|
||||
|
||||
getAccessToken(): string | null {
|
||||
return this.accessToken;
|
||||
}
|
||||
|
||||
clearTokens() {
|
||||
this.accessToken = null;
|
||||
this.idToken = null;
|
||||
this.refreshToken = null;
|
||||
sessionStorage.clear();
|
||||
}
|
||||
}
|
||||
|
||||
export const tokenManager = new TokenManager();
|
||||
```
|
||||
|
||||
### Content Security Policy
|
||||
|
||||
Configure CSP headers in your UI application to prevent XSS:
|
||||
|
||||
```html
|
||||
<meta http-equiv="Content-Security-Policy"
|
||||
content="default-src 'self';
|
||||
connect-src 'self' https://api.yourdomain.com https://cognito-idp.us-east-2.amazonaws.com;
|
||||
script-src 'self';
|
||||
style-src 'self' 'unsafe-inline';">
|
||||
```
|
||||
|
||||
### Additional Security Measures
|
||||
|
||||
1. **Always use HTTPS** for both UI and API
|
||||
2. **Validate token on the client** before making requests
|
||||
3. **Implement logout** that clears all stored tokens
|
||||
4. **Set reasonable token lifetimes** in Cognito configuration
|
||||
5. **Monitor for suspicious activity** via API logs
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Required Headers
|
||||
|
||||
```
|
||||
Authorization: Bearer <access_token>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
### Common Endpoints
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/identity` | GET | Get user roles and permissions |
|
||||
| `/health` | GET | API health check (no auth required) |
|
||||
| `/clients` | GET | List all clients |
|
||||
| `/client/{id}` | GET | Get client details |
|
||||
| `/client/{id}/document` | GET | List client documents |
|
||||
| `/client/{id}/document` | POST | Upload document (multipart/form-data) |
|
||||
| `/client/{id}/document/batch` | POST | Upload ZIP batch |
|
||||
| `/document/{id}` | GET | Get document details |
|
||||
| `/clients/{clientId}/folders` | GET | List folders |
|
||||
|
||||
### Public Endpoints (No Authentication Required)
|
||||
|
||||
| Endpoint | Description |
|
||||
|----------|-------------|
|
||||
| `/health` | Health check and version info |
|
||||
| `/swagger/*` | API documentation |
|
||||
| `/metrics` | Prometheus metrics |
|
||||
|
||||
### TypeScript Type Definitions
|
||||
|
||||
```typescript
|
||||
// Identity types
|
||||
interface IdentityRole {
|
||||
key: string;
|
||||
name: string;
|
||||
description?: string;
|
||||
permissions: string[];
|
||||
}
|
||||
|
||||
interface IdentityResponse {
|
||||
roles: IdentityRole[];
|
||||
}
|
||||
|
||||
// Error types
|
||||
interface ErrorResponse {
|
||||
message: string;
|
||||
}
|
||||
|
||||
// Client types
|
||||
interface Client {
|
||||
client_id: string;
|
||||
name: string;
|
||||
can_sync: boolean;
|
||||
}
|
||||
|
||||
// Document types
|
||||
interface Document {
|
||||
id: string;
|
||||
client_id: string;
|
||||
hash: string;
|
||||
filename?: string;
|
||||
folder_id?: string;
|
||||
fields?: Record<string, any>;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
For API issues or questions:
|
||||
- Check the Swagger documentation at `/swagger/index.html`
|
||||
- Review API logs for detailed error information
|
||||
- Contact the backend team for CORS or authentication configuration changes
|
||||
@@ -0,0 +1,349 @@
|
||||
# Query API Endpoints Summary
|
||||
|
||||
This document provides a comprehensive summary of all REST API endpoints available in the Query Orchestration Platform.
|
||||
|
||||
## Quick Reference (All Endpoints)
|
||||
|
||||
| Path | Methods | Service |
|
||||
|----------------------------------------|--------------------------|----------------------|
|
||||
| /login | GET | AuthService |
|
||||
| /login-callback | GET | AuthService |
|
||||
| /logout | GET | AuthService |
|
||||
| /home | GET | AuthService |
|
||||
| /identity | GET | AuthService |
|
||||
| /client | POST | ClientService |
|
||||
| /clients | GET | ClientService |
|
||||
| /client/{id} | GET, PATCH | ClientService |
|
||||
| /client/{id}/status | GET | ClientService |
|
||||
| /client/{id}/collector | GET, PATCH | CollectorService |
|
||||
| /client/{id}/folders | GET | FolderService |
|
||||
| /folders | POST | FolderService |
|
||||
| /folders/{folderId} | PATCH | FolderService |
|
||||
| /folders/{folderId}/documents | GET | FolderService |
|
||||
| /folders/{folderId}/metrics | GET | FolderService |
|
||||
| /client/{id}/document | GET, POST | DocumentsService |
|
||||
| /client/{id}/document/batch | GET, POST | DocumentsService |
|
||||
| /client/{id}/document/batch/{batch_id} | GET, DELETE | DocumentsService |
|
||||
| /document/{id} | GET | DocumentsService |
|
||||
| /documents/{documentId}/labels | GET, POST | LabelService |
|
||||
| /labels/{labelName}/documents | GET | LabelService |
|
||||
| /field-extractions | GET, POST | FieldExtractionService |
|
||||
| /field-extractions/version | GET | FieldExtractionService |
|
||||
| /field-extractions/history | GET | FieldExtractionService |
|
||||
| /query | GET, POST | QueryService |
|
||||
| /query/{id} | GET, PATCH | QueryService |
|
||||
| /query/{id}/test | POST | QueryService |
|
||||
| /client/{id}/export | POST | ExportService |
|
||||
| /export/{id} | GET | ExportService |
|
||||
| /admin/users | GET, POST | AdminService |
|
||||
| /admin/users/{email} | GET, HEAD, PATCH, DELETE | AdminService |
|
||||
| /admin/users/{email}/disable | POST | AdminService |
|
||||
| /admin/users/{email}/enable | POST | AdminService |
|
||||
|
||||
---
|
||||
|
||||
## Authentication Service
|
||||
|
||||
### OAuth2 Authentication Flow
|
||||
- **GET /login** - Initiate login flow by redirecting to Cognito IDP
|
||||
- Redirects to AWS Cognito login page
|
||||
- No authentication required
|
||||
|
||||
- **GET /login-callback** - Handle OAuth2 callback and exchange code for tokens
|
||||
- Query Parameters:
|
||||
- `code` (required) - Authorization code from Cognito
|
||||
- `state` (required) - CSRF protection parameter
|
||||
- Sets JWT cookie and redirects to dashboard
|
||||
|
||||
- **GET /logout** - Logout user and invalidate session
|
||||
- Requires authentication
|
||||
- Redirects to Cognito logout page or application home
|
||||
|
||||
- **GET /home** - Get the home page menu for authenticated users
|
||||
- Requires authentication
|
||||
- Returns HTML content for authenticated user menu
|
||||
|
||||
- **GET /identity** - Get current user identity and roles
|
||||
- Requires authentication
|
||||
- Returns user's assigned roles from Permit.io with permissions
|
||||
- Returns: `{user_id, email, roles[]}`
|
||||
|
||||
## Client Service
|
||||
|
||||
### Client Management
|
||||
- **POST /client** - Create a new client with provided details
|
||||
- Request Body: `{id, name}`
|
||||
- Returns: `{id}` (201 Created)
|
||||
|
||||
- **GET /clients** - List all clients in the system
|
||||
- Returns: `{clients[]}` where each client contains `{id, name}`
|
||||
|
||||
- **GET /client/{id}** - Get a specific client by its ID
|
||||
- Returns: `{id, name, can_sync}` (200 OK)
|
||||
|
||||
- **PATCH /client/{id}** - Update an existing client with new details
|
||||
- Request Body: `{name?, can_sync?}` (optional fields)
|
||||
- Returns: 200 OK on success
|
||||
|
||||
- **GET /client/{id}/status** - Retrieve the sync status for a client
|
||||
- Returns: `{status}` where status is one of: `IN_SYNC`, `NOT_SYNCED`, `NOT_SYNCING`
|
||||
|
||||
## Collector Service
|
||||
|
||||
### Collector Configuration Management
|
||||
- **GET /client/{id}/collector** - Get a collector configuration by client ID
|
||||
- Returns: `{client_id, active_version, latest_version, minimum_cleaner_version, minimum_text_version, fields[]}`
|
||||
|
||||
- **PATCH /client/{id}/collector** - Set or update collector configuration for a client
|
||||
- Request Body: `{minimum_cleaner_version?, minimum_text_version?, active_version?, fields[]?}`
|
||||
- Returns: 204 No Content on success
|
||||
|
||||
## Folder Service
|
||||
|
||||
### Folder Management
|
||||
- **GET /client/{id}/folders** - List all folders for a client
|
||||
- Query Parameters:
|
||||
- `metrics` (optional, default: false) - When true, includes processing metrics for each folder
|
||||
- Returns: `{folders[]}` where each folder contains `{id, path, parent_id, metrics?}`
|
||||
- Root-level folders have null parentId
|
||||
|
||||
- **POST /folders** - Create a new folder
|
||||
- Request Body: `{client_id, path, parent_id?}`
|
||||
- Returns: `{id, client_id, path, parent_id}` (201 Created)
|
||||
|
||||
- **PATCH /folders/{folderId}** - Rename a folder
|
||||
- Request Body: `{path}`
|
||||
- Returns: `{id, client_id, path, parent_id}` (200 OK)
|
||||
|
||||
- **GET /folders/{folderId}/documents** - Get documents in a folder
|
||||
- Query Parameters:
|
||||
- `textRecord` (optional, default: false) - When true, includes full text extraction record
|
||||
- Returns: `{documents[]}` with document details including filename and labels
|
||||
|
||||
- **GET /folders/{folderId}/metrics** - Get folder processing metrics
|
||||
- Returns: `{total_documents, processed_documents, label_counts{}}`
|
||||
|
||||
## Documents Service
|
||||
|
||||
### Document Management
|
||||
- **GET /client/{id}/document** - List all documents for a specific client
|
||||
- Returns: Array of `{id, hash}` document summaries
|
||||
|
||||
- **POST /client/{id}/document** - Upload a single PDF file for a client
|
||||
- Content-Type: `multipart/form-data`
|
||||
- Form Data:
|
||||
- `file` (required) - PDF file to upload (binary)
|
||||
- `filename` (optional) - Custom filename (max 4096 chars)
|
||||
- Returns: 200 OK on successful upload
|
||||
|
||||
- **GET /document/{id}** - Get document details by its ID
|
||||
- Query Parameters:
|
||||
- `textRecord` (optional, default: false) - When true, includes full text extraction record
|
||||
- Returns: `{id, client_id, hash, folder_id, filename, labels[], fields{}, text_record?}`
|
||||
|
||||
### Batch Document Processing
|
||||
- **POST /client/{id}/document/batch** - Upload multiple PDFs as ZIP archive for batch processing
|
||||
- Content-Type: `multipart/form-data`
|
||||
- Form Data:
|
||||
- `archive` (required) - ZIP file containing PDF documents (max 1GB)
|
||||
- Returns: `{batch_id, status, status_url}` (202 Accepted)
|
||||
|
||||
- **GET /client/{id}/document/batch** - List batch uploads for a client with pagination
|
||||
- Query Parameters:
|
||||
- `limit` (optional, default: 20, max: 100) - Number of items to return
|
||||
- `offset` (optional, default: 0) - Number of items to skip
|
||||
- Returns: `{batches[], total_count}`
|
||||
|
||||
- **GET /client/{id}/document/batch/{batch_id}** - Get detailed status of a specific batch upload
|
||||
- Returns: `{batch_id, client_id, original_filename, status, total_documents, processed_documents, failed_documents, invalid_type_documents, progress_percent, created_at, completed_at?, failed_filenames[]}`
|
||||
|
||||
- **DELETE /client/{id}/document/batch/{batch_id}** - Cancel a batch upload currently processing
|
||||
- Returns: 204 No Content on successful cancellation
|
||||
|
||||
## Label Service
|
||||
|
||||
### Document Label Management
|
||||
- **POST /documents/{documentId}/labels** - Apply a label to a document
|
||||
- Request Body: `{label_name}`
|
||||
- Labels track document processing state/workflow
|
||||
- Returns: `{id, document_id, label_name, applied_at}` (201 Created)
|
||||
|
||||
- **GET /documents/{documentId}/labels** - Get all labels for a document
|
||||
- Returns: `{labels[]}` ordered by most recent first
|
||||
- Each label contains: `{id, document_id, label_name, applied_at}`
|
||||
|
||||
- **GET /labels/{labelName}/documents** - Get all documents with a specific label
|
||||
- Query Parameters:
|
||||
- `clientId` (required) - Filter by client ID
|
||||
- Returns: `{documents[]}` with document summaries
|
||||
|
||||
## Field Extraction Service
|
||||
|
||||
### Document Field Extraction Management
|
||||
- **POST /field-extractions** - Create a new field extraction
|
||||
- Request Body: `{document_id, single_fields{}, array_fields{}}`
|
||||
- Creates versioned field extraction record
|
||||
- Returns: `{id, document_id, version, single_fields{}, array_fields{}, created_at}` (201 Created)
|
||||
|
||||
- **GET /field-extractions** - Get current (most recent) field extraction for a document
|
||||
- Query Parameters:
|
||||
- `documentId` (required) - The document ID
|
||||
- Returns: `{id, document_id, version, single_fields{}, array_fields{}, created_at}`
|
||||
|
||||
- **GET /field-extractions/version** - Get field extraction by specific version
|
||||
- Query Parameters:
|
||||
- `documentId` (required) - The document ID
|
||||
- `version` (required) - The version number to retrieve
|
||||
- Returns: `{id, document_id, version, single_fields{}, array_fields{}, created_at}`
|
||||
|
||||
- **GET /field-extractions/history** - Get field extraction version history
|
||||
- Query Parameters:
|
||||
- `documentId` (required) - The document ID
|
||||
- Returns: `{versions[]}` ordered by most recent first
|
||||
- Each version contains: `{version, created_at}`
|
||||
|
||||
## Query Service
|
||||
|
||||
### Query Definition and Execution
|
||||
- **GET /query** - List queries based on filter criteria
|
||||
- Returns: `{queries[]}` where each query contains `{id, type, active_version, latest_version, config?, required_queries[]?}`
|
||||
|
||||
- **POST /query** - Create a new query with provided details
|
||||
- Request Body: `{type, config?, required_queries[]?}`
|
||||
- Supported Types: `JSON_EXTRACTOR`, `CONTEXT_FULL`
|
||||
- Returns: `{id}` (201 Created)
|
||||
|
||||
- **GET /query/{id}** - Get a specific query by its ID
|
||||
- Returns: `{id, type, active_version, latest_version, config?, required_queries[]?}`
|
||||
|
||||
- **PATCH /query/{id}** - Update an existing query with new details
|
||||
- Request Body: `{config?, active_version?, required_queries[]?}`
|
||||
- Returns: 200 OK on success
|
||||
|
||||
- **POST /query/{id}/test** - Execute a test run of a query with parameters
|
||||
- Request Body: `{document_id, query_version}`
|
||||
- Returns: `{value}` - Result of the query test
|
||||
|
||||
## Export Service
|
||||
|
||||
### Export Management
|
||||
- **POST /client/{id}/export** - Trigger an export process for a client
|
||||
- Request Body: `{ingestion_filters?, field_filters[]?}`
|
||||
- Ingestion Filters: `{start_date?, end_date?}`
|
||||
- Field Filters: `{field_name, condition, values[]}`
|
||||
- Conditions: `less_than`, `greater_than`, `closed_interval`, `open_interval`, `left_closed_interval`, `right_closed_interval`, `include`, `exclude`
|
||||
- Returns: `{id}` (201 Created)
|
||||
|
||||
- **GET /export/{id}** - Check the current state of an export
|
||||
- Returns: `{client_id, status, output_location?}`
|
||||
- Status values: `completed`, `in_progress`, `failed`
|
||||
|
||||
## Admin Service
|
||||
|
||||
### User Management
|
||||
|
||||
#### User Creation and Listing
|
||||
- **POST /admin/users** - Create a new user
|
||||
- Request Body: `{email, first_name, last_name, roles[]}`
|
||||
- Creates user in both AWS Cognito and Permit.io
|
||||
- Roles: Must exist in Permit.io (e.g., `super_admin`, `user_admin`, `auditor`, `client_user`)
|
||||
- Returns: `{cognito_subject_id, email, first_name, last_name, status, enabled, permit_synced, roles_assigned[], created_at}` (201 Created or 200 OK if already exists)
|
||||
- **Idempotent**: Returns 200 OK if user already exists with matching attributes
|
||||
|
||||
- **GET /admin/users** - List users with pagination, filtering, and sorting
|
||||
- Query Parameters:
|
||||
- `page` (optional, default: 1, max: 10000) - Page number
|
||||
- `page_size` (optional, default: 50, max: 100) - Results per page
|
||||
- `search` (optional) - Search term for email/name filtering
|
||||
- `status` (optional, default: `all`) - Filter by status: `enabled`, `disabled`, `all`
|
||||
- `sort_by` (optional, default: `email`) - Sort field: `email`, `created_at`, `last_name`
|
||||
- `sort_order` (optional, default: `asc`) - Sort direction: `asc`, `desc`
|
||||
- Returns: `{users[], total, page, page_size, has_more}`
|
||||
|
||||
#### Individual User Operations
|
||||
- **GET /admin/users/{email}** - Get user by email
|
||||
- Returns: `{cognito_subject_id, email, first_name, last_name, status, enabled, roles[], created_at, updated_at}`
|
||||
|
||||
- **HEAD /admin/users/{email}** - Check if user exists
|
||||
- Returns: 200 OK with headers:
|
||||
- `X-User-Exists-Cognito: boolean` - Whether user exists in Cognito
|
||||
- `X-User-Exists-PermitIO: boolean` - Whether user exists in Permit.io
|
||||
- Returns: 404 Not Found if user doesn't exist in either system
|
||||
|
||||
- **PATCH /admin/users/{email}** - Update user attributes and roles
|
||||
- Request Body: `{first_name?, last_name?, roles[]?}`
|
||||
- **Role Management**: Providing `roles[]` REPLACES all existing roles (not additive)
|
||||
- To add a role: Include all current roles PLUS the new role
|
||||
- To remove a role: Include only roles you want to keep
|
||||
- To remove all roles: Send empty array `[]`
|
||||
- Omit `roles` field to leave roles unchanged
|
||||
- Returns: `{cognito_subject_id, email, first_name, last_name, status, enabled, roles[], created_at, updated_at}` (200 OK)
|
||||
|
||||
- **DELETE /admin/users/{email}** - Delete user permanently
|
||||
- Query Parameters:
|
||||
- `confirm=true` (required) - Must be set to confirm deletion
|
||||
- Deletes from both AWS Cognito and Permit.io
|
||||
- **Irreversible operation**
|
||||
- Returns: `{success, email, cognito_subject_id, deleted_from{cognito, permit}, timestamp}` (200 OK or 207 Multi-Status for partial deletion)
|
||||
|
||||
#### User Account Status Management
|
||||
- **POST /admin/users/{email}/disable** - Disable user
|
||||
- Disables user in AWS Cognito, preventing authentication
|
||||
- User data and roles are preserved
|
||||
- **Idempotent**: Safe to call multiple times
|
||||
- Returns: `{success, email, action, cognito_subject_id, timestamp}` (200 OK)
|
||||
|
||||
- **POST /admin/users/{email}/enable** - Enable user
|
||||
- Re-enables a previously disabled user in AWS Cognito
|
||||
- Restores authentication capability
|
||||
- **Idempotent**: Safe to call multiple times
|
||||
- Returns: `{success, email, action, cognito_subject_id, timestamp}` (200 OK)
|
||||
|
||||
## Common Response Codes
|
||||
|
||||
All endpoints may return the following standard HTTP response codes:
|
||||
|
||||
- **200 OK** - Request succeeded
|
||||
- **201 Created** - Resource created successfully
|
||||
- **202 Accepted** - Request accepted for async processing
|
||||
- **204 No Content** - Request succeeded with no response body
|
||||
- **207 Multi-Status** - Partial success (e.g., deletion from only one system)
|
||||
- **302 Found** - Redirect (for auth flows)
|
||||
- **400 Bad Request** - Invalid request body or parameters
|
||||
- **401 Unauthorized** - Authentication required or failed
|
||||
- **403 Forbidden** - Insufficient permissions
|
||||
- **404 Not Found** - Resource does not exist
|
||||
- **409 Conflict** - Resource already exists or state conflict
|
||||
- **429 Too Many Requests** - Rate limit exceeded (includes `Retry-After` header)
|
||||
- **500 Internal Server Error** - Server-side error
|
||||
- **502 Bad Gateway** - External service error (e.g., Cognito, Permit.io)
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
All endpoints implement dual-layer rate limiting:
|
||||
- **Global Rate Limit**: 500 requests/second across all IPs (1000 burst)
|
||||
- **Per-IP Rate Limit**: 5 requests/second per IP (10 burst)
|
||||
|
||||
Rate limit headers included in all responses:
|
||||
- `RateLimit: limit;window=time-window` (e.g., `100;window=60`)
|
||||
- `Retry-After: seconds` (only on 429 responses)
|
||||
|
||||
## Security
|
||||
|
||||
- **Authentication**: JWT Bearer tokens via AWS Cognito OAuth2
|
||||
- **Authorization**: Role-based access control (RBAC) via Permit.io
|
||||
- **All endpoints require authentication** except:
|
||||
- `GET /login` - Public login initiation
|
||||
- `GET /login-callback` - OAuth2 callback handler
|
||||
|
||||
## API Versioning
|
||||
|
||||
- **Base URL**: `https://doczy.com/v1` (production)
|
||||
- **Local Development**: `http://localhost:8080`
|
||||
- **API Version**: 0.0.1
|
||||
- **OpenAPI Specification**: `serviceAPIs/queryAPI.yaml`
|
||||
|
||||
## Documentation
|
||||
|
||||
- **Swagger UI**: Available at `/swagger/index.html` when running locally
|
||||
- **OpenAPI Spec**: Auto-generated from `serviceAPIs/queryAPI.yaml`
|
||||
Reference in New Issue
Block a user