# Chatbot Backend Guide This guide covers the chatbot backend that queryAPI exposes for the Aaria Agent Service integration. It is the operator entry point; design rationale lives in `plans/chatbot_plan_codex.v10.md`. ## Overview queryAPI exposes a synchronous HTTP facade over the Aaria FastAPI agent service. Client UIs talk to queryAPI; queryAPI calls FastAPI and persists the conversation in Postgres so future turns inherit the prior context. Who calls it: - Client UIs as `client_user`: create sessions, post turns, list turns, patch document/folder scope. - Super-admins: list, get, soft-delete sessions across all users for a given client. What it does NOT do (out of scope per plan v10): - Full-text search. - GIN indexes. - Costing / billing. - Persisted structured data or result rows from the agent. - User-initiated session deletes (only super-admin DELETE). ## Architecture ### Data model (`bot_*` tables, migration 131) | Table | Purpose | | ----------------------- | ----------------------------------------------------------------------------------------------- | | `bot_sessions` | Owner-scoped chatbot conversations. Soft-deletable via `is_deleted`. Derives `last_terminal_ordinal` and `last_seen_ordinal` from `bot_turns`. | | `bot_session_documents` | Per-session document scope. Empty set means "all documents for the client" (plan §4 fallback). | | `bot_session_folders` | Per-session folder scope. Folder ids preserved in the FastAPI payload; expansion to documents happens in M3 payload assembly. | | `bot_turns` | Append-only turn log. Status is one of `in_flight`, `completed`, `errored`, `abandoned`, `session_deleted`. | Two CHECK constraints govern `bot_turns`: 1. `bot_turns_status_values` enforces the enum set. 2. `bot_turns_status_fields_consistent` enforces the (status, completion, error) consistency matrix from plan §4 plus a guard clause for unknown statuses. The guard clause `OR status NOT IN (...)` lets `bot_turns_status_values` own the rejection of unknown values without `_status_fields_consistent` firing first under Postgres's alphabetical-by-name CHECK evaluation order. The plan §4 literal three-clause matrix is preserved verbatim for documented statuses. The partial unique index `bot_turns_one_inflight_per_session` enforces "at most one in-flight turn per session". This is the primary mechanism for serializing concurrent submissions. ### AddTurn flow (plan §6) ```text AddTurn(input): validate prompt (1 <= rune count <= MaxTurnChars) attemptID = uuid.New() tx1: LockBotSessionForOwner(...) AbandonExpiredInflightForOwner(grace = max(2 * RequestTimeoutSeconds, 60s)) re-derive last_seen_ordinal / last_terminal_ordinal GetBotTurn(input.Ordinal) if existing exists: classify by status; reset on errored/abandoned (prompt must match, ordinal must equal last_terminal_ordinal); reject on others. else: check ordinal matches natural target (last_seen+1 if has_inflight, else last_terminal+1); InsertBotTurnPrompt; partial unique index may reject -> ErrPriorTurnInFlight. if input.Ordinal == 1: SetBotSessionTitle(first 120 runes of prompt) where title='' end tx1 Build conversation_history (last RecentTurns completed turns x 2 messages, asc). Build chat_scope (stored docs + folder-expanded docs, dedup; folder ids preserved). Call FastAPI /agent/chat OUTSIDE any DB tx (network round-trip). tx2: LockBotSessionForOwner(...) if no row -> session was soft-deleted mid-call: MarkBotTurnSessionDeleted(...) with attempt_id compare-and-set return ErrSessionDeletedDuringCall (HTTP 410) if FastAPI returned a typed error: FailBotTurn(...) with error JSON containing attempt count return mapped agent_* sentinel (HTTP 502) if FastAPI succeeded but answer.text is empty/oversize: FailBotTurn(...) with agent_invalid_response error JSON return ErrAgentInvalidResponse (HTTP 502) success path: CompleteBotTurn(...) with attempt_id compare-and-set summarize metadata (latency, tokens) apply state update: null -> preserve prior; non-null -> strip cached_results, write if under 64KB cap, else log warn + preserve end tx2 ``` The two-transaction model is deliberate: no DB connection is held while calling FastAPI (which can take tens of seconds), and tx2's attempt_id compare-and-set rejects stale callbacks (`zero rows affected -> ErrTurnSuperseded`). Plan §6 + §11. ### State management `bot_sessions.state jsonb` mirrors the agent's session state across turns. On a successful turn: - FastAPI `updated_session_state == null` -> preserve prior state. The service does NOT call `UpdateBotSessionState`. - `updated_session_state` non-null -> `StripCachedResults` removes the top-level `cached_results` key (a hash-keyed MAP, often the largest field). Then `SessionStateOversize` checks the result against `chatbot.StateMaxBytes = 64 * 1024`. Under cap -> persist; over cap -> log warn and preserve prior state. The canonical example of a `session_state` payload is `plans/chatbot.stuff/sample.response.json`. It has 10 top-level keys: `turn`, `resolved_entities`, `last_scope`, `last_intent`, `last_query_sql`, `last_query_plan`, `last_result_hash`, `cached_results`, `last_scope_signature`, `validator_retry_count`. After strip, the persisted payload has the remaining 9 top-level keys. ### Retry policy (FastAPI client, plan §7) - Up to 3 attempts per HTTP call. - Retry on HTTP 429, 500, 502, 503, 504, and transport timeouts. - Do NOT retry other 4xx, malformed JSON, HTTP 200 with `status:"error"`, or HTTP 200 with empty/null `answer.text`. - Reuse the same `request_id` (`{sessionID}:{ordinal}`) on every attempt. The caller mints it once; the client must not regenerate per-attempt. - One parent context enforces total wall time. Per-attempt timeout = `cfg.RequestTimeoutSeconds` via `context.WithTimeout` derived from the parent. - Header literal: `X-API-Key: `. Production values are JSON-encoded strings (`{"AGENT_SERVICE_API_KEY":"value"}`) per plan §11. ### Error JSON shape All `bot_turns.error` writes use the plan §6 canonical shape: ```json { "code": "", "message": "", "attempt": } ``` The `attempt` field carries the FastAPI client's `AttemptCount` so operators can distinguish single-failure from exhausted-retry scenarios. ## Configuration reference Five env vars, plan §8. Validated at startup via `chatbot.ChatbotConfig.Validate` (called between `serviceconfig.InitializeConfig` and `cfg.InitializeAuthConfig`). | Variable | Default | Validation | Notes | | --------------------------------- | ---------------------- | ----------------------------------- | ---------------------------------------------------- | | `CHATBOT_SERVICE_URL` | (required, notEmpty) | env-loader rejects empty | FastAPI base URL. | | `CHATBOT_API_KEY` | (required, notEmpty) | env-loader rejects empty | Sent verbatim as `X-API-Key` header. | | `CHATBOT_REQUEST_TIMEOUT_SECONDS` | 60 | `>= 1` | Per-attempt FastAPI timeout. | | `CHATBOT_MAX_TURN_CHARS` | 2000 | `>= 1` | Caps prompt and `answer.text` rune count. | | `CHATBOT_RECENT_TURNS` | 5 | `1 <= x <= 5` | Conversation-history length and session-list preview cap. | Derived values (constants/helpers, not env vars): - `chatbot.StateMaxBytes = 64 * 1024` — session-state cap (plan §8). - `chatbot.ChatbotConfig.InflightGraceSeconds() = 2 * RequestTimeoutSeconds` per plan §8 literal. The service uses `effectiveGraceSeconds(cfg) = max(2 * RequestTimeoutSeconds, 60)` as the actual abandonment threshold to keep small `RequestTimeoutSeconds` in test fixtures from prematurely abandoning legitimately in-flight turns. Production `RequestTimeoutSeconds=60` already exceeds the 60-second floor. ## API reference ### User routes (`/client/{clientId}/...`) All routes are singular `/client/...` per plan §3 so Permit.io's existing `client_user` policy applies. | Method | Path | OperationId | Description | | ------ | ----------------------------------------------------------------- | ------------------------- | -------------------------------------------------------- | | POST | `/client/{clientId}/bot/sessions` | `createBotSession` | Create a new owner-scoped session. | | GET | `/client/{clientId}/bot/sessions` | `listBotSessions` | List the caller's sessions; bounds: `limit in [1,200]`, `offset >= 0`. | | GET | `/client/{clientId}/bot/sessions/{sessionId}` | `getBotSession` | Read one owner-scoped session. | | PATCH | `/client/{clientId}/bot/sessions/{sessionId}/scope` | `patchBotSessionScope` | Add/remove documents and folders from session scope. | | POST | `/client/{clientId}/bot/sessions/{sessionId}/turns` | `createBotTurn` | Submit a new turn. Plan §6 algorithm. | | GET | `/client/{clientId}/bot/sessions/{sessionId}/turns` | `listBotTurns` | List all turns for an owner-scoped session, ASC. | | GET | `/client/{clientId}/documents/{documentId}/schema` | `getDocumentSchema` | Return the document's bound JSON Schema or `null` (omitted in JSON when unbound). | | GET | `/client/{clientId}/documents/{documentId}/all-metadata` | `getDocumentAllMetadata` | Bundle of (document, labels, customMetadata) for one doc. | ### Super-admin routes | Method | Path | OperationId | Description | | ------ | ----------------------------------------------------- | --------------------------------- | -------------------------------------------- | | GET | `/super-admin/bot/sessions?clientId=...` | `listBotSessionsAsSuperAdmin` | List all sessions for a client (cross-user). | | GET | `/super-admin/bot/sessions/{sessionId}?clientId=...` | `getBotSessionAsSuperAdmin` | Read one session by id (cross-user). | | DELETE | `/super-admin/bot/sessions/{sessionId}?clientId=...` | `deleteBotSessionAsSuperAdmin` | Soft-delete; transitions any in-flight turns to `session_deleted`. | ### `BotSessionResponse.lastTurn` `lastTurn` returns `last_terminal_ordinal` (plan §4 derived note), NOT `last_seen_ordinal`. The latter is computed from `bot_turns` but used only internally by `AddTurn` for ordinal classification. Clients should display `lastTurn` as "the last completed turn this session has". ### `DocumentSchemaResponse.schema` When the document has no `custom_schema_id` binding, the response is HTTP 200 with the `schema` field omitted (the field is `*map[...]` with `omitempty`, so JSON does NOT emit `"schema": null`; the key is absent). Clients should treat both "key absent" and "key present and null" as the unbound case. ### `BotLabelRecord` vs `LabelRecord` The all-metadata endpoint uses `BotLabelRecord` (plain-string `appliedBy`) instead of the existing `LabelRecord` (email-validated `appliedBy`). Both shapes wrap the same `documentLabels.appliedBy varchar` column; the divergence exists because the email-regex Marshal validator on `LabelRecord` rejects non-email seed values that pre-date the email-typing convention. Long-term cleanup is to relax `LabelRecord.AppliedBy` to plain string. Until then, the chatbot endpoints emit `BotLabelRecord`. ### HTTP status mapping | Status | Sentinels / cases | | ------ | -------------------------------------------------------------------------------------------- | | 400 | `prompt_empty`, `prompt_too_long`, `add_remove_conflict`, `document_cross_client`, `folder_cross_client`, `clientId is required` (super-admin), invalid `limit`/`offset` | | 401 | Missing Cognito subject | | 404 | Cross-client / cross-user / soft-deleted session; document not found in client scope | | 409 | `turn_in_flight`, `prior_turn_in_flight`, `already_complete`, `turn_session_deleted`, `ordinal_out_of_range`, `turn_superseded` (legacy: `prompt_mismatch` — service-layer assertion only; not reachable from the wire as of 2026-05-07) | | 410 | `session_deleted_during_call` | | 502 | `agent_application_error`, `agent_missing_answer`, `agent_invalid_response`, `agent_transport_error` | | 500 | Unexpected backend errors | `mapBotTurnError` dispatch order matters because `ErrAlreadyComplete` chains to `ErrOrdinalOutOfRange` via a custom `Is()` method. The sentinels are matched in declaration order; `ErrAlreadyComplete` must be checked before `ErrOrdinalOutOfRange` so the message string "already_complete" is returned in the HTTP retry-of-completed scenario. The chain exists so plan §11's concurrency-test loss set (`ErrTurnInFlight | ErrPriorTurnInFlight | ErrOrdinalOutOfRange`) covers the case where a concurrent submitter's ordinal lands on the row the winner just completed. ### Test-pattern note Test assertions on HTTP error message bodies use substring matches. `turn_in_flight` matches both `turn_in_flight` and `prior_turn_in_flight` (the latter contains the former). M4's `TestAddTurn_SameOrdinalStillInflight_409` exploits this: the test name says "same ordinal", but the algorithm returns `prior_turn_in_flight` because the natural target is `last_seen+1` when has_inflight; the substring assertion still passes. Both 409s have the same operator semantics ("can't submit while a turn is in flight"). ## Operator runbook ### Failed-turn classifications When a `bot_turns` row has a non-`completed` status, operators consult `bot_turns.error` for the canonical error JSON and pick the recovery path below. #### `errored` The FastAPI agent returned a typed error or the response failed validation. The error JSON includes `code`, `message`, and `attempt`. Common codes: | Code | Meaning | Recovery | | -------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------------- | | `agent_application_error` | FastAPI returned 200 with `status:"error"`. Code/message in the inner `error` field. | Client may resubmit the same prompt at the same ordinal; the algorithm reset path runs. | | `agent_missing_answer` | FastAPI returned 200 with empty/null/whitespace `answer.text`. | Same as above. May indicate a prompt the agent cannot answer; client UX should suggest rephrasing. | | `agent_invalid_response` | FastAPI returned 200 but body failed structural validation (malformed JSON or oversize completion). | Same as above. May indicate an upstream agent change; check upstream logs. | | `agent_transport_error` | FastAPI exhausted its 3-attempt retry budget on transport-level failures (429/5xx/timeouts). | Same as above. May indicate FastAPI overload; check upstream health and rate-limit. | To retry: client POSTs to the same `/turns` route with the same prompt. The handler picks the `errored` row's ordinal via prompt-match; `AddTurn` runs the existing-row reset path; a new `attempt_id` is minted; tx2 records the new outcome. If the client posts a *different* prompt instead of retrying, the handler advances to `last_terminal_ordinal+1` (a fresh `in_flight` at ordinal N+1) rather than rejecting with 409 `prompt_mismatch`. The errored row at ordinal N stays in history. See `plans/chatbot.stuff/fix.chat.bugs.1.md` §3 Option A for the rationale. #### `abandoned` The grace window expired without a terminal status. Most often this means the queryAPI process died mid-call and the next caller's `AbandonExpiredInflightForOwner` swept the stale row. The error JSON carries: ```json {"code":"turn_abandoned_grace_expired","message":"no terminal status reached within grace window"} ``` Recovery is identical to `errored`: client resubmits the same prompt. The reset path applies. #### `session_deleted` A super-admin DELETEd the session while the turn was in flight, OR an older retry path hit the session-deleted-during-call branch. The error JSON carries: ```json {"code":"session_deleted_by_admin","message":"session deleted while turn in flight","attempt":} ``` OR (from the AddTurn tx2 path): ```json {"code":"session_deleted_during_call","message":"session was soft-deleted while FastAPI call was in flight","attempt":} ``` Recovery: NONE. The session is permanent. The client must create a new session and resubmit. The historical `session_deleted` row remains for audit; LIST turns surfaces it with the original prompt and the error JSON. ### Performance notes `decorateSessionList` (the M2 list-sessions handler path) does N+1 scope reads: one `loadScope` round-trip per session in the list. Default `limit=20`, hard cap `limit=200`. Under default load the overhead is negligible; if a deployment surfaces latency on the list endpoint, consider a single batch read of all scope rows for the listed session ids. Plan v10 marked this as acceptable for v1. ### Rate-limit overrides `RATE_LIMIT_ENDPOINT_OVERRIDES` is a JSON object keyed by `METHOD /path` using **Echo's path template form** (`:clientId`, `:sessionId`), NOT the OpenAPI `{clientId}` form. The middleware uses `c.Path()` which returns Echo's registered path string; an override keyed by `{clientId}` will silently fail to match. Example for the chatbot turn endpoint: ```bash RATE_LIMIT_ENDPOINT_OVERRIDES='{"POST /client/:clientId/bot/sessions/:sessionId/turns":{"global_rate":50,"global_burst":100,"rate":2,"burst":4}}' ``` **M6 operator-facing behavior change**: setting an override here NOW throttles the actual request rate (per-(IP, route) budget). Before M6, overrides only affected the `RateLimit` and `Retry-After` headers; the Allow() decision used the default rate/burst. Plan §10 M6 + plan §11. ## References - Plan: `plans/chatbot_plan_codex.v10.md` (design rationale and authoritative spec). - FastAPI swagger: `plans/chatbot.stuff/openapi.chatbot.swagger.json`. - Sample `session_state` fixture: `plans/chatbot.stuff/sample.response.json`. - Cloud FastAPI deployment (dev): see plan §11 for the URL and the literal `X-API-Key` value. ## Known warts These are not bugs; they are conscious tradeoffs documented for maintainers. 1. **CHECK guard clause** in `bot_turns_status_fields_consistent`. The plan §4 literal three-clause matrix is preserved verbatim; the trailing `OR status NOT IN (...)` is an evaluation-order guard so `bot_turns_status_values` owns the rejection of unknown enum values. 2. **`ErrAlreadyComplete` chain to `ErrOrdinalOutOfRange`** via custom `Is()` so plan §11's concurrency-test loss set covers the case where a loser's ordinal lands on the winner's just-completed row. 3. **`BotLabelRecord` vs `LabelRecord`** divergence (above). 4. **`schema=null`** uses field omission (omitempty) rather than explicit JSON null. Clients should accept both. 5. **`M5DocumentID` alias** is cosmetically distinct from `DocumentID` in the generated types. Both are `openapi_types.UUID`. Cosmetic only. 6. **Rate-limit override `:param` template** form, not OpenAPI `{param}` form (above). 7. **N+1 scope reads** in session list (above). 8. **Grace-window floor of 60s** in `effectiveGraceSeconds`; the plan §8 literal `2 * RequestTimeoutSeconds` is preserved on the chatbot config method, but the service applies the floor at the call site so test fixtures with `RequestTimeoutSeconds=1` (2s grace) do not prematurely abandon legitimately in-flight turns.