query-orchestration/docs/ai.generated/12-chatbot-guide.md

# Chatbot Backend Guide

This guide covers the chatbot backend that queryAPI exposes for the Aaria
Agent Service integration. It is the operator entry point; design
rationale lives in `plans/chatbot_plan_codex.v10.md`.

## Overview

queryAPI exposes a synchronous HTTP facade over the Aaria FastAPI agent
service. Client UIs talk to queryAPI; queryAPI calls FastAPI and
persists the conversation in Postgres so future turns inherit the
prior context.

Who calls it:
- Client UIs as `client_user`: create sessions, post turns, list turns,
  patch document/folder scope.
- Super-admins: list, get, soft-delete sessions across all users for a
  given client.

What it does NOT do (out of scope per plan v10):
- Full-text search.
- GIN indexes.
- Costing / billing.
- Persisted structured data or result rows from the agent.
- User-initiated session deletes (only super-admin DELETE).

## Architecture

### Data model (`bot_*` tables, migration 131)

| Table                   | Purpose                                                                                         |
| ----------------------- | ----------------------------------------------------------------------------------------------- |
| `bot_sessions`          | Owner-scoped chatbot conversations. Soft-deletable via `is_deleted`. Derives `last_terminal_ordinal` and `last_seen_ordinal` from `bot_turns`. |
| `bot_session_documents` | Per-session document scope. Empty set means "all documents for the client" (plan §4 fallback). |
| `bot_session_folders`   | Per-session folder scope. Folder ids preserved in the FastAPI payload; expansion to documents happens in M3 payload assembly. |
| `bot_turns`             | Append-only turn log. Status is one of `in_flight`, `completed`, `errored`, `abandoned`, `session_deleted`. |

Two CHECK constraints govern `bot_turns`:

1. `bot_turns_status_values` enforces the enum set.
2. `bot_turns_status_fields_consistent` enforces the (status, completion,
   error) consistency matrix from plan §4 plus a guard clause for
   unknown statuses. The guard clause `OR status NOT IN (...)` lets
   `bot_turns_status_values` own the rejection of unknown values
   without `_status_fields_consistent` firing first under Postgres's
   alphabetical-by-name CHECK evaluation order. The plan §4 literal
   three-clause matrix is preserved verbatim for documented statuses.

The partial unique index `bot_turns_one_inflight_per_session` enforces
"at most one in-flight turn per session". This is the primary
mechanism for serializing concurrent submissions.

### AddTurn flow (plan §6)

```text
AddTurn(input):
  validate prompt (1 <= rune count <= MaxTurnChars)
  attemptID = uuid.New()

  tx1:
    LockBotSessionForOwner(...)
    AbandonExpiredInflightForOwner(grace = max(2 * RequestTimeoutSeconds, 60s))
    re-derive last_seen_ordinal / last_terminal_ordinal
    GetBotTurn(input.Ordinal)
    if existing exists:
      classify by status; reset on errored/abandoned (prompt must match,
      ordinal must equal last_terminal_ordinal); reject on others.
    else:
      check ordinal matches natural target (last_seen+1 if has_inflight,
      else last_terminal+1); InsertBotTurnPrompt; partial unique index
      may reject -> ErrPriorTurnInFlight.
    if input.Ordinal == 1:
      SetBotSessionTitle(first 120 runes of prompt) where title=''
  end tx1

  Build conversation_history (last RecentTurns completed turns x 2 messages, asc).
  Build chat_scope (stored docs + folder-expanded docs, dedup; folder ids preserved).
  Call FastAPI /agent/chat OUTSIDE any DB tx (network round-trip).

  tx2:
    LockBotSessionForOwner(...)
    if no row -> session was soft-deleted mid-call:
      MarkBotTurnSessionDeleted(...) with attempt_id compare-and-set
      return ErrSessionDeletedDuringCall (HTTP 410)
    if FastAPI returned a typed error:
      FailBotTurn(...) with error JSON containing attempt count
      return mapped agent_* sentinel (HTTP 502)
    if FastAPI succeeded but answer.text is empty/oversize:
      FailBotTurn(...) with agent_invalid_response error JSON
      return ErrAgentInvalidResponse (HTTP 502)
    success path:
      CompleteBotTurn(...) with attempt_id compare-and-set
      summarize metadata (latency, tokens)
      apply state update: null -> preserve prior; non-null -> strip
      cached_results, write if under 64KB cap, else log warn + preserve
  end tx2
```

The two-transaction model is deliberate: no DB connection is held
while calling FastAPI (which can take tens of seconds), and tx2's
attempt_id compare-and-set rejects stale callbacks (`zero rows
affected -> ErrTurnSuperseded`). Plan §6 + §11.

### State management

`bot_sessions.state jsonb` mirrors the agent's session state across
turns. On a successful turn:

- FastAPI `updated_session_state == null` -> preserve prior state. The
  service does NOT call `UpdateBotSessionState`.
- `updated_session_state` non-null -> `StripCachedResults` removes the
  top-level `cached_results` key (a hash-keyed MAP, often the largest
  field). Then `SessionStateOversize` checks the result against
  `chatbot.StateMaxBytes = 64 * 1024`. Under cap -> persist; over cap
  -> log warn and preserve prior state.

The canonical example of a `session_state` payload is
`plans/chatbot.stuff/sample.response.json`. It has 10 top-level keys:
`turn`, `resolved_entities`, `last_scope`, `last_intent`,
`last_query_sql`, `last_query_plan`, `last_result_hash`,
`cached_results`, `last_scope_signature`, `validator_retry_count`.
After strip, the persisted payload has the remaining 9 top-level keys.

### Retry policy (FastAPI client, plan §7)

- Up to 3 attempts per HTTP call.
- Retry on HTTP 429, 500, 502, 503, 504, and transport timeouts.
- Do NOT retry other 4xx, malformed JSON, HTTP 200 with `status:"error"`,
  or HTTP 200 with empty/null `answer.text`.
- Reuse the same `request_id` (`{sessionID}:{ordinal}`) on every
  attempt. The caller mints it once; the client must not regenerate
  per-attempt.
- One parent context enforces total wall time. Per-attempt timeout =
  `cfg.RequestTimeoutSeconds` via `context.WithTimeout` derived from
  the parent.
- Header literal: `X-API-Key: <cfg.APIKey verbatim>`. Production
  values are JSON-encoded strings (`{"AGENT_SERVICE_API_KEY":"value"}`)
  per plan §11.

### Error JSON shape

All `bot_turns.error` writes use the plan §6 canonical shape:

```json
{
  "code": "<sentinel-string>",
  "message": "<human-readable>",
  "attempt": <int>
}
```

The `attempt` field carries the FastAPI client's `AttemptCount` so
operators can distinguish single-failure from exhausted-retry scenarios.

## Configuration reference

Five env vars, plan §8. Validated at startup via
`chatbot.ChatbotConfig.Validate` (called between
`serviceconfig.InitializeConfig` and `cfg.InitializeAuthConfig`).

| Variable                          | Default                | Validation                          | Notes                                                |
| --------------------------------- | ---------------------- | ----------------------------------- | ---------------------------------------------------- |
| `CHATBOT_SERVICE_URL`             | (required, notEmpty)   | env-loader rejects empty            | FastAPI base URL.                                    |
| `CHATBOT_API_KEY`                 | (required, notEmpty)   | env-loader rejects empty            | Sent verbatim as `X-API-Key` header.                 |
| `CHATBOT_REQUEST_TIMEOUT_SECONDS` | 60                     | `>= 1`                              | Per-attempt FastAPI timeout.                         |
| `CHATBOT_MAX_TURN_CHARS`          | 2000                   | `1 <= x <= 65536`                   | Caps prompt and `answer.text` rune count. Upper bound (`chatbot.MaxTurnCharsCeiling`) mirrors the OpenAPI `prompt.maxLength`. |
| `CHATBOT_RECENT_TURNS`            | 5                      | `1 <= x <= 5`                       | Conversation-history length and session-list preview cap. |

Derived values (constants/helpers, not env vars):

- `chatbot.StateMaxBytes = 64 * 1024` — session-state cap (plan §8).
- `chatbot.MaxTurnCharsCeiling = 65536` — upper bound for `CHATBOT_MAX_TURN_CHARS`, locked to the OpenAPI `BotTurnAddRequest.prompt.maxLength`. Validate() rejects any value above this.
- `chatbot.ChatbotConfig.InflightGraceSeconds() = 2 * RequestTimeoutSeconds`
  per plan §8 literal. The service uses
  `effectiveGraceSeconds(cfg) = max(2 * RequestTimeoutSeconds, 60)` as
  the actual abandonment threshold to keep small `RequestTimeoutSeconds`
  in test fixtures from prematurely abandoning legitimately in-flight
  turns. Production `RequestTimeoutSeconds=60` already exceeds the
  60-second floor.

## API reference

### User routes (`/client/{clientId}/...`)

All routes are singular `/client/...` per plan §3 so Permit.io's
existing `client_user` policy applies.

| Method | Path                                                              | OperationId               | Description                                              |
| ------ | ----------------------------------------------------------------- | ------------------------- | -------------------------------------------------------- |
| POST   | `/client/{clientId}/bot/sessions`                                 | `createBotSession`        | Create a new owner-scoped session.                       |
| GET    | `/client/{clientId}/bot/sessions`                                 | `listBotSessions`         | List the caller's sessions; bounds: `limit in [1,200]`, `offset >= 0`. |
| GET    | `/client/{clientId}/bot/sessions/{sessionId}`                     | `getBotSession`           | Read one owner-scoped session.                           |
| PATCH  | `/client/{clientId}/bot/sessions/{sessionId}/scope`               | `patchBotSessionScope`    | Add/remove documents and folders from session scope.     |
| POST   | `/client/{clientId}/bot/sessions/{sessionId}/turns`               | `createBotTurn`           | Submit a new turn. Plan §6 algorithm.                    |
| GET    | `/client/{clientId}/bot/sessions/{sessionId}/turns`               | `listBotTurns`            | List all turns for an owner-scoped session, ASC.         |
| GET    | `/client/{clientId}/documents/{documentId}/schema`                | `getDocumentSchema`       | Return the document's bound JSON Schema or `null` (omitted in JSON when unbound). |
| GET    | `/client/{clientId}/documents/{documentId}/all-metadata`          | `getDocumentAllMetadata`  | Bundle of (document, labels, customMetadata) for one doc. |

### Super-admin routes

| Method | Path                                                  | OperationId                       | Description                                  |
| ------ | ----------------------------------------------------- | --------------------------------- | -------------------------------------------- |
| GET    | `/super-admin/bot/sessions?clientId=...`              | `listBotSessionsAsSuperAdmin`     | List all sessions for a client (cross-user). |
| GET    | `/super-admin/bot/sessions/{sessionId}?clientId=...`  | `getBotSessionAsSuperAdmin`       | Read one session by id (cross-user).         |
| DELETE | `/super-admin/bot/sessions/{sessionId}?clientId=...`  | `deleteBotSessionAsSuperAdmin`    | Soft-delete; transitions any in-flight turns to `session_deleted`. |

### `BotSessionResponse.lastTurn`

`lastTurn` returns `last_terminal_ordinal` (plan §4 derived note), NOT
`last_seen_ordinal`. The latter is computed from `bot_turns` but used
only internally by `AddTurn` for ordinal classification. Clients should
display `lastTurn` as "the last completed turn this session has".

### `DocumentSchemaResponse.schema`

When the document has no `custom_schema_id` binding, the response is
HTTP 200 with the `schema` field omitted (the field is `*map[...]` with
`omitempty`, so JSON does NOT emit `"schema": null`; the key is
absent). Clients should treat both "key absent" and "key present and
null" as the unbound case.

### `BotLabelRecord` vs `LabelRecord`

The all-metadata endpoint uses `BotLabelRecord` (plain-string
`appliedBy`) instead of the existing `LabelRecord` (email-validated
`appliedBy`). Both shapes wrap the same `documentLabels.appliedBy
varchar` column; the divergence exists because the email-regex Marshal
validator on `LabelRecord` rejects non-email seed values that pre-date
the email-typing convention. Long-term cleanup is to relax
`LabelRecord.AppliedBy` to plain string. Until then, the chatbot
endpoints emit `BotLabelRecord`.

### HTTP status mapping

| Status | Sentinels / cases                                                                            |
| ------ | -------------------------------------------------------------------------------------------- |
| 400    | `prompt_empty`, `prompt_too_long`, `add_remove_conflict`, `document_cross_client`, `folder_cross_client`, `clientId is required` (super-admin), invalid `limit`/`offset` |
| 401    | Missing Cognito subject                                                                      |
| 404    | Cross-client / cross-user / soft-deleted session; document not found in client scope          |
| 409    | `turn_in_flight`, `prior_turn_in_flight`, `already_complete`, `turn_session_deleted`, `ordinal_out_of_range`, `turn_superseded` (legacy: `prompt_mismatch` — service-layer assertion only; not reachable from the wire as of 2026-05-07) |
| 410    | `session_deleted_during_call`                                                                |
| 502    | `agent_application_error`, `agent_missing_answer`, `agent_invalid_response`, `agent_transport_error` |
| 500    | Unexpected backend errors                                                                    |

`mapBotTurnError` dispatch order matters because `ErrAlreadyComplete`
chains to `ErrOrdinalOutOfRange` via a custom `Is()` method. The
sentinels are matched in declaration order; `ErrAlreadyComplete` must
be checked before `ErrOrdinalOutOfRange` so the message string
"already_complete" is returned in the HTTP retry-of-completed scenario.
The chain exists so plan §11's concurrency-test loss set
(`ErrTurnInFlight | ErrPriorTurnInFlight | ErrOrdinalOutOfRange`) covers
the case where a concurrent submitter's ordinal lands on the row the
winner just completed.

### Test-pattern note

Test assertions on HTTP error message bodies use substring matches.
`turn_in_flight` matches both `turn_in_flight` and `prior_turn_in_flight`
(the latter contains the former). M4's `TestAddTurn_SameOrdinalStillInflight_409`
exploits this: the test name says "same ordinal", but the algorithm
returns `prior_turn_in_flight` because the natural target is `last_seen+1`
when has_inflight; the substring assertion still passes. Both 409s
have the same operator semantics ("can't submit while a turn is in
flight").

## Operator runbook

### Failed-turn classifications

When a `bot_turns` row has a non-`completed` status, operators consult
`bot_turns.error` for the canonical error JSON and pick the recovery
path below.

#### `errored`

The FastAPI agent returned a typed error or the response failed
validation. The error JSON includes `code`, `message`, and `attempt`.
Common codes:

| Code                       | Meaning                                                                  | Recovery                                                                |
| -------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------------- |
| `agent_application_error`  | FastAPI returned 200 with `status:"error"`. Code/message in the inner `error` field. | Client may resubmit the same prompt at the same ordinal; the algorithm reset path runs. |
| `agent_missing_answer`     | FastAPI returned 200 with empty/null/whitespace `answer.text`.            | Same as above. May indicate a prompt the agent cannot answer; client UX should suggest rephrasing. |
| `agent_invalid_response`   | FastAPI returned 200 but body failed structural validation (malformed JSON or oversize completion). | Same as above. May indicate an upstream agent change; check upstream logs. |
| `agent_transport_error`    | FastAPI exhausted its 3-attempt retry budget on transport-level failures (429/5xx/timeouts). | Same as above. May indicate FastAPI overload; check upstream health and rate-limit. |

To retry: client POSTs to the same `/turns` route with the same prompt.
The handler picks the `errored` row's ordinal via prompt-match;
`AddTurn` runs the existing-row reset path; a new `attempt_id` is
minted; tx2 records the new outcome.

If the client posts a *different* prompt instead of retrying, the
handler advances to `last_terminal_ordinal+1` (a fresh `in_flight` at
ordinal N+1) rather than rejecting with 409 `prompt_mismatch`. The
errored row at ordinal N stays in history. See
`plans/chatbot.stuff/fix.chat.bugs.1.md` §3 Option A for the rationale.

#### `abandoned`

The grace window expired without a terminal status. Most often this
means the queryAPI process died mid-call and the next caller's
`AbandonExpiredInflightForOwner` swept the stale row. The error JSON
carries:

```json
{"code":"turn_abandoned_grace_expired","message":"no terminal status reached within grace window"}
```

Recovery is identical to `errored`: client resubmits the same prompt.
The reset path applies.

#### `session_deleted`

A super-admin DELETEd the session while the turn was in flight, OR an
older retry path hit the session-deleted-during-call branch. The error
JSON carries:

```json
{"code":"session_deleted_by_admin","message":"session deleted while turn in flight","attempt":<int>}
```

OR (from the AddTurn tx2 path):

```json
{"code":"session_deleted_during_call","message":"session was soft-deleted while FastAPI call was in flight","attempt":<int>}
```

Recovery: NONE. The session is permanent. The client must create a new
session and resubmit. The historical `session_deleted` row remains for
audit; LIST turns surfaces it with the original prompt and the error
JSON.

### Performance notes

`decorateSessionList` (the M2 list-sessions handler path) does N+1
scope reads: one `loadScope` round-trip per session in the list.
Default `limit=20`, hard cap `limit=200`. Under default load the
overhead is negligible; if a deployment surfaces latency on the list
endpoint, consider a single batch read of all scope rows for the
listed session ids. Plan v10 marked this as acceptable for v1.

### Rate-limit overrides

`RATE_LIMIT_ENDPOINT_OVERRIDES` is a JSON object keyed by
`METHOD /path` using **Echo's path template form** (`:clientId`,
`:sessionId`), NOT the OpenAPI `{clientId}` form. The middleware uses
`c.Path()` which returns Echo's registered path string; an override
keyed by `{clientId}` will silently fail to match.

Example for the chatbot turn endpoint:

```bash
RATE_LIMIT_ENDPOINT_OVERRIDES='{"POST /client/:clientId/bot/sessions/:sessionId/turns":{"global_rate":50,"global_burst":100,"rate":2,"burst":4}}'
```

**M6 operator-facing behavior change**: setting an override here NOW
throttles the actual request rate (per-(IP, route) budget). Before M6,
overrides only affected the `RateLimit` and `Retry-After` headers; the
Allow() decision used the default rate/burst. Plan §10 M6 + plan §11.

## References

- Plan: `plans/chatbot_plan_codex.v10.md` (design rationale and
  authoritative spec).
- FastAPI swagger: `plans/chatbot.stuff/openapi.chatbot.swagger.json`.
- Sample `session_state` fixture: `plans/chatbot.stuff/sample.response.json`.
- Cloud FastAPI deployment (dev): see plan §11 for the URL and the
  literal `X-API-Key` value.

## Known warts

These are not bugs; they are conscious tradeoffs documented for
maintainers.

1. **CHECK guard clause** in `bot_turns_status_fields_consistent`. The
   plan §4 literal three-clause matrix is preserved verbatim; the
   trailing `OR status NOT IN (...)` is an evaluation-order guard so
   `bot_turns_status_values` owns the rejection of unknown enum values.
2. **`ErrAlreadyComplete` chain to `ErrOrdinalOutOfRange`** via custom
   `Is()` so plan §11's concurrency-test loss set covers the case
   where a loser's ordinal lands on the winner's just-completed row.
3. **`BotLabelRecord` vs `LabelRecord`** divergence (above).
4. **`schema=null`** uses field omission (omitempty) rather than
   explicit JSON null. Clients should accept both.
5. **`M5DocumentID` alias** is cosmetically distinct from `DocumentID`
   in the generated types. Both are `openapi_types.UUID`. Cosmetic only.
6. **Rate-limit override `:param` template** form, not OpenAPI
   `{param}` form (above).
7. **N+1 scope reads** in session list (above).
8. **Grace-window floor of 60s** in `effectiveGraceSeconds`; the plan
   §8 literal `2 * RequestTimeoutSeconds` is preserved on the chatbot
   config method, but the service applies the floor at the call site
   so test fixtures with `RequestTimeoutSeconds=1` (2s grace) do not
   prematurely abandon legitimately in-flight turns.