query-orchestration/docs/ai.generated/12-chatbot-guide.md

Chatbot Backend Guide

This guide covers the chatbot backend that queryAPI exposes for the Aaria Agent Service integration. It is the operator entry point; design rationale lives in plans/chatbot_plan_codex.v10.md.

Overview

queryAPI exposes a synchronous HTTP facade over the Aaria FastAPI agent service. Client UIs talk to queryAPI; queryAPI calls FastAPI and persists the conversation in Postgres so future turns inherit the prior context.

Who calls it:

• Client UIs as client_user: create sessions, post turns, list turns, patch document/folder scope.
• Super-admins: list, get, soft-delete sessions across all users for a given client.

What it does NOT do (out of scope per plan v10):

• Full-text search.
• GIN indexes.
• Costing / billing.
• Persisted structured data or result rows from the agent.
• User-initiated session deletes (only super-admin DELETE).

Architecture

Data model (bot_* tables, migration 131)

Table	Purpose
bot_sessions	Owner-scoped chatbot conversations. Soft-deletable via is_deleted. Derives last_terminal_ordinal and last_seen_ordinal from bot_turns.
bot_session_documents	Per-session document scope. Empty set means "all documents for the client" (plan §4 fallback).
bot_session_folders	Per-session folder scope. Folder ids preserved in the FastAPI payload; expansion to documents happens in M3 payload assembly.
bot_turns	Append-only turn log. Status is one of in_flight, completed, errored, abandoned, session_deleted.

Two CHECK constraints govern bot_turns:

1. bot_turns_status_values enforces the enum set.
2. bot_turns_status_fields_consistent enforces the (status, completion, error) consistency matrix from plan §4 plus a guard clause for unknown statuses. The guard clause OR status NOT IN (...) lets bot_turns_status_values own the rejection of unknown values without _status_fields_consistent firing first under Postgres's alphabetical-by-name CHECK evaluation order. The plan §4 literal three-clause matrix is preserved verbatim for documented statuses.

The partial unique index bot_turns_one_inflight_per_session enforces "at most one in-flight turn per session". This is the primary mechanism for serializing concurrent submissions.

AddTurn flow (plan §6)

AddTurn(input):
  validate prompt (1 <= rune count <= MaxTurnChars)
  attemptID = uuid.New()

  tx1:
    LockBotSessionForOwner(...)
    AbandonExpiredInflightForOwner(grace = max(2 * RequestTimeoutSeconds, 60s))
    re-derive last_seen_ordinal / last_terminal_ordinal
    GetBotTurn(input.Ordinal)
    if existing exists:
      classify by status; reset on errored/abandoned (prompt must match,
      ordinal must equal last_terminal_ordinal); reject on others.
    else:
      check ordinal matches natural target (last_seen+1 if has_inflight,
      else last_terminal+1); InsertBotTurnPrompt; partial unique index
      may reject -> ErrPriorTurnInFlight.
    if input.Ordinal == 1:
      SetBotSessionTitle(first 120 runes of prompt) where title=''
  end tx1

  Build conversation_history (last RecentTurns completed turns x 2 messages, asc).
  Build chat_scope (stored docs + folder-expanded docs, dedup; folder ids preserved).
  Call FastAPI /agent/chat OUTSIDE any DB tx (network round-trip).

  tx2:
    LockBotSessionForOwner(...)
    if no row -> session was soft-deleted mid-call:
      MarkBotTurnSessionDeleted(...) with attempt_id compare-and-set
      return ErrSessionDeletedDuringCall (HTTP 410)
    if FastAPI returned a typed error:
      FailBotTurn(...) with error JSON containing attempt count
      return mapped agent_* sentinel (HTTP 502)
    if FastAPI succeeded but answer.text is empty/oversize:
      FailBotTurn(...) with agent_invalid_response error JSON
      return ErrAgentInvalidResponse (HTTP 502)
    success path:
      CompleteBotTurn(...) with attempt_id compare-and-set
      summarize metadata (latency, tokens)
      apply state update: null -> preserve prior; non-null -> strip
      cached_results, write if under 64KB cap, else log warn + preserve
  end tx2

The two-transaction model is deliberate: no DB connection is held while calling FastAPI (which can take tens of seconds), and tx2's attempt_id compare-and-set rejects stale callbacks (zero rows affected -> ErrTurnSuperseded). Plan §6 + §11.

State management

bot_sessions.state jsonb mirrors the agent's session state across turns. On a successful turn:

• FastAPI updated_session_state == null -> preserve prior state. The service does NOT call UpdateBotSessionState.
• updated_session_state non-null -> StripCachedResults removes the top-level cached_results key (a hash-keyed MAP, often the largest field). Then SessionStateOversize checks the result against chatbot.StateMaxBytes = 64 * 1024. Under cap -> persist; over cap -> log warn and preserve prior state.

The canonical example of a session_state payload is plans/chatbot.stuff/sample.response.json. It has 10 top-level keys: turn, resolved_entities, last_scope, last_intent, last_query_sql, last_query_plan, last_result_hash, cached_results, last_scope_signature, validator_retry_count. After strip, the persisted payload has the remaining 9 top-level keys.

Retry policy (FastAPI client, plan §7)

• Up to 3 attempts per HTTP call.
• Retry on HTTP 429, 500, 502, 503, 504, and transport timeouts.
• Do NOT retry other 4xx, malformed JSON, HTTP 200 with status:"error", or HTTP 200 with empty/null answer.text.
• Reuse the same request_id ({sessionID}:{ordinal}) on every attempt. The caller mints it once; the client must not regenerate per-attempt.
• One parent context enforces total wall time. Per-attempt timeout = cfg.RequestTimeoutSeconds via context.WithTimeout derived from the parent.
• Header literal: X-API-Key: <cfg.APIKey verbatim>. Production values are JSON-encoded strings ({"AGENT_SERVICE_API_KEY":"value"}) per plan §11.

Error JSON shape

All bot_turns.error writes use the plan §6 canonical shape:

{
  "code": "<sentinel-string>",
  "message": "<human-readable>",
  "attempt": <int>
}

The attempt field carries the FastAPI client's AttemptCount so operators can distinguish single-failure from exhausted-retry scenarios.

Configuration reference

Five env vars, plan §8. Validated at startup via chatbot.ChatbotConfig.Validate (called between serviceconfig.InitializeConfig and cfg.InitializeAuthConfig).

Variable	Default	Validation	Notes
CHATBOT_SERVICE_URL	(required, notEmpty)	env-loader rejects empty	FastAPI base URL.
CHATBOT_API_KEY	(required, notEmpty)	env-loader rejects empty	Sent verbatim as X-API-Key header.
CHATBOT_REQUEST_TIMEOUT_SECONDS	60	>= 1	Per-attempt FastAPI timeout.
CHATBOT_MAX_TURN_CHARS	2000	1 <= x <= 65536	Caps prompt and answer.text rune count. Upper bound (chatbot.MaxTurnCharsCeiling) mirrors the OpenAPI prompt.maxLength.
CHATBOT_RECENT_TURNS	5	1 <= x <= 5	Conversation-history length and session-list preview cap.

Derived values (constants/helpers, not env vars):

• chatbot.StateMaxBytes = 64 * 1024 — session-state cap (plan §8).
• chatbot.MaxTurnCharsCeiling = 65536 — upper bound for CHATBOT_MAX_TURN_CHARS, locked to the OpenAPI BotTurnAddRequest.prompt.maxLength. Validate() rejects any value above this.
• chatbot.ChatbotConfig.InflightGraceSeconds() = 2 * RequestTimeoutSeconds per plan §8 literal. The service uses effectiveGraceSeconds(cfg) = max(2 * RequestTimeoutSeconds, 60) as the actual abandonment threshold to keep small RequestTimeoutSeconds in test fixtures from prematurely abandoning legitimately in-flight turns. Production RequestTimeoutSeconds=60 already exceeds the 60-second floor.

API reference

User routes (/client/{clientId}/...)

All routes are singular /client/... per plan §3 so Permit.io's existing client_user policy applies.

Method	Path	OperationId	Description
POST	/client/{clientId}/bot/sessions	createBotSession	Create a new owner-scoped session.
GET	/client/{clientId}/bot/sessions	listBotSessions	List the caller's sessions; bounds: limit in [1,200], offset >= 0.
GET	/client/{clientId}/bot/sessions/{sessionId}	getBotSession	Read one owner-scoped session.
PATCH	/client/{clientId}/bot/sessions/{sessionId}/scope	patchBotSessionScope	Add/remove documents and folders from session scope.
POST	/client/{clientId}/bot/sessions/{sessionId}/turns	createBotTurn	Submit a new turn. Plan §6 algorithm.
GET	/client/{clientId}/bot/sessions/{sessionId}/turns	listBotTurns	List all turns for an owner-scoped session, ASC.
GET	/client/{clientId}/documents/{documentId}/schema	getDocumentSchema	Return the document's bound JSON Schema or null (omitted in JSON when unbound).
GET	/client/{clientId}/documents/{documentId}/all-metadata	getDocumentAllMetadata	Bundle of (document, labels, customMetadata) for one doc.

Super-admin routes

Method	Path	OperationId	Description
GET	/super-admin/bot/sessions?clientId=...	listBotSessionsAsSuperAdmin	List all sessions for a client (cross-user).
GET	/super-admin/bot/sessions/{sessionId}?clientId=...	getBotSessionAsSuperAdmin	Read one session by id (cross-user).
DELETE	/super-admin/bot/sessions/{sessionId}?clientId=...	deleteBotSessionAsSuperAdmin	Soft-delete; transitions any in-flight turns to session_deleted.

BotSessionResponse.lastTurn

lastTurn returns last_terminal_ordinal (plan §4 derived note), NOT last_seen_ordinal. The latter is computed from bot_turns but used only internally by AddTurn for ordinal classification. Clients should display lastTurn as "the last completed turn this session has".

DocumentSchemaResponse.schema

When the document has no custom_schema_id binding, the response is HTTP 200 with the schema field omitted (the field is *map[...] with omitempty, so JSON does NOT emit "schema": null; the key is absent). Clients should treat both "key absent" and "key present and null" as the unbound case.

BotLabelRecord vs LabelRecord

The all-metadata endpoint uses BotLabelRecord (plain-string appliedBy) instead of the existing LabelRecord (email-validated appliedBy). Both shapes wrap the same documentLabels.appliedBy varchar column; the divergence exists because the email-regex Marshal validator on LabelRecord rejects non-email seed values that pre-date the email-typing convention. Long-term cleanup is to relax LabelRecord.AppliedBy to plain string. Until then, the chatbot endpoints emit BotLabelRecord.

HTTP status mapping

Status	Sentinels / cases
400	prompt_empty, prompt_too_long, add_remove_conflict, document_cross_client, folder_cross_client, clientId is required (super-admin), invalid limit/offset
401	Missing Cognito subject
404	Cross-client / cross-user / soft-deleted session; document not found in client scope
409	turn_in_flight, prior_turn_in_flight, already_complete, turn_session_deleted, ordinal_out_of_range, turn_superseded (legacy: prompt_mismatch — service-layer assertion only; not reachable from the wire as of 2026-05-07)
410	session_deleted_during_call
502	agent_application_error, agent_missing_answer, agent_invalid_response, agent_transport_error
500	Unexpected backend errors

mapBotTurnError dispatch order matters because ErrAlreadyComplete chains to ErrOrdinalOutOfRange via a custom Is() method. The sentinels are matched in declaration order; ErrAlreadyComplete must be checked before ErrOrdinalOutOfRange so the message string "already_complete" is returned in the HTTP retry-of-completed scenario. The chain exists so plan §11's concurrency-test loss set (ErrTurnInFlight | ErrPriorTurnInFlight | ErrOrdinalOutOfRange) covers the case where a concurrent submitter's ordinal lands on the row the winner just completed.

Test-pattern note

Test assertions on HTTP error message bodies use substring matches. turn_in_flight matches both turn_in_flight and prior_turn_in_flight (the latter contains the former). M4's TestAddTurn_SameOrdinalStillInflight_409 exploits this: the test name says "same ordinal", but the algorithm returns prior_turn_in_flight because the natural target is last_seen+1 when has_inflight; the substring assertion still passes. Both 409s have the same operator semantics ("can't submit while a turn is in flight").

Operator runbook

Failed-turn classifications

When a bot_turns row has a non-completed status, operators consult bot_turns.error for the canonical error JSON and pick the recovery path below.

errored

The FastAPI agent returned a typed error or the response failed validation. The error JSON includes code, message, and attempt. Common codes:

Code	Meaning	Recovery
agent_application_error	FastAPI returned 200 with status:"error". Code/message in the inner error field.	Client may resubmit the same prompt at the same ordinal; the algorithm reset path runs.
agent_missing_answer	FastAPI returned 200 with empty/null/whitespace answer.text.	Same as above. May indicate a prompt the agent cannot answer; client UX should suggest rephrasing.
agent_invalid_response	FastAPI returned 200 but body failed structural validation (malformed JSON or oversize completion).	Same as above. May indicate an upstream agent change; check upstream logs.
agent_transport_error	FastAPI exhausted its 3-attempt retry budget on transport-level failures (429/5xx/timeouts).	Same as above. May indicate FastAPI overload; check upstream health and rate-limit.

To retry: client POSTs to the same /turns route with the same prompt. The handler picks the errored row's ordinal via prompt-match; AddTurn runs the existing-row reset path; a new attempt_id is minted; tx2 records the new outcome.

If the client posts a different prompt instead of retrying, the handler advances to last_terminal_ordinal+1 (a fresh in_flight at ordinal N+1) rather than rejecting with 409 prompt_mismatch. The errored row at ordinal N stays in history. See plans/chatbot.stuff/fix.chat.bugs.1.md §3 Option A for the rationale.

abandoned

The grace window expired without a terminal status. Most often this means the queryAPI process died mid-call and the next caller's AbandonExpiredInflightForOwner swept the stale row. The error JSON carries:

{"code":"turn_abandoned_grace_expired","message":"no terminal status reached within grace window"}

Recovery is identical to errored: client resubmits the same prompt. The reset path applies.

session_deleted

A super-admin DELETEd the session while the turn was in flight, OR an older retry path hit the session-deleted-during-call branch. The error JSON carries:

{"code":"session_deleted_by_admin","message":"session deleted while turn in flight","attempt":<int>}

OR (from the AddTurn tx2 path):

{"code":"session_deleted_during_call","message":"session was soft-deleted while FastAPI call was in flight","attempt":<int>}

Recovery: NONE. The session is permanent. The client must create a new session and resubmit. The historical session_deleted row remains for audit; LIST turns surfaces it with the original prompt and the error JSON.

Performance notes

decorateSessionList (the M2 list-sessions handler path) does N+1 scope reads: one loadScope round-trip per session in the list. Default limit=20, hard cap limit=200. Under default load the overhead is negligible; if a deployment surfaces latency on the list endpoint, consider a single batch read of all scope rows for the listed session ids. Plan v10 marked this as acceptable for v1.

Rate-limit overrides

RATE_LIMIT_ENDPOINT_OVERRIDES is a JSON object keyed by METHOD /path using Echo's path template form (:clientId, :sessionId), NOT the OpenAPI {clientId} form. The middleware uses c.Path() which returns Echo's registered path string; an override keyed by {clientId} will silently fail to match.

Example for the chatbot turn endpoint:

RATE_LIMIT_ENDPOINT_OVERRIDES='{"POST /client/:clientId/bot/sessions/:sessionId/turns":{"global_rate":50,"global_burst":100,"rate":2,"burst":4}}'

M6 operator-facing behavior change: setting an override here NOW throttles the actual request rate (per-(IP, route) budget). Before M6, overrides only affected the RateLimit and Retry-After headers; the Allow() decision used the default rate/burst. Plan §10 M6 + plan §11.

References

• Plan: plans/chatbot_plan_codex.v10.md (design rationale and authoritative spec).
• FastAPI swagger: plans/chatbot.stuff/openapi.chatbot.swagger.json.
• Sample session_state fixture: plans/chatbot.stuff/sample.response.json.
• Cloud FastAPI deployment (dev): see plan §11 for the URL and the literal X-API-Key value.

Known warts

These are not bugs; they are conscious tradeoffs documented for maintainers.

1. CHECK guard clause in bot_turns_status_fields_consistent. The plan §4 literal three-clause matrix is preserved verbatim; the trailing OR status NOT IN (...) is an evaluation-order guard so bot_turns_status_values owns the rejection of unknown enum values.
2. ErrAlreadyComplete chain to ErrOrdinalOutOfRange via custom Is() so plan §11's concurrency-test loss set covers the case where a loser's ordinal lands on the winner's just-completed row.
3. BotLabelRecord vs LabelRecord divergence (above).
4. schema=null uses field omission (omitempty) rather than explicit JSON null. Clients should accept both.
5. M5DocumentID alias is cosmetically distinct from DocumentID in the generated types. Both are openapi_types.UUID. Cosmetic only.
6. Rate-limit override :param template form, not OpenAPI {param} form (above).
7. N+1 scope reads in session list (above).
8. Grace-window floor of 60s in effectiveGraceSeconds; the plan §8 literal 2 * RequestTimeoutSeconds is preserved on the chatbot config method, but the service applies the floor at the call site so test fixtures with RequestTimeoutSeconds=1 (2s grace) do not prematurely abandon legitimately in-flight turns.