query-orchestration/docs/ai.generated/02-service-architecture.md

Service Architecture

Service Catalog

Active Services

queryAPI - Main HTTP API Service

• Port: 8080
• Type: HTTP REST API
• Responsibilities:
  • Client, document, folder, label, collector, export, field extraction, admin, and EULA endpoints
  • Authentication/authorization middleware integration (Cognito + Permit.io)
  • Batch upload background processing coordination

storeEventRunner - S3 Event Processor

• Port: 8081
• Type: Queue consumer
• Input: S3 event notification messages
• Output: DOCINIT queue messages

docInitRunner - Document Initialization

• Port: 8082
• Type: Queue consumer
• Input: DOCINIT queue messages
• Output: DOCSYNC queue messages
• Responsibilities: document registration and deduplication by client/hash

docSyncRunner - Document Sync Gate

• Port: 8083
• Type: Queue consumer
• Input: DOCSYNC queue messages
• Output: DOCCLEAN queue messages
• Responsibilities: enforce client sync eligibility before processing

docCleanRunner - Document Validation/Cleaning

• Port: 8084
• Type: Queue consumer
• Input: DOCCLEAN queue messages
• Output: none (pipeline currently ends here)
• Responsibilities: content validation, clean record creation, idempotent processing

clientSyncRunner - Client Reprocessing Trigger

• Port: 8089
• Type: Queue consumer
• Input: CLIENTSYNC queue messages
• Output: DOCSYNC queue messages (one per client document)

Deprecated Compatibility Services

The following services are retained as deployment compatibility placeholders and currently run health-only behavior:

• docTextRunner (8085)
• querySyncRunner (8087)
• queryRunner (8088)
• queryVersionSyncRunner (8090)

Data Flow Architecture

Active Document Processing Pipeline

sequenceDiagram
    participant Client
    participant S3
    participant Store as storeEventRunner
    participant Init as docInitRunner
    participant Sync as docSyncRunner
    participant Clean as docCleanRunner

    Client->>S3: Upload Document
    S3->>Store: S3 Event Notification
    Store->>Init: DOCINIT Queue
    Init->>Sync: DOCSYNC Queue
    Sync->>Clean: DOCCLEAN Queue

Client Synchronization Flow

sequenceDiagram
    participant Admin
    participant API as queryAPI
    participant CS as clientSyncRunner
    participant Sync as docSyncRunner

    Admin->>API: Trigger client sync
    API->>CS: CLIENTSYNC Queue
    CS->>Sync: DOCSYNC Queue (per client document)

Communication Patterns

Inter-Service Messaging

• Services use AWS SQS for asynchronous decoupling
• Queue consumers are idempotent and retry-friendly

Queue Architecture (Active)

Queue Name	Producer	Consumer	Message Format
DOCINIT	storeEventRunner	docInitRunner	{bucket, key, hash}
DOCSYNC	docInitRunner, clientSyncRunner	docSyncRunner	{document_id}
DOCCLEAN	docSyncRunner	docCleanRunner	{document_id}
CLIENTSYNC	queryAPI	clientSyncRunner	{client_id}

Queue Architecture (Deprecated Compatibility)

Queue Name	Notes
DOCTEXT	Kept in some deployments for compatibility; not part of active pipeline
QUERYSYNC / QUERY / QUERYVERSIONSYNC	Legacy query pipeline queues; query subsystem removed