query-orchestration/docs/ai.generated/service.details.md

Service Processing Details

This document describes current service behavior in the processing pipeline.

Active Pipeline Services

1. storeEventRunner

Purpose: Entry point for upload events.

Code locations:

• cmd/storeEventRunner/
• api/storeEventRunner/
• internal/document/store/

Behavior:

• Consumes S3 event notifications
• Validates supported event/object patterns
• Extracts bucket/key/hash metadata
• Publishes to DOCINIT

Output: {bucket, key, hash} to DOCINIT.

---

2. docInitRunner

Purpose: Document registration and deduplication.

Code locations:

• cmd/docInitRunner/
• api/docInitRunner/
• internal/document/init/

Behavior:

• Parses upload metadata from S3 key
• Deduplicates by client/hash
• Creates document and entry records for new documents
• Publishes new documents to DOCSYNC

Output: {document_id} to DOCSYNC for new documents.

---

3. docSyncRunner

Purpose: Sync eligibility gate.

Code locations:

• cmd/docSyncRunner/
• api/docSyncRunner/
• internal/document/sync/

Behavior:

• Loads document + client state
• Checks client sync status
• Publishes eligible documents to DOCCLEAN

Output: {document_id} to DOCCLEAN when eligible.

---

4. docCleanRunner

Purpose: Document validation/cleaning stage.

Code locations:

• cmd/docCleanRunner/
• api/docCleanRunner/
• internal/document/clean/

Behavior:

• Idempotent clean processing
• Validates MIME/content constraints and S3-backed inputs
• Stores clean version entries

Output: none. The active pipeline currently ends after clean.

---

5. clientSyncRunner

Purpose: Re-queue all documents for a client.

Code locations:

• cmd/clientSyncRunner/
• api/clientSyncRunner/
• internal/client/sync/

Behavior:

• Consumes CLIENTSYNC
• Enumerates client documents
• Publishes each to DOCSYNC

Output: {document_id} to DOCSYNC for each client document.

---

6. batch folder cleanup

Purpose: Remove empty folders auto-created by legacy ZIP batch extraction when all file outcomes are terminal.

Code locations:

• internal/document/batch/foldercleanup/
• internal/server/api/batch_worker.go
• internal/database/queries/batch.sql

Behavior:

• Locks the batch row before finalizing cleanup
• Requires terminal batch status plus persisted outcome evidence
• Detaches rejected current-batch documents and upload rows from candidate folders
• Deletes only safe empty auto-created folders
• Retries ready cleanup from the QueryAPI background worker

Output: updates batch_uploads.folder_cleanup_completed_at; may null rejected documents.folderId and documentUploads.folder_id; may delete safe candidate folders.

Deprecated Compatibility Services

These services are retained for deployment compatibility and are currently health-only/no-op:

• cmd/docTextRunner/
• cmd/querySyncRunner/
• cmd/queryRunner/
• cmd/queryVersionSyncRunner/

Active Data Flow Summary

S3 Upload Event
  -> storeEventRunner
  -> docInitRunner
  -> docSyncRunner
  -> docCleanRunner
  -> batch folder cleanup (for legacy ZIP batches)

Client Sync Trigger
  -> clientSyncRunner
  -> docSyncRunner
  -> docCleanRunner