query-orchestration/plans/mutable.metadata.review.combo.v3.codex.md

Mutable Metadata Plan v3 Review (Codex)

A review of plans/mutable.metadata.plan.combo.v3.md and plans/mutable.metadata.plan.combo.v3.tracking.md since they are to be used as a pair to implement the plans/mutable.metadata.plan.combo.v3.md plan.

I reviewed the current legacy field-extraction flow first so the plan review is anchored to real behavior:

• api/queryAPI/fieldextractions.go:20-69 binds the request, calls FieldExtraction.CreateFieldExtraction, and currently maps any create error to HTTP 500.
• internal/fieldextraction/service.go:27-100 creates the extraction row, then locks existing version rows, computes max(version)+1, and inserts the version row.
• Authorization today is checked as (user, action, first-path-segment resource) with no document/client object passed into Permit.io: internal/cognitoauth/middleware.go:84-110, internal/cognitoauth/permitio.go:79-89.

Findings

1. Blocking: the tracking doc’s migration numbers collide with live migrations already in the repo. References:

   • plans/mutable.metadata.plan.combo.v3.tracking.md:48-75
   • internal/database/migrations: existing 00000000000120_* through 00000000000123_* are already present (00000000000120_remove_text_extraction, 00000000000121_create_eula_tables, 00000000000122_make_eula_effective_date_nullable, 00000000000123_create_ui_settings) Impact:
   • Phase 1 cannot start as written.
   • Any implementation that follows the tracking doc literally will create filename collisions and broken migration ordering. Recommendation:
   • Renumber all new mutable-metadata migrations to the next free sequence and update every tracking reference before coding.

2. Blocking: the plan grants /custom-metadata read/write to client_user, but the current auth model is not tenant-scoped. References:

   • Plan grants: plans/mutable.metadata.plan.combo.v3.md:730-780
   • Current auth check: internal/cognitoauth/middleware.go:105-110, internal/cognitoauth/permitio.go:79-89
   • Current documented limitation: cmd/auth_related/permit.setup/permit_policies.yaml:155-184 Impact:
   • custom-metadata:get/post would be authorized purely by resource/action on the first path segment.
   • Without new application-level document/client ownership checks, a client_user or user_admin who knows another document UUID can read/write that document’s custom metadata across clients. Recommendation:
   • Resolve scoping before implementation. Either:
   • Add explicit document/client ownership enforcement in the custom-metadata service/controller and track it as first-class work, or
   • Do not grant client_user/user_admin write access to /custom-metadata until tenant scoping exists.

3. High: Phase 5a in the tracking doc misses the HTTP-layer work needed to return the documented 409. References:

   • Tracking: plans/mutable.metadata.plan.combo.v3.tracking.md:290-299
   • Current controller behavior: api/queryAPI/fieldextractions.go:45-50 Impact:
   • Adding only a service pre-check will not produce the promised HTTP 409.
   • As written, POST /field-extractions will still return 500 for the new mutual-exclusivity error. Recommendation:
   • Add explicit controller error mapping and HTTP tests for the legacy write guard, not just service tests.

4. High: the reset-metadata section is internally inconsistent about documents that already have legacy field extractions. References:

   • plans/mutable.metadata.plan.combo.v3.md:643-653 Impact:
   • The behavior bullets say a legacy-extraction document “succeeds trivially.”
   • The validation section says the same condition returns 409 Conflict.
   • The tracking doc appears to assume 409, so the plan and tracker are not actually aligned. Recommendation:
   • Pick one behavior and update both docs. I recommend 409, because it matches the mutual-exclusivity rule and avoids false-success responses on legacy documents.

5. High: the custom-metadata versioning plan overstates its concurrency safety, especially on first write. References:

   • Plan claim: plans/mutable.metadata.plan.combo.v3.md:152-153
   • Tracking task: plans/mutable.metadata.plan.combo.v3.tracking.md:239-245
   • Current legacy pattern: internal/fieldextraction/service.go:70-100 Impact:
   • SELECT ... FOR UPDATE on the max-version row does not serialize the first write for a document, because there is no row to lock yet.
   • Two concurrent first writes can still race and one will fail on UNIQUE (document_id, version). Recommendation:
   • Decide on a real serialization strategy now:
   • Lock the document row before version assignment, or
   • Retry on unique-violation explicitly.
   • Add a “concurrent first write” test to the tracker, not just a generic concurrency test.

6. Medium: actor/audit provenance is underspecified and currently inconsistent with the plan’s audit goals. References:

   • Request bodies still carry caller-supplied createdBy: plans/mutable.metadata.plan.combo.v3.md:566-571, 847-858
   • Tracking only explicitly extracts caller identity for reset: plans/mutable.metadata.plan.combo.v3.tracking.md:431-447, 502
   • Reset refers to a not-yet-defined “same log sink used by AssignSchema / DeleteSchema”: plans/mutable.metadata.plan.combo.v3.md:638, 1069 Impact:
   • If created_by is meant to be audit-authoritative, client-supplied JSON is not trustworthy.
   • The tracker does not currently include the work to derive actor identity from auth context for schema create/update/delete, folder assign, or metadata writes.
   • The audit sink itself is assumed rather than designed. Recommendation:
   • Decide now whether actor identity is authoritative or merely display metadata.
   • If authoritative, derive it from auth context everywhere and add explicit tracking tasks.
   • Define the concrete audit sink/API instead of leaving it implied.

Bottom Line

I would not start implementation until Findings 1 and 2 are resolved. Findings 3 through 6 should be folded into the tracking doc before work starts, otherwise the implementation will either drift from the plan or discover these gaps mid-phase.