query-orchestration/plans/mutable.metadata.review.combo.v3.codex.md

# Mutable Metadata Plan v3 Review (Codex)
A review of plans/mutable.metadata.plan.combo.v3.md and plans/mutable.metadata.plan.combo.v3.tracking.md since they are to be used as a pair to implement the plans/mutable.metadata.plan.combo.v3.md plan. 


I reviewed the current legacy field-extraction flow first so the plan review is anchored to real behavior:

- `api/queryAPI/fieldextractions.go:20-69` binds the request, calls `FieldExtraction.CreateFieldExtraction`, and currently maps any create error to HTTP 500.
- `internal/fieldextraction/service.go:27-100` creates the extraction row, then locks existing version rows, computes `max(version)+1`, and inserts the version row.
- Authorization today is checked as `(user, action, first-path-segment resource)` with no document/client object passed into Permit.io: `internal/cognitoauth/middleware.go:84-110`, `internal/cognitoauth/permitio.go:79-89`.

## Findings

1. **Blocking: the tracking doc’s migration numbers collide with live migrations already in the repo.**
   References:
   - `plans/mutable.metadata.plan.combo.v3.tracking.md:48-75`
   - `internal/database/migrations`: existing `00000000000120_*` through `00000000000123_*` are already present (`00000000000120_remove_text_extraction`, `00000000000121_create_eula_tables`, `00000000000122_make_eula_effective_date_nullable`, `00000000000123_create_ui_settings`)
   Impact:
   - Phase 1 cannot start as written.
   - Any implementation that follows the tracking doc literally will create filename collisions and broken migration ordering.
   Recommendation:
   - Renumber all new mutable-metadata migrations to the next free sequence and update every tracking reference before coding.

2. **Blocking: the plan grants `/custom-metadata` read/write to `client_user`, but the current auth model is not tenant-scoped.**
   References:
   - Plan grants: `plans/mutable.metadata.plan.combo.v3.md:730-780`
   - Current auth check: `internal/cognitoauth/middleware.go:105-110`, `internal/cognitoauth/permitio.go:79-89`
   - Current documented limitation: `cmd/auth_related/permit.setup/permit_policies.yaml:155-184`
   Impact:
   - `custom-metadata:get/post` would be authorized purely by resource/action on the first path segment.
   - Without new application-level document/client ownership checks, a `client_user` or `user_admin` who knows another document UUID can read/write that document’s custom metadata across clients.
   Recommendation:
   - Resolve scoping before implementation. Either:
   - Add explicit document/client ownership enforcement in the custom-metadata service/controller and track it as first-class work, or
   - Do not grant `client_user`/`user_admin` write access to `/custom-metadata` until tenant scoping exists.

3. **High: Phase 5a in the tracking doc misses the HTTP-layer work needed to return the documented 409.**
   References:
   - Tracking: `plans/mutable.metadata.plan.combo.v3.tracking.md:290-299`
   - Current controller behavior: `api/queryAPI/fieldextractions.go:45-50`
   Impact:
   - Adding only a service pre-check will not produce the promised HTTP 409.
   - As written, `POST /field-extractions` will still return 500 for the new mutual-exclusivity error.
   Recommendation:
   - Add explicit controller error mapping and HTTP tests for the legacy write guard, not just service tests.

4. **High: the reset-metadata section is internally inconsistent about documents that already have legacy field extractions.**
   References:
   - `plans/mutable.metadata.plan.combo.v3.md:643-653`
   Impact:
   - The behavior bullets say a legacy-extraction document “succeeds trivially.”
   - The validation section says the same condition returns `409 Conflict`.
   - The tracking doc appears to assume `409`, so the plan and tracker are not actually aligned.
   Recommendation:
   - Pick one behavior and update both docs. I recommend `409`, because it matches the mutual-exclusivity rule and avoids false-success responses on legacy documents.

5. **High: the custom-metadata versioning plan overstates its concurrency safety, especially on first write.**
   References:
   - Plan claim: `plans/mutable.metadata.plan.combo.v3.md:152-153`
   - Tracking task: `plans/mutable.metadata.plan.combo.v3.tracking.md:239-245`
   - Current legacy pattern: `internal/fieldextraction/service.go:70-100`
   Impact:
   - `SELECT ... FOR UPDATE` on the max-version row does not serialize the first write for a document, because there is no row to lock yet.
   - Two concurrent first writes can still race and one will fail on `UNIQUE (document_id, version)`.
   Recommendation:
   - Decide on a real serialization strategy now:
   - Lock the document row before version assignment, or
   - Retry on unique-violation explicitly.
   - Add a “concurrent first write” test to the tracker, not just a generic concurrency test.

6. **Medium: actor/audit provenance is underspecified and currently inconsistent with the plan’s audit goals.**
   References:
   - Request bodies still carry caller-supplied `createdBy`: `plans/mutable.metadata.plan.combo.v3.md:566-571`, `847-858`
   - Tracking only explicitly extracts caller identity for reset: `plans/mutable.metadata.plan.combo.v3.tracking.md:431-447`, `502`
   - Reset refers to a not-yet-defined “same log sink used by `AssignSchema` / `DeleteSchema`”: `plans/mutable.metadata.plan.combo.v3.md:638`, `1069`
   Impact:
   - If `created_by` is meant to be audit-authoritative, client-supplied JSON is not trustworthy.
   - The tracker does not currently include the work to derive actor identity from auth context for schema create/update/delete, folder assign, or metadata writes.
   - The audit sink itself is assumed rather than designed.
   Recommendation:
   - Decide now whether actor identity is authoritative or merely display metadata.
   - If authoritative, derive it from auth context everywhere and add explicit tracking tasks.
   - Define the concrete audit sink/API instead of leaving it implied.

## Bottom Line

I would not start implementation until Findings 1 and 2 are resolved. Findings 3 through 6 should be folded into the tracking doc before work starts, otherwise the implementation will either drift from the plan or discover these gaps mid-phase.