feat: add sdg training data bridge
CI / validate (push) Successful in 38s

This commit is contained in:
2026-07-21 13:14:52 -05:00
parent f1a8bb0f35
commit 1d683ab380
13 changed files with 933 additions and 1 deletions
+14
View File
@@ -19,6 +19,8 @@
- Codex logger smoke test: `scripts/codex-run-logger.sh`
- Precommit/CI check: `scripts/precommit-check.sh`
- Codex environment: `.codex/environments/environment.toml`
- Training data dry run: `python3 scripts/generate_training_data.py --dry-run`
- Local training data bridge: `python3 scripts/generate_training_data.py`
## Environment
@@ -46,6 +48,17 @@ Use `.env` for local values and keep it out of git. Update `.env.example`, `READ
- Do not manually edit generated dictionary files.
- Generated metadata includes `dictionary_version`, `registry_checksum`, `static_base_checksum`, and `codebook_checksum`.
## SDG Bridge Rules
- `SDG_BRIDGE_PLAN.md` documents the local-to-Data-Designer bridge design.
- `scripts/generate_training_data.py` reads generated dictionary artifacts and produces instruction-format seed datasets.
- Default output is `../workspace_Data/data/module_controller_intents_<unix_ts>/`.
- Default manifest patch target is `../workspace_Data/manifest.llm.json`.
- Use `--dry-run` for validation; it must not write output data or patch manifests.
- Mimir/Data Designer expansion only runs when `--sdg-host`, `--sdg-user`, and `--sdg-key` are all provided.
- Do not start or restart Mimir llama.cpp/Keiro from this bridge.
- If Mimir expansion fails, the bridge keeps the local seed dataset, writes `SDG_ERROR.txt`, patches the manifest with `generated_local_remote_failed`, and exits non-zero.
## Dispatch Rules
- Dispatch only exact `action_id` values present in `dictionary/actions.index.json`.
@@ -81,4 +94,5 @@ Use `.env` for local values and keep it out of git. Update `.env.example`, `READ
- Keep the CLI Rust-first and small.
- Prefer adding scripts under `src/scripts/` and registering them in `src/registry.rs`.
- Add focused tests for parsing, dictionary generation, action lookup, and payload validation.
- Add stdlib Python tests for bridge behavior in `tests/` when changing `scripts/generate_training_data.py`.
- Run `cargo test` before handing off code changes.