feat: deploy training data to zoe root
CI / validate (push) Successful in 38s

This commit is contained in:
2026-07-22 09:13:54 -05:00
parent 1d683ab380
commit 8a697cf64f
11 changed files with 124 additions and 22 deletions
+10 -3
View File
@@ -26,6 +26,7 @@ cargo run -- run to_slug --payload '{"text":"Hello, World!"}'
cargo run -- run write_note --payload '{"filename":"hello.txt","content":"hello"}'
scripts/precommit-check.sh
python3 scripts/generate_training_data.py --dry-run
scripts/deploy_training_data.sh
```
`serve` starts:
@@ -116,7 +117,7 @@ git config core.hooksPath .githooks
## SDG Bridge
`scripts/generate_training_data.py` generates instruction-format seed data from current dictionary artifacts.
`scripts/generate_training_data.py` generates instruction-format seed data from current dictionary artifacts. The default export root is `/Volumes/Zoe/custom-local-llm`.
Dry run:
@@ -129,10 +130,16 @@ Local dataset generation:
```sh
python3 scripts/generate_training_data.py \
--dictionary-dir dictionary/ \
--output-dir ../workspace_Data/data/ \
--manifest ../workspace_Data/manifest.llm.json
--output-dir /Volumes/Zoe/custom-local-llm/training-data/ \
--manifest /Volumes/Zoe/custom-local-llm/manifest.llm.json
```
Optional Mimir/Data Designer expansion is enabled only when `--sdg-host`, `--sdg-user`, and `--sdg-key` are supplied. The bridge does not start or restart Mimir inference services.
If remote expansion fails, the timestamped local seed dataset remains in place, `SDG_ERROR.txt` is written in that dataset directory, and the manifest entry is marked `generated_local_remote_failed`.
Woodpecker deploys a new training-data export to Zoe on `main` pushes by running:
```sh
scripts/deploy_training_data.sh
```