llm-tools/tools/k8s.llm.md

Kubernetes (k8s) Guide

A local Kubernetes cluster runs on Tabitha under the silma-ai namespace. This is Silma's local data and infrastructure plane — databases, vector stores, message queues, secrets, observability, and workflow orchestration.

---

Cluster

Field	Value
Context	docker-desktop
Namespace	silma-ai
Manifests	/Users/Tabitha/.openclaw/k8s/silma-ai/

---

Starting the Stack

# Apply all manifests (idempotent)
bash /Users/Tabitha/.openclaw/k8s/silma-ai/scripts/apply.sh

# Or via npm from the OCPlatform root
cd /Users/Tabitha/.openclaw && npm run k8s:silma-ai:ensure

# Port-forward all services to localhost (run in a separate terminal, stays in foreground)
bash /Users/Tabitha/.openclaw/k8s/silma-ai/scripts/port-forward.sh

---

Service Map

All services below are accessible at localhost:<port> only when the port-forward script is running.

Service	Port(s)	Auth	Notes
PostgreSQL + pgvector	5432	secret: postgres-credentials	DB: silma — primary relational + vector store
Redis	6379	none	In-memory cache, pub/sub
Elasticsearch	9200	none	Full-text search
Kafka	9092	none	Event streaming (Zookeeper internal on 2181)
Qdrant	6333 (HTTP), 6334 (gRPC)	none	Vector DB for Silma's semantic memory
MinIO	9000 (API), 9001 (console)	secret: minio-credentials	S3-compatible object storage
Vault	18200	token: devroot	Secrets management — see tools/vault.llm.md
Prometheus	9090	none	Metrics scraper
Grafana	3000	secret: grafana-admin	Dashboards and observability UI
Temporal	7233 (gRPC)	none	Workflow orchestration engine
Temporal UI	8233	none	Web UI for Temporal workflows

---

LLM Inference (Host — NOT in k8s)

LLM inference runs natively on Tabitha's host (not in Docker) to get Metal/GPU acceleration:

cd /Users/Tabitha/.openclaw && npm run api:llm

# Endpoints (OpenAI-compatible):
# Gemma-4-E4B:  http://localhost:8080
# Gemma-4-26B:  http://localhost:8081

---

Common kubectl Commands

# Check all pods
kubectl get pods -n silma-ai

# Check a specific pod's logs
kubectl logs -n silma-ai <pod-name> --tail=50
kubectl logs -n silma-ai <pod-name> -f   # follow

# Describe a pod (for debugging CrashLoopBackOff etc.)
kubectl describe pod -n silma-ai <pod-name>

# Get secrets (base64-encoded)
kubectl get secret -n silma-ai postgres-credentials -o jsonpath='{.data.password}' | base64 -d

# Restart a deployment
kubectl rollout restart deployment -n silma-ai <deployment-name>

# Get all resources in namespace
kubectl get all -n silma-ai

# Check PVCs (persistent storage)
kubectl get pvc -n silma-ai

---

Connecting to Services

PostgreSQL

# After port-forward is active:
psql -h localhost -p 5432 -U silma -d silma
# Password: from kubectl get secret postgres-credentials

Qdrant (Silma's vector memory)

from qdrant_client import QdrantClient
client = QdrantClient(host="localhost", port=6333)
client.get_collections()

MinIO

import boto3
s3 = boto3.client('s3',
    endpoint_url='http://localhost:9000',
    aws_access_key_id='minioadmin',
    aws_secret_access_key='minioadmin')
s3.list_buckets()

Redis

redis-cli -h localhost -p 6379 ping

---

Storage

PVC data is persisted on Hagia (/Volumes/Hagia/k8s/volumes/). If Hagia is not mounted, PVCs will be unavailable and pods will fail to start.

# Verify Hagia is mounted before starting k8s services
ls /Volumes/Hagia/k8s/volumes/

---

Notes for AI Agents

• Always run the port-forward script first before trying to connect to any k8s service
• Tabitha must have Docker Desktop running with the docker-desktop context active
• If pods are in CrashLoopBackOff, check if Hagia is mounted — most PVCs depend on it
• Vault (localhost:18200) requires the port-forward — see tools/vault.llm.md for secret access patterns
• LLM inference is on the host, not in k8s — don't look for it with kubectl