# Kubernetes (k8s) Guide

A local Kubernetes cluster runs on Tabitha under the `silma-ai` namespace. This is Silma's local data and infrastructure plane — databases, vector stores, message queues, secrets, observability, and workflow orchestration.

---

## Cluster

| Field | Value |
|-------|-------|
| **Context** | `docker-desktop` |
| **Namespace** | `silma-ai` |
| **Manifests** | `/Users/Tabitha/.openclaw/k8s/silma-ai/` |

---

## Starting the Stack

```bash
# Apply all manifests (idempotent)
bash /Users/Tabitha/.openclaw/k8s/silma-ai/scripts/apply.sh

# Or via npm from the OCPlatform root
cd /Users/Tabitha/.openclaw && npm run k8s:silma-ai:ensure

# Port-forward all services to localhost (run in a separate terminal, stays in foreground)
bash /Users/Tabitha/.openclaw/k8s/silma-ai/scripts/port-forward.sh
```

---

## Service Map

All services below are accessible at `localhost:<port>` **only when the port-forward script is running.**

| Service | Port(s) | Auth | Notes |
|---------|---------|------|-------|
| **PostgreSQL + pgvector** | `5432` | secret: `postgres-credentials` | DB: `silma` — primary relational + vector store |
| **Redis** | `6379` | none | In-memory cache, pub/sub |
| **Elasticsearch** | `9200` | none | Full-text search |
| **Kafka** | `9092` | none | Event streaming (Zookeeper internal on 2181) |
| **Qdrant** | `6333` (HTTP), `6334` (gRPC) | none | Vector DB for Silma's semantic memory |
| **MinIO** | `9000` (API), `9001` (console) | secret: `minio-credentials` | S3-compatible object storage |
| **Vault** | `18200` | token: `devroot` | Secrets management — see `tools/vault.llm.md` |
| **Prometheus** | `9090` | none | Metrics scraper |
| **Grafana** | `3000` | secret: `grafana-admin` | Dashboards and observability UI |
| **Temporal** | `7233` (gRPC) | none | Workflow orchestration engine |
| **Temporal UI** | `8233` | none | Web UI for Temporal workflows |

---

## LLM Inference (Host — NOT in k8s)

LLM inference runs natively on Tabitha's host (not in Docker) to get Metal/GPU acceleration:

```bash
cd /Users/Tabitha/.openclaw && npm run api:llm

# Endpoints (OpenAI-compatible):
# Gemma-4-E4B:  http://localhost:8080
# Gemma-4-26B:  http://localhost:8081
```

---

## Common kubectl Commands

```bash
# Check all pods
kubectl get pods -n silma-ai

# Check a specific pod's logs
kubectl logs -n silma-ai <pod-name> --tail=50
kubectl logs -n silma-ai <pod-name> -f   # follow

# Describe a pod (for debugging CrashLoopBackOff etc.)
kubectl describe pod -n silma-ai <pod-name>

# Get secrets (base64-encoded)
kubectl get secret -n silma-ai postgres-credentials -o jsonpath='{.data.password}' | base64 -d

# Restart a deployment
kubectl rollout restart deployment -n silma-ai <deployment-name>

# Get all resources in namespace
kubectl get all -n silma-ai

# Check PVCs (persistent storage)
kubectl get pvc -n silma-ai
```

---

## Connecting to Services

### PostgreSQL
```bash
# After port-forward is active:
psql -h localhost -p 5432 -U silma -d silma
# Password: from kubectl get secret postgres-credentials
```

### Qdrant (Silma's vector memory)
```python
from qdrant_client import QdrantClient
client = QdrantClient(host="localhost", port=6333)
client.get_collections()
```

### MinIO
```python
import boto3
s3 = boto3.client('s3',
    endpoint_url='http://localhost:9000',
    aws_access_key_id='minioadmin',
    aws_secret_access_key='minioadmin')
s3.list_buckets()
```

### Redis
```bash
redis-cli -h localhost -p 6379 ping
```

---

## Storage

PVC data is persisted on Hagia (`/Volumes/Hagia/k8s/volumes/`). If Hagia is not mounted, PVCs will be unavailable and pods will fail to start.

```bash
# Verify Hagia is mounted before starting k8s services
ls /Volumes/Hagia/k8s/volumes/
```

---

## Notes for AI Agents

- **Always run the port-forward script first** before trying to connect to any k8s service
- Tabitha must have Docker Desktop running with the `docker-desktop` context active
- If pods are in `CrashLoopBackOff`, check if Hagia is mounted — most PVCs depend on it
- Vault (`localhost:18200`) requires the port-forward — see `tools/vault.llm.md` for secret access patterns
- LLM inference is on the host, not in k8s — don't look for it with `kubectl`