chore: Added chrysopedia-lightrag service to docker-compose.yml with Qd…

- "docker-compose.yml" - ".env.lightrag" GSD-Task: S01/T01
2026-04-03 21:26:26 +00:00 · 2026-04-03 21:26:26 +00:00 · 80097df4da
commit 80097df4da
parent 75693f3b35
10 changed files with 13312 additions and 18 deletions
--- a/.gsd/completed-units-M018.json
+++ b/.gsd/completed-units-M018.json
@ -0,0 +1 @@
+[]
--- a/.gsd/milestones/M019/slices/S01/S01-PLAN.md
+++ b/.gsd/milestones/M019/slices/S01/S01-PLAN.md
@ -1,6 +1,135 @@
 # S01: [B] LightRAG Deployment + Docker Integration

-**Goal:** Deploy LightRAG framework alongside existing stack with Qdrant backend and music production entity/relationship extraction
+**Goal:** LightRAG service running in Docker Compose stack on ub01, connected to existing Qdrant and Ollama services, with DGX Sparks as LLM backend. Entity extraction produces music production entities from test input. Query returns meaningful results.
 **Demo:** After this: LightRAG service running in Docker, connected to Qdrant, entity extraction prompts producing music production entities from test input

 ## Tasks
+- [x] **T01: Added chrysopedia-lightrag service to docker-compose.yml with Qdrant vector storage, Ollama embeddings, and DGX Sparks LLM config in .env.lightrag** — Add the chrysopedia-lightrag service definition to docker-compose.yml and create the .env.lightrag configuration file. This is the infrastructure config task — all file changes are local, verified with `docker compose config`.
+
+## Steps
+
+1. Read the existing `docker-compose.yml` to understand the current service layout, network, and patterns.
+2. Add the `chrysopedia-lightrag` service to `docker-compose.yml` after the `chrysopedia-ollama` service (before the application services). Use:
+   - Image: `ghcr.io/hkuds/lightrag:latest`
+   - Container name: `chrysopedia-lightrag`
+   - Port: `127.0.0.1:9621:9621`
+   - Volume: `/vmPool/r/services/chrysopedia_lightrag:/app/data`
+   - env_file: `.env.lightrag`
+   - Environment: `TIKTOKEN_CACHE_DIR=/app/data/tiktoken`
+   - depends_on: `chrysopedia-qdrant` (service_healthy), `chrysopedia-ollama` (service_healthy)
+   - Network: `chrysopedia`
+   - Health check: `curl -sf http://localhost:9621/health || exit 1` with interval 15s, timeout 5s, retries 5, start_period 30s
+   - restart: unless-stopped
+3. Create `.env.lightrag` with these settings:
+   - `HOST=0.0.0.0`, `PORT=9621`, `WORKSPACE=chrysopedia`
+   - LLM: `LLM_BINDING=openai`, `LLM_MODEL=fyn-llm-agent-chat`, `LLM_BINDING_HOST=https://chat.forgetyour.name/api`, `LLM_BINDING_API_KEY=` (placeholder — to be set on ub01)
+   - Embedding: `EMBEDDING_BINDING=ollama`, `EMBEDDING_BINDING_HOST=http://chrysopedia-ollama:11434`, `EMBEDDING_MODEL=nomic-embed-text`, `EMBEDDING_DIM=768`
+   - Storage: `LIGHTRAG_VECTOR_STORAGE=QdrantVectorDBStorage`, `QDRANT_URL=http://chrysopedia-qdrant:6333`, `LIGHTRAG_GRAPH_STORAGE=NetworkXStorage`
+   - Entity types: `ENTITY_TYPES=["Creator", "Technique", "Plugin", "Synthesizer", "Effect", "Genre", "DAW", "SamplePack", "SignalChain", "Concept", "Frequency", "SoundDesignElement"]`
+   - Cache: `ENABLE_LLM_CACHE_FOR_EXTRACT=true`
+4. Add `.env.lightrag` to `.gitignore` if not already covered by a wildcard pattern. Check if `.env*` or `.env.lightrag` is already ignored.
+5. Run `docker compose config` to validate — must exit 0 with no errors.
+
+## Must-Haves
+
+- [ ] chrysopedia-lightrag service in docker-compose.yml with correct image, ports, volumes, depends_on, healthcheck
+- [ ] .env.lightrag with all required LightRAG configuration
+- [ ] .env.lightrag excluded from git tracking
+- [ ] `docker compose config` exits 0
+
+## Verification
+
+- `docker compose config 2>&1 | grep -q chrysopedia-lightrag && echo PASS || echo FAIL`
+- `docker compose config` exits 0 (validates all service definitions, env var references, network config)
+- `grep -q 'env.lightrag' .gitignore && echo PASS || echo FAIL`
+
+## Failure Modes
+
+| Dependency | On error | On timeout | On malformed response |
+|------------|----------|-----------|----------------------|
+| docker compose config | Fix syntax error in compose file | N/A | N/A |
+| .env.lightrag file | Verify all env var names match LightRAG's expected names | N/A | N/A |
+  - Estimate: 30m
+  - Files: docker-compose.yml, .env.lightrag, .gitignore
+  - Verify: docker compose config 2>&1 | grep -q chrysopedia-lightrag && echo PASS
+- [ ] **T02: Deploy LightRAG on ub01 and verify entity extraction with music production content** — Deploy the LightRAG container on ub01, set the real API key, verify health, and test entity extraction with music production content. This is the operational proof task.
+
+## Steps
+
+1. SSH to ub01 and create the bind mount directory:
+   ```
+   ssh ub01 'mkdir -p /vmPool/r/services/chrysopedia_lightrag'
+   ```
+2. Push local changes (docker-compose.yml, .env.lightrag) to the repo so ub01 can pull them:
+   - Stage and commit docker-compose.yml and .env.lightrag changes
+   - Push to the remote
+   - On ub01: `cd /vmPool/r/repos/xpltdco/chrysopedia && git pull`
+3. On ub01, copy .env.lightrag and set the real LLM_BINDING_API_KEY:
+   - The key is the same as LLM_API_KEY in the main .env file on ub01
+   - `ssh ub01` then edit `.env.lightrag` in the repo directory to set the real key
+4. Pull the LightRAG Docker image and bring up the service:
+   ```
+   ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && docker compose pull chrysopedia-lightrag && docker compose up -d chrysopedia-lightrag'
+   ```
+5. Wait for health check to pass (up to 60s):
+   ```
+   ssh ub01 'for i in $(seq 1 12); do docker inspect --format="{{.State.Health.Status}}" chrysopedia-lightrag 2>/dev/null | grep -q healthy && echo HEALTHY && break; sleep 5; done'
+   ```
+6. Verify the /health endpoint:
+   ```
+   ssh ub01 'curl -sf http://localhost:9621/health'
+   ```
+7. Verify existing services still healthy (R010 re-verification):
+   ```
+   ssh ub01 'docker ps --filter name=chrysopedia --format "{{.Names}} {{.Status}}"'
+   ```
+   All existing services should still show as healthy.
+8. Test entity extraction with music production content:
+   ```
+   ssh ub01 'curl -sf -X POST http://localhost:9621/documents/text -H "Content-Type: application/json" -d \'{ "text": "COPYCATT uses Serum for bass sound design with sidechain compression from the kick drum. The signal chain runs through OTT for multiband compression, then into a Valhalla Room reverb send. The technique involves layering a sub bass with a mid-range reese bass, using LFO modulation on the wavetable position in Serum to create movement.", "file_source": "test_music_production.txt" }\''
+   ```
+   Should return success. Check container logs for entity extraction output.
+9. Test query to verify entities were extracted and are searchable:
+   ```
+   ssh ub01 'curl -sf -X POST http://localhost:9621/query -H "Content-Type: application/json" -d \'{ "query": "What plugins does COPYCATT use for bass sound design?", "mode": "mix" }\''
+   ```
+   Should return a response mentioning Serum, OTT, Valhalla Room.
+10. If entity extraction or query fails, check logs:
+    ```
+    ssh ub01 'docker logs chrysopedia-lightrag --tail 50'
+    ```
+    Common failures: LLM API key invalid (401), embedding model not loaded (connection refused), Qdrant unreachable (DNS). Fix and retry.
+
+## Must-Haves
+
+- [ ] Bind mount directory exists on ub01
+- [ ] LightRAG container running and healthy on ub01
+- [ ] /health endpoint returns 200
+- [ ] All existing Chrysopedia services still healthy
+- [ ] Entity extraction succeeds with music production text
+- [ ] Query returns meaningful response with extracted entities
+
+## Verification
+
+- `ssh ub01 'curl -sf http://localhost:9621/health'` returns 200
+- `ssh ub01 'docker ps --filter name=chrysopedia-lightrag --format "{{.Status}}"'` contains "healthy"
+- Entity extraction POST returns success (HTTP 200)
+- Query POST returns response containing music production entities
+- `ssh ub01 'docker ps --filter name=chrysopedia --format "{{.Names}} {{.Status}}"'` shows all services healthy
+
+## Failure Modes
+
+| Dependency | On error | On timeout | On malformed response |
+|------------|----------|-----------|----------------------|
+| DGX Sparks LLM API | Check API key, verify endpoint URL. If 401/403: key is wrong. If 500: try with Ollama fallback. | Increase LightRAG timeout config or check DGX Sparks availability | Log response body, may need model name adjustment |
+| Ollama embedding | Verify chrysopedia-ollama is healthy, check `ollama list` for nomic-embed-text model | Ollama may be loading model — wait and retry | Check embedding dimension config matches model output |
+| Qdrant vector store | Verify chrysopedia-qdrant is healthy, check port 6333 reachable from LightRAG container | Qdrant may be recovering — wait for health check | Collection creation may fail — check Qdrant logs |
+
+## Negative Tests
+
+- **Malformed inputs**: If entity extraction returns empty/garbage, check ENTITY_TYPES env var is valid JSON array
+- **Error paths**: If LLM returns 401, the API key is wrong — compare with main .env LLM_API_KEY
+- **Boundary conditions**: First-time startup may be slow (downloading tiktoken data, creating Qdrant collections) — allow 60s start_period
+  - Estimate: 45m
+  - Files: docker-compose.yml, .env.lightrag
+  - Verify: ssh ub01 'curl -sf http://localhost:9621/health && echo HEALTH_OK'; ssh ub01 'docker ps --filter name=chrysopedia-lightrag --format "{{.Status}}" | grep -q healthy && echo CONTAINER_OK'
--- a/.gsd/milestones/M019/slices/S01/S01-RESEARCH.md
+++ b/.gsd/milestones/M019/slices/S01/S01-RESEARCH.md
@ -0,0 +1,185 @@
+# S01 Research: [B] LightRAG Deployment + Docker Integration
+
+## Summary
+
+LightRAG is a graph-enhanced RAG framework that combines knowledge graph entity/relationship extraction with vector search. It ships an official Docker image (`ghcr.io/hkuds/lightrag:latest`) with a built-in REST API server. The framework natively supports Qdrant as a vector backend and NetworkX for graph storage (the D032-specified MVP approach). The main integration work is: (1) adding a LightRAG container to the existing Chrysopedia Docker Compose stack, (2) configuring it to use the existing Qdrant and Ollama services, (3) configuring DGX Sparks as the LLM backend via OpenAI-compatible binding, (4) customizing entity types for the music production domain, and (5) verifying entity extraction on test input.
+
+## Recommendation
+
+Use the official `ghcr.io/hkuds/lightrag:latest` Docker image with environment-variable-only configuration (no custom Dockerfile needed). Configure:
+- **Vector storage:** `QdrantVectorDBStorage` pointing to `chrysopedia-qdrant:6333`
+- **Graph storage:** `NetworkXStorage` (default, persisted to working dir)
+- **LLM binding:** `openai` pointing to DGX Sparks at `https://chat.forgetyour.name/api`
+- **Embedding binding:** `ollama` pointing to `chrysopedia-ollama:11434`
+- **Entity types:** Custom music production types via `ENTITY_TYPES` env var
+- **Workspace:** `chrysopedia` (isolates data in shared Qdrant via payload partitioning)
+
+## Implementation Landscape
+
+### Existing Infrastructure (on ub01)
+
+| Service | Container | Internal Address | Notes |
+|---------|-----------|-----------------|-------|
+| PostgreSQL 16 | chrysopedia-db | chrysopedia-db:5432 | Not used by LightRAG (uses file-based KV) |
+| Redis 7 | chrysopedia-redis | chrysopedia-redis:6379 | Could be used for LightRAG KV cache, but JSON default is simpler for MVP |
+| Qdrant v1.13.2 | chrysopedia-qdrant | chrysopedia-qdrant:6333 | Shared — LightRAG creates its own collections with workspace prefix |
+| Ollama | chrysopedia-ollama | chrysopedia-ollama:11434 | Runs nomic-embed-text (768 dim) |
+| API | chrysopedia-api | chrysopedia-api:8000 | FastAPI — LLM at DGX Sparks (`https://chat.forgetyour.name/api`) |
+| Web UI | chrysopedia-web-8096 | port 8096 | React frontend |
+| Worker | chrysopedia-worker | — | Celery worker |
+| Watcher | chrysopedia-watcher | — | Transcript folder watcher |
+| MCP | chrysopedia-mcp | port 8101 | AI agent tools |
+
+**Network:** `chrysopedia` bridge (172.32.0.0/24)
+**Compose project:** `xpltd_chrysopedia`
+**Repo:** `/vmPool/r/repos/xpltdco/chrysopedia`
+
+### LightRAG Architecture (Key Findings)
+
+**Four storage types:**
+1. **KV_STORAGE** — LLM response cache, text chunks, document info. Default: `JsonKVStorage` (file-based). Persists to `WORKING_DIR`.
+2. **VECTOR_STORAGE** — Entity, relation, and chunk vectors. Using: `QdrantVectorDBStorage`.
+3. **GRAPH_STORAGE** — Entity relation graph. Using: `NetworkXStorage` (default, file-based GraphML). Per D032, NetworkX for MVP, Neo4j migration path at >100K entities.
+4. **DOC_STATUS_STORAGE** — Document indexing status. Default: `JsonDocStatusStorage` (file-based).
+
+**Qdrant workspace isolation:** QdrantVectorDBStorage uses payload-based partitioning (Qdrant's recommended multitenancy approach) — shared collections with payload filtering. Set `WORKSPACE=chrysopedia` to avoid any collision with the existing `chrysopedia` collection used by the current search system. LightRAG creates its own collections automatically.
+
+**Entity types customization:** Set via `ENTITY_TYPES` env var (JSON array string). Default types are generic ("Person", "Creature", "Organization", etc.). For music production domain, define custom types like:
+```
+ENTITY_TYPES='["Creator", "Technique", "Plugin", "Synthesizer", "Effect", "Genre", "DAW", "SamplePack", "SignalChain", "Concept", "Frequency", "SoundDesignElement"]'
+```
+
+**REST API endpoints (key ones for this slice):**
+- `POST /documents/text` — Insert text for indexing (triggers entity extraction)
+- `POST /documents/texts` — Insert multiple texts
+- `POST /query` — Query with modes: naive, local, global, hybrid, mix
+- `GET /health` — Health check
+- `POST /graph/entity/create` — Manual entity creation
+- Built-in WebUI at the server root (port 9621)
+
+**LLM configuration for DGX Sparks:**
+```env
+LLM_BINDING=openai
+LLM_MODEL=fyn-llm-agent-chat
+LLM_BINDING_HOST=https://chat.forgetyour.name/api
+LLM_BINDING_API_KEY=<same key as chrysopedia-api>
+```
+
+**Embedding configuration (reusing existing Ollama):**
+```env
+EMBEDDING_BINDING=ollama
+EMBEDDING_BINDING_HOST=http://chrysopedia-ollama:11434
+EMBEDDING_MODEL=nomic-embed-text
+EMBEDDING_DIM=768
+```
+
+### Docker Integration Plan
+
+**New service in docker-compose.yml:**
+```yaml
+chrysopedia-lightrag:
+  image: ghcr.io/hkuds/lightrag:latest
+  container_name: chrysopedia-lightrag
+  restart: unless-stopped
+  ports:
+    - "127.0.0.1:9621:9621"
+  volumes:
+    - /vmPool/r/services/chrysopedia_lightrag:/app/data
+  env_file:
+    - .env.lightrag
+  environment:
+    - TIKTOKEN_CACHE_DIR=/app/data/tiktoken
+  depends_on:
+    chrysopedia-qdrant:
+      condition: service_healthy
+    chrysopedia-ollama:
+      condition: service_healthy
+  networks:
+    - chrysopedia
+  healthcheck:
+    test: ["CMD-SHELL", "curl -sf http://localhost:9621/health || exit 1"]
+    interval: 15s
+    timeout: 5s
+    retries: 5
+    start_period: 30s
+```
+
+**Separate .env.lightrag file** (cleaner than mixing vars with main .env):
+```env
+HOST=0.0.0.0
+PORT=9621
+WORKSPACE=chrysopedia
+
+# LLM
+LLM_BINDING=openai
+LLM_MODEL=fyn-llm-agent-chat
+LLM_BINDING_HOST=https://chat.forgetyour.name/api
+LLM_BINDING_API_KEY=${LLM_API_KEY}
+
+# Embedding
+EMBEDDING_BINDING=ollama
+EMBEDDING_BINDING_HOST=http://chrysopedia-ollama:11434
+EMBEDDING_MODEL=nomic-embed-text
+EMBEDDING_DIM=768
+
+# Storage
+LIGHTRAG_VECTOR_STORAGE=QdrantVectorDBStorage
+QDRANT_URL=http://chrysopedia-qdrant:6333
+LIGHTRAG_GRAPH_STORAGE=NetworkXStorage
+
+# Entity extraction
+ENTITY_TYPES='["Creator", "Technique", "Plugin", "Synthesizer", "Effect", "Genre", "DAW", "SamplePack", "SignalChain", "Concept", "Frequency", "SoundDesignElement"]'
+ENABLE_LLM_CACHE_FOR_EXTRACT=true
+```
+
+### Bind mount path
+Create `/vmPool/r/services/chrysopedia_lightrag` on ub01 for LightRAG data persistence (rag_storage, inputs, tiktoken cache).
+
+## Risks & Mitigations
+
+| Risk | Impact | Mitigation |
+|------|--------|------------|
+| DGX Sparks API compatibility with LightRAG's OpenAI binding | high — entity extraction fails silently or produces garbage | Test with a simple `/documents/text` insert before full integration. The API uses standard `/v1/chat/completions`. If it fails, fall back to Ollama with a local model. |
+| Qdrant collection namespace collision | medium — LightRAG overwrites existing chrysopedia vectors | LightRAG uses workspace-prefixed collection names + payload filtering. The existing `chrysopedia` collection is managed by the current pipeline's QdrantManager. No collision as long as WORKSPACE is set. Verify by listing collections after first insert. |
+| LightRAG image size / resource usage on ub01 | low — but ub01 is the home server | The image includes a WebUI frontend (bun build). Memory footprint with NetworkX graph is modest at MVP scale. Monitor with `docker stats`. |
+| nomic-embed-text dimension mismatch | low — but would cause silent errors | Existing system uses 768 dims. LightRAG's EMBEDDING_DIM=768 must match. If LightRAG defaults to a different dim, inserts will fail with a Qdrant dimension mismatch error (loud failure). |
+| Entity extraction quality with music production content | medium — generic prompts may miss domain concepts | Custom ENTITY_TYPES helps. First test should use real technique page content. If extraction quality is poor, the fix is ENTITY_TYPES tuning (no code change). |
+| LLM_BINDING_API_KEY in .env.lightrag needs the same key as main stack | low — operational | Reference the same key. Could use env var interpolation in docker-compose or a shared .env. |
+
+## Natural Task Seams
+
+1. **Docker Compose + .env.lightrag** — Add the service definition, create the env file, create the bind mount directory. Pure infrastructure. Verify: `docker compose config` validates, `docker compose up chrysopedia-lightrag` starts, health check passes.
+
+2. **LLM + Embedding connectivity verification** — With the container running, test that LightRAG can reach both the DGX Sparks LLM endpoint and the Ollama embedding endpoint. Verify: insert a trivial text via the REST API, check logs for successful LLM calls and embedding generation.
+
+3. **Music production entity extraction test** — Insert real technique page content (from existing DB). Check that entities and relationships are extracted with correct music production types. Verify: query the `/graph` endpoints or check the working directory for the GraphML file, confirm entities like "Serum", "sidechain compression", "COPYCATT" appear with appropriate types.
+
+## Don't Hand-Roll
+
+- **Docker image:** Use `ghcr.io/hkuds/lightrag:latest`. Do NOT build from source — the official image includes the WebUI, all storage backends, and tiktoken cache setup.
+- **Entity extraction prompts:** LightRAG has built-in extraction prompts that use the ENTITY_TYPES list. Do NOT write custom extraction prompts — configure via the env var.
+- **Qdrant collection management:** LightRAG creates and manages its own Qdrant collections. Do NOT manually create collections for it.
+
+## Key Files to Modify
+
+| File | Change |
+|------|--------|
+| `docker-compose.yml` | Add `chrysopedia-lightrag` service |
+| `.env.lightrag` (new) | LightRAG-specific environment configuration |
+| (ub01 filesystem) | Create `/vmPool/r/services/chrysopedia_lightrag` directory |
+
+## Verification Strategy
+
+1. `docker compose config` — validates compose syntax
+2. `docker compose up -d chrysopedia-lightrag` — container starts
+3. `curl http://localhost:9621/health` — API responds
+4. `curl -X POST http://localhost:9621/documents/text -H 'Content-Type: application/json' -d '{"text": "COPYCATT uses Serum for bass sound design with sidechain compression from the kick drum...", "file_source": "test_input.txt"}'` — insert succeeds, entities extracted
+5. `curl http://localhost:9621/query -X POST -H 'Content-Type: application/json' -d '{"query": "What plugins does COPYCATT use?", "mode": "mix"}'` — returns meaningful response mentioning Serum
+
+## Sources
+
+- LightRAG GitHub: https://github.com/HKUDS/LightRAG
+- LightRAG API docs: https://github.com/HKUDS/LightRAG/blob/main/lightrag/api/README.md
+- LightRAG env.example: https://github.com/HKUDS/LightRAG/blob/main/env.example
+- Docker image: `ghcr.io/hkuds/lightrag:latest`
+- PyPI: `lightrag-hku` v1.4.12 (latest as of research date)
--- a/.gsd/milestones/M019/slices/S01/tasks/T01-PLAN.md
+++ b/.gsd/milestones/M019/slices/S01/tasks/T01-PLAN.md
@ -0,0 +1,69 @@
+---
+estimated_steps: 37
+estimated_files: 3
+skills_used: []
+---
+
+# T01: Add LightRAG service to Docker Compose and create .env.lightrag
+
+Add the chrysopedia-lightrag service definition to docker-compose.yml and create the .env.lightrag configuration file. This is the infrastructure config task — all file changes are local, verified with `docker compose config`.
+
+## Steps
+
+1. Read the existing `docker-compose.yml` to understand the current service layout, network, and patterns.
+2. Add the `chrysopedia-lightrag` service to `docker-compose.yml` after the `chrysopedia-ollama` service (before the application services). Use:
+   - Image: `ghcr.io/hkuds/lightrag:latest`
+   - Container name: `chrysopedia-lightrag`
+   - Port: `127.0.0.1:9621:9621`
+   - Volume: `/vmPool/r/services/chrysopedia_lightrag:/app/data`
+   - env_file: `.env.lightrag`
+   - Environment: `TIKTOKEN_CACHE_DIR=/app/data/tiktoken`
+   - depends_on: `chrysopedia-qdrant` (service_healthy), `chrysopedia-ollama` (service_healthy)
+   - Network: `chrysopedia`
+   - Health check: `curl -sf http://localhost:9621/health || exit 1` with interval 15s, timeout 5s, retries 5, start_period 30s
+   - restart: unless-stopped
+3. Create `.env.lightrag` with these settings:
+   - `HOST=0.0.0.0`, `PORT=9621`, `WORKSPACE=chrysopedia`
+   - LLM: `LLM_BINDING=openai`, `LLM_MODEL=fyn-llm-agent-chat`, `LLM_BINDING_HOST=https://chat.forgetyour.name/api`, `LLM_BINDING_API_KEY=` (placeholder — to be set on ub01)
+   - Embedding: `EMBEDDING_BINDING=ollama`, `EMBEDDING_BINDING_HOST=http://chrysopedia-ollama:11434`, `EMBEDDING_MODEL=nomic-embed-text`, `EMBEDDING_DIM=768`
+   - Storage: `LIGHTRAG_VECTOR_STORAGE=QdrantVectorDBStorage`, `QDRANT_URL=http://chrysopedia-qdrant:6333`, `LIGHTRAG_GRAPH_STORAGE=NetworkXStorage`
+   - Entity types: `ENTITY_TYPES=["Creator", "Technique", "Plugin", "Synthesizer", "Effect", "Genre", "DAW", "SamplePack", "SignalChain", "Concept", "Frequency", "SoundDesignElement"]`
+   - Cache: `ENABLE_LLM_CACHE_FOR_EXTRACT=true`
+4. Add `.env.lightrag` to `.gitignore` if not already covered by a wildcard pattern. Check if `.env*` or `.env.lightrag` is already ignored.
+5. Run `docker compose config` to validate — must exit 0 with no errors.
+
+## Must-Haves
+
+- [ ] chrysopedia-lightrag service in docker-compose.yml with correct image, ports, volumes, depends_on, healthcheck
+- [ ] .env.lightrag with all required LightRAG configuration
+- [ ] .env.lightrag excluded from git tracking
+- [ ] `docker compose config` exits 0
+
+## Verification
+
+- `docker compose config 2>&1 | grep -q chrysopedia-lightrag && echo PASS || echo FAIL`
+- `docker compose config` exits 0 (validates all service definitions, env var references, network config)
+- `grep -q 'env.lightrag' .gitignore && echo PASS || echo FAIL`
+
+## Failure Modes
+
+| Dependency | On error | On timeout | On malformed response |
+|------------|----------|-----------|----------------------|
+| docker compose config | Fix syntax error in compose file | N/A | N/A |
+| .env.lightrag file | Verify all env var names match LightRAG's expected names | N/A | N/A |
+
+## Inputs
+
+- ``docker-compose.yml` — existing Docker Compose stack definition`
+- ``.env.example` — reference for existing env var patterns`
+- ``.gitignore` — check if .env files are already excluded`
+
+## Expected Output
+
+- ``docker-compose.yml` — updated with chrysopedia-lightrag service definition`
+- ``.env.lightrag` — new file with LightRAG configuration`
+- ``.gitignore` — updated to exclude .env.lightrag if needed`
+
+## Verification
+
+docker compose config 2>&1 | grep -q chrysopedia-lightrag && echo PASS
--- a/.gsd/milestones/M019/slices/S01/tasks/T01-SUMMARY.md
+++ b/.gsd/milestones/M019/slices/S01/tasks/T01-SUMMARY.md
@ -0,0 +1,81 @@
+---
+id: T01
+parent: S01
+milestone: M019
+provides: []
+requires: []
+affects: []
+key_files: ["docker-compose.yml", ".env.lightrag"]
+key_decisions: ["Used env_file required:true since LightRAG cannot start without config", "Set KV and DocStatus storage to JSON file-based for initial simplicity", "Placed service after Ollama and before application services to match dependency ordering"]
+patterns_established: []
+drill_down_paths: []
+observability_surfaces: []
+duration: ""
+verification_result: "All four verification checks passed: docker compose config renders the chrysopedia-lightrag service, exits 0 with no errors, and .env.lightrag is excluded from git tracking via the existing .env.* wildcard pattern."
+completed_at: 2026-04-03T21:26:22.766Z
+blocker_discovered: false
+---
+
+# T01: Added chrysopedia-lightrag service to docker-compose.yml with Qdrant vector storage, Ollama embeddings, and DGX Sparks LLM config in .env.lightrag
+
+> Added chrysopedia-lightrag service to docker-compose.yml with Qdrant vector storage, Ollama embeddings, and DGX Sparks LLM config in .env.lightrag
+
+## What Happened
+---
+id: T01
+parent: S01
+milestone: M019
+key_files:
+  - docker-compose.yml
+  - .env.lightrag
+key_decisions:
+  - Used env_file required:true since LightRAG cannot start without config
+  - Set KV and DocStatus storage to JSON file-based for initial simplicity
+  - Placed service after Ollama and before application services to match dependency ordering
+duration: ""
+verification_result: passed
+completed_at: 2026-04-03T21:26:22.767Z
+blocker_discovered: false
+---
+
+# T01: Added chrysopedia-lightrag service to docker-compose.yml with Qdrant vector storage, Ollama embeddings, and DGX Sparks LLM config in .env.lightrag
+
+**Added chrysopedia-lightrag service to docker-compose.yml with Qdrant vector storage, Ollama embeddings, and DGX Sparks LLM config in .env.lightrag**
+
+## What Happened
+
+Added the chrysopedia-lightrag service definition to docker-compose.yml between Ollama and FastAPI services. Uses ghcr.io/hkuds/lightrag:latest, binds port 9621, mounts persistent storage, depends on Qdrant and Ollama healthy. Created .env.lightrag with OpenAI-compatible LLM binding for DGX Sparks, Ollama embedding binding, Qdrant vector storage, NetworkX graph storage, JSON KV/DocStatus storage, and music production entity types. File is already git-ignored by the .env.* wildcard.
+
+## Verification
+
+All four verification checks passed: docker compose config renders the chrysopedia-lightrag service, exits 0 with no errors, and .env.lightrag is excluded from git tracking via the existing .env.* wildcard pattern.
+
+## Verification Evidence
+
+| # | Command | Exit Code | Verdict | Duration |
+|---|---------|-----------|---------|----------|
+| 1 | `docker compose config 2>&1 | grep -q chrysopedia-lightrag` | 0 | ✅ pass | 500ms |
+| 2 | `docker compose config > /dev/null 2>&1` | 0 | ✅ pass | 500ms |
+| 3 | `grep -q 'env.lightrag|.env.*' .gitignore` | 0 | ✅ pass | 10ms |
+| 4 | `git check-ignore -q .env.lightrag` | 0 | ✅ pass | 10ms |
+
+
+## Deviations
+
+Added explicit KV/DocStatus storage settings, SUMMARY_LANGUAGE, MAX_ASYNC, MAX_TOKENS, OLLAMA_EMBEDDING_NUM_CTX, and RERANK_BINDING based on upstream env.example review. Used env_file required:true.
+
+## Known Issues
+
+LLM_BINDING_API_KEY is a placeholder — must be set on ub01 before LLM requests will work.
+
+## Files Created/Modified
+
+- `docker-compose.yml`
+- `.env.lightrag`
+
+
+## Deviations
+Added explicit KV/DocStatus storage settings, SUMMARY_LANGUAGE, MAX_ASYNC, MAX_TOKENS, OLLAMA_EMBEDDING_NUM_CTX, and RERANK_BINDING based on upstream env.example review. Used env_file required:true.
+
+## Known Issues
+LLM_BINDING_API_KEY is a placeholder — must be set on ub01 before LLM requests will work.
--- a/.gsd/milestones/M019/slices/S01/tasks/T02-PLAN.md
+++ b/.gsd/milestones/M019/slices/S01/tasks/T02-PLAN.md
@ -0,0 +1,100 @@
+---
+estimated_steps: 68
+estimated_files: 2
+skills_used: []
+---
+
+# T02: Deploy LightRAG on ub01 and verify entity extraction with music production content
+
+Deploy the LightRAG container on ub01, set the real API key, verify health, and test entity extraction with music production content. This is the operational proof task.
+
+## Steps
+
+1. SSH to ub01 and create the bind mount directory:
+   ```
+   ssh ub01 'mkdir -p /vmPool/r/services/chrysopedia_lightrag'
+   ```
+2. Push local changes (docker-compose.yml, .env.lightrag) to the repo so ub01 can pull them:
+   - Stage and commit docker-compose.yml and .env.lightrag changes
+   - Push to the remote
+   - On ub01: `cd /vmPool/r/repos/xpltdco/chrysopedia && git pull`
+3. On ub01, copy .env.lightrag and set the real LLM_BINDING_API_KEY:
+   - The key is the same as LLM_API_KEY in the main .env file on ub01
+   - `ssh ub01` then edit `.env.lightrag` in the repo directory to set the real key
+4. Pull the LightRAG Docker image and bring up the service:
+   ```
+   ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && docker compose pull chrysopedia-lightrag && docker compose up -d chrysopedia-lightrag'
+   ```
+5. Wait for health check to pass (up to 60s):
+   ```
+   ssh ub01 'for i in $(seq 1 12); do docker inspect --format="{{.State.Health.Status}}" chrysopedia-lightrag 2>/dev/null | grep -q healthy && echo HEALTHY && break; sleep 5; done'
+   ```
+6. Verify the /health endpoint:
+   ```
+   ssh ub01 'curl -sf http://localhost:9621/health'
+   ```
+7. Verify existing services still healthy (R010 re-verification):
+   ```
+   ssh ub01 'docker ps --filter name=chrysopedia --format "{{.Names}} {{.Status}}"'
+   ```
+   All existing services should still show as healthy.
+8. Test entity extraction with music production content:
+   ```
+   ssh ub01 'curl -sf -X POST http://localhost:9621/documents/text -H "Content-Type: application/json" -d \'{ "text": "COPYCATT uses Serum for bass sound design with sidechain compression from the kick drum. The signal chain runs through OTT for multiband compression, then into a Valhalla Room reverb send. The technique involves layering a sub bass with a mid-range reese bass, using LFO modulation on the wavetable position in Serum to create movement.", "file_source": "test_music_production.txt" }\''
+   ```
+   Should return success. Check container logs for entity extraction output.
+9. Test query to verify entities were extracted and are searchable:
+   ```
+   ssh ub01 'curl -sf -X POST http://localhost:9621/query -H "Content-Type: application/json" -d \'{ "query": "What plugins does COPYCATT use for bass sound design?", "mode": "mix" }\''
+   ```
+   Should return a response mentioning Serum, OTT, Valhalla Room.
+10. If entity extraction or query fails, check logs:
+    ```
+    ssh ub01 'docker logs chrysopedia-lightrag --tail 50'
+    ```
+    Common failures: LLM API key invalid (401), embedding model not loaded (connection refused), Qdrant unreachable (DNS). Fix and retry.
+
+## Must-Haves
+
+- [ ] Bind mount directory exists on ub01
+- [ ] LightRAG container running and healthy on ub01
+- [ ] /health endpoint returns 200
+- [ ] All existing Chrysopedia services still healthy
+- [ ] Entity extraction succeeds with music production text
+- [ ] Query returns meaningful response with extracted entities
+
+## Verification
+
+- `ssh ub01 'curl -sf http://localhost:9621/health'` returns 200
+- `ssh ub01 'docker ps --filter name=chrysopedia-lightrag --format "{{.Status}}"'` contains "healthy"
+- Entity extraction POST returns success (HTTP 200)
+- Query POST returns response containing music production entities
+- `ssh ub01 'docker ps --filter name=chrysopedia --format "{{.Names}} {{.Status}}"'` shows all services healthy
+
+## Failure Modes
+
+| Dependency | On error | On timeout | On malformed response |
+|------------|----------|-----------|----------------------|
+| DGX Sparks LLM API | Check API key, verify endpoint URL. If 401/403: key is wrong. If 500: try with Ollama fallback. | Increase LightRAG timeout config or check DGX Sparks availability | Log response body, may need model name adjustment |
+| Ollama embedding | Verify chrysopedia-ollama is healthy, check `ollama list` for nomic-embed-text model | Ollama may be loading model — wait and retry | Check embedding dimension config matches model output |
+| Qdrant vector store | Verify chrysopedia-qdrant is healthy, check port 6333 reachable from LightRAG container | Qdrant may be recovering — wait for health check | Collection creation may fail — check Qdrant logs |
+
+## Negative Tests
+
+- **Malformed inputs**: If entity extraction returns empty/garbage, check ENTITY_TYPES env var is valid JSON array
+- **Error paths**: If LLM returns 401, the API key is wrong — compare with main .env LLM_API_KEY
+- **Boundary conditions**: First-time startup may be slow (downloading tiktoken data, creating Qdrant collections) — allow 60s start_period
+
+## Inputs
+
+- ``docker-compose.yml` — updated with LightRAG service from T01`
+- ``.env.lightrag` — configuration file from T01`
+
+## Expected Output
+
+- ``docker-compose.yml` — deployed on ub01 (no further local changes)`
+- ``.env.lightrag` — deployed on ub01 with real API key set`
+
+## Verification
+
+ssh ub01 'curl -sf http://localhost:9621/health && echo HEALTH_OK'; ssh ub01 'docker ps --filter name=chrysopedia-lightrag --format "{{.Status}}" | grep -q healthy && echo CONTAINER_OK'
--- a/.gsd/reports/M018-2026-04-03T21-17-51.html
+++ b/.gsd/reports/M018-2026-04-03T21-17-51.html
--- a/.gsd/reports/index.html
+++ b/.gsd/reports/index.html
@ -130,7 +130,7 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
    </div>
    <div class="hdr-right">
      <span class="gen-lbl">Updated</span>
-      <span class="gen">Mar 31, 2026, 05:52 AM</span>
+      <span class="gen">Apr 3, 2026, 09:17 PM</span>
    </div>
  </div>
 </header>
@ -148,6 +148,10 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
        <div class="toc-group-label">M009</div>
        <ul><li><a href="M009-2026-03-31T05-52-28.html">Mar 31, 2026, 05:52 AM</a> <span class="toc-kind toc-milestone">milestone</span></li></ul>
      </div>
+      <div class="toc-group">
+        <div class="toc-group-label">M018</div>
+        <ul><li><a href="M018-2026-04-03T21-17-51.html">Apr 3, 2026, 09:17 PM</a> <span class="toc-kind toc-milestone">milestone</span></li></ul>
+      </div>
  </aside>

  <!-- Main content -->
@ -156,37 +160,39 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
      <h2>Project Overview</h2>
      
    <div class="idx-summary">
-      <div class="idx-stat"><span class="idx-val">$180.97</span><span class="idx-lbl">Total Cost</span></div>
-      <div class="idx-stat"><span class="idx-val">256.54M</span><span class="idx-lbl">Total Tokens</span></div>
-      <div class="idx-stat"><span class="idx-val">8h 17m</span><span class="idx-lbl">Duration</span></div>
-      <div class="idx-stat"><span class="idx-val">35/39</span><span class="idx-lbl">Slices</span></div>
-      <div class="idx-stat"><span class="idx-val">9/10</span><span class="idx-lbl">Milestones</span></div>
-      <div class="idx-stat"><span class="idx-val">2</span><span class="idx-lbl">Reports</span></div>
+      <div class="idx-stat"><span class="idx-val">$365.18</span><span class="idx-lbl">Total Cost</span></div>
+      <div class="idx-stat"><span class="idx-val">519.78M</span><span class="idx-lbl">Total Tokens</span></div>
+      <div class="idx-stat"><span class="idx-val">15h 36m</span><span class="idx-lbl">Duration</span></div>
+      <div class="idx-stat"><span class="idx-val">73/123</span><span class="idx-lbl">Slices</span></div>
+      <div class="idx-stat"><span class="idx-val">18/25</span><span class="idx-lbl">Milestones</span></div>
+      <div class="idx-stat"><span class="idx-val">3</span><span class="idx-lbl">Reports</span></div>
    </div>
    <div class="idx-progress">
-      <div class="idx-bar-track"><div class="idx-bar-fill" style="width:90%"></div></div>
-      <span class="idx-pct">90% complete</span>
+      <div class="idx-bar-track"><div class="idx-bar-fill" style="width:59%"></div></div>
+      <span class="idx-pct">59% complete</span>
    </div>
      <div class="sparkline-wrap"><h3>Cost Progression</h3>
    <div class="sparkline">
      <svg viewBox="0 0 600 60" width="600" height="60" class="spark-svg">
-        <polyline points="12.0,13.7 588.0,12.0" class="spark-line" fill="none"/>
-        <circle cx="12.0" cy="13.7" r="3" class="spark-dot">
+        <polyline points="12.0,31.0 300.0,30.2 588.0,12.0" class="spark-line" fill="none"/>
+        <circle cx="12.0" cy="31.0" r="3" class="spark-dot">
      <title>M008: M008: Credibility Debt Cleanup — Broken Links, Test Data, Jargon, Empty Metrics — $172.23</title>
-    </circle><circle cx="588.0" cy="12.0" r="3" class="spark-dot">
+    </circle><circle cx="300.0" cy="30.2" r="3" class="spark-dot">
      <title>M009: Homepage &amp; First Impression — $180.97</title>
+    </circle><circle cx="588.0" cy="12.0" r="3" class="spark-dot">
+      <title>M018: M018: Phase 2 Research &amp; Documentation — Site Audit and Forgejo Wiki Bootstrap — $365.18</title>
    </circle>
        <text x="12" y="58" class="spark-lbl">$172.23</text>
-        <text x="588" y="58" text-anchor="end" class="spark-lbl">$180.97</text>
+        <text x="588" y="58" text-anchor="end" class="spark-lbl">$365.18</text>
      </svg>
      <div class="spark-axis">
-        <span class="spark-tick" style="left:2.0%" title="2026-03-31T05:31:26.249Z">M008</span><span class="spark-tick" style="left:98.0%" title="2026-03-31T05:52:28.456Z">M009</span>
+        <span class="spark-tick" style="left:2.0%" title="2026-03-31T05:31:26.249Z">M008</span><span class="spark-tick" style="left:50.0%" title="2026-03-31T05:52:28.456Z">M009</span><span class="spark-tick" style="left:98.0%" title="2026-04-03T21:17:51.201Z">M018</span>
      </div>
    </div></div>
    </section>

    <section class="idx-cards">
-      <h2>Progression <span class="sec-count">2</span></h2>
+      <h2>Progression <span class="sec-count">3</span></h2>
      <div class="cards-grid">
      <a class="report-card" href="M008-2026-03-31T05-31-26.html">
        <div class="card-top">
@ -209,7 +215,7 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
        
        
      </a>
-      <a class="report-card card-latest" href="M009-2026-03-31T05-52-28.html">
+      <a class="report-card" href="M009-2026-03-31T05-52-28.html">
        <div class="card-top">
          <span class="card-label">M009: Homepage &amp; First Impression</span>
          <span class="card-kind card-kind-milestone">milestone</span>
@ -228,6 +234,27 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
          <span>35/39 slices</span>
        </div>
        <div class="card-delta"><span>+$8.74</span><span>+3 slices</span><span>+1 milestone</span></div>
+        
+      </a>
+      <a class="report-card card-latest" href="M018-2026-04-03T21-17-51.html">
+        <div class="card-top">
+          <span class="card-label">M018: M018: Phase 2 Research &amp; Documentation — Site Audit and Forgejo Wiki Bootstrap</span>
+          <span class="card-kind card-kind-milestone">milestone</span>
+        </div>
+        <div class="card-date">Apr 3, 2026, 09:17 PM</div>
+        <div class="card-progress">
+          <div class="card-bar-track">
+            <div class="card-bar-fill" style="width:59%"></div>
+          </div>
+          <span class="card-pct">59%</span>
+        </div>
+        <div class="card-stats">
+          <span>$365.18</span>
+          <span>519.78M</span>
+          <span>15h 36m</span>
+          <span>73/123 slices</span>
+        </div>
+        <div class="card-delta"><span>+$184.21</span><span>+38 slices</span><span>+9 milestones</span></div>
        <div class="card-latest-badge">Latest</div>
      </a></div>
    </section>
@ -242,7 +269,7 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
    <span class="ftr-sep">—</span>
    <span>/home/aux/projects/content-to-kb-automator</span>
    <span class="ftr-sep">—</span>
-    <span>Updated Mar 31, 2026, 05:52 AM</span>
+    <span>Updated Apr 3, 2026, 09:17 PM</span>
  </div>
 </footer>
 </body>
--- a/.gsd/reports/reports.json
+++ b/.gsd/reports/reports.json
@ -35,6 +35,22 @@
      "doneMilestones": 9,
      "totalMilestones": 10,
      "phase": "planning"
+    },
+    {
+      "filename": "M018-2026-04-03T21-17-51.html",
+      "generatedAt": "2026-04-03T21:17:51.201Z",
+      "milestoneId": "M018",
+      "milestoneTitle": "M018: Phase 2 Research & Documentation — Site Audit and Forgejo Wiki Bootstrap",
+      "label": "M018: M018: Phase 2 Research & Documentation — Site Audit and Forgejo Wiki Bootstrap",
+      "kind": "milestone",
+      "totalCost": 365.1780352499999,
+      "totalTokens": 519778986,
+      "totalDuration": 56190545,
+      "doneSlices": 73,
+      "totalSlices": 123,
+      "doneMilestones": 18,
+      "totalMilestones": 25,
+      "phase": "planning"
    }
  ]
 }
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -77,6 +77,35 @@ services:
      start_period: 30s
    stop_grace_period: 15s

+  # ── LightRAG (graph-based RAG knowledge base) ──
+  chrysopedia-lightrag:
+    image: ghcr.io/hkuds/lightrag:latest
+    container_name: chrysopedia-lightrag
+    restart: unless-stopped
+    env_file:
+      - path: .env.lightrag
+        required: true
+    environment:
+      TIKTOKEN_CACHE_DIR: /app/data/tiktoken
+    ports:
+      - "127.0.0.1:9621:9621"
+    volumes:
+      - /vmPool/r/services/chrysopedia_lightrag:/app/data
+    depends_on:
+      chrysopedia-qdrant:
+        condition: service_healthy
+      chrysopedia-ollama:
+        condition: service_healthy
+    networks:
+      - chrysopedia
+    healthcheck:
+      test: ["CMD-SHELL", "curl -sf http://localhost:9621/health || exit 1"]
+      interval: 15s
+      timeout: 5s
+      retries: 5
+      start_period: 30s
+    stop_grace_period: 15s
+
  # ── FastAPI application ──
  chrysopedia-api:
    build: