Agent Context

Meta	Value
Repo	`xpltdco/chrysopedia`
Language	Python 3.11+ (backend), TypeScript (frontend)
Framework	FastAPI + Celery + SQLAlchemy (async) + React 18 + Vite
Entry Point	`backend/main.py`
Test Command	`cd backend && python -m pytest tests/ -v`
Build Command	`docker compose build && docker compose up -d`
Compose Services	11 (PostgreSQL, Redis, Qdrant, Ollama, LightRAG, API, Worker, Watcher, nginx, init, healthcheck)
Database	PostgreSQL 16 (async via asyncpg)
Cache/Queue	Redis 7 (Celery broker, classification cache, review toggle)
Vector Store	Qdrant 1.13.2 (nomic-embed-text embeddings via Ollama)
LLM	OpenAI-compatible API (DGX Sparks Qwen primary, Ollama fallback)
Last Updated	2026-04-04

File Index (Read These First)

Priority	File	Purpose
1	`backend/main.py`	FastAPI app setup, router registration
2	`backend/models.py`	18 SQLAlchemy ORM models
3	`backend/config.py`	Settings class (env vars, LRU cached)
4	`backend/database.py`	Async + sync engine factories
5	`backend/schemas.py`	Pydantic request/response schemas
6	`backend/pipeline/stages.py`	6-stage Celery LLM pipeline
7	`backend/services/search_service.py`	Qdrant + keyword search
8	`backend/watcher.py`	Filesystem transcript ingestion
9	`backend/redis_client.py`	Async Redis client
10	`backend/routers/`	11 FastAPI routers (auth, consent, admin, etc.)
11	`frontend/src/App.tsx`	React router, layout, main structure
12	`frontend/src/App.css`	All 5,820 lines of styling (monolithic BEM)
13	`frontend/src/api/public-client.ts`	API client (~600 lines, all TS interfaces)
14	`frontend/src/pages/`	11 page components
15	`docker-compose.yml`	11-service orchestration
16	`Dockerfile.api`	Backend/worker image
17	`Dockerfile.web`	Frontend/nginx image
18	`prompts/`	LLM prompt templates (XML-style, disk-loaded)

Common Modification Patterns

To add an API endpoint:

Create/edit router in backend/routers/
Add Pydantic schemas in backend/schemas.py
Register router in backend/main.py
Add frontend API call in frontend/src/api/public-client.ts

To modify the database schema:

Edit models in backend/models.py
Apply DDL manually (no Alembic migrations currently)
Rebuild API + worker: docker compose build chrysopedia-api && docker compose up -d chrysopedia-api chrysopedia-worker

To add a pipeline stage:

Add stage function in backend/pipeline/stages.py
Create prompt template in prompts/
Register in the Celery task chain
Prompts are disk-loaded at runtime (SHA-256 tracked for reproducibility)

To add a frontend page:

Create component in frontend/src/pages/
Add route in frontend/src/App.tsx
Style in frontend/src/App.css (monolithic, BEM conventions)

Gotchas

Dual SQLAlchemy engines (D004) — FastAPI uses async engine (asyncpg). Celery uses sync engine (psycopg2). Never use the async session in Celery tasks.
asyncpg timezone trap — Use datetime.now(timezone.utc).replace(tzinfo=None). asyncpg rejects timezone-aware datetimes.
SQLAlchemy reserved names — Never use relationship, query, metadata as column names — they shadow SQLAlchemy internals.
Vite build constants — Must wrap with JSON.stringify() in vite.config.ts (e.g., JSON.stringify(version)).
Docker ARG/ENV ordering — In Dockerfile.web: ARG → ENV → RUN npm run build. ENV must precede the build step.
ZFS file watching — Use watchdog.observers.polling.PollingObserver, not inotify (doesn't work on ZFS).
Nginx stale DNS — After API container rebuild, restart nginx: docker compose restart chrysopedia-web-8096
Port 8000 conflict — ub01:8000 may be used by another service; internal API runs on 8000 inside Docker network, exposed as 8096.
Non-blocking embeddings (D005) — Stage 6 (embedding/Qdrant indexing) failures don't block pipeline output. A technique page can exist without embeddings.
Redis classification cache — Stage 4 classification results stored in Redis with 24h TTL, not in the database.
Monolithic CSS — All 5,820 lines in one file (App.css). No CSS modules, no preprocessor. BEM naming with 77 custom properties.
LightRAG service — Runs on port 9621 (localhost only). Data stored in /vmPool/r/services/chrysopedia_lightrag/. Separate .env.lightrag config.

Pipeline Stages

Stage	Purpose	LLM Config
1	Segmentation	Chat model
2	Key Moments extraction	Chat model
3	(Reserved)	—
4	Classification	Chat model (cached in Redis)
5	Synthesis (page generation)	Thinking model
6	Embedding + Qdrant indexing	Ollama (nomic-embed-text)

Verification Commands

# SSH to ub01
ssh ub01
cd /vmPool/r/repos/xpltdco/chrysopedia

# Check running services
docker ps --filter name=chrysopedia

# View API logs
docker logs -f chrysopedia-api

# View worker logs
docker logs -f chrysopedia-worker

# Health check
curl http://ub01:8096/health

# Database shell
psql -h ub01 -p 5433 -U chrysopedia

# Full rebuild
docker compose build && docker compose up -d

Chrysopedia Wiki

Architecture

Features

Reference

Operations