# S05 — Search-First Web UI — Research

**Date:** 2026-03-29

## Summary

S05 builds the public-facing web UI for Chrysopedia — the search-first landing page, technique page display, creators browse, and topics browse. This is a medium-complexity frontend + backend slice that wires together existing infrastructure (Qdrant vectors, PostgreSQL models, React+Vite+TypeScript frontend) into new pages and API endpoints. The backend needs a search endpoint that embeds query text via the embedding API, queries Qdrant for semantic results, and enriches them with DB metadata. The frontend needs 6 new pages/views and a new API client module.

The riskiest piece is the search endpoint: it requires an async embedding call (to convert query text to a vector) followed by an async Qdrant search, both of which must complete within 500ms (R015). The existing `EmbeddingClient` is sync (designed for Celery), so the search endpoint needs an async variant using `openai.AsyncOpenAI` and `AsyncQdrantClient`. Everything else — technique page display, creators browse, topics browse — is standard CRUD over existing DB models.

The frontend is a React 18 + Vite + TypeScript SPA. S04 established the pattern: typed API client with `fetch()`, React Router v6, plain CSS (no Tailwind/UI library). S05 extends this with new routes and pages. The existing `App.tsx` routes everything to `/admin/*` — S05 adds the public UI at root paths (`/`, `/search`, `/techniques/:slug`, `/creators`, `/creators/:slug`, `/topics`).

## Recommendation

Build backend-first: start with the search API endpoint (highest risk — requires async embedding + Qdrant integration), then the read-only data endpoints (technique pages, creators list with counts, topics hierarchy), then the frontend pages. The search endpoint is the critical path — everything else is well-understood CRUD.

Use `AsyncQdrantClient` from `qdrant-client` (already in requirements.txt, same package provides both sync and async) and `openai.AsyncOpenAI` (already in requirements.txt) for the search endpoint. Create a new `backend/routers/search.py` for the search API and new Pydantic response schemas. Keep the existing sync `EmbeddingClient` and `QdrantManager` for Celery — the async search service is a separate, thin layer for the FastAPI request path.

For keyword fallback (R005 says "semantic where possible, with keyword fallback"), use PostgreSQL `ILIKE` queries on technique_page.title, key_moment.title, and creator.name. This handles the case where the embedding service is unavailable or returns empty results.

## Implementation Landscape

### Key Files

**Backend — existing (read, extend):**
- `backend/config.py` — Settings with `qdrant_url`, `qdrant_collection`, `embedding_api_url`, `embedding_model`, `embedding_dimensions`. No changes needed.
- `backend/models.py` — All 7 models already exist: Creator, SourceVideo, TranscriptSegment, KeyMoment, TechniquePage, RelatedTechniqueLink, Tag. TechniquePage has `body_sections` (JSONB), `signal_chains` (JSONB), `topic_tags` (ARRAY), `topic_category`, `plugins` (ARRAY). No schema changes needed.
- `backend/schemas.py` — Has `CreatorRead`, `TechniquePageRead`, `KeyMomentRead`. Needs new response schemas for search results and enriched technique page detail.
- `backend/database.py` — Async engine + `get_session` dependency. Used as-is.
- `backend/main.py` — Mount new routers here (search, techniques, topics).
- `backend/routers/creators.py` — Has `list_creators` (alphabetical order) and `get_creator` (by slug). Needs extension: randomized sort, genre filter, technique count, video count per creator.
- `backend/pipeline/qdrant_client.py` — Sync `QdrantManager` for Celery pipeline write-path. Read-path (search) needs new async code.
- `backend/pipeline/embedding_client.py` — Sync `EmbeddingClient`. Search needs async variant.

**Backend — new files to create:**
- `backend/routers/search.py` — `GET /api/v1/search?q=...&scope=all|topics|creators&limit=20` — orchestrates async embedding + Qdrant search + keyword fallback + DB enrichment.
- `backend/routers/techniques.py` — `GET /api/v1/techniques` (list, filterable by category/creator), `GET /api/v1/techniques/{slug}` (full detail with key_moments, related links, creator info).
- `backend/routers/topics.py` — `GET /api/v1/topics` (hierarchy: top-level categories with sub-topics, counts of technique pages and creators per sub-topic). Reads from `canonical_tags.yaml` + DB aggregation.
- `backend/search_service.py` — Async search service class: `AsyncOpenAI` for embedding, `AsyncQdrantClient` for vector search. Thin wrapper, initialized from Settings.

**Frontend — existing (modify):**
- `frontend/src/App.tsx` — Add new routes for public pages (`/`, `/search`, `/techniques/:slug`, `/creators`, `/creators/:slug`, `/topics`). Keep admin routes as-is.
- `frontend/src/App.css` — Extend with styles for new pages (search bar, technique page, browse lists).
- `frontend/src/api/client.ts` — Extend (or create parallel `search-client.ts`) with typed functions for search, techniques, creators, topics endpoints.

**Frontend — new files to create:**
- `frontend/src/pages/Home.tsx` — Landing page: search bar, Topics card, Creators card, recently added section.
- `frontend/src/pages/SearchResults.tsx` — Full search results page after Enter or "See all results".
- `frontend/src/pages/TechniquePage.tsx` — Full technique page display (R006): header, study guide prose, key moments index, related techniques, plugins.
- `frontend/src/pages/CreatorsBrowse.tsx` — Creators list (R007): genre filter pills, type-to-narrow, sort toggle (random/alpha/views), creator rows with counts.
- `frontend/src/pages/CreatorDetail.tsx` — Creator's technique page list.
- `frontend/src/pages/TopicsBrowse.tsx` — Two-level topic hierarchy (R008): top-level categories → sub-topics with counts.

### Build Order

1. **Search service + search endpoint** (highest risk, unblocks frontend search). Create `backend/search_service.py` with async embedding + Qdrant query. Create `backend/routers/search.py`. This proves the semantic search path works end-to-end. Verify with `curl`.

2. **Technique page + topics + creators API endpoints** (unblocks all frontend pages). Create `backend/routers/techniques.py`, `backend/routers/topics.py`, extend `backend/routers/creators.py`. Add enriched response schemas to `backend/schemas.py`. Mount all new routers in `main.py`.

3. **Frontend: Landing page + search + navigation shell** (proves the primary interaction). Build `Home.tsx` with search bar and live typeahead, `SearchResults.tsx` for full results. This is the highest-value UI — it's the R015 30-second retrieval path.

4. **Frontend: Technique page display** (R006 — the core content unit). Build `TechniquePage.tsx` rendering all sections: header with tags, study guide prose from `body_sections` JSONB, key moments index, related links, plugins.

5. **Frontend: Browse pages** (R007, R008, R014). Build `CreatorsBrowse.tsx` (randomized default sort for R014), `CreatorDetail.tsx`, `TopicsBrowse.tsx`.

### Verification Approach

**Backend verification:**
- `cd backend && python -m pytest tests/ -v` — all existing 40 tests still pass (regression).
- New integration tests for search endpoint: mock the embedding API and Qdrant, verify response shape and <500ms timing.
- New integration tests for techniques/topics/creators endpoints: verify data shape, pagination, filtering.
- `curl` smoke tests against running API: `GET /api/v1/search?q=snare`, `GET /api/v1/techniques`, `GET /api/v1/topics`, `GET /api/v1/creators?sort=random`.

**Frontend verification:**
- `cd frontend && npx tsc -b` — zero TypeScript errors.
- `cd frontend && npm run build` — clean production build.
- Manual browser verification: navigate to each page, confirm data renders, search returns results, technique page displays all sections.

## Constraints

- **Existing frontend uses plain CSS, no UI library.** S04 established this pattern — continue with plain CSS. Adding Tailwind or shadcn/ui would be a divergence. The existing `App.css` has ~350 lines of well-structured styles.
- **EmbeddingClient is sync (Celery).** The search endpoint runs in async FastAPI — cannot reuse `EmbeddingClient` directly. Must create async embedding using `openai.AsyncOpenAI`.
- **QdrantManager is sync (Celery).** Same issue — use `AsyncQdrantClient` for the search read path.
- **Qdrant payloads have limited metadata.** Technique page payloads contain: `type`, `page_id`, `creator_id`, `title`, `topic_category`, `topic_tags`, `summary`. Key moment payloads contain: `type`, `moment_id`, `source_video_id`, `title`, `start_time`, `end_time`, `content_type`. The search endpoint must enrich results with DB data (creator names, slugs, etc.) after Qdrant returns IDs.
- **Stage 4 classification in Redis, not DB.** The `KeyMoment` model lacks `topic_tags`/`topic_category` columns. However, `TechniquePage` has both. For search, technique pages are the primary entity — this is fine.
- **No existing single-resource endpoints for technique pages or key moments.** These need to be created.
- **TypeScript strict mode** — `noUnusedLocals`, `noUnusedParameters`, `noUncheckedIndexedAccess` are all enabled. Frontend code must be clean.

## Common Pitfalls

- **Async embedding timeout in search path.** The embedding API call adds latency to every search request. If the embedding service is slow (>200ms), the 500ms target becomes tight. Mitigation: set a short timeout (300ms) on the embedding call and fall back to keyword search on timeout.
- **Qdrant connection failure in search.** Unlike the pipeline (where embedding/Qdrant failures are non-blocking side-effects), search failures are user-visible. The search endpoint must gracefully degrade to keyword-only search when Qdrant is unavailable.
- **Random sort on creators page (R014) with pagination.** `ORDER BY random()` in SQL gives different results per page load, making offset-based pagination inconsistent. Since the creators list is likely <100 entries, fetch all and shuffle server-side, or use a seed-based random sort (`ORDER BY md5(id::text || seed)`) with the seed passed from the client.
- **body_sections JSONB structure is undefined.** The `TechniquePage.body_sections` column stores the study guide prose as JSONB, but the exact schema depends on what stage 5 (synthesis) produces. The frontend must handle variable structures gracefully. Check `stage5_synthesis.txt` prompt to understand the expected format.
- **Vite dev proxy only covers `/api`.** The existing proxy in `vite.config.ts` proxies `/api` to `http://localhost:8001`. This is sufficient for all new endpoints since they're all under `/api/v1/`.

## Open Risks

- **body_sections JSONB format is unknown until real pipeline output is examined.** The synthesis prompt (stage 5) defines the structure, but no real pipeline run has been done. The technique page frontend component must be flexible enough to handle whatever JSONB shape the LLM produces. Fallback: render as plain text if structure is unrecognized.
- **Qdrant collection may be empty** if no pipeline runs have completed embedding. The search endpoint must handle empty results gracefully and fall back to keyword search.
- **Embedding API latency** is unknown — it depends on the deployment (local Ollama vs DGX Sparks). The 500ms search target may not be achievable with slow embedding. Client-side debounce (300ms) helps, but server-side latency must be measured.
