---
id: M014
title: "Multi-Source Technique Pages — Nested Sections, Composition, Citations, and Section Search"
status: complete
completed_at: 2026-04-03T02:20:34.440Z
key_decisions:
  - D024: Sections with subsections use empty-string content; substance lives in subsections — avoids duplication between section-level and subsection content
  - Offset-based citation scheme for composition: existing moments keep [0]-[N-1], new get [N]-[N+M-1], no renumbering of existing citations
  - Compose detection uses creator_id + LOWER(category) for case-insensitive page matching
  - Per-section embeddings use deterministic uuid5 keyed on page_id:section_slug for idempotent re-indexing
  - Correlated scalar subqueries for admin technique page counts instead of joins with GROUP BY
  - Format-discriminated rendering: body_sections_format field selects v1 or v2 renderer, keeping both paths independent
key_files:
  - prompts/stage5_synthesis.txt
  - prompts/stage5_compose.txt
  - backend/pipeline/schemas.py
  - backend/pipeline/citation_utils.py
  - backend/pipeline/stages.py
  - backend/pipeline/qdrant_client.py
  - backend/search_service.py
  - backend/routers/pipeline.py
  - backend/routers/techniques.py
  - backend/schemas.py
  - backend/models.py
  - alembic/versions/012_multi_source_format.py
  - frontend/src/pages/TechniquePage.tsx
  - frontend/src/components/TableOfContents.tsx
  - frontend/src/utils/citations.tsx
  - frontend/src/pages/AdminTechniquePages.tsx
  - frontend/src/pages/SearchResults.tsx
  - frontend/src/components/SearchAutocomplete.tsx
lessons_learned:
  - Offset-based citation indexing (existing [0]-[N-1], new [N]-[N+M-1]) is cleaner than renumbering — avoids invalidating existing citation references during composition
  - Format-discriminated rendering (v1/v2 switch on a DB column) is a safe way to evolve content structure without breaking existing pages
  - Deterministic UUIDs (uuid5 on entity_id:slug) are essential for idempotent Qdrant point upserts — avoids orphan points on re-indexing
  - Correlated scalar subqueries are cleaner than GROUP BY joins when an endpoint needs multiple independent count aggregations with different filter compositions
  - Compound slug IDs (sectionSlug--subSlug) prevent anchor collisions between sections and subsections in the same page
---

# M014: Multi-Source Technique Pages — Nested Sections, Composition, Citations, and Section Search

**Restructured technique pages to support multi-video composition with nested H2/H3 sections, inline citation markers, table of contents, admin multi-source management, and section-level search with deep linking.**

## What Happened

M014 delivered a fundamental upgrade to technique page structure and content pipeline. Previously, technique pages were single-video, flat-dict affairs. Now they support multi-source composition (new video moments merge into existing pages), nested H2/H3 sections with a clickable table of contents, inline [N] citation markers linking claims to source key moments, per-section Qdrant embeddings with deep-link search results, and an admin view for managing multi-source pages.

Seven slices executed in dependency order:

S01 established the v2 body_sections format — BodySection/BodySubSection Pydantic models, citation_utils for extracting and validating [N] markers, and a rewritten synthesis prompt (stage5_synthesis.txt v5). 28 unit tests.

S02 created the composition prompt (stage5_compose.txt) with offset-based citation re-indexing for merging new moments into existing pages, plus a compose CLI subcommand on the test harness. 16 unit tests.

S03 laid the data foundation: Alembic migration 012 added body_sections_format column and technique_page_videos association table. API responses wired with source_videos field. Deployed and verified on ub01.

S04 wired compose-or-create branching into stage5_synthesis — queries existing pages by creator_id + LOWER(category), branches to compose path if a match exists, otherwise runs standard synthesis. All pages now get body_sections_format='v2' and TechniquePageVideo tracking. 12 unit tests.

S05 built format-aware frontend rendering: v2 pages get a TableOfContents component, nested H2/H3 sections with slugified anchor IDs, and citation superscript links. v1 pages render unchanged. Deployed to ub01.

S06 added an admin technique pages view at /admin/techniques with paginated API endpoint (source/version counts, filters, sort) and expandable source video rows.

S07 completed the stack with per-section Qdrant embeddings (deterministic UUIDs, stale point cleanup), technique_section search result type, and deep link scrolling to hash fragments on technique pages. 22 unit tests.

## Success Criteria Results

The roadmap defined success through slice-level demos. All seven delivered:

- **S01 — v2 body_sections format**: ✅ BodySection/BodySubSection models, citation validation, v5 prompt — 28 tests passing
- **S02 — Compose mode**: ✅ Composition prompt + test harness compose subcommand — 16 tests passing
- **S03 — Data model + migration**: ✅ Alembic 012 applied on ub01, API returns body_sections_format and source_videos
- **S04 — Compose-or-create logic**: ✅ Stage 5 branches on existing pages, sets v2 format, tracks source videos — 12 tests passing
- **S05 — Frontend v2 rendering**: ✅ TOC, nested sections, citation links, v1 unchanged — frontend builds with 0 TS errors, deployed
- **S06 — Admin multi-source view**: ✅ Endpoint with counts/filters, React table with expandable rows — verified via curl + browser
- **S07 — Section search + deep linking**: ✅ Per-section embeddings, technique_section results, hash scroll — 22 tests passing

## Definition of Done Results

- All 7 slices complete with ✅ checkboxes: ✅
- All 7 slice summaries exist with verification_result: passed: ✅
- 37 files changed, ~6,450 lines added (non-.gsd code): ✅
- Frontend builds with zero TypeScript errors: ✅
- Backend imports and endpoints verified on ub01: ✅
- 78 total unit tests across S01 (28), S02 (16), S04 (12), S07 (22): ✅

## Requirement Outcomes

- **R006 (Technique Page Display)**: Remains validated. Now supports both v1 and v2 formats — v2 adds nested sections with TOC and citations.
- **R012 (Incremental Content Addition)**: Remains validated. Composition prompt and pipeline compose-or-create logic fulfill the multi-source update mechanism.
- **R009 (Qdrant Vector Search)**: Remains validated. Now includes per-section embeddings alongside page-level and key moment embeddings.
- **R005 (Search-First Web UI)**: Remains validated. Search results now include technique_section type with section-level deep links.

## Deviations

Root-level conftest.py added (not planned) to fix sys.path for project-root test discovery. Docker Compose service name chrysopedia-web used instead of chrysopedia-web-8096 from plan. T02 in S04 replaced integration-level branching tests with source-code assertions + focused unit tests due to session mock fragility.

## Follow-ups

Visual QA of v2 rendering once real multi-source pipeline runs populate v2 pages in production. Review stashed git edits on ub01. Consider deterministic UUIDs for page-level and key moment Qdrant points (currently uuid4 — see KNOWLEDGE.md entry on QdrantManager).
