feat: Added paginated GET /admin/pipeline/technique-pages endpoint with…

- "backend/routers/pipeline.py"
- "backend/schemas.py"

GSD-Task: S06/T01
This commit is contained in:
jlightner 2026-04-03 01:55:35 +00:00
parent 5a5295ae3f
commit 495d1fa489
11 changed files with 661 additions and 4 deletions

View file

@ -10,6 +10,6 @@ Restructure technique pages to be broader (per-creator+category across videos),
| S02 | Composition Prompt + Test Harness Compose Mode | high | S01 | ✅ | Run test harness --compose mode with existing page + new moments → merged output with deduplication, new sections, updated citations. |
| S03 | Data Model + Migration | low | — | ✅ | Alembic migration runs clean. API response includes body_sections_format and source_videos fields. |
| S04 | Pipeline Compose-or-Create Logic | high | S01, S02, S03 | ✅ | Process two COPYCATT videos. Second video's moments composed into existing page. technique_page_videos has both video IDs. |
| S05 | Frontend — Nested Rendering, TOC, Citations | medium | S03 | | Format-2 page renders with TOC, nested sections, clickable citations. Format-1 pages unchanged. |
| S05 | Frontend — Nested Rendering, TOC, Citations | medium | S03 | | Format-2 page renders with TOC, nested sections, clickable citations. Format-1 pages unchanged. |
| S06 | Admin UI — Multi-Source Pipeline Management | medium | S03, S04 | ⬜ | Admin view for multi-source page shows source dropdown, composition history, per-video chunking inspection. |
| S07 | Search — Per-Section Embeddings + Deep Linking | medium | S04, S05 | ⬜ | Search 'LFO grain position' → section-level result → click → navigates to page#section and scrolls. |

View file

@ -0,0 +1,88 @@
---
id: S05
parent: M014
milestone: M014
provides:
- V2 format-aware technique page rendering (TOC, nested sections, citations) for S07 deep linking
requires:
- slice: S03
provides: body_sections_format and source_videos fields in API response schema
affects:
- S07
key_files:
- frontend/src/pages/TechniquePage.tsx
- frontend/src/components/TableOfContents.tsx
- frontend/src/utils/citations.tsx
- frontend/src/api/public-client.ts
- frontend/src/App.css
key_decisions:
- Subsection IDs use compound slugs (sectionSlug--subSlug) to avoid anchor collision between sections and subsections
- D024 enforced: sections with subsections skip empty content paragraph
- Invalid citation indices render as plain text rather than broken links
patterns_established:
- Format-discriminated rendering: body_sections_format field selects v1 (dict) or v2 (array) renderer, keeping both paths independent
- Citation parsing as a standalone utility (parseCitations) reusable for any component rendering technique page content
observability_surfaces:
- none
drill_down_paths:
- .gsd/milestones/M014/slices/S05/tasks/T01-SUMMARY.md
- .gsd/milestones/M014/slices/S05/tasks/T02-SUMMARY.md
duration: ""
verification_result: passed
completed_at: 2026-04-03T01:47:43.749Z
blocker_discovered: false
---
# S05: Frontend — Nested Rendering, TOC, Citations
**Format-2 technique pages render with nested H2/H3 sections, a clickable TOC, and citation superscript links; format-1 pages unchanged; deployed to production.**
## What Happened
Built format-aware rendering in TechniquePage.tsx that detects body_sections_format=2 (list-of-objects) vs format=1 (dict) and renders accordingly. V2 path renders a TableOfContents component with CSS-counter-numbered nested anchor links, H2 sections with slugified IDs, H3 subsections with compound slug IDs (sectionSlug--subSlug), and citation parsing. The parseCitations utility in citations.tsx converts [N] and [N,M] markers to superscript anchor links targeting key moment IDs; invalid indices render as plain text. V1 dict rendering is completely untouched.
TypeScript types updated with BodySectionV2, BodySubSectionV2 interfaces, body_sections_format discriminator, and source_videos field. snapshotToOverlay updated to pass through list-format body_sections.
CSS additions: TOC card with sticky positioning, subsection left-border styling, citation superscript links with hover states, scroll-margin-top on anchored sections for smooth scrolling past the fixed header.
Deployed to ub01 production: pushed 11 commits, built chrysopedia-web container (56 modules, 0 Vite/TS errors), verified HTTP 200 on site root and /health. No v2 pages exist in production yet (S04 populated them on test data), so v2 rendering is verified structurally via the TypeScript build. V1 pages render unchanged.
## Verification
Frontend build: `cd frontend && npm run build` exits 0, 56 modules, no errors. Production: curl http://ub01:8096/ returns 200. Health: curl http://ub01:8096/health returns 200. JS bundle hash matches build output.
## Requirements Advanced
- R006 — Technique page display now supports v2 nested sections with TOC and citations in addition to existing v1 format
## Requirements Validated
None.
## New Requirements Surfaced
None.
## Requirements Invalidated or Re-scoped
None.
## Deviations
Docker Compose service name is chrysopedia-web, not chrysopedia-web-8096 as listed in the slice plan verify command. The plan's verify command had shell quoting issues. T02 used the correct service name.
## Known Limitations
V2 rendering only verified structurally (TypeScript build) — no live v2 pages exist in production yet. Visual verification of TOC, citations, and nested sections requires S04 to populate v2 pages first. ub01 has a stashed git state with local backend edits from prior work.
## Follow-ups
Visual QA of v2 rendering once S04 data is in production. Review stashed edits on ub01.
## Files Created/Modified
- `frontend/src/pages/TechniquePage.tsx` — Format-aware rendering: v2 array → TOC + nested H2/H3 sections + citation links; v1 dict unchanged
- `frontend/src/components/TableOfContents.tsx` — New component: CSS-counter-numbered nested anchor link list for v2 sections
- `frontend/src/utils/citations.tsx` — New utility: parseCitations converts [N] and [N,M] markers to superscript anchor links
- `frontend/src/api/public-client.ts` — Added BodySectionV2, BodySubSectionV2 types, body_sections_format, source_videos fields
- `frontend/src/App.css` — TOC card styles, subsection borders, citation superscripts, scroll-margin-top for anchored sections

View file

@ -0,0 +1,63 @@
# S05: Frontend — Nested Rendering, TOC, Citations — UAT
**Milestone:** M014
**Written:** 2026-04-03T01:47:43.749Z
# S05 UAT — Frontend Nested Rendering, TOC, Citations
## Preconditions
- Chrysopedia running at http://ub01:8096 with healthy API
- At least one v1-format technique page exists (current production data)
- For v2 tests: either S04 has populated v2 pages, or use browser dev tools to mock API response
## Test Cases
### TC1: V1 Format Backward Compatibility
1. Navigate to any existing technique page (e.g., http://ub01:8096/techniques/copycatt-bass-sound-design)
2. **Expected:** Page renders identically to pre-S05 behavior — dict-keyed sections as H2 headings with paragraph content below each
3. **Expected:** No TOC component visible on v1 pages
4. **Expected:** No JavaScript console errors
### TC2: V2 Format — Table of Contents Rendering
1. Navigate to a v2-format technique page (body_sections_format = 2)
2. **Expected:** A TOC card appears near the top with numbered section links
3. **Expected:** TOC entries are nested — H2 sections at top level, H3 subsections indented beneath
4. Click a TOC entry
5. **Expected:** Page scrolls to the corresponding section heading, accounting for fixed header offset
### TC3: V2 Format — Nested Section Structure
1. On a v2-format technique page, inspect the section headings
2. **Expected:** Top-level sections render as H2 with slugified id attributes
3. **Expected:** Subsections render as H3 with compound id attributes (parentSlug--childSlug)
4. **Expected:** Sections that have subsections do NOT render an empty paragraph before the subsection list (D024)
5. **Expected:** Sections without subsections render their content directly
### TC4: V2 Format — Citation Links
1. On a v2-format technique page, locate citation markers in prose text (e.g., [1] or [2,3])
2. **Expected:** Citations render as superscript links, not plain bracketed text
3. **Expected:** Clicking a citation scrolls to the corresponding key moment in the moments list
4. **Expected:** Citations with invalid indices (referencing non-existent key moments) render as plain text
### TC5: CSS and Visual Polish
1. On a v2-format technique page, inspect visual styling
2. **Expected:** TOC has card-like styling (background, border, padding)
3. **Expected:** Subsections have left-border visual treatment
4. **Expected:** Citation superscripts have distinct hover state
5. Resize viewport to mobile width
6. **Expected:** TOC and sections reflow cleanly on narrow viewports
### TC6: Build Integrity
1. Run `cd frontend && npm run build`
2. **Expected:** TypeScript compilation and Vite build succeed with 0 errors
3. **Expected:** Output includes all 56+ modules
### TC7: Production Deployment Health
1. `curl -s -o /dev/null -w '%{http_code}' http://ub01:8096/`**Expected:** 200
2. `curl -s -o /dev/null -w '%{http_code}' http://ub01:8096/health`**Expected:** 200
3. Open any technique page in browser, check Network tab
4. **Expected:** JS bundle loads successfully, no 404s on assets
## Edge Cases
- **V2 page with no subsections on any section:** All sections render content directly, no compound IDs
- **V2 page with empty key_moments array:** Citation links have no targets — should render as plain text
- **Very long TOC (10+ sections):** TOC should be scrollable or wrap cleanly

View file

@ -0,0 +1,48 @@
{
"schemaVersion": 1,
"taskId": "T02",
"unitId": "M014/S05/T02",
"timestamp": 1775180782079,
"passed": false,
"discoverySource": "task-plan",
"checks": [
{
"command": "ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia",
"exitCode": 2,
"durationMs": 5,
"verdict": "fail"
},
{
"command": "git pull",
"exitCode": 0,
"durationMs": 1762,
"verdict": "pass"
},
{
"command": "docker compose build chrysopedia-web-8096",
"exitCode": 1,
"durationMs": 153,
"verdict": "fail"
},
{
"command": "docker compose up -d chrysopedia-web-8096'",
"exitCode": 2,
"durationMs": 5,
"verdict": "fail"
},
{
"command": "sleep 5",
"exitCode": 0,
"durationMs": 5005,
"verdict": "pass"
},
{
"command": "curl -s -o /dev/null -w '%{http_code}' http://ub01:8096/",
"exitCode": 0,
"durationMs": 22,
"verdict": "pass"
}
],
"retryAttempt": 1,
"maxRetries": 2
}

View file

@ -1,6 +1,71 @@
# S06: Admin UI — Multi-Source Pipeline Management
**Goal:** Admin UI reflects multi-source reality with per-source pipeline inspection, composition history, and moment→section visibility.
**Goal:** Admin view for multi-source technique pages — list with source/version counts, expandable rows showing source videos and composition history, cross-links to pipeline admin.
**Demo:** After this: Admin view for multi-source page shows source dropdown, composition history, per-video chunking inspection.
## Tasks
- [x] **T01: Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters** — Add a new admin endpoint that returns technique pages with aggregated source video counts, version counts, body_sections_format, and creator info. Supports filtering by multi_source_only (boolean) and creator (slug). This provides the data source for the admin technique pages UI.
The endpoint lives in `backend/routers/pipeline.py` alongside other admin endpoints. It queries TechniquePage with joins to Creator, counts from technique_page_videos and technique_page_versions, and returns a paginated response.
## Steps
1. Read `backend/routers/pipeline.py` to understand existing admin endpoint patterns (imports, auth, response style).
2. Read `backend/schemas.py` to see existing schema patterns.
3. Add a new Pydantic response schema `AdminTechniquePageItem` in `backend/schemas.py` with fields: id, title, slug, creator_name, creator_slug, topic_category, body_sections_format, source_video_count (int), version_count (int), created_at, updated_at. Add `AdminTechniquePageListResponse` with items list and total count.
4. Add `GET /admin/pipeline/technique-pages` endpoint in `backend/routers/pipeline.py`. Query uses: `select(TechniquePage)` with `selectinload(TechniquePage.creator)`, correlated subqueries for source_video_count (count from technique_page_videos where technique_page_id matches) and version_count (count from technique_page_versions). Support query params: `multi_source_only: bool = False` (filter source_video_count > 1), `creator: str | None` (filter by creator slug), `sort: str = 'recent'` (recent/alpha/creator), `offset: int = 0`, `limit: int = 50`.
5. Test endpoint manually via curl against the running API on ub01.
## Must-Haves
- [ ] Endpoint returns correct source_video_count and version_count per page
- [ ] multi_source_only=true filters to pages with >1 source video
- [ ] creator filter works by slug
- [ ] Response includes body_sections_format field
- [ ] Pagination works (offset/limit/total)
## Verification
- `ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && PYTHONPATH=backend python -c "from schemas import AdminTechniquePageItem, AdminTechniquePageListResponse; print(AdminTechniquePageItem.model_fields.keys())"'` exits 0
- After deploying: `curl -s http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -m json.tool | head -30` returns valid JSON with items array containing source_video_count and version_count fields
- Estimate: 45m
- Files: backend/routers/pipeline.py, backend/schemas.py
- Verify: ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && PYTHONPATH=backend python -c "from schemas import AdminTechniquePageItem; print(AdminTechniquePageItem.model_fields.keys())"' && curl -sf http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -c 'import json,sys; d=json.load(sys.stdin); assert "items" in d; print(f"OK: {d["total"]} pages")'
- [ ] **T02: Build AdminTechniquePages page with route and dropdown entry** — Create the frontend page for admin technique page management. Table showing technique pages with source video counts, version counts, format badges, and creator. Expandable rows show source videos list (from existing `/techniques/{slug}` detail endpoint) and link to version history. Filters for multi-source only and creator. Cross-links to pipeline admin and public technique pages.
Follows patterns from `AdminPipeline.tsx` (table layout, expandable rows, badges, fetch pattern).
## Steps
1. Read `frontend/src/pages/AdminPipeline.tsx` (first ~100 lines) for fetch patterns, state management, table structure.
2. Read `frontend/src/api/public-client.ts` for the `request<T>` pattern and existing type exports.
3. Add TypeScript types and fetch function to `frontend/src/api/public-client.ts`: `AdminTechniquePageItem` interface (id, title, slug, creator_name, creator_slug, topic_category, body_sections_format, source_video_count, version_count, created_at, updated_at) and `fetchAdminTechniquePages(params)` function calling `GET /admin/pipeline/technique-pages`.
4. Create `frontend/src/pages/AdminTechniquePages.tsx`:
- State: items array, loading, error, filters (multiSourceOnly boolean, creatorFilter string), expandedSlug
- Fetch on mount and filter change using the new client function
- Table columns: Title (link to public page), Creator (link to creator detail), Category, Format (v1/v2 badge), Sources (count), Versions (count), Updated
- Click row to expand → fetch technique detail via existing `fetchTechnique(slug)` → show source videos list with filenames and added_at dates, link to each video in pipeline admin via `/admin/pipeline?video={id}`
- Filter bar: "Multi-source only" toggle checkbox, creator text filter
- Use existing CSS patterns from AdminPipeline (admin-page class, table styles)
5. Add route in `frontend/src/App.tsx`: `<Route path="/admin/techniques" element={<AdminTechniquePages />} />`
6. Add "Techniques" link in `frontend/src/components/AdminDropdown.tsx` menu items.
7. Build frontend to verify no TypeScript errors.
## Must-Haves
- [ ] Table renders with all columns (title, creator, category, format, sources, versions, updated)
- [ ] Row expand shows source videos with filenames and dates
- [ ] Multi-source filter toggle works
- [ ] Cross-link to pipeline admin works (via video ID query param)
- [ ] Cross-link to public technique page works
- [ ] Route `/admin/techniques` works
- [ ] Admin dropdown includes Techniques entry
## Verification
- `cd frontend && npm run build` exits 0 (no TypeScript errors)
- After deploying: navigate to `http://ub01:8096/admin/techniques` in browser, page renders with technique data
- Admin dropdown shows three items: Reports, Pipeline, Techniques
- Estimate: 1h30m
- Files: frontend/src/api/public-client.ts, frontend/src/pages/AdminTechniquePages.tsx, frontend/src/App.tsx, frontend/src/components/AdminDropdown.tsx
- Verify: cd frontend && npm run build 2>&1 | tail -5

View file

@ -0,0 +1,74 @@
# S06 Research: Admin UI — Multi-Source Pipeline Management
## Depth: Light Research
This is straightforward admin UI work using established patterns already in the codebase. The AdminPipeline page (1500+ lines) is a mature component with video list, event logs, chunking inspector, bulk actions, and run history. The data model (S03) and compose logic (S04) are complete. The needed API surface mostly exists already.
## Summary
The roadmap demo: "Admin view for multi-source page shows source dropdown, composition history, per-video chunking inspection."
Three capabilities are needed:
1. **Multi-source technique page list** — Admin needs a view of technique pages showing which source videos contributed to each, grouped/filterable by creator. No such admin endpoint or page exists yet.
2. **Composition history** — Version history showing what was composed when. The `TechniquePageVersion` model and `/{slug}/versions` API endpoint already exist. The frontend `TechniquePage.tsx` already has version switching. What's missing is surfacing this from an admin management perspective (which pages have multiple sources, which were composed vs created fresh).
3. **Per-video chunking inspection** — Already exists as `ChunkingInspector` component in `AdminPipeline.tsx` and the `GET /admin/pipeline/chunking/{video_id}` backend endpoint. This is done.
## Implementation Landscape
### What Exists
**Backend:**
- `GET /techniques` — list with category/creator filter, pagination, sort
- `GET /techniques/{slug}` — detail with `source_videos` and `version_count`
- `GET /techniques/{slug}/versions` — version list with `pipeline_metadata`
- `GET /techniques/{slug}/versions/{n}` — version detail with `content_snapshot`
- `GET /admin/pipeline/chunking/{video_id}` — full chunking data
- `TechniquePageVideo` association table tracking page↔video links with `added_at`
- `TechniquePageVersion` with `pipeline_metadata` JSONB (stores prompt hashes, model config)
**Frontend:**
- `AdminPipeline.tsx` — 1500-line page with video management, chunking inspector, bulk actions
- `TechniquePage.tsx` — version switcher already built
- `public-client.ts` — types for `SourceVideoSummary`, `TechniquePageVersionSummary`, fetchers for techniques/versions
- Route: `/admin/pipeline` (video management), no technique admin page yet
**Schemas:**
- `SourceVideoSummary`: id, filename, content_type, added_at
- `TechniquePageVersionSummary`: version_number, created_at, pipeline_metadata
- `TechniquePageDetail` already includes `source_videos[]` and `version_count`
### What's Needed
**New backend endpoint:**
- `GET /admin/pipeline/technique-pages` — Admin-focused technique page list with source video counts, body_sections_format, version counts, and latest composition date. Needs joins to `technique_page_videos` and `technique_page_versions`. Filterable by multi-source (>1 video) and by creator.
**New frontend page or section:**
- Either a new page at `/admin/techniques` or a new tab/section in `AdminPipeline.tsx`. Given that AdminPipeline is already 1500 lines, a separate page makes more sense.
- Shows technique pages in a table: title, creator, source video count, format (v1/v2), version count, last updated
- Expandable row showing source videos list (from existing `/techniques/{slug}` response) and link to version history
- Filter: "Multi-source only" toggle, creator dropdown
- Link to pipeline admin filtered by video ID (already supported via `?video=` query param)
**Frontend wiring:**
- New route `/admin/techniques`
- Entry in `AdminDropdown` component
- New fetch function in `public-client.ts`
### Natural Seams
1. **T01: Backend endpoint**`GET /admin/pipeline/technique-pages` with multi-source filtering. Small, self-contained.
2. **T02: Frontend page** — New `AdminTechniquePages.tsx` with table, expand/collapse for source videos and composition history, filters. Uses existing API fetchers for version detail.
3. **T03: Integration** — Route, dropdown entry, cross-links between pipeline admin ↔ technique admin.
### Constraints
- The existing `GET /techniques` endpoint returns paginated public data. The admin endpoint needs additional fields (source video count, format, version count) without modifying the public response — so a separate admin endpoint is cleaner.
- `AdminPipeline.tsx` already links to pipeline admin via `?video=` — the technique admin should link back using the same pattern.
- ChunkingInspector already exists in AdminPipeline — the technique admin page should link to it rather than duplicate it.
## Recommendation
Build a lightweight admin technique pages endpoint and a new frontend page. ~3 tasks: backend endpoint, frontend page, integration wiring. The heaviest task is the frontend page, but it follows the exact same patterns as AdminPipeline (table with expandable rows, filters, badges).

View file

@ -0,0 +1,47 @@
---
estimated_steps: 17
estimated_files: 2
skills_used: []
---
# T01: Add GET /admin/pipeline/technique-pages endpoint
Add a new admin endpoint that returns technique pages with aggregated source video counts, version counts, body_sections_format, and creator info. Supports filtering by multi_source_only (boolean) and creator (slug). This provides the data source for the admin technique pages UI.
The endpoint lives in `backend/routers/pipeline.py` alongside other admin endpoints. It queries TechniquePage with joins to Creator, counts from technique_page_videos and technique_page_versions, and returns a paginated response.
## Steps
1. Read `backend/routers/pipeline.py` to understand existing admin endpoint patterns (imports, auth, response style).
2. Read `backend/schemas.py` to see existing schema patterns.
3. Add a new Pydantic response schema `AdminTechniquePageItem` in `backend/schemas.py` with fields: id, title, slug, creator_name, creator_slug, topic_category, body_sections_format, source_video_count (int), version_count (int), created_at, updated_at. Add `AdminTechniquePageListResponse` with items list and total count.
4. Add `GET /admin/pipeline/technique-pages` endpoint in `backend/routers/pipeline.py`. Query uses: `select(TechniquePage)` with `selectinload(TechniquePage.creator)`, correlated subqueries for source_video_count (count from technique_page_videos where technique_page_id matches) and version_count (count from technique_page_versions). Support query params: `multi_source_only: bool = False` (filter source_video_count > 1), `creator: str | None` (filter by creator slug), `sort: str = 'recent'` (recent/alpha/creator), `offset: int = 0`, `limit: int = 50`.
5. Test endpoint manually via curl against the running API on ub01.
## Must-Haves
- [ ] Endpoint returns correct source_video_count and version_count per page
- [ ] multi_source_only=true filters to pages with >1 source video
- [ ] creator filter works by slug
- [ ] Response includes body_sections_format field
- [ ] Pagination works (offset/limit/total)
## Verification
- `ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && PYTHONPATH=backend python -c "from schemas import AdminTechniquePageItem, AdminTechniquePageListResponse; print(AdminTechniquePageItem.model_fields.keys())"'` exits 0
- After deploying: `curl -s http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -m json.tool | head -30` returns valid JSON with items array containing source_video_count and version_count fields
## Inputs
- ``backend/routers/pipeline.py` — existing admin endpoint patterns`
- ``backend/schemas.py` — existing Pydantic schema patterns`
- ``backend/models.py` — TechniquePage, TechniquePageVideo, TechniquePageVersion models`
## Expected Output
- ``backend/routers/pipeline.py` — new GET /admin/pipeline/technique-pages endpoint`
- ``backend/schemas.py` — new AdminTechniquePageItem and AdminTechniquePageListResponse schemas`
## Verification
ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && PYTHONPATH=backend python -c "from schemas import AdminTechniquePageItem; print(AdminTechniquePageItem.model_fields.keys())"' && curl -sf http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -c 'import json,sys; d=json.load(sys.stdin); assert "items" in d; print(f"OK: {d["total"]} pages")'

View file

@ -0,0 +1,81 @@
---
id: T01
parent: S06
milestone: M014
provides: []
requires: []
affects: []
key_files: ["backend/routers/pipeline.py", "backend/schemas.py"]
key_decisions: ["Used correlated scalar subqueries for count fields rather than joins with GROUP BY for cleaner filter composition"]
patterns_established: []
drill_down_paths: []
observability_surfaces: []
duration: ""
verification_result: "Schema import verified inside Docker container. Endpoint returns valid JSON with 20 pages. multi_source_only=true returns 0 (correct — empty association table). creator=skope returns 2. Alpha sort returns correctly ordered titles. Pagination with offset=5&limit=2 returns 2 items with total=20."
completed_at: 2026-04-03T01:55:31.738Z
blocker_discovered: false
---
# T01: Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters
> Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters
## What Happened
---
id: T01
parent: S06
milestone: M014
key_files:
- backend/routers/pipeline.py
- backend/schemas.py
key_decisions:
- Used correlated scalar subqueries for count fields rather than joins with GROUP BY for cleaner filter composition
duration: ""
verification_result: passed
completed_at: 2026-04-03T01:55:31.739Z
blocker_discovered: false
---
# T01: Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters
**Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters**
## What Happened
Added AdminTechniquePageItem and AdminTechniquePageListResponse Pydantic schemas to backend/schemas.py. Added the endpoint in backend/routers/pipeline.py using correlated scalar subqueries against technique_page_videos and technique_page_versions for count fields. Supports multi_source_only, creator slug filter, sort (recent/alpha/creator), and offset/limit pagination. Deployed to ub01 via docker compose rebuild and verified all filters and pagination work correctly.
## Verification
Schema import verified inside Docker container. Endpoint returns valid JSON with 20 pages. multi_source_only=true returns 0 (correct — empty association table). creator=skope returns 2. Alpha sort returns correctly ordered titles. Pagination with offset=5&limit=2 returns 2 items with total=20.
## Verification Evidence
| # | Command | Exit Code | Verdict | Duration |
|---|---------|-----------|---------|----------|
| 1 | `docker exec chrysopedia-api python -c 'from schemas import AdminTechniquePageItem...'` | 0 | ✅ pass | 1000ms |
| 2 | `curl -sf http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -c 'assert items in d'` | 0 | ✅ pass | 1000ms |
| 3 | `curl -sf ...?multi_source_only=true` | 0 | ✅ pass | 1000ms |
| 4 | `curl -sf ...?creator=skope` | 0 | ✅ pass | 1000ms |
| 5 | `curl -sf ...?sort=alpha&limit=3` | 0 | ✅ pass | 1000ms |
| 6 | `curl -sf ...?offset=5&limit=2` | 0 | ✅ pass | 1000ms |
## Deviations
Verified schema import inside Docker rather than on host (pydantic not installed on host).
## Known Issues
technique_page_videos and technique_page_versions tables are empty so counts are always 0. Queries are correct and will return real counts when populated.
## Files Created/Modified
- `backend/routers/pipeline.py`
- `backend/schemas.py`
## Deviations
Verified schema import inside Docker rather than on host (pydantic not installed on host).
## Known Issues
technique_page_videos and technique_page_versions tables are empty so counts are always 0. Queries are correct and will return real counts when populated.

View file

@ -0,0 +1,62 @@
---
estimated_steps: 28
estimated_files: 4
skills_used: []
---
# T02: Build AdminTechniquePages page with route and dropdown entry
Create the frontend page for admin technique page management. Table showing technique pages with source video counts, version counts, format badges, and creator. Expandable rows show source videos list (from existing `/techniques/{slug}` detail endpoint) and link to version history. Filters for multi-source only and creator. Cross-links to pipeline admin and public technique pages.
Follows patterns from `AdminPipeline.tsx` (table layout, expandable rows, badges, fetch pattern).
## Steps
1. Read `frontend/src/pages/AdminPipeline.tsx` (first ~100 lines) for fetch patterns, state management, table structure.
2. Read `frontend/src/api/public-client.ts` for the `request<T>` pattern and existing type exports.
3. Add TypeScript types and fetch function to `frontend/src/api/public-client.ts`: `AdminTechniquePageItem` interface (id, title, slug, creator_name, creator_slug, topic_category, body_sections_format, source_video_count, version_count, created_at, updated_at) and `fetchAdminTechniquePages(params)` function calling `GET /admin/pipeline/technique-pages`.
4. Create `frontend/src/pages/AdminTechniquePages.tsx`:
- State: items array, loading, error, filters (multiSourceOnly boolean, creatorFilter string), expandedSlug
- Fetch on mount and filter change using the new client function
- Table columns: Title (link to public page), Creator (link to creator detail), Category, Format (v1/v2 badge), Sources (count), Versions (count), Updated
- Click row to expand → fetch technique detail via existing `fetchTechnique(slug)` → show source videos list with filenames and added_at dates, link to each video in pipeline admin via `/admin/pipeline?video={id}`
- Filter bar: "Multi-source only" toggle checkbox, creator text filter
- Use existing CSS patterns from AdminPipeline (admin-page class, table styles)
5. Add route in `frontend/src/App.tsx`: `<Route path="/admin/techniques" element={<AdminTechniquePages />} />`
6. Add "Techniques" link in `frontend/src/components/AdminDropdown.tsx` menu items.
7. Build frontend to verify no TypeScript errors.
## Must-Haves
- [ ] Table renders with all columns (title, creator, category, format, sources, versions, updated)
- [ ] Row expand shows source videos with filenames and dates
- [ ] Multi-source filter toggle works
- [ ] Cross-link to pipeline admin works (via video ID query param)
- [ ] Cross-link to public technique page works
- [ ] Route `/admin/techniques` works
- [ ] Admin dropdown includes Techniques entry
## Verification
- `cd frontend && npm run build` exits 0 (no TypeScript errors)
- After deploying: navigate to `http://ub01:8096/admin/techniques` in browser, page renders with technique data
- Admin dropdown shows three items: Reports, Pipeline, Techniques
## Inputs
- ``frontend/src/api/public-client.ts` — existing request<T> pattern and types`
- ``frontend/src/pages/AdminPipeline.tsx` — reference for admin page patterns (table, expand, badges)`
- ``frontend/src/components/AdminDropdown.tsx` — dropdown menu to add entry to`
- ``frontend/src/App.tsx` — route definitions`
- ``backend/schemas.py` — AdminTechniquePageItem schema from T01`
## Expected Output
- ``frontend/src/pages/AdminTechniquePages.tsx` — new admin technique pages page`
- ``frontend/src/api/public-client.ts` — AdminTechniquePageItem type + fetchAdminTechniquePages function`
- ``frontend/src/App.tsx` — new /admin/techniques route`
- ``frontend/src/components/AdminDropdown.tsx` — Techniques menu entry added`
## Verification
cd frontend && npm run build 2>&1 | tail -5

View file

@ -22,9 +22,9 @@ from sqlalchemy.ext.asyncio import AsyncSession
from config import get_settings
from database import get_session
from models import PipelineEvent, PipelineRun, PipelineRunStatus, SourceVideo, Creator, KeyMoment, TranscriptSegment, ProcessingStatus
from models import PipelineEvent, PipelineRun, PipelineRunStatus, SourceVideo, Creator, KeyMoment, TranscriptSegment, ProcessingStatus, TechniquePage, TechniquePageVideo, TechniquePageVersion
from redis_client import get_redis
from schemas import DebugModeResponse, DebugModeUpdate, TokenStageSummary, TokenSummaryResponse
from schemas import DebugModeResponse, DebugModeUpdate, TokenStageSummary, TokenSummaryResponse, AdminTechniquePageItem, AdminTechniquePageListResponse
logger = logging.getLogger("chrysopedia.pipeline")
@ -188,6 +188,108 @@ async def list_pipeline_videos(
}
# ── Admin: Technique Pages ───────────────────────────────────────────────────
@router.get(
"/admin/pipeline/technique-pages",
response_model=AdminTechniquePageListResponse,
)
async def list_admin_technique_pages(
multi_source_only: bool = False,
creator: Annotated[str | None, Query(description="Filter by creator slug")] = None,
sort: Annotated[str, Query(description="Sort: recent, alpha, creator")] = "recent",
offset: Annotated[int, Query(ge=0)] = 0,
limit: Annotated[int, Query(ge=1, le=200)] = 50,
db: AsyncSession = Depends(get_session),
):
"""List technique pages with source video counts, version counts, and creator info.
Supports filtering by multi-source pages only and by creator slug.
"""
# Correlated subquery: source video count per page
video_count_sq = (
select(func.count())
.select_from(TechniquePageVideo)
.where(TechniquePageVideo.technique_page_id == TechniquePage.id)
.correlate(TechniquePage)
.scalar_subquery()
.label("source_video_count")
)
# Correlated subquery: version count per page
version_count_sq = (
select(func.count())
.select_from(TechniquePageVersion)
.where(TechniquePageVersion.technique_page_id == TechniquePage.id)
.correlate(TechniquePage)
.scalar_subquery()
.label("version_count")
)
stmt = (
select(
TechniquePage.id,
TechniquePage.title,
TechniquePage.slug,
TechniquePage.topic_category,
TechniquePage.body_sections_format,
TechniquePage.created_at,
TechniquePage.updated_at,
Creator.name.label("creator_name"),
Creator.slug.label("creator_slug"),
video_count_sq,
version_count_sq,
)
.join(Creator, TechniquePage.creator_id == Creator.id)
)
# Filters
if multi_source_only:
stmt = stmt.where(video_count_sq > 1)
if creator:
stmt = stmt.where(Creator.slug == creator)
# Count total before pagination
count_stmt = select(func.count()).select_from(stmt.subquery())
total = (await db.execute(count_stmt)).scalar() or 0
# Sort
if sort == "alpha":
stmt = stmt.order_by(TechniquePage.title.asc())
elif sort == "creator":
stmt = stmt.order_by(Creator.name.asc(), TechniquePage.title.asc())
else: # "recent" default
stmt = stmt.order_by(TechniquePage.updated_at.desc())
stmt = stmt.offset(offset).limit(limit)
result = await db.execute(stmt)
rows = result.all()
items = [
AdminTechniquePageItem(
id=r.id,
title=r.title,
slug=r.slug,
creator_name=r.creator_name,
creator_slug=r.creator_slug,
topic_category=r.topic_category,
body_sections_format=r.body_sections_format,
source_video_count=r.source_video_count or 0,
version_count=r.version_count or 0,
created_at=r.created_at,
updated_at=r.updated_at,
)
for r in rows
]
return AdminTechniquePageListResponse(
items=items,
total=total,
offset=offset,
limit=limit,
)
# ── Admin: Retrigger ─────────────────────────────────────────────────────────
@router.post("/admin/pipeline/trigger/{video_id}")

View file

@ -432,3 +432,30 @@ class TokenSummaryResponse(BaseModel):
video_id: str
stages: list[TokenStageSummary] = Field(default_factory=list)
grand_total_tokens: int
# ── Admin: Technique Pages ───────────────────────────────────────────────────
class AdminTechniquePageItem(BaseModel):
"""Technique page with aggregated source/version counts for admin view."""
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
title: str
slug: str
creator_name: str
creator_slug: str
topic_category: str
body_sections_format: str
source_video_count: int = 0
version_count: int = 0
created_at: datetime
updated_at: datetime
class AdminTechniquePageListResponse(BaseModel):
"""Paginated list of technique pages for admin view."""
items: list[AdminTechniquePageItem] = Field(default_factory=list)
total: int = 0
offset: int = 0
limit: int = 50