feat: Added paginated GET /admin/pipeline/technique-pages endpoint with…
- "backend/routers/pipeline.py" - "backend/schemas.py" GSD-Task: S06/T01
This commit is contained in:
parent
5a5295ae3f
commit
495d1fa489
11 changed files with 661 additions and 4 deletions
|
|
@ -10,6 +10,6 @@ Restructure technique pages to be broader (per-creator+category across videos),
|
|||
| S02 | Composition Prompt + Test Harness Compose Mode | high | S01 | ✅ | Run test harness --compose mode with existing page + new moments → merged output with deduplication, new sections, updated citations. |
|
||||
| S03 | Data Model + Migration | low | — | ✅ | Alembic migration runs clean. API response includes body_sections_format and source_videos fields. |
|
||||
| S04 | Pipeline Compose-or-Create Logic | high | S01, S02, S03 | ✅ | Process two COPYCATT videos. Second video's moments composed into existing page. technique_page_videos has both video IDs. |
|
||||
| S05 | Frontend — Nested Rendering, TOC, Citations | medium | S03 | ⬜ | Format-2 page renders with TOC, nested sections, clickable citations. Format-1 pages unchanged. |
|
||||
| S05 | Frontend — Nested Rendering, TOC, Citations | medium | S03 | ✅ | Format-2 page renders with TOC, nested sections, clickable citations. Format-1 pages unchanged. |
|
||||
| S06 | Admin UI — Multi-Source Pipeline Management | medium | S03, S04 | ⬜ | Admin view for multi-source page shows source dropdown, composition history, per-video chunking inspection. |
|
||||
| S07 | Search — Per-Section Embeddings + Deep Linking | medium | S04, S05 | ⬜ | Search 'LFO grain position' → section-level result → click → navigates to page#section and scrolls. |
|
||||
|
|
|
|||
88
.gsd/milestones/M014/slices/S05/S05-SUMMARY.md
Normal file
88
.gsd/milestones/M014/slices/S05/S05-SUMMARY.md
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
---
|
||||
id: S05
|
||||
parent: M014
|
||||
milestone: M014
|
||||
provides:
|
||||
- V2 format-aware technique page rendering (TOC, nested sections, citations) for S07 deep linking
|
||||
requires:
|
||||
- slice: S03
|
||||
provides: body_sections_format and source_videos fields in API response schema
|
||||
affects:
|
||||
- S07
|
||||
key_files:
|
||||
- frontend/src/pages/TechniquePage.tsx
|
||||
- frontend/src/components/TableOfContents.tsx
|
||||
- frontend/src/utils/citations.tsx
|
||||
- frontend/src/api/public-client.ts
|
||||
- frontend/src/App.css
|
||||
key_decisions:
|
||||
- Subsection IDs use compound slugs (sectionSlug--subSlug) to avoid anchor collision between sections and subsections
|
||||
- D024 enforced: sections with subsections skip empty content paragraph
|
||||
- Invalid citation indices render as plain text rather than broken links
|
||||
patterns_established:
|
||||
- Format-discriminated rendering: body_sections_format field selects v1 (dict) or v2 (array) renderer, keeping both paths independent
|
||||
- Citation parsing as a standalone utility (parseCitations) reusable for any component rendering technique page content
|
||||
observability_surfaces:
|
||||
- none
|
||||
drill_down_paths:
|
||||
- .gsd/milestones/M014/slices/S05/tasks/T01-SUMMARY.md
|
||||
- .gsd/milestones/M014/slices/S05/tasks/T02-SUMMARY.md
|
||||
duration: ""
|
||||
verification_result: passed
|
||||
completed_at: 2026-04-03T01:47:43.749Z
|
||||
blocker_discovered: false
|
||||
---
|
||||
|
||||
# S05: Frontend — Nested Rendering, TOC, Citations
|
||||
|
||||
**Format-2 technique pages render with nested H2/H3 sections, a clickable TOC, and citation superscript links; format-1 pages unchanged; deployed to production.**
|
||||
|
||||
## What Happened
|
||||
|
||||
Built format-aware rendering in TechniquePage.tsx that detects body_sections_format=2 (list-of-objects) vs format=1 (dict) and renders accordingly. V2 path renders a TableOfContents component with CSS-counter-numbered nested anchor links, H2 sections with slugified IDs, H3 subsections with compound slug IDs (sectionSlug--subSlug), and citation parsing. The parseCitations utility in citations.tsx converts [N] and [N,M] markers to superscript anchor links targeting key moment IDs; invalid indices render as plain text. V1 dict rendering is completely untouched.
|
||||
|
||||
TypeScript types updated with BodySectionV2, BodySubSectionV2 interfaces, body_sections_format discriminator, and source_videos field. snapshotToOverlay updated to pass through list-format body_sections.
|
||||
|
||||
CSS additions: TOC card with sticky positioning, subsection left-border styling, citation superscript links with hover states, scroll-margin-top on anchored sections for smooth scrolling past the fixed header.
|
||||
|
||||
Deployed to ub01 production: pushed 11 commits, built chrysopedia-web container (56 modules, 0 Vite/TS errors), verified HTTP 200 on site root and /health. No v2 pages exist in production yet (S04 populated them on test data), so v2 rendering is verified structurally via the TypeScript build. V1 pages render unchanged.
|
||||
|
||||
## Verification
|
||||
|
||||
Frontend build: `cd frontend && npm run build` exits 0, 56 modules, no errors. Production: curl http://ub01:8096/ returns 200. Health: curl http://ub01:8096/health returns 200. JS bundle hash matches build output.
|
||||
|
||||
## Requirements Advanced
|
||||
|
||||
- R006 — Technique page display now supports v2 nested sections with TOC and citations in addition to existing v1 format
|
||||
|
||||
## Requirements Validated
|
||||
|
||||
None.
|
||||
|
||||
## New Requirements Surfaced
|
||||
|
||||
None.
|
||||
|
||||
## Requirements Invalidated or Re-scoped
|
||||
|
||||
None.
|
||||
|
||||
## Deviations
|
||||
|
||||
Docker Compose service name is chrysopedia-web, not chrysopedia-web-8096 as listed in the slice plan verify command. The plan's verify command had shell quoting issues. T02 used the correct service name.
|
||||
|
||||
## Known Limitations
|
||||
|
||||
V2 rendering only verified structurally (TypeScript build) — no live v2 pages exist in production yet. Visual verification of TOC, citations, and nested sections requires S04 to populate v2 pages first. ub01 has a stashed git state with local backend edits from prior work.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
Visual QA of v2 rendering once S04 data is in production. Review stashed edits on ub01.
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `frontend/src/pages/TechniquePage.tsx` — Format-aware rendering: v2 array → TOC + nested H2/H3 sections + citation links; v1 dict unchanged
|
||||
- `frontend/src/components/TableOfContents.tsx` — New component: CSS-counter-numbered nested anchor link list for v2 sections
|
||||
- `frontend/src/utils/citations.tsx` — New utility: parseCitations converts [N] and [N,M] markers to superscript anchor links
|
||||
- `frontend/src/api/public-client.ts` — Added BodySectionV2, BodySubSectionV2 types, body_sections_format, source_videos fields
|
||||
- `frontend/src/App.css` — TOC card styles, subsection borders, citation superscripts, scroll-margin-top for anchored sections
|
||||
63
.gsd/milestones/M014/slices/S05/S05-UAT.md
Normal file
63
.gsd/milestones/M014/slices/S05/S05-UAT.md
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
# S05: Frontend — Nested Rendering, TOC, Citations — UAT
|
||||
|
||||
**Milestone:** M014
|
||||
**Written:** 2026-04-03T01:47:43.749Z
|
||||
|
||||
# S05 UAT — Frontend Nested Rendering, TOC, Citations
|
||||
|
||||
## Preconditions
|
||||
- Chrysopedia running at http://ub01:8096 with healthy API
|
||||
- At least one v1-format technique page exists (current production data)
|
||||
- For v2 tests: either S04 has populated v2 pages, or use browser dev tools to mock API response
|
||||
|
||||
## Test Cases
|
||||
|
||||
### TC1: V1 Format Backward Compatibility
|
||||
1. Navigate to any existing technique page (e.g., http://ub01:8096/techniques/copycatt-bass-sound-design)
|
||||
2. **Expected:** Page renders identically to pre-S05 behavior — dict-keyed sections as H2 headings with paragraph content below each
|
||||
3. **Expected:** No TOC component visible on v1 pages
|
||||
4. **Expected:** No JavaScript console errors
|
||||
|
||||
### TC2: V2 Format — Table of Contents Rendering
|
||||
1. Navigate to a v2-format technique page (body_sections_format = 2)
|
||||
2. **Expected:** A TOC card appears near the top with numbered section links
|
||||
3. **Expected:** TOC entries are nested — H2 sections at top level, H3 subsections indented beneath
|
||||
4. Click a TOC entry
|
||||
5. **Expected:** Page scrolls to the corresponding section heading, accounting for fixed header offset
|
||||
|
||||
### TC3: V2 Format — Nested Section Structure
|
||||
1. On a v2-format technique page, inspect the section headings
|
||||
2. **Expected:** Top-level sections render as H2 with slugified id attributes
|
||||
3. **Expected:** Subsections render as H3 with compound id attributes (parentSlug--childSlug)
|
||||
4. **Expected:** Sections that have subsections do NOT render an empty paragraph before the subsection list (D024)
|
||||
5. **Expected:** Sections without subsections render their content directly
|
||||
|
||||
### TC4: V2 Format — Citation Links
|
||||
1. On a v2-format technique page, locate citation markers in prose text (e.g., [1] or [2,3])
|
||||
2. **Expected:** Citations render as superscript links, not plain bracketed text
|
||||
3. **Expected:** Clicking a citation scrolls to the corresponding key moment in the moments list
|
||||
4. **Expected:** Citations with invalid indices (referencing non-existent key moments) render as plain text
|
||||
|
||||
### TC5: CSS and Visual Polish
|
||||
1. On a v2-format technique page, inspect visual styling
|
||||
2. **Expected:** TOC has card-like styling (background, border, padding)
|
||||
3. **Expected:** Subsections have left-border visual treatment
|
||||
4. **Expected:** Citation superscripts have distinct hover state
|
||||
5. Resize viewport to mobile width
|
||||
6. **Expected:** TOC and sections reflow cleanly on narrow viewports
|
||||
|
||||
### TC6: Build Integrity
|
||||
1. Run `cd frontend && npm run build`
|
||||
2. **Expected:** TypeScript compilation and Vite build succeed with 0 errors
|
||||
3. **Expected:** Output includes all 56+ modules
|
||||
|
||||
### TC7: Production Deployment Health
|
||||
1. `curl -s -o /dev/null -w '%{http_code}' http://ub01:8096/` → **Expected:** 200
|
||||
2. `curl -s -o /dev/null -w '%{http_code}' http://ub01:8096/health` → **Expected:** 200
|
||||
3. Open any technique page in browser, check Network tab
|
||||
4. **Expected:** JS bundle loads successfully, no 404s on assets
|
||||
|
||||
## Edge Cases
|
||||
- **V2 page with no subsections on any section:** All sections render content directly, no compound IDs
|
||||
- **V2 page with empty key_moments array:** Citation links have no targets — should render as plain text
|
||||
- **Very long TOC (10+ sections):** TOC should be scrollable or wrap cleanly
|
||||
48
.gsd/milestones/M014/slices/S05/tasks/T02-VERIFY.json
Normal file
48
.gsd/milestones/M014/slices/S05/tasks/T02-VERIFY.json
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
{
|
||||
"schemaVersion": 1,
|
||||
"taskId": "T02",
|
||||
"unitId": "M014/S05/T02",
|
||||
"timestamp": 1775180782079,
|
||||
"passed": false,
|
||||
"discoverySource": "task-plan",
|
||||
"checks": [
|
||||
{
|
||||
"command": "ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia",
|
||||
"exitCode": 2,
|
||||
"durationMs": 5,
|
||||
"verdict": "fail"
|
||||
},
|
||||
{
|
||||
"command": "git pull",
|
||||
"exitCode": 0,
|
||||
"durationMs": 1762,
|
||||
"verdict": "pass"
|
||||
},
|
||||
{
|
||||
"command": "docker compose build chrysopedia-web-8096",
|
||||
"exitCode": 1,
|
||||
"durationMs": 153,
|
||||
"verdict": "fail"
|
||||
},
|
||||
{
|
||||
"command": "docker compose up -d chrysopedia-web-8096'",
|
||||
"exitCode": 2,
|
||||
"durationMs": 5,
|
||||
"verdict": "fail"
|
||||
},
|
||||
{
|
||||
"command": "sleep 5",
|
||||
"exitCode": 0,
|
||||
"durationMs": 5005,
|
||||
"verdict": "pass"
|
||||
},
|
||||
{
|
||||
"command": "curl -s -o /dev/null -w '%{http_code}' http://ub01:8096/",
|
||||
"exitCode": 0,
|
||||
"durationMs": 22,
|
||||
"verdict": "pass"
|
||||
}
|
||||
],
|
||||
"retryAttempt": 1,
|
||||
"maxRetries": 2
|
||||
}
|
||||
|
|
@ -1,6 +1,71 @@
|
|||
# S06: Admin UI — Multi-Source Pipeline Management
|
||||
|
||||
**Goal:** Admin UI reflects multi-source reality with per-source pipeline inspection, composition history, and moment→section visibility.
|
||||
**Goal:** Admin view for multi-source technique pages — list with source/version counts, expandable rows showing source videos and composition history, cross-links to pipeline admin.
|
||||
**Demo:** After this: Admin view for multi-source page shows source dropdown, composition history, per-video chunking inspection.
|
||||
|
||||
## Tasks
|
||||
- [x] **T01: Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters** — Add a new admin endpoint that returns technique pages with aggregated source video counts, version counts, body_sections_format, and creator info. Supports filtering by multi_source_only (boolean) and creator (slug). This provides the data source for the admin technique pages UI.
|
||||
|
||||
The endpoint lives in `backend/routers/pipeline.py` alongside other admin endpoints. It queries TechniquePage with joins to Creator, counts from technique_page_videos and technique_page_versions, and returns a paginated response.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read `backend/routers/pipeline.py` to understand existing admin endpoint patterns (imports, auth, response style).
|
||||
2. Read `backend/schemas.py` to see existing schema patterns.
|
||||
3. Add a new Pydantic response schema `AdminTechniquePageItem` in `backend/schemas.py` with fields: id, title, slug, creator_name, creator_slug, topic_category, body_sections_format, source_video_count (int), version_count (int), created_at, updated_at. Add `AdminTechniquePageListResponse` with items list and total count.
|
||||
4. Add `GET /admin/pipeline/technique-pages` endpoint in `backend/routers/pipeline.py`. Query uses: `select(TechniquePage)` with `selectinload(TechniquePage.creator)`, correlated subqueries for source_video_count (count from technique_page_videos where technique_page_id matches) and version_count (count from technique_page_versions). Support query params: `multi_source_only: bool = False` (filter source_video_count > 1), `creator: str | None` (filter by creator slug), `sort: str = 'recent'` (recent/alpha/creator), `offset: int = 0`, `limit: int = 50`.
|
||||
5. Test endpoint manually via curl against the running API on ub01.
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] Endpoint returns correct source_video_count and version_count per page
|
||||
- [ ] multi_source_only=true filters to pages with >1 source video
|
||||
- [ ] creator filter works by slug
|
||||
- [ ] Response includes body_sections_format field
|
||||
- [ ] Pagination works (offset/limit/total)
|
||||
|
||||
## Verification
|
||||
|
||||
- `ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && PYTHONPATH=backend python -c "from schemas import AdminTechniquePageItem, AdminTechniquePageListResponse; print(AdminTechniquePageItem.model_fields.keys())"'` exits 0
|
||||
- After deploying: `curl -s http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -m json.tool | head -30` returns valid JSON with items array containing source_video_count and version_count fields
|
||||
- Estimate: 45m
|
||||
- Files: backend/routers/pipeline.py, backend/schemas.py
|
||||
- Verify: ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && PYTHONPATH=backend python -c "from schemas import AdminTechniquePageItem; print(AdminTechniquePageItem.model_fields.keys())"' && curl -sf http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -c 'import json,sys; d=json.load(sys.stdin); assert "items" in d; print(f"OK: {d["total"]} pages")'
|
||||
- [ ] **T02: Build AdminTechniquePages page with route and dropdown entry** — Create the frontend page for admin technique page management. Table showing technique pages with source video counts, version counts, format badges, and creator. Expandable rows show source videos list (from existing `/techniques/{slug}` detail endpoint) and link to version history. Filters for multi-source only and creator. Cross-links to pipeline admin and public technique pages.
|
||||
|
||||
Follows patterns from `AdminPipeline.tsx` (table layout, expandable rows, badges, fetch pattern).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read `frontend/src/pages/AdminPipeline.tsx` (first ~100 lines) for fetch patterns, state management, table structure.
|
||||
2. Read `frontend/src/api/public-client.ts` for the `request<T>` pattern and existing type exports.
|
||||
3. Add TypeScript types and fetch function to `frontend/src/api/public-client.ts`: `AdminTechniquePageItem` interface (id, title, slug, creator_name, creator_slug, topic_category, body_sections_format, source_video_count, version_count, created_at, updated_at) and `fetchAdminTechniquePages(params)` function calling `GET /admin/pipeline/technique-pages`.
|
||||
4. Create `frontend/src/pages/AdminTechniquePages.tsx`:
|
||||
- State: items array, loading, error, filters (multiSourceOnly boolean, creatorFilter string), expandedSlug
|
||||
- Fetch on mount and filter change using the new client function
|
||||
- Table columns: Title (link to public page), Creator (link to creator detail), Category, Format (v1/v2 badge), Sources (count), Versions (count), Updated
|
||||
- Click row to expand → fetch technique detail via existing `fetchTechnique(slug)` → show source videos list with filenames and added_at dates, link to each video in pipeline admin via `/admin/pipeline?video={id}`
|
||||
- Filter bar: "Multi-source only" toggle checkbox, creator text filter
|
||||
- Use existing CSS patterns from AdminPipeline (admin-page class, table styles)
|
||||
5. Add route in `frontend/src/App.tsx`: `<Route path="/admin/techniques" element={<AdminTechniquePages />} />`
|
||||
6. Add "Techniques" link in `frontend/src/components/AdminDropdown.tsx` menu items.
|
||||
7. Build frontend to verify no TypeScript errors.
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] Table renders with all columns (title, creator, category, format, sources, versions, updated)
|
||||
- [ ] Row expand shows source videos with filenames and dates
|
||||
- [ ] Multi-source filter toggle works
|
||||
- [ ] Cross-link to pipeline admin works (via video ID query param)
|
||||
- [ ] Cross-link to public technique page works
|
||||
- [ ] Route `/admin/techniques` works
|
||||
- [ ] Admin dropdown includes Techniques entry
|
||||
|
||||
## Verification
|
||||
|
||||
- `cd frontend && npm run build` exits 0 (no TypeScript errors)
|
||||
- After deploying: navigate to `http://ub01:8096/admin/techniques` in browser, page renders with technique data
|
||||
- Admin dropdown shows three items: Reports, Pipeline, Techniques
|
||||
- Estimate: 1h30m
|
||||
- Files: frontend/src/api/public-client.ts, frontend/src/pages/AdminTechniquePages.tsx, frontend/src/App.tsx, frontend/src/components/AdminDropdown.tsx
|
||||
- Verify: cd frontend && npm run build 2>&1 | tail -5
|
||||
|
|
|
|||
74
.gsd/milestones/M014/slices/S06/S06-RESEARCH.md
Normal file
74
.gsd/milestones/M014/slices/S06/S06-RESEARCH.md
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
# S06 Research: Admin UI — Multi-Source Pipeline Management
|
||||
|
||||
## Depth: Light Research
|
||||
|
||||
This is straightforward admin UI work using established patterns already in the codebase. The AdminPipeline page (1500+ lines) is a mature component with video list, event logs, chunking inspector, bulk actions, and run history. The data model (S03) and compose logic (S04) are complete. The needed API surface mostly exists already.
|
||||
|
||||
## Summary
|
||||
|
||||
The roadmap demo: "Admin view for multi-source page shows source dropdown, composition history, per-video chunking inspection."
|
||||
|
||||
Three capabilities are needed:
|
||||
|
||||
1. **Multi-source technique page list** — Admin needs a view of technique pages showing which source videos contributed to each, grouped/filterable by creator. No such admin endpoint or page exists yet.
|
||||
|
||||
2. **Composition history** — Version history showing what was composed when. The `TechniquePageVersion` model and `/{slug}/versions` API endpoint already exist. The frontend `TechniquePage.tsx` already has version switching. What's missing is surfacing this from an admin management perspective (which pages have multiple sources, which were composed vs created fresh).
|
||||
|
||||
3. **Per-video chunking inspection** — Already exists as `ChunkingInspector` component in `AdminPipeline.tsx` and the `GET /admin/pipeline/chunking/{video_id}` backend endpoint. This is done.
|
||||
|
||||
## Implementation Landscape
|
||||
|
||||
### What Exists
|
||||
|
||||
**Backend:**
|
||||
- `GET /techniques` — list with category/creator filter, pagination, sort
|
||||
- `GET /techniques/{slug}` — detail with `source_videos` and `version_count`
|
||||
- `GET /techniques/{slug}/versions` — version list with `pipeline_metadata`
|
||||
- `GET /techniques/{slug}/versions/{n}` — version detail with `content_snapshot`
|
||||
- `GET /admin/pipeline/chunking/{video_id}` — full chunking data
|
||||
- `TechniquePageVideo` association table tracking page↔video links with `added_at`
|
||||
- `TechniquePageVersion` with `pipeline_metadata` JSONB (stores prompt hashes, model config)
|
||||
|
||||
**Frontend:**
|
||||
- `AdminPipeline.tsx` — 1500-line page with video management, chunking inspector, bulk actions
|
||||
- `TechniquePage.tsx` — version switcher already built
|
||||
- `public-client.ts` — types for `SourceVideoSummary`, `TechniquePageVersionSummary`, fetchers for techniques/versions
|
||||
- Route: `/admin/pipeline` (video management), no technique admin page yet
|
||||
|
||||
**Schemas:**
|
||||
- `SourceVideoSummary`: id, filename, content_type, added_at
|
||||
- `TechniquePageVersionSummary`: version_number, created_at, pipeline_metadata
|
||||
- `TechniquePageDetail` already includes `source_videos[]` and `version_count`
|
||||
|
||||
### What's Needed
|
||||
|
||||
**New backend endpoint:**
|
||||
- `GET /admin/pipeline/technique-pages` — Admin-focused technique page list with source video counts, body_sections_format, version counts, and latest composition date. Needs joins to `technique_page_videos` and `technique_page_versions`. Filterable by multi-source (>1 video) and by creator.
|
||||
|
||||
**New frontend page or section:**
|
||||
- Either a new page at `/admin/techniques` or a new tab/section in `AdminPipeline.tsx`. Given that AdminPipeline is already 1500 lines, a separate page makes more sense.
|
||||
- Shows technique pages in a table: title, creator, source video count, format (v1/v2), version count, last updated
|
||||
- Expandable row showing source videos list (from existing `/techniques/{slug}` response) and link to version history
|
||||
- Filter: "Multi-source only" toggle, creator dropdown
|
||||
- Link to pipeline admin filtered by video ID (already supported via `?video=` query param)
|
||||
|
||||
**Frontend wiring:**
|
||||
- New route `/admin/techniques`
|
||||
- Entry in `AdminDropdown` component
|
||||
- New fetch function in `public-client.ts`
|
||||
|
||||
### Natural Seams
|
||||
|
||||
1. **T01: Backend endpoint** — `GET /admin/pipeline/technique-pages` with multi-source filtering. Small, self-contained.
|
||||
2. **T02: Frontend page** — New `AdminTechniquePages.tsx` with table, expand/collapse for source videos and composition history, filters. Uses existing API fetchers for version detail.
|
||||
3. **T03: Integration** — Route, dropdown entry, cross-links between pipeline admin ↔ technique admin.
|
||||
|
||||
### Constraints
|
||||
|
||||
- The existing `GET /techniques` endpoint returns paginated public data. The admin endpoint needs additional fields (source video count, format, version count) without modifying the public response — so a separate admin endpoint is cleaner.
|
||||
- `AdminPipeline.tsx` already links to pipeline admin via `?video=` — the technique admin should link back using the same pattern.
|
||||
- ChunkingInspector already exists in AdminPipeline — the technique admin page should link to it rather than duplicate it.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Build a lightweight admin technique pages endpoint and a new frontend page. ~3 tasks: backend endpoint, frontend page, integration wiring. The heaviest task is the frontend page, but it follows the exact same patterns as AdminPipeline (table with expandable rows, filters, badges).
|
||||
47
.gsd/milestones/M014/slices/S06/tasks/T01-PLAN.md
Normal file
47
.gsd/milestones/M014/slices/S06/tasks/T01-PLAN.md
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
---
|
||||
estimated_steps: 17
|
||||
estimated_files: 2
|
||||
skills_used: []
|
||||
---
|
||||
|
||||
# T01: Add GET /admin/pipeline/technique-pages endpoint
|
||||
|
||||
Add a new admin endpoint that returns technique pages with aggregated source video counts, version counts, body_sections_format, and creator info. Supports filtering by multi_source_only (boolean) and creator (slug). This provides the data source for the admin technique pages UI.
|
||||
|
||||
The endpoint lives in `backend/routers/pipeline.py` alongside other admin endpoints. It queries TechniquePage with joins to Creator, counts from technique_page_videos and technique_page_versions, and returns a paginated response.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read `backend/routers/pipeline.py` to understand existing admin endpoint patterns (imports, auth, response style).
|
||||
2. Read `backend/schemas.py` to see existing schema patterns.
|
||||
3. Add a new Pydantic response schema `AdminTechniquePageItem` in `backend/schemas.py` with fields: id, title, slug, creator_name, creator_slug, topic_category, body_sections_format, source_video_count (int), version_count (int), created_at, updated_at. Add `AdminTechniquePageListResponse` with items list and total count.
|
||||
4. Add `GET /admin/pipeline/technique-pages` endpoint in `backend/routers/pipeline.py`. Query uses: `select(TechniquePage)` with `selectinload(TechniquePage.creator)`, correlated subqueries for source_video_count (count from technique_page_videos where technique_page_id matches) and version_count (count from technique_page_versions). Support query params: `multi_source_only: bool = False` (filter source_video_count > 1), `creator: str | None` (filter by creator slug), `sort: str = 'recent'` (recent/alpha/creator), `offset: int = 0`, `limit: int = 50`.
|
||||
5. Test endpoint manually via curl against the running API on ub01.
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] Endpoint returns correct source_video_count and version_count per page
|
||||
- [ ] multi_source_only=true filters to pages with >1 source video
|
||||
- [ ] creator filter works by slug
|
||||
- [ ] Response includes body_sections_format field
|
||||
- [ ] Pagination works (offset/limit/total)
|
||||
|
||||
## Verification
|
||||
|
||||
- `ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && PYTHONPATH=backend python -c "from schemas import AdminTechniquePageItem, AdminTechniquePageListResponse; print(AdminTechniquePageItem.model_fields.keys())"'` exits 0
|
||||
- After deploying: `curl -s http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -m json.tool | head -30` returns valid JSON with items array containing source_video_count and version_count fields
|
||||
|
||||
## Inputs
|
||||
|
||||
- ``backend/routers/pipeline.py` — existing admin endpoint patterns`
|
||||
- ``backend/schemas.py` — existing Pydantic schema patterns`
|
||||
- ``backend/models.py` — TechniquePage, TechniquePageVideo, TechniquePageVersion models`
|
||||
|
||||
## Expected Output
|
||||
|
||||
- ``backend/routers/pipeline.py` — new GET /admin/pipeline/technique-pages endpoint`
|
||||
- ``backend/schemas.py` — new AdminTechniquePageItem and AdminTechniquePageListResponse schemas`
|
||||
|
||||
## Verification
|
||||
|
||||
ssh ub01 'cd /vmPool/r/repos/xpltdco/chrysopedia && PYTHONPATH=backend python -c "from schemas import AdminTechniquePageItem; print(AdminTechniquePageItem.model_fields.keys())"' && curl -sf http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -c 'import json,sys; d=json.load(sys.stdin); assert "items" in d; print(f"OK: {d["total"]} pages")'
|
||||
81
.gsd/milestones/M014/slices/S06/tasks/T01-SUMMARY.md
Normal file
81
.gsd/milestones/M014/slices/S06/tasks/T01-SUMMARY.md
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
---
|
||||
id: T01
|
||||
parent: S06
|
||||
milestone: M014
|
||||
provides: []
|
||||
requires: []
|
||||
affects: []
|
||||
key_files: ["backend/routers/pipeline.py", "backend/schemas.py"]
|
||||
key_decisions: ["Used correlated scalar subqueries for count fields rather than joins with GROUP BY for cleaner filter composition"]
|
||||
patterns_established: []
|
||||
drill_down_paths: []
|
||||
observability_surfaces: []
|
||||
duration: ""
|
||||
verification_result: "Schema import verified inside Docker container. Endpoint returns valid JSON with 20 pages. multi_source_only=true returns 0 (correct — empty association table). creator=skope returns 2. Alpha sort returns correctly ordered titles. Pagination with offset=5&limit=2 returns 2 items with total=20."
|
||||
completed_at: 2026-04-03T01:55:31.738Z
|
||||
blocker_discovered: false
|
||||
---
|
||||
|
||||
# T01: Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters
|
||||
|
||||
> Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters
|
||||
|
||||
## What Happened
|
||||
---
|
||||
id: T01
|
||||
parent: S06
|
||||
milestone: M014
|
||||
key_files:
|
||||
- backend/routers/pipeline.py
|
||||
- backend/schemas.py
|
||||
key_decisions:
|
||||
- Used correlated scalar subqueries for count fields rather than joins with GROUP BY for cleaner filter composition
|
||||
duration: ""
|
||||
verification_result: passed
|
||||
completed_at: 2026-04-03T01:55:31.739Z
|
||||
blocker_discovered: false
|
||||
---
|
||||
|
||||
# T01: Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters
|
||||
|
||||
**Added paginated GET /admin/pipeline/technique-pages endpoint with source video counts, version counts, and multi-source/creator/sort filters**
|
||||
|
||||
## What Happened
|
||||
|
||||
Added AdminTechniquePageItem and AdminTechniquePageListResponse Pydantic schemas to backend/schemas.py. Added the endpoint in backend/routers/pipeline.py using correlated scalar subqueries against technique_page_videos and technique_page_versions for count fields. Supports multi_source_only, creator slug filter, sort (recent/alpha/creator), and offset/limit pagination. Deployed to ub01 via docker compose rebuild and verified all filters and pagination work correctly.
|
||||
|
||||
## Verification
|
||||
|
||||
Schema import verified inside Docker container. Endpoint returns valid JSON with 20 pages. multi_source_only=true returns 0 (correct — empty association table). creator=skope returns 2. Alpha sort returns correctly ordered titles. Pagination with offset=5&limit=2 returns 2 items with total=20.
|
||||
|
||||
## Verification Evidence
|
||||
|
||||
| # | Command | Exit Code | Verdict | Duration |
|
||||
|---|---------|-----------|---------|----------|
|
||||
| 1 | `docker exec chrysopedia-api python -c 'from schemas import AdminTechniquePageItem...'` | 0 | ✅ pass | 1000ms |
|
||||
| 2 | `curl -sf http://ub01:8096/api/v1/admin/pipeline/technique-pages | python3 -c 'assert items in d'` | 0 | ✅ pass | 1000ms |
|
||||
| 3 | `curl -sf ...?multi_source_only=true` | 0 | ✅ pass | 1000ms |
|
||||
| 4 | `curl -sf ...?creator=skope` | 0 | ✅ pass | 1000ms |
|
||||
| 5 | `curl -sf ...?sort=alpha&limit=3` | 0 | ✅ pass | 1000ms |
|
||||
| 6 | `curl -sf ...?offset=5&limit=2` | 0 | ✅ pass | 1000ms |
|
||||
|
||||
|
||||
## Deviations
|
||||
|
||||
Verified schema import inside Docker rather than on host (pydantic not installed on host).
|
||||
|
||||
## Known Issues
|
||||
|
||||
technique_page_videos and technique_page_versions tables are empty so counts are always 0. Queries are correct and will return real counts when populated.
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `backend/routers/pipeline.py`
|
||||
- `backend/schemas.py`
|
||||
|
||||
|
||||
## Deviations
|
||||
Verified schema import inside Docker rather than on host (pydantic not installed on host).
|
||||
|
||||
## Known Issues
|
||||
technique_page_videos and technique_page_versions tables are empty so counts are always 0. Queries are correct and will return real counts when populated.
|
||||
62
.gsd/milestones/M014/slices/S06/tasks/T02-PLAN.md
Normal file
62
.gsd/milestones/M014/slices/S06/tasks/T02-PLAN.md
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
---
|
||||
estimated_steps: 28
|
||||
estimated_files: 4
|
||||
skills_used: []
|
||||
---
|
||||
|
||||
# T02: Build AdminTechniquePages page with route and dropdown entry
|
||||
|
||||
Create the frontend page for admin technique page management. Table showing technique pages with source video counts, version counts, format badges, and creator. Expandable rows show source videos list (from existing `/techniques/{slug}` detail endpoint) and link to version history. Filters for multi-source only and creator. Cross-links to pipeline admin and public technique pages.
|
||||
|
||||
Follows patterns from `AdminPipeline.tsx` (table layout, expandable rows, badges, fetch pattern).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read `frontend/src/pages/AdminPipeline.tsx` (first ~100 lines) for fetch patterns, state management, table structure.
|
||||
2. Read `frontend/src/api/public-client.ts` for the `request<T>` pattern and existing type exports.
|
||||
3. Add TypeScript types and fetch function to `frontend/src/api/public-client.ts`: `AdminTechniquePageItem` interface (id, title, slug, creator_name, creator_slug, topic_category, body_sections_format, source_video_count, version_count, created_at, updated_at) and `fetchAdminTechniquePages(params)` function calling `GET /admin/pipeline/technique-pages`.
|
||||
4. Create `frontend/src/pages/AdminTechniquePages.tsx`:
|
||||
- State: items array, loading, error, filters (multiSourceOnly boolean, creatorFilter string), expandedSlug
|
||||
- Fetch on mount and filter change using the new client function
|
||||
- Table columns: Title (link to public page), Creator (link to creator detail), Category, Format (v1/v2 badge), Sources (count), Versions (count), Updated
|
||||
- Click row to expand → fetch technique detail via existing `fetchTechnique(slug)` → show source videos list with filenames and added_at dates, link to each video in pipeline admin via `/admin/pipeline?video={id}`
|
||||
- Filter bar: "Multi-source only" toggle checkbox, creator text filter
|
||||
- Use existing CSS patterns from AdminPipeline (admin-page class, table styles)
|
||||
5. Add route in `frontend/src/App.tsx`: `<Route path="/admin/techniques" element={<AdminTechniquePages />} />`
|
||||
6. Add "Techniques" link in `frontend/src/components/AdminDropdown.tsx` menu items.
|
||||
7. Build frontend to verify no TypeScript errors.
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] Table renders with all columns (title, creator, category, format, sources, versions, updated)
|
||||
- [ ] Row expand shows source videos with filenames and dates
|
||||
- [ ] Multi-source filter toggle works
|
||||
- [ ] Cross-link to pipeline admin works (via video ID query param)
|
||||
- [ ] Cross-link to public technique page works
|
||||
- [ ] Route `/admin/techniques` works
|
||||
- [ ] Admin dropdown includes Techniques entry
|
||||
|
||||
## Verification
|
||||
|
||||
- `cd frontend && npm run build` exits 0 (no TypeScript errors)
|
||||
- After deploying: navigate to `http://ub01:8096/admin/techniques` in browser, page renders with technique data
|
||||
- Admin dropdown shows three items: Reports, Pipeline, Techniques
|
||||
|
||||
## Inputs
|
||||
|
||||
- ``frontend/src/api/public-client.ts` — existing request<T> pattern and types`
|
||||
- ``frontend/src/pages/AdminPipeline.tsx` — reference for admin page patterns (table, expand, badges)`
|
||||
- ``frontend/src/components/AdminDropdown.tsx` — dropdown menu to add entry to`
|
||||
- ``frontend/src/App.tsx` — route definitions`
|
||||
- ``backend/schemas.py` — AdminTechniquePageItem schema from T01`
|
||||
|
||||
## Expected Output
|
||||
|
||||
- ``frontend/src/pages/AdminTechniquePages.tsx` — new admin technique pages page`
|
||||
- ``frontend/src/api/public-client.ts` — AdminTechniquePageItem type + fetchAdminTechniquePages function`
|
||||
- ``frontend/src/App.tsx` — new /admin/techniques route`
|
||||
- ``frontend/src/components/AdminDropdown.tsx` — Techniques menu entry added`
|
||||
|
||||
## Verification
|
||||
|
||||
cd frontend && npm run build 2>&1 | tail -5
|
||||
|
|
@ -22,9 +22,9 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
|||
|
||||
from config import get_settings
|
||||
from database import get_session
|
||||
from models import PipelineEvent, PipelineRun, PipelineRunStatus, SourceVideo, Creator, KeyMoment, TranscriptSegment, ProcessingStatus
|
||||
from models import PipelineEvent, PipelineRun, PipelineRunStatus, SourceVideo, Creator, KeyMoment, TranscriptSegment, ProcessingStatus, TechniquePage, TechniquePageVideo, TechniquePageVersion
|
||||
from redis_client import get_redis
|
||||
from schemas import DebugModeResponse, DebugModeUpdate, TokenStageSummary, TokenSummaryResponse
|
||||
from schemas import DebugModeResponse, DebugModeUpdate, TokenStageSummary, TokenSummaryResponse, AdminTechniquePageItem, AdminTechniquePageListResponse
|
||||
|
||||
logger = logging.getLogger("chrysopedia.pipeline")
|
||||
|
||||
|
|
@ -188,6 +188,108 @@ async def list_pipeline_videos(
|
|||
}
|
||||
|
||||
|
||||
# ── Admin: Technique Pages ───────────────────────────────────────────────────
|
||||
|
||||
@router.get(
|
||||
"/admin/pipeline/technique-pages",
|
||||
response_model=AdminTechniquePageListResponse,
|
||||
)
|
||||
async def list_admin_technique_pages(
|
||||
multi_source_only: bool = False,
|
||||
creator: Annotated[str | None, Query(description="Filter by creator slug")] = None,
|
||||
sort: Annotated[str, Query(description="Sort: recent, alpha, creator")] = "recent",
|
||||
offset: Annotated[int, Query(ge=0)] = 0,
|
||||
limit: Annotated[int, Query(ge=1, le=200)] = 50,
|
||||
db: AsyncSession = Depends(get_session),
|
||||
):
|
||||
"""List technique pages with source video counts, version counts, and creator info.
|
||||
|
||||
Supports filtering by multi-source pages only and by creator slug.
|
||||
"""
|
||||
# Correlated subquery: source video count per page
|
||||
video_count_sq = (
|
||||
select(func.count())
|
||||
.select_from(TechniquePageVideo)
|
||||
.where(TechniquePageVideo.technique_page_id == TechniquePage.id)
|
||||
.correlate(TechniquePage)
|
||||
.scalar_subquery()
|
||||
.label("source_video_count")
|
||||
)
|
||||
|
||||
# Correlated subquery: version count per page
|
||||
version_count_sq = (
|
||||
select(func.count())
|
||||
.select_from(TechniquePageVersion)
|
||||
.where(TechniquePageVersion.technique_page_id == TechniquePage.id)
|
||||
.correlate(TechniquePage)
|
||||
.scalar_subquery()
|
||||
.label("version_count")
|
||||
)
|
||||
|
||||
stmt = (
|
||||
select(
|
||||
TechniquePage.id,
|
||||
TechniquePage.title,
|
||||
TechniquePage.slug,
|
||||
TechniquePage.topic_category,
|
||||
TechniquePage.body_sections_format,
|
||||
TechniquePage.created_at,
|
||||
TechniquePage.updated_at,
|
||||
Creator.name.label("creator_name"),
|
||||
Creator.slug.label("creator_slug"),
|
||||
video_count_sq,
|
||||
version_count_sq,
|
||||
)
|
||||
.join(Creator, TechniquePage.creator_id == Creator.id)
|
||||
)
|
||||
|
||||
# Filters
|
||||
if multi_source_only:
|
||||
stmt = stmt.where(video_count_sq > 1)
|
||||
if creator:
|
||||
stmt = stmt.where(Creator.slug == creator)
|
||||
|
||||
# Count total before pagination
|
||||
count_stmt = select(func.count()).select_from(stmt.subquery())
|
||||
total = (await db.execute(count_stmt)).scalar() or 0
|
||||
|
||||
# Sort
|
||||
if sort == "alpha":
|
||||
stmt = stmt.order_by(TechniquePage.title.asc())
|
||||
elif sort == "creator":
|
||||
stmt = stmt.order_by(Creator.name.asc(), TechniquePage.title.asc())
|
||||
else: # "recent" default
|
||||
stmt = stmt.order_by(TechniquePage.updated_at.desc())
|
||||
|
||||
stmt = stmt.offset(offset).limit(limit)
|
||||
result = await db.execute(stmt)
|
||||
rows = result.all()
|
||||
|
||||
items = [
|
||||
AdminTechniquePageItem(
|
||||
id=r.id,
|
||||
title=r.title,
|
||||
slug=r.slug,
|
||||
creator_name=r.creator_name,
|
||||
creator_slug=r.creator_slug,
|
||||
topic_category=r.topic_category,
|
||||
body_sections_format=r.body_sections_format,
|
||||
source_video_count=r.source_video_count or 0,
|
||||
version_count=r.version_count or 0,
|
||||
created_at=r.created_at,
|
||||
updated_at=r.updated_at,
|
||||
)
|
||||
for r in rows
|
||||
]
|
||||
|
||||
return AdminTechniquePageListResponse(
|
||||
items=items,
|
||||
total=total,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
)
|
||||
|
||||
|
||||
# ── Admin: Retrigger ─────────────────────────────────────────────────────────
|
||||
|
||||
@router.post("/admin/pipeline/trigger/{video_id}")
|
||||
|
|
|
|||
|
|
@ -432,3 +432,30 @@ class TokenSummaryResponse(BaseModel):
|
|||
video_id: str
|
||||
stages: list[TokenStageSummary] = Field(default_factory=list)
|
||||
grand_total_tokens: int
|
||||
|
||||
|
||||
# ── Admin: Technique Pages ───────────────────────────────────────────────────
|
||||
|
||||
class AdminTechniquePageItem(BaseModel):
|
||||
"""Technique page with aggregated source/version counts for admin view."""
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: uuid.UUID
|
||||
title: str
|
||||
slug: str
|
||||
creator_name: str
|
||||
creator_slug: str
|
||||
topic_category: str
|
||||
body_sections_format: str
|
||||
source_video_count: int = 0
|
||||
version_count: int = 0
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class AdminTechniquePageListResponse(BaseModel):
|
||||
"""Paginated list of technique pages for admin view."""
|
||||
items: list[AdminTechniquePageItem] = Field(default_factory=list)
|
||||
total: int = 0
|
||||
offset: int = 0
|
||||
limit: int = 50
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue