diff --git a/.gsd/milestones/M021/M021-ROADMAP.md b/.gsd/milestones/M021/M021-ROADMAP.md index 5534dd1..bb19a24 100644 --- a/.gsd/milestones/M021/M021-ROADMAP.md +++ b/.gsd/milestones/M021/M021-ROADMAP.md @@ -10,7 +10,7 @@ LightRAG becomes the primary search engine. Chat engine goes live (encyclopedic | S02 | [B] Creator-Scoped Retrieval Cascade | medium | S01 | ✅ | Question on Keota's profile first checks Keota's content, then sound design domain, then full KB, then graceful fallback | | S03 | [B] Chat Engine MVP | high | S02 | ✅ | User asks a question, receives a streamed response with citations linking to source videos and technique pages | | S04 | [B] Highlight Detection v1 | medium | — | ✅ | Scored highlight candidates generated from existing pipeline data for a sample of videos | -| S05 | [A] Audio Mode + Chapter Markers | medium | — | ⬜ | Media player with waveform visualization in audio mode and chapter markers on the timeline | +| S05 | [A] Audio Mode + Chapter Markers | medium | — | ✅ | Media player with waveform visualization in audio mode and chapter markers on the timeline | | S06 | [A] Auto-Chapters Review UI | low | — | ⬜ | Creator reviews detected chapters: drag boundaries, rename, reorder, approve for publication | | S07 | [A] Impersonation Polish + Write Mode | low | — | ⬜ | Impersonation write mode with confirmation modal. Audit log admin view shows all sessions. | | S08 | Forgejo KB Update — Chat, Retrieval, Highlights | low | S01, S02, S03, S04, S05, S06, S07 | ⬜ | Forgejo wiki updated with chat engine, retrieval routing, and highlight detection docs | diff --git a/.gsd/milestones/M021/slices/S05/S05-SUMMARY.md b/.gsd/milestones/M021/slices/S05/S05-SUMMARY.md new file mode 100644 index 0000000..2b6f0ee --- /dev/null +++ b/.gsd/milestones/M021/slices/S05/S05-SUMMARY.md @@ -0,0 +1,114 @@ +--- +id: S05 +parent: M021 +milestone: M021 +provides: + - GET /videos/{video_id}/stream endpoint for media file serving + - GET /videos/{video_id}/chapters endpoint returning KeyMoment-based chapter data + - fetchChapters() frontend API client function + - AudioWaveform component for waveform visualization + - ChapterMarkers component for seek bar chapter overlay + - HTMLMediaElement-widened useMediaSync hook +requires: + [] +affects: + - S06 +key_files: + - backend/routers/videos.py + - backend/schemas.py + - frontend/src/api/videos.ts + - frontend/src/components/AudioWaveform.tsx + - frontend/src/components/ChapterMarkers.tsx + - frontend/src/components/PlayerControls.tsx + - frontend/src/hooks/useMediaSync.ts + - frontend/src/pages/WatchPage.tsx + - frontend/src/App.css +key_decisions: + - Stream endpoint uses FileResponse with mimetypes.guess_type for content-type detection + - Chapters endpoint maps KeyMoment records directly to ChapterMarkerRead schema + - wavesurfer.js uses MediaElement backend with shared audio ref so useMediaSync controls playback identically to video mode + - useMediaSync widened from HTMLVideoElement to HTMLMediaElement for audio/video polymorphism + - Chapter ticks use button elements for keyboard accessibility + - RegionsPlugin registered at WaveSurfer creation, regions added on ready event +patterns_established: + - Conditional media rendering pattern: WatchPage checks video_url to decide AudioWaveform vs VideoPlayer + - HTMLMediaElement polymorphism: useMediaSync works for both audio and video elements without code changes + - Non-critical data fetching: chapters are fetched with silent error catching so page works even if chapters fail +observability_surfaces: + - none +drill_down_paths: + - .gsd/milestones/M021/slices/S05/tasks/T01-SUMMARY.md + - .gsd/milestones/M021/slices/S05/tasks/T02-SUMMARY.md + - .gsd/milestones/M021/slices/S05/tasks/T03-SUMMARY.md +duration: "" +verification_result: passed +completed_at: 2026-04-04T05:54:51.869Z +blocker_discovered: false +--- + +# S05: [A] Audio Mode + Chapter Markers + +**Media player renders waveform visualization for audio-only content and chapter markers on the timeline derived from KeyMoment data.** + +## What Happened + +This slice added three capabilities across backend and frontend: + +**T01 — Backend endpoints + API client:** Added `GET /videos/{video_id}/stream` (serves media files via FileResponse with MIME type detection) and `GET /videos/{video_id}/chapters` (returns KeyMoment records as chapter markers sorted by start_time). Added `ChapterMarkerRead` and `ChaptersResponse` Pydantic schemas. Added `fetchChapters()` to the frontend API client. + +**T02 — Audio waveform rendering:** Installed wavesurfer.js. Widened `useMediaSync` ref type from `HTMLVideoElement` to `HTMLMediaElement` so the same playback hook works for both audio and video. Created `AudioWaveform.tsx` that renders a hidden `