# S01 Research: Web Media Player MVP

## Summary

This slice adds a custom video player with HLS playback, speed controls, and synchronized transcript sidebar to the Chrysopedia frontend. The work is **high-risk** because the required video infrastructure (file storage, transcoding, streaming) does not exist yet — the system currently processes only transcript JSON files.

### Key Findings

1. **No video files on the server.** `/vmPool/r/services/chrysopedia_data/` contains `transcripts/` and `video_meta/` but zero MP4/MKV files. `SourceVideo.file_path` stores relative paths like `Chee/01 Day 1 - Drums - Part 1.mp4` but these are metadata references — the actual videos live elsewhere (likely the source machine used for Whisper transcription).

2. **No HLS infrastructure.** ffmpeg is not installed on ub01 or in any container. No HLS transcoding pipeline exists. No `.m3u8` playlists or `.ts` segments exist.

3. **No media serving endpoint.** The nginx config only proxies `/api/` to the FastAPI backend and serves the SPA. There's no `/media/` location for video files.

4. **MinIO is planned but not deployed.** D035 chose MinIO for file/object storage but the compose stack doesn't include a MinIO service yet.

5. **Rich transcript data exists.** 206,767 transcript segments across 383 videos with `start_time`, `end_time`, `text` fields. Word-level timing is in the raw JSON files. KeyMoments have `raw_transcript` text and `start_time`/`end_time` timestamps.

6. **Frontend has no media dependencies.** `package.json` dependencies: react, react-dom, react-router-dom. No video player library.

## Recommendation

**Phased approach — build the player UI now against a mock/demo video, wire real video infrastructure separately.**

The player component, transcript sync, speed controls, and UI integration are substantial frontend work independent of video storage. Build and verify the player components against a single test video (or a publicly available HLS test stream), then wire real video serving when MinIO + ffmpeg transcoding land.

### Phase 1 (this slice): Player UI + Transcript Sync
- New `VideoPlayer` component using **hls.js** for HLS playback, falling back to native `<video>` for MP4
- Custom controls: play/pause, seek bar, speed selector (0.5x, 0.75x, 1x, 1.25x, 1.5x, 2x), volume, fullscreen
- `TranscriptSidebar` component: scrollable segment list synced to playback position, auto-scroll to active segment, click-to-seek
- New `/watch/:videoId` route with player + transcript layout
- Integration on `TechniquePage`: key moment timestamps become clickable → navigate to player at that timestamp
- New API endpoint: `GET /api/v1/videos/:id/transcript` returning segments for a video
- New API field: `video_url` on video responses (nullable, empty for now until MinIO is up)

### Phase 2 (separate slice): Video Storage + Transcoding
- MinIO service in docker-compose
- Upload endpoint or batch import for video files
- ffmpeg-based HLS transcoding (background task)
- `video_url` population with signed MinIO URLs

## Implementation Landscape

### New files needed

**Backend:**
- `backend/routers/videos.py` — extend with `GET /videos/{id}` (single video detail), `GET /videos/{id}/transcript` (transcript segments ordered by start_time)
- `backend/schemas.py` — add `video_url: str | None = None` to `SourceVideoRead`, add `TranscriptForPlayerResponse` schema

**Frontend:**
- `frontend/src/components/VideoPlayer.tsx` — HLS player component wrapping hls.js + HTML5 `<video>` with custom controls
- `frontend/src/components/TranscriptSidebar.tsx` — time-synced transcript display
- `frontend/src/components/PlayerControls.tsx` — custom playback controls (speed, seek, volume)
- `frontend/src/pages/WatchPage.tsx` — new route `/watch/:videoId` composing player + transcript
- `frontend/src/hooks/useMediaSync.ts` — shared state between player and transcript (currentTime, playing, playbackRate)
- `frontend/src/api/videos.ts` — API functions for video detail + transcript

**Config:**
- `frontend/package.json` — add `hls.js` dependency
- `docker/nginx.conf` — no changes needed yet (video serving deferred)

### Existing files modified

- `frontend/src/App.tsx` — add `/watch/:videoId` route
- `frontend/src/pages/TechniquePage.tsx` — make key moment timestamps clickable (link to `/watch/:videoId?t=startTime`)
- `frontend/src/api/techniques.ts` — no changes needed (KeyMomentSummary already has source_video_id)
- `frontend/src/App.css` — add player/transcript styles (or new CSS file)

### Key library: hls.js

- NPM: `hls.js` (current: v1.5.x)
- Zero dependencies, ~300KB gzipped
- Attaches to a native `<video>` element via MediaSource Extensions
- Pattern: `const hls = new Hls(); hls.loadSource(url); hls.attachMedia(videoEl);`
- Events: `MANIFEST_PARSED` (ready), `ERROR` (with `fatal` flag), `LEVEL_SWITCHED`
- Cleanup: `hls.destroy()` in React useEffect return
- Native HLS fallback: Safari supports HLS natively — check `video.canPlayType('application/vnd.apple.mpegstream')` and skip hls.js if true
- Speed control: use native `videoEl.playbackRate = rate`, works with hls.js attached

### Data flow for transcript sync

```
TranscriptSegment (DB)
  → GET /api/v1/videos/:id/transcript → [{start_time, end_time, text, segment_index}]
  → TranscriptSidebar component
  → useMediaSync hook tracks video.currentTime via timeupdate event (~4Hz)
  → Binary search on segments array to find active segment
  → Auto-scroll to active segment, highlight it
  → Click segment → video.currentTime = segment.start_time
```

### Video URL strategy (interim)

Until MinIO is deployed, `video_url` will be `null` for all videos. The player page should show a clear "Video not available" state when `video_url` is null. For development/demo, use a public HLS test stream (e.g., Apple's Bipbop stream or similar) so the player can be built and tested end-to-end.

Alternatively, the player can be tested with a local MP4 served via the API (FastAPI `FileResponse` from a test file). This is better for verifying transcript sync since we'd have matching timestamps.

### Seams for task decomposition

1. **Backend: Video detail + transcript endpoint** — standalone, testable API work
2. **Frontend: VideoPlayer component + hls.js integration** — can be built/tested with a test stream URL, independent of transcript
3. **Frontend: TranscriptSidebar + sync hook** — needs the transcript API, independent of actual video
4. **Frontend: WatchPage route + layout** — composes player + transcript
5. **Frontend: TechniquePage integration** — make timestamps clickable, link to watch page
6. **CSS/styling** — player chrome, transcript sidebar, responsive layout

### Risks and constraints

| Risk | Impact | Mitigation |
|------|--------|------------|
| No video files available for testing | Can't verify real HLS playback end-to-end | Use public HLS test stream for player testing, mock transcript data for sync testing |
| hls.js bundle size (~300KB) | Adds to initial load | Lazy-load via dynamic import on WatchPage (already have Suspense pattern) |
| Transcript segment count per video | Some videos have 500+ segments — could cause scroll performance issues | Virtualized list or windowed rendering if needed; start without and measure |
| Safari native HLS vs hls.js | Different code paths for Safari vs Chrome/Firefox | Standard hls.js pattern: check `canPlayType` first, fall back to hls.js |
| `timeupdate` event frequency | ~4Hz is enough for segment-level sync but choppy for word-level | Segment-level sync is the MVP; word-level highlighting is a follow-up |

## Don't Hand-Roll

- **Video player library:** Use `hls.js`, not a custom MSE implementation. It handles all HLS complexity (adaptive bitrate, buffer management, error recovery).
- **Scroll-into-view for transcript:** Use `Element.scrollIntoView({ behavior: 'smooth', block: 'nearest' })`, not manual scroll position calculation.
- **Binary search for active segment:** Use a proper binary search on the sorted segments array, not a linear scan on every timeupdate (O(log n) vs O(n) for 500+ segments).

## Sources

- hls.js GitHub: github.com/video-dev/hls.js — API docs confirm `attachMedia`, `loadSource`, events pattern
- Existing codebase: `backend/models.py` (SourceVideo, TranscriptSegment, KeyMoment models), `frontend/src/pages/TechniquePage.tsx` (key moment display pattern), `frontend/src/api/techniques.ts` (API client pattern)
- Infrastructure: `docker-compose.yml` (no media serving), `docker/nginx.conf` (API proxy only), `backend/config.py` (no video storage config)
