# S01: Pipeline Debug Mode — Full LLM I/O Capture and Token Accounting — UAT

**Milestone:** M007
**Written:** 2026-03-30T18:57:12.919Z

## UAT: Pipeline Debug Mode — Full LLM I/O Capture and Token Accounting

### Preconditions
- Chrysopedia stack running on ub01 (docker ps shows chrysopedia-api, chrysopedia-worker, chrysopedia-db, chrysopedia-redis healthy)
- At least one video with completed pipeline events in database
- SSH access to ub01

### Test 1: Debug Mode Toggle Round-Trip

1. GET debug mode state: `curl -sf http://ub01:8096/api/v1/admin/pipeline/debug-mode`
   - **Expected:** JSON with `debug_mode` boolean field
2. PUT debug mode to true: `curl -sf -X PUT http://ub01:8096/api/v1/admin/pipeline/debug-mode -H 'Content-Type: application/json' -d '{"debug_mode":true}'`
   - **Expected:** `{"debug_mode": true}`
3. GET debug mode again
   - **Expected:** `{"debug_mode": true}` — persisted in Redis
4. PUT debug mode to false
   - **Expected:** `{"debug_mode": false}`
5. GET debug mode again
   - **Expected:** `{"debug_mode": false}`

### Test 2: Token Summary for Existing Video

1. Pick a video_id with pipeline events (e.g., one shown in admin pipeline page)
2. GET token summary: `curl -sf http://ub01:8096/api/v1/admin/pipeline/token-summary/{video_id}`
   - **Expected:** JSON with `video_id`, `stages` array (each with stage name, call_count, total_prompt_tokens, total_completion_tokens, total_tokens), and `grand_total_tokens`
   - **Expected:** grand_total_tokens equals sum of all stage total_tokens

### Test 3: Token Summary for Non-Existent Video

1. GET token summary with fake ID: `curl -sf http://ub01:8096/api/v1/admin/pipeline/token-summary/00000000-0000-0000-0000-000000000000`
   - **Expected:** `{"video_id": "00000000-...", "stages": [], "grand_total_tokens": 0}` — empty, not 404

### Test 4: End-to-End Debug Capture

1. Enable debug mode: PUT `{"debug_mode": true}`
2. Trigger pipeline for a video: `curl -X POST http://ub01:8096/api/v1/admin/pipeline/trigger/{video_id}`
3. Wait for pipeline to complete (check events or worker logs)
4. GET events: `curl -sf http://ub01:8096/api/v1/admin/pipeline/events/{video_id}`
5. Find an `llm_call` event in the items array
   - **Expected:** `system_prompt_text` is a non-empty string (the system prompt sent to the LLM)
   - **Expected:** `user_prompt_text` is a non-empty string (the user prompt with transcript content)
   - **Expected:** `response_text` is a non-empty string (the LLM's response)
6. Find a `start` or `complete` event
   - **Expected:** `system_prompt_text`, `user_prompt_text`, `response_text` are all null (only llm_call events carry debug data)

### Test 5: Debug Off Produces No Capture

1. Disable debug mode: PUT `{"debug_mode": false}`
2. Trigger pipeline for a different video (or re-trigger same one)
3. Wait for pipeline to complete
4. GET events for that video, find llm_call events
   - **Expected:** `system_prompt_text`, `user_prompt_text`, `response_text` are all null
   - **Expected:** Other event fields (stage, event_type, prompt_tokens, completion_tokens, payload) are still populated normally

### Test 6: Event Listing Includes Debug Fields

1. GET events for any video: `curl -sf http://ub01:8096/api/v1/admin/pipeline/events/{video_id}`
   - **Expected:** Each event object in `items` array has keys `system_prompt_text`, `user_prompt_text`, `response_text` (may be null)
   - **Expected:** Response shape is `{items: [...], total: N, offset: 0, limit: N}`

### Edge Cases

- **Redis down during debug mode read:** _is_debug_mode() silently falls back to config.debug_mode (default false). Pipeline continues without capturing I/O.
- **Debug mode toggled mid-pipeline:** Events before toggle use old state, events after use new state. The check happens once per _make_llm_callback creation (once per stage invocation), not per LLM call within a stage.
- **Very large prompts:** No truncation — full text stored. Monitor pipeline_events table size if debug is left on for many pipeline runs.
