---
estimated_steps: 29
estimated_files: 5
skills_used: []
---

# T01: Add debug columns, migration, debug toggle endpoints, and token summary API

Set up all infrastructure for debug mode: Alembic migration adding 3 TEXT columns to pipeline_events, PipelineEvent model update, debug mode GET/PUT endpoints (Redis-backed, following review_mode pattern from D007), token summary aggregation endpoint, and extended event listing response.

All work is on ub01 via SSH at /vmPool/r/repos/xpltdco/chrysopedia.

## Steps

1. Add `system_prompt_text`, `user_prompt_text`, `response_text` columns (all `Text, nullable=True`) to `PipelineEvent` model in `backend/models.py` after the existing `payload` column.

2. Create Alembic migration `alembic/versions/006_debug_columns.py` that adds the 3 TEXT columns to `pipeline_events`. Use `op.add_column()` with `nullable=True` — no data rewrite needed.

3. Add `debug_mode: bool = False` to `Settings` class in `backend/config.py`.

4. Add Pydantic schemas in `backend/schemas.py`:
   - `DebugModeResponse(BaseModel)` with `debug_mode: bool`
   - `DebugModeUpdate(BaseModel)` with `debug_mode: bool`
   - `TokenStageSummary(BaseModel)` with `stage: str`, `call_count: int`, `total_prompt_tokens: int`, `total_completion_tokens: int`, `total_tokens: int`
   - `TokenSummaryResponse(BaseModel)` with `video_id: str`, `stages: list[TokenStageSummary]`, `grand_total_tokens: int`

5. Add 3 new endpoints in `backend/routers/pipeline.py`:
   - `GET /admin/pipeline/debug-mode` — read from Redis key `chrysopedia:debug_mode`, fall back to `settings.debug_mode`. Follow exact pattern from review router's `get_mode()`.
   - `PUT /admin/pipeline/debug-mode` — write to Redis. Follow exact pattern from review router's `set_mode()`.
   - `GET /admin/pipeline/token-summary/{video_id}` — query `pipeline_events` grouped by stage where `event_type='llm_call'`, sum token columns, return `TokenSummaryResponse`.

6. Extend the existing `list_pipeline_events` response dict to include `system_prompt_text`, `user_prompt_text`, `response_text` fields (nullable).

7. Run migration: `docker exec chrysopedia-api alembic upgrade head`. Rebuild API container if needed: `docker compose build chrysopedia-api && docker compose up -d chrysopedia-api`.

8. Verify: curl debug-mode GET returns `{"debug_mode": false}`, PUT sets to true, GET reads back true. Curl token-summary for any video_id returns valid (possibly empty) response.

## Must-Haves

- [ ] 3 TEXT columns added to pipeline_events table via migration
- [ ] PipelineEvent model has matching mapped columns
- [ ] Debug mode GET/PUT endpoints work with Redis (fall back to config)
- [ ] Token summary endpoint returns per-stage aggregation
- [ ] Event listing includes new fields

## Failure Modes

| Dependency | On error | On timeout | On malformed response |
|------------|----------|-----------|----------------------|
| Redis (debug mode read) | Fall back to config.debug_mode default | Same as error — use default | N/A (simple GET/SET of string) |
| PostgreSQL (token summary) | Return 500 with error detail | Standard DB timeout handling | N/A (aggregation query) |

## Inputs

- ``backend/models.py` — existing PipelineEvent model (line ~386)`
- ``backend/config.py` — existing Settings class with review_mode pattern`
- ``backend/schemas.py` — existing ReviewModeResponse/Update as template`
- ``backend/routers/pipeline.py` — existing pipeline endpoints and list_pipeline_events`
- ``backend/routers/review.py` — review_mode GET/PUT pattern to follow (lines 337-375)`
- ``alembic/versions/` — existing migrations for revision chain`

## Expected Output

- ``alembic/versions/006_debug_columns.py` — migration adding 3 TEXT columns`
- ``backend/models.py` — PipelineEvent with system_prompt_text, user_prompt_text, response_text columns`
- ``backend/config.py` — Settings with debug_mode field`
- ``backend/schemas.py` — DebugModeResponse, DebugModeUpdate, TokenStageSummary, TokenSummaryResponse`
- ``backend/routers/pipeline.py` — 3 new endpoints + extended event listing`

## Verification

ssh ub01 'docker exec chrysopedia-api alembic upgrade head' && curl -sf http://ub01:8096/admin/pipeline/debug-mode | grep debug_mode
