# S02 Research: Composition Prompt + Test Harness Compose Mode

## Summary

This slice adds the ability to compose new video content into existing technique pages — the core multi-source capability. Two deliverables: (1) a `stage5_compose.txt` prompt that instructs the LLM to merge new moments into an existing page, and (2) a `--compose` mode for the test harness that exercises this prompt offline.

The work is structurally parallel to S01 (prompt + test harness update + tests) but the LLM task is harder: composition requires deduplication, section merging, citation re-indexing, and preserving the quality of existing prose while integrating new material.

## Recommendation

**Targeted research.** The patterns are established by S01 (Pydantic schemas, test harness structure, citation utils). The new work is the composition prompt design and the harness plumbing to feed existing page + new moments to the LLM.

## Implementation Landscape

### What Exists

| File | Role | Relevance |
|------|------|-----------|
| `backend/pipeline/test_harness.py` | Offline synthesis test runner | Needs `compose` subcommand added alongside existing `run` and `promote` |
| `backend/pipeline/schemas.py` | `SynthesizedPage`, `BodySection`, `SynthesisResult` | Composition output should use the same `SynthesisResult` schema — no new output schema needed |
| `backend/pipeline/citation_utils.py` | `validate_citations()`, `extract_citations()` | Reused as-is for compose output validation |
| `backend/pipeline/test_harness_v2_format.py` | v2 format tests | Pattern to follow for compose-mode tests |
| `prompts/stage5_synthesis.txt` | v5 synthesis prompt (from S01) | Compose prompt references same voice/tone/structure rules but has different instructions |
| `backend/pipeline/stages.py` `_merge_pages_by_slug()` | Merges chunk-split pages by slug via LLM | Closest existing pattern. Uses `stage5_merge.txt` prompt (which doesn't exist on disk yet — it's referenced but never created). Compose is similar but merges new moments into an existing page rather than merging partial pages from the same video. |
| `backend/pipeline/llm_client.py` | `LLMClient.complete()`, `estimate_max_tokens()` | Used by test harness already; compose mode uses the same interface |
| `backend/pipeline/export_fixture.py` | Exports video moments to fixture JSON | May need extension to also export existing page JSON for compose fixtures |

### What Needs Building

#### 1. Composition Prompt (`prompts/stage5_compose.txt`)

The LLM receives:
- `<existing_page>`: Full JSON of the current `SynthesizedPage` (body_sections, citations, moment_indices, etc.)
- `<new_moments>`: The new moments from a second video, same format as synthesis input
- `<creator>`: Creator name

The prompt must instruct the LLM to:
- **Preserve existing prose quality** — don't rewrite sections that are fine
- **Integrate new material** — add to existing sections where new moments cover the same sub-topic, or create new sections for genuinely new territory
- **Deduplicate** — if new moments cover ground already in the page, enrich rather than repeat
- **Re-index citations** — existing citations [0]-[N] refer to the original moment list. New moments get indices [N+1]-[N+M]. The composed output must use a unified index space. The prompt must explain the indexing scheme clearly.
- **Maintain section ordering** — workflow-order rule from synthesis prompt still applies
- **Output format** — same `SynthesisResult` / `SynthesizedPage` with v2 body_sections

Key design decision: The compose prompt should receive the existing page's `moment_indices` mapped to their moment data so the LLM understands what's already covered. This means the user prompt includes both old moments (for context) and new moments (to integrate), with clear labeling.

#### 2. Test Harness Compose Mode

Add a `compose` subcommand to `test_harness.py`:

```
python -m pipeline.test_harness compose \
  --existing-page /tmp/existing_page.json \
  --fixture fixtures/new_video.json \
  --prompt prompts/stage5_compose.txt \
  --output /tmp/composed.json
```

Arguments:
- `--existing-page`: JSON file containing a single `SynthesizedPage` (output from a prior synthesis run)
- `--fixture`: New moments fixture (same format as `run` mode)
- `--category`: Optional filter (only compose for matching category)
- `--prompt`: Compose prompt file (defaults to `stage5_compose.txt`)
- `--output`: Output path
- `--model`, `--modality`: LLM overrides (same as `run`)

The compose runner needs to:
1. Load the existing page JSON and validate it as `SynthesizedPage`
2. Load the new fixture moments (same as `run` mode)
3. Filter new moments by category to match the existing page's `topic_category`
4. Build a user prompt with `<existing_page>`, `<new_moments>`, and `<creator>` tags
5. Call the LLM with the compose system prompt
6. Parse the response as `SynthesisResult` (the LLM returns an updated version of the page)
7. Run citation validation on the output, with moment_count = old_moments + new_moments
8. Log deduplication metrics: how many old sections survived, how many new sections appeared, word count delta

#### 3. Tests (`backend/pipeline/test_harness_compose.py`)

Unit tests for compose-specific logic (no LLM calls):
- User prompt construction: existing page + new moments formatted correctly
- Citation re-indexing math: old moments 0-4, new moments 5-9, validate combined
- Word count comparison: before/after composition
- Category matching: only compose moments matching the page's topic_category
- Edge cases: empty new moments, single new moment, existing page with no subsections

### Seam Analysis

The work divides cleanly into three tasks:

**T01: Composition prompt** — Write `prompts/stage5_compose.txt`. Self-contained file. Can reference the synthesis prompt's voice/tone section but must have its own instructions for merge semantics, deduplication, and citation re-indexing. Verifiable by structural checks (required sections present, example output valid).

**T02: Test harness compose subcommand** — Add `compose` subcommand to `test_harness.py`. Depends on T01 only for the default prompt path. The harness code itself is pure Python with no LLM dependency in tests. Needs a helper to build the compose user prompt (existing page JSON + new moments text).

**T03: Compose-mode tests** — Unit tests for the compose plumbing. Depends on T02 code existing. Tests the prompt construction, citation math, category filtering, and word count diffing. No LLM calls.

### Citation Re-indexing Strategy

This is the trickiest part. In compose mode:
- The existing page has citations [0]-[N-1] referring to original moments
- New moments are numbered [N]-[N+M-1] in the unified space
- The LLM needs to understand both sets so it can:
  - Preserve existing [0]-[N-1] citations in untouched prose
  - Add [N]-[N+M-1] citations in new/updated prose
  - Potentially add cross-references (new moment enriches an existing section → add [N+k] alongside existing [j])

The user prompt should present:
```
<existing_moments>
[0] Title: ... (already cited in existing page)
[1] Title: ...
...
[N-1] Title: ...
</existing_moments>

<new_moments>
[N] Title: ... (new content to integrate)
[N+1] Title: ...
...
[N+M-1] Title: ...
</new_moments>

<existing_page>
{full page JSON}
</existing_page>
```

This way the LLM sees the complete index space and can cite any moment by its global index.

### Deduplication Signal

The prompt should instruct the LLM to compare new moment summaries against existing section content. If a new moment covers material already present:
- Don't add a new section
- Optionally add the new moment's citation to the existing passage if it provides a second source
- Only add new prose if the new moment contains genuinely new detail (different settings, different context, additional nuance)

This is a judgment call for the LLM — the prompt needs clear examples of "enriching" vs "duplicating."

### Risks

1. **LLM may rewrite too aggressively** — Given the full page, the LLM might want to rewrite everything. The prompt must be explicit: preserve existing prose verbatim where it's correct, only modify sections where new information changes or enriches them.
2. **Citation index confusion** — The LLM needs to correctly distinguish old indices from new ones. Clear labeling in the user prompt is critical.
3. **Output size** — Composed pages are larger (more moments, more sections). Token limits may need adjustment. `estimate_max_tokens` should handle this since it's based on input size.

### Don't Hand-Roll

- Citation validation: use existing `validate_citations()` from `citation_utils.py`
- Word counting: use the same pattern from `test_harness_v2_format.py`
- LLM interaction: use existing `LLMClient.complete()` and `estimate_max_tokens()`
- Schema validation: use existing `SynthesisResult.model_validate_json()`

### Verification Strategy

- **T01 (prompt):** Structural assertions — required sections present, example output parses as valid SynthesisResult, citation rules section explains the dual-index scheme
- **T02 (harness):** `python -m pipeline.test_harness compose --help` exits 0; compose user prompt builder produces expected format with existing page and new moments
- **T03 (tests):** `python -m pytest backend/pipeline/test_harness_compose.py -v` — all pass

### Notes for Planner

- The `compose` subcommand mirrors the `run` subcommand structure but with different inputs. Keep the pattern consistent.
- The compose prompt is the highest-risk deliverable — it drives whether the LLM produces good composed output. The test harness is mechanical.
- S04 (Pipeline Compose-or-Create Logic) will use both the compose prompt and the test harness patterns when wiring into the real pipeline. The harness is the offline proving ground.
- The existing `_merge_pages_by_slug` in stages.py merges partial pages from chunk splitting. Composition is conceptually different (existing page + new moments) but the LLM call pattern is the same.
- No new Pydantic schemas needed — compose output is the same `SynthesisResult` with `SynthesizedPage` objects.
