# S02: Composition Prompt + Test Harness Compose Mode

**Goal:** Composition prompt (stage5_compose.txt) and test harness --compose mode enable offline testing of merging new video moments into existing technique pages, producing unified v2 output with correct citation re-indexing and deduplication.
**Demo:** After this: Run test harness --compose mode with existing page + new moments → merged output with deduplication, new sections, updated citations.

## Tasks
- [x] **T01: Created composition prompt with merge rules, citation re-indexing, deduplication guidance, and two valid v2 JSON examples** — Create the composition prompt that instructs the LLM to merge new video moments into an existing technique page. The prompt receives three inputs: existing page JSON, new moments list, and creator name. Must define merge semantics (preserve existing prose, integrate new material into existing sections or create new ones), deduplication rules (enrich rather than repeat), citation re-indexing scheme (existing [0]-[N-1], new [N]-[N+M-1]), and output format (same SynthesisResult/SynthesizedPage v2 schema as synthesis).

Steps:
1. Read `prompts/stage5_synthesis.txt` to understand voice/tone rules and v2 output format — the compose prompt must reference the same writing standards but NOT duplicate them (the compose prompt is self-contained).
2. Write `prompts/stage5_compose.txt` with these sections:
   - Role/context preamble explaining the composition task
   - Input format: `<existing_page>`, `<existing_moments>`, `<new_moments>`, `<creator>` XML tags
   - Merge rules: preserve existing prose verbatim where correct; add new sections for genuinely new territory; enrich existing sections with new detail where topics overlap
   - Deduplication guidance with examples of 'enriching' vs 'duplicating'
   - Citation re-indexing rules: existing moments keep indices [0]-[N-1], new moments get [N]-[N+M-1], composed output uses the unified index space
   - Section ordering: maintain workflow-order rule from synthesis
   - Output format: same SynthesisResult JSON with v2 body_sections (list[BodySection])
   - Include a concrete JSON example showing composed output with mixed old/new citations
3. Validate the prompt structurally: check that the JSON example parses as valid SynthesisResult, citation rules section mentions the dual-index scheme, merge rules section is present.
  - Estimate: 45m
  - Files: prompts/stage5_compose.txt, prompts/stage5_synthesis.txt
  - Verify: python3 -c "
import json, pathlib, re
p = pathlib.Path('prompts/stage5_compose.txt')
assert p.exists(), 'prompt file missing'
txt = p.read_text()
assert len(txt) > 2000, f'prompt too short: {len(txt)} chars'
for section in ['existing_page', 'new_moments', 'existing_moments', 'creator', 'Citation', 'dedup', 'Merge']:
    assert any(s.lower() in txt.lower() for s in [section]), f'missing section: {section}'
# Check JSON example is valid
json_match = re.search(r'```json\n(.*?)```', txt, re.DOTALL)
assert json_match, 'no JSON example found'
json.loads(json_match.group(1))
print('All structural checks passed')
"
- [x] **T02: Added compose subcommand with build_compose_prompt(), run_compose(), and CLI wiring for offline composition testing** — Add a `compose` subcommand to `backend/pipeline/test_harness.py` that loads an existing page JSON + new moments fixture, builds a compose user prompt, calls the LLM via the compose prompt, and validates the output. Also extract the compose user-prompt builder as a testable function.

Steps:
1. Read `backend/pipeline/test_harness.py` fully to understand the existing `run` and `promote` subcommand patterns.
2. Add a `build_compose_prompt()` function that takes:
   - `existing_page: dict` (SynthesizedPage as dict, from prior synthesis output)
   - `existing_moments: list[tuple[MockKeyMoment, dict]]` (the original moments the page was built from)
   - `new_moments: list[tuple[MockKeyMoment, dict]]` (new video's moments to compose in)
   - `creator_name: str`
   Returns: the user prompt string with `<existing_page>`, `<existing_moments>`, `<new_moments>`, and `<creator>` XML tags. Existing moments get indices [0]-[N-1], new moments get [N]-[N+M-1]. Uses `build_moments_text()` for formatting moment lists.
3. Add a `run_compose()` function (parallel to `run_synthesis()`) that:
   - Loads the existing page JSON and validates as SynthesizedPage
   - Loads the new fixture via existing `load_fixture()`
   - Filters new moments by category to match `existing_page.topic_category`
   - Needs the existing moments for citation context — accept them as a separate fixture or extract from the existing page's moment_indices + a moments fixture
   - Calls `build_compose_prompt()` to build the user prompt
   - Calls LLM with compose system prompt
   - Parses response as SynthesisResult
   - Runs citation validation with moment_count = len(existing_moments) + len(new_moments)
   - Logs compose-specific metrics: word count before/after, sections before/after, new sections added
4. Register `compose` subcommand in the `main()` argparse with args: `--existing-page` (required), `--fixture` (required), `--existing-fixture` (required, original moments fixture for citation context), `--prompt`, `--output`, `--category`, `--model`, `--modality`.
5. Wire the subcommand handler to call `run_compose()` and write output JSON.
6. Verify: `cd backend && python -m pipeline.test_harness compose --help` exits 0.
  - Estimate: 1h
  - Files: backend/pipeline/test_harness.py
  - Verify: cd backend && python -m pipeline.test_harness compose --help
- [x] **T03: 16 unit tests covering compose prompt XML structure, citation offset math, category filtering, and edge cases — all passing** — Write unit tests for the compose harness plumbing — no LLM calls. Tests cover prompt construction, citation re-indexing math, category filtering, word count comparison, and edge cases.

Steps:
1. Read the updated `backend/pipeline/test_harness.py` (from T02) to understand `build_compose_prompt()` signature and `run_compose()` interface.
2. Read `backend/pipeline/test_harness_v2_format.py` for test patterns (helper functions, BodySection construction, etc.).
3. Create `backend/pipeline/test_harness_compose.py` with these test classes:
   - `TestBuildComposePrompt`: 
     - Existing page + 3 old moments + 2 new moments → prompt contains all XML tags, old moments indexed [0]-[2], new moments indexed [3]-[4]
     - Creator name appears in `<creator>` tag
     - Existing page JSON appears in `<existing_page>` tag and is valid JSON
     - Moment text uses same format as `build_moments_text()`
   - `TestCitationReindexing`:
     - 5 old + 3 new moments → valid citation range is [0]-[7]
     - validate_citations() with moment_count=8 accepts citations in [0]-[7]
     - validate_citations() with moment_count=8 rejects citation [8]
   - `TestCategoryFiltering`:
     - Build fixture with moments in 2 categories → compose with existing page of category A → only category A new moments used
   - `TestEdgeCases`:
     - Empty new moments list → prompt still valid, just no new_moments content
     - Single new moment → prompt has one entry at index [N]
     - Existing page with no subsections → handled correctly
4. Run tests: `cd backend && python -m pytest pipeline/test_harness_compose.py -v`
  - Estimate: 45m
  - Files: backend/pipeline/test_harness_compose.py
  - Verify: cd backend && python -m pytest pipeline/test_harness_compose.py -v
