---
estimated_steps: 25
estimated_files: 2
skills_used: []
---

# T01: Build PromptVariantGenerator and OptimizationLoop engine

Create two new modules: `variant_generator.py` with an LLM-powered prompt mutation engine, and `optimizer.py` with an optimization loop that orchestrates generate→score→select cycles.

## Context

The existing `ScoreRunner.synthesize_and_score()` already chains Stage 5 synthesis through a modified prompt and scores the result. The `VoiceDial` class demonstrates the pattern of appending modifier text to the base prompt. This task builds the higher-level automation: an LLM generates prompt variants targeting weak dimensions, and a loop selects winners across iterations.

## Key Interfaces

- `LLMClient.complete(system_prompt, user_prompt, response_model, modality)` — drives both variant generation and synthesis
- `ScoreRunner.synthesize_and_score(moments, creator_name, voice_level)` — scores a variant (uses voice_level=0.5 for neutral baseline)
- `_load_prompt('stage5_synthesis.txt')` — loads the base prompt to optimize
- `_get_stage_config(5)` — returns (model_override, modality) for Stage 5

## Steps

1. Create `backend/pipeline/quality/variant_generator.py`:
   - Define a meta-prompt constant (`VARIANT_META_PROMPT`) that instructs the LLM to act as a prompt engineer. Given the current best prompt, its per-dimension scores, and the scoring rubric summary, produce a single modified variant targeting the weakest dimension(s). The meta-prompt MUST instruct the LLM to preserve the JSON output format section of the synthesis prompt unchanged.
   - `PromptVariantGenerator.__init__(self, client: LLMClient)` — stores the LLM client
   - `generate(self, base_prompt: str, scores: ScoreResult, n: int = 2) -> list[str]` — calls the LLM `n` times with the meta-prompt, each time asking for a variant. Validates each variant: must differ from base by ≥50 chars, must still contain the JSON format instruction markers (e.g. `SynthesisResult` or `"pages"` key reference). Returns list of valid variant prompt strings. Logs and skips invalid variants.

2. Create `backend/pipeline/quality/optimizer.py`:
   - `@dataclass OptimizationResult`: best_prompt (str), best_score (ScoreResult), history (list of dicts with iteration, variant_index, prompt_text, score fields), elapsed_seconds (float)
   - `OptimizationLoop.__init__(self, client: LLMClient, stage: int, fixture_path: str, iterations: int, variants_per_iter: int)` — stores config, creates ScoreRunner and PromptVariantGenerator internally
   - `run(self) -> OptimizationResult` — loads base prompt via `_load_prompt(f'stage{stage}_synthesis.txt')`, loads fixture data from JSON file, runs baseline `synthesize_and_score()`, then iterates: generate N variants → score each → keep best → repeat. Prints progress per iteration. Stores full history. Returns OptimizationResult.
   - Handle LLM errors gracefully: if a variant's synthesis/scoring returns a ScoreResult with `error`, log it and skip that variant (don't abort the loop).

3. Verify both modules import cleanly from project root.

## Must-Haves

- Meta-prompt instructs LLM to preserve JSON output format section
- Variant validation: minimum diff from base, format markers present
- Invalid/errored variants are skipped, not fatal
- Progress output per iteration
- OptimizationResult contains full history for downstream reporting

## Inputs

- ``backend/pipeline/quality/scorer.py` — ScoreRunner and ScoreResult classes used by optimizer`
- ``backend/pipeline/quality/voice_dial.py` — pattern reference for prompt modification`
- ``backend/pipeline/llm_client.py` — LLMClient used by both generator and optimizer`
- ``backend/pipeline/stages.py` — _load_prompt() and _get_stage_config() used by optimizer`
- ``backend/pipeline/quality/fixtures/sample_moments.json` — fixture format reference`

## Expected Output

- ``backend/pipeline/quality/variant_generator.py` — PromptVariantGenerator class with LLM-powered prompt mutation`
- ``backend/pipeline/quality/optimizer.py` — OptimizationLoop class with generate→score→select cycle and OptimizationResult dataclass`

## Verification

python -c "from pipeline.quality.variant_generator import PromptVariantGenerator; print('generator ok')" && python -c "from pipeline.quality.optimizer import OptimizationLoop, OptimizationResult; print('optimizer ok')"
