---
id: S02
parent: M013
milestone: M013
provides:
  - ScoreRunner with score_page() and synthesize_and_score() methods
  - VoiceDial class for 3-band prompt modification
  - CLI score subcommand with --file, --slug, --voice-level
  - 6-moment fixture file for offline testing
requires:
  []
affects:
  - S03
key_files:
  - backend/pipeline/quality/scorer.py
  - backend/pipeline/quality/voice_dial.py
  - backend/pipeline/quality/__main__.py
  - backend/pipeline/quality/__init__.py
  - backend/pipeline/quality/fixtures/sample_moments.json
  - pipeline
key_decisions:
  - Hardcoded scoring rubric in scorer.py rather than external prompt file — faster iteration during quality toolkit development
  - Three discrete voice bands (low/mid/high) at 0.33/0.67 boundaries instead of continuous interpolation — simpler to reason about and test
  - Mid band returns base prompt unmodified since existing stage5 prompt already targets ~0.6 voice preservation
  - Project-root symlink + __init__.py sys.path bootstrap to support running from any CWD
patterns_established:
  - sys.path bootstrap in package __init__.py using os.path.realpath() to resolve through symlinks
  - VoiceDial band-based prompt modification pattern reusable for other prompt dimension dials
observability_surfaces:
  - Score report output with per-dimension scores and justification excerpts
  - Band identification in voice-level output (band=low/mid/high)
  - Clean connectivity error with endpoint URL on LLM failure
drill_down_paths:
  - .gsd/milestones/M013/slices/S02/tasks/T01-SUMMARY.md
  - .gsd/milestones/M013/slices/S02/tasks/T02-SUMMARY.md
duration: ""
verification_result: passed
completed_at: 2026-04-01T09:01:48.179Z
blocker_discovered: false
---

# S02: Stage 5 Quality Scorer & Voice Preservation Dial

**Built 5-dimension LLM-as-judge scorer with CLI `score` subcommand and 3-band voice preservation dial that modifies Stage 5 synthesis prompts, runnable from both project root and backend/ directory.**

## What Happened

T01 created the scoring infrastructure: `ScoreRunner` with a `ScoreResult` dataclass covering 5 quality dimensions (structural, content_specificity, voice_preservation, readability, factual_fidelity), each scored 0.0–1.0 by an LLM judge via a hardcoded rubric prompt. The `score` subcommand was added to the existing `pipeline.quality` CLI with mutually exclusive `--file`/`--slug` inputs and an optional `--voice-level` parameter. A 6-moment fixture file (`sample_moments.json`) with realistic music production content provides offline testing data.

T02 added the `VoiceDial` class implementing three discrete bands (low ≤0.33, mid 0.34–0.66, high ≥0.67) that modify the Stage 5 synthesis system prompt. Low band appends voice suppression instructions (neutral third-person, no direct quotes). Mid band passes the base prompt through unmodified (it already targets ~0.6 voice preservation). High band appends amplification instructions (maximize direct quotes, preserve personality). `ScoreRunner.synthesize_and_score()` chains re-synthesis through the modified prompt followed by scoring.

Post-task fix: The verification gate ran `python -m pipeline.quality score --help` from the project root, but the `pipeline` package lives under `backend/`. Added a `pipeline` symlink at project root pointing to `backend/pipeline`, and added a `sys.path` bootstrap in `backend/pipeline/quality/__init__.py` that resolves through the symlink to add `backend/` to `sys.path`. This ensures `config` and `pipeline.llm_client` imports resolve regardless of CWD.

## Verification

All slice verification checks pass from both project root and backend/ directory:

1. `python -m pipeline.quality score --help` — shows --file, --slug, --voice-level args (exit 0)
2. `python -c "from pipeline.quality.scorer import ScoreRunner, ScoreResult; print('ok')"` — import succeeds (exit 0)
3. `python -c "from pipeline.quality.voice_dial import VoiceDial; vd = VoiceDial('base'); assert vd.modify(0.1) != vd.modify(0.5); assert vd.modify(0.5) != vd.modify(0.9)"` — three distinct bands confirmed (exit 0)
4. `python -m pipeline.quality score --file backend/pipeline/quality/fixtures/sample_moments.json` — clean connectivity error, exit 1, no traceback
5. `python -m pipeline.quality score --file backend/pipeline/quality/fixtures/sample_moments.json --voice-level 0.3` — re-synthesis path, clean connectivity error, exit 1, no traceback
6. Fixture validation: 6 moments with required fields present

## Requirements Advanced

None.

## Requirements Validated

None.

## New Requirements Surfaced

None.

## Requirements Invalidated or Re-scoped

None.

## Deviations

Added project-root `pipeline` symlink and `sys.path` bootstrap in `__init__.py` to support running `python -m pipeline.quality` from the project root (not just from `backend/`). This was not in the original plan but required by the verification gate.

## Known Limitations

`--slug` path prints "DB loading not yet implemented" and exits 1 (deferred by design). Voice-level path requires `prompts/stage5_synthesis.txt` to be reachable from CWD — works in Docker (WORKDIR=/app) and from `backend/`, but from project root the prompt file path resolves differently.

## Follow-ups

None.

## Files Created/Modified

- `backend/pipeline/quality/scorer.py` — New: ScoreResult dataclass + ScoreRunner with score_page() and synthesize_and_score() methods
- `backend/pipeline/quality/voice_dial.py` — New: VoiceDial class with 3-band prompt modification (low/mid/high)
- `backend/pipeline/quality/__main__.py` — Added score subcommand with --file, --slug, --voice-level args
- `backend/pipeline/quality/__init__.py` — Added sys.path bootstrap for project-root execution
- `backend/pipeline/quality/fixtures/sample_moments.json` — New: 6-moment fixture with realistic music production content
- `backend/pipeline/quality/fixtures/__init__.py` — New: empty package marker
- `pipeline` — New: symlink to backend/pipeline for project-root execution
