# S01: General FYN-LLM Fitness Suite — UAT

**Milestone:** M013
**Written:** 2026-04-01T08:46:13.884Z

## UAT: S01 — General FYN-LLM Fitness Suite

### Preconditions
- Backend source at `backend/` with `pipeline/quality/` package present
- Python 3.x with project dependencies installed
- No live LLM endpoint required for error-path tests (tests 1-4)
- Live FYN-LLM endpoint required for functional tests (tests 5-7)

### Test Cases

#### TC1: Import Check
1. Run `cd backend && python -c 'from pipeline.quality.fitness import FitnessRunner; print("ok")'`
2. **Expected:** Prints "ok", exits 0

#### TC2: CLI Help
1. Run `cd backend && python -m pipeline.quality --help`
2. **Expected:** Shows usage with `fitness` subcommand listed and described

#### TC3: No Subcommand
1. Run `cd backend && python -m pipeline.quality`
2. **Expected:** Prints usage message, exits non-zero

#### TC4: Connectivity Error (no LLM running)
1. Ensure no LLM is running on localhost
2. Run `cd backend && python -m pipeline.quality fitness`
3. **Expected:** Prints "Cannot reach LLM endpoint at {url}" with both primary and fallback URLs. Exits 1. No Python traceback visible in output.

#### TC5: Full Fitness Run (requires live FYN-LLM)
1. Ensure FYN-LLM is reachable at configured endpoint
2. Run `cd backend && python -m pipeline.quality fitness`
3. **Expected:** Prints formatted report with 9 test results across 4 categories. Each test shows name, PASS/FAIL, elapsed time. Summary line at end shows pass count / total. Exits 0 if all pass, 1 if any fail.

#### TC6: Mandelbrot Thinking Mode (requires live FYN-LLM)
1. Run fitness suite with live LLM
2. **Expected:** Mandelbrot test uses thinking modality (visible in verbose output or code inspection). Tests LLM's reasoning capability, not just chat completion.

#### TC7: JSON Compliance with Malformed Response (requires live FYN-LLM)
1. If LLM returns non-JSON for a JSON test
2. **Expected:** Test marks as FAIL with clear detail message ("Failed to parse JSON" or similar). No crash or traceback.

### Edge Cases

#### EC1: Empty LLM Response
- If LLM returns empty string, affected tests should FAIL with descriptive detail, not crash with IndexError or similar.

#### EC2: Timeout
- If LLM endpoint is reachable but extremely slow, the connectivity probe should eventually time out and report the failure clearly.
