docs: update 10 wiki pages with M025 feature content — notifications, rate limiting, onboarding, transparency, export, fallback, mobile, quality toolkit

API-Surface.md

@@ -200,6 +200,35 @@ All under prefix `/api/v1/admin/pipeline/`.
 
 **New endpoints should follow the `{items, total, offset, limit}` paginated pattern.**
 
+## Notifications (M025/S01)
+
+| Method | Path | Auth | Notes |
+|--------|------|------|-------|
+| GET | `/api/v1/notifications/preferences` | Bearer JWT | Returns notification_preferences JSONB |
+| PUT | `/api/v1/notifications/preferences` | Bearer JWT | Update digest frequency, enabled flags |
+| GET | `/api/v1/notifications/unsubscribe?token=` | Public | JWT-signed token with expiry; renders styled HTML result |
+
+## Creator Transparency & Export (M025/S05, S07)
+
+| Method | Path | Auth | Notes |
+|--------|------|------|-------|
+| GET | `/api/v1/creator/transparency` | Bearer JWT | Full entity graph: technique pages, key moments, cross-refs, source videos, tags |
+| GET | `/api/v1/creator/export` | Bearer JWT | GDPR-style ZIP download (12 data tables + export_metadata.json) |
+
+## Onboarding (M025/S03)
+
+| Method | Path | Auth | Notes |
+|--------|------|------|-------|
+| POST | `/api/v1/auth/onboarding-complete` | Bearer JWT | Sets `onboarding_completed = true` on User |
+
+## Usage & Rate Limiting (M025/S04)
+
+| Method | Path | Auth | Notes |
+|--------|------|------|-------|
+| GET | `/api/v1/admin/usage` | Admin | Token consumption: period totals, top creators/users, daily counts |
+
+Chat requests are rate-limited per-user (30/hr), per-IP (10/hr), and per-creator (60/hr) via Redis sliding-window. Returns HTTP 429 with `retry_after` header when exceeded.
+
 ## Authentication
 
 JWT-based authentication added in M019. See [[Authentication]] for full details.

Architecture.md

@@ -74,7 +74,7 @@ Chrysopedia is a self-hosted music production knowledge base that synthesizes te
 | Frontend | React 18 + TypeScript + Vite |
 | Backend | FastAPI + Celery + SQLAlchemy (async) |
 | Database | PostgreSQL 16 |
-| Cache/Broker | Redis 7 (Celery broker + review mode toggle + classification cache) |
+| Cache/Broker | Redis 7 (Celery broker + Beat scheduler + review mode toggle + classification cache + rate limit counters) |
 | Vector Store | Qdrant 1.13.2 |
 | Knowledge Graph | LightRAG (graph-based RAG, port 9621) |
 | Embeddings | Ollama (nomic-embed-text) via OpenAI-compatible /v1/embeddings |
@@ -90,6 +90,7 @@ Chrysopedia is a self-hosted music production knowledge base that synthesizes te
 4. **Storage:** Technique pages + key moments → PostgreSQL, embeddings → Qdrant
 5. **Serving:** React SPA fetches from FastAPI, search queries hit Qdrant then PostgreSQL fallback
 6. **Auth:** JWT-protected endpoints for creator consent management and admin features (see [[Authentication]])
+7. **Notifications:** Celery Beat runs daily email digest task (09:00 UTC) -- queries new content per followed creator, composes HTML, sends via SMTP. Deduplication via `EmailDigestLog` table (M025/S01)
 
 ---
 
@@ -97,6 +98,7 @@ Chrysopedia is a self-hosted music production knowledge base that synthesizes te
 → PostgreSQL, embeddings → Qdrant
 5. **Serving:** React SPA fetches from FastAPI, search queries hit Qdrant then PostgreSQL fallback
 6. **Auth:** JWT-protected endpoints for creator consent management and admin features (see [[Authentication]])
+7. **Notifications:** Celery Beat runs daily email digest task (09:00 UTC) -- queries new content per followed creator, composes HTML, sends via SMTP. Deduplication via `EmailDigestLog` table (M025/S01)
 
 ---
 

Chat-Engine.md

@@ -225,6 +225,36 @@ End-to-end video metadata flow from search results through SSE to frontend sourc
 - `utils/formatTime.ts` — shared hour-aware time formatter used across timestamp badges and player controls.
 - Source cards now show: timestamp badge (links to `/watch/:id?t=N` when `start_time` defined) and video filename metadata.
 
+
+## LLM Fallback Resilience (M025/S08)
+
+ChatService maintains two AsyncOpenAI clients: primary (DGX endpoint) and fallback (Ollama). When the primary `create()` call fails with `APIConnectionError`, `APITimeoutError`, or `InternalServerError`, the entire streaming call is retried with the fallback client. The SSE `done` event includes `fallback_used: true/false` so the frontend and usage logging know which model actually served the response.
+
+- **Config:** `LLM_FALLBACK_URL` and `LLM_FALLBACK_MODEL` in docker-compose.yml
+- **Logging:** `chat_llm_fallback` WARNING when primary fails and fallback activates
+- **Usage tracking:** `ChatUsageLog.model` records actual model name (primary or fallback)
+
+## Refined System Prompt (M025/S09)
+
+The system prompt was rewritten from 5 lines to a structured template covering:
+
+- **Citation density:** Cite every factual claim inline with `[N]` markers
+- **Response format:** Short paragraphs, bullet lists for step-by-step, bold key terms
+- **Domain terminology:** Music production context awareness
+- **Conflicting sources:** Present both perspectives with attribution
+- **Response length:** 2-4 paragraphs default, adjust to query complexity
+
+Kept under 20 lines using markdown headers for structure. All 26 existing chat tests pass unchanged.
+
+## Chat Quality Evaluation Toolkit (M025/S09)
+
+A 5-dimension LLM-as-judge evaluation framework:
+
+- **Scorer** (`backend/pipeline/quality/chat_scorer.py`): Grades responses on citation_accuracy, response_structure, domain_expertise, source_grounding, personality_fidelity
+- **Eval Harness** (`backend/pipeline/quality/chat_eval.py`): SSE-parsing runner that calls the live chat endpoint and feeds responses to the scorer
+- **Test Suite** (`backend/pipeline/quality/fixtures/chat_test_suite.yaml`): 10 queries across technical, conceptual, creator-scoped, and cross-creator categories
+- **CLI:** `python -m pipeline.quality chat_eval` subcommand for automated evaluation runs
+- **Baseline report:** Documented in S09-QUALITY-REPORT.md with JSON results
 ## Key Files
 
 - `backend/chat_service.py` — ChatService with history load/save, retrieve-prompt-stream pipeline

Configuration.md

@@ -145,6 +145,38 @@ LLM settings are configured per pipeline stage:
 
 Prompt templates are loaded from disk (`prompts/` directory) at runtime. SHA-256 hashes are tracked for reproducibility.
 
+
+## SMTP / Email Notifications (M025/S01)
+
+| Variable | Default | Notes |
+|----------|---------|-------|
+| `smtp_host` | `""` (empty) | SMTP server hostname. Empty = email disabled (graceful no-op) |
+| `smtp_port` | `587` | SMTP port (587 for STARTTLS) |
+| `smtp_user` | `""` | SMTP username |
+| `smtp_password` | `""` | SMTP password |
+| `smtp_from_email` | `""` | Sender email address |
+| `smtp_from_name` | `"Chrysopedia"` | Sender display name |
+
+When `smtp_host` is empty, the digest task logs a warning and returns without sending. No emails are lost -- the next run picks up unsent content.
+
+## Rate Limiting (M025/S04)
+
+| Variable | Default | Notes |
+|----------|---------|-------|
+| `rate_limit_user_per_hour` | `30` | Max chat requests per authenticated user per hour |
+| `rate_limit_ip_per_hour` | `10` | Max chat requests per IP per hour (anonymous) |
+| `rate_limit_creator_per_hour` | `60` | Max chat requests per creator scope per hour |
+
+Rate limiter uses Redis sorted sets (sliding window). Fails open on Redis errors -- logs WARNING, allows request.
+
+## LLM Fallback (M025/S08)
+
+| Variable | Default | Notes |
+|----------|---------|-------|
+| `LLM_FALLBACK_URL` | `http://chrysopedia-ollama:11434/v1` | Fallback LLM endpoint (Ollama) |
+| `LLM_FALLBACK_MODEL` | `qwen2.5:7b` | Fallback model name |
+
+ChatService auto-falls back from primary to fallback on `APIConnectionError`, `APITimeoutError`, or `InternalServerError`. Fallback activation is logged at WARNING and tracked in SSE `done` event and `ChatUsageLog`.
 ## Network
 
 - **Compose subnet:** 172.32.0.0/24

Data-Model.md

@@ -252,6 +252,46 @@ Append-only versioned record of per-field consent changes.
 - **User passwords** are stored as bcrypt hashes via `bcrypt.hashpw()`
 - **Consent audit** uses version numbers assigned in application code (`max(version) + 1` per video_consent_id)
 
+
+## M025 Models
+
+### EmailDigestLog (M025/S01)
+
+| Field | Type | Notes |
+|-------|------|-------|
+| id | UUID PK | |
+| user_id | FK -> User | |
+| content_summary | JSONB | Digest content grouped by creator |
+| sent_at | Timestamp | |
+
+Tracks every successful email send for deduplication. The digest task checks `MAX(sent_at)` per user to find new content since last send.
+
+### ChatUsageLog (M025/S04)
+
+| Field | Type | Notes |
+|-------|------|-------|
+| id | UUID PK | |
+| user_id | FK -> User | Nullable (anonymous users) |
+| client_ip | String | For IP-based rate limiting |
+| creator_slug | String | Nullable |
+| query | Text | User's chat query |
+| prompt_tokens | Integer | |
+| completion_tokens | Integer | |
+| total_tokens | Integer | |
+| cascade_tier | String | creator / domain / global / none |
+| model | String | Actual model used (primary or fallback) |
+| latency_ms | Integer | End-to-end response time |
+| created_at | Timestamp | Indexed for time-range aggregation |
+
+Non-blocking INSERT -- errors logged but never block the SSE response.
+
+### User Model Changes (M025/S01, S03)
+
+| Field | Type | Notes |
+|-------|------|-------|
+| notification_preferences | JSONB | `{"digest_enabled": bool, "digest_frequency": "daily"|"weekly"}` |
+| onboarding_completed | Boolean | Default false; set true via POST /auth/onboarding-complete |
+
 ---
 
 *See also: [[Architecture]], [[API-Surface]], [[Pipeline]], [[Authentication]]*

Decisions.md

@@ -69,6 +69,15 @@ Architectural and pattern decisions made during Chrysopedia development. Append-
 | D043 | M023/S02 | Personality weight modulation strategy | 3-tier intensity (<0.4 subtle, 0.4–0.8 voice, ≥0.8 embody) with temperature scaling 0.3–0.5. **Superseded by D044.** | Initial stepped approach; replaced by continuous interpolation. |
 | D044 | M023/S04 | Personality weight modulation strategy (revision) | 5-tier continuous interpolation. Progressive field inclusion: <0.2 no personality; 0.2+ tone; 0.4+ descriptors; 0.6+ phrases (count scaled); 0.8+ vocabulary/style; 0.9+ summary. Temperature: 0.3 + weight × 0.2. | 3-tier step function had jarring transitions. Continuous interpolation with progressive field inclusion gives finer control. 0.0–0.19 dead zone ensures purely encyclopedic mode. |
 
+## M025 Decisions
+
+| # | When | Decision | Choice | Rationale |
+|---|------|----------|--------|-----------|
+| D045 | M025/S01 | Signed unsubscribe token library | PyJWT (already a dependency) over itsdangerous | Avoids adding a new dependency; PyJWT supports exp claims natively for self-expiring tokens |
+| D046 | M025/S10 | Sticky section bar for technique pages | Built ReadingHeader with sentinel-based IntersectionObserver | Sticky elements never leave viewport for standard IO detection; sentinel div pattern solves this reliably |
+| D047 | M025/S04 | Rate limiter failure mode | Fail-open (allow request on Redis error) | Availability over strictness for a single-admin platform; rate limiting is cost protection, not security |
+| D048 | M025/S08 | LLM fallback strategy | Catch-and-retry with secondary client on transient errors | Matches sync LLMClient pipeline pattern; propagates fallback_used through SSE and usage logging |
+
 ---
 
 *See also: [[Architecture]], [[Development-Guide]]*

Deployment.md

@@ -118,6 +118,30 @@ docker compose build chrysopedia-api && docker compose up -d chrysopedia-api chr
 docker compose restart chrysopedia-api chrysopedia-worker
 ```
 
+
+## Email Digest Scheduling (M025/S01)
+
+The Celery worker runs with the `--beat` flag to enable periodic task scheduling:
+
+```
+command: celery -A worker worker --loglevel=info --beat
+```
+
+**Celery Beat schedule:**
+- `send-digest-emails`: Runs daily at 09:00 UTC via `crontab(hour=9, minute=0)`
+- Collocated with `worker.py` celery_app config (no separate beat container needed)
+
+## LLM Fallback Configuration (M025/S08)
+
+Add to the API service environment in `docker-compose.yml`:
+
+```yaml
+environment:
+  LLM_FALLBACK_URL: http://chrysopedia-ollama:11434/v1
+  LLM_FALLBACK_MODEL: qwen2.5:7b
+```
+
+The fallback endpoint is the local Ollama instance running on the same Docker network. No external dependency -- fallback works even when the primary DGX endpoint is unreachable.
 ## Port Mapping
 
 | Service | Container Port | Host Port | Binding |

Frontend.md

@@ -193,6 +193,45 @@ API modules:
 
 Relative `/api/v1` base URL (nginx proxies to API container).
 
+
+## M025 Pages & Components
+
+### CreatorOnboarding (M025/S03)
+
+- **Route:** `/creator/onboarding`
+- **File:** `frontend/src/pages/CreatorOnboarding.tsx` + `.module.css`
+- **Auth:** ProtectedRoute (requires login)
+- **Description:** 3-step wizard (Welcome -> Consent Setup -> Dashboard Tour). Step 2 fetches real consent data. Stepper UI with numbered circles and connecting lines. Responsive at 375px.
+- **Trigger:** Login redirects here when `onboarding_completed === false`
+
+### AdminUsage (M025/S04)
+
+- **Route:** `/admin/usage`
+- **File:** `frontend/src/pages/AdminUsage.tsx` + `.module.css`
+- **Auth:** Admin only (via AdminDropdown menu)
+- **Description:** Token usage dashboard with summary cards (today/week/month totals), CSS bar chart for daily activity, and breakdown tables (top creators, top users).
+- **API:** `GET /api/v1/admin/usage`
+
+### CreatorTransparency (M025/S05)
+
+- **Route:** `/creator/transparency`
+- **File:** `frontend/src/pages/CreatorTransparency.tsx` + `.module.css`
+- **Auth:** ProtectedRoute (requires linked creator profile)
+- **Description:** Full entity graph viewer with tag summary bar and 4 collapsible sections: Technique Pages, Key Moments (grouped by source video), Cross-References, Source Videos. Uses `grid-template-rows: 0fr/1fr` animation for collapse/expand.
+- **Nav:** Sidebar link between Tiers and Posts on CreatorDashboard
+
+### ReadingHeader (M025/S10)
+
+- **File:** `frontend/src/components/ReadingHeader.tsx`
+- **Description:** Sticky section bar on technique pages. Fixed-position bar slides in via CSS transform when user scrolls past the title. Uses sentinel-based IntersectionObserver with callback ref pattern.
+- **Integration:** Rendered by TechniquePage.tsx, receives `activeId` from parent
+
+### Export My Data (M025/S07)
+
+- **Location:** Button on CreatorDashboard
+- **API:** `exportCreatorData()` in `frontend/src/api/creator-dashboard.ts`
+- **Pattern:** `fetch` with Bearer token -> `response.blob()` -> hidden anchor + `URL.createObjectURL` -> click -> revoke
+- **UX:** Loading spinner during download, inline error display on failure
 ## CSS Architecture
 
 | Property | Value |

Home.md

@@ -8,7 +8,7 @@ Producers can search for specific techniques and find timestamped key moments, s
 
 - [[Architecture]] — System architecture, Docker services, network topology
 - [[Data-Model]] — SQLAlchemy models, relationships, enums
-- [[API-Surface]] — All 70+ API endpoints grouped by domain
+- [[API-Surface]] — All 80+ API endpoints grouped by domain
 - [[Frontend]] — Routes, components, hooks, CSS architecture
 - [[Pipeline]] — 6-stage LLM extraction pipeline, prompt system
 - [[Chat-Engine]] — Streaming Q&A with multi-turn memory
@@ -17,7 +17,7 @@ Producers can search for specific techniques and find timestamped key moments, s
 - [[Search-Retrieval]] — LightRAG + Qdrant retrieval cascade
 - [[Deployment]] — Docker Compose setup, rebuild commands
 - [[Development-Guide]] — Local dev setup, common gotchas
-- [[Decisions]] — Architectural decisions register (D001–D044)
+- [[Decisions]] — Architectural decisions register (D001–D046)
 
 ## Features
 
@@ -50,6 +50,24 @@ Producers can search for specific techniques and find timestamped key moments, s
 - **Inline Collapsible Player** — On technique pages between summary and body, with bibliography seek wiring and multi-source-video selector (M024/S02)
 - **Citation UX** — Timestamp badge links to `/watch/:id?t=N`, video filename on source cards, shared `chatCitations.tsx` + `formatTime.ts` utilities (M024/S05)
 
+### Notifications & Cost Management (M025)
+- **Email Digest Notifications** — Daily Celery Beat emails to followers when creators publish, with signed-token unsubscribe (M025/S01)
+- **Rate Limiting** — Redis sliding-window per-user/IP/creator rate limiter for chat, fail-open design (M025/S04)
+- **Usage Dashboard** — Admin token consumption tracking with period aggregation, top creators/users, daily charts (M025/S04)
+
+### Creator Experience (M025)
+- **Onboarding Wizard** — 3-step welcome→consent→tour flow for new creators, triggered on first login (M025/S03)
+- **AI Transparency Page** — Full entity graph viewer: technique pages, key moments, cross-references, source videos, tags (M025/S05)
+- **Data Export** — GDPR-style ZIP download of all creator-owned content (12 tables + metadata) (M025/S07)
+
+### Resilience & Quality (M025)
+- **LLM Fallback** — ChatService automatic primary→Ollama failover on connection/timeout/server errors (M025/S08)
+- **Prompt Optimization** — Refined system prompt with citation density, structure, and domain guidance (M025/S09)
+- **Chat Quality Toolkit** — 5-dimension LLM-as-judge scorer, SSE eval harness, 10-query test suite (M025/S09)
+- **Mobile Responsiveness** — ≤400px safety-net breakpoints across all pages, zero horizontal overflow at 375px (M025/S02)
+- **Reading Header** — Sticky section bar on technique pages with sentinel-based scroll detection (M025/S10)
+- **Graph Evaluation** — NetworkX benchmark report with Neo4j migration plan at 50K/90K node thresholds (M025/S06)
+
 ### Platform
 - **Authentication** — JWT with invite codes, admin/creator roles
 - **Consent System** — Per-video granular consent with audit trail
@@ -57,7 +75,7 @@ Producers can search for specific techniques and find timestamped key moments, s
 
 ## Current Scale
 
-- **83** technique pages
+- **95** technique pages
 - **25–26** creators
 - **200** source videos
 - **7** topic categories
@@ -78,4 +96,4 @@ Producers can search for specific techniques and find timestamped key moments, s
 
 ---
 
-*Last updated: 2026-04-04 — M024 shorts publishing, embed support, timeline pins, auto-captioning, templates, citation UX
+*Last updated: 2026-04-04 — M025 notifications, rate limiting, onboarding, transparency, export, fallback, mobile, quality toolkit

_Sidebar.md

@@ -1,6 +1,7 @@
 ### Chrysopedia Wiki
 
 - [[Home]]
+- [[Newcomer-Guide|Newcomer Guide]]
 
 **Architecture**
 - [[Architecture]]