From 31b72c731c230bfbd3db77ac0a49ee1fd6e99742 Mon Sep 17 00:00:00 2001 From: jlightner Date: Fri, 3 Apr 2026 19:49:14 +0000 Subject: [PATCH] =?UTF-8?q?feat:=20Compiled=20467-line=20Site=20Audit=20Re?= =?UTF-8?q?port=20merging=20source=20code=20research=E2=80=A6?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - ".gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md" GSD-Task: S01/T02 --- .gsd/milestones/M018/slices/S01/S01-PLAN.md | 2 +- .../M018/slices/S01/SITE-AUDIT-REPORT.md | 467 ++++++++++++++++++ .../M018/slices/S01/tasks/T01-VERIFY.json | 16 + .../M018/slices/S01/tasks/T02-SUMMARY.md | 74 +++ 4 files changed, 558 insertions(+), 1 deletion(-) create mode 100644 .gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md create mode 100644 .gsd/milestones/M018/slices/S01/tasks/T01-VERIFY.json create mode 100644 .gsd/milestones/M018/slices/S01/tasks/T02-SUMMARY.md diff --git a/.gsd/milestones/M018/slices/S01/S01-PLAN.md b/.gsd/milestones/M018/slices/S01/S01-PLAN.md index ac11cbe..61a1c7f 100644 --- a/.gsd/milestones/M018/slices/S01/S01-PLAN.md +++ b/.gsd/milestones/M018/slices/S01/S01-PLAN.md @@ -17,7 +17,7 @@ Key constraint: The site is at http://ub01:8096. API base is /api/v1. Admin endp - Estimate: 45m - Files: .gsd/milestones/M018/slices/S01/AUDIT-FINDINGS.md, .gsd/milestones/M018/slices/S01/S01-RESEARCH.md - Verify: test -f .gsd/milestones/M018/slices/S01/AUDIT-FINDINGS.md && grep -c '##' .gsd/milestones/M018/slices/S01/AUDIT-FINDINGS.md | grep -q '[3-9]' -- [ ] **T02: Compile Site Audit Report from research and audit findings** — Compile the final Site Audit Report by merging the S01-RESEARCH.md source code analysis with the T01 AUDIT-FINDINGS.md live verification results. The report is the primary deliverable of this slice — a single markdown document that Phase 2 planning references for all implementation decisions. +- [x] **T02: Compiled 467-line Site Audit Report merging source code research with live audit findings, documenting all 12 routes, 41 API endpoints, 13 data models, CSS architecture, and 8 Phase 2 integration risks** — Compile the final Site Audit Report by merging the S01-RESEARCH.md source code analysis with the T01 AUDIT-FINDINGS.md live verification results. The report is the primary deliverable of this slice — a single markdown document that Phase 2 planning references for all implementation decisions. Steps: 1. Read S01-RESEARCH.md and AUDIT-FINDINGS.md diff --git a/.gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md b/.gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md new file mode 100644 index 0000000..2bf43b0 --- /dev/null +++ b/.gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md @@ -0,0 +1,467 @@ +# Chrysopedia — Site Audit Report + +**Audit Date:** 2026-04-03 +**Site URL:** http://ub01:8096 +**API Version:** v0.1.0 | **App Version:** v0.8.0 +**Auditor:** GSD Auto-Mode (M018/S01) + +--- + +## Executive Summary + +Chrysopedia is a music production knowledge base that synthesizes technique articles from video transcripts using an LLM pipeline. The site runs as a self-hosted Docker Compose stack on `ub01` with 8 containers serving a React SPA frontend, FastAPI backend, Celery worker, PostgreSQL, Redis, Qdrant vector store, Ollama embedding service, and nginx reverse proxy. + +**Scale at time of audit:** 83 technique pages, 25–26 creators, 200 source videos, 7 topic categories. Single-admin tool with no authentication layer. + +**Tech stack:** +- **Frontend:** React 18 + TypeScript + Vite (no UI library, no state management library, no CSS framework) +- **Backend:** FastAPI (async) + SQLAlchemy (async for API, sync for Celery) + Celery + PostgreSQL 16 + Redis 7 + Qdrant 1.13.2 + Ollama +- **Infrastructure:** Docker Compose on ub01 (172.32.0.0/24 subnet), nginx reverse proxy exposing port 8096 + +**Key architectural characteristics:** +- Zero external frontend dependencies beyond React, react-router-dom, and Vite +- Monolithic CSS (5,820 lines, single file, BEM naming, 77 custom properties) +- No authentication on any endpoint (admin routes are network-access-controlled) +- Dual SQLAlchemy strategy: async engine for FastAPI, sync engine for Celery pipeline +- Non-blocking pipeline side effects (embedding failures don't block page synthesis) + +--- + +## Route Map + +All 12 frontend routes verified in live browser session. No JavaScript blocking errors observed on any route. + +| # | Route Pattern | Page Component | Auth | Render Status | Notes | +|---|---|---|---|---|---| +| 1 | `/` | Home | Public | ✅ OK | Hero search, 3-step process cards, popular topics, stats counters, Topics/Creators feature cards | +| 2 | `/search` | SearchResults | Public | ✅ OK | Sort (Relevance/Newest/Oldest/A-Z), tag/content match highlights, partial matches section | +| 3 | `/techniques/:slug` | TechniquePage | Public | ✅ OK | v2 body sections, ToC sidebar, citation superscripts, tag/plugin badges, key moments | +| 4 | `/creators` | CreatorsBrowse | Public | ✅ OK | Sort (Random/A-Z/Views), name filter, 12 genre filter buttons, creator cards with stats | +| 5 | `/creators/:slug` | CreatorDetail | Public | ✅ OK | Avatar, video count, genre breakdown, technique list. Known: duplicate genre badges from casing | +| 6 | `/topics` | TopicsBrowse | Public | ✅ OK | 7 category cards with icons, expandable sub-topic lists, topic filter input | +| 7 | `/topics/:category/:subtopic` | SubTopicPage | Public | ✅ OK | Breadcrumb nav, category badge, creator-grouped technique cards, sort dropdown | +| 8 | `/about` | About | Public | ✅ OK | Static: What Is, 6-step pipeline explanation, Who Maintains This | +| 9 | `/admin/reports` | AdminReports | Admin* | ✅ OK | Status filter tabs (All/Open/Acknowledged/Resolved/Dismissed), 0 reports at audit time | +| 10 | `/admin/pipeline` | AdminPipeline | Admin* | ✅ OK | 200 videos, status tabs, recent activity, worker status, debug toggle, per-video actions | +| 11 | `/admin/techniques` | AdminTechniquePages | Admin* | ✅ OK | 83 articles, 16 creators, table with filters/sort. Multi-source and creator filters | +| 12 | `*` (fallback) | → Redirect `/` | — | ✅ OK | SPA fallback returns 200 shell; react-router redirects unknown paths to `/` | + +*Admin routes have no authentication gate — accessible to anyone with network access. + +**Routing architecture:** Routes defined in a single `` block in `App.tsx`. The nginx server block returns the SPA `index.html` for all paths, with react-router-dom v6 handling client-side routing. New routes are added to the `` block — no other configuration required. + +--- + +## Component Inventory + +### Page Components (11) + +| Component | Route | Key Features | +|---|---|---| +| Home | `/` | Hero search bar, useCountUp animation for stats, popular topic tags, nav cards | +| SearchResults | `/search` | Debounced search input, SortDropdown, result cards with highlights, partial match fallback | +| TechniquePage | `/techniques/:slug` | v1/v2 format-discriminated rendering (D024), TableOfContents sidebar, IntersectionObserver scroll-spy (D030), citation superscripts, key moments | +| CreatorsBrowse | `/creators` | Random default sort (D014), genre filter pills, name filter, CreatorAvatar | +| CreatorDetail | `/creators/:slug` | Avatar, genre breakdown badges, technique list with sort | +| TopicsBrowse | `/topics` | Category cards with colored left borders (D020), expandable sections (CSS grid 0fr/1fr animation), filter input | +| SubTopicPage | `/topics/:cat/:sub` | Breadcrumb navigation, creator-grouped cards, sort dropdown | +| About | `/about` | Static content, numbered pipeline step cards | +| AdminReports | `/admin/reports` | Status filter tabs, report list (empty at audit time) | +| AdminPipeline | `/admin/pipeline` | Video list with status tabs, worker status, debug toggle, per-video actions (Retrigger/Revoke/Re-run Stage) | +| AdminTechniquePages | `/admin/techniques` | Technique table with multi-source filter, creator filter, format badge, version count | + +### Shared Components (11+) + +| Component | Used By | Purpose | +|---|---|---| +| SearchAutocomplete | Nav bar, mobile panel | Global search with Ctrl+Shift+F shortcut. `globalShortcut` prop prevents duplicate handlers (KNOWLEDGE entry) | +| AdminDropdown | Nav bar | Hover-open at desktop (D028), tap-toggle on mobile | +| AppFooter | All pages | Version display, build date, GitHub link, About link | +| TableOfContents | TechniquePage | Sticky sidebar ToC with scroll-spy active indicator | +| SortDropdown | SearchResults, CreatorsBrowse, SubTopicPage | Reusable sort selector | +| TagList | TechniquePage, SearchResults | Renders tag/badge pills | +| CategoryIcons | TopicsBrowse | SVG icons per topic category | +| CreatorAvatar | CreatorsBrowse, CreatorDetail | Avatar display with fallback | +| CopyLinkButton | TechniquePage | Clipboard copy with tooltip | +| SocialIcons | CreatorDetail | Social media link icons | +| ReportIssueModal | TechniquePage | Content report submission form | + +### Hooks (3) + +| Hook | Purpose | +|---|---| +| useCountUp | Animated counter for homepage stats (technique/creator counts) | +| useSortPreference | Persists sort preference per page in localStorage | +| useDocumentTitle | Sets `` per page — all 10 pages instrumented (D022) | + +### Utilities (2) + +| File | Purpose | +|---|---| +| citations.tsx | Citation superscript rendering and linking for technique page body sections | +| catSlug.ts | Category/subtopic slug generation and normalization | + +### API Client + +Single module `public-client.ts` (~600 lines) with typed `request<T>` helper. Relative `/api/v1` base URL (nginx proxies to API container). TypeScript interfaces for all response types defined in the same file. Vite dev proxy configured for local development (`/api` → `localhost:8001`). + +**State management:** Local component state only (useState/useEffect). No Redux, Zustand, Context providers, or external state management. + +--- + +## API Surface Map + +### Public Endpoints (10) + +| # | Method | Path | Response Shape | Verified | Notes | +|---|---|---|---|---|---| +| 1 | GET | `/health` | `{status, service, version, database}` | ✅ 200 | version="0.1.0", db=connected | +| 2 | GET | `/api/v1/stats` | `{technique_count, creator_count}` | ✅ 200 | 83 techniques, 26 creators | +| 3 | GET | `/api/v1/search?q=` | `{items, partial_matches, total, query, fallback_used}` | ✅ 200 | ⚠️ Uses `items` key, not `results`. Has keyword fallback (D009) | +| 4 | GET | `/api/v1/search/suggestions?q=` | `{suggestions: [{text, type}]}` | ✅ 200 | Technique name autocomplete | +| 5 | GET | `/api/v1/search/popular` | `{items: [{query, count}]}` | ✅ 200 | Popular searches from search_log (D025) | +| 6 | GET | `/api/v1/techniques?limit=&offset=` | `{items, total, offset, limit}` | ✅ 200 | Paginated, total=83 | +| 7 | GET | `/api/v1/techniques/random` | `{slug}` | ✅ 200 | Returns JSON slug, NOT a redirect | +| 8 | GET | `/api/v1/techniques/{slug}` | 22-field object (see below) | ✅ 200 | Rich response with eager-loaded relations | +| 9 | GET | `/api/v1/techniques/{slug}/versions` | `{items, total}` | ✅ 200 | Paginated version list | +| 10 | GET | `/api/v1/techniques/{slug}/versions/{n}` | Version detail | — | Router-confirmed, not probed | + +**Technique detail fields (22):** title, slug, topic_category, topic_tags, summary, body_sections, body_sections_format, signal_chains, plugins, id, creator_id, creator_name, creator_slug, source_quality, view_count, key_moment_count, created_at, updated_at, key_moments, creator_info, related_links, version_count, source_videos. + +### Browse Endpoints (5) + +| # | Method | Path | Response Shape | Verified | Notes | +|---|---|---|---|---|---| +| 1 | GET | `/api/v1/creators?sort=&genre=` | `{items, total, offset, limit}` | ✅ 200 | sort=random\|alpha\|views, genre filter. total=25 (vs 26 in stats) | +| 2 | GET | `/api/v1/creators/{slug}` | 16-field object | ✅ 200 | Includes genre_breakdown, techniques list, social_links | +| 3 | GET | `/api/v1/topics` | `[{name, description, sub_topics}]` | ✅ 200 | ⚠️ Returns bare list, not paginated dict | +| 4 | GET | `/api/v1/topics/{cat}/{sub}` | `{items, total, offset, limit}` | ✅ 200 | Paginated technique list for subtopic | +| 5 | GET | `/api/v1/topics/{cat}` | `{items, total, offset, limit}` | ✅ 200 | Paginated technique list for category | + +### Report Endpoints (3) + +| # | Method | Path | Response Shape | Verified | Notes | +|---|---|---|---|---|---| +| 1 | POST | `/api/v1/reports` | Report object | ✅ 422 | Empty body → validation error (confirms endpoint exists) | +| 2 | GET | `/api/v1/admin/reports` | Report list | ✅ 200 | No auth required | +| 3 | PATCH | `/api/v1/admin/reports/{id}` | Updated report | — | Router-confirmed, not probed (destructive) | + +### Pipeline Admin Endpoints (20+) + +All under prefix `/api/v1/admin/pipeline/` (NOT `/api/v1/pipeline/`). + +| # | Method | Path | Purpose | Verified | +|---|---|---|---|---| +| 1 | GET | `/admin/pipeline/videos` | Paginated video list with pipeline status | ✅ 200 | +| 2 | POST | `/admin/pipeline/trigger/{video_id}` | Trigger pipeline for video | Router-confirmed | +| 3 | POST | `/admin/pipeline/clean-retrigger/{video_id}` | Clean retrigger (wipe + reprocess) | Router-confirmed | +| 4 | POST | `/admin/pipeline/revoke/{video_id}` | Revoke active pipeline task | Router-confirmed | +| 5 | POST | `/admin/pipeline/rerun-stage/{video_id}` | Re-run specific pipeline stage | Router-confirmed | +| 6 | GET | `/admin/pipeline/events` | Pipeline event log | Router-confirmed | +| 7 | GET | `/admin/pipeline/runs` | Pipeline run history | Router-confirmed | +| 8 | GET | `/admin/pipeline/chunking-inspector/{video_id}` | Inspect chunking results | Router-confirmed | +| 9 | GET | `/admin/pipeline/embed-status` | Embedding/Qdrant health | Router-confirmed | +| 10 | GET | `/admin/pipeline/debug-mode` | Get debug mode state | ✅ 200 | +| 11 | POST | `/admin/pipeline/debug-mode` | Set debug mode state | Router-confirmed | +| 12 | GET | `/admin/pipeline/token-summary` | Token usage summary | Router-confirmed | +| 13 | GET | `/admin/pipeline/stale-pages` | Pages needing regeneration | ✅ 200 | +| 14 | POST | `/admin/pipeline/bulk-resynthesize` | Regenerate all technique pages | Router-confirmed | +| 15 | POST | `/admin/pipeline/wipe-all-output` | Delete all pipeline output | Router-confirmed | +| 16 | POST | `/admin/pipeline/optimize-prompt` | Trigger prompt optimization | Router-confirmed | +| 17 | POST | `/admin/pipeline/reindex-all` | Rebuild Qdrant index | Router-confirmed | +| 18 | GET | `/admin/pipeline/worker-status` | Celery worker health | ✅ 200 | +| 19 | GET | `/admin/pipeline/recent-activity` | Recent pipeline events | ✅ 200 | +| 20 | POST | `/admin/pipeline/creator-profile/{creator_id}` | Update creator profile | Router-confirmed | +| 21 | POST | `/admin/pipeline/avatar-fetch/{creator_id}` | Fetch creator avatar | Router-confirmed | + +### Other Endpoints (2) + +| # | Method | Path | Response Shape | Verified | Notes | +|---|---|---|---|---|---| +| 1 | POST | `/api/v1/ingest` | Ingest result | ✅ 422 | Transcript upload (empty body → validation) | +| 2 | GET | `/api/v1/videos` | `[{...}]` | ✅ 200 | ⚠️ Returns bare list, not paginated dict | + +### Response Shape Inconsistencies + +Three endpoints return bare lists instead of paginated dicts: +- `GET /api/v1/topics` → bare list +- `GET /api/v1/videos` → bare list +- Creator count mismatch: `/stats` says 26, `/creators` returns 25 (likely a 0-technique creator excluded from browse) + +**New endpoints should follow the paginated `{items, total, offset, limit}` pattern.** + +--- + +## Data Model + +13 SQLAlchemy models in `backend/models.py`. Key relationships shown below. + +### Core Content Models + +| Model | Key Fields | Relationships | +|---|---|---| +| **Creator** | name, slug, genres, avatar_url, bio, social_links, featured | → SourceVideo (1:N), → TechniquePage (1:N) | +| **SourceVideo** | filename, youtube_url, folder_name, processing_status, pipeline_stage | → Creator (N:1), → TranscriptSegment (1:N), → KeyMoment (1:N) | +| **TranscriptSegment** | start_time, end_time, text | → SourceVideo (N:1) | +| **KeyMoment** | title, summary, start_time, end_time, topic_category, topic_tags | → SourceVideo (N:1) | +| **TechniquePage** | title, slug, summary, body_sections (JSONB), body_sections_format, signal_chains, plugins, topic_category, topic_tags, source_quality, view_count | → Creator (N:1), → Tag (M:N via technique_page_tags), → KeyMoment (M:N via TechniquePageVideo) | +| **TechniquePageVersion** | version_number, content_snapshot (JSONB), pipeline_metadata (JSONB) | → TechniquePage (N:1) | + +### Supporting Models + +| Model | Purpose | +|---|---| +| **RelatedTechniqueLink** | Directed link between technique pages (source → target with label) | +| **Tag** | Normalized tag with M:N join to TechniquePage | +| **TechniquePageVideo** | Join table: TechniquePage ↔ SourceVideo (multi-source pages) | +| **ContentReport** | User-submitted content reports with status workflow | +| **SearchLog** | Query logging for popular searches feature (D025) | +| **PipelineRun** | Pipeline execution tracking per video | +| **PipelineEvent** | Granular pipeline stage events for admin monitoring | + +### Schema Notes + +- No Alembic migrations directory — schema changes require manual DDL or adding Alembic setup +- `body_sections_format` discriminator enables v1/v2 format coexistence (D024) +- `topic_category` casing is inconsistent across records (known data quality issue — see KNOWLEDGE) +- Stage 4 classification data (topic_tags per moment) stored in Redis, not DB columns + +--- + +## CSS Architecture + +### Overview + +| Property | Value | +|---|---| +| File | `frontend/src/App.css` | +| Lines | 5,820 | +| Unique classes | ~589 | +| Naming convention | BEM (`block__element--modifier`) | +| Theme | Dark-only (no light mode) | +| Custom properties | 77 in `:root` (D017) | +| Accent color | Cyan `#22d3ee` | +| Font stack | System fonts (no custom fonts) | +| Preprocessor | None | +| CSS Modules | None | +| Component library | None | + +### Custom Property Categories (77 total) + +The 77 CSS custom properties in `:root` cover these semantic categories: + +- **Surface colors:** page background, card backgrounds, nav, footer, input backgrounds +- **Text colors:** primary, secondary, muted, inverse, link, heading +- **Accent colors:** primary cyan, hover/active states, focus rings +- **Badge colors:** Per-category badge pairs (background + text) for 7 topic categories +- **Status colors:** Success/warning/error/info for admin states +- **Border colors:** Default, hover, focus, divider +- **Shadow colors:** Elevation shadows, glow effects +- **Overlay colors:** Modal/dropdown overlays + +### Breakpoints + +Four distinct breakpoint values used across 10 `@media` blocks: + +| Breakpoint | Usage | +|---|---| +| 480px | Narrow mobile — compact card layouts | +| 600px | Wider mobile — grid adjustments | +| 640px | Small tablet — content width changes | +| 768px | Desktop → mobile transition — sidebar collapse, layout shift | + +**Inconsistency risk:** Four different mobile breakpoints means some components respond at different widths. Phase 2 should standardize to 2–3 breakpoints (e.g., 480px mobile, 768px tablet, 1024px desktop). + +### Layout Patterns + +- **Page max-width:** 64rem (widened from 48rem in M005, D019) +- **Technique page:** CSS grid 2-column (1fr main + 22rem sidebar), collapses at 768px +- **Card layouts:** CSS grid with `auto-fill, minmax(...)` for responsive grids +- **Collapsible sections:** `grid-template-rows: 0fr/1fr` animation pattern +- **Sticky elements:** ToC sidebar, reading header (via `position: sticky`) + +### Dead CSS Risk + +With 5,820 lines and ~589 classes in a single file, dead CSS accumulation is likely. No tooling (PurgeCSS, unused-css-detector) is configured. Phase 2 additions will increase this debt. **Recommendation:** Run a dead CSS analysis before starting Phase 2 frontend work, and consider splitting App.css by feature area or adopting CSS modules for new components. + +--- + +## Infrastructure Stack + +### Docker Services + +| Service | Image | Container | Port | Volume | +|---|---|---|---|---| +| PostgreSQL 16 | postgres:16-alpine | chrysopedia-db | 5433:5432 | chrysopedia_postgres_data | +| Redis 7 | redis:7-alpine | chrysopedia-redis | 6379 (internal) | — | +| Qdrant 1.13.2 | qdrant/qdrant:v1.13.2 | chrysopedia-qdrant | 6333 (internal) | chrysopedia_qdrant_data | +| Ollama | ollama/ollama:latest | chrysopedia-ollama | 11434 (internal) | chrysopedia_ollama_data | +| API (FastAPI) | Dockerfile.api | chrysopedia-api | 8000 (internal) | Bind: backend/, prompts/ | +| Worker (Celery) | Dockerfile.api (same image) | chrysopedia-worker | — | Bind: backend/, prompts/ | +| Watcher | Dockerfile.api (same image) | chrysopedia-watcher | — | Bind: watch dir | +| Web (nginx) | Dockerfile.web | chrysopedia-web-8096 | 8096:80 | — | + +### Network + +- Compose subnet: 172.32.0.0/24 (D015) +- External access: nginx on port 8096 proxies to API container (port 8000 internal) +- Vite dev proxy: `/api` → `localhost:8001` + +### Build Pipeline + +- **Frontend:** Vite build producing static assets in `dist/`. Dockerfile.web runs `npm run build` then serves via nginx. +- **Backend:** Dockerfile.api installs Python dependencies, runs uvicorn (API) or celery (worker/watcher). +- **Build-time injection:** `__APP_VERSION__`, `__BUILD_DATE__`, `__GIT_COMMIT__` via Vite `define` (must use `JSON.stringify` — see KNOWLEDGE). +- **Docker ARG→ENV ordering:** ARG must precede ENV which must precede `RUN npm run build` (KNOWLEDGE entry). + +### Deployment + +```bash +# Standard deploy on ub01 +cd /vmPool/r/repos/xpltdco/chrysopedia +git pull +docker compose build && docker compose up -d +``` + +No CI/CD pipeline. Manual git pull + compose rebuild. Alembic not configured — schema changes require manual DDL. + +--- + +## Phase 2 Integration Recommendations + +### Adding New Routes + +1. Add `<Route path="/new-path" element={<NewPage />} />` to the `<Routes>` block in `App.tsx` +2. Create page component in `frontend/src/pages/NewPage.tsx` +3. Call `useDocumentTitle('Page Title — Chrysopedia')` in the component +4. No nginx configuration needed — SPA fallback handles all paths +5. **Risk:** No code splitting configured. All page components are bundled together. Phase 2 should add `React.lazy()` + `Suspense` for new heavy pages. + +### Adding New API Endpoints + +1. Create router file: `backend/routers/foo.py` with `APIRouter(prefix="/foo", tags=["foo"])` +2. Register in `backend/main.py`: `app.include_router(foo.router, prefix="/api/v1")` +3. Define Pydantic schemas in `backend/schemas.py` +4. Use paginated response pattern: `{items, total, offset, limit}` (not bare lists) +5. Admin endpoints go under `/api/v1/admin/` prefix +6. **Risk:** No request validation middleware, no rate limiting, no auth. Phase 2 should decide on auth strategy early (D033 defers Stripe Connect, but admin endpoints need protection at minimum). + +### Adding New Frontend API Calls + +1. Add TypeScript interface to `public-client.ts` +2. Add function using `request<T>` helper with appropriate HTTP method +3. Base URL is relative `/api/v1` — nginx proxies to backend +4. **Risk:** `public-client.ts` is already ~600 lines. Consider splitting by domain (search, techniques, creators, admin) in Phase 2. + +### Adding New Database Models + +1. Add model class to `backend/models.py` +2. Add Pydantic schema to `backend/schemas.py` +3. **No Alembic configured** — create migration manually or set up Alembic first +4. Use `datetime.now(timezone.utc).replace(tzinfo=None)` for timestamp defaults (D002) +5. Avoid column names that shadow ORM functions (e.g., `relationship`) — see KNOWLEDGE +6. **Risk:** All 13 models in one file. Phase 2 with significant new models should split `models.py` by domain. + +### CSS Additions + +1. Append to `frontend/src/App.css` using BEM naming convention +2. Use existing CSS custom properties for all colors — never hardcode hex values +3. Use existing breakpoint values (prefer 768px for the mobile/desktop split) +4. **Risk (highest):** Monolithic 5,820-line CSS file with no scoping. Any selector conflict affects the entire app. **Strong recommendation:** adopt CSS modules for all new Phase 2 components, and progressively migrate existing components during major refactors. At minimum, namespace new selectors (e.g., `.p2-chat__container` for Phase 2 chat feature). + +### Key Phase 2 Risks + +| # | Risk | Impact | Mitigation | +|---|---|---|---| +| 1 | Monolithic CSS (5,820 lines, no scoping) | Selector conflicts, dead CSS accumulation, maintenance cost | CSS modules for new components, PurgeCSS audit, progressive splitting | +| 2 | No authentication | Admin endpoints open to network, no user sessions for creator tools | Auth middleware decision needed before M019 (D033 defers Stripe) | +| 3 | No Alembic | Schema changes are manual DDL, no migration history, no rollback | Set up Alembic in M019 foundations milestone | +| 4 | No code splitting | Bundle grows with every new page, longer initial load | React.lazy + Suspense for new pages, route-based splitting | +| 5 | Single-file API client (~600 lines) | Merge conflicts, hard to find endpoints | Split by domain in M019 | +| 6 | Pipeline actively running | Schema/API changes can conflict with in-flight pipeline tasks | Freeze pipeline before major schema changes | +| 7 | All stale pages (83/83) | Bulk resynthesize triggered accidentally would overwrite all content | Add confirmation gate to bulk resynthesize endpoint | +| 8 | No CI/CD | Manual deploy, no automated testing in pipeline | Add GitHub Actions in M019 or M020 | + +### Integration Points with Phase 2 Architecture (D031) + +| Integration Point | Milestone | What Connects | +|---|---|---| +| INT-1: RAG Knowledge Graph | M020 (Core Experiences) | LightRAG (D032) hooks into existing Qdrant + adds NetworkX graph. New `/api/v1/chat` endpoint. | +| INT-2: Creator Portal Auth | M022 (Creator Tools) | Auth middleware (new) wraps existing admin + new creator endpoints. | +| INT-3: MinIO File Storage | M022 (Creator Tools) | MinIO (D035) serves signed URLs for presets/samples. New upload endpoints. | +| INT-4: Stripe Connect | Phase 3 (deferred) | Payment integration for creator tiers (D033 — demo-only in Phase 2). | + +--- + +## Appendix: Audit Evidence + +### Route Verification Summary + +All 12 routes visited in a live browser session (desktop viewport 1280×800) at http://ub01:8096 on 2026-04-03. Every page rendered with consistent nav bar (logo, search, Home/Topics/Creators, Admin dropdown) and footer (version, build date, links). + +**Visual observations:** +- Dark theme only — no light mode toggle +- No loading skeletons — pages render fully or show empty states +- Admin pages accessible without login (consistent with single-admin design) +- Minor 422 console warnings on admin pages (benign — from fetch of endpoints requiring parameters) + +### Endpoint Probe Results + +| Category | Total | Probed | Confirmed via Router | 200 OK | 422 (validation) | Not Probed | +|---|---|---|---|---|---|---| +| Public | 10 | 9 | 1 | 9 | 0 | 1 | +| Browse | 5 | 5 | 0 | 5 | 0 | 0 | +| Reports | 3 | 2 | 1 | 1 | 1 | 1 | +| Pipeline Admin | 21 | 5 | 16 | 5 | 0 | 16 | +| Other | 2 | 2 | 0 | 1 | 1 | 0 | +| **Total** | **41** | **23** | **18** | **21** | **2** | **18** | + +18 endpoints not probed were confirmed from FastAPI router source code (POST/destructive operations not safe to execute during live audit with active pipeline). + +### Data Snapshot at Audit Time + +| Metric | Value | +|---|---| +| Technique pages | 83 | +| Creators (stats endpoint) | 26 | +| Creators (browse endpoint) | 25 | +| Source videos | 200 | +| Topic categories | 7 | +| Pipeline: queued | 148 | +| Pipeline: in progress | 3 | +| Pipeline: errored | 13 | +| Pipeline: complete | 36 | +| Stale pages | 83/83 | +| Celery workers | 1 online | +| Debug mode | Enabled | +| Content reports | 0 | + +### Discrepancies Log + +7 discrepancies found between source code research and live site verification: + +| # | Area | Expected | Actual | Severity | +|---|---|---|---|---| +| 1 | Search response key | `results` | `items` + `partial_matches` + `fallback_used` | Medium | +| 2 | `/techniques/random` | HTTP redirect | JSON `{slug}` | Low | +| 3 | Creator count | 26 | 25 in browse (26 in stats) | Low | +| 4 | `/api/v1/topics` shape | Paginated dict | Bare list | Low | +| 5 | `/api/v1/videos` shape | Not documented | Bare list (not paginated) | Low | +| 6 | Pipeline admin prefix | `/api/v1/pipeline/` | `/api/v1/admin/pipeline/` | Medium | +| 7 | Creator genre badges | Clean categories | Duplicate badges from casing inconsistency | Low | + +### Relevant Decision Register Entries + +Key decisions from DECISIONS.md that affect Phase 2 integration: + +- **D002:** Timestamp handling — `datetime.now(timezone.utc).replace(tzinfo=None)` +- **D009:** Async SearchService pattern for FastAPI, sync clients for Celery +- **D015:** Docker subnet 172.32.0.0/24 +- **D017:** 77 CSS custom properties, cyan accent +- **D019:** 64rem max-width, 2-column grid layout +- **D024:** Format-discriminated rendering for v1/v2 body sections +- **D025:** Search query logging in PostgreSQL + Redis cache +- **D031:** Phase 2 milestone structure (M018–M025) +- **D032:** LightRAG with Qdrant + NetworkX for chat/knowledge graph +- **D033:** Demo monetization — Stripe Connect deferred to Phase 3 +- **D035:** MinIO for file/object storage diff --git a/.gsd/milestones/M018/slices/S01/tasks/T01-VERIFY.json b/.gsd/milestones/M018/slices/S01/tasks/T01-VERIFY.json new file mode 100644 index 0000000..f3f6ce6 --- /dev/null +++ b/.gsd/milestones/M018/slices/S01/tasks/T01-VERIFY.json @@ -0,0 +1,16 @@ +{ + "schemaVersion": 1, + "taskId": "T01", + "unitId": "M018/S01/T01", + "timestamp": 1775245534335, + "passed": true, + "discoverySource": "task-plan", + "checks": [ + { + "command": "test -f .gsd/milestones/M018/slices/S01/AUDIT-FINDINGS.md", + "exitCode": 0, + "durationMs": 8, + "verdict": "pass" + } + ] +} diff --git a/.gsd/milestones/M018/slices/S01/tasks/T02-SUMMARY.md b/.gsd/milestones/M018/slices/S01/tasks/T02-SUMMARY.md new file mode 100644 index 0000000..3a46752 --- /dev/null +++ b/.gsd/milestones/M018/slices/S01/tasks/T02-SUMMARY.md @@ -0,0 +1,74 @@ +--- +id: T02 +parent: S01 +milestone: M018 +provides: [] +requires: [] +affects: [] +key_files: [".gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md"] +key_decisions: [] +patterns_established: [] +drill_down_paths: [] +observability_surfaces: [] +duration: "" +verification_result: "File exists at 467 lines with 9 H2 sections — passes all three verification checks (file exists, ≥300 lines, ≥5 sections)." +completed_at: 2026-04-03T19:49:11.326Z +blocker_discovered: false +--- + +# T02: Compiled 467-line Site Audit Report merging source code research with live audit findings, documenting all 12 routes, 41 API endpoints, 13 data models, CSS architecture, and 8 Phase 2 integration risks + +> Compiled 467-line Site Audit Report merging source code research with live audit findings, documenting all 12 routes, 41 API endpoints, 13 data models, CSS architecture, and 8 Phase 2 integration risks + +## What Happened +--- +id: T02 +parent: S01 +milestone: M018 +key_files: + - .gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md +key_decisions: + - (none) +duration: "" +verification_result: passed +completed_at: 2026-04-03T19:49:11.326Z +blocker_discovered: false +--- + +# T02: Compiled 467-line Site Audit Report merging source code research with live audit findings, documenting all 12 routes, 41 API endpoints, 13 data models, CSS architecture, and 8 Phase 2 integration risks + +**Compiled 467-line Site Audit Report merging source code research with live audit findings, documenting all 12 routes, 41 API endpoints, 13 data models, CSS architecture, and 8 Phase 2 integration risks** + +## What Happened + +Merged S01-RESEARCH.md (source code analysis) and AUDIT-FINDINGS.md (live site verification from T01) into a single comprehensive SITE-AUDIT-REPORT.md. The report has 9 major sections: Executive Summary, Route Map, Component Inventory, API Surface Map, Data Model, CSS Architecture, Infrastructure Stack, Phase 2 Integration Recommendations, and Appendix with audit evidence. Cross-referenced 12 DECISIONS.md entries where they affect Phase 2 integration. Identified 8 key Phase 2 risks with mitigations and documented 4 integration points from the Phase 2 milestone structure. + +## Verification + +File exists at 467 lines with 9 H2 sections — passes all three verification checks (file exists, ≥300 lines, ≥5 sections). + +## Verification Evidence + +| # | Command | Exit Code | Verdict | Duration | +|---|---------|-----------|---------|----------| +| 1 | `test -f .gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md && test $(wc -l < .gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md) -ge 300 && grep -c '^## ' .gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md | grep -q '[5-9]'` | 0 | ✅ pass | 100ms | + + +## Deviations + +Research counted 43 endpoints; report reconciled to 41 after deduplication. No material deviation. + +## Known Issues + +None. + +## Files Created/Modified + +- `.gsd/milestones/M018/slices/S01/SITE-AUDIT-REPORT.md` + + +## Deviations +Research counted 43 endpoints; report reconciled to 41 after deduplication. No material deviation. + +## Known Issues +None.