diff --git a/API-Surface.md b/API-Surface.md index f09f61c..2e9655f 100644 --- a/API-Surface.md +++ b/API-Surface.md @@ -1,6 +1,6 @@ # API Surface -61 API endpoints grouped by domain. All served by FastAPI under `/api/v1/`. +71 API endpoints grouped by domain. All served by FastAPI under `/api/v1/`. ## Public Endpoints (10) @@ -84,6 +84,25 @@ All under prefix `/api/v1/consent/`. All require Bearer JWT. | GET | `/consent/videos/{video_id}/history` | Creator (owner) or Admin | Versioned audit trail of consent changes for a video. | | GET | `/consent/admin/summary` | Admin only | Aggregate consent flag counts across all videos. | +## Posts Endpoints (5) — M023/S01 + +All under prefix `/api/v1/posts/`. Public list/get; create/update/delete require Bearer JWT with creator ownership. + +| Method | Path | Auth | Purpose | +|--------|------|------|---------| +| GET | `/api/v1/posts?creator_id=` | Public (auth-aware) | List published posts; shows drafts to owner | +| GET | `/api/v1/posts/{id}` | Public (auth-aware) | Single post detail | +| POST | `/api/v1/posts` | Bearer JWT (creator) | Create post with title, body (Tiptap JSON), is_published | +| PUT | `/api/v1/posts/{id}` | Bearer JWT (creator owner) | Update post fields | +| DELETE | `/api/v1/posts/{id}` | Bearer JWT (creator owner) | Delete post + cascade delete attachments from MinIO | + +## File Endpoints (2) — M023/S01 + +| Method | Path | Auth | Purpose | +|--------|------|------|---------| +| POST | `/api/v1/files/upload` | Bearer JWT | Multipart file upload to MinIO, returns PostAttachment | +| GET | `/api/v1/files/{attachment_id}/download` | Public | Presigned MinIO download URL (redirect) | + ## Report Endpoints (3) | Method | Path | Purpose | @@ -137,6 +156,15 @@ All under prefix `/api/v1/admin/pipeline/`. |--------|------|---------| | POST | `/api/v1/admin/creators/{slug}/extract-profile` | Queue personality profile extraction task | + +### Shorts Admin (3) — M023/S03 + +| Method | Path | Purpose | +|--------|------|---------| +| POST | `/api/v1/admin/shorts/{highlight_id}/generate` | Queue short generation for approved highlight (3 presets) — 202 response | +| GET | `/api/v1/admin/shorts/{highlight_id}` | List generated shorts for a highlight with per-preset status | +| GET | `/api/v1/admin/shorts/{short_id}/download` | Presigned MinIO download URL for completed short | + ## Other Endpoints (2) | Method | Path | Notes | diff --git a/Chat-Engine.md b/Chat-Engine.md index 8353f9a..bcb5927 100644 --- a/Chat-Engine.md +++ b/Chat-Engine.md @@ -140,6 +140,73 @@ Floating chat bubble on creator detail pages. Fixed-position bottom-right. - **Conversation threading** — `conversationId` generated via `crypto.randomUUID()` on first send, threaded through `streamChat()`, updated from done event - **Reset on close** — messages and conversationId cleared when panel closes +## Personality Interpolation (M023/S02, S04) + +The chat engine modulates the system prompt based on (0.0–1.0) sent in the request. When a creator is specified, their JSONB is loaded and progressively injected into the system prompt. + +### 5-Tier System (D044) + +| Weight Range | Tier | Profile Fields Included | Instruction | +|-------------|------|------------------------|-------------| +| < 0.2 | None | — | Pure encyclopedic (no personality block) | +| 0.2–0.39 | Subtle Reference | Basic tone | "Subtly reference this creator's style" | +| 0.4–0.59 | Creator Tone | + descriptors, explanation approach | "Adopt this creator's teaching tone" | +| 0.6–0.79 | Creator Voice | + signature phrases (count scaled with weight) | "Channel this creator's voice" | +| 0.8–0.89 | Full Voice | + vocabulary, style markers | "Speak in this creator's voice" | +| 0.9–1.0 | Full Embodiment | + summary paragraph | "Fully embody this creator" | + +### Temperature Scaling + +Linear: + +- weight=0.0 → temperature 0.3 (encyclopedic, precise) +- weight=1.0 → temperature 0.5 (still grounded, slight creative variance) + +### Request Format + + + +### Graceful Fallback + +If the creator has no (null JSONB), the system falls back to pure encyclopedic mode regardless of weight value. DB errors during profile fetch are caught and logged — never crash the stream. + +## Personality Interpolation (M023/S02, S04) + +The chat engine modulates the system prompt based on `personality_weight` (0.0–1.0) sent in the request. When a creator is specified, their `personality_profile` JSONB is loaded and progressively injected into the system prompt. + +### 5-Tier System (D044) + +| Weight Range | Tier | Profile Fields Included | Instruction | +|-------------|------|------------------------|-------------| +| < 0.2 | None | — | Pure encyclopedic (no personality block) | +| 0.2–0.39 | Subtle Reference | Basic tone | "Subtly reference this creator's style" | +| 0.4–0.59 | Creator Tone | + descriptors, explanation approach | "Adopt this creator's teaching tone" | +| 0.6–0.79 | Creator Voice | + signature phrases (count scaled with weight) | "Channel this creator's voice" | +| 0.8–0.89 | Full Voice | + vocabulary, style markers | "Speak in this creator's voice" | +| 0.9–1.0 | Full Embodiment | + summary paragraph | "Fully embody this creator" | + +### Temperature Scaling + +Linear: `temperature = 0.3 + weight * 0.2` + +- weight=0.0 → temperature 0.3 (encyclopedic, precise) +- weight=1.0 → temperature 0.5 (still grounded, slight creative variance) + +### Request Format + +```json +{ + "query": "How do I make a reese bass?", + "creator": "creator-slug", + "personality_weight": 0.7, + "conversation_id": "uuid" +} +``` + +### Graceful Fallback + +If the creator has no `personality_profile` (null JSONB), the system falls back to pure encyclopedic mode regardless of weight value. DB errors during profile fetch are caught and logged — never crash the stream. + ## Key Files - `backend/chat_service.py` — ChatService with history load/save, retrieve-prompt-stream pipeline @@ -157,6 +224,8 @@ Floating chat bubble on creator detail pages. Fixed-position bottom-right. - **Auto-generate conversation_id** — Server creates UUID when client omits it, ensuring consistent `done` event shape - **Widget resets on close** — Clean slate UX; no persistence across open/close cycles - **Client-side suggested questions** — Generated from technique titles/categories without API call +- **5-tier interpolation** — Progressive field inclusion replaces 3-tier step function (D044 supersedes D043) +- **5-tier interpolation** — Progressive field inclusion replaces 3-tier step function (D044 supersedes D043) - **Citation parsing duplicated** — ChatPage and ChatWidget each parse citations independently (extracted utility deferred) - **Standalone ASGI test client** — Tests use mocked DB to avoid PostgreSQL dependency diff --git a/Configuration.md b/Configuration.md index 13d9768..8175071 100644 --- a/Configuration.md +++ b/Configuration.md @@ -55,6 +55,21 @@ Configuration is managed via environment variables in two `.env` files loaded by | `GIT_COMMIT` | `unknown` | Git commit hash (injected into frontend) | | `DEBUG` | `false` | Debug mode | +### MinIO Object Storage (M023/S01) + +| Variable | Description | +|----------|-------------| +| `MINIO_ENDPOINT` | MinIO server endpoint (default: `chrysopedia-minio:9000`) | +| `MINIO_ACCESS_KEY` | MinIO access key | +| `MINIO_SECRET_KEY` | MinIO secret key | +| `MINIO_BUCKET_NAME` | Bucket name for file uploads (default: `chrysopedia`) | + +### Video Source (M023/S03) + +| Variable | Default | Description | +|----------|---------|-------------| +| `VIDEO_SOURCE_PATH` | `/videos` | Path to source video files for shorts generation | + ### Watch Directory | Variable | Description | @@ -109,6 +124,13 @@ Injected at Docker build time via `docker-compose.yml` build args: - **Port:** 9621 (localhost only on host) - **Volume:** `chrysopedia_lightrag_data` +### MinIO (M023/S01) + +- **Port:** Internal only (9000 API, 9001 console) +- **Volume:** `chrysopedia_minio_data` +- **Healthcheck:** `mc ready local` +- **Access:** API-generated presigned URLs only (no public port) + ### Web (nginx) - **Port:** 8096:80 (host:container) diff --git a/Data-Model.md b/Data-Model.md index 05b321a..8e01a95 100644 --- a/Data-Model.md +++ b/Data-Model.md @@ -1,6 +1,6 @@ # Data Model -20 SQLAlchemy models in `backend/models.py`. +23 SQLAlchemy models in `backend/models.py`. ## Entity Relationship Overview @@ -19,7 +19,11 @@ Creator (1) ──→ (N) SourceVideo (1) ──→ (N) TranscriptSegment │ ├──→ (N) CreatorFollow ←── User │ + ├──→ (N) Post (1) ──→ (N) PostAttachment + │ └──→ (0..1) User ──→ (N) InviteCode (created_by) + +HighlightCandidate (1) ──→ (N) GeneratedShort ``` ## Core Content Models @@ -221,6 +225,8 @@ Append-only versioned record of per-field consent changes. | **ConsentField** | kb_inclusion, training_usage, public_display | | **HighlightStatus** | candidate, approved, rejected (M021/S04) | | **ChapterStatus** | draft, approved, hidden (M021/S06) | +| **FormatPreset** | vertical, square, horizontal (M023/S03) | +| **ShortStatus** | pending, processing, complete, failed (M023/S03) | ## Migrations @@ -230,6 +236,8 @@ Append-only versioned record of per-field consent changes. | 021 | Add trim_start/trim_end to highlight_candidates (M022/S01) | | 022 | Add creator_follows table (M022/S02) | | 023 | Add personality_profile JSONB to creators (M022/S06) | +| 024 | Add posts and post_attachments tables (M023/S01) | +| 025 | Add generated_shorts table with FormatPreset and ShortStatus enums (M023/S03) | ## Schema Notes @@ -243,3 +251,42 @@ Append-only versioned record of per-field consent changes. --- *See also: [[Architecture]], [[API-Surface]], [[Pipeline]], [[Authentication]]* + +### Post (M023/S01) + +| Field | Type | Notes | +|-------|------|-------| +| id | UUID PK | | +| title | Text | Post title | +| body | JSONB | Tiptap rich text JSON (canonical format) | +| is_published | Boolean | Default false — draft until published | +| creator_id | FK → Creator | | +| created_at | Timestamp | | +| updated_at | Timestamp | | + +### PostAttachment (M023/S01) + +| Field | Type | Notes | +|-------|------|-------| +| id | UUID PK | | +| post_id | FK → Post | Cascade delete | +| filename | String | Original upload filename | +| content_type | String | MIME type | +| object_key | String | MinIO object key | +| file_size | Integer | Bytes | +| created_at | Timestamp | | + +### GeneratedShort (M023/S03) + +| Field | Type | Notes | +|-------|------|-------| +| id | UUID PK | | +| highlight_candidate_id | FK → HighlightCandidate | | +| preset | Enum(FormatPreset) | vertical / square / horizontal | +| status | Enum(ShortStatus) | pending / processing / complete / failed | +| object_key | String | MinIO key (shorts/{highlight_id}/{preset}.mp4) | +| file_size | Integer | Nullable — bytes | +| duration_secs | Float | Nullable | +| error_message | Text | Nullable — on failure | +| created_at | Timestamp | | +| updated_at | Timestamp | | diff --git a/Decisions.md b/Decisions.md index e99a1f4..14e6835 100644 --- a/Decisions.md +++ b/Decisions.md @@ -61,6 +61,14 @@ Architectural and pattern decisions made during Chrysopedia development. Append- | D021 | M011 findings triage | 12/16 approved; denied beginner paths, YouTube links, hide admin, CTA label | | D030 | ToC scroll-spy rootMargin | `0px 0px -70% 0px` — active when in top 30% of viewport | +## M023 Decisions + +| # | When | Decision | Choice | Rationale | +|---|------|----------|--------|-----------| +| D042 | M023/S01 | Rich text editor for creator posts | Tiptap (headless, React) with StarterKit + Link + Placeholder. Store Tiptap JSON in JSONB column, render client-side via @tiptap/html. | Headless architecture fits dark theme customization. JSON storage is lossless and enables future server-side rendering. No HTML sanitization needed. | +| D043 | M023/S02 | Personality weight modulation strategy | 3-tier intensity (<0.4 subtle, 0.4–0.8 voice, ≥0.8 embody) with temperature scaling 0.3–0.5. **Superseded by D044.** | Initial stepped approach; replaced by continuous interpolation. | +| D044 | M023/S04 | Personality weight modulation strategy (revision) | 5-tier continuous interpolation. Progressive field inclusion: <0.2 no personality; 0.2+ tone; 0.4+ descriptors; 0.6+ phrases (count scaled); 0.8+ vocabulary/style; 0.9+ summary. Temperature: 0.3 + weight × 0.2. | 3-tier step function had jarring transitions. Continuous interpolation with progressive field inclusion gives finer control. 0.0–0.19 dead zone ensures purely encyclopedic mode. | + --- *See also: [[Architecture]], [[Development-Guide]]* diff --git a/Frontend.md b/Frontend.md index 9f6eec6..3f9308a 100644 --- a/Frontend.md +++ b/Frontend.md @@ -20,6 +20,9 @@ React 18 + TypeScript + Vite SPA. No UI library, no state management library, no | `/admin/reports` | AdminReports | Admin* | Content reports | | `/admin/pipeline` | AdminPipeline | Admin* | Pipeline management | | `/admin/techniques` | AdminTechniquePages | Admin* | Technique page admin | +| `/creator/posts` | PostsList | Creator JWT | Creator post management — status badges, edit/delete (M023/S01) | +| `/creator/posts/new` | PostEditor | Creator JWT | Tiptap rich text editor with file attachments (M023/S01) | +| `/creator/posts/:postId/edit` | PostEditor | Creator JWT | Edit existing post (M023/S01) | | `*` | → Redirect `/` | — | SPA fallback | *Admin routes have no authentication gate. @@ -41,6 +44,7 @@ React 18 + TypeScript + Vite SPA. No UI library, no state management library, no | CopyLinkButton | Clipboard copy with tooltip | | SocialIcons | Social media link icons (9 platforms) | | ReportIssueModal | Content report submission | +| PostsFeed | Published posts feed with Tiptap JSON→HTML rendering and file downloads (M023/S01) | | ChatWidget | Floating chat bubble on creator pages — SSE streaming, citations, suggested questions (M022/S03) | | PersonalityProfile | Collapsible creator personality display — 3 sub-cards (Teaching Style, Vocabulary, Style) (M022/S06) | @@ -87,6 +91,46 @@ Collapsible personality display on creator detail pages. - **Gracefully hidden** when profile is null - **Files:** `PersonalityProfile.tsx`, styles in `App.css` +### PostEditor (M023/S01) + +Rich text editor at `/creator/posts/new` and `/creator/posts/:postId/edit`. + +- **Tiptap v3** with StarterKit + Link + Placeholder extensions +- **Formatting toolbar** — bold, italic, lists, headings, links +- **File attachment zone** — drag-and-drop with multipart upload to MinIO +- **Publish toggle** — draft/published state +- **Sequential save flow** — create post → upload files +- **Files:** `PostEditor.tsx`, `PostEditor.module.css` + +### PostsList (M023/S01) + +Creator post management at `/creator/posts`. + +- **Status badges** — Draft / Published +- **Edit/Delete actions** with confirmation dialog +- **SidebarNav integration** — accessible from creator dashboard +- **Files:** `PostsList.tsx`, `PostsList.module.css` + +### Shorts UI (M023/S03) + +Shorts generation controls in HighlightQueue. + +- **Generate button** — visible on approved highlights with no in-progress shorts +- **Per-preset status badges** — color-coded pending/processing/complete/failed with pulsing animation +- **Download links** — open presigned MinIO URLs in new tab +- **5s polling** — while any shorts are processing, auto-stops when all settle +- **Files:** `HighlightQueue.tsx`, `HighlightQueue.module.css` (updated) + +### Personality Slider (M023/S02, S04) + +ChatWidget personality control. + +- **Range input** (0.0–1.0, step 0.1) with gradient track fill via `--slider-fill` CSS custom property +- **Tier labels** — Encyclopedic / Subtle Reference / Creator Tone / Creator Voice / Full Embodiment +- **Numeric display** with tabular-nums for stable width +- **Labels match backend 5-tier interpolation boundaries** (0.2/0.4/0.6/0.8) +- **Files:** `ChatWidget.tsx`, `ChatWidget.module.css` (updated) + ## Hooks | Hook | Purpose | @@ -106,7 +150,9 @@ API modules: - `chat.ts` — SSE streaming client for POST /api/v1/chat using `fetch()` + `ReadableStream`, `ChatDoneMeta` type - `videos.ts` — chapter management functions - `auth.ts` — authentication + impersonation functions -- `highlights.ts` — creator highlight review functions (M022/S01) +- `highlights.ts` +- `posts.ts` — post CRUD and file upload/download functions (M023/S01) +- `shorts.ts` — short generation, listing, and download functions (M023/S03) — creator highlight review functions (M022/S01) - `follows.ts` — follow/unfollow/status/list functions (M022/S02) - `creators.ts` — creator detail with personality_profile and follower_count types (M022/S02, S06) diff --git a/Home.md b/Home.md index fb9d468..cdede57 100644 --- a/Home.md +++ b/Home.md @@ -8,7 +8,7 @@ Producers can search for specific techniques and find timestamped key moments, s - [[Architecture]] — System architecture, Docker services, network topology - [[Data-Model]] — SQLAlchemy models, relationships, enums -- [[API-Surface]] — All 60+ API endpoints grouped by domain +- [[API-Surface]] — All 70+ API endpoints grouped by domain - [[Frontend]] — Routes, components, hooks, CSS architecture - [[Pipeline]] — 6-stage LLM extraction pipeline, prompt system - [[Chat-Engine]] — Streaming Q&A with multi-turn memory @@ -17,7 +17,7 @@ Producers can search for specific techniques and find timestamped key moments, s - [[Search-Retrieval]] — LightRAG + Qdrant retrieval cascade - [[Deployment]] — Docker Compose setup, rebuild commands - [[Development-Guide]] — Local dev setup, common gotchas -- [[Decisions]] — Architectural decisions register (D001–D041) +- [[Decisions]] — Architectural decisions register (D001–D044) ## Features @@ -33,6 +33,9 @@ Producers can search for specific techniques and find timestamped key moments, s - **Creator Tiers** — Free/Pro/Premium tier configuration with Coming Soon placeholders (M022) - **Highlight Detection v2** — 10-dimension scoring with audio proxy signals, creator review queue (M022) - **Chat Widget** — Floating creator-scoped chat bubble with streaming SSE and citations (M022) +- **Post Editor + File Sharing** — Tiptap rich text editor, MinIO file attachments, CRUD API (M023) +- **Shorts Generation Pipeline** — ffmpeg clip extraction in 3 format presets from approved highlights (M023) +- **Personality Interpolation** — 5-tier continuous personality weight system for chat (M023) - **Multi-Turn Chat Memory** — Redis-backed conversation history with conversation_id threading (M022) - **Creator Dashboard** — Video management, chapter editing, consent controls @@ -64,4 +67,4 @@ Producers can search for specific techniques and find timestamped key moments, s --- -*Last updated: 2026-04-04 — M022 follow system, personality profiles, highlight v2, chat widget, multi-turn memory, creator tiers* +*Last updated: 2026-04-04 — M023 post editor, file sharing, shorts pipeline, personality interpolation diff --git a/_Sidebar.md b/_Sidebar.md index 0d8f5bc..3236c84 100644 --- a/_Sidebar.md +++ b/_Sidebar.md @@ -15,6 +15,7 @@ - [[Search-Retrieval]] - [[Highlights]] - [[Personality-Profiles]] +- Posts (via Post Editor) **Reference** - [[API-Surface]]