docs: update wiki with M023 features — post editor, file sharing, shorts pipeline, personality interpolation, D042-D044
Updated pages: Home, Data-Model, API-Surface, Frontend, Chat-Engine, Decisions, Configuration, _Sidebar New content: - Post/PostAttachment/GeneratedShort models and migrations 024-025 - 10 new API endpoints (5 posts CRUD, 2 files, 3 shorts admin) — total 71 - PostEditor (Tiptap v3), PostsFeed, PostsList, shorts UI, personality slider - 5-tier personality interpolation system (D044) - MinIO and video_source_path configuration - Decisions D042 (Tiptap), D043 (3-tier, superseded), D044 (5-tier interpolation)
jlightner
parent
49ab6d029a
commit
d1bd67a9eb
8 changed files with 230 additions and 6 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# API Surface
|
||||
|
||||
61 API endpoints grouped by domain. All served by FastAPI under `/api/v1/`.
|
||||
71 API endpoints grouped by domain. All served by FastAPI under `/api/v1/`.
|
||||
|
||||
## Public Endpoints (10)
|
||||
|
||||
|
|
@ -84,6 +84,25 @@ All under prefix `/api/v1/consent/`. All require Bearer JWT.
|
|||
| GET | `/consent/videos/{video_id}/history` | Creator (owner) or Admin | Versioned audit trail of consent changes for a video. |
|
||||
| GET | `/consent/admin/summary` | Admin only | Aggregate consent flag counts across all videos. |
|
||||
|
||||
## Posts Endpoints (5) — M023/S01
|
||||
|
||||
All under prefix `/api/v1/posts/`. Public list/get; create/update/delete require Bearer JWT with creator ownership.
|
||||
|
||||
| Method | Path | Auth | Purpose |
|
||||
|--------|------|------|---------|
|
||||
| GET | `/api/v1/posts?creator_id=` | Public (auth-aware) | List published posts; shows drafts to owner |
|
||||
| GET | `/api/v1/posts/{id}` | Public (auth-aware) | Single post detail |
|
||||
| POST | `/api/v1/posts` | Bearer JWT (creator) | Create post with title, body (Tiptap JSON), is_published |
|
||||
| PUT | `/api/v1/posts/{id}` | Bearer JWT (creator owner) | Update post fields |
|
||||
| DELETE | `/api/v1/posts/{id}` | Bearer JWT (creator owner) | Delete post + cascade delete attachments from MinIO |
|
||||
|
||||
## File Endpoints (2) — M023/S01
|
||||
|
||||
| Method | Path | Auth | Purpose |
|
||||
|--------|------|------|---------|
|
||||
| POST | `/api/v1/files/upload` | Bearer JWT | Multipart file upload to MinIO, returns PostAttachment |
|
||||
| GET | `/api/v1/files/{attachment_id}/download` | Public | Presigned MinIO download URL (redirect) |
|
||||
|
||||
## Report Endpoints (3)
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|
|
@ -137,6 +156,15 @@ All under prefix `/api/v1/admin/pipeline/`.
|
|||
|--------|------|---------|
|
||||
| POST | `/api/v1/admin/creators/{slug}/extract-profile` | Queue personality profile extraction task |
|
||||
|
||||
|
||||
### Shorts Admin (3) — M023/S03
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|--------|------|---------|
|
||||
| POST | `/api/v1/admin/shorts/{highlight_id}/generate` | Queue short generation for approved highlight (3 presets) — 202 response |
|
||||
| GET | `/api/v1/admin/shorts/{highlight_id}` | List generated shorts for a highlight with per-preset status |
|
||||
| GET | `/api/v1/admin/shorts/{short_id}/download` | Presigned MinIO download URL for completed short |
|
||||
|
||||
## Other Endpoints (2)
|
||||
|
||||
| Method | Path | Notes |
|
||||
|
|
|
|||
|
|
@ -140,6 +140,73 @@ Floating chat bubble on creator detail pages. Fixed-position bottom-right.
|
|||
- **Conversation threading** — `conversationId` generated via `crypto.randomUUID()` on first send, threaded through `streamChat()`, updated from done event
|
||||
- **Reset on close** — messages and conversationId cleared when panel closes
|
||||
|
||||
## Personality Interpolation (M023/S02, S04)
|
||||
|
||||
The chat engine modulates the system prompt based on (0.0–1.0) sent in the request. When a creator is specified, their JSONB is loaded and progressively injected into the system prompt.
|
||||
|
||||
### 5-Tier System (D044)
|
||||
|
||||
| Weight Range | Tier | Profile Fields Included | Instruction |
|
||||
|-------------|------|------------------------|-------------|
|
||||
| < 0.2 | None | — | Pure encyclopedic (no personality block) |
|
||||
| 0.2–0.39 | Subtle Reference | Basic tone | "Subtly reference this creator's style" |
|
||||
| 0.4–0.59 | Creator Tone | + descriptors, explanation approach | "Adopt this creator's teaching tone" |
|
||||
| 0.6–0.79 | Creator Voice | + signature phrases (count scaled with weight) | "Channel this creator's voice" |
|
||||
| 0.8–0.89 | Full Voice | + vocabulary, style markers | "Speak in this creator's voice" |
|
||||
| 0.9–1.0 | Full Embodiment | + summary paragraph | "Fully embody this creator" |
|
||||
|
||||
### Temperature Scaling
|
||||
|
||||
Linear:
|
||||
|
||||
- weight=0.0 → temperature 0.3 (encyclopedic, precise)
|
||||
- weight=1.0 → temperature 0.5 (still grounded, slight creative variance)
|
||||
|
||||
### Request Format
|
||||
|
||||
|
||||
|
||||
### Graceful Fallback
|
||||
|
||||
If the creator has no (null JSONB), the system falls back to pure encyclopedic mode regardless of weight value. DB errors during profile fetch are caught and logged — never crash the stream.
|
||||
|
||||
## Personality Interpolation (M023/S02, S04)
|
||||
|
||||
The chat engine modulates the system prompt based on `personality_weight` (0.0–1.0) sent in the request. When a creator is specified, their `personality_profile` JSONB is loaded and progressively injected into the system prompt.
|
||||
|
||||
### 5-Tier System (D044)
|
||||
|
||||
| Weight Range | Tier | Profile Fields Included | Instruction |
|
||||
|-------------|------|------------------------|-------------|
|
||||
| < 0.2 | None | — | Pure encyclopedic (no personality block) |
|
||||
| 0.2–0.39 | Subtle Reference | Basic tone | "Subtly reference this creator's style" |
|
||||
| 0.4–0.59 | Creator Tone | + descriptors, explanation approach | "Adopt this creator's teaching tone" |
|
||||
| 0.6–0.79 | Creator Voice | + signature phrases (count scaled with weight) | "Channel this creator's voice" |
|
||||
| 0.8–0.89 | Full Voice | + vocabulary, style markers | "Speak in this creator's voice" |
|
||||
| 0.9–1.0 | Full Embodiment | + summary paragraph | "Fully embody this creator" |
|
||||
|
||||
### Temperature Scaling
|
||||
|
||||
Linear: `temperature = 0.3 + weight * 0.2`
|
||||
|
||||
- weight=0.0 → temperature 0.3 (encyclopedic, precise)
|
||||
- weight=1.0 → temperature 0.5 (still grounded, slight creative variance)
|
||||
|
||||
### Request Format
|
||||
|
||||
```json
|
||||
{
|
||||
"query": "How do I make a reese bass?",
|
||||
"creator": "creator-slug",
|
||||
"personality_weight": 0.7,
|
||||
"conversation_id": "uuid"
|
||||
}
|
||||
```
|
||||
|
||||
### Graceful Fallback
|
||||
|
||||
If the creator has no `personality_profile` (null JSONB), the system falls back to pure encyclopedic mode regardless of weight value. DB errors during profile fetch are caught and logged — never crash the stream.
|
||||
|
||||
## Key Files
|
||||
|
||||
- `backend/chat_service.py` — ChatService with history load/save, retrieve-prompt-stream pipeline
|
||||
|
|
@ -157,6 +224,8 @@ Floating chat bubble on creator detail pages. Fixed-position bottom-right.
|
|||
- **Auto-generate conversation_id** — Server creates UUID when client omits it, ensuring consistent `done` event shape
|
||||
- **Widget resets on close** — Clean slate UX; no persistence across open/close cycles
|
||||
- **Client-side suggested questions** — Generated from technique titles/categories without API call
|
||||
- **5-tier interpolation** — Progressive field inclusion replaces 3-tier step function (D044 supersedes D043)
|
||||
- **5-tier interpolation** — Progressive field inclusion replaces 3-tier step function (D044 supersedes D043)
|
||||
- **Citation parsing duplicated** — ChatPage and ChatWidget each parse citations independently (extracted utility deferred)
|
||||
- **Standalone ASGI test client** — Tests use mocked DB to avoid PostgreSQL dependency
|
||||
|
||||
|
|
|
|||
|
|
@ -55,6 +55,21 @@ Configuration is managed via environment variables in two `.env` files loaded by
|
|||
| `GIT_COMMIT` | `unknown` | Git commit hash (injected into frontend) |
|
||||
| `DEBUG` | `false` | Debug mode |
|
||||
|
||||
### MinIO Object Storage (M023/S01)
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `MINIO_ENDPOINT` | MinIO server endpoint (default: `chrysopedia-minio:9000`) |
|
||||
| `MINIO_ACCESS_KEY` | MinIO access key |
|
||||
| `MINIO_SECRET_KEY` | MinIO secret key |
|
||||
| `MINIO_BUCKET_NAME` | Bucket name for file uploads (default: `chrysopedia`) |
|
||||
|
||||
### Video Source (M023/S03)
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `VIDEO_SOURCE_PATH` | `/videos` | Path to source video files for shorts generation |
|
||||
|
||||
### Watch Directory
|
||||
|
||||
| Variable | Description |
|
||||
|
|
@ -109,6 +124,13 @@ Injected at Docker build time via `docker-compose.yml` build args:
|
|||
- **Port:** 9621 (localhost only on host)
|
||||
- **Volume:** `chrysopedia_lightrag_data`
|
||||
|
||||
### MinIO (M023/S01)
|
||||
|
||||
- **Port:** Internal only (9000 API, 9001 console)
|
||||
- **Volume:** `chrysopedia_minio_data`
|
||||
- **Healthcheck:** `mc ready local`
|
||||
- **Access:** API-generated presigned URLs only (no public port)
|
||||
|
||||
### Web (nginx)
|
||||
|
||||
- **Port:** 8096:80 (host:container)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Data Model
|
||||
|
||||
20 SQLAlchemy models in `backend/models.py`.
|
||||
23 SQLAlchemy models in `backend/models.py`.
|
||||
|
||||
## Entity Relationship Overview
|
||||
|
||||
|
|
@ -19,7 +19,11 @@ Creator (1) ──→ (N) SourceVideo (1) ──→ (N) TranscriptSegment
|
|||
│
|
||||
├──→ (N) CreatorFollow ←── User
|
||||
│
|
||||
├──→ (N) Post (1) ──→ (N) PostAttachment
|
||||
│
|
||||
└──→ (0..1) User ──→ (N) InviteCode (created_by)
|
||||
|
||||
HighlightCandidate (1) ──→ (N) GeneratedShort
|
||||
```
|
||||
|
||||
## Core Content Models
|
||||
|
|
@ -221,6 +225,8 @@ Append-only versioned record of per-field consent changes.
|
|||
| **ConsentField** | kb_inclusion, training_usage, public_display |
|
||||
| **HighlightStatus** | candidate, approved, rejected (M021/S04) |
|
||||
| **ChapterStatus** | draft, approved, hidden (M021/S06) |
|
||||
| **FormatPreset** | vertical, square, horizontal (M023/S03) |
|
||||
| **ShortStatus** | pending, processing, complete, failed (M023/S03) |
|
||||
|
||||
## Migrations
|
||||
|
||||
|
|
@ -230,6 +236,8 @@ Append-only versioned record of per-field consent changes.
|
|||
| 021 | Add trim_start/trim_end to highlight_candidates (M022/S01) |
|
||||
| 022 | Add creator_follows table (M022/S02) |
|
||||
| 023 | Add personality_profile JSONB to creators (M022/S06) |
|
||||
| 024 | Add posts and post_attachments tables (M023/S01) |
|
||||
| 025 | Add generated_shorts table with FormatPreset and ShortStatus enums (M023/S03) |
|
||||
|
||||
## Schema Notes
|
||||
|
||||
|
|
@ -243,3 +251,42 @@ Append-only versioned record of per-field consent changes.
|
|||
---
|
||||
|
||||
*See also: [[Architecture]], [[API-Surface]], [[Pipeline]], [[Authentication]]*
|
||||
|
||||
### Post (M023/S01)
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| id | UUID PK | |
|
||||
| title | Text | Post title |
|
||||
| body | JSONB | Tiptap rich text JSON (canonical format) |
|
||||
| is_published | Boolean | Default false — draft until published |
|
||||
| creator_id | FK → Creator | |
|
||||
| created_at | Timestamp | |
|
||||
| updated_at | Timestamp | |
|
||||
|
||||
### PostAttachment (M023/S01)
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| id | UUID PK | |
|
||||
| post_id | FK → Post | Cascade delete |
|
||||
| filename | String | Original upload filename |
|
||||
| content_type | String | MIME type |
|
||||
| object_key | String | MinIO object key |
|
||||
| file_size | Integer | Bytes |
|
||||
| created_at | Timestamp | |
|
||||
|
||||
### GeneratedShort (M023/S03)
|
||||
|
||||
| Field | Type | Notes |
|
||||
|-------|------|-------|
|
||||
| id | UUID PK | |
|
||||
| highlight_candidate_id | FK → HighlightCandidate | |
|
||||
| preset | Enum(FormatPreset) | vertical / square / horizontal |
|
||||
| status | Enum(ShortStatus) | pending / processing / complete / failed |
|
||||
| object_key | String | MinIO key (shorts/{highlight_id}/{preset}.mp4) |
|
||||
| file_size | Integer | Nullable — bytes |
|
||||
| duration_secs | Float | Nullable |
|
||||
| error_message | Text | Nullable — on failure |
|
||||
| created_at | Timestamp | |
|
||||
| updated_at | Timestamp | |
|
||||
|
|
|
|||
|
|
@ -61,6 +61,14 @@ Architectural and pattern decisions made during Chrysopedia development. Append-
|
|||
| D021 | M011 findings triage | 12/16 approved; denied beginner paths, YouTube links, hide admin, CTA label |
|
||||
| D030 | ToC scroll-spy rootMargin | `0px 0px -70% 0px` — active when in top 30% of viewport |
|
||||
|
||||
## M023 Decisions
|
||||
|
||||
| # | When | Decision | Choice | Rationale |
|
||||
|---|------|----------|--------|-----------|
|
||||
| D042 | M023/S01 | Rich text editor for creator posts | Tiptap (headless, React) with StarterKit + Link + Placeholder. Store Tiptap JSON in JSONB column, render client-side via @tiptap/html. | Headless architecture fits dark theme customization. JSON storage is lossless and enables future server-side rendering. No HTML sanitization needed. |
|
||||
| D043 | M023/S02 | Personality weight modulation strategy | 3-tier intensity (<0.4 subtle, 0.4–0.8 voice, ≥0.8 embody) with temperature scaling 0.3–0.5. **Superseded by D044.** | Initial stepped approach; replaced by continuous interpolation. |
|
||||
| D044 | M023/S04 | Personality weight modulation strategy (revision) | 5-tier continuous interpolation. Progressive field inclusion: <0.2 no personality; 0.2+ tone; 0.4+ descriptors; 0.6+ phrases (count scaled); 0.8+ vocabulary/style; 0.9+ summary. Temperature: 0.3 + weight × 0.2. | 3-tier step function had jarring transitions. Continuous interpolation with progressive field inclusion gives finer control. 0.0–0.19 dead zone ensures purely encyclopedic mode. |
|
||||
|
||||
---
|
||||
|
||||
*See also: [[Architecture]], [[Development-Guide]]*
|
||||
|
|
|
|||
48
Frontend.md
48
Frontend.md
|
|
@ -20,6 +20,9 @@ React 18 + TypeScript + Vite SPA. No UI library, no state management library, no
|
|||
| `/admin/reports` | AdminReports | Admin* | Content reports |
|
||||
| `/admin/pipeline` | AdminPipeline | Admin* | Pipeline management |
|
||||
| `/admin/techniques` | AdminTechniquePages | Admin* | Technique page admin |
|
||||
| `/creator/posts` | PostsList | Creator JWT | Creator post management — status badges, edit/delete (M023/S01) |
|
||||
| `/creator/posts/new` | PostEditor | Creator JWT | Tiptap rich text editor with file attachments (M023/S01) |
|
||||
| `/creator/posts/:postId/edit` | PostEditor | Creator JWT | Edit existing post (M023/S01) |
|
||||
| `*` | → Redirect `/` | — | SPA fallback |
|
||||
|
||||
*Admin routes have no authentication gate.
|
||||
|
|
@ -41,6 +44,7 @@ React 18 + TypeScript + Vite SPA. No UI library, no state management library, no
|
|||
| CopyLinkButton | Clipboard copy with tooltip |
|
||||
| SocialIcons | Social media link icons (9 platforms) |
|
||||
| ReportIssueModal | Content report submission |
|
||||
| PostsFeed | Published posts feed with Tiptap JSON→HTML rendering and file downloads (M023/S01) |
|
||||
| ChatWidget | Floating chat bubble on creator pages — SSE streaming, citations, suggested questions (M022/S03) |
|
||||
| PersonalityProfile | Collapsible creator personality display — 3 sub-cards (Teaching Style, Vocabulary, Style) (M022/S06) |
|
||||
|
||||
|
|
@ -87,6 +91,46 @@ Collapsible personality display on creator detail pages.
|
|||
- **Gracefully hidden** when profile is null
|
||||
- **Files:** `PersonalityProfile.tsx`, styles in `App.css`
|
||||
|
||||
### PostEditor (M023/S01)
|
||||
|
||||
Rich text editor at `/creator/posts/new` and `/creator/posts/:postId/edit`.
|
||||
|
||||
- **Tiptap v3** with StarterKit + Link + Placeholder extensions
|
||||
- **Formatting toolbar** — bold, italic, lists, headings, links
|
||||
- **File attachment zone** — drag-and-drop with multipart upload to MinIO
|
||||
- **Publish toggle** — draft/published state
|
||||
- **Sequential save flow** — create post → upload files
|
||||
- **Files:** `PostEditor.tsx`, `PostEditor.module.css`
|
||||
|
||||
### PostsList (M023/S01)
|
||||
|
||||
Creator post management at `/creator/posts`.
|
||||
|
||||
- **Status badges** — Draft / Published
|
||||
- **Edit/Delete actions** with confirmation dialog
|
||||
- **SidebarNav integration** — accessible from creator dashboard
|
||||
- **Files:** `PostsList.tsx`, `PostsList.module.css`
|
||||
|
||||
### Shorts UI (M023/S03)
|
||||
|
||||
Shorts generation controls in HighlightQueue.
|
||||
|
||||
- **Generate button** — visible on approved highlights with no in-progress shorts
|
||||
- **Per-preset status badges** — color-coded pending/processing/complete/failed with pulsing animation
|
||||
- **Download links** — open presigned MinIO URLs in new tab
|
||||
- **5s polling** — while any shorts are processing, auto-stops when all settle
|
||||
- **Files:** `HighlightQueue.tsx`, `HighlightQueue.module.css` (updated)
|
||||
|
||||
### Personality Slider (M023/S02, S04)
|
||||
|
||||
ChatWidget personality control.
|
||||
|
||||
- **Range input** (0.0–1.0, step 0.1) with gradient track fill via `--slider-fill` CSS custom property
|
||||
- **Tier labels** — Encyclopedic / Subtle Reference / Creator Tone / Creator Voice / Full Embodiment
|
||||
- **Numeric display** with tabular-nums for stable width
|
||||
- **Labels match backend 5-tier interpolation boundaries** (0.2/0.4/0.6/0.8)
|
||||
- **Files:** `ChatWidget.tsx`, `ChatWidget.module.css` (updated)
|
||||
|
||||
## Hooks
|
||||
|
||||
| Hook | Purpose |
|
||||
|
|
@ -106,7 +150,9 @@ API modules:
|
|||
- `chat.ts` — SSE streaming client for POST /api/v1/chat using `fetch()` + `ReadableStream`, `ChatDoneMeta` type
|
||||
- `videos.ts` — chapter management functions
|
||||
- `auth.ts` — authentication + impersonation functions
|
||||
- `highlights.ts` — creator highlight review functions (M022/S01)
|
||||
- `highlights.ts`
|
||||
- `posts.ts` — post CRUD and file upload/download functions (M023/S01)
|
||||
- `shorts.ts` — short generation, listing, and download functions (M023/S03) — creator highlight review functions (M022/S01)
|
||||
- `follows.ts` — follow/unfollow/status/list functions (M022/S02)
|
||||
- `creators.ts` — creator detail with personality_profile and follower_count types (M022/S02, S06)
|
||||
|
||||
|
|
|
|||
9
Home.md
9
Home.md
|
|
@ -8,7 +8,7 @@ Producers can search for specific techniques and find timestamped key moments, s
|
|||
|
||||
- [[Architecture]] — System architecture, Docker services, network topology
|
||||
- [[Data-Model]] — SQLAlchemy models, relationships, enums
|
||||
- [[API-Surface]] — All 60+ API endpoints grouped by domain
|
||||
- [[API-Surface]] — All 70+ API endpoints grouped by domain
|
||||
- [[Frontend]] — Routes, components, hooks, CSS architecture
|
||||
- [[Pipeline]] — 6-stage LLM extraction pipeline, prompt system
|
||||
- [[Chat-Engine]] — Streaming Q&A with multi-turn memory
|
||||
|
|
@ -17,7 +17,7 @@ Producers can search for specific techniques and find timestamped key moments, s
|
|||
- [[Search-Retrieval]] — LightRAG + Qdrant retrieval cascade
|
||||
- [[Deployment]] — Docker Compose setup, rebuild commands
|
||||
- [[Development-Guide]] — Local dev setup, common gotchas
|
||||
- [[Decisions]] — Architectural decisions register (D001–D041)
|
||||
- [[Decisions]] — Architectural decisions register (D001–D044)
|
||||
|
||||
## Features
|
||||
|
||||
|
|
@ -33,6 +33,9 @@ Producers can search for specific techniques and find timestamped key moments, s
|
|||
- **Creator Tiers** — Free/Pro/Premium tier configuration with Coming Soon placeholders (M022)
|
||||
- **Highlight Detection v2** — 10-dimension scoring with audio proxy signals, creator review queue (M022)
|
||||
- **Chat Widget** — Floating creator-scoped chat bubble with streaming SSE and citations (M022)
|
||||
- **Post Editor + File Sharing** — Tiptap rich text editor, MinIO file attachments, CRUD API (M023)
|
||||
- **Shorts Generation Pipeline** — ffmpeg clip extraction in 3 format presets from approved highlights (M023)
|
||||
- **Personality Interpolation** — 5-tier continuous personality weight system for chat (M023)
|
||||
- **Multi-Turn Chat Memory** — Redis-backed conversation history with conversation_id threading (M022)
|
||||
- **Creator Dashboard** — Video management, chapter editing, consent controls
|
||||
|
||||
|
|
@ -64,4 +67,4 @@ Producers can search for specific techniques and find timestamped key moments, s
|
|||
|
||||
---
|
||||
|
||||
*Last updated: 2026-04-04 — M022 follow system, personality profiles, highlight v2, chat widget, multi-turn memory, creator tiers*
|
||||
*Last updated: 2026-04-04 — M023 post editor, file sharing, shorts pipeline, personality interpolation
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@
|
|||
- [[Search-Retrieval]]
|
||||
- [[Highlights]]
|
||||
- [[Personality-Profiles]]
|
||||
- Posts (via Post Editor)
|
||||
|
||||
**Reference**
|
||||
- [[API-Surface]]
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue