Commit graph

16 commits

Author SHA1 Message Date
jlightner
dfc5aa2ae7 chore: Added GeneratedShort model with FormatPreset/ShortStatus enums,…
- "backend/models.py"
- "backend/config.py"
- "docker/Dockerfile.api"
- "docker-compose.yml"
- "alembic/versions/025_add_generated_shorts.py"

GSD-Task: S03/T01
2026-04-04 09:43:36 +00:00
jlightner
f0f36a3f76 feat: Added MinIO Docker service, Post/PostAttachment models with migra…
- "docker-compose.yml"
- "backend/config.py"
- "backend/minio_client.py"
- "backend/models.py"
- "backend/schemas.py"
- "backend/requirements.txt"
- "docker/nginx.conf"
- "alembic/versions/024_add_posts_and_attachments.py"

GSD-Task: S01/T01
2026-04-04 09:02:40 +00:00
jlightner
17b43d9778 feat: Added LightRAG /query/data as primary search engine with file_sou…
- "backend/config.py"
- "backend/search_service.py"

GSD-Task: S01/T01
2026-04-04 04:44:24 +00:00
jlightner
906b6491fe fix: static 96k max_tokens for all pipeline stages — dynamic estimator was truncating thinking model output
The dynamic token estimator calculated max_tokens from input size × stage ratio,
which produced ~9k for stage 5 compose calls. Thinking models consume unpredictable
budget for internal reasoning, leaving 0 visible output tokens.

Changed: hard_limit 32768→96000, estimate_max_tokens now returns hard_limit directly.
2026-04-03 08:18:28 +00:00
jlightner
fd1fd6c6f9 fix: Pipeline LLM audit — temperature=0, realistic token ratios, structured request_params
Audit findings & fixes:
- temperature was never set (API defaulted to 1.0) → now explicit 0.0 for deterministic JSON
- llm_max_tokens=65536 exceeded hard_limit=32768 → aligned to 32768
- Output ratio estimates were 5-30x too high (based on actual pipeline data):
  stage2: 0.6→0.05, stage3: 2.0→0.3, stage4: 0.5→0.3, stage5: 2.5→0.8
- request_params now structured as api_params (what's sent to LLM) vs pipeline_config
  (internal estimator settings) — no more ambiguous 'hard_limit' in request params
- temperature=0.0 sent on both primary and fallback endpoints
2026-04-01 07:20:09 +00:00
jlightner
c344b8c670 fix: Moment-to-page linking via moment_indices in stage 5 synthesis
When the LLM splits a category group into multiple technique pages,
moments were blanket-linked to the last page in the loop, leaving all
other pages as orphans with 0 key moments (48 out of 204 pages affected).

Added moment_indices field to SynthesizedPage schema and synthesis prompt
so the LLM explicitly declares which input moments each page covers.
Stage 5 now uses these indices for targeted linking instead of the broken
blanket approach. Tags are also computed per-page from linked moments
only, fixing cross-contamination (e.g. "stereo imaging" tag appearing
on gain staging pages).

Deleted 48 orphan technique pages from the database.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 00:34:37 -05:00
jlightner
52e7e3bbc2 feat: remove review workflow — unused gate that blocked nothing
773 key moments sat at 'pending' with 0 approved/edited/rejected.
review_status was never checked by any public-facing query — all content
was always visible regardless of review state.

Removed:
- backend/routers/review.py (10 endpoints)
- backend/tests/test_review.py
- frontend ReviewQueue, MomentDetail pages
- frontend client.ts (review-only API client)
- frontend ModeToggle, StatusBadge components
- Review link from AdminDropdown, Moments link from pipeline rows
- ReviewStatus, PageReviewStatus enums from models
- review_mode config flag
- review_status columns (migration 007)
- ~80 lines of mode-toggle CSS

Pipeline now always sets processing_status to 'published'.
Migration 007 drops columns, enums, and migrates 'reviewed' → 'published'.
2026-03-31 02:34:12 +00:00
jlightner
4b0914b12b fix: restore complete project tree from ub01 canonical state
Auto-mode commit 7aa33cd accidentally deleted 78 files (14,814 lines) during M005
execution. Subsequent commits rebuilt some frontend files but backend/, alembic/,
tests/, whisper/, docker configs, and prompts were never restored in this repo.

This commit restores the full project tree by syncing from ub01's working directory,
which has all M001-M007 features running in production containers.

Restored: backend/ (config, models, routers, database, redis, search_service, worker),
alembic/ (6 migrations), docker/ (Dockerfiles, nginx, compose), prompts/ (4 stages),
tests/, whisper/, README.md, .env.example, chrysopedia-spec.md
2026-03-31 02:10:41 +00:00
jlightner
7aa33cd17f fix: Fixed syntax errors in pipeline event instrumentation — _emit_even…
- "backend/pipeline/stages.py"

GSD-Task: S01/T01
2026-03-30 08:27:53 +00:00
jlightner
17347da87e feat: Switch to FYN-LLM-Agent models — chat for stages 2/4, think for stages 3/5 2026-03-30 05:42:27 +00:00
jlightner
f67e676264 fix: Bump max_tokens to 65536 (model supports 94K context, extraction needs headroom) 2026-03-30 04:57:44 +00:00
jlightner
6fb497d03a chore: Bump LLM max_tokens to 32768, commit M002/M003 GSD artifacts
- max_tokens bumped from 16384 to 32768 (extraction responses still hitting limits)
- All GSD planning/completion artifacts for M002 (deployment) and M003 (DNS + LLM routing)
- KNOWLEDGE.md updated with XPLTD domain setup flow and container healthcheck patterns
- DECISIONS.md updated with D015 (subnet) and D016 (Ollama for embeddings)
2026-03-30 04:22:45 +00:00
jlightner
cf759f3739 fix: Add max_tokens=16384 to LLM requests (OpenWebUI defaults to 1000, truncating pipeline JSON) 2026-03-30 04:08:29 +00:00
jlightner
4aa4b08a7f feat: Per-stage LLM model routing with thinking modality and think-tag stripping
- Added 8 per-stage config fields: llm_stage{2-5}_model and llm_stage{2-5}_modality
- LLMClient.complete() accepts modality ('chat'/'thinking') and model_override
- Thinking modality: appends JSON instructions to system prompt, strips <think> tags
- strip_think_tags() handles multiline, multiple blocks, and edge cases
- Pipeline stages 2-5 read per-stage config and pass to LLM client
- Updated .env.example with per-stage model/modality documentation
- All 59 tests pass including new think-tag stripping test
2026-03-30 02:12:14 +00:00
jlightner
12cc86aef9 chore: Extended Settings with 12 LLM/embedding/Qdrant config fields, cr…
- "backend/config.py"
- "backend/worker.py"
- "backend/pipeline/schemas.py"
- "backend/pipeline/llm_client.py"
- "backend/requirements.txt"
- "backend/pipeline/__init__.py"
- "backend/pipeline/stages.py"

GSD-Task: S03/T01
2026-03-29 22:30:31 +00:00
jlightner
07126138b5 chore: Built FastAPI app with DB-connected health check, Pydantic schem…
- "backend/main.py"
- "backend/config.py"
- "backend/schemas.py"
- "backend/routers/__init__.py"
- "backend/routers/health.py"
- "backend/routers/creators.py"
- "backend/routers/videos.py"

GSD-Task: S01/T03
2026-03-29 21:54:57 +00:00