chrysopedia/backend
jlightner c6c15defee feat: Dynamic token estimation for per-stage max_tokens
- Add estimate_tokens() and estimate_max_tokens() to llm_client with
  stage-specific output ratios (0.3x segmentation, 1.2x extraction,
  0.15x classification, 1.5x synthesis)
- Add max_tokens override parameter to LLMClient.complete()
- Wire all 4 pipeline stages to estimate max_tokens from actual prompt
  content with 20% buffer and 2048 floor
- Add LLM_MAX_TOKENS_HARD_LIMIT=32768 config (dynamic estimator ceiling)
- Log token estimates alongside every LLM request

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 05:55:17 -05:00
..
pipeline feat: Dynamic token estimation for per-stage max_tokens 2026-03-30 05:55:17 -05:00
routers feat: Pipeline events, admin dashboard, and version switcher UI 2026-03-30 05:55:07 -05:00
tests test: Added version list/detail API endpoints, Pydantic schemas, versio… 2026-03-30 07:27:40 +00:00
config.py feat: Dynamic token estimation for per-stage max_tokens 2026-03-30 05:55:17 -05:00
database.py fix: Created SQLAlchemy models for all 7 entities, Alembic async migrat… 2026-03-29 21:48:36 +00:00
main.py feat: Content issue reporting — submit from technique pages, manage in admin reports page 2026-03-30 02:53:56 -05:00
models.py feat: Pipeline events, admin dashboard, and version switcher UI 2026-03-30 05:55:07 -05:00
pytest.ini test: Added 6 integration tests proving ingestion, creator auto-detecti… 2026-03-29 22:16:15 +00:00
redis_client.py test: Built 9 review queue API endpoints (queue, stats, approve, reject… 2026-03-29 23:13:43 +00:00
requirements.txt feat: Created 4 prompt templates and implemented 5 Celery tasks (stages… 2026-03-29 22:36:06 +00:00
schemas.py feat: Content issue reporting — submit from technique pages, manage in admin reports page 2026-03-30 02:53:56 -05:00
search_service.py feat: Created async search service with embedding+Qdrant+keyword fallba… 2026-03-29 23:55:52 +00:00
worker.py chore: Extended Settings with 12 LLM/embedding/Qdrant config fields, cr… 2026-03-29 22:30:31 +00:00