chrysopedia/backend/pipeline
jlightner e80094dc05 feat: Truncation detection, batched classification, and pipeline auto-resume
Three resilience improvements to the pipeline:

1. LLMResponse(str) subclass carries finish_reason metadata from the LLM.
   _safe_parse_llm_response detects truncation (finish=length) and raises
   LLMTruncationError instead of wastefully retrying with a JSON nudge
   that makes the prompt even longer.

2. Stage 4 classification now batches moments (20 per call) instead of
   sending all moments in a single LLM call. Prevents context window
   overflow for videos with many moments. Batch results are merged with
   reindexed moment_index values.

3. run_pipeline auto-resumes from the last completed stage on error/retry
   instead of always restarting from stage 2. Queries pipeline_events for
   the most recent run to find completed stages. clean_reprocess trigger
   still forces a full restart.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 17:48:19 -05:00
..
__init__.py fix: restore complete project tree from ub01 canonical state 2026-03-31 02:10:41 +00:00
embedding_client.py fix: restore complete project tree from ub01 canonical state 2026-03-31 02:10:41 +00:00
llm_client.py feat: Truncation detection, batched classification, and pipeline auto-resume 2026-03-31 17:48:19 -05:00
qdrant_client.py feat: Add bulk pipeline reprocessing — creator filter, multi-select, clean retrigger 2026-03-31 15:24:59 +00:00
schemas.py fix: restore complete project tree from ub01 canonical state 2026-03-31 02:10:41 +00:00
stages.py feat: Truncation detection, batched classification, and pipeline auto-resume 2026-03-31 17:48:19 -05:00