7ef116e2f9
Define User, Project, Experiment, Run, StageResult, Score, ResponseCache, and WebhookConfig with UUID primary keys, JSON columns, enum types (ExperimentStatus, RunStatus), full relationship cascades, and indexes. Uses sqlalchemy.JSON (not JSONB) for SQLite compatibility in single-container mode. 16 tests added covering table creation, CRUD, uniqueness constraints, default values, and cascade deletes — all passing. |
||
|---|---|---|
| alembic | ||
| Auto Run Docs | ||
| backend | ||
| docker | ||
| frontend | ||
| .env.example | ||
| .gitignore | ||
| alembic.ini | ||
| CLAUDE.md | ||
| docker-compose.yml | ||
| env.example | ||
| promptlooper-spec.md | ||
| README.md | ||
PromptLooper
The one who loops prompts — a universal LLM pipeline tuning workbench.
PromptLooper is a self-hosted tool for systematically optimizing LLM prompts, model selection, and inference parameters. It runs experiments across prompt x model x parameter combinations, caches every response, scores results against pluggable evaluation functions, and surfaces the best configurations through a real-time observability dashboard with human-in-the-loop steering.
It ships as a single Docker container (SQLite mode) for zero-config quickstart, or a Docker Compose stack (Postgres + Redis) for production use. An MCP server enables any AI agent to drive PromptLooper programmatically — creating experiments, running sweeps, and reading results without human intervention.
Quick Start
Single Container (zero dependencies)
docker run -p 8400:8400 -v promptlooper-data:/data ghcr.io/xpltdco/promptlooper
Open http://localhost:8400 — you'll be prompted to create an admin account on first boot.
Production (Docker Compose)
git clone git@git.xpltd.co:xpltdco/promptlooper.git
cd promptlooper
cp .env.example .env
# Edit .env — set POSTGRES_PASSWORD and JWT_SECRET at minimum
docker compose up -d
Features
- Systematic experimentation — grid, random, and guided sweeps across prompt x model x parameter space
- Response caching — SHA-256 deduplication means re-runs cost zero tokens
- Pluggable scoring — embedding similarity, format compliance, keyword presence, LLM-as-judge, human rating, custom webhooks
- Real-time dashboard — live progress, leaderboard, side-by-side comparison, steering controls
- MCP server — AI agents can create experiments, run sweeps, and export results programmatically
- Single-container mode — SQLite + in-process queue when no external dependencies are configured
Development
# Start backing services
docker compose up -d promptlooper-db promptlooper-redis
# Backend
cd backend && pip install -r requirements.txt
alembic upgrade head
uvicorn main:app --reload --host 0.0.0.0 --port 8000
# Frontend (separate terminal)
cd frontend && npm install && npm run dev
Testing
cd backend && pytest
cd frontend && npm test