diff --git a/Agent-Context.-.md b/Agent-Context.-.md new file mode 100644 index 0000000..4dc88a4 --- /dev/null +++ b/Agent-Context.-.md @@ -0,0 +1,180 @@ +# Agent Context + +| Meta | Value | +|------|-------| +| **Repo** | `xpltdco/tubearr` | +| **Language** | TypeScript | +| **Framework** | Fastify 5 + Drizzle ORM + React 19 | +| **Entry Point** | `src/index.ts` | +| **Test Command** | `npm test` | +| **Build Command** | `npm run build && npm run build:frontend` | +| **Docker Image** | `tubearr:latest` | +| **Upstream Port** | `8989` | +| **Database** | SQLite (LibSQL) at `TUBEARR_DB_PATH` | +| **Node Version** | >= 18.0.0 (Docker uses 22) | +| **Module System** | ES Modules (`"type": "module"`) | +| **Last Updated** | 2026-04-04 | + +## File Index (Read These First) + +Priority-ordered list of the most important files for understanding the codebase: + +| Priority | File | Purpose | +|----------|------|---------| +| 1 | `src/index.ts` | Application entry point — startup sequence, service initialization | +| 2 | `src/config/index.ts` | All environment variables parsed and exported as `config` | +| 3 | `src/server/index.ts` | Fastify server builder — plugin registration, middleware, route mounting | +| 4 | `src/db/schema/index.ts` | All Drizzle schema exports (re-exports from individual schema files) | +| 5 | `src/db/schema/channels.ts` | Channels table definition | +| 6 | `src/db/schema/content.ts` | Content items + format profiles table definitions | +| 7 | `src/db/schema/queue.ts` | Queue items table definition | +| 8 | `src/services/queue.ts` | Download queue management — concurrency control, retry logic | +| 9 | `src/services/download.ts` | Download orchestration — yt-dlp args, file organization, quality analysis | +| 10 | `src/services/scheduler.ts` | Cron-based channel monitoring — scans for new content | +| 11 | `src/sources/yt-dlp.ts` | yt-dlp subprocess wrapper — command execution, error classification | +| 12 | `src/sources/youtube.ts` | YouTube platform source — channel/content fetching | +| 13 | `src/server/middleware/auth.ts` | Authentication — same-origin bypass + API key validation | +| 14 | `src/server/routes/channel.ts` | Channel CRUD routes (largest route file) | +| 15 | `src/server/routes/queue.ts` | Queue management routes | +| 16 | `src/services/event-bus.ts` | Event pub/sub for WebSocket progress streaming | +| 17 | `src/services/notification.ts` | Notification dispatch (Discord, email, Pushover, Telegram) | +| 18 | `src/frontend/src/App.tsx` | React router root — all frontend routes defined here | +| 19 | `package.json` | Dependencies, scripts, engine requirements | +| 20 | `Dockerfile` | Multi-stage build — deps, build, runtime | + +## Dependency Map + +``` +src/index.ts +├── src/config/index.ts (AppConfig) +├── src/db/index.ts (database init) +│ └── src/db/schema/*.ts (Drizzle tables) +├── src/db/migrate.ts (migration runner) +├── src/server/index.ts (Fastify builder) +│ ├── src/server/middleware/auth.ts +│ ├── src/server/middleware/error-handler.ts +│ └── src/server/routes/*.ts (14 route files) +│ └── src/db/repositories/*.ts (data access) +├── src/services/rate-limiter.ts +├── src/services/file-organizer.ts +├── src/services/cookie-manager.ts +├── src/services/quality-analyzer.ts +├── src/services/download.ts +│ ├── src/sources/yt-dlp.ts (subprocess) +│ ├── src/services/rate-limiter.ts +│ ├── src/services/file-organizer.ts +│ └── src/services/quality-analyzer.ts +├── src/services/event-bus.ts +├── src/services/queue.ts +│ └── src/services/download.ts +├── src/services/scheduler.ts +│ ├── src/sources/youtube.ts +│ ├── src/sources/soundcloud.ts +│ └── src/sources/generic.ts +├── src/services/notification.ts +└── src/services/health.ts +``` + +## Common Modification Patterns + +### To add a new API endpoint: +1. Create or edit route file in `src/server/routes/` +2. If new file: register in `src/server/index.ts` via `app.register()` +3. Add repository method in `src/db/repositories/` for DB access +4. Add service method in `src/services/` for business logic +5. Add frontend API call in `src/frontend/src/api/` +6. Add React Query hook in `src/frontend/src/hooks/` + +### To modify the database schema: +1. Edit table definition in `src/db/schema/*.ts` +2. Run `npm run db:generate` to create migration SQL +3. Run `npm run db:migrate` to apply +4. Update affected repository in `src/db/repositories/` + +### To add a new platform source: +1. Create `src/sources/newplatform.ts` implementing `PlatformSource` interface from `src/sources/platform-source.ts` +2. Implement: `resolveChannel(url)`, `fetchRecentContent(channel, mode)`, `getContentMetadata(url)` +3. Register in `src/index.ts` PlatformRegistry +4. Add rate limiter env var in `src/config/index.ts` + +### To add a new notification type: +1. Add type to `notificationSettings.type` enum options in `src/db/schema/notifications.ts` +2. Add dispatch logic in `src/services/notification.ts` +3. Add config form in frontend + +## Gotchas + +1. **ES Modules** — The project uses `"type": "module"`. Imports must use `.js` extensions in compiled output. The `@/` path alias maps to `src/` but behaves differently in backend (tsx/tsc) vs frontend (Vite). + +2. **Same-origin auth bypass** — Browser requests skip API key auth. If testing auth logic, use a tool like curl with no Origin header to trigger key-based auth. + +3. **yt-dlp as subprocess** — Downloads happen via `execFile` (not `exec`). Arguments are passed as arrays, not strings. Shell injection is prevented by design. Error messages come from yt-dlp's stderr. + +4. **SQLite WAL mode** — The database file has companion `.db-wal` and `.db-shm` files. All three must be present together. Never copy just the `.db` file from a running instance. + +5. **Service initialization order** — Services in `src/index.ts` must be initialized in dependency order. DownloadService depends on RateLimiter, FileOrganizer, CookieManager, QualityAnalyzer. QueueService depends on DownloadService. SchedulerService depends on QueueService. + +6. **Vite middleware in dev** — In development (`NODE_ENV=development`), the Fastify server serves the frontend via Vite middleware on the same port. In production, it serves pre-built static files from `dist/frontend/`. Don't add `@fastify/static` manually — it's handled in `src/server/index.ts`. + +7. **API key auto-generation** — On first startup, if `TUBEARR_API_KEY` env var is not set, a UUID is generated, logged to stdout once, and stored in `systemConfig` table. The key is cached in a mutable holder object so regeneration takes effect without restart. + +8. **Content status transitions** — Status flows: `monitored` → `queued` → `downloading` → `downloaded` or `failed`. The `ignored` status is a dead end. Queue items have their own separate status tracking. + +9. **Format profile cascade** — Deleting a format profile sets `formatProfileId` to NULL on channels and platform settings (ON DELETE SET NULL), not cascade delete. + +10. **Timestamps are strings** — SQLite doesn't have a native datetime type. All timestamps are stored as ISO 8601 text strings. Drizzle handles this transparently. + +## Verification Commands + +```bash +# Type check (catches type errors without building) +npx tsc --noEmit + +# Run test suite +npm test + +# Build backend + frontend +npm run build && npm run build:frontend + +# Generate migration (after schema changes) +npm run db:generate + +# Apply migrations +npm run db:migrate + +# Start dev server +npm run dev + +# Check yt-dlp availability +which yt-dlp && yt-dlp --version + +# Docker build +docker build -t tubearr:latest . + +# Docker run +docker compose up -d + +# Health check +curl http://localhost:8989/ping +``` + +## Architecture Summary + +``` +Single-process Node.js app: + Fastify HTTP server (port 8989) + ├── REST API (/api/v1/*) — authenticated, JSON + ├── WebSocket (/ws) — real-time download progress + └── Static files (dist/frontend/) — React SPA (production) + or Vite middleware — React SPA (development) + + Background services (in-process): + ├── SchedulerService — cron jobs per channel (croner) + ├── QueueService — concurrent download processing + └── EventBus — pub/sub for WebSocket events + + External dependencies: + ├── yt-dlp (subprocess) — media downloading + ├── ffmpeg (subprocess) — post-processing + └── SQLite (embedded) — all application state +``` \ No newline at end of file