1 Agent-Context

xpltd_admin edited this page 2026-04-03 22:44:10 -06:00

Table of Contents

• Agent Context
• File Index (Read These First)
• Dependency Map
• Common Modification Patterns
• To add a new API endpoint:
• To modify the database schema:
• To add a new platform source:
• To add a new notification type:
• Gotchas
• Verification Commands
• Architecture Summary

Agent Context

Meta	Value
Repo	xpltdco/tubearr
Language	TypeScript
Framework	Fastify 5 + Drizzle ORM + React 19
Entry Point	src/index.ts
Test Command	npm test
Build Command	npm run build && npm run build:frontend
Docker Image	tubearr:latest
Upstream Port	8989
Database	SQLite (LibSQL) at TUBEARR_DB_PATH
Node Version	>= 18.0.0 (Docker uses 22)
Module System	ES Modules ("type": "module")
Last Updated	2026-04-04

File Index (Read These First)

Priority-ordered list of the most important files for understanding the codebase:

Priority	File	Purpose
1	src/index.ts	Application entry point — startup sequence, service initialization
2	src/config/index.ts	All environment variables parsed and exported as config
3	src/server/index.ts	Fastify server builder — plugin registration, middleware, route mounting
4	src/db/schema/index.ts	All Drizzle schema exports (re-exports from individual schema files)
5	src/db/schema/channels.ts	Channels table definition
6	src/db/schema/content.ts	Content items + format profiles table definitions
7	src/db/schema/queue.ts	Queue items table definition
8	src/services/queue.ts	Download queue management — concurrency control, retry logic
9	src/services/download.ts	Download orchestration — yt-dlp args, file organization, quality analysis
10	src/services/scheduler.ts	Cron-based channel monitoring — scans for new content
11	src/sources/yt-dlp.ts	yt-dlp subprocess wrapper — command execution, error classification
12	src/sources/youtube.ts	YouTube platform source — channel/content fetching
13	src/server/middleware/auth.ts	Authentication — same-origin bypass + API key validation
14	src/server/routes/channel.ts	Channel CRUD routes (largest route file)
15	src/server/routes/queue.ts	Queue management routes
16	src/services/event-bus.ts	Event pub/sub for WebSocket progress streaming
17	src/services/notification.ts	Notification dispatch (Discord, email, Pushover, Telegram)
18	src/frontend/src/App.tsx	React router root — all frontend routes defined here
19	package.json	Dependencies, scripts, engine requirements
20	Dockerfile	Multi-stage build — deps, build, runtime

Dependency Map

src/index.ts
├── src/config/index.ts (AppConfig)
├── src/db/index.ts (database init)
│   └── src/db/schema/*.ts (Drizzle tables)
├── src/db/migrate.ts (migration runner)
├── src/server/index.ts (Fastify builder)
│   ├── src/server/middleware/auth.ts
│   ├── src/server/middleware/error-handler.ts
│   └── src/server/routes/*.ts (14 route files)
│       └── src/db/repositories/*.ts (data access)
├── src/services/rate-limiter.ts
├── src/services/file-organizer.ts
├── src/services/cookie-manager.ts
├── src/services/quality-analyzer.ts
├── src/services/download.ts
│   ├── src/sources/yt-dlp.ts (subprocess)
│   ├── src/services/rate-limiter.ts
│   ├── src/services/file-organizer.ts
│   └── src/services/quality-analyzer.ts
├── src/services/event-bus.ts
├── src/services/queue.ts
│   └── src/services/download.ts
├── src/services/scheduler.ts
│   ├── src/sources/youtube.ts
│   ├── src/sources/soundcloud.ts
│   └── src/sources/generic.ts
├── src/services/notification.ts
└── src/services/health.ts

Common Modification Patterns

To add a new API endpoint:

1. Create or edit route file in src/server/routes/
2. If new file: register in src/server/index.ts via app.register()
3. Add repository method in src/db/repositories/ for DB access
4. Add service method in src/services/ for business logic
5. Add frontend API call in src/frontend/src/api/
6. Add React Query hook in src/frontend/src/hooks/

To modify the database schema:

1. Edit table definition in src/db/schema/*.ts
2. Run npm run db:generate to create migration SQL
3. Run npm run db:migrate to apply
4. Update affected repository in src/db/repositories/

To add a new platform source:

1. Create src/sources/newplatform.ts implementing PlatformSource interface from src/sources/platform-source.ts
2. Implement: resolveChannel(url), fetchRecentContent(channel, mode), getContentMetadata(url)
3. Register in src/index.ts PlatformRegistry
4. Add rate limiter env var in src/config/index.ts

To add a new notification type:

1. Add type to notificationSettings.type enum options in src/db/schema/notifications.ts
2. Add dispatch logic in src/services/notification.ts
3. Add config form in frontend

Gotchas

1. ES Modules — The project uses "type": "module". Imports must use .js extensions in compiled output. The @/ path alias maps to src/ but behaves differently in backend (tsx/tsc) vs frontend (Vite).

2. Same-origin auth bypass — Browser requests skip API key auth. If testing auth logic, use a tool like curl with no Origin header to trigger key-based auth.

3. yt-dlp as subprocess — Downloads happen via execFile (not exec). Arguments are passed as arrays, not strings. Shell injection is prevented by design. Error messages come from yt-dlp's stderr.

4. SQLite WAL mode — The database file has companion .db-wal and .db-shm files. All three must be present together. Never copy just the .db file from a running instance.

5. Service initialization order — Services in src/index.ts must be initialized in dependency order. DownloadService depends on RateLimiter, FileOrganizer, CookieManager, QualityAnalyzer. QueueService depends on DownloadService. SchedulerService depends on QueueService.

6. Vite middleware in dev — In development (NODE_ENV=development), the Fastify server serves the frontend via Vite middleware on the same port. In production, it serves pre-built static files from dist/frontend/. Don't add @fastify/static manually — it's handled in src/server/index.ts.

7. API key auto-generation — On first startup, if TUBEARR_API_KEY env var is not set, a UUID is generated, logged to stdout once, and stored in systemConfig table. The key is cached in a mutable holder object so regeneration takes effect without restart.

8. Content status transitions — Status flows: monitored → queued → downloading → downloaded or failed. The ignored status is a dead end. Queue items have their own separate status tracking.

9. Format profile cascade — Deleting a format profile sets formatProfileId to NULL on channels and platform settings (ON DELETE SET NULL), not cascade delete.

10. Timestamps are strings — SQLite doesn't have a native datetime type. All timestamps are stored as ISO 8601 text strings. Drizzle handles this transparently.

Verification Commands

# Type check (catches type errors without building)
npx tsc --noEmit

# Run test suite
npm test

# Build backend + frontend
npm run build && npm run build:frontend

# Generate migration (after schema changes)
npm run db:generate

# Apply migrations
npm run db:migrate

# Start dev server
npm run dev

# Check yt-dlp availability
which yt-dlp && yt-dlp --version

# Docker build
docker build -t tubearr:latest .

# Docker run
docker compose up -d

# Health check
curl http://localhost:8989/ping

Architecture Summary

Single-process Node.js app:
  Fastify HTTP server (port 8989)
    ├── REST API (/api/v1/*)          — authenticated, JSON
    ├── WebSocket (/ws)               — real-time download progress
    └── Static files (dist/frontend/) — React SPA (production)
         or Vite middleware            — React SPA (development)

  Background services (in-process):
    ├── SchedulerService — cron jobs per channel (croner)
    ├── QueueService     — concurrent download processing
    └── EventBus         — pub/sub for WebSocket events

  External dependencies:
    ├── yt-dlp  (subprocess) — media downloading
    ├── ffmpeg  (subprocess) — post-processing
    └── SQLite  (embedded)   — all application state