From e4b2a7b3ed767e12042e410cb0f4364fda2bb61c Mon Sep 17 00:00:00 2001 From: xpltd_admin Date: Fri, 3 Apr 2026 22:43:20 -0600 Subject: [PATCH] Create Development-Guide wiki page with code patterns, testing, and CI/CD --- Development-Guide.-.md | 235 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 235 insertions(+) create mode 100644 Development-Guide.-.md diff --git a/Development-Guide.-.md b/Development-Guide.-.md new file mode 100644 index 0000000..8c3b0dd --- /dev/null +++ b/Development-Guide.-.md @@ -0,0 +1,235 @@ +# Development Guide + +| Meta | Value | +|------|-------| +| **Repo** | `xpltdco/tubearr` | +| **Page** | `Development-Guide` | +| **Audience** | developers, agents | +| **Last Updated** | 2026-04-04 | +| **Status** | current | + +## Code Organization Patterns + +### Adding a New API Endpoint + +1. **Create/edit route file** in `src/server/routes/` — define the Fastify route with method, URL, handler +2. **Register the route** in `src/server/index.ts` (if new file) — `app.register(import('./routes/my-route.js'))` +3. **Add repository method** in `src/db/repositories/` if the endpoint needs new database queries +4. **Add service method** in `src/services/` if there's business logic beyond simple CRUD +5. **Add frontend API function** in `src/frontend/src/api/` — create a function that calls the endpoint +6. **Add React Query hook** in `src/frontend/src/hooks/` — wrap the API function with `useQuery` or `useMutation` + +### Adding a New Database Table + +1. **Define schema** in `src/db/schema/` — create a new file or add to an existing one using Drizzle's `sqliteTable()` +2. **Export schema** from `src/db/schema/index.ts` +3. **Generate migration:** `npm run db:generate` +4. **Apply migration:** `npm run db:migrate` +5. **Create repository** in `src/db/repositories/` — data access functions for the new table +6. **Write tests** in `src/__tests__/` for the repository + +### Adding a New Platform Source + +1. **Create source file** in `src/sources/` — implement the `PlatformSource` interface +2. **Required methods:** `resolveChannel(url)`, `fetchRecentContent(channel, mode)`, `getContentMetadata(url)` +3. **Register source** in `src/index.ts` in the PlatformRegistry initialization +4. **Add platform type** to the `platform` column options in `src/db/schema/channels.ts` +5. **Add rate limiter config** — new env var `TUBEARR_RATELIMIT_{PLATFORM}_MS` in `src/config/index.ts` + +### Adding a Frontend Page + +1. **Create page component** in `src/frontend/src/pages/` +2. **Add route** in `src/frontend/src/App.tsx` — `} />` +3. **Add navigation link** in the sidebar/nav component +4. **Create hooks** in `src/frontend/src/hooks/` for any API data the page needs + +## Testing + +### Test Framework + +**Vitest** — fast, Vite-native test runner with TypeScript support. + +**Config:** `vitest.config.ts` +- Test files: `src/__tests__/**/*.test.ts` +- Environment: `node` (not jsdom — backend tests) +- Timeout: 15 seconds per test +- Path alias: `@/` → `src/` + +### Running Tests + +```bash +# Run all tests once +npm test + +# Run in watch mode (during development) +npx vitest + +# Run a specific test file +npx vitest src/__tests__/services/queue.test.ts + +# Run tests matching a pattern +npx vitest --filter "download" +``` + +### Test Structure + +Tests are organized by layer in `src/__tests__/`: + +``` +src/__tests__/ +├── services/ # Service-level tests (queue, download, scheduler) +├── repositories/ # Database repository tests +├── routes/ # API route integration tests +├── sources/ # Platform source tests +└── utils/ # Utility function tests +``` + +### Writing Tests + +Tests use Vitest's `describe`/`it`/`expect` API: + +```typescript +import { describe, it, expect, vi } from 'vitest' + +describe('QueueService', () => { + it('should enqueue a content item', async () => { + // Arrange + const service = createQueueService(mockDb) + + // Act + const result = await service.enqueue(42) + + // Assert + expect(result.status).toBe('pending') + expect(result.contentItemId).toBe(42) + }) +}) +``` + +**Mocking:** Use `vi.mock()` for module-level mocks and `vi.fn()` for function mocks. Database tests typically mock the Drizzle client. + +## CI/CD Pipeline + +### Forgejo Actions + +**Workflow:** `.forgejo/workflows/ci.yml` + +**Triggers:** Push or PR to `master` branch. + +**Steps:** +1. `actions/checkout@v4` — checkout code +2. `npm ci` — install dependencies (clean install) +3. `npx tsc --noEmit` — TypeScript type checking (no compilation output) +4. `npm test` — run Vitest suite + +### What CI Checks + +| Check | Command | Purpose | +|-------|---------|---------| +| Type safety | `tsc --noEmit` | Catch type errors without building | +| Tests | `npm test` | Run all Vitest test suites | + +### Local CI Simulation + +Run the same checks locally before pushing: + +```bash +npx tsc --noEmit && npm test +``` + +## Coding Conventions + +### TypeScript + +- **Strict mode** enabled (`strict: true` in tsconfig) +- **ES modules** — the project uses `"type": "module"` in package.json +- **Path aliases** — use `@/` to import from `src/` (e.g., `import { config } from '@/config'`) +- **Target:** ES2022 — modern JavaScript features available + +### File Naming + +- **kebab-case** for all files: `format-profile.ts`, `queue-repository.ts` +- One module per file — each route, service, repository, or schema gets its own file +- Test files mirror source structure: `src/services/queue.ts` → `src/__tests__/services/queue.test.ts` + +### Code Patterns + +- **Repository pattern** — all database access goes through repository functions, never direct Drizzle calls in routes +- **Service pattern** — business logic lives in services, routes are thin handlers that validate input and call services +- **Fastify plugins** — routes are registered as Fastify plugins using `fastify-plugin` +- **Error classification** — yt-dlp errors are classified into categories (`rate_limit`, `geo_blocked`, etc.) for smart retry logic + +### Import Style + +```typescript +// External dependencies +import Fastify from 'fastify' +import { eq } from 'drizzle-orm' + +// Internal imports (use path alias) +import { config } from '@/config' +import { channels } from '@/db/schema' +import { getChannel } from '@/db/repositories/channel-repository' +``` + +## Branch Strategy + +- **`master`** — primary branch, CI runs on every push +- Feature branches for non-trivial changes, merged via PR +- No formal branching model (e.g., GitFlow) — keep it simple + +## Development Tips + +### Hot Reload + +`npm run dev` uses tsx watch — the backend restarts automatically when you save a TypeScript file. The frontend uses Vite HMR for instant updates without full page reload. + +### Database Inspection + +To inspect the SQLite database directly: + +```bash +# Local dev +sqlite3 ./data/tubearr.db + +# Docker +docker exec -it tubearr sqlite3 /config/tubearr.db + +# Useful queries +.tables -- list all tables +.schema channels -- show table schema +SELECT * FROM system_config; -- view app settings +SELECT COUNT(*) FROM content_items; -- count content items +``` + +### Debugging yt-dlp + +Test yt-dlp commands directly to debug download issues: + +```bash +# Check if yt-dlp can access a URL +yt-dlp --dump-json "https://youtube.com/watch?v=xxx" + +# Test format selection +yt-dlp -F "https://youtube.com/watch?v=xxx" + +# Simulate a download (dry run) +yt-dlp --simulate "https://youtube.com/watch?v=xxx" +``` + +### API Testing + +Use curl with the API key to test endpoints: + +```bash +# List channels +curl -H "X-Api-Key: your-key" http://localhost:8989/api/v1/channel + +# Add a channel +curl -X POST -H "X-Api-Key: your-key" -H "Content-Type: application/json" \ + -d '{"url": "https://youtube.com/@example"}' \ + http://localhost:8989/api/v1/channel + +# Check queue +curl -H "X-Api-Key: your-key" http://localhost:8989/api/v1/queue +``` \ No newline at end of file