mirror of
https://github.com/xpltdco/media-rip.git
synced 2026-04-02 18:43:59 -06:00
Docker self-hosting: fix persistence, add data_dir config
Critical fix: - Dockerfile env var was MEDIARIP__DATABASE__PATH (ignored) — now MEDIARIP__SERVER__DB_PATH DB was landing at /app/mediarip.db (lost on restart) instead of /data/mediarip.db Persistence model: - /downloads → media files (bind mount recommended) - /data → SQLite DB, session cookies, error logs (named volume) - /themes → custom CSS themes (read-only bind mount) - /app/config.yaml → optional YAML config (read-only bind mount) Other changes: - Add server.data_dir config field (default: /data) for explicit session storage - Cookie storage uses data_dir instead of fragile path math from output_dir parent - Lifespan creates data_dir on startup - .dockerignore excludes tests, dev DB, egg-info - docker-compose.yml: inline admin/purge config examples - docker-compose.example.yml: parameterized with env vars - .env.example: session mode, clearer docs - README: Docker volumes table, admin setup docs, full config reference - PROJECT.md: reflects completed v1.0 state - REQUIREMENTS.md: all 26 requirements validated
This commit is contained in:
xpltd
parent
85f57a3e41
commit
5a6eb00906
14 changed files with 188 additions and 103 deletions
|
|
@ -20,6 +20,10 @@ coverage/
|
|||
tmp/
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
backend/tests/
|
||||
backend/mediarip.db*
|
||||
backend/media_rip.egg-info/
|
||||
|
||||
# Ignore git
|
||||
.git/
|
||||
|
|
@ -31,3 +35,8 @@ tmp/
|
|||
*.code-workspace
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# Ignore docs / meta
|
||||
LICENSE
|
||||
README.md
|
||||
CHANGELOG.md
|
||||
|
|
|
|||
13
.env.example
13
.env.example
|
|
@ -3,13 +3,20 @@
|
|||
# Copy this file to .env and fill in your values.
|
||||
# Used with docker-compose.example.yml (secure deployment with Caddy).
|
||||
|
||||
# Your domain name (for Caddy auto-TLS)
|
||||
# ── Required for Caddy auto-TLS ──
|
||||
DOMAIN=media.example.com
|
||||
|
||||
# Admin credentials
|
||||
# ── Admin credentials ──
|
||||
# Username for the admin panel
|
||||
ADMIN_USERNAME=admin
|
||||
|
||||
# Bcrypt password hash — generate with:
|
||||
# python -c "import bcrypt; print(bcrypt.hashpw(b'YOUR_PASSWORD', bcrypt.gensalt()).decode())"
|
||||
# docker run --rm python:3.12-slim python -c \
|
||||
# "import bcrypt; print(bcrypt.hashpw(b'YOUR_PASSWORD', bcrypt.gensalt()).decode())"
|
||||
ADMIN_PASSWORD_HASH=
|
||||
|
||||
# ── Session mode (optional) ──
|
||||
# isolated = each browser has its own queue (default)
|
||||
# shared = all users see all downloads
|
||||
# open = no session tracking
|
||||
SESSION_MODE=isolated
|
||||
|
|
|
|||
|
|
@ -12,23 +12,38 @@ A user can paste any yt-dlp-supported URL, see exactly what they're about to dow
|
|||
|
||||
## Current State
|
||||
|
||||
S01 (Foundation + Download Engine) complete. Backend foundation built: FastAPI app with yt-dlp download engine, SQLite/WAL persistence, pydantic-settings config system, SSE broker, and 4 API endpoints. 68 tests passing including real YouTube download integration tests proving the sync-to-async bridge works. Ready for S02 (SSE transport + session system).
|
||||
**v1.0.0 — Feature-complete and ship-ready.**
|
||||
|
||||
M001 (v1.0 full build — 6 slices) and M002 (UI/UX polish — 3 slices) are complete. 213 tests passing (179 backend, 34 frontend). Code pushed to GitHub. Docker image, CI/CD workflows, and deployment examples are in place.
|
||||
|
||||
All core capabilities implemented: URL submission + download, live format extraction, real-time SSE progress with reconnect replay, download queue management, playlist support with parent/child jobs, session isolation (isolated/shared/open), cookie auth upload, purge system (scheduled/manual/never), three built-in themes + custom theme system, admin panel with bcrypt auth, unsupported URL reporting, health endpoint, session export/import, link sharing, source-aware output templates, mobile-responsive layout, and zero outbound telemetry.
|
||||
|
||||
## Architecture / Key Patterns
|
||||
|
||||
- **Backend:** Python 3.12 + FastAPI, yt-dlp as library (not subprocess), aiosqlite for SQLite, sse-starlette for SSE, APScheduler 3.x for cron, bcrypt for admin auth
|
||||
- **Frontend:** Vue 3 + TypeScript + Pinia + Vite
|
||||
- **Transport:** SSE (server-push only, no WebSocket)
|
||||
- **Persistence:** SQLite with WAL mode
|
||||
- **Persistence:** SQLite with WAL mode — `/data/mediarip.db` in Docker
|
||||
- **Critical pattern:** `ThreadPoolExecutor` + `loop.call_soon_threadsafe` bridges sync yt-dlp into async FastAPI — the load-bearing architectural seam
|
||||
- **Session isolation:** Per-browser cookie-scoped queues (isolated/shared/open modes)
|
||||
- **Config hierarchy:** Hardcoded defaults → config.yaml → env var overrides → SQLite admin writes
|
||||
- **Distribution:** Single multi-stage Docker image, GHCR + Docker Hub, amd64 + arm64
|
||||
- **Config hierarchy:** Hardcoded defaults → config.yaml → env var overrides (MEDIARIP__*) → SQLite admin writes
|
||||
- **Distribution:** Single multi-stage Docker image (ghcr.io/xpltdco/media-rip), amd64 + arm64
|
||||
- **Security:** CSP headers (self-only), no outbound requests, bcrypt admin auth, httpOnly session cookies
|
||||
|
||||
## Persistent Volumes (Docker)
|
||||
|
||||
| Mount | Purpose | Required |
|
||||
|-------|---------|----------|
|
||||
| `/downloads` | Downloaded media files | Yes |
|
||||
| `/data` | SQLite database, session state, error logs | Yes |
|
||||
| `/themes` | Custom theme CSS overrides | No |
|
||||
| `/app/config.yaml` | YAML configuration file | No |
|
||||
|
||||
## Capability Contract
|
||||
|
||||
See `.gsd/REQUIREMENTS.md` for the explicit capability contract, requirement status, and coverage mapping.
|
||||
|
||||
## Milestone Sequence
|
||||
## Milestone History
|
||||
|
||||
- [ ] M001: media.rip() v1.0 — Full-featured self-hosted yt-dlp web frontend, Docker-distributed
|
||||
- ✅ M001: media.rip() v1.0 — Full-featured self-hosted yt-dlp web frontend (6 slices)
|
||||
- ✅ M002: UI/UX Polish — Ship-Ready Frontend (3 slices)
|
||||
|
|
|
|||
|
|
@ -4,11 +4,11 @@ This file is the explicit capability and coverage contract for the project.
|
|||
|
||||
Use it to track what is actively in scope, what has been validated by completed work, what is intentionally deferred, and what is explicitly out of scope.
|
||||
|
||||
## Active
|
||||
## Validated
|
||||
|
||||
### R001 — URL submission + download for any yt-dlp-supported site
|
||||
- Class: core-capability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: User pastes any URL supported by yt-dlp and the system downloads it to the configured output directory
|
||||
- Why it matters: The fundamental product primitive — everything else depends on this working
|
||||
- Source: user
|
||||
|
|
@ -19,7 +19,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R002 — Live format/quality extraction and selection
|
||||
- Class: core-capability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: GET /api/formats?url= calls yt-dlp extract_info to return available formats; user picks resolution, codec, ext before downloading
|
||||
- Why it matters: Power users won't use a tool that hides quality choice. Competitors use presets — live extraction is a step up
|
||||
- Source: user
|
||||
|
|
@ -30,7 +30,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R003 — Real-time SSE progress
|
||||
- Class: core-capability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Server-sent events stream delivers job status transitions (queued→extracting→downloading→completed/failed) with download progress (percent, speed, ETA) per session
|
||||
- Why it matters: No progress = no trust. Users need to see something is happening
|
||||
- Source: user
|
||||
|
|
@ -41,7 +41,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R004 — SSE init replay on reconnect
|
||||
- Class: continuity
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: When a client reconnects to the SSE endpoint, the server replays current job states from the DB as synthetic events before entering the live queue
|
||||
- Why it matters: Without this, page refresh clears the queue view even though downloads are running. Breaks session isolation's value proposition entirely
|
||||
- Source: user
|
||||
|
|
@ -52,7 +52,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R005 — Download queue: view, cancel, filter, sort
|
||||
- Class: primary-user-loop
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Users see all their downloads in a unified queue with status, progress, and can cancel or remove entries. Filter by status, sort by date/name
|
||||
- Why it matters: Table stakes for any download manager UX
|
||||
- Source: user
|
||||
|
|
@ -63,7 +63,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R006 — Playlist support: parent + collapsible child jobs
|
||||
- Class: core-capability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Playlist URLs create a parent job with collapsible child video rows. Parent status reflects aggregate child progress. Mixed success/failure shown per child
|
||||
- Why it matters: Playlists are a primary use case for self-hosters. MeTube treats them as flat — collapsible parent/child is a step up
|
||||
- Source: user
|
||||
|
|
@ -74,7 +74,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R007 — Session isolation: isolated (default) / shared / open modes
|
||||
- Class: differentiator
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Operator selects session mode server-wide. Isolated: each browser sees only its own downloads via httpOnly UUID cookie. Shared: all sessions see all downloads. Open: no session tracking
|
||||
- Why it matters: The primary differentiator from MeTube (issue #591 closed as "won't fix"). The feature that created demand for forks
|
||||
- Source: user
|
||||
|
|
@ -85,7 +85,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R008 — Cookie auth: per-session cookies.txt upload
|
||||
- Class: core-capability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Users upload a Netscape-format cookies.txt file scoped to their session. Enables downloading paywalled/private content. Files purged on session clear
|
||||
- Why it matters: The practical reason people move off MeTube. Enables authenticated downloads without embedding credentials in the app
|
||||
- Source: research
|
||||
|
|
@ -96,7 +96,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R009 — Purge system: scheduled/manual/never, independent file + log TTL
|
||||
- Class: operability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Operator configures purge mode (scheduled cron, manual-only, never). File TTL and log TTL are independent values. Purge activity written to audit log. Purge must skip active downloads
|
||||
- Why it matters: Ephemeral storage is the contract with users. Operators need control over disk lifecycle
|
||||
- Source: user
|
||||
|
|
@ -107,7 +107,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R010 — Three built-in themes: cyberpunk (default), dark, light
|
||||
- Class: differentiator
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Three themes baked into the Docker image. Cyberpunk is default: #00a8ff/#ff6b2b, JetBrains Mono, scanlines, grid overlay. Dark and light are clean alternatives
|
||||
- Why it matters: Visual identity differentiator — every other tool ships with plain material/tailwind defaults. Cyberpunk makes first impressions memorable
|
||||
- Source: user
|
||||
|
|
@ -118,7 +118,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R011 — Drop-in custom theme system via volume mount
|
||||
- Class: differentiator
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Operators drop a theme folder into /themes volume mount. Theme pack: theme.css (CSS variable overrides) + metadata.json + optional preview.png + optional assets/. Appears in picker without recompile
|
||||
- Why it matters: The feature MeTube refuses to build. Lowers theming floor to "edit a CSS file"
|
||||
- Source: user
|
||||
|
|
@ -129,7 +129,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R012 — CSS variable contract (base.css) as stable theme API
|
||||
- Class: constraint
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: A documented, stable set of CSS custom properties (--color-bg, --color-accent-primary, --font-ui, --radius-sm, --effect-overlay, etc.) that all themes override. Token names cannot change after v1.0 ships — they are the public API for custom themes
|
||||
- Why it matters: Changing token names after operators write custom themes breaks those themes. This is a one-way door
|
||||
- Source: user
|
||||
|
|
@ -140,7 +140,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R013 — Mobile-responsive layout
|
||||
- Class: primary-user-loop
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: <768px breakpoint: bottom tab bar (Submit/Queue/Settings), full-width URL input, card list for queue (swipe-to-cancel), bottom sheet for format options. All tap targets minimum 44px
|
||||
- Why it matters: >50% of self-hoster interactions happen on phone or tablet. No existing yt-dlp web UI does mobile well
|
||||
- Source: user
|
||||
|
|
@ -151,7 +151,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R014 — Admin panel with secure auth
|
||||
- Class: operability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Admin panel with username/password login (HTTPBasic + bcrypt). First-boot credential setup with forced change prompt. Session list, storage view, manual purge trigger, live config editor, unsupported URL log download. Security posture: timing-safe comparison (secrets.compare_digest), Secure/HttpOnly/SameSite=Strict cookies behind TLS, security headers on admin routes (HSTS, X-Content-Type-Options, X-Frame-Options), startup warning when admin enabled without TLS detected
|
||||
- Why it matters: Shipping an admin panel with crappy auth undermines the trust proposition of the entire product. Operators deserve qBittorrent/Sonarr-level login UX, not raw tokens
|
||||
- Source: user
|
||||
|
|
@ -162,7 +162,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R015 — Unsupported URL reporting with audit log
|
||||
- Class: failure-visibility
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: When yt-dlp fails with extraction error, job shows failed badge + "Report unsupported site" button. Click appends to log (domain-only by default, full URL opt-in). Admin downloads log. Zero automatic outbound reporting
|
||||
- Why it matters: Users see exactly what gets logged. Trust feature — transparency in failure handling
|
||||
- Source: user
|
||||
|
|
@ -173,7 +173,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R016 — Health endpoint
|
||||
- Class: operability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: GET /api/health returns status, version, yt_dlp_version, uptime
|
||||
- Why it matters: Uptime Kuma and similar monitoring tools are table stakes for self-hosters
|
||||
- Source: user
|
||||
|
|
@ -184,7 +184,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R017 — Session export/import
|
||||
- Class: continuity
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Export session as JSON archive (download history + queue state + preferences). Import restores history into a new session. Does not require sign-in, stays anonymous-first
|
||||
- Why it matters: Enables identity continuity on persistent instances without a real account system. No competitor offers this
|
||||
- Source: research
|
||||
|
|
@ -195,7 +195,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R018 — Link sharing (completed file shareable URL)
|
||||
- Class: primary-user-loop
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Completed downloads are served at predictable URLs. Users can copy a direct download link to share with others
|
||||
- Why it matters: Removes the "now what?" question after downloading — users share a ripped file with a friend via URL
|
||||
- Source: research
|
||||
|
|
@ -217,7 +217,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R020 — Zero automatic outbound telemetry
|
||||
- Class: constraint
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: The container makes zero automatic outbound network requests. No CDN calls, no Google Fonts, no update checks, no analytics. All fonts and assets bundled or self-hosted
|
||||
- Why it matters: Trust is the core proposition. Competing tools have subtle external requests. This is an explicit design constraint, not an afterthought
|
||||
- Source: user
|
||||
|
|
@ -228,7 +228,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R021 — Docker: single multi-stage image, GHCR + Docker Hub, amd64 + arm64
|
||||
- Class: launchability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Single Dockerfile, multi-stage build (Node frontend builder → Python deps → slim runtime with ffmpeg). Published to ghcr.io/xpltd/media-rip and docker.io/xpltd/media-rip. Both amd64 and arm64 architectures
|
||||
- Why it matters: Docker is the distribution mechanism for self-hosted tools. arm64 users (Raspberry Pi, Apple Silicon NAS) are a significant audience
|
||||
- Source: user
|
||||
|
|
@ -239,7 +239,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R022 — CI/CD: lint + test on PR, build + push on tag
|
||||
- Class: launchability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: GitHub Actions: ci.yml runs ruff + pytest + eslint + vue-tsc + vitest + Docker smoke on PRs. publish.yml builds multi-platform image and pushes to both registries on v*.*.* tags. Generates GitHub Release with changelog
|
||||
- Why it matters: Ensures the image stays functional as yt-dlp extractors evolve. Automated quality gate
|
||||
- Source: user
|
||||
|
|
@ -250,7 +250,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R023 — Config system: config.yaml + env var overrides + admin live writes
|
||||
- Class: operability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Three-layer config: hardcoded defaults → config.yaml (read-only at start) → env var overrides (MEDIARIP__SECTION__KEY) → SQLite admin writes (live, no restart). All fields optional — zero-config works out of the box
|
||||
- Why it matters: Operators need infrastructure-as-code (YAML, env vars) AND live UI config without restart
|
||||
- Source: user
|
||||
|
|
@ -272,7 +272,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R025 — Per-download output template override
|
||||
- Class: core-capability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: Users can override the output template on a per-download basis, in addition to the source-aware defaults (R019)
|
||||
- Why it matters: Power users want control over file naming for specific downloads
|
||||
- Source: user
|
||||
|
|
@ -283,7 +283,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
### R026 — Secure deployment example
|
||||
- Class: launchability
|
||||
- Status: active
|
||||
- Status: validated
|
||||
- Description: docker-compose.example.yml ships with a reverse proxy + TLS configuration as the default documented deployment path, not an afterthought
|
||||
- Why it matters: Making the secure path the default path prevents operators from accidentally running admin auth over cleartext
|
||||
- Source: user
|
||||
|
|
@ -410,32 +410,32 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
| ID | Class | Status | Primary owner | Supporting | Proof |
|
||||
|---|---|---|---|---|---|
|
||||
| R001 | core-capability | active | M001/S01 | none | unmapped |
|
||||
| R002 | core-capability | active | M001/S01 | M001/S03 | unmapped |
|
||||
| R003 | core-capability | active | M001/S02 | M001/S03 | unmapped |
|
||||
| R004 | continuity | active | M001/S02 | none | unmapped |
|
||||
| R005 | primary-user-loop | active | M001/S03 | none | unmapped |
|
||||
| R006 | core-capability | active | M001/S03 | M001/S01 | unmapped |
|
||||
| R007 | differentiator | active | M001/S02 | M001/S03 | unmapped |
|
||||
| R008 | core-capability | active | M001/S04 | none | unmapped |
|
||||
| R009 | operability | active | M001/S04 | none | unmapped |
|
||||
| R010 | differentiator | active | M001/S05 | none | unmapped |
|
||||
| R011 | differentiator | active | M001/S05 | none | unmapped |
|
||||
| R012 | constraint | active | M001/S05 | M001/S03 | unmapped |
|
||||
| R013 | primary-user-loop | active | M001/S03 | none | unmapped |
|
||||
| R014 | operability | active | M001/S04 | none | unmapped |
|
||||
| R015 | failure-visibility | active | M001/S04 | none | unmapped |
|
||||
| R016 | operability | active | M001/S02 | none | unmapped |
|
||||
| R017 | continuity | active | M001/S04 | none | unmapped |
|
||||
| R018 | primary-user-loop | active | M001/S04 | none | unmapped |
|
||||
| R001 | core-capability | validated | M001/S01 | none | unmapped |
|
||||
| R002 | core-capability | validated | M001/S01 | M001/S03 | unmapped |
|
||||
| R003 | core-capability | validated | M001/S02 | M001/S03 | unmapped |
|
||||
| R004 | continuity | validated | M001/S02 | none | unmapped |
|
||||
| R005 | primary-user-loop | validated | M001/S03 | none | unmapped |
|
||||
| R006 | core-capability | validated | M001/S03 | M001/S01 | unmapped |
|
||||
| R007 | differentiator | validated | M001/S02 | M001/S03 | unmapped |
|
||||
| R008 | core-capability | validated | M001/S04 | none | unmapped |
|
||||
| R009 | operability | validated | M001/S04 | none | unmapped |
|
||||
| R010 | differentiator | validated | M001/S05 | none | unmapped |
|
||||
| R011 | differentiator | validated | M001/S05 | none | unmapped |
|
||||
| R012 | constraint | validated | M001/S05 | M001/S03 | unmapped |
|
||||
| R013 | primary-user-loop | validated | M001/S03 | none | unmapped |
|
||||
| R014 | operability | validated | M001/S04 | none | unmapped |
|
||||
| R015 | failure-visibility | validated | M001/S04 | none | unmapped |
|
||||
| R016 | operability | validated | M001/S02 | none | unmapped |
|
||||
| R017 | continuity | validated | M001/S04 | none | unmapped |
|
||||
| R018 | primary-user-loop | validated | M001/S04 | none | unmapped |
|
||||
| R019 | core-capability | validated | M001/S01 | none | 9 unit tests (S01 test_output_template.py) |
|
||||
| R020 | constraint | active | M001/S06 | all | unmapped |
|
||||
| R021 | launchability | active | M001/S06 | none | unmapped |
|
||||
| R022 | launchability | active | M001/S06 | none | unmapped |
|
||||
| R023 | operability | active | M001/S01 | M001/S04 | unmapped |
|
||||
| R020 | constraint | validated | M001/S06 | all | unmapped |
|
||||
| R021 | launchability | validated | M001/S06 | none | unmapped |
|
||||
| R022 | launchability | validated | M001/S06 | none | unmapped |
|
||||
| R023 | operability | validated | M001/S01 | M001/S04 | unmapped |
|
||||
| R024 | core-capability | validated | M001/S01 | none | integration test (S01 test_concurrent_downloads) |
|
||||
| R025 | core-capability | active | M001/S03 | none | unmapped |
|
||||
| R026 | launchability | active | M001/S06 | none | unmapped |
|
||||
| R025 | core-capability | validated | M001/S03 | none | unmapped |
|
||||
| R026 | launchability | validated | M001/S06 | none | unmapped |
|
||||
| R027 | primary-user-loop | deferred | none | none | unmapped |
|
||||
| R028 | failure-visibility | deferred | none | none | unmapped |
|
||||
| R029 | primary-user-loop | deferred | none | none | unmapped |
|
||||
|
|
@ -449,7 +449,7 @@ Use it to track what is actively in scope, what has been validated by completed
|
|||
|
||||
## Coverage Summary
|
||||
|
||||
- Active requirements: 24
|
||||
- Mapped to slices: 24
|
||||
- Validated: 2
|
||||
- Active requirements: 0
|
||||
- Mapped to slices: 26
|
||||
- Validated: 26
|
||||
- Unmapped active requirements: 0
|
||||
|
|
|
|||
|
|
@ -55,7 +55,8 @@ USER mediarip
|
|||
|
||||
# Environment defaults
|
||||
ENV MEDIARIP__DOWNLOADS__OUTPUT_DIR=/downloads \
|
||||
MEDIARIP__DATABASE__PATH=/data/mediarip.db \
|
||||
MEDIARIP__SERVER__DB_PATH=/data/mediarip.db \
|
||||
MEDIARIP__SERVER__DATA_DIR=/data \
|
||||
PYTHONUNBUFFERED=1
|
||||
|
||||
EXPOSE 8000
|
||||
|
|
|
|||
54
README.md
54
README.md
|
|
@ -3,16 +3,20 @@
|
|||
A self-hostable yt-dlp web frontend. Paste a URL, pick quality, download — with session isolation, real-time progress, and a cyberpunk default theme.
|
||||
|
||||
|
||||
|
||||
|
||||
## Features
|
||||
|
||||
- **Paste & download** — Any URL yt-dlp supports. Format picker with live quality extraction.
|
||||
- **Real-time progress** — Server-Sent Events stream download progress to the browser instantly.
|
||||
- **Session isolation** — Each browser gets its own download queue. No cross-talk.
|
||||
- **Playlist support** — Collapsible parent/child jobs with per-video status tracking.
|
||||
- **Three built-in themes** — Cyberpunk (default), Dark, Light. Switch in the header.
|
||||
- **Custom themes** — Drop a CSS file into `/themes` volume. No rebuild needed.
|
||||
- **Admin panel** — Session management, storage info, manual purge. Protected by HTTP Basic + bcrypt.
|
||||
- **Zero telemetry** — No outbound requests. Your downloads are your business.
|
||||
- **Admin panel** — Session management, storage info, manual purge, error logs. Protected by bcrypt auth.
|
||||
- **Cookie auth** — Upload cookies.txt per session for paywalled/private content.
|
||||
- **Auto-purge** — Configurable scheduled cleanup of old downloads and logs.
|
||||
- **Zero telemetry** — No outbound requests. No CDN, no fonts, no analytics. CSP enforced.
|
||||
- **Mobile-friendly** — Responsive layout with bottom tabs on small screens.
|
||||
|
||||
## Quickstart
|
||||
|
|
@ -25,6 +29,17 @@ Open [http://localhost:8080](http://localhost:8080) and paste a URL.
|
|||
|
||||
Downloads are saved to `./downloads/`.
|
||||
|
||||
## Docker Volumes
|
||||
|
||||
| Mount | Purpose | Persists |
|
||||
|-------|---------|----------|
|
||||
| `/downloads` | Downloaded media files | ✅ Bind mount recommended |
|
||||
| `/data` | SQLite database, session cookies, error logs | ✅ Named volume recommended |
|
||||
| `/themes` | Custom theme CSS overrides (optional) | Read-only bind mount |
|
||||
| `/app/config.yaml` | YAML config file (optional) | Read-only bind mount |
|
||||
|
||||
**Important:** The `/data` volume contains the database (download history, admin state, error logs) and session cookie files. Use a named volume or bind mount to persist across container restarts.
|
||||
|
||||
## Configuration
|
||||
|
||||
All settings have sensible defaults. Override via environment variables or `config.yaml`:
|
||||
|
|
@ -32,6 +47,8 @@ All settings have sensible defaults. Override via environment variables or `conf
|
|||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `MEDIARIP__SERVER__PORT` | `8000` | Internal server port |
|
||||
| `MEDIARIP__SERVER__DB_PATH` | `/data/mediarip.db` | SQLite database path |
|
||||
| `MEDIARIP__SERVER__DATA_DIR` | `/data` | Persistent data directory |
|
||||
| `MEDIARIP__DOWNLOADS__OUTPUT_DIR` | `/downloads` | Where files are saved |
|
||||
| `MEDIARIP__DOWNLOADS__MAX_CONCURRENT` | `3` | Maximum parallel downloads |
|
||||
| `MEDIARIP__SESSION__MODE` | `isolated` | `isolated`, `shared`, or `open` |
|
||||
|
|
@ -41,6 +58,7 @@ All settings have sensible defaults. Override via environment variables or `conf
|
|||
| `MEDIARIP__ADMIN__PASSWORD_HASH` | _(empty)_ | Bcrypt hash of admin password |
|
||||
| `MEDIARIP__PURGE__ENABLED` | `false` | Enable auto-purge of old downloads |
|
||||
| `MEDIARIP__PURGE__MAX_AGE_HOURS` | `168` | Delete downloads older than this |
|
||||
| `MEDIARIP__PURGE__CRON` | `0 3 * * *` | Purge schedule (cron syntax) |
|
||||
| `MEDIARIP__THEMES_DIR` | `/themes` | Custom themes directory |
|
||||
|
||||
### Session Modes
|
||||
|
|
@ -49,6 +67,25 @@ All settings have sensible defaults. Override via environment variables or `conf
|
|||
- **shared**: All sessions see all downloads. Good for household/team use.
|
||||
- **open**: No session tracking at all.
|
||||
|
||||
### Admin Panel
|
||||
|
||||
Enable the admin panel to manage sessions, view storage, trigger manual purge, and review error logs:
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml environment section
|
||||
MEDIARIP__ADMIN__ENABLED: "true"
|
||||
MEDIARIP__ADMIN__USERNAME: "admin"
|
||||
MEDIARIP__ADMIN__PASSWORD_HASH: "$2b$12$..." # see below
|
||||
```
|
||||
|
||||
Generate a bcrypt password hash:
|
||||
```bash
|
||||
docker run --rm python:3.12-slim python -c \
|
||||
"import bcrypt; print(bcrypt.hashpw(b'YOUR_PASSWORD', bcrypt.gensalt()).decode())"
|
||||
```
|
||||
|
||||
Admin state (login, settings changes) persists in the SQLite database at `/data/mediarip.db`.
|
||||
|
||||
## Custom Themes
|
||||
|
||||
1. Create a folder in your themes volume: `./themes/my-theme/`
|
||||
|
|
@ -70,7 +107,7 @@ See the built-in themes in `frontend/src/themes/` for fully commented examples.
|
|||
|
||||
## Secure Deployment
|
||||
|
||||
For production with TLS:
|
||||
For production with TLS, use the included Caddy reverse proxy:
|
||||
|
||||
```bash
|
||||
cp docker-compose.example.yml docker-compose.yml
|
||||
|
|
@ -79,12 +116,7 @@ cp .env.example .env
|
|||
docker compose up -d
|
||||
```
|
||||
|
||||
This uses Caddy as a reverse proxy with automatic Let's Encrypt TLS.
|
||||
|
||||
Generate an admin password hash:
|
||||
```bash
|
||||
python -c "import bcrypt; print(bcrypt.hashpw(b'YOUR_PASSWORD', bcrypt.gensalt()).decode())"
|
||||
```
|
||||
Caddy automatically provisions Let's Encrypt TLS certificates for your domain.
|
||||
|
||||
## Development
|
||||
|
||||
|
|
@ -95,7 +127,7 @@ cd backend
|
|||
python -m venv .venv
|
||||
.venv/bin/pip install -r requirements.txt
|
||||
.venv/bin/pip install pytest pytest-asyncio pytest-anyio httpx ruff
|
||||
.venv/bin/python -m pytest tests/ -v
|
||||
.venv/bin/python -m pytest tests/ -v -m "not integration"
|
||||
```
|
||||
|
||||
### Frontend
|
||||
|
|
@ -120,6 +152,7 @@ npm run build # Production build
|
|||
| `/api/formats` | GET | Extract available formats for a URL |
|
||||
| `/api/events` | GET | SSE stream for real-time progress |
|
||||
| `/api/cookies` | POST | Upload cookies.txt for authenticated downloads |
|
||||
| `/api/cookies` | DELETE | Remove cookies.txt for current session |
|
||||
| `/api/themes` | GET | List available custom themes |
|
||||
| `/api/admin/*` | GET/POST | Admin endpoints (requires auth) |
|
||||
|
||||
|
|
@ -130,6 +163,7 @@ npm run build # Production build
|
|||
- **Transport**: Server-Sent Events for real-time progress
|
||||
- **Database**: SQLite with WAL mode
|
||||
- **Styling**: CSS custom properties (no Tailwind, no component library)
|
||||
- **Container**: Multi-stage build, non-root user, amd64 + arm64
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
|
|
@ -39,6 +39,7 @@ class ServerConfig(BaseModel):
|
|||
port: int = 8000
|
||||
log_level: str = "info"
|
||||
db_path: str = "mediarip.db"
|
||||
data_dir: str = "/data"
|
||||
|
||||
|
||||
class DownloadsConfig(BaseModel):
|
||||
|
|
|
|||
|
|
@ -57,6 +57,10 @@ async def lifespan(app: FastAPI):
|
|||
)
|
||||
|
||||
# --- Database ---
|
||||
# Ensure data directory exists for DB and session state
|
||||
data_dir = Path(config.server.data_dir)
|
||||
data_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
db = await init_db(config.server.db_path)
|
||||
logger.info("Database initialised at %s", config.server.db_path)
|
||||
|
||||
|
|
|
|||
|
|
@ -13,12 +13,10 @@ logger = logging.getLogger("mediarip.cookies")
|
|||
|
||||
router = APIRouter(tags=["cookies"])
|
||||
|
||||
COOKIES_DIR = "data/sessions"
|
||||
|
||||
|
||||
def _cookie_path(output_base: str, session_id: str) -> Path:
|
||||
def _cookie_path(data_dir: str, session_id: str) -> Path:
|
||||
"""Return the cookies.txt path for a session."""
|
||||
return Path(output_base).parent / COOKIES_DIR / session_id / "cookies.txt"
|
||||
return Path(data_dir) / "sessions" / session_id / "cookies.txt"
|
||||
|
||||
|
||||
@router.post("/cookies")
|
||||
|
|
@ -29,7 +27,7 @@ async def upload_cookies(
|
|||
) -> dict:
|
||||
"""Upload a Netscape-format cookies.txt for the current session.
|
||||
|
||||
File is stored at data/sessions/{session_id}/cookies.txt.
|
||||
File is stored at {data_dir}/sessions/{session_id}/cookies.txt.
|
||||
CRLF line endings are normalized to LF.
|
||||
"""
|
||||
content = await file.read()
|
||||
|
|
@ -38,7 +36,7 @@ async def upload_cookies(
|
|||
text = content.decode("utf-8", errors="replace").replace("\r\n", "\n")
|
||||
|
||||
config = request.app.state.config
|
||||
cookie_file = _cookie_path(config.downloads.output_dir, session_id)
|
||||
cookie_file = _cookie_path(config.server.data_dir, session_id)
|
||||
cookie_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
cookie_file.write_text(text, encoding="utf-8")
|
||||
|
||||
|
|
@ -54,7 +52,7 @@ async def delete_cookies(
|
|||
) -> dict:
|
||||
"""Delete the cookies.txt for the current session."""
|
||||
config = request.app.state.config
|
||||
cookie_file = _cookie_path(config.downloads.output_dir, session_id)
|
||||
cookie_file = _cookie_path(config.server.data_dir, session_id)
|
||||
|
||||
if cookie_file.is_file():
|
||||
cookie_file.unlink()
|
||||
|
|
@ -64,12 +62,12 @@ async def delete_cookies(
|
|||
return {"status": "not_found"}
|
||||
|
||||
|
||||
def get_cookie_path_for_session(output_dir: str, session_id: str) -> str | None:
|
||||
def get_cookie_path_for_session(data_dir: str, session_id: str) -> str | None:
|
||||
"""Return the cookies.txt path if it exists for a session, else None.
|
||||
|
||||
Called by DownloadService to pass cookiefile to yt-dlp.
|
||||
"""
|
||||
path = _cookie_path(output_dir, session_id)
|
||||
path = _cookie_path(data_dir, session_id)
|
||||
if path.is_file():
|
||||
return str(path)
|
||||
return None
|
||||
|
|
|
|||
|
|
@ -26,7 +26,10 @@ def test_config(tmp_path: Path) -> AppConfig:
|
|||
"""Return an AppConfig with downloads.output_dir pointing at a temp dir."""
|
||||
dl_dir = tmp_path / "downloads"
|
||||
dl_dir.mkdir()
|
||||
return AppConfig(downloads={"output_dir": str(dl_dir)})
|
||||
return AppConfig(
|
||||
server={"data_dir": str(tmp_path / "data")},
|
||||
downloads={"output_dir": str(dl_dir)},
|
||||
)
|
||||
|
||||
|
||||
@pytest_asyncio.fixture()
|
||||
|
|
@ -75,7 +78,7 @@ async def client(tmp_path: Path):
|
|||
|
||||
# Build config pointing at temp resources
|
||||
config = AppConfig(
|
||||
server={"db_path": db_path},
|
||||
server={"db_path": db_path, "data_dir": str(tmp_path / "data")},
|
||||
downloads={"output_dir": str(dl_dir)},
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -33,7 +33,10 @@ async def download_env(tmp_path):
|
|||
dl_dir.mkdir()
|
||||
db_path = str(tmp_path / "test.db")
|
||||
|
||||
config = AppConfig(downloads={"output_dir": str(dl_dir)})
|
||||
config = AppConfig(
|
||||
server={"data_dir": str(tmp_path / "data")},
|
||||
downloads={"output_dir": str(dl_dir)},
|
||||
)
|
||||
db = await init_db(db_path)
|
||||
loop = asyncio.get_running_loop()
|
||||
broker = SSEBroker(loop)
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ async def file_client(tmp_path):
|
|||
dl_dir.mkdir()
|
||||
|
||||
config = AppConfig(
|
||||
server={"db_path": db_path},
|
||||
server={"db_path": db_path, "data_dir": str(tmp_path / "data")},
|
||||
downloads={"output_dir": str(dl_dir)},
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,12 +1,10 @@
|
|||
# media.rip() — Docker Compose example with Caddy reverse proxy
|
||||
# media.rip() — Docker Compose with Caddy reverse proxy (recommended for production)
|
||||
#
|
||||
# This is the recommended deployment configuration.
|
||||
# Caddy automatically provisions TLS certificates via Let's Encrypt.
|
||||
#
|
||||
# Usage:
|
||||
# 1. Replace YOUR_DOMAIN with your actual domain
|
||||
# 2. Set a strong admin password hash (see below)
|
||||
# 3. Run: docker compose up -d
|
||||
# Setup:
|
||||
# 1. Copy .env.example to .env and fill in your values
|
||||
# 2. Run: docker compose -f docker-compose.example.yml up -d
|
||||
#
|
||||
# Generate a bcrypt password hash:
|
||||
# docker run --rm python:3.12-slim python -c \
|
||||
|
|
@ -18,19 +16,21 @@ services:
|
|||
container_name: media-rip
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- downloads:/downloads
|
||||
- data:/data
|
||||
# Optional: custom themes
|
||||
# - ./themes:/themes:ro
|
||||
# Optional: config file
|
||||
# - ./config.yaml:/app/config.yaml:ro
|
||||
- downloads:/downloads # Downloaded media files
|
||||
- data:/data # Database, sessions, error logs
|
||||
# Optional:
|
||||
# - ./themes:/themes:ro # Custom theme CSS overrides
|
||||
# - ./config.yaml:/app/config.yaml:ro # YAML config file
|
||||
environment:
|
||||
# Admin panel (optional — remove to disable)
|
||||
# Admin panel
|
||||
MEDIARIP__ADMIN__ENABLED: "true"
|
||||
MEDIARIP__ADMIN__USERNAME: "admin"
|
||||
MEDIARIP__ADMIN__USERNAME: "${ADMIN_USERNAME:-admin}"
|
||||
MEDIARIP__ADMIN__PASSWORD_HASH: "${ADMIN_PASSWORD_HASH}"
|
||||
# Session mode: isolated (default), shared, or open
|
||||
# MEDIARIP__SESSION__MODE: "isolated"
|
||||
MEDIARIP__SESSION__MODE: "${SESSION_MODE:-isolated}"
|
||||
# Auto-purge (optional)
|
||||
# MEDIARIP__PURGE__ENABLED: "true"
|
||||
# MEDIARIP__PURGE__MAX_AGE_HOURS: "168"
|
||||
expose:
|
||||
- "8000"
|
||||
healthcheck:
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@
|
|||
#
|
||||
# The app will be available at http://localhost:8080
|
||||
# Downloads are persisted in ./downloads/
|
||||
# Database + session state persisted in the mediarip-data volume.
|
||||
|
||||
services:
|
||||
mediarip:
|
||||
|
|
@ -13,11 +14,20 @@ services:
|
|||
ports:
|
||||
- "8080:8000"
|
||||
volumes:
|
||||
- ./downloads:/downloads # Downloaded files
|
||||
- ./themes:/themes # Custom themes (optional)
|
||||
- mediarip-data:/data # Database + internal state
|
||||
- ./downloads:/downloads # Downloaded media files (browsable)
|
||||
- mediarip-data:/data # Database, sessions, error logs
|
||||
# Optional:
|
||||
# - ./themes:/themes:ro # Custom theme CSS overrides
|
||||
# - ./config.yaml:/app/config.yaml:ro # YAML config file
|
||||
environment:
|
||||
- MEDIARIP__SESSION__MODE=isolated
|
||||
# Admin panel (disabled by default):
|
||||
# - MEDIARIP__ADMIN__ENABLED=true
|
||||
# - MEDIARIP__ADMIN__USERNAME=admin
|
||||
# - MEDIARIP__ADMIN__PASSWORD_HASH=$2b$12$...your.bcrypt.hash...
|
||||
# Auto-purge (disabled by default):
|
||||
# - MEDIARIP__PURGE__ENABLED=true
|
||||
# - MEDIARIP__PURGE__MAX_AGE_HOURS=168
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/api/health')"]
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue