chore: Added SMTP config, User notification_preferences JSONB, EmailDig…
- "backend/config.py" - "backend/models.py" - "backend/schemas.py" - "backend/services/email.py" - "alembic/versions/029_add_email_digest.py" GSD-Task: S01/T01
This commit is contained in:
jlightner
parent
0f9e76babd
commit
34a45d1c8e
15 changed files with 17630 additions and 23 deletions
1
.gsd/completed-units-M024.json
Normal file
1
.gsd/completed-units-M024.json
Normal file
|
|
@ -0,0 +1 @@
|
|||
[]
|
||||
|
|
@ -1,6 +1,160 @@
|
|||
# S01: [A] Notification System (Email Digests)
|
||||
|
||||
**Goal:** Build email notification system for follower post alerts
|
||||
**Goal:** Followers receive batched email digests when followed creators publish new content (posts or technique pages). Digest job runs on a configurable schedule via Celery Beat. Users can toggle digest preferences and unsubscribe via signed link.
|
||||
**Demo:** After this: Followers receive email digests when followed creators post new content
|
||||
|
||||
## Tasks
|
||||
- [x] **T01: Added SMTP config, User notification_preferences JSONB, EmailDigestLog model, migration 029, email digest composer+sender, and notification preference schemas** — Foundation task: add SMTP settings to config.py, add notification_preferences JSONB column to User model, create EmailDigestLog model, write Alembic migration 029, and implement the email service module (HTML digest composer + smtplib sender).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Add SMTP fields to `backend/config.py` Settings class: `smtp_host` (str, default ''), `smtp_port` (int, default 587), `smtp_user` (str, default ''), `smtp_password` (str, default ''), `smtp_from_address` (str, default ''), `smtp_tls` (bool, default True). All default to empty/safe values so the app starts without SMTP configured.
|
||||
|
||||
2. Add `notification_preferences` column to User model in `backend/models.py`: `mapped_column(JSONB, nullable=False, server_default='{"email_digests": true, "digest_frequency": "daily"}')`. This stores per-user prefs as a JSONB dict.
|
||||
|
||||
3. Create `EmailDigestLog` model in `backend/models.py`:
|
||||
- `id` (UUID PK)
|
||||
- `user_id` (FK to users.id, CASCADE)
|
||||
- `digest_sent_at` (datetime, default _now)
|
||||
- `content_summary` (JSONB — list of {creator_id, post_ids, technique_page_ids} included in this digest)
|
||||
- Index on `(user_id, digest_sent_at)` for efficient last-sent queries
|
||||
|
||||
4. Create Alembic migration `alembic/versions/029_add_email_digest.py`: add `notification_preferences` column to users table, create `email_digest_log` table.
|
||||
|
||||
5. Create `backend/services/email.py`:
|
||||
- `compose_digest_html(user_display_name, creator_content_groups, unsubscribe_url)` — returns HTML string. Groups content by creator, lists new posts and technique pages with titles and links. Simple inline-CSS HTML template (no external template engine).
|
||||
- `send_email(to_address, subject, html_body, settings)` — uses `smtplib.SMTP`/`SMTP_SSL` based on `smtp_tls`. Returns bool (success/failure). Catches `smtplib.SMTPException` and logs error. 10-second timeout per connection.
|
||||
- `is_smtp_configured(settings)` — returns True only if smtp_host and smtp_from_address are non-empty.
|
||||
|
||||
6. Add Pydantic response/request schemas to `backend/schemas.py`: `NotificationPreferences` (email_digests: bool, digest_frequency: str), `NotificationPreferencesUpdate` (same fields, optional).
|
||||
|
||||
## Failure Modes
|
||||
|
||||
| Dependency | On error | On timeout | On malformed response |
|
||||
|------------|----------|-----------|----------------------|
|
||||
| SMTP server | Log error, return False from send_email | 10s socket timeout, log + return False | N/A (we compose, not receive) |
|
||||
| PostgreSQL (migration) | Migration fails, rollback | Standard Alembic timeout | N/A |
|
||||
|
||||
## Negative Tests
|
||||
|
||||
- `is_smtp_configured` returns False when host or from_address is empty
|
||||
- `send_email` returns False and logs on SMTP connection failure
|
||||
- `compose_digest_html` handles empty content groups (returns minimal "no new content" message)
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] SMTP settings in config.py with safe defaults (empty = unconfigured)
|
||||
- [ ] notification_preferences JSONB on User with server_default
|
||||
- [ ] EmailDigestLog model with user_id + digest_sent_at index
|
||||
- [ ] Alembic migration 029 applies cleanly
|
||||
- [ ] Email service composes valid HTML and sends via smtplib
|
||||
- [ ] Schemas for notification preferences
|
||||
- Estimate: 1.5h
|
||||
- Files: backend/config.py, backend/models.py, backend/schemas.py, backend/services/email.py, alembic/versions/029_add_email_digest.py
|
||||
- Verify: cd backend && python -c "from models import EmailDigestLog, User; from services.email import compose_digest_html, send_email, is_smtp_configured; from config import get_settings; s = get_settings(); assert not is_smtp_configured(s); print('OK')" && cd ../alembic && python -c "import importlib.util; spec = importlib.util.spec_from_file_location('m', 'versions/029_add_email_digest.py'); mod = importlib.util.module_from_spec(spec); print('Migration module loads OK')"
|
||||
- [ ] **T02: Digest Celery task with Beat scheduling and Docker config** — Build the digest orchestration: a Celery task that queries new content since each user's last digest, groups by followed creator, composes and sends emails via the email service, and logs successful sends. Configure Celery Beat to run it daily. Update docker-compose.yml to add --beat flag to the worker.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Create `backend/tasks/notifications.py` with `send_digest_emails` Celery task:
|
||||
- Import celery_app from worker module
|
||||
- Use sync SQLAlchemy engine (same pattern as pipeline/stages.py — create sync engine from DATABASE_URL converted to postgresql://)
|
||||
- Query all users where `notification_preferences->>'email_digests'` is true
|
||||
- For each user, find their last EmailDigestLog entry (max digest_sent_at)
|
||||
- Query CreatorFollow to get followed creator_ids
|
||||
- For each followed creator: query Posts where `is_published=True AND created_at > last_digest_at`, query TechniquePages where `created_at > last_digest_at` (join through TechniquePageVideo → SourceVideo to match creator_id)
|
||||
- Skip user if no new content across all followed creators
|
||||
- Call `compose_digest_html` with grouped content, generate signed unsubscribe URL
|
||||
- Call `send_email` — on success, INSERT EmailDigestLog with content_summary JSONB
|
||||
- If `is_smtp_configured()` returns False, log warning and return early (graceful no-op)
|
||||
- Log structured messages: task start, per-user send result, task complete with count
|
||||
|
||||
2. Add Celery Beat schedule to `backend/worker.py`:
|
||||
```python
|
||||
celery_app.conf.beat_schedule = {
|
||||
'send-digest-emails': {
|
||||
'task': 'tasks.notifications.send_digest_emails',
|
||||
'schedule': crontab(hour=9, minute=0), # daily at 9am UTC
|
||||
},
|
||||
}
|
||||
```
|
||||
Import `from celery.schedules import crontab` at top.
|
||||
|
||||
3. Import `tasks.notifications` in worker.py alongside the existing `pipeline.stages` import so the task decorator registers.
|
||||
|
||||
4. Update `docker-compose.yml` worker command from `["celery", "-A", "worker", "worker", "--loglevel=info", "--concurrency=1"]` to `["celery", "-A", "worker", "worker", "--beat", "--loglevel=info", "--concurrency=1"]`.
|
||||
|
||||
5. Generate signed unsubscribe tokens: use `itsdangerous.URLSafeTimedSerializer` with `settings.app_secret_key`. Token encodes user_id. Unsubscribe URL format: `{base_url}/api/v1/notifications/unsubscribe?token={token}`. Token valid for 30 days.
|
||||
|
||||
## Failure Modes
|
||||
|
||||
| Dependency | On error | On timeout | On malformed response |
|
||||
|------------|----------|-----------|----------------------|
|
||||
| SMTP (via email service) | Log error per user, continue to next user | 10s timeout in email service | N/A |
|
||||
| PostgreSQL | Task fails with exception, Celery retries | Standard SQLAlchemy timeout | N/A |
|
||||
| Redis (Celery broker) | Task not dispatched, Beat retries next schedule | Celery handles reconnection | N/A |
|
||||
|
||||
## Load Profile
|
||||
|
||||
- Shared resources: sync DB session (one per task run), SMTP connection (one per email)
|
||||
- Per-operation cost: ~3 DB queries per user (last digest, follows, content), 1 SMTP send per user
|
||||
- 10x breakpoint: N/A — <50 users, volume is trivial. At 500+ users, batch SMTP connections.
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] Digest task queries correct new content per followed creator
|
||||
- [ ] Graceful no-op when SMTP is unconfigured
|
||||
- [ ] EmailDigestLog written only on successful send (deduplication)
|
||||
- [ ] Celery Beat schedule configured for daily run
|
||||
- [ ] Docker worker command includes --beat flag
|
||||
- [ ] Signed unsubscribe URL generated per email
|
||||
- Estimate: 1.5h
|
||||
- Files: backend/tasks/__init__.py, backend/tasks/notifications.py, backend/worker.py, docker-compose.yml
|
||||
- Verify: cd backend && python -c "from tasks.notifications import send_digest_emails; print('Task imports OK')" && python -c "from worker import celery_app; assert 'send-digest-emails' in celery_app.conf.beat_schedule; print('Beat schedule OK')" && grep -q '\-\-beat' ../docker-compose.yml && echo 'Docker beat flag OK'
|
||||
- [ ] **T03: Notification preferences API, unsubscribe endpoint, frontend settings, and integration test** — Wire the user-facing surfaces: API endpoints for reading/updating notification preferences, a public unsubscribe endpoint (no auth required, uses signed token), a frontend settings toggle in the creator settings page, and an integration test that verifies the full digest flow with mocked SMTP.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Create `backend/routers/notifications.py`:
|
||||
- `GET /notifications/preferences` — returns current user's notification_preferences JSONB. Requires auth.
|
||||
- `PUT /notifications/preferences` — updates notification_preferences. Validates against NotificationPreferencesUpdate schema. Requires auth.
|
||||
- `GET /notifications/unsubscribe` — accepts `token` query param. Decodes with `itsdangerous.URLSafeTimedSerializer` (max_age=30 days). Sets `notification_preferences.email_digests = False` for the user. Returns simple HTML page confirming unsubscription. No auth required.
|
||||
|
||||
2. Mount the router in `backend/main.py`: `app.include_router(notifications.router, prefix="/api/v1")`
|
||||
|
||||
3. Create `frontend/src/api/notifications.ts`:
|
||||
- `getNotificationPreferences()` — GET /api/v1/notifications/preferences
|
||||
- `updateNotificationPreferences(prefs)` — PUT /api/v1/notifications/preferences
|
||||
|
||||
4. Add notification settings section to `frontend/src/pages/CreatorSettings.tsx`:
|
||||
- New section "Email Notifications" with a toggle switch for email digests (on/off)
|
||||
- Frequency selector (daily/weekly) shown only when digests are enabled
|
||||
- Fetch current prefs on mount, PUT on change with optimistic UI update
|
||||
- Use existing form/toggle patterns from the page
|
||||
|
||||
5. Create integration test `backend/tests/test_notifications.py`:
|
||||
- Test GET/PUT preferences endpoints (auth required, returns correct shape)
|
||||
- Test unsubscribe endpoint with valid and expired tokens
|
||||
- Test digest task end-to-end: create test user, creator, follow, published post, mock SMTP, run `send_digest_emails`, assert email was "sent" (mock called with correct args), assert EmailDigestLog created
|
||||
- Test digest task skips when no new content
|
||||
- Test digest task no-op when SMTP unconfigured
|
||||
|
||||
6. Add `itsdangerous` to `requirements.txt` if not already present.
|
||||
|
||||
## Negative Tests
|
||||
|
||||
- PUT preferences with invalid digest_frequency value → 422
|
||||
- Unsubscribe with expired token (>30 days) → error page
|
||||
- Unsubscribe with tampered token → error page
|
||||
- GET/PUT preferences without auth → 401
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] GET/PUT notification preferences endpoints with auth
|
||||
- [ ] Unsubscribe endpoint works without auth via signed token
|
||||
- [ ] Frontend toggle for email digests in creator settings
|
||||
- [ ] Integration test covers digest task happy path with mocked SMTP
|
||||
- [ ] Integration test covers unsubscribe with valid and invalid tokens
|
||||
- Estimate: 2h
|
||||
- Files: backend/routers/notifications.py, backend/main.py, backend/tests/test_notifications.py, frontend/src/api/notifications.ts, frontend/src/pages/CreatorSettings.tsx, requirements.txt
|
||||
- Verify: cd backend && python -m pytest tests/test_notifications.py -v --timeout=30 2>&1 | tail -20
|
||||
|
|
|
|||
80
.gsd/milestones/M025/slices/S01/S01-RESEARCH.md
Normal file
80
.gsd/milestones/M025/slices/S01/S01-RESEARCH.md
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
# S01 Research — Notification System (Email Digests)
|
||||
|
||||
## Summary
|
||||
|
||||
Build an email digest system: when a creator publishes new content (posts, technique pages), followers receive batched email notifications. No email infrastructure exists yet — this is greenfield. The follow system (CreatorFollow model, CRUD endpoints, frontend API) is fully built and working.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Use Python stdlib `smtplib`/`email.mime` for sending, Celery Beat for scheduling, and a new `email_notification_log` table for deduplication. Add a `notification_preferences` JSONB column to the User model. No external email library needed — the volume is tiny (single-admin, invite-only tool with <50 users).
|
||||
|
||||
SMTP credentials come from environment variables. The digest job runs as a periodic Celery Beat task (daily or configurable). Each run: query new content since last digest per creator, find followers, compose HTML email, send, log.
|
||||
|
||||
## Implementation Landscape
|
||||
|
||||
### What exists
|
||||
|
||||
| Component | Location | Status |
|
||||
|---|---|---|
|
||||
| CreatorFollow model | `backend/models.py:748` | Complete — user_id, creator_id, created_at, unique constraint |
|
||||
| Follow CRUD endpoints | `backend/routers/follows.py` | Complete — follow/unfollow/status/list |
|
||||
| Follow frontend API | `frontend/src/api/follows.ts` | Complete — all 4 functions |
|
||||
| User model with email | `backend/models.py:152` | Has `email` field (String 255, unique, not null) |
|
||||
| Post model | `backend/models.py:773` | Has `is_published`, `created_at`, `creator_id` |
|
||||
| TechniquePage model | `backend/models.py:291` | Has `created_at`, `updated_at`, creator via SourceVideo join |
|
||||
| Celery worker | `docker-compose.yml:143`, `backend/worker.py` | Running, concurrency=1, no Beat scheduler |
|
||||
| Redis | `docker-compose.yml` | Running, used as Celery broker |
|
||||
| Config/Settings | `backend/config.py` | Pydantic BaseSettings, no SMTP config yet |
|
||||
| Alembic migrations | `alembic/versions/` | 28 migrations, latest is `028_add_shorts_template.py` |
|
||||
|
||||
### What needs building
|
||||
|
||||
1. **SMTP config** — Add `smtp_host`, `smtp_port`, `smtp_user`, `smtp_password`, `smtp_from_address`, `smtp_tls` to `Settings` in `config.py`
|
||||
2. **Notification preferences** — Add `notification_preferences` JSONB column to User model (email_digests: bool, digest_frequency: "daily"|"weekly") + Alembic migration
|
||||
3. **Email notification log** — New `EmailDigestLog` model tracking (user_id, digest_sent_at, content_ids_included) for deduplication
|
||||
4. **Email service** — `backend/services/email.py` using stdlib `smtplib` + `email.mime` for composing and sending HTML digest emails
|
||||
5. **Digest task** — Celery task in `backend/pipeline/stages.py` or a new `backend/tasks/notifications.py` that queries new content, groups by creator, sends digests
|
||||
6. **Celery Beat schedule** — Add Beat config to `worker.py` and a Beat container/process to docker-compose
|
||||
7. **Notification preferences API** — Endpoints to get/update user notification preferences
|
||||
8. **Frontend notification settings** — Small UI for users to toggle digest emails on/off
|
||||
9. **Unsubscribe link** — One-click unsubscribe in digest email (signed URL or token)
|
||||
|
||||
### Natural seams for task decomposition
|
||||
|
||||
1. **DB + Config foundation** (migration, model changes, SMTP config) — unblocks everything
|
||||
2. **Email service** (compose + send logic, template) — independent, unit-testable
|
||||
3. **Digest Celery task + Beat setup** (orchestration, scheduling, Docker changes) — depends on 1+2
|
||||
4. **API + Frontend** (notification preferences endpoints, settings UI, unsubscribe) — depends on 1
|
||||
5. **Integration verification** (end-to-end test with real SMTP or test stub)
|
||||
|
||||
### Key design decisions needed
|
||||
|
||||
**Content triggers:** Two types of "new content" exist:
|
||||
- **Posts:** `Post.is_published == True` with `created_at > last_digest_sent_at`. Direct `creator_id` on the model.
|
||||
- **Technique pages:** `TechniquePage.created_at > last_digest_sent_at`. Creator linkage requires join through `TechniquePageVideo → SourceVideo → Creator`. New technique pages come from pipeline completion (stage 5 sets `processing_status = complete`).
|
||||
|
||||
**Digest grouping:** Group by creator — "Creator X published 2 new articles and 1 post since your last digest." Each follower gets one email per digest run, with all followed creators' new content summarized.
|
||||
|
||||
**Celery Beat deployment:** Two options:
|
||||
- Add `--beat` flag to existing worker command: `celery -A worker worker --beat --loglevel=info` (simpler, fine for single-worker)
|
||||
- Separate Beat container (safer for multi-worker, but overkill here)
|
||||
|
||||
Recommendation: `--beat` on existing worker since concurrency=1 already.
|
||||
|
||||
**SMTP provider:** Environment variables, no provider locked in. For ub01 self-hosted, could use any external SMTP relay (Mailgun, SES, or even local Postfix). The code just needs `smtplib.SMTP(host, port)`.
|
||||
|
||||
### Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| No SMTP server configured at deploy time | Make digest task no-op when SMTP settings are empty. Log warning, don't crash. |
|
||||
| Email sending blocks Celery worker | Send emails in the digest task with reasonable timeouts (10s per email). Volume is <50 emails per run. |
|
||||
| TechniquePage→Creator join is complex | Pre-query: get creator_ids with new technique pages via `TechniquePageVideo → SourceVideo.creator_id` join, then match against CreatorFollow |
|
||||
| User has no email verification | Accept this — invite-only system, emails are entered at registration. Add verification later if needed. |
|
||||
|
||||
### Verification approach
|
||||
|
||||
- **Unit tests:** Email composition (HTML template renders correctly), digest query (correct content grouped by creator), deduplication (no duplicate sends)
|
||||
- **Integration test:** Mock SMTP server, trigger digest task, verify emails sent to correct followers with correct content
|
||||
- **Docker verification:** Celery Beat schedule fires, task runs without error in logs
|
||||
- **Manual:** Configure SMTP, follow a creator, publish content, wait for digest or trigger manually, check inbox
|
||||
71
.gsd/milestones/M025/slices/S01/tasks/T01-PLAN.md
Normal file
71
.gsd/milestones/M025/slices/S01/tasks/T01-PLAN.md
Normal file
|
|
@ -0,0 +1,71 @@
|
|||
---
|
||||
estimated_steps: 32
|
||||
estimated_files: 5
|
||||
skills_used: []
|
||||
---
|
||||
|
||||
# T01: DB models, migration, SMTP config, and email service
|
||||
|
||||
Foundation task: add SMTP settings to config.py, add notification_preferences JSONB column to User model, create EmailDigestLog model, write Alembic migration 029, and implement the email service module (HTML digest composer + smtplib sender).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Add SMTP fields to `backend/config.py` Settings class: `smtp_host` (str, default ''), `smtp_port` (int, default 587), `smtp_user` (str, default ''), `smtp_password` (str, default ''), `smtp_from_address` (str, default ''), `smtp_tls` (bool, default True). All default to empty/safe values so the app starts without SMTP configured.
|
||||
|
||||
2. Add `notification_preferences` column to User model in `backend/models.py`: `mapped_column(JSONB, nullable=False, server_default='{"email_digests": true, "digest_frequency": "daily"}')`. This stores per-user prefs as a JSONB dict.
|
||||
|
||||
3. Create `EmailDigestLog` model in `backend/models.py`:
|
||||
- `id` (UUID PK)
|
||||
- `user_id` (FK to users.id, CASCADE)
|
||||
- `digest_sent_at` (datetime, default _now)
|
||||
- `content_summary` (JSONB — list of {creator_id, post_ids, technique_page_ids} included in this digest)
|
||||
- Index on `(user_id, digest_sent_at)` for efficient last-sent queries
|
||||
|
||||
4. Create Alembic migration `alembic/versions/029_add_email_digest.py`: add `notification_preferences` column to users table, create `email_digest_log` table.
|
||||
|
||||
5. Create `backend/services/email.py`:
|
||||
- `compose_digest_html(user_display_name, creator_content_groups, unsubscribe_url)` — returns HTML string. Groups content by creator, lists new posts and technique pages with titles and links. Simple inline-CSS HTML template (no external template engine).
|
||||
- `send_email(to_address, subject, html_body, settings)` — uses `smtplib.SMTP`/`SMTP_SSL` based on `smtp_tls`. Returns bool (success/failure). Catches `smtplib.SMTPException` and logs error. 10-second timeout per connection.
|
||||
- `is_smtp_configured(settings)` — returns True only if smtp_host and smtp_from_address are non-empty.
|
||||
|
||||
6. Add Pydantic response/request schemas to `backend/schemas.py`: `NotificationPreferences` (email_digests: bool, digest_frequency: str), `NotificationPreferencesUpdate` (same fields, optional).
|
||||
|
||||
## Failure Modes
|
||||
|
||||
| Dependency | On error | On timeout | On malformed response |
|
||||
|------------|----------|-----------|----------------------|
|
||||
| SMTP server | Log error, return False from send_email | 10s socket timeout, log + return False | N/A (we compose, not receive) |
|
||||
| PostgreSQL (migration) | Migration fails, rollback | Standard Alembic timeout | N/A |
|
||||
|
||||
## Negative Tests
|
||||
|
||||
- `is_smtp_configured` returns False when host or from_address is empty
|
||||
- `send_email` returns False and logs on SMTP connection failure
|
||||
- `compose_digest_html` handles empty content groups (returns minimal "no new content" message)
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] SMTP settings in config.py with safe defaults (empty = unconfigured)
|
||||
- [ ] notification_preferences JSONB on User with server_default
|
||||
- [ ] EmailDigestLog model with user_id + digest_sent_at index
|
||||
- [ ] Alembic migration 029 applies cleanly
|
||||
- [ ] Email service composes valid HTML and sends via smtplib
|
||||
- [ ] Schemas for notification preferences
|
||||
|
||||
## Inputs
|
||||
|
||||
- ``backend/config.py` — existing Settings class to extend with SMTP fields`
|
||||
- ``backend/models.py` — existing User and base model patterns to follow`
|
||||
- ``backend/schemas.py` — existing Pydantic schema patterns`
|
||||
|
||||
## Expected Output
|
||||
|
||||
- ``backend/config.py` — Settings class with SMTP fields added`
|
||||
- ``backend/models.py` — User.notification_preferences column + EmailDigestLog model added`
|
||||
- ``backend/schemas.py` — NotificationPreferences and NotificationPreferencesUpdate schemas`
|
||||
- ``backend/services/email.py` — new module with compose_digest_html, send_email, is_smtp_configured`
|
||||
- ``alembic/versions/029_add_email_digest.py` — migration adding notification_preferences column and email_digest_log table`
|
||||
|
||||
## Verification
|
||||
|
||||
cd backend && python -c "from models import EmailDigestLog, User; from services.email import compose_digest_html, send_email, is_smtp_configured; from config import get_settings; s = get_settings(); assert not is_smtp_configured(s); print('OK')" && cd ../alembic && python -c "import importlib.util; spec = importlib.util.spec_from_file_location('m', 'versions/029_add_email_digest.py'); mod = importlib.util.module_from_spec(spec); print('Migration module loads OK')"
|
||||
85
.gsd/milestones/M025/slices/S01/tasks/T01-SUMMARY.md
Normal file
85
.gsd/milestones/M025/slices/S01/tasks/T01-SUMMARY.md
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
---
|
||||
id: T01
|
||||
parent: S01
|
||||
milestone: M025
|
||||
provides: []
|
||||
requires: []
|
||||
affects: []
|
||||
key_files: ["backend/config.py", "backend/models.py", "backend/schemas.py", "backend/services/email.py", "alembic/versions/029_add_email_digest.py"]
|
||||
key_decisions: ["OSError catch alongside SMTPException in send_email to handle DNS/network failures from socket layer"]
|
||||
patterns_established: []
|
||||
drill_down_paths: []
|
||||
observability_surfaces: []
|
||||
duration: ""
|
||||
verification_result: "Ran task plan verification commands: model imports succeed, is_smtp_configured returns False with empty defaults, migration module loads. Ran 5 negative test cases: empty host/from_address rejected, unconfigured send returns False, empty content groups produce minimal HTML, SMTP connection failure returns False with logged exception, schema defaults correct."
|
||||
completed_at: 2026-04-04T12:11:08.433Z
|
||||
blocker_discovered: false
|
||||
---
|
||||
|
||||
# T01: Added SMTP config, User notification_preferences JSONB, EmailDigestLog model, migration 029, email digest composer+sender, and notification preference schemas
|
||||
|
||||
> Added SMTP config, User notification_preferences JSONB, EmailDigestLog model, migration 029, email digest composer+sender, and notification preference schemas
|
||||
|
||||
## What Happened
|
||||
---
|
||||
id: T01
|
||||
parent: S01
|
||||
milestone: M025
|
||||
key_files:
|
||||
- backend/config.py
|
||||
- backend/models.py
|
||||
- backend/schemas.py
|
||||
- backend/services/email.py
|
||||
- alembic/versions/029_add_email_digest.py
|
||||
key_decisions:
|
||||
- OSError catch alongside SMTPException in send_email to handle DNS/network failures from socket layer
|
||||
duration: ""
|
||||
verification_result: passed
|
||||
completed_at: 2026-04-04T12:11:08.433Z
|
||||
blocker_discovered: false
|
||||
---
|
||||
|
||||
# T01: Added SMTP config, User notification_preferences JSONB, EmailDigestLog model, migration 029, email digest composer+sender, and notification preference schemas
|
||||
|
||||
**Added SMTP config, User notification_preferences JSONB, EmailDigestLog model, migration 029, email digest composer+sender, and notification preference schemas**
|
||||
|
||||
## What Happened
|
||||
|
||||
Added six SMTP fields to config.py Settings with safe empty defaults. Added notification_preferences JSONB column to User model with server_default. Created EmailDigestLog model with composite index on (user_id, digest_sent_at). Created Alembic migration 029. Built email service module with compose_digest_html (inline-CSS grouped by creator), send_email (smtplib with STARTTLS, 10s timeout, bool return), and is_smtp_configured gate. Added NotificationPreferences and NotificationPreferencesUpdate Pydantic schemas.
|
||||
|
||||
## Verification
|
||||
|
||||
Ran task plan verification commands: model imports succeed, is_smtp_configured returns False with empty defaults, migration module loads. Ran 5 negative test cases: empty host/from_address rejected, unconfigured send returns False, empty content groups produce minimal HTML, SMTP connection failure returns False with logged exception, schema defaults correct.
|
||||
|
||||
## Verification Evidence
|
||||
|
||||
| # | Command | Exit Code | Verdict | Duration |
|
||||
|---|---------|-----------|---------|----------|
|
||||
| 1 | `python -c 'from models import EmailDigestLog, User; from services.email import ...; assert not is_smtp_configured(s)'` | 0 | ✅ pass | 1200ms |
|
||||
| 2 | `python -c 'import importlib.util; spec = ...(029_add_email_digest.py)'` | 0 | ✅ pass | 300ms |
|
||||
| 3 | `Negative tests: 5 cases (empty config, connection failure, empty groups)` | 0 | ✅ pass | 2100ms |
|
||||
| 4 | `Schemas import + defaults validation` | 0 | ✅ pass | 400ms |
|
||||
|
||||
|
||||
## Deviations
|
||||
|
||||
Added OSError catch in send_email alongside SMTPException — DNS resolution failures raise socket.gaierror (subclass of OSError), not SMTPException.
|
||||
|
||||
## Known Issues
|
||||
|
||||
None.
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `backend/config.py`
|
||||
- `backend/models.py`
|
||||
- `backend/schemas.py`
|
||||
- `backend/services/email.py`
|
||||
- `alembic/versions/029_add_email_digest.py`
|
||||
|
||||
|
||||
## Deviations
|
||||
Added OSError catch in send_email alongside SMTPException — DNS resolution failures raise socket.gaierror (subclass of OSError), not SMTPException.
|
||||
|
||||
## Known Issues
|
||||
None.
|
||||
83
.gsd/milestones/M025/slices/S01/tasks/T02-PLAN.md
Normal file
83
.gsd/milestones/M025/slices/S01/tasks/T02-PLAN.md
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
---
|
||||
estimated_steps: 44
|
||||
estimated_files: 4
|
||||
skills_used: []
|
||||
---
|
||||
|
||||
# T02: Digest Celery task with Beat scheduling and Docker config
|
||||
|
||||
Build the digest orchestration: a Celery task that queries new content since each user's last digest, groups by followed creator, composes and sends emails via the email service, and logs successful sends. Configure Celery Beat to run it daily. Update docker-compose.yml to add --beat flag to the worker.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Create `backend/tasks/notifications.py` with `send_digest_emails` Celery task:
|
||||
- Import celery_app from worker module
|
||||
- Use sync SQLAlchemy engine (same pattern as pipeline/stages.py — create sync engine from DATABASE_URL converted to postgresql://)
|
||||
- Query all users where `notification_preferences->>'email_digests'` is true
|
||||
- For each user, find their last EmailDigestLog entry (max digest_sent_at)
|
||||
- Query CreatorFollow to get followed creator_ids
|
||||
- For each followed creator: query Posts where `is_published=True AND created_at > last_digest_at`, query TechniquePages where `created_at > last_digest_at` (join through TechniquePageVideo → SourceVideo to match creator_id)
|
||||
- Skip user if no new content across all followed creators
|
||||
- Call `compose_digest_html` with grouped content, generate signed unsubscribe URL
|
||||
- Call `send_email` — on success, INSERT EmailDigestLog with content_summary JSONB
|
||||
- If `is_smtp_configured()` returns False, log warning and return early (graceful no-op)
|
||||
- Log structured messages: task start, per-user send result, task complete with count
|
||||
|
||||
2. Add Celery Beat schedule to `backend/worker.py`:
|
||||
```python
|
||||
celery_app.conf.beat_schedule = {
|
||||
'send-digest-emails': {
|
||||
'task': 'tasks.notifications.send_digest_emails',
|
||||
'schedule': crontab(hour=9, minute=0), # daily at 9am UTC
|
||||
},
|
||||
}
|
||||
```
|
||||
Import `from celery.schedules import crontab` at top.
|
||||
|
||||
3. Import `tasks.notifications` in worker.py alongside the existing `pipeline.stages` import so the task decorator registers.
|
||||
|
||||
4. Update `docker-compose.yml` worker command from `["celery", "-A", "worker", "worker", "--loglevel=info", "--concurrency=1"]` to `["celery", "-A", "worker", "worker", "--beat", "--loglevel=info", "--concurrency=1"]`.
|
||||
|
||||
5. Generate signed unsubscribe tokens: use `itsdangerous.URLSafeTimedSerializer` with `settings.app_secret_key`. Token encodes user_id. Unsubscribe URL format: `{base_url}/api/v1/notifications/unsubscribe?token={token}`. Token valid for 30 days.
|
||||
|
||||
## Failure Modes
|
||||
|
||||
| Dependency | On error | On timeout | On malformed response |
|
||||
|------------|----------|-----------|----------------------|
|
||||
| SMTP (via email service) | Log error per user, continue to next user | 10s timeout in email service | N/A |
|
||||
| PostgreSQL | Task fails with exception, Celery retries | Standard SQLAlchemy timeout | N/A |
|
||||
| Redis (Celery broker) | Task not dispatched, Beat retries next schedule | Celery handles reconnection | N/A |
|
||||
|
||||
## Load Profile
|
||||
|
||||
- Shared resources: sync DB session (one per task run), SMTP connection (one per email)
|
||||
- Per-operation cost: ~3 DB queries per user (last digest, follows, content), 1 SMTP send per user
|
||||
- 10x breakpoint: N/A — <50 users, volume is trivial. At 500+ users, batch SMTP connections.
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] Digest task queries correct new content per followed creator
|
||||
- [ ] Graceful no-op when SMTP is unconfigured
|
||||
- [ ] EmailDigestLog written only on successful send (deduplication)
|
||||
- [ ] Celery Beat schedule configured for daily run
|
||||
- [ ] Docker worker command includes --beat flag
|
||||
- [ ] Signed unsubscribe URL generated per email
|
||||
|
||||
## Inputs
|
||||
|
||||
- ``backend/services/email.py` — email service from T01`
|
||||
- ``backend/models.py` — EmailDigestLog, User, CreatorFollow, Post, TechniquePage, TechniquePageVideo, SourceVideo models from T01`
|
||||
- ``backend/config.py` — SMTP settings from T01`
|
||||
- ``backend/worker.py` — existing Celery app config`
|
||||
- ``docker-compose.yml` — existing worker service definition`
|
||||
|
||||
## Expected Output
|
||||
|
||||
- ``backend/tasks/__init__.py` — empty package init`
|
||||
- ``backend/tasks/notifications.py` — send_digest_emails Celery task with content query, grouping, email send, and logging`
|
||||
- ``backend/worker.py` — Beat schedule added, tasks.notifications import added`
|
||||
- ``docker-compose.yml` — worker command updated with --beat flag`
|
||||
|
||||
## Verification
|
||||
|
||||
cd backend && python -c "from tasks.notifications import send_digest_emails; print('Task imports OK')" && python -c "from worker import celery_app; assert 'send-digest-emails' in celery_app.conf.beat_schedule; print('Beat schedule OK')" && grep -q '\-\-beat' ../docker-compose.yml && echo 'Docker beat flag OK'
|
||||
73
.gsd/milestones/M025/slices/S01/tasks/T03-PLAN.md
Normal file
73
.gsd/milestones/M025/slices/S01/tasks/T03-PLAN.md
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
---
|
||||
estimated_steps: 33
|
||||
estimated_files: 6
|
||||
skills_used: []
|
||||
---
|
||||
|
||||
# T03: Notification preferences API, unsubscribe endpoint, frontend settings, and integration test
|
||||
|
||||
Wire the user-facing surfaces: API endpoints for reading/updating notification preferences, a public unsubscribe endpoint (no auth required, uses signed token), a frontend settings toggle in the creator settings page, and an integration test that verifies the full digest flow with mocked SMTP.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Create `backend/routers/notifications.py`:
|
||||
- `GET /notifications/preferences` — returns current user's notification_preferences JSONB. Requires auth.
|
||||
- `PUT /notifications/preferences` — updates notification_preferences. Validates against NotificationPreferencesUpdate schema. Requires auth.
|
||||
- `GET /notifications/unsubscribe` — accepts `token` query param. Decodes with `itsdangerous.URLSafeTimedSerializer` (max_age=30 days). Sets `notification_preferences.email_digests = False` for the user. Returns simple HTML page confirming unsubscription. No auth required.
|
||||
|
||||
2. Mount the router in `backend/main.py`: `app.include_router(notifications.router, prefix="/api/v1")`
|
||||
|
||||
3. Create `frontend/src/api/notifications.ts`:
|
||||
- `getNotificationPreferences()` — GET /api/v1/notifications/preferences
|
||||
- `updateNotificationPreferences(prefs)` — PUT /api/v1/notifications/preferences
|
||||
|
||||
4. Add notification settings section to `frontend/src/pages/CreatorSettings.tsx`:
|
||||
- New section "Email Notifications" with a toggle switch for email digests (on/off)
|
||||
- Frequency selector (daily/weekly) shown only when digests are enabled
|
||||
- Fetch current prefs on mount, PUT on change with optimistic UI update
|
||||
- Use existing form/toggle patterns from the page
|
||||
|
||||
5. Create integration test `backend/tests/test_notifications.py`:
|
||||
- Test GET/PUT preferences endpoints (auth required, returns correct shape)
|
||||
- Test unsubscribe endpoint with valid and expired tokens
|
||||
- Test digest task end-to-end: create test user, creator, follow, published post, mock SMTP, run `send_digest_emails`, assert email was "sent" (mock called with correct args), assert EmailDigestLog created
|
||||
- Test digest task skips when no new content
|
||||
- Test digest task no-op when SMTP unconfigured
|
||||
|
||||
6. Add `itsdangerous` to `requirements.txt` if not already present.
|
||||
|
||||
## Negative Tests
|
||||
|
||||
- PUT preferences with invalid digest_frequency value → 422
|
||||
- Unsubscribe with expired token (>30 days) → error page
|
||||
- Unsubscribe with tampered token → error page
|
||||
- GET/PUT preferences without auth → 401
|
||||
|
||||
## Must-Haves
|
||||
|
||||
- [ ] GET/PUT notification preferences endpoints with auth
|
||||
- [ ] Unsubscribe endpoint works without auth via signed token
|
||||
- [ ] Frontend toggle for email digests in creator settings
|
||||
- [ ] Integration test covers digest task happy path with mocked SMTP
|
||||
- [ ] Integration test covers unsubscribe with valid and invalid tokens
|
||||
|
||||
## Inputs
|
||||
|
||||
- ``backend/services/email.py` — email service from T01`
|
||||
- ``backend/models.py` — models from T01`
|
||||
- ``backend/schemas.py` — notification preference schemas from T01`
|
||||
- ``backend/config.py` — SMTP settings from T01`
|
||||
- ``backend/tasks/notifications.py` — digest task from T02`
|
||||
- ``frontend/src/pages/CreatorSettings.tsx` — existing settings page to extend`
|
||||
|
||||
## Expected Output
|
||||
|
||||
- ``backend/routers/notifications.py` — GET/PUT preferences + unsubscribe endpoints`
|
||||
- ``backend/main.py` — notifications router mounted`
|
||||
- ``backend/tests/test_notifications.py` — integration tests for preferences, unsubscribe, and digest task`
|
||||
- ``frontend/src/api/notifications.ts` — API client functions`
|
||||
- ``frontend/src/pages/CreatorSettings.tsx` — notification settings section added`
|
||||
|
||||
## Verification
|
||||
|
||||
cd backend && python -m pytest tests/test_notifications.py -v --timeout=30 2>&1 | tail -20
|
||||
16761
.gsd/reports/M024-2026-04-04T12-02-28.html
Normal file
16761
.gsd/reports/M024-2026-04-04T12-02-28.html
Normal file
File diff suppressed because one or more lines are too long
|
|
@ -130,7 +130,7 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
|
|||
</div>
|
||||
<div class="hdr-right">
|
||||
<span class="gen-lbl">Updated</span>
|
||||
<span class="gen">Apr 4, 2026, 10:23 AM</span>
|
||||
<span class="gen">Apr 4, 2026, 12:02 PM</span>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
|
@ -168,6 +168,10 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
|
|||
<div class="toc-group-label">M023</div>
|
||||
<ul><li><a href="M023-2026-04-04T10-23-24.html">Apr 4, 2026, 10:23 AM</a> <span class="toc-kind toc-milestone">milestone</span></li></ul>
|
||||
</div>
|
||||
<div class="toc-group">
|
||||
<div class="toc-group-label">M024</div>
|
||||
<ul><li><a href="M024-2026-04-04T12-02-28.html">Apr 4, 2026, 12:02 PM</a> <span class="toc-kind toc-milestone">milestone</span></li></ul>
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
<!-- Main content -->
|
||||
|
|
@ -176,47 +180,49 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
|
|||
<h2>Project Overview</h2>
|
||||
|
||||
<div class="idx-summary">
|
||||
<div class="idx-stat"><span class="idx-val">$550.85</span><span class="idx-lbl">Total Cost</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">777.32M</span><span class="idx-lbl">Total Tokens</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">23h 59m</span><span class="idx-lbl">Duration</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">106/123</span><span class="idx-lbl">Slices</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">23/25</span><span class="idx-lbl">Milestones</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">7</span><span class="idx-lbl">Reports</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">$590.21</span><span class="idx-lbl">Total Cost</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">835.08M</span><span class="idx-lbl">Total Tokens</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">25h 38m</span><span class="idx-lbl">Duration</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">112/123</span><span class="idx-lbl">Slices</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">24/25</span><span class="idx-lbl">Milestones</span></div>
|
||||
<div class="idx-stat"><span class="idx-val">8</span><span class="idx-lbl">Reports</span></div>
|
||||
</div>
|
||||
<div class="idx-progress">
|
||||
<div class="idx-bar-track"><div class="idx-bar-fill" style="width:86%"></div></div>
|
||||
<span class="idx-pct">86% complete</span>
|
||||
<div class="idx-bar-track"><div class="idx-bar-fill" style="width:91%"></div></div>
|
||||
<span class="idx-pct">91% complete</span>
|
||||
</div>
|
||||
<div class="sparkline-wrap"><h3>Cost Progression</h3>
|
||||
<div class="sparkline">
|
||||
<svg viewBox="0 0 600 60" width="600" height="60" class="spark-svg">
|
||||
<polyline points="12.0,36.7 108.0,36.2 204.0,24.1 300.0,21.1 396.0,16.3 492.0,14.2 588.0,12.0" class="spark-line" fill="none"/>
|
||||
<circle cx="12.0" cy="36.7" r="3" class="spark-dot">
|
||||
<polyline points="12.0,37.5 94.3,37.0 176.6,25.7 258.9,22.9 341.1,18.4 423.4,16.5 505.7,14.4 588.0,12.0" class="spark-line" fill="none"/>
|
||||
<circle cx="12.0" cy="37.5" r="3" class="spark-dot">
|
||||
<title>M008: M008: Credibility Debt Cleanup — Broken Links, Test Data, Jargon, Empty Metrics — $172.23</title>
|
||||
</circle><circle cx="108.0" cy="36.2" r="3" class="spark-dot">
|
||||
</circle><circle cx="94.3" cy="37.0" r="3" class="spark-dot">
|
||||
<title>M009: Homepage & First Impression — $180.97</title>
|
||||
</circle><circle cx="204.0" cy="24.1" r="3" class="spark-dot">
|
||||
</circle><circle cx="176.6" cy="25.7" r="3" class="spark-dot">
|
||||
<title>M018: M018: Phase 2 Research & Documentation — Site Audit and Forgejo Wiki Bootstrap — $365.18</title>
|
||||
</circle><circle cx="300.0" cy="21.1" r="3" class="spark-dot">
|
||||
</circle><circle cx="258.9" cy="22.9" r="3" class="spark-dot">
|
||||
<title>M019: Foundations — Auth, Consent & LightRAG — $411.26</title>
|
||||
</circle><circle cx="396.0" cy="16.3" r="3" class="spark-dot">
|
||||
</circle><circle cx="341.1" cy="18.4" r="3" class="spark-dot">
|
||||
<title>M021: Intelligence Online — Chat, Chapters & Search Cutover — $485.08</title>
|
||||
</circle><circle cx="492.0" cy="14.2" r="3" class="spark-dot">
|
||||
</circle><circle cx="423.4" cy="16.5" r="3" class="spark-dot">
|
||||
<title>M022: Creator Tools & Personality — $516.43</title>
|
||||
</circle><circle cx="588.0" cy="12.0" r="3" class="spark-dot">
|
||||
</circle><circle cx="505.7" cy="14.4" r="3" class="spark-dot">
|
||||
<title>M023: MVP Integration — Demo Build — $550.85</title>
|
||||
</circle><circle cx="588.0" cy="12.0" r="3" class="spark-dot">
|
||||
<title>M024: Polish, Shorts Pipeline & Citations — $590.21</title>
|
||||
</circle>
|
||||
<text x="12" y="58" class="spark-lbl">$172.23</text>
|
||||
<text x="588" y="58" text-anchor="end" class="spark-lbl">$550.85</text>
|
||||
<text x="588" y="58" text-anchor="end" class="spark-lbl">$590.21</text>
|
||||
</svg>
|
||||
<div class="spark-axis">
|
||||
<span class="spark-tick" style="left:2.0%" title="2026-03-31T05:31:26.249Z">M008</span><span class="spark-tick" style="left:18.0%" title="2026-03-31T05:52:28.456Z">M009</span><span class="spark-tick" style="left:34.0%" title="2026-04-03T21:17:51.201Z">M018</span><span class="spark-tick" style="left:50.0%" title="2026-04-03T23:30:16.641Z">M019</span><span class="spark-tick" style="left:66.0%" title="2026-04-04T06:50:37.759Z">M021</span><span class="spark-tick" style="left:82.0%" title="2026-04-04T08:51:51.223Z">M022</span><span class="spark-tick" style="left:98.0%" title="2026-04-04T10:23:24.247Z">M023</span>
|
||||
<span class="spark-tick" style="left:2.0%" title="2026-03-31T05:31:26.249Z">M008</span><span class="spark-tick" style="left:15.7%" title="2026-03-31T05:52:28.456Z">M009</span><span class="spark-tick" style="left:29.4%" title="2026-04-03T21:17:51.201Z">M018</span><span class="spark-tick" style="left:43.1%" title="2026-04-03T23:30:16.641Z">M019</span><span class="spark-tick" style="left:56.9%" title="2026-04-04T06:50:37.759Z">M021</span><span class="spark-tick" style="left:70.6%" title="2026-04-04T08:51:51.223Z">M022</span><span class="spark-tick" style="left:84.3%" title="2026-04-04T10:23:24.247Z">M023</span><span class="spark-tick" style="left:98.0%" title="2026-04-04T12:02:28.692Z">M024</span>
|
||||
</div>
|
||||
</div></div>
|
||||
</section>
|
||||
|
||||
<section class="idx-cards">
|
||||
<h2>Progression <span class="sec-count">7</span></h2>
|
||||
<h2>Progression <span class="sec-count">8</span></h2>
|
||||
<div class="cards-grid">
|
||||
<a class="report-card" href="M008-2026-03-31T05-31-26.html">
|
||||
<div class="card-top">
|
||||
|
|
@ -344,7 +350,7 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
|
|||
<div class="card-delta"><span>+$31.35</span><span>+7 slices</span><span>+1 milestone</span></div>
|
||||
|
||||
</a>
|
||||
<a class="report-card card-latest" href="M023-2026-04-04T10-23-24.html">
|
||||
<a class="report-card" href="M023-2026-04-04T10-23-24.html">
|
||||
<div class="card-top">
|
||||
<span class="card-label">M023: MVP Integration — Demo Build</span>
|
||||
<span class="card-kind card-kind-milestone">milestone</span>
|
||||
|
|
@ -363,6 +369,27 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
|
|||
<span>106/123 slices</span>
|
||||
</div>
|
||||
<div class="card-delta"><span>+$34.42</span><span>+5 slices</span><span>+1 milestone</span></div>
|
||||
|
||||
</a>
|
||||
<a class="report-card card-latest" href="M024-2026-04-04T12-02-28.html">
|
||||
<div class="card-top">
|
||||
<span class="card-label">M024: Polish, Shorts Pipeline & Citations</span>
|
||||
<span class="card-kind card-kind-milestone">milestone</span>
|
||||
</div>
|
||||
<div class="card-date">Apr 4, 2026, 12:02 PM</div>
|
||||
<div class="card-progress">
|
||||
<div class="card-bar-track">
|
||||
<div class="card-bar-fill" style="width:91%"></div>
|
||||
</div>
|
||||
<span class="card-pct">91%</span>
|
||||
</div>
|
||||
<div class="card-stats">
|
||||
<span>$590.21</span>
|
||||
<span>835.08M</span>
|
||||
<span>25h 38m</span>
|
||||
<span>112/123 slices</span>
|
||||
</div>
|
||||
<div class="card-delta"><span>+$39.36</span><span>+6 slices</span><span>+1 milestone</span></div>
|
||||
<div class="card-latest-badge">Latest</div>
|
||||
</a></div>
|
||||
</section>
|
||||
|
|
@ -377,7 +404,7 @@ footer{border-top:1px solid var(--border-1);padding:16px 32px}
|
|||
<span class="ftr-sep">—</span>
|
||||
<span>/home/aux/projects/content-to-kb-automator</span>
|
||||
<span class="ftr-sep">—</span>
|
||||
<span>Updated Apr 4, 2026, 10:23 AM</span>
|
||||
<span>Updated Apr 4, 2026, 12:02 PM</span>
|
||||
</div>
|
||||
</footer>
|
||||
</body>
|
||||
|
|
|
|||
|
|
@ -115,6 +115,22 @@
|
|||
"doneMilestones": 23,
|
||||
"totalMilestones": 25,
|
||||
"phase": "planning"
|
||||
},
|
||||
{
|
||||
"filename": "M024-2026-04-04T12-02-28.html",
|
||||
"generatedAt": "2026-04-04T12:02:28.692Z",
|
||||
"milestoneId": "M024",
|
||||
"milestoneTitle": "Polish, Shorts Pipeline & Citations",
|
||||
"label": "M024: Polish, Shorts Pipeline & Citations",
|
||||
"kind": "milestone",
|
||||
"totalCost": 590.205655,
|
||||
"totalTokens": 835082375,
|
||||
"totalDuration": 92306404,
|
||||
"doneSlices": 112,
|
||||
"totalSlices": 123,
|
||||
"doneMilestones": 24,
|
||||
"totalMilestones": 25,
|
||||
"phase": "planning"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
|
|||
48
alembic/versions/029_add_email_digest.py
Normal file
48
alembic/versions/029_add_email_digest.py
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
"""Add notification_preferences to users and email_digest_log table.
|
||||
|
||||
Revision ID: 029_add_email_digest
|
||||
Revises: 028_add_shorts_template
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.dialects.postgresql import JSONB, UUID
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision = "029_add_email_digest"
|
||||
down_revision = "028_add_shorts_template"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# notification_preferences JSONB on users
|
||||
op.add_column(
|
||||
"users",
|
||||
sa.Column(
|
||||
"notification_preferences",
|
||||
JSONB,
|
||||
nullable=False,
|
||||
server_default='{"email_digests": true, "digest_frequency": "daily"}',
|
||||
),
|
||||
)
|
||||
|
||||
# email_digest_log table
|
||||
op.create_table(
|
||||
"email_digest_log",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True, server_default=sa.func.gen_random_uuid()),
|
||||
sa.Column("user_id", UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False),
|
||||
sa.Column("digest_sent_at", sa.DateTime, server_default=sa.func.now(), nullable=False),
|
||||
sa.Column("content_summary", JSONB, nullable=True),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_email_digest_log_user_sent",
|
||||
"email_digest_log",
|
||||
["user_id", "digest_sent_at"],
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_email_digest_log_user_sent", table_name="email_digest_log")
|
||||
op.drop_table("email_digest_log")
|
||||
op.drop_column("users", "notification_preferences")
|
||||
|
|
@ -83,6 +83,14 @@ class Settings(BaseSettings):
|
|||
video_metadata_path: str = "/data/video_meta"
|
||||
video_source_path: str = "/videos"
|
||||
|
||||
# SMTP (email digests)
|
||||
smtp_host: str = ""
|
||||
smtp_port: int = 587
|
||||
smtp_user: str = ""
|
||||
smtp_password: str = ""
|
||||
smtp_from_address: str = ""
|
||||
smtp_tls: bool = True
|
||||
|
||||
# Git commit SHA (set at Docker build time or via env var)
|
||||
git_commit_sha: str = "unknown"
|
||||
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ from sqlalchemy import (
|
|||
Enum,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
String,
|
||||
Text,
|
||||
|
|
@ -168,6 +169,10 @@ class User(Base):
|
|||
is_active: Mapped[bool] = mapped_column(
|
||||
Boolean, default=True, server_default="true"
|
||||
)
|
||||
notification_preferences: Mapped[dict] = mapped_column(
|
||||
JSONB, nullable=False,
|
||||
server_default='{"email_digests": true, "digest_frequency": "daily"}',
|
||||
)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
default=_now, server_default=func.now()
|
||||
)
|
||||
|
|
@ -179,6 +184,26 @@ class User(Base):
|
|||
creator: Mapped[Creator | None] = sa_relationship()
|
||||
|
||||
|
||||
class EmailDigestLog(Base):
|
||||
"""Record of a digest email sent to a user."""
|
||||
__tablename__ = "email_digest_log"
|
||||
__table_args__ = (
|
||||
Index("ix_email_digest_log_user_sent", "user_id", "digest_sent_at"),
|
||||
)
|
||||
|
||||
id: Mapped[uuid.UUID] = _uuid_pk()
|
||||
user_id: Mapped[uuid.UUID] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="CASCADE"), nullable=False,
|
||||
)
|
||||
digest_sent_at: Mapped[datetime] = mapped_column(
|
||||
default=_now, server_default=func.now()
|
||||
)
|
||||
content_summary: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
||||
|
||||
# relationships
|
||||
user: Mapped[User] = sa_relationship()
|
||||
|
||||
|
||||
class InviteCode(Base):
|
||||
"""Single-use or limited-use invite codes for registration gating."""
|
||||
__tablename__ = "invite_codes"
|
||||
|
|
|
|||
|
|
@ -847,3 +847,17 @@ class ShortsTemplateUpdate(BaseModel):
|
|||
outro_duration_secs: float = Field(default=2.0, ge=1.0, le=5.0)
|
||||
show_intro: bool = False
|
||||
show_outro: bool = False
|
||||
|
||||
|
||||
# ── Notification Preferences ─────────────────────────────────────────────────
|
||||
|
||||
class NotificationPreferences(BaseModel):
|
||||
"""Current notification preferences for a user."""
|
||||
email_digests: bool = True
|
||||
digest_frequency: str = "daily"
|
||||
|
||||
|
||||
class NotificationPreferencesUpdate(BaseModel):
|
||||
"""Partial update for notification preferences."""
|
||||
email_digests: bool | None = None
|
||||
digest_frequency: str | None = None
|
||||
|
|
|
|||
161
backend/services/email.py
Normal file
161
backend/services/email.py
Normal file
|
|
@ -0,0 +1,161 @@
|
|||
"""Email digest composition and SMTP delivery.
|
||||
|
||||
Provides three public functions:
|
||||
- is_smtp_configured(settings) — gate check before attempting sends
|
||||
- compose_digest_html(user_display_name, creator_content_groups, unsubscribe_url)
|
||||
- send_email(to_address, subject, html_body, settings) — delivers via smtplib
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import smtplib
|
||||
from email.mime.multipart import MIMEMultipart
|
||||
from email.mime.text import MIMEText
|
||||
from html import escape
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ── Public API ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def is_smtp_configured(settings: Settings) -> bool:
|
||||
"""Return True only if SMTP host and from-address are set."""
|
||||
return bool(settings.smtp_host) and bool(settings.smtp_from_address)
|
||||
|
||||
|
||||
def compose_digest_html(
|
||||
user_display_name: str,
|
||||
creator_content_groups: list[dict],
|
||||
unsubscribe_url: str,
|
||||
) -> str:
|
||||
"""Build an HTML email body for the digest.
|
||||
|
||||
Args:
|
||||
user_display_name: Recipient's display name.
|
||||
creator_content_groups: List of dicts, each with:
|
||||
- creator_name (str)
|
||||
- posts (list of {title, url})
|
||||
- technique_pages (list of {title, url})
|
||||
unsubscribe_url: Signed link to toggle off digests.
|
||||
|
||||
Returns:
|
||||
HTML string ready for MIMEText.
|
||||
"""
|
||||
if not creator_content_groups:
|
||||
return _minimal_html(user_display_name, unsubscribe_url)
|
||||
|
||||
sections: list[str] = []
|
||||
for group in creator_content_groups:
|
||||
creator_name = escape(group.get("creator_name", "Unknown"))
|
||||
items_html = ""
|
||||
|
||||
for post in group.get("posts", []):
|
||||
title = escape(post.get("title", "Untitled"))
|
||||
url = escape(post.get("url", "#"))
|
||||
items_html += (
|
||||
f'<li style="margin-bottom:6px;">'
|
||||
f'<a href="{url}" style="color:#3b82f6;text-decoration:none;">{title}</a>'
|
||||
f' <span style="color:#9ca3af;font-size:12px;">(post)</span></li>'
|
||||
)
|
||||
|
||||
for page in group.get("technique_pages", []):
|
||||
title = escape(page.get("title", "Untitled"))
|
||||
url = escape(page.get("url", "#"))
|
||||
items_html += (
|
||||
f'<li style="margin-bottom:6px;">'
|
||||
f'<a href="{url}" style="color:#3b82f6;text-decoration:none;">{title}</a>'
|
||||
f' <span style="color:#9ca3af;font-size:12px;">(technique)</span></li>'
|
||||
)
|
||||
|
||||
if items_html:
|
||||
sections.append(
|
||||
f'<h3 style="margin:16px 0 8px;font-size:16px;color:#f1f5f9;">{creator_name}</h3>'
|
||||
f'<ul style="list-style:none;padding:0;margin:0;">{items_html}</ul>'
|
||||
)
|
||||
|
||||
body_content = "\n".join(sections) if sections else "<p>No new content this period.</p>"
|
||||
|
||||
return _wrap_html(user_display_name, body_content, unsubscribe_url)
|
||||
|
||||
|
||||
def send_email(
|
||||
to_address: str,
|
||||
subject: str,
|
||||
html_body: str,
|
||||
settings: Settings,
|
||||
) -> bool:
|
||||
"""Send an HTML email via SMTP.
|
||||
|
||||
Returns True on success, False on any SMTP error (logged).
|
||||
Uses a 10-second connection timeout.
|
||||
"""
|
||||
if not is_smtp_configured(settings):
|
||||
logger.warning("send_email called but SMTP is not configured")
|
||||
return False
|
||||
|
||||
msg = MIMEMultipart("alternative")
|
||||
msg["Subject"] = subject
|
||||
msg["From"] = settings.smtp_from_address
|
||||
msg["To"] = to_address
|
||||
msg.attach(MIMEText(html_body, "html", "utf-8"))
|
||||
|
||||
try:
|
||||
if settings.smtp_tls:
|
||||
server = smtplib.SMTP(settings.smtp_host, settings.smtp_port, timeout=10)
|
||||
server.starttls()
|
||||
else:
|
||||
server = smtplib.SMTP(settings.smtp_host, settings.smtp_port, timeout=10)
|
||||
|
||||
if settings.smtp_user:
|
||||
server.login(settings.smtp_user, settings.smtp_password)
|
||||
|
||||
server.sendmail(settings.smtp_from_address, to_address, msg.as_string())
|
||||
server.quit()
|
||||
logger.info("Digest email sent to %s", to_address)
|
||||
return True
|
||||
except smtplib.SMTPException:
|
||||
logger.exception("SMTP error sending digest to %s", to_address)
|
||||
return False
|
||||
except OSError:
|
||||
logger.exception("Network error connecting to SMTP server for %s", to_address)
|
||||
return False
|
||||
|
||||
|
||||
# ── Private helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _minimal_html(user_display_name: str, unsubscribe_url: str) -> str:
|
||||
"""Fallback HTML when there's no new content."""
|
||||
return _wrap_html(
|
||||
user_display_name,
|
||||
'<p style="color:#94a3b8;">No new content from your followed creators this period.</p>',
|
||||
unsubscribe_url,
|
||||
)
|
||||
|
||||
|
||||
def _wrap_html(user_display_name: str, body_content: str, unsubscribe_url: str) -> str:
|
||||
"""Wrap body content in the standard digest email shell."""
|
||||
name = escape(user_display_name)
|
||||
unsub = escape(unsubscribe_url)
|
||||
return f"""\
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><meta charset="utf-8"></head>
|
||||
<body style="margin:0;padding:0;background:#0f172a;font-family:system-ui,-apple-system,sans-serif;">
|
||||
<div style="max-width:600px;margin:0 auto;padding:24px;">
|
||||
<h1 style="font-size:22px;color:#e2e8f0;margin-bottom:4px;">Chrysopedia Digest</h1>
|
||||
<p style="color:#94a3b8;margin-top:0;">Hi {name}, here's what's new from creators you follow.</p>
|
||||
<hr style="border:none;border-top:1px solid #1e293b;margin:16px 0;">
|
||||
{body_content}
|
||||
<hr style="border:none;border-top:1px solid #1e293b;margin:24px 0 12px;">
|
||||
<p style="font-size:12px;color:#64748b;text-align:center;">
|
||||
<a href="{unsub}" style="color:#64748b;text-decoration:underline;">Unsubscribe from digests</a>
|
||||
</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>"""
|
||||
Loading…
Add table
Reference in a new issue