---
id: T01
parent: S03
milestone: M007
provides: []
requires: []
affects: []
key_files: ["backend/watcher.py", "backend/requirements.txt"]
key_decisions: ["Used httpx sync client since watcher runs in synchronous watchdog callback threads", "Capped error sidecar response body at 500 chars", "Ignore files in processed/ and failed/ subdirs to prevent re-processing loops"]
patterns_established: []
drill_down_paths: []
observability_surfaces: []
duration: ""
verification_result: "All four slice-level verification checks pass: py_compile exits 0, watchdog present in requirements.txt, PollingObserver present in watcher.py, processed and failed strings present in watcher.py."
completed_at: 2026-03-30T19:17:44.067Z
blocker_discovered: false
---

# T01: Built backend/watcher.py with PollingObserver-based folder watching, file stability detection, JSON validation, multipart POST to /api/v1/ingest, and processed/failed file disposition with error sidecars

> Built backend/watcher.py with PollingObserver-based folder watching, file stability detection, JSON validation, multipart POST to /api/v1/ingest, and processed/failed file disposition with error sidecars

## What Happened
---
id: T01
parent: S03
milestone: M007
key_files:
  - backend/watcher.py
  - backend/requirements.txt
key_decisions:
  - Used httpx sync client since watcher runs in synchronous watchdog callback threads
  - Capped error sidecar response body at 500 chars
  - Ignore files in processed/ and failed/ subdirs to prevent re-processing loops
duration: ""
verification_result: passed
completed_at: 2026-03-30T19:17:44.068Z
blocker_discovered: false
---

# T01: Built backend/watcher.py with PollingObserver-based folder watching, file stability detection, JSON validation, multipart POST to /api/v1/ingest, and processed/failed file disposition with error sidecars

**Built backend/watcher.py with PollingObserver-based folder watching, file stability detection, JSON validation, multipart POST to /api/v1/ingest, and processed/failed file disposition with error sidecars**

## What Happened

Created backend/watcher.py as a standalone Python script using watchdog's PollingObserver for ZFS/NFS reliability. Monitors WATCH_FOLDER for new .json files, waits for file size stability (SCP/rsync handling), validates JSON structure against required keys, POSTs as multipart upload via httpx to the ingest API. Successfully processed files move to processed/, failures move to failed/ with .error sidecar files. Also created backend/requirements.txt synced from ub01 production with watchdog added.

## Verification

All four slice-level verification checks pass: py_compile exits 0, watchdog present in requirements.txt, PollingObserver present in watcher.py, processed and failed strings present in watcher.py.

## Verification Evidence

| # | Command | Exit Code | Verdict | Duration |
|---|---------|-----------|---------|----------|
| 1 | `python -m py_compile backend/watcher.py` | 0 | ✅ pass | 500ms |
| 2 | `grep -q 'watchdog' backend/requirements.txt` | 0 | ✅ pass | 50ms |
| 3 | `grep -q 'PollingObserver' backend/watcher.py` | 0 | ✅ pass | 50ms |
| 4 | `grep -q 'processed' backend/watcher.py && grep -q 'failed' backend/watcher.py` | 0 | ✅ pass | 50ms |


## Deviations

None.

## Known Issues

None.

## Files Created/Modified

- `backend/watcher.py`
- `backend/requirements.txt`


## Deviations
None.

## Known Issues
None.
