- Rust 78.4%
- JavaScript 11.3%
- HTML 5.5%
- CSS 3%
- D2 1.6%
- Other 0.2%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
Dashboard - Add webcam-based face enrollment on the users page: capture frames in the browser, post them to a new enrollment handler that invokes vision-service with a timeout and serializes concurrent enrollments. - Add `/api/esp/events` and an in-memory ring buffer (`esp_events.rs`) so the serial monitor can replay recent encrypted UART frames; sensor events are excluded from access totals. - Persist ESP32 `EVENT:AUTH:*` / `EVENT:SENSOR:*` frames from the UART link into `auth_attempts`, with HELLO heartbeats to survive ESP32 reboots. - Default HLS segments to 1 s for a lower-latency live view. Firmware - Build for the `xtensa-esp32s3-espidf` target with build-std and ldproxy. - Emit PIR, RFID, keypad and PIN decision telemetry over the encrypted link. - Treat `ESP_ERR_TIMEOUT` on an empty UART ring buffer as "no data". Vision - Validate and upsert enrollments (reject split id/name, replace duplicates) and log pipeline decisions. Docs/diagrams - Add `docs/encryption.md` (D2 + PNG), `diagrams/system_simple.d2`, and update the Pi 5 setup, analytics, dashboard, firmware and vision READMEs. |
||
| crates | ||
| data | ||
| diagrams | ||
| docs | ||
| .gitignore | ||
| AGENTS.md | ||
| Cargo.lock | ||
| Cargo.toml | ||
| README.md | ||
| shell.nix | ||
smarty
Smarty is a small physical access-control system for a door:
- A Rust dashboard (
crates/dashboard) with an admin UI, SQLite audit log, camera HLS proxy, lock API, device auth API, and a live vision overlay. - An ESP32-S3 firmware (
crates/firmware) for PIR, RFID, keypad, servo, I2C LCD, UART-to-Pi and a task watchdog. - A Pi vision service (
crates/vision-service) that runs face detection, passive anti-spoofing, active liveness and recognition on the camera feed. - A Raspberry Pi 5 setup that runs the dashboard and publishes the Camera Module as an RTSP/HLS feed.
Browser ──HTTP──> dashboard (Axum, one process)
├─ SQLite data/door.db
├─ ffmpeg (1x) RTSP -> data/hls (last 10 min)
├─ SSE live face boxes / scores
└─ /dev/serial0 ──> ESP32-S3
├─ PIR / RFID / keypad
├─ servo lock
└─ I2C LCD
Pi 5 Camera ──RTSP──> dashboard HLS
└──────> vision-service
├─ SCRFD detection
├─ MiniFASNet passive anti-spoofing
├─ active head-movement liveness
└─ ArcFace recognition
│
└──POST /api/vision/result──> dashboard
Workspace
| Path | Purpose |
|---|---|
crates/dashboard |
Axum web app: admin UI, lock API, device auth API, camera HLS proxy, live vision overlay |
crates/vision |
Face detection, anti-spoofing, active liveness, recognition and enrollment |
crates/vision-protocol |
Shared live-event types between the Pi service and dashboard |
crates/vision-service |
Pi camera capture + vision pipeline + dashboard client |
crates/uart-protocol |
Encrypted, authenticated Pi ↔ ESP32 UART framing |
crates/firmware |
ESP32-S3 firmware (separate Cargo workspace, Xtensa target) |
docs/pi5-setup.md |
Pi 5 OS, camera, UART, vision, deployment and verification |
cad/ |
Reserved for OpenSCAD sources (empty for now) |
data/ |
Runtime SQLite, HLS and vision models/enrollments (git-ignored) |
diagrams/ |
D2 hardware, power and software diagrams |
AGENTS.md |
Repository conventions and commands for contributors/agents |
The Rust dashboard/firmware are the source of truth. The firmware is deliberately
a separate workspace because it builds for xtensa-esp32s3-none-elf. The
original Python dashboard and Arduino sketches have been retired.
Quick start (dashboard)
nix-shell
cargo run -p smarty-dashboard -- init-db
cargo run -p smarty-dashboard -- create-admin --username admin
cargo run -p smarty-dashboard
# open http://localhost:8000
Development without a Pi works: GPIO/serial/ffmpeg failures fall back to no-ops,
and CAMERA_ENABLED=0 disables the camera.
Vision / liveness
The Pi service posts a live VisionEvent for every processed frame. The
dashboard broadcasts it over SSE to the camera page, which draws the face box,
identity, similarity, passive anti-spoofing result and active-liveness progress
on top of the HLS video.
An unlock requires all of:
- Exactly one detected face.
- Passive anti-spoofing passes (MiniFASNetV2).
- Active liveness passes (the user turns their head slightly).
- Face recognition matches
enrollments.json. - The identity matches an active
door_users.face_labelin the dashboard.
See crates/vision/README.md and
crates/vision-service/README.md.
In-dashboard analytics and tests
The Analytics page (/analytics) shows live door/camera/power status,
auth success/failure, spoof and recognition rates, lock reliability, latency
histograms, recent audit rows, and threshold/target checks. It also has
click-to-run software tests and an optional labelled-image vision benchmark.
See docs/analytics.md.
An optional /metrics endpoint remains available for Prometheus, but the
in-dashboard page does not depend on Prometheus or Grafana.
Encrypted Pi ↔ ESP32 link
All dashboard ↔ ESP32 communication is ChaCha20-Poly1305 authenticated
encryption with per-frame sessions/sequence numbers. There is no plaintext
fallback. Set the same 32-byte key (openssl rand -hex 32) in the dashboard's
UART_SHARED_KEY and compile it into the firmware with SMARTY_UART_KEY.
See crates/firmware/README.md and
docs/pi5-setup.md.
Credential hashing
New administrator passwords and door PINs use salted PBKDF2-HMAC-SHA256 (210,000 iterations, Werkzeug-compatible format). Legacy scrypt hashes are verified and re-hashed on the next successful use. Face embeddings are not hashed because recognition requires the vectors; protect the git-ignored enrollment file instead.
Rolling 10-minute video buffer
The dashboard ffmpeg process keeps the last VIDEO_HISTORY_SECONDS (default
600 seconds) of HLS segments in data/hls. Older segments are deleted by
ffmpeg, and any pre-existing segments/playlist are removed at startup. The
camera page can seek within the playlist's rolling window.
Firmware
The ESP32-S3 firmware is flashed separately:
cd crates/firmware
cargo build --release
cargo run --release # espflash flash --monitor
Full prerequisites, wiring, pin map and the UART protocol are in
crates/firmware/README.md.
Raspberry Pi 5
The complete Pi 5 guide is docs/pi5-setup.md. It covers:
- Camera Module →
rpicam-vid→mediamtx→ RTSP. - Pi GPIO14/15 ↔ ESP32 GPIO6/7 UART wiring.
- Enabling the Pi UART and freeing the serial console.
- Models, enrollments and the vision service.
- Systemd units for mediamtx, the camera publisher, the vision service and the dashboard.
- Verification and troubleshooting.
Configuration
All dashboard settings are environment variables; the complete table is in
crates/dashboard/README.md. The most important:
export DATABASE=data/door.db
export HLS_DIR=data/hls
export VIDEO_HISTORY_SECONDS=600
export CAMERA_RTSP_URL='rtsp://127.0.0.1:8554/cam'
export DEVICE_TOKEN='change-me'
# Physical lock output: gpio (Pi relay), uart (ESP32 servo), or none.
export LOCK_TRANSPORT=uart
export SERIAL_PORT=/dev/serial0
export SERIAL_BAUD=115200
export UART_SHARED_KEY='<same 64-hex key as the firmware>'
# Face-triggered unlock cooldown.
export VISION_UNLOCK_COOLDOWN_SECONDS=30
# Optional Prometheus token; not needed for the in-dashboard analytics.
export METRICS_TOKEN=''
Never commit .env files, RTSP credentials, data/*.db, data/vision/*, or
HLS output.
Device / firmware API
curl -X POST http://localhost:8000/api/auth/pin -H 'Content-Type: application/json' -d '{"pin":"1234"}'
curl -X POST http://localhost:8000/api/auth/rfid -H 'Content-Type: application/json' -d '{"uid":"A1B2C3D4"}'
curl -X POST http://localhost:8000/api/auth/face -H 'Content-Type: application/json' -d '{"label":"jane_doe"}'
The vision service uses the stricter POST /api/vision/result endpoint, which
also carries passive anti-spoofing and active-liveness results.
Deployment
nix-shell --run 'cargo build --release -p smarty-dashboard -p vision-service'
sudo cp crates/dashboard/smarty-dashboard.service /etc/systemd/system/
sudo cp crates/vision-service/smarty-vision.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now smarty-dashboard smarty-vision
The units assume /home/pi/smarty, keep secrets in /etc/smarty.env, and use a
single process for the dashboard because the lock transport and ffmpeg child are
process-wide singletons.
Diagrams and CAD
The diagrams/ directory contains D2 sources and rendered PNGs for the power
and hardware design. Re-render with
d2 diagrams/<name>.d2 diagrams/<name>.png
cad/ is reserved for OpenSCAD sources. It is intentionally empty for now.