Missing-person detection,
in real time.
A vision system that detects and tracks missing persons across video, webcam, and RTSP streams. YOLOv8 finds people, ArcFace embeds faces, ByteTrack maintains identity, and a three-state confidence-accumulation machine eliminates false positives before an alert is ever raised.
The brief
Find a person in a crowd — without crying wolf.
Naive face recognition fires on a single high-similarity frame, which makes it noisy and untrusted in real deployments. The goal was a system whose alerts could be acted on: identity confirmed across multiple frames, locked once confirmed, and re-verified continuously without flickering between candidates.
Confidence-accumulation tracker
ArcFace embeddings per identity
Periodic re-verification interval
Bbox-lost timeout before drop
Architecture
Six stages from pixel to alert.
- STAGE 011
YOLOv8 — person detection
Each frame is passed through YOLOv8 to detect every person in view. Detections become candidates for the rest of the pipeline.
- STAGE 022
InsightFace ArcFace — 512-d face embedding
Each cropped person is searched for faces with InsightFace. Detected faces are embedded into 512-dimensional vectors using ArcFace ResNet100.
- STAGE 033
Cosine similarity — match against database
Embeddings are compared against the missing-person database via cosine similarity. Per-track scores feed the state machine, not raw alerts.
- STAGE 044
ByteTrack + state machine
ByteTrack keeps frame-to-frame identity association via Kalman + IoU. Scores accumulate inside a three-state machine before any alert is raised.
- STAGE 055
Locked tracking — bbox + appearance + velocity
Once locked, identity is frozen. Tracking continues via bbox IoU, HSV body-appearance EMA, and constant-velocity prediction during occlusion.
- STAGE 066
Periodic re-verification
Every 30 frames the lock is re-checked. Three consecutive failures or a 2-second bbox-lost timeout drop the track back to IDLE.
Tracking state machine
A single high-similarity frame should not raise an alarm.
No candidate. Every detection is screened — but nothing is acted on.
Similarity above threshold for 2+ consecutive frames. Evidence accumulates without raising an alert.
Weighted score confirmed across multiple frames — identity frozen, alert fired, persistent tracker engages.
Transition rules
- IDLE → WATCHINGsimilarity > 0.35 for 2 consecutive frames
- WATCHING → LOCKEDweighted score > 0.40 and at least 1 frame > 0.50
- WATCHING → IDLEno accumulation in 10 frames or 3 consecutive frames < 0.30
- LOCKED → IDLE (re-verify)similarity < 0.25 on 3 consecutive 30-frame re-checks
- LOCKED → IDLE (bbox lost)bounding box not seen for 2 seconds
Demos
Watch the lock earn its alert.
Database overview
Similarity distribution
Threshold tuning
Tech stack
Open-source, end-to-end.
Detection & Recognition
- YOLOv8 (Ultralytics)
- InsightFace
- ArcFace ResNet100
- ONNX Runtime
Tracking
- ByteTrack
- Supervision
- Kalman filter
- HSV histogram EMA
Runtime
- Python 3.11
- OpenCV
- NumPy
- Docker / Docker Compose
Inputs supported
- Video file
- Webcam
- RTSP stream
- MJPEG
Outcomes
Alerts you can act on.
False positives, eliminated by design
Single-frame spikes are rejected. Identity is only locked after weighted score across multiple frames passes the bar.
Identity that doesn't flicker
Frozen-label soft lock with periodic re-verification — no jitter between similar candidates, no labels swapping mid-frame.
Robust through occlusion
Bbox + body appearance + velocity prediction keeps the track alive when the face turns away or is briefly hidden.
Deploys anywhere
Runs on a video file, a webcam, or an RTSP stream — packaged in Docker, with a docker-compose for detection and training.