Embedded Systems & Computer Vision

Driver monitoring & diagnostic lab

On-device driver-state monitoring developed during Donald’s research-engineering internship at Shanghai Ben’an Intelligent: eight detector classes, vehicle-aware alerts, and replay-driven validation.

2025 – Present · Remote

Shanghai Ben’an Intelligent · Research Engineer, Driver State Monitoring (DMS) · Internship

PythonMediaPipeOpenCVTemporal state machinesCalibrationReplay tooling
Why it sits here. An end-to-end internship engineering story: on-device deployment, context-aware alerts, calibration traceability, and a 616-test validation suite.
02 / Closer to the silicon. Closer to the world.STUDY IN SPACE

01 / IMPLEMENTATION & CONTRIBUTION

What the work involves

Deployed and validated real-time inference for eight safety-relevant detector classes. Designed the hierarchical alert state machine and vehicle-context gating, persisted per-camera calibration profiles, and built the diagnostic lab CLI.

Technical depth

Eye closure, yawning, distraction, phone use, smoking, face loss, lens occlusion, and eye anomaly detection; hierarchical face-loss and lens-occlusion fallbacks; speed, ignition, and gear gating; per-camera calibration; shadow-mode hard-case capture, human review, and replay-based metrics.

The project family

dms

02 / RESULTS

What came out of it

Validated on real cabin video and shipped a 616-test pytest suite with replay-driven regressions, inference-cadence checks, and per-detector coverage metrics in lockstep with model development.

03 / SUPPORTING EVIDENCE

Follow the source

Implementation notes, project records, and supporting artifacts.

Source context & project scope

Employment, on-device deployment, and cabin-video validation reflect Donald’s latest internship record. The 616-test suite is a project result, not a new website-run test.

The supplied record does not claim safety certification, verified CAN integration, or quantified fleet-wide sensitivity/specificity.

Deployed and validated real-time on-device driver-state monitoring, connecting safety-relevant alerts, low-power edge inference, and replay-driven validation. Deployed and validated eight safety-relevant detector classes: eye closure, yawning, distraction, phone use, smoking, face loss, lens occlusion, and eye anomaly. Shipped a 616-test pytest suite with replay-driven regressions, inference-cadence checks, and per-detector coverage metrics alongside model development. Designed the alert state machine with hierarchical face-loss and lens-occlusion fallbacks. Speed, ignition, and gear gating suppress false alarms in non-driving states; persisted per-camera calibration profiles support fleet-grade traceability. Fit the model pipeline to the low-power CV181x edge SoC by tightening inference cadence and per-stage compute budgets. Validated end-to-end on real cabin video and built the lab CLI for shadow-mode hard-case capture, human review, and replay-based metric reporting.

CANDIDATE · 2025 – Present

Architecture, eight detection states, calibration workflow, shared lab and historical test count.

SOURCE · 2026-09-17

FrameContext and VehicleContext explicitly carry source timestamps and motion-alert gating.

SOURCE · 2026-09-17

Secondary hand-inference scheduling and shared detector orchestration.

SOURCE · 2026-09-17

Documented smoking-latch, stale-obstruction and early-warning fixes.

SOURCE · 2026-09-17

Diagnostic replay implementation; sample data was not inspected.

SOURCE · 2026-09-17
CONTINUE IN EMBEDDED SYSTEMS & COMPUTER VISION

DMS on CV181x