Autonomous driving
Data engines, on-device ADAS, HD lane mapping and from-scratch visual-inertial SLAM, tuned for unstructured Indian traffic.
- Auto-labeling
- ODD scoring
- HD maps
- VIO / GNSS
- MCAP
Perception · Autonomy · Edge
Autonomous systems that run on the device, on Indian roads.
Computer vision, on-device autonomy and vehicle reverse engineering. Hardware-synced sensor rigs and the data engines they feed, ADAS and visual-inertial SLAM that ship to real hardware, plus CAN bus tooling for the vehicles nobody published a DBC for.
01 / Focus
Four things, done properly
Everything below ships to a phone, a Jetson, or a car, not to a notebook.
Computer vision
Detection, tracking, re-identification, ANPR and vision-language incident verification, running locally on one GPU.
- YOLOv8
- ByteTrack
- ReID
- ANPR
- VLM
Reverse engineering
CAN and CAN-FD capture, signal boundary and checksum analysis, empirically proven DBC derivation, EV telematics.
- SocketCAN
- DBC
- UDS / ISO-TP
- OBD-II
Safety intelligence
The Safety Ecosystem Framework, multi-agent road-risk monitoring, and autonomous emergency-response orchestration.
- Multi-agent
- Geospatial
- Sensor fusion
- SSRN
02 / Data collection
Before the model, the rig
Autonomy is a data problem long before it is a modelling problem. This is the layer everything above it inherits.
A perception stack is only ever as honest as the data under it, and on a vehicle that means solving unglamorous problems first: getting many camera heads onto one host without melting the bandwidth budget, getting a position fix you can actually trust, getting an inertial reference good enough to judge everything else against, and getting all of it onto one clock so that a frame, a pose and a CAN message recorded at the same instant can be proven to belong together.
Cameras
GMSL2
Automotive serialized video over a single coax, so multiple camera heads run at real cable lengths without the bandwidth and EMI problems that USB and raw MIPI hit on a vehicle. Deserializer boards aggregate the array onto one capture host with a shared frame trigger.
Positioning
GNSS RTK
Real-time kinematic fixes from NTRIP correction streams, with the raw observables (pseudorange, carrier phase, Doppler) logged alongside. That keeps post-processed kinematic solutions available later, and lets GNSS enter a factor graph as a measurement rather than as a black-box fix.
Inertial
Xsens IMU
Industrial MTi-series inertial measurement with characterised bias stability and factory calibration. It is both the source of preintegrated motion constraints between camera keyframes and the reference that phone-grade IMUs get evaluated against.
Vehicle
CAN integration
Speed, steering angle, wheel ticks, gear and brake state taken off the bus and stamped into the same log as the sensors. On vehicles with no public signal definitions, the DBC is derived and validated before anything downstream is allowed to depend on it.
Timing
Hardware sync
One clock domain. GNSS PPS disciplines the host, camera exposure is triggered from a common source, and every sample carries a timestamp on the same monotonic base. Alignment becomes a property of the rig instead of a post-processing guess.
Logging
MCAP / Foxglove
Sessions land as standards-compliant MCAP that opens in Foxglove with zero manual offset, replays through the exact live pipeline, and carries its calibration and metadata with it. Recordings stay useful years after the drive.
The pathway to self-driving
01
Capture
A hardware-synchronised multi-sensor rig producing recordings whose timing you can defend. Phones become acquisition devices where fleet scale matters more than sensor grade.
02
Ground truth
Calibration, telemetry extraction and time alignment turn raw recordings into a dataset. Domain-gap evaluation says how far the footage sits from what a model was trained on.
03
Label and govern
Machine labels pass a calibrated confidence gate into human review, rare and risky moments are mined out of the rest, and the models retrain in a closed loop over one governed ontology.
04
Perceive on the device
Detection, drivable-area segmentation and tracking running inside the power and thermal envelope of the target hardware, not on a workstation.
05
Localise and map
A pose you own rather than one a vendor SDK hands you: visual features, IMU preintegration and raw GNSS fused in a factor graph, feeding lane-level HD maps in standard formats.
06
Certify the ODD
Score the road, and score your own ability to see it. Per corridor, per hour, per weather condition, which stretches are viable for vision-based autonomy and which dimension fails where.
07
Reach the vehicle
Nothing actuates until the bus is understood. Signal definitions derived and proven from capture, then diagnostics and integration on top of a DBC that has been validated rather than assumed.
03 / Selected work
28 systems
Autonomous driving, perception, vehicle reverse engineering, safety and agents. Filter by domain.
AV
LabeloxAV
A data engine for autonomous driving, built for Indian roads. Raw fleet footage is machine-labeled through a three-path fusion pipeline, every label passes a confidence gate into human review, rare and risky moments are mined, and the models retrain themselves in a closed loop. One ontology, 178 governed classes: autorickshaws, cattle on the carriageway, overloaded two-wheelers, hand carts, potholes.
- Python
- Docker
- Active learning
- HD maps
AV
Zyra ADAS
An L2 shadow-mode ADAS on an Android phone. YOLOv8n object detection and classical lane tracking run on Vulkan-accelerated NCNN inference behind a Flutter UI over a C++ NDK engine. It watches the road and predicts what a real L2 system would do, side by side with what you actually do. Perception without vehicle control.
- Dart / Flutter
- C++17
- NCNN
- Vulkan
AV
Lyne ADAS
A warning-only driver assistance system that auto-tunes its own workload from a 2 GB budget phone to an NPU flagship. Forward-collision warning, real drivable-area segmentation, vulnerable-road-user detection, headway monitoring and GTSRB traffic-sign recognition, with every model on-device and nothing leaving the phone.
- Kotlin
- Jetpack Compose
- LiteRT
- CameraX
AV
ODYX
From-scratch visual-inertial SLAM with GNSS fusion, computed entirely on the device. A GTSAM fixed-lag smoother fuses visual features, IMU preintegration and raw multi-constellation GNSS into a globally georeferenced, drift-corrected 6-DoF pose. It exposes a PoseProvider interface so other apps drop it in instead of ARCore.
- C++
- GTSAM
- RTKLIB
- DBoW3
- NDK
AV
CORRIDYX
Not "can the car drive here" but "can a camera even see here, right now, in this weather, at this hour". Every 150 m of road a fleet phone touches gets scored on lane visibility, GNSS quality, illumination, agent chaos and roughness, plus GLARYX, a four-head perception-weather watchdog that catches a smudged lens, sun glare, rain and night grain before they silently break the detector.
- Kotlin
- LiteRT
- ODD
- On-device
AV
LANYX
Live top-down lane mapping from a dash-mounted phone. Lane markings are detected in the camera feed, lifted onto the ground plane with ARCore's tracked pose, accumulated into a memory-bounded density grid, then vectorized into real lane geometry (Zhang-Suen thinning, skeleton graph trace, Douglas-Peucker) and exported as GeoJSON, OpenDRIVE, Lanelet2 or KML.
- Kotlin
- ARCore
- LiteRT
- OpenDRIVE
AV
Blurabbit HD-MAP
Turns any Android phone into an HD mapping device for self-driving cars. On-device camera, GNSS and IMU fuse into lane-level HD maps with OpenDRIVE, Lanelet2 and GeoJSON export. Crowdsourced mapping without a survey vehicle.
- Kotlin
- SLAM
- Lanelet2
- Crowdsourcing
AV
CHRONYX-CORE
A time-aligned multi-sensor capture kernel for Android. Camera, IMU, GNSS and audio are stamped on one BOOTTIME clock and written straight to an MCAP file that opens in Foxglove with zero manual offset, the unglamorous problem that ruins most phone-collected robotics datasets.
- Kotlin
- Camera2
- MCAP
- Foxglove
AV
DriveLogger
Android phones as autonomous-vehicle data-acquisition devices. Synchronized multi-sensor recording exported as standards-compliant MCAP, built for the autonomous data collection community.
- Kotlin
- MCAP
- Sensor sync
AV
Orvex
The full lifecycle of multi-sensor data, from raw GoPro / Insta360 / Android recordings through telemetry extraction, calibration, synchronization, dataset assembly, auto-labeling, training, 3D reconstruction, SLAM validation and edge deployment. Built for Indian road conditions.
- Python
- Calibration
- Depth
- Edge deploy
AV
RabbitAV
A windshield-mounted Android phone becomes two fused systems for Indian roads. On-device ADAS gives forward collision warning, headway monitoring and vulnerable-road-user alerts with every inference on the phone. Alongside it, an IMU and vision hazard mapper: the accelerometer feels the jolt, a camera lookback frame classifies pothole against speed breaker, GPS pins it, and hazards accumulate into a local map that warns you on the next approach. Floor device class is 3-4 GB RAM with CPU-only INT8 inference.
- Kotlin
- YOLOv8n INT8
- IMU fusion
- On-device
AV
DriveLab
A desktop workbench for dashcam footage and the front door to the data engine: GPS telemetry extraction, domain-gap evaluation against Indian roads, annotation and fine-tuning, and privacy-blurred Mapillary export. Part of LabeloxAV.
- Python
- PyQt6
- YOLO
- Dataset tools
AV
EasySLAM
A production-ready Python SLAM package that stays approachable for beginners without capping what researchers can do with it. ORB-SLAM, FastSLAM, GraphSLAM, visual-inertial and RGB-D backends behind one API, with webcam, RealSense, stereo, LiDAR and TUM/KITTI dataset inputs, semantic mapping, EKF sensor fusion, map merging and a PyQt6 visualizer. Published on PyPI.
- Python
- Open3D
- PyQt6
- PyPI
CV
Sentigon
An agentic security operations center that runs entirely on your own hardware. Live camera feeds pass through a fleet of 12 cooperating agents (perception, reasoning, action, supervision) that detect, verify, reason about and respond to physical-security threats in real time. Every model is local: no cloud, no API key, no telemetry.
- TypeScript
- Next.js
- FastAPI
- Ollama
CV
SentigonEdge
The same stack condensed onto a single NVIDIA Jetson AGX Orin. RTSP ingest with a pre-roll ring buffer so evidence exists from before an incident fired, TensorRT detection plus ByteTrack, ANPR and cross-camera ReID, stateful behaviour signatures, per-zone anomaly baselines learned online, and a local Qwen2.5-VL verdict on every high-value candidate. Nothing is required to leave the box.
- Python
- TensorRT
- Jetson Orin
- ByteTrack
- VLM
CV
SentigonV2
The workstation build of the same intelligence layer: a local-first monorepo where the whole stack runs on a single machine with one GPU, and the heavier vision-language tier can be offloaded to a rented cloud GPU through a config switch, without touching application code.
- Python
- Next.js
- ByteTrack
- Event bus
CV
VisionAI Aegis
Enterprise-grade video surveillance and analytics: real-time intelligent video analysis with face recognition, license-plate recognition, anomaly detection and multi-camera tracking, all orchestrated through a web operator dashboard.
- Python
- OpenCV
- GPU
- Dashboard
CV
Incident Lens AI
A forensic video analysis suite that transforms raw crash footage into a defensible legal case file in seconds: reconstruction, evidence extraction and structured reporting from ordinary dashcam and CCTV video.
- TypeScript
- Forensics
- VLM
CV
Halox Traffic
On-device Indian traffic-violation detection and ANPR with tamper-evident evidence collection. Offline-first, device-tiered so it degrades gracefully on weaker hardware, and field-tested outdoors rather than on a benchmark clip.
- Kotlin
- ONNX
- ANPR
- On-device
CV
Advanced rPPG
Contactless heart-rate and heart-rate-variability measurement from ordinary video. MediaPipe face tracking feeds three rPPG algorithms (chrominance, POS and green-channel) with time-domain, frequency-domain and non-linear HRV analysis, plus a live signal-quality assessment that tells you when not to trust the reading.
- Python
- MediaPipe
- PyQt5
- DSP
RE
CanLab
A CAN bus reverse-engineering workstation. Load a capture, inspect frames and signals, run offline analysis to find counters, checksums and signal boundaries, get AI help interpreting an unknown ID, then build and export a DBC. Includes UDS, ISO-TP, J1939, OBD-II, XCP and DoIP diagnostics. For isolated bench use only, it adds injection, replay, fuzzing and a MitM gateway.
- Python 3.11
- PyQt6
- SocketCAN
- DBC
- UDS
RE
BluCANRE
CAN reverse engineering of the Tata Tigor EV / Xpres-T EV (Ziptron, ~26 kWh). The 94-cell BMS voltage map is empirically proven across 121,329 cell frames from a 32-minute capture: every cell in 3275-3283 mV, pack sum steady at 308.12-308.49 V, three independent decoders in agreement. Everything not derivable from the data is documented as not derivable, and the whole platform is gated behind a validated DBC.
- Python
- opendbc
- BMS
- Telematics
RE
Drivox CAN-RE
The CANalyst-II is a proprietary ControlCAN USB device that SavvyCAN cannot open. Rather than bridging with a second process fighting over one USB handle, Drivox has one process own the adapter and fan the single stream out to a live TUI with passenger hotkey event tagging, a CSV, and virtual SocketCAN interfaces, so SavvyCAN watches the same synchronized session.
- Python
- ControlCAN
- vcan
- SavvyCAN
RE
canmcpd
A proactive CAN daemon with an MCP interface. It owns the adapter, captures continuously and runs standing inference over the bus (per-ID cadence model, novelty detection, optional decoded-signal checks), then exposes live state and history to any MCP client over Streamable HTTP. Anomalies accrue whether or not an agent is attached, so the answer comes from continuous monitoring rather than a cold query.
- Python
- FastMCP
- python-can
- asyncio
Safety
SEF: Safety Ecosystem Framework
An architecture for ambient safety intelligence that unifies smartphones, CCTV networks, IoT sensors and vehicle telematics into one ecosystem for proactive threat detection and emergency-response coordination. Published on SSRN and Zenodo.
- Research
- Distributed
- IoT
- SSRN
Safety
SurakshaNet
26 specialized agents in three operational clusters continuously analyse weather, traffic flow, accident history, live camera feeds, flood risk and hospital capacity, then fuse their assessments through Bayesian belief networks and Byzantine fault-tolerant voting into a single calibrated risk score per road segment in Kerala. Elevated risk triggers causal attribution and dispatch.
- Python
- Multi-agent
- Geospatial
- Bayesian
Safety
GoldenHour Engine
An autonomous emergency-response orchestration platform for road accidents, the coordination layer between a detected incident and the resources that have to reach it inside the golden hour.
- Python
- Agents
- Emergency response
Agents
deepgent
A domain-locked autonomous engineering agent for autonomous vehicles, computer vision and embedded systems. It takes a task from natural language to a verified artifact running on target hardware at spec. AV, CV, embedded and robotics-adjacent edge AI only, and "compiles and tests pass" is an intermediate state, never completion. Built on the Claude Agent SDK, with a dense Blender-style desktop GUI over twelve surfaces.
- Python
- Claude Agent SDK
- qasync
- MCP
No systems in this category.
04 / Try it
Six apps you can install today
Mount a phone on the windshield and drive. Everything here runs on the device, offline, with no account and no telemetry.
Before you install
These are test builds distributed outside the Play Store, so Android will ask you to allow installs from this source the first time. They are driver assistance and mapping tools for evaluation: they warn, they record, they map. None of them control a vehicle. Keep your eyes on the road and treat every alert as advisory.
Androidv1.2.0
Lyne ADAS
Forward collision warning, drivable-area segmentation, vulnerable-road-user alerts, headway monitoring and traffic-sign recognition. Warning only, entirely on-device, works offline.
Androidv0.1.0-phase10
Zyra ADAS
L2 shadow-mode ADAS. YOLOv8n detection and lane tracking on Vulkan-accelerated NCNN. It predicts what a real L2 system would do, side by side with what you actually do. No vehicle control.
Androidv0.1.0
ODYX
From-scratch visual-inertial SLAM with GNSS fusion. A GTSAM factor graph over camera, 200 Hz IMU and raw multi-constellation GNSS, computing georeferenced 6-DoF pose without ARCore.
Androidv0.1.0
LANYX
Live top-down lane mapping from a dash-mounted phone. Detected markings are lifted onto the ground plane, accumulated into a map, vectorized into lane geometry, and exported as GeoJSON, OpenDRIVE, Lanelet2 or KML.
Androidv1.0.0
RabbitAV
On-device ADAS plus an IMU and vision road hazard mapper. The accelerometer feels the jolt, a camera lookback frame classifies pothole against speed breaker, GPS pins it, and hazards warn you on the next approach.
Androidv1.0.0
Blurabbit HD-MAP
Turns any Android phone into an HD mapping device. On-device camera, GNSS and IMU fuse into lane-level HD maps with OpenDRIVE, Lanelet2 and GeoJSON export.
Debug build. Larger and slower than a release build.
Contributing
Field reports are the most useful thing you can send: road, device, weather, time of day, and what the system got wrong. Open an issue on the relevant repository with a clip or a log and it goes into the next round of mined data. Pull requests are welcome on all of them.
05 / Research
Three research threads
One published paper, and two lines of applied research running in the field.
Scholarly article
Ambient Safety Intelligence: A Framework for Proactive Public Safety Through Distributed Multi-Source Data Correlation
Current public safety systems operate on fundamentally reactive principles, activating only after incidents occur and relying on fragmented infrastructure that fails to leverage the extensive network of connected devices now ubiquitous in modern urban environments. This paper presents the Safety Ecosystem Framework, a novel architecture for ambient safety intelligence that unifies heterogeneous data sources (smartphones, closed-circuit television networks, Internet of Things sensors, and vehicle telematics) into a cohesive ecosystem for proactive threat detection and emergency response coordination.
Perception that knows when it cannot see
Autonomy research aimed at roads that published datasets never covered. Three questions drive it. What does an ontology for unstructured Indian traffic actually need, and how do you govern 178 classes without the labels drifting? Can a fleet of dash-mounted phones certify where vision-based autonomy is viable, per 150 m of road, per hour of day, per weather condition? And can a perception stack score its own ability to perceive, so a smudged lens is reported as a smudged lens rather than as a road with no lane markings?
The last one matters most. A stack that silently degrades is more dangerous than one that stops. GLARYX runs four heads every frame, on lens contamination, glare, atmospherics and night quality, and publishes a single trust value any sibling app on the device can read before deciding whether to believe its own detections.
A DBC you can prove, not one you guessed
Manufacturers keep CAN signal definitions proprietary, so anyone working on Indian EVs starts from an empty grid. The research question is not "can a signal be decoded" but "what standard of evidence should a decoded signal have to meet before anything is built on top of it".
The working answer is a validated DBC gate. On the Tata Tigor EV, the 94-cell BMS voltage map holds across 121,329 cell frames from a 32-minute capture, with every cell inside 3275-3283 mV, pack sum steady at 308.12-308.49 V, and three independent decoders in agreement. Everything not derivable from the capture is documented as not derivable rather than inferred, and a self-test proves the validator rejects injected defects. Cell-delta trending on that one message set is what predicts pack failure weeks ahead.
06 / Field notes
Systems, running
Recorded walkthroughs: real hardware, real roads, unedited failure modes included.
07 / About
Sherin Joseph Roy
Bangalore, India. Co-founder at DeepMost AI.
I work on computer vision, autonomous systems and AI-driven safety intelligence, and specifically on making them work in the field rather than in a demo. Most of what I build runs on the edge: an Android phone on a windshield, a Jetson Orin in a cabinet, a laptop wired to a CAN bus.
A lot of that work is aimed at conditions global datasets never saw. Autorickshaws, cattle on the carriageway, overloaded two-wheelers, unlit roads in monsoon. The honest version of the problem is not "can the model detect a car" but "can the camera even see, right now, in this weather, at this hour", which is why several of these projects score their own ability to perceive before they score the world.
The rest is vehicles from the inside out: reverse-engineering CAN buses to derive DBCs that are empirically proven rather than guessed, and building the capture and validation tooling that proof requires. Alongside it, the Safety Ecosystem Framework, published research on correlating smartphones, CCTV, IoT sensors and vehicle telematics into proactive public safety.
Rig
- GMSL2 cameras
- GNSS RTK / NTRIP
- Xsens MTi IMU
- PPS hardware sync
- MCAP / Foxglove
Perception
- YOLOv8 / RT-DETR
- ByteTrack
- ArcFace / ReID
- OpenCV
- PyTorch
Edge
- TensorRT
- NCNN / Vulkan
- LiteRT / TFLite
- ONNX
- Jetson Orin
Localization
- GTSAM
- RTKLIB
- ARCore
- MCAP / Foxglove
- Lanelet2 / OpenDRIVE
Vehicle
- SocketCAN
- python-can
- opendbc
- UDS / ISO-TP
- J1939 / OBD-II
Systems
- Python
- Kotlin
- C++
- Rust
- FastAPI / Next.js
08 / Contact
Get in touch
Open to collaboration on autonomy, perception and vehicle systems.