LOCUS: Learning Safe, Generalizable Perception-based Hybrid Control with Certificates

Massachusetts Institute of Technology
Our system receives lidar sensor measurements and uses a hybrid control architecture to navigate towards the goal

Learning to recognize unsafe behaviors from observations lets us guarantee safety for robots.



Many robotic tasks require high-dimensional sensors such as cameras and Lidar to navigate complex environments, but developing certifiably safe feedback controllers around these sensors remains a challenging open problem, particularly when learning is involved.

Previous works have proved the safety of perception-feedback controllers by separating the perception and control subsystems and making strong assumptions on the abilities of the perception subsystem. In this work, we introduce a novel learning-enabled perception-feedback hybrid controller, where we use Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) to show the safety and liveness of a full-stack perception-feedback controller. We use neural networks to learn a CBF and CLF for the full-stack system directly in the observation space of the robot, without the need to assume a separate perception-based state estimator. Our hybrid controller, called LOCUS (Learning-enabled Observation-feedback Control Using Switching), can safely navigate unknown environments, consistently reach its goal, and generalizes safely to environments outside of the training dataset. We demonstrate LOCUS in experiments both in simulation and in hardware, where it successfully navigates a changing environment using feedback from a Lidar sensor.

Train in one environment...

The CLF and CBF are trained in a single environment

generalize safely...

The CLF and CBF are trained in a single environment

and transfer to hardware without retraining!

Related Links

This work is part of a broader research thread around learned certificates, which allow us to build data-driven proofs of controller correctness. For a survey of the field of learned certificates, see this paper.

Other work on learned certificates from our lab include:


author={Dawson, Charles and Lowenkamp, Bethany and Goff, Dylan and Fan, Chuchu},
journal={IEEE Robotics and Automation Letters},
title={Learning Safe, Generalizable Perception-Based Hybrid Control With Certificates},