Learning-based Bayesian Inference for Testing of Autonomous Systems

For safe operation of robotic systems, it is important to accurately understand its failure modes using prior testing. Hardware testing of robotic infrastructure is known to be extremely slow and costly. Instead, failure prediction in simulation can help in analyzing the system before deployment. Conventionally, large-scale na\"ive Monte Carlo simulations are used for testing; however, this method is only suitable for testing average system performance. For safety-critical systems, worst-case performance is more crucial as failures are often rare events, and the size of test batches increases substantially to discover rare occurrences of failures. Rare-event sampling methods can be helpful; however, they exhibit slow convergence and cannot handle constraints. This research introduces a novel sampling-based testing framework for autonomous systems which bridges these gaps by utilizing a discretized gradient-based second-order Langevin algorithm combined with learning-based techniques for constrained sampling of failure modes. Our method can predict more diverse failures by exploring the search space efficiently and ensures feasibility with respect to temporal and implicit constraints. We demonstrate the application of our testing methodology on two categories of testing problems, via simulations and hardware experiments. Our method discovers upto 2X failures compared to naive Random Walk sampling, with only half of the sample size.