One of the main challenges in autonomous racing is to design algorithms for motion planning at high speed, and across complex racing courses. End-to-end trajectory synthesis has been previously proposed where the trajectory for the ego vehicle is computed based on camera images from the racecar. This is done in a supervised learning setting using behavioral cloning techniques. In this paper, we address the limitations of behavioral cloning methods for trajectory synthesis by introducing Differential Bayesian Filtering (DBF), which uses probabilistic Bézier curves as a basis for inferring optimal autonomous racing trajectories based on Bayesian inference. We introduce a trajectory sampling mechanism and combine it with a filtering process which is able to push the car to its physical driving limits. The performance of DBF is evaluated on the DeepRacing Formula One simulation environment and compared with several other trajectory synthesis approaches as well as human driving performance. DBF achieves the fastest lap time, and the fastest speed, by pushing the racecar closer to its limits of control while always remaining inside track bounds.

Multi-agent autonomous racing still remains a largely unsolved research challenge. The high-speed and close proximity situations that arise in multi-agent autonomous racing present an ideal condition to design algorithms which trade off aggressive overtaking maneuvers and minimize the risk of collision with the opponent. In this article we study a two-vehicle autonomous racing setup and present AutoPass—a novel framework for overtaking in a multi-agent setting. AutoPass uses the structure of an automaton to break down the complex task of overtaking into sub-maneuvers that balance overtaking likelihood and risk with safety of the ego vehicle. We present real-world implementation of 1/10-scale autonomous racing cars to demonstrate the effectiveness of AutoPass for the overtaking task. Our results indicate that the overtake success ratio for the AutoPass framework is 0.395, or 23 times more likely, compared to a purely reactive system at 0.017, while traditional Robot Operating System (ROS)-based path planners (depending on the navigation plugin used) are placed between 0.115 and 0.286.

This paper presents an adaptive lookahead pure-pursuit lateral controller for optimizing racing metrics such as lap time, average lap speed, and deviation from a reference trajectory in an autonomous racing scenario. We propose a greedy algorithm to compute and assign optimal lookahead distances for the pure-pursuit controller for each waypoint on a reference trajectory for improving the race metrics. We use a ROS based autonomous racing simulator to evaluate the adaptive pure-pursuit algorithm and compare our method with several other pure-pursuit based lateral controllers. We also demonstrate our approach on a scaled real testbed using a F1/10 autonomous racecar. Our method results in a significant improvement (20%) in the racing metrics for an autonomous racecar.

A popular metric for measuring progress in autonomous driving has been the "miles per intervention". This is nowhere near a sufficient metric and it does not allow for a fair comparison between the capabilities of two autonomous vehicles (AVs). In this paper we propose Scenario2Vector - a Scenario Description Language (SDL) based embedding for traffic situations that allows us to automatically search for similar traffic situations from large AV data-sets. Our SDL embedding distills a traffic situation experienced by an AV into its canonical components - actors, actions, and the traffic scene. We can then use this embedding to evaluate similarity of different traffic situations in vector space. We have also created a first of its kind, Traffic Scenario Similarity (TSS) dataset which contains human ranking annotations for the similarity between traffic scenarios. Using the TSS data, we compare our SDL embedding -with textual caption based search methods such as Sentence2Vector. We find that Scenario2Vector outperforms Sentence2Vector by 13% ; and is a promising step towards enabling fair comparisons among AVs by inspecting how they perform in similar traffic situations. We hope that Scenario2Vector can have a similar impact to the AV community that Word2Vec/Sent2Vec have had in Natural Language Processing datasets.

Multi-agent autonomous racing is a challenging problem for autonomous vehicles due to the split-second, and complex decisions that vehicles must continuously make during a race. The presence of other agents on the track requires continuous monitoring of the ego vehicle’s surroundings, and necessitates predicting the behavior of other vehicles so the ego can quickly react to a changing environment with informed decisions. In our previous work we have developed the DeepRacing AI framework for autonomous formula one racing. Our DeepRacing framework was the first implementation to use the highly photorealisitc Formula One game as a simulation testbed for autonomous racing. We have successfully demonstrated single agent high speed autonomous racing using Bezier curve trajectories. In this paper, we extend the capabilities of the DeepRacing framework towards multi-agent autonomous racing. To do so, we first develop and learn a virtual camera model from game data that the user can configure to emulate the presence of a camera sensor on the vehicle. Next we propose and train a deep recurrent neural network that can predict the future poses of opponent agents in the field of view of the virtual camera using vehicles position, velocity, and heading data with respect to the ego vehicle racecar. We demonstrate early promising results for both these contributions in the game. These added features will extend the DeepRacing framework to become more suitable for multi-agent autonomous racing algorithm development

Demonstrating high-speed autonomous racing can be considered as a grand challenge for vision based end-to-end deep learning models. DeepRacing AI is a novel end-to-end framework for trajectory synthesis for autonomous racing. We train and demonstrate the effectiveness of our approach using a high fidelity and photo-realistic Formula One gaming environment - used by real racing drivers. This is the first work that has used the highly realistic F1 game as a simulation environment for deep learning models. We present a novel method for single agent autonomous racing by training a deep neural network to predict a parameterized representation of a trajectory. Our Bezier curve based trajectory synthesis approach outperforms several other end-to-end DNN approaches for autonomous racing. In addition to evaluating our methodology in a closed-loop manner in the game; we also implement the DeepRacing algorithm on a 1/10 scale autonomous racing test-bed and show its ability to handle real-world data at high speeds.

This paper presents the ROS F1/10 simulator - a ROS and Gazebo based autonomous racing simulator designed to mirror the behaviour and performance of the physical F1/10 platform. The simulator can be used to verify the performance of autonomous racing algorithms before testing on the real F1/10 racecar. The simulator supports most major ROS SLAM and navigation packages with tutorials on how to get started available on www.f1tenth.dev under open-source license and we continue to add support for more algorithms and features. The paper presents details on the simulator architecture, design, and features and presents several research use-cases including an example of developing in the sim and transferring the performance to the race racecar.

The kind of closed-loop verification likely to be required for autonomous vehicle (AV) safety testing is beyond the reach of traditional test methodologies and discrete verification. Validation puts the autonomous vehicle system to the test in scenarios or situations that the system would likely encounter in everyday driving after its release. These scenarios can either be controlled directly in a physical (closed-course proving ground) or virtual (simulation of predefined scenarios) environment, or they can arise spontaneously during operation in the real world (open-road testing or simulation of randomly generated scenarios). In AV testing, simulation serves primarily two purposes: to assist the development of a robust autonomous vehicle and to test and validate the AV before release. A challenge arises from the sheer number of scenario variations that can be constructed from each of the above sources due to the high number of variables involved (most of which are continuous). Even with continuous variables discretized, the possible number of combinations becomes practically infeasible to test. To overcome this challenge we propose using reinforcement learning (RL) to generate failure examples and unexpected traffic situations for the AV software implementation. Although reinforcement learning algorithms have achieved notable results in games and some robotic manipulations, this technique has not been widely scaled up to the more challenging real world applications like autonomous driving.

We consider the challenging problem of high speed autonomous racing in a realistic Formula One environment. DeepRacing is a novel end-to-end framework, and a virtual testbed for training and evaluating algorithms for autonomous racing. The virtual testbed is implemented using the realistic F1 series of video games, developed by Codemasters, which many Formula One drivers use for training. This virtual testbed is released under an open-source license both as a standalone C++ API and as a binding to the popular Robot Operating System 2 (ROS2) framework. This open-source API allows anyone to use the high fidelity physics and photo-realistic capabilities of the F1 game as a simulator, and without hacking any game engine code. We use this framework to evaluate several neural network methodologies for autonomous racing. Specifically, we consider several fully end-to-end models that directly predict steering and acceleration commands for an autonomous race car as well as a model that predicts a list of waypoints to follow in the car's local coordinate system, with the task of selecting a steering/throttle angle left to a classical control algorithm. We also present a novel method of autonomous racing by training a deep neural network to predict a parameterized representation of a trajectory rather than a list of waypoints. We evaluate these models performance in our open-source simulator and show that trajectory prediction far outperforms end-to-end driving. Additionally, we show that open-loop performance for an end-to-end model, i.e. root-mean-square error for a model's predicted control values, does not necessarily correlate with increased driving performance in the closed-loop sense, i.e. actual ability to race around a track. Finally, we show that our proposed model of parameterized trajectory prediction outperforms both end-to-end control and waypoint prediction.

We consider the challenging problem of high speed autonomous racing in realistic dynamic environments. DeepRacing is a novel end-to-end framework, and a virtual testbed for training and evaluating algorithms for autonomous racing. The virtual testbed is implemented using the realistic Formula One (F1) Codemasters game, which is used by many F1 drivers for training. We present AdmiralNet - a Convolution Neural Network (CNN) integrated with Long Short-Term Memory (LSTM) cells that can be tuned for the autonomous racing task in the highly realistic F1 game. We evaluate AdmiralNet's performance on unseen race tracks, and also evaluate the degree of transference between the simulation and the real world by implementing end-to-end racing on a physical 1/10 scale autonomous racecar.

In 2005 DARPA labeled the realization of viable autonomous vehicles (AVs) a grand challenge; a short time later the idea became a moonshot that could change the automotive industry. Today, the question of safety stands between reality and solved. Given the right platform the CPS community is poised to offer unique insights. However, testing the limits of safety and performance on real vehicles is costly and hazardous. The use of such vehicles is also outside the reach of most researchers and students. In this paper, we present F1/10: an open-source, affordable, and high-performance 1/10 scale autonomous vehicle testbed. The F1/10 testbed carries a full suite of sensors, perception, planning, control, and networking software stacks that are similar to full scale solutions. We demonstrate key examples of the research enabled by the F1/10 testbed, and how the platform can be used to augment research and education in autonomous systems, making autonomy more accessible.