Virtual Reality Based Simulator for Evaluation of Autonomous Vehicles
Motivation
Successful deployment of autonomous vehicles (AV) on public roads requires a thorough evaluation of the autonomous driving algorithms in various scenarios. Physical training, validation and evaluation of the vehicles for such scenarios is necessary, expensive and risky to realize, specifically the ones involving multiple vehicles, objects or pedestrians. Virtual Reality based simulation testing platforms can allow for complex scenarios recreation and evaluation of AV behavior algorithms in a safe, inexpensive and repeatable manner and provide a testbed for human factors studies and assessment of vehicles.
The VR testing platform described in this research can significantly speed the rate of technology development by realizing a platform that can provide accurate depiction of an actual roadway test scenario to one or more system components under evaluation. Further, the platform allows human-in-the-loop feedback that could potentially provide invaluable system design feedback, without which years of transportation trials may be required to identify unexpected design flaws.
VR Simulator Testbed Architecture
The Virtual Reality Simulation testbed architecture has various system components that are seamlessly integrated via various network communication technologies to operate collectively as a real-time high fidelity simulation system:
The Control Module (CM) developed by co-simulating Carsim and Matlab/Simulink implements the collection of AV behavior algorithms and vehicle dynamics.
The Visualization Module (VM), developed in Unity 3D, comprises of the computer-generated environment hosting a simulated AV and supports immersive visualization of the Control Module’s response for the autonomous vehicle in Virtual Reality. An NVIDIA GeForce GTX 1080 renders the Virtual Reality environment while guaranteeing real time performance for the users who experience the simulator through the use of a stereoscopic Head Mounted Display (HMD). The users get to perceive the autonomous vehicle from the perspective that they would have if they were actually riding in the vehicle.
The Clemson Connected Vehicle Testbed (C-CVT) deployed at Clemson University’s main campus is used to facilitate the transmission of a real world vehicle’s data to the simulation testbed’s core modules. The C-CVT is a Dedicated Short Range Communication (DSRC) infrastructure setup that allows for Vehicle to Vehicle (V2V) and Vehicle to Infrastructure (V2I) communication capabilities. The DSRC infrastructure of C-CVT comprises a set of Road Side Units (Fixed Edge Nodes) within the university’s campus and On Board Units (OBU) which are placed inside vehicles (Mobile Edge Nodes) and facilitate communication with Fixed Edge Nodes using DSRC technology. OBU also supplies the required GPS location and estimated speed data for a vehicle as it is driven on the road.
Data collected from the real world vehicle is published to a Communication System Node running an MQTT broker. MQTT is a lightweight publish-subscribe based messaging protocol. The VM subscribes to the MQTT broker to receive the vehicle’s published information. This allows real-time data communication from the vehicle to the simulation platform with low latency.
Virtual Environment Design
The virtual environment for this application is a one-to-one representation of the Clemson University’s campus. The road network and the models of the buildings has been laid out in such a way that they are co-located with the real version. Figure 2 depicts the Clemson University Football Stadium and the neighboring buildings and roads.
Case Study
The simulator aims at testing and evaluating autonomous algorithms and human experience of being a typical passenger of an autonomous vehicle.
The case study consists in a virtual autonomous-driven vehicle (virtual-vehicle), and one human-driven real vehicle (real-vehicle). The driver in the real-vehicle can see in real time the behavior of the virtual-vehicle and thus adjust its maneuver depending on it. This interaction between the driver of the real-vehicle and the virtual-vehicle is created using tablet devices mounted on the cars’ dashboards and will be referred to as vehicle-in-the-loop. There can be two different cases, either the real-vehicle precedes the virtual-vehicle, or it follows it. In the first case, the tablet will act as a rear-view mirror and will show the virtual-vehicle which follows the real-vehicle according to some autonomous algorithms. In the second instead, the tablet will show the virtual-vehicle from the front view of the real-vehicle. The two scenarios are represented in the figure on the right.
Overview video