To start with, we worked on developing an AI that could do taxi missions in GTA V (follow a map to drive from point A to point B). This AI worked using only the information displayed on screen while playing the game (it was fed screen frames as input data and outputted a keyboard key to press: W, A, S or D). We took what we learned from the GTA V simulation and used it to design a prototype car. We now had 8 cameras installed on the car that generated a full 360-degree image of the surroundings instead of the single camera that was being simulated in GTA V. We also added a bunch of Ultrasonic ranging sensors so that the system could detect the distance to the nearest obstacles and thus filter out most of the wrong instructions generated by the AI which yield a significant improvement. For the motors as they aren’t our primary concern we used 2 “hoverboards” that we reverse engineered so that we could control each of the wheels individually. The car has a max speed of around 20km/h. The whole thing is powered by a 1kWh 12V lithium ion battery that I built from smaller 18650 cells. We then gathered training data for the AI. As this isn’t a simulation and a crash would be disastrous our first goal was letting the car autonomously perform a whole lap on a karting track without hitting any obstacles or getting off the road.