Simulating success

Chris Hoyle, Technical Director for rFpro, explains to Alex Kreetzer the benefits of testing AI through simulation.

Chris Hoyle has been involved in programming since a teenager, creating free software as a hobby for other people on the internet. One day, some software that he created was picked up by a couple of motor racing teams; one of which was involved within Formula 1. Over the course of six months, this hobby quickly turned into a new career building high-end simulators for F1. This became an overwhelming success for Hoyle, who was soon approached by most of the F1 teams, along with a number of different racing divisions. In fact, the demand for the software became so high that he started running out of teams to sell to. Due to this, Hoyle decided to move into the road car business which, again, saw a huge demand - with customers using it to assess ride comfort and durability, rather than the track-focused simulation software. rFpro is now being used by most of the top OEMs, tier ones and engineering consultants. In 2015, the company was approached for the first time by a customer who wanted to use the simulation software for AI research. This wasn't an area that Hoyle and his team knew much about but, over two and a half years of research and development, they created the first official version.

You would expect the larger companies involved in the autonomous sector to take most of the business from the smaller companies. However, people are realising that simulation technology is more effective than physically testing self-driving vehicles. "The pioneers in this industry, people like Mobileye, Tesla and Nvidia, have built massive offshore teams of people who are manually enabling training data, “says Hoyle. “Human beings will annotate every frame of the video, which takes over half an hour. To get this video into your data through this process, you're looking at nine or ten years of work. Of course, with thousands of researchers working on these islands, it will be a lot shorter, but it is incredibly expensive, very slow and error-prone." By creating data without this time-consuming process, simulation software can significantly reduce the development time to milliseconds through an error-free process that ensures 100% accuracy.

Testing, testing

The are no incremental costs to using simulation software, as you will not be paying thousands of people over an extended period of time. However, Hoyle explains that if you wanted to do so, the software can even incorporate up to 50 humans into the virtual world to analyse different user-centric outcomes in a range of real-life situations. By doing so, rFpro creates a virtual world where customers can focus on other cars, pedestrians and public transport, which will prepare autonomous vehicles for real-world driving. "The best way to test autonomous vehicles is to incorporate the human aspect, because we have random and unpredictable behaviours that cannot be predicted,” he says. “By having autonomous models driving in sparsely populated cities with nice weather alongside other computer controlled road users which drive well and are easy to predict, you will learn almost nothing. So, we simulate autonomous vehicles in built-up areas like Manhattan which will have people spilling into the roads and congestion to push the limits of what is capable.”

This allows researchers to identify problems without putting people at risk. On public roads, you need to be testing for millions of miles to understand how everything will work. The only way of achieving this without taking nearly a decade to work through new data is by utilising simulation. This allows OEMs and suppliers to analyse the technology from a people-centric position. Although, Hoyle warns, the industry needs to take its time when testing to make autonomous cars as safe as possible. "When people ask me how long it will be till they see autonomous cars on the road, I tend to predict as far into the future as possible. Some people are saying that it is around the corner, but I do not know on what basis they are predicting this. We think that using this approach, the vendors are going to end up with databases of thousands of manoeuvres and scenarios which will all be built up over time.”

For example, there will be a continuous simulation of processes involved with difficult junctions, developing the models to make sure that any new functionalities break previous outcomes that were introduced. “At some point in the future, all of these tests will be passed regularly through simulation, but we will get to a stage where we can statistically prove that the rate at which we are identifying new failure modes makes the autonomous model safer by some factor than human drivers,” Hoyle continues. “Once you've reached this stage, the industry watchdogs will take a statistical sample of your database, verify it through their own test and then move into a public road environment. Nonetheless, no one has reached this stage yet and it has only been down to marketing hype.”

The tortoise and the hare

It is clear that many players think that the faster they get the technology out the better. This is unsurprising in a profit-driven world, although there needs to be a focus on the rules and regulations of testing and rolling out the technology. Autonomous cars are already safer than human drivers, but there will still be glitches and issues through testing; in some cases, production cars have injured or killed consumers. This further cements the advantages of simulation, especially in the early stages of autonomous development. "The Human Factors Research Unit at the University of Southampton has already released papers that show that supervised testing of autonomous vehicles is inherently risky," says Hoyle. "The workload for the supervising driver is higher than the workload if they were driving the vehicles themselves. On top of that, you've got the problem of maintaining concentration and attention when so much of their traditional hands-on role has been taken away from them."

It has already been proved that these drivers are not likely to be as safe as if they were in control. So, is this an adequate backup for a software system that is known to have bugs in it? Haye doesn’t think so: "Until you've reached the point where you are able to statistically prove that your autonomous model has achieved a level of safety, you shouldn’t be testing your vehicle amongst other road users," adds Hoyle. Safety, although it can delay a supplier getting to market, must be the first priority for the industry. This can be managed effectively through simulation, taking much less time than physical testing and allowing companies to progress through the safety regulations with more accurate results. "If you look at the industry now, you'll see that there are AI vendors with no OEM background taking a very different approach to the AI departments within OEM vendors. I think that we've seen a lot of rapid progress in the non-traditional area of the market, but I suspect that the OEMs supported by people who have been developing, testing and elevating vehicles for many years are going to relentlessly make steady progress. I would be surprised if many of the new entrants will survive without being bought or having their IP folded into the major OEMs and tier one suppliers." 

Tailoring the AI

When a brand is developing a new innovation like AI, it needs to put its own spin on the technology in order to retain its brand identity. For example, BMW will want its AI to have a sporty feel to it, whereas Mercedes will want to have a chauffeur-like quality that is comfortable and luxurious. To do this, says Hoyle, you need to understand vehicle dynamics. "When people get to test autonomous vehicles they tend to find it rather unpleasant and it's because all these models have been taught to do is attempt to drive safely. You need to model the less-obvious things such as ride comfort.” Software providers have to learn how to model these factors and match this kind of behaviour so that they take into consideration the passenger inside. "Unless you understand these vehicle dynamics through a testing environment that allows you to wrap it around the model of the vehicle, you are going to deliver a very bland and unpleasant autonomous car," warns Hoyle.

The second aspect is vehicle control. The traditional OEM and suppliers understand that the first thing that you have to do is learn to control the vehicle. In an emergency situation, it is not enough to slam on the brakes; the vehicle must know the best outcome through split-second decisions. "If there is a crash on the motorway, it is more important to get out of trouble, rather than continue to follow the rules of the road," says Hoyle. "It doesn't matter if you're going sideways at 90mph, as long as the vehicle avoids any obstacles in the road, such as an overturned vehicle." Fortunately, some people are starting to get this right, which is a great step in the right direction in terms of AI.

Two years ago, the industry was promised a solution to all of its problems from the hardware vendors. However, they are still claiming that this kind of innovation is two years away, showing how challenging the process has been with autonomous software. We are now seeing a significant push for autonomous software on our roads, so it is going to be down to the R&D surrounding the technology. "For us, it's a case of scaling everything up to the point where we are able to simulate millions of miles overnight on a reasonable-sized chunk of hardware. What works in our favour is that we do almost everything on the graphics card and the rate of progress on these is still doubling, allowing us to throw more and more at it." This is a bright light for self-driving software which should continue to blossom into the vision that the industry has for it.  

You may also like:

Don't miss out

Be the first to know about the latest news in the automotive and transportation industry, through our weekly mailer and bi-monthly publication: New Mobility Visions.

New Mobility | Visions | Three6Zero Limited © 2018