View all newsletters
Receive our newsletter - data, insights and analysis delivered to you

AI Shoots Down Human Pilot in Virtual Dogfight as Reinforcement Learning Proves its Worth

An AI "robust enough to beat any opponent, even a human"

By Matthew Gooding

An AI has beaten a human fighter pilot to emerge as the Top Gun in a virtual dogfight which also has been hailed as a victory for the reinforcement learning technique.

The algorithm from US defense contractor Heron Systems emerged victorious from the US military‘s AlphaDogfight challenge. It defeated seven other AIs piloting simulated F-16 fighters, before taking on an experienced pilot wearing a virtual reality headset and piloting a simulator. The final score was a 5-0 win for the machine over its human counterpart.

Heron trained its system using the reinforcement learning technique, where the algorithm is asked to perform the same action over and over again in a virtual world until it gains something akin to an understanding of the task. Runner-up Lockheed Martin also used similar techniques, suggesting the method could play a key role in future development of autonomous air-to-air combat.

Ben Bell, Heron’s lead engineer for machine learning, said training the AI involved creating a training environment featuring a “league” of 100 unique agents.

Speaking at a Q&A after the dogfight, he said: “We didn’t try to combine any kind of expert systems. Our advantage was only using reinforcement learning so we tried to create an action scheme where the neural network could control the plane in a way that was smooth, which you didn’t see our competitors doing, and at a high enough rate where it could make some high aspect shots.

“Our training architecture is the second most important thing we did differently. We started off early with a league of agents. We want to create multiple different agents that fly in different patterns, with different reward structures and different neural network architecture. So the final agent we used has trained against 102 other totally unique agents, and that made us robust enough to beat any opponent, even the human pilot.

Content from our partners
Green for go: Transforming trade in the UK
Manufacturers are switching to personalised customer experience amid fierce competition
How many ends in end-to-end service orchestration?

“Even a week before trial one we had agents that weren’t very good at flying at all. It hasn’t been easy but we’ve managed to turn it round.”

The challenge was set by Defense Advanced Research Projects Agency (DARPA), as part of it’s Air Combat Evolution, or ACE, programme, which apparently “seeks to automate air-to-air combat and build human trust in AI as a step toward improved human-machine teaming.” We’re not sure if Maverick and Goose would approve.

Heron and the other competitors had a year to build their systems, and had to go through a series of qualifiers to reach the final trial.

“It’s been amazing to see how far the teams have advanced AI for autonomous dogfighting in less than a year,” said Colonel Dan “Animal” Javorsek, program manager in DARPA’s Strategic Technology Office.

Read more: “Data is Our Ammunition” – British Army CIO

Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.