View all newsletters
Receive our newsletter - data, insights and analysis delivered to you

Facebook takes a look into deep learning for mobile AR/VR

Facebook revealed how it created the deep learning capability for use on mobile.

By Sabrina Dougall

Facebook has released an update on its visual perception technology for VR and AR technologies. Software engineers at the social network company have posted details of their computation model which uses deep learning to enable effective full-body tracking capability on a mobile.

“We recently developed a new technology that can accurately detect body poses and segment a person from their background,” said the Facebook AI Camera Team in a research post.

Although the tracking capability itself is not revolutionary, the research team went into some detail about how it created the necessary neural network model for use on a smart phone in real-time.Facebook

Circumventing the cumbersome ResNet typically leveraged for Facebook’s initial Mask R-CNN model, engineers were able to optimize the number of convolution layers as well as the width of each layer to lighten the compute load.

“Developing computer vision models for mobile devices is a challenging task,” researchers wrote. “A mobile model has to be small, fast and accurate without large memory requirements.

Data scientists noted the small model size and fast runtime made possible through the modularity of Caffe 2 – not to be confused with Intel Coffee Lake CPUs – Facebook’s scalable deep learning framework, first made open source in April 2017. Engineers utilised “a mobile CPU and GPU libraries including NNPack, SNPE and Metal” to boost computation speed.

“Our final model is only a few megabytes and is very accurate,” researchers said.

Content from our partners
Green for go: Transforming trade in the UK
Manufacturers are switching to personalised customer experience amid fierce competition
How many ends in end-to-end service orchestration?

Facebook AI Research (FAIR) has made Caffe2 operators open-source, including GenerateProposalsOp, BBoxTransformOp, BoxWithNMSLimit, and RoIAlignOp.

How to hack a connected car
Best artificial intelligence courses 2018
WEF Davos: Artificial intelligence forces fundamental business shift

Earlier in January, Facebook partnered with the University of Washington to establish a new research centre for AR/VR technologies. The UW Reality Lab, built within the School of Computer Science & Engineering and located in Seattle, is also home to the Oculus Research division of Facebook which produces the company’s popular headset.

“As we work to give people the power to build community and bring the world closer together, augmented reality and virtual reality will form a growing role as the technical foundation for many experiences,” said Michael Cohen, Director of Computational Photography at Facebook and UW Reality Lab Advisory Board Member.

Topics in this article : , , , , ,
Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.