View all newsletters
Receive our newsletter - data, insights and analysis delivered to you

Facebook takes a look into deep learning for mobile AR/VR

Facebook revealed how it created the deep learning capability for use on mobile.

By Sabrina Dougall

Facebook has released an update on its visual perception technology for VR and AR technologies. Software engineers at the social network company have posted details of their computation model which uses deep learning to enable effective full-body tracking capability on a mobile.

“We recently developed a new technology that can accurately detect body poses and segment a person from their background,” said the Facebook AI Camera Team in a research post.

Although the tracking capability itself is not revolutionary, the research team went into some detail about how it created the necessary neural network model for use on a smart phone in real-time.Facebook

Circumventing the cumbersome ResNet typically leveraged for Facebook’s initial Mask R-CNN model, engineers were able to optimize the number of convolution layers as well as the width of each layer to lighten the compute load.

“Developing computer vision models for mobile devices is a challenging task,” researchers wrote. “A mobile model has to be small, fast and accurate without large memory requirements.

Data scientists noted the small model size and fast runtime made possible through the modularity of Caffe 2 – not to be confused with Intel Coffee Lake CPUs – Facebook’s scalable deep learning framework, first made open source in April 2017. Engineers utilised “a mobile CPU and GPU libraries including NNPack, SNPE and Metal” to boost computation speed.

“Our final model is only a few megabytes and is very accurate,” researchers said.

Content from our partners
Why email is still the number one threat vector
Why HR must take firm steps to become a more data-driven function
Why enterprises of all sizes must  embrace smart manufacturing solutions

Facebook AI Research (FAIR) has made Caffe2 operators open-source, including GenerateProposalsOp, BBoxTransformOp, BoxWithNMSLimit, and RoIAlignOp.

How to hack a connected car
Best artificial intelligence courses 2018
WEF Davos: Artificial intelligence forces fundamental business shift

Earlier in January, Facebook partnered with the University of Washington to establish a new research centre for AR/VR technologies. The UW Reality Lab, built within the School of Computer Science & Engineering and located in Seattle, is also home to the Oculus Research division of Facebook which produces the company’s popular headset.

“As we work to give people the power to build community and bring the world closer together, augmented reality and virtual reality will form a growing role as the technical foundation for many experiences,” said Michael Cohen, Director of Computational Photography at Facebook and UW Reality Lab Advisory Board Member.

Topics in this article: , , , , ,
Websites in our network
NEWSLETTER Sign up Tick the boxes of the newsletters you would like to receive. Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
I consent to New Statesman Media Group collecting my details provided via this form in accordance with the Privacy Policy
SUBSCRIBED

THANK YOU