Smart glasses are making a comeback. Have they got any smarter?

The hall erupted in whoops and applause as Sergey Brin bounded onto the stage. Already, the audience at Google’s 2012 I/O developer conference had been treated to piecemeal company announcements about Android software improvements and search algorithm breakthroughs. What the company’s co-founder was about to show them, though, was entirely unexpected: Google’s first attempt at developing a pair of smart glasses. Brin’s attempt to demonstrate the device’s potential, however, would involve around a dozen parachutists broadcasting their jump from an airship down to the convention centre. “This can go wrong in about 500 different ways,” warned Brin.

Thankfully, Google’s first public demonstration of its smart glasses didn’t result in any high-velocity accidents. The product’s release a year later, however, proved a slow-motion disaster. Criticism from privacy advocates came thick and fast, with many claiming that the device insufficiently alerted passers-by to when they were being recorded. A public backlash ensued. One user, journalist Mat Honan, keenly observed how uncomfortable people became around the glasses. “I’m not wearing my $1,500 face computer on public transit, where there’s a good chance it might be yanked from my face,” he wrote in an essay for Wired. Two months later, another user would record this happening to her in a bar in San Francisco.

It could have been that the Google Glass folks just did not anticipate the privacy backlash.
Apu Kapadia, Indiana University

“It could have been that the Google Glass folks just did not anticipate the privacy backlash,” says Apu Kapadia, a computer science professor at Indiana University and an expert on smart glasses. After all, from a designer’s perspective, the fact that the product’s display above the user’s right eye is clearly on while recording should constitute enough warning for sensitive passers-by. Even so, that wasn’t something non-users were used to. “Absent that physical action of bringing a camera to your face, it was not clear that recording had started,” says Kapadia, “even though, technically, you could stare at the person and maybe figure it out.”

Sales of Google Glass proved lacklustre, seemingly dooming the concept of consumer-friendly smart glasses for a generation. In recent months, however, several other tech companies have gambled on precisely the opposite. Last September saw Facebook announce its partnership with Ray-Ban to launch ‘Stories,’ a pair of smart glasses retailing at $299 that would allow the user not only to stream videos, but also receive phone calls. The news came amid a flurry of other releases that year, including Oppo’s Air Glass and TCL’s NXTWEAR G glasses. Such devices have been seen as the harbingers of the ‘metaverse,’ a (contested) vision of the future wherein individuals will be able to immerse themselves in virtual worlds anchored in the physical, all with the aid of augmented reality software.

There were even rumours that Google itself was contemplating a return to the world of smart glasses, this time with a device that incorporated AR software. The search giant may have been encouraged by the curious afterlife of its pioneering spectacles. Even after the consumer version was discontinued in 2015, Google Glass has continued to find uses in sectors including logistics and healthcare, where the hands-free nature of the devices is advantageous for staff working for long hours on complex, repetitive tasks.

Proponents of smart glasses also argue that new technologies now meet the privacy challenges that doomed earlier products. Facial recognition algorithms, for example, can be used to automatically block unknown faces in frame, while visual AI systems are capable of spotting clues that the user has walked into a bathroom or a private home, and turn the camera off.

S.A. Applin is skeptical. The anthropologist, who has spent her career analysing the impact of algorithms and automation on human society, is concerned that such technologies may have unforeseen limitations. “People take agency,” says Applin, “and are able to do things that may not have been considered by those creating the privacy protections, thus making them less effective.”

Protecting privacy

Much of Google and Facebook’s business model rests on the premise that customers are searching for platforms from which to share data about their lives, careers and businesses and learn about others. It is also true that the hardware used to achieve this – namely, laptops and phones – hasn’t changed much. Smart glasses, meanwhile, provide added mobility and new perspectives that allow novel formats for photos and video. Their hands-free nature may also appeal to those keen on maintaining some measure of social distancing in the wake of the pandemic, as Applin and her co-author, Dr. Catherine Flick, noted in a recent paper.

But the concerns of those who feel uncomfortable at the prospect of being surreptitiously recorded by a smart glasses user haven’t disappeared – especially given how much more advanced these products are compared to their antecedents. Ray-Ban Stories, for example, have been criticised for how easy it is to accidentally take photos, and seem to be the first salvo in Meta’s attempt to create AR glasses that can only work by collecting vast amounts of mapping and location data (with some exceptions.) While it might be argued that most people are used to being recorded all the time by other people’s phones and CCTV cameras, these devices arguably benefit from a concept that researcher Helen Nissenbaum dubbed ‘contextual integrity.’ For example, when we walk through an airport, most people will accept that they are being recorded; similarly, the few seconds when someone takes their phone out of their pocket and lifts it to eye level indicate that a recording is about to take place.

Smart glasses, by contrast, lack many obvious visual and contextual clues in their use. While newer models have incorporated bright lights to indicate that recording is in progress, some have argued that it’s still not easy for someone to tell when the user has hit record. Additionally, the reactions of the public to a new generation of consumer smart glasses – whether they’re wearing them, or observing those that do – is unpredictable, says Applin, simply because concepts of privacy vary from culture to culture. “It is many things to many cultures,” she says, “and as we alluded to in our paper, technologists must take into account cultural differences when designing and deploying any new technology.”

However, there is evidence to suggest attitudes toward smart glasses have softened since 2014. Social media has exploded in popularity during that time, which Kapadia argues has coincided with a generational shift in attitudes toward personal privacy. Patently, Gen Zers are much more used to being photographed and filmed by mobile phones than their elders. “As each generation becomes, in some sense, more open to newer technology and the photo medium, maybe – maybe – the tech companies see an opportunity,” says Kapadia.

Even so, it’s a mistake to think that privacy is a non-issue for teenagers. Evidence from interviews with young people by Kapadia and several of his colleagues about their attitudes toward privacy suggest that teenagers consistently modify their behaviours in settings where footage is expected to be taken with the intent to publish on social media. One interviewee, for example, remarked that they were less inclined to take part in underage drinking at house parties because of the chance their parents might see a photo on social media.

One would expect this behaviour to be accentuated if smart glasses become the primary vehicle for capturing and disseminating images and video on social media. “That’s maybe sad, in a way,” says Kapadia. It could be argued, though, that that division of personae isn’t so different from the conduct of older generations on Facebook as opposed to LinkedIn. Rather, “each generation changes the way they behave and then has a new bar for privacy” as technology advances, explains Kapadia.

Technology has also sufficiently advanced since 2014 to protect the privacy of those who feel uncomfortable being filmed or photographed by smart glasses, say their proponents. Facial recognition systems, for example, could block out unknown faces, while visual AI software could interpret contextual clues in the local environment and switch off the in-built camera whenever the user walks into a family home or a public bathroom.

Critics argue that such technologies are not sufficiently advanced yet. Another major concern, says Kapadia, is cybersecurity. The horror stories of voyeuristic hackers switching on laptop cameras while keeping the recording light off are well-known. The (very minor) silver lining in these tales is that such criminals are usually only afforded a limited vantage point. Hacking smart glasses, however, could potentially allow third parties an uninterrupted view of how we conduct our daily lives.

Smart glasses, smart workplace

Despite the hype around Facebook and Ray Ban’s collaboration, consumer smart glasses still only form a small part of the overall market compared to business applications. It will be that way for a little while yet, says Paul Travers. “The broader markets are harder to solve a problem for,” explains the CEO of Vuzix, one of the world’s leading smart glasses developers. This means that the popularity of smart glasses is dependent on whether they’re fashionable. “And the tech right now is not small enough to make fashion-forward products.”

The tech right now is not small enough to make fashion-forward products.
Paul Travers, Vuzix

It is small enough, however, for the workplace. Vuzix is one of several companies around the world developing smart glasses for enterprise applications. Travers is particularly proud of the M400, which carries an eight-core processor and 4K streaming video cameras while only weighing 2.8 ounces (most normal eyeglasses weigh just under an ounce.) Vuzix sells approximately 15,000-20,000 of these glasses and others like it every year, Travers claims.  Their chief selling point, he explains, is their usefulness in streamlining training regimes, giving instructors a live head-height view of everything that the student is doing and allowing the learner to superimpose schematics and other visual aids on their surroundings.

One particularly profitable segment for Vuzix has been warehousing, where supervisors have been using its glasses to remotely train new staff. “A brand-new person, who’s never packed a pallet before, is told, step-by-step as he’s scanning the goods, where it needs to go,” explains Travers. Healthcare is another fruitful area, where smart glasses have been used to plan and assist in numerous surgeries.

More controversially, these devices have also been seen resting on the noses of security personnel. In recent years, reports have emerged of smart glasses loaded with facial recognition software being used by both Russian and Chinese police. Vuzix itself has also collaborated with software companies to adapt facial recognition software for its glasses, most notably in 2019 when it partnered with developers NNTC to produce pairs for security personnel in the UAE.

The use of facial recognition services by law enforcement has already proven controversial. A logical next step for its integration in smart glasses might be the ability of police officers to instantly view additional data related to a facial match, like a suspect’s home address. This touches on wider ethical questions about what kinds of data smart glass users should be able to pull up about other individuals. There is a danger, says Kapadia, of smart glass developers trying to use visual AI to quantify subjective aspects of a person’s identity; their race, perhaps, or their gender.

For his part, Travers is convinced that it is justifiable for law enforcement to use smart glasses inside certain spaces where the necessity for active surveillance is necessarily high (the same spaces where, he adds, Vuzix smart glasses might be encountered.) “Anybody who thinks they’re walking through a border crossing and they’re not going to have their face scanned is smoking crack,” he says. When it comes to police using them outside of those confined circumstances, however, Travers is wary. “I will say, though, in a lot of countries, people just live with it. In America, it’s a little different.”

Whether or not US police departments will be outfitted with smart glasses remains to be seen. What is clear is that regulatory provisions specific to these types of devices are few and far between (although theoretical safeguards do exist in existing privacy and human rights statutes.) Any future rules on smart glasses have to be carefully adapted to market realities, explains Applin.

“Maybe we can eventually hold big manufacturers to regulations,” she says. Even so, regulators also need to be aware of the possibility of a cottage industry emerging of developers looking to sell their own versions of smart glasses to the public. “People are going to be 3D printing these in their garages and writing their own software,” says Applin. “And those glasses are going to be taking pictures.”