Augmented Reality: Benedict Evans pondering an augmented reality

Benedict Evans, a guy who always thinks from the point of view of a venture capitalist because that's his job

What might that look like for a billion people? (((That's a good thing to ask.)))

The first level to AR is what we had with Google Glass - a screen floats things in space in front of you, but it's not connected to the world in any way. In fact, Google Glass was conceptually pretty similar to a smart watch, except that you looked up and to the right instead of down and to the left. It gave you a new screen but it didn't have any sense of what was in the world in front of you. With more advanced technology, though, we could extend this to make your entire field of view a sphere that could show windows, 3D objects or anything else, floating in space.

However, with what one might call 'true AR', or, perhaps 'mixed reality', the device starts to have some awareness of your surroundings, and can place things in the world such that if you suspend disbelief you could imagine that they're really there. Unlike Google Glass, the headset will do 3D mapping of your surroundings and track your head position all the time. That would make it possible to place a 'virtual TV' on a wall and have it stay there as you walk around, or make the whole wall a display. Or you could put Minecraft (or Populous) on the coffee table and raise and lower mountains with your hands, as though you're modelling clay. By extension, anyone else wearing the same glasses could be able to see the same things - you could make a wall or conference table into a display and your whole team could use it at once, or you and your children could play god around the same Minecraft map. Or that little robot could hide behind the sofa, or you could hide it there for your children to find. (This has some overlap with some use cases for VR, of course, especially when people talk about adding external cameras.) This is mixed reality as a screen - or perhaps, as turning the world around you into infinite screen.

But there's somewhere else to go with this, because you're still only really doing SLAM - you're mapping the 3D surfaces of a room, but not understanding it. Now, suppose I meet you at a networking event and I see your LinkedIn profile card over your head, or see a note from Salesforce telling me you're a key target customer, or a note from Truecaller saying you're going to try to sell me insurance and to walk away. Or, as Black Mirror suggested, you could just block people. That is, the cluster of image sensors in the 'glasses' aren't just mapping the objects around me but recognising them. This is the real augmentation of reality - you're not just showing things in parallel to the world but as part of it. Your glasses can show you the things that you might look at on a smartphone or a 2000 inch screen, but they can also unbundle that screen into the real world, and change it. So, there's a spectrum - on one hand, you can enrich (or pollute) the entire world by making everything a screen, but on the other, this might let you place the subtlest of cues or changes into the world - not just translate every sign into your own language when you travel, but correct 'American English' into English. If today people install a Chrome extension that replaces 'millennial' with 'snake people', what would an MR extension change? How about making your boss puke rainbows? (Fun is, after all, the most important real problem to solve.)

Do you wear the glasses all day, though, once they become small enough? If not, many of the more ambient applications don't work as well. You might, perhaps, have a combination of a watch or phone that's the 'always on' device, and glasses that you put on like reading glasses in the right context. That might address some of the social questions that Google Glass ran into - taking out a phone, looking at a watch or putting on a pair of glasses send a clear signal that people can understand in a way that wearing a Google Glass in a bar did not.

This touches on a related question - do AR and VR merge?...