The camera is the new keyboard

Your smartphone camera is about to reshape everything, but it has nothing to do with snapping photos. 

Sure, phone makers are getting really, really good at creating cameras. Between hardware improvements and advancements in computational photography, we’ve never had better optics in our pockets than we do with the latest generation of flagships.

But that’s not why cameras matter more than ever before. It’s because cameras are at the heart of a paradigm shift that’s transforming the way we interact with the world around us. From augmented reality to camera-based search, cameras are increasingly becoming the interface through which we experience technology and find information.

Cameras take over

Say you’re walking down the street, and you pass by a restaurant you want to know more about. Right now, you might whip out your phone and type the name into Google, or Yelp, or some other service. But in the near future, you’ll be able to simply point your camera at the restaurant to get the same information.

Right now, we’re used to spending hours a day tapping away on our phones’ keyboards to access what we need. But much of that tapping would be unnecessary if we could just share visual context about the world around us. Instead of Googling how to fix your toilet, what if you could just hold up your camera to the broken part and get repair instructions?

Though we’ve been slowly approaching this shift for some time, it’s never been more apparent or more dramatic than it was in 2017. Some observers have taken to saying “camera is the new keyboard” as a kind of shorthand for what’s happening.

It’s already happening

One early sign of this shift is the unlikely comeback of the QR code — a feature once so maligned it inspired a 2013 marketing tome called QR Codes Kill Kittens. 

Yet just four years later, Spotify, Pinterest, Venmo, Snapchat, and Facebook Messenger all have their own versions of the feature. Even Apple added QR code-scanning to the iPhone’s camera — itself a tacit admission that QR codes aren’t going anywhere.

But for an even better look at where the future is heading, look to camera-based search, which stands to dramatically change how we find information. Google and Pinterest both launched products this year that allow you to conduct specific types of searches just by pointing your camera at objects in your surroundings. 

Google’s version of the tool can, for now, recognize text, household objects and landmarks. Pinterest’s tool is more about inspiration — point your camera at food to find relevant recipes or at a pair of shoes for style ideas. 

While those tools, both coincidentally called “Lens,” are still in the early stages, they paint a pretty clear picture as to where the tech is heading. “Search what you see,” Google summed up in a blog post earlier this year.

Microsoft is also experimenting with computer vision in the camera. The company introduced a new app called Seeing AI that’s meant to help the visually impaired “see” the world around them The app will narrate what’s around you as you point the camera at your surroundings.

Like Google and Pinterest’s Lens apps, Seeing AI is still in early days so the app has more than a few kinks. But it’s not difficult to imagine how the technology could be transformative to the visually impaired. 

What happens next?

Still, there are potential issues. While it’s difficult to deny the efficiency of pointing your camera at something and instantly getting helpful information about that thing, it does raise the question of who gets to decide what information comes up. This is fundamentally different from getting a page of blue links as search results. Sure, those are ranked, but it’s still ultimately up to the searcher to choose one. With AR search, you likely won’t even know alternative info is there.

Moreover, cameras and computer vision will accelerate the idea that combing through multiple pages of results feel as obsolete as texting with a numeric keypad. The price of that convenience, however, will be granting more power to whomever is deciding what the one “right” answer is any given situation. 

AR changes the privacy equation, too. By opening up these apps to the world around us, we’re handing huge amounts of private information over to these services, which will now know even more about what we’re doing, feeling, and thinking at any given moment. That’s not something that can be easily walked back. 

These are valid concerns — and ones that tech companies should address sooner rather than later as these changes are only going to accelerate. This year more than ever we’ve seen what happens when tech companies release tools to the public and are slow to adapt to the abuses of those tools that inevitably happen. 

Regardless, the smart camera is rapidly become a reality. Everywhere you look, tech companies are adding layer upon layer of intelligence to their cameras at a surprising rate. And while none of these applications are, on their own, enough to change our lives, put them all together and it adds up to a world where the camera — augmented by powerful chips, precise sensors, and a vast database in the cloud — becomes the most important app on your phone.