At F8 today, Facebook is announcing a bunch of utterly crazy s--- that we'll soon be able to do to the pictures we take. That includes Facebook, WhatsApp, Facebook Messenger, and Instagram and affects, oh, somewhere approaching 2 billion people.
But while the company is talking a lot about cameras, it would be a mistake to look at what it is rolling out as a mere photography tool. Yes, there are cool picture effects. But what Facebook is really trying to do is to fully insert itself in the real world. Facebook's augmented reality camera effects are an early attempt to let the digital infiltrate the physical, a way for the company to become the conduit between everything you see in the world around you, and all the information that exists, via your smartphone.
"Facebook is so much about marrying the physical world with online," the company's CEO Mark Zuckerberg told BuzzFeed News in an interview late last week. "When you can make it so that you can intermix digital and physical parts of the world, that's going to make a lot of our experiences better and our lives richer."
It is certainly going to make life weirder. At an earlier demo, when a group of 18 Facebook engineers gathered to show their work to an outsider for the first time, they were clearly nervous. One pointed his phone at a table, and a 3D propeller plane popped up on screen, circling around a water bottle that rested on the tabletop. Another used his phone's camera to turn the room into a planetarium, with planets and stars hewing across the ceiling as shooting stars fired from side to side. Still another took a normal photo of a face — and then made it smile, frown, and gape with the push of a button. Little wonder they seemed on edge: The stuff they were showing off was wild and largely unprecedented.
More from Buzzfeed:
This new Camera Platform, as the company calls it, is a major bet that the camera isn't simply a tool used to capture images. It's something you'll use when you want to share photos and videos, sure, but also when you want to overlay digital experiences on the real world. Imagine, Zuckerberg urged, using Facebook's camera to view pieces of digital art affixed to a wall. Or to play a digital game overlaid on a tabletop. Or to leave a digital object in a room for someone to later discover — perhaps even future generations. Imagine using your phone to take a 2D photo, and then transform that photo into a 3D space. Imagine manipulating a friend's expression to make them smile, or frown, or, well, whatever. Imagine changing your home into Hogwarts for a Harry Potter-obsessed daughter. That's what Facebook is doing. "We see the beginning of an important platform," Zuckerberg said.
Onstage at F8 Tuesday morning, Zuckerberg doubled down on this point: "The camera needs to be more central than the text box in all of our apps," he said. "We're making the camera the first augmented reality platform."
And you thought this was just about Snapchat.
AI at War
It's easy to draw comparisons to Snapchat. And certainly the camera platform's Snapchat-like effects are likely to grab the most attention early on. But the more interesting stuff that Facebook is trying to pull off involves layering the digital and physical worlds on top of each other — bringing the former into the latter, and vice versa. There will be three big augmented reality areas Facebook is pushing into. The ability to display information on top of the world in front of you, the ability to add new digital objects to your environment (think: Pokemon Go), and the ability to enhance existing objects.
For example, Facebook's Camera can map out two-dimensional photographs in 3D. The company hopes developers will someday build digital products that behave and interact in those formerly 2D spaces, just as they would in the rich three-dimensional world we live in. Picture this: In one demo, Facebook showed off various 3D scenes created entirely from a handful of 2D photos. The scenes had real depth to them — you could peer around a tree in a forest, or tilt your head to see behind a bed in a room. With a few clicks, the lights went down in the room. The forest flooded with water. It was magic.
The demo was on an Oculus headset, but Facebook's ambition is to bring these kind of scenes directly into the News Feed itself, no Oculus required. It wants people to be able to create and interact with them directly on their phones.
The ultimate idea here is to turn the real world into an extension of Facebook itself. "There's all these different random effects which are fun, but also foundational to a platform where people can create 3D objects and put them into the world," Zuckerberg explained.