Inside Facebook’s Biggest Artificial Intelligence Project Ever – Fortune

6 months ago Comments Off on Inside Facebook’s Biggest Artificial Intelligence Project Ever – Fortune

When a computer can parse that much information and make judgements, it’s a disconcerting reminder that every single aspect of our digital lives is being atomized, sliced, and diced in ways that show advertisers, researchers, and even governments a picture of our private thoughts and actions. Just as troubling: The notion that machine learning algorithms may not get things right.

And none of this accounts for the fact that many people don’t even know that machine learning methods are altering their experience of a product. The reason a person may not see a post in his or her News Feed may be because an algorithm filtered it out. In 2014, an MIT study discovered that 62.5% of participants in the study were not aware that Facebook filtered their News Feed.

“The best AI algorithms can generalize, and they can predict what you want, but they are never perfect,” Candela says. It’s one reason why Schroepfer believes that Facebook remains far from turning everything over to artificial intelligence technologies.

“I think you still have people in the decision loop,” Schroepfer says. “We are building things for other people, and it’s hard for me to believe that, even with our advanced technology, machines can figure out what other people want.”

Schroepfer says all of this work is meant to build a social network that can better anticipate what a user wants to see or experience. If you have a bad day, he wants Facebook to show you humorous cat videos. If you haven’t talked to your mother in a week, he wants Facebook to recognize that and actively serve you an update about her life.

“The problem with Facebook right now is, you’re not telling us enough about what you want,” Schroepfer says. “We’re trying to guess at it. Part of the problem is we don’t know what to ask, and we’re not sure what to do with it when you tell us. Because our systems aren’t yet really set up to optimize for that.”

The Applied Machine Learning team is a chance to establish those systems. FAIR, meanwhile, is an opportunity to build a better understanding of how to make computers learn.

Get Data Sheet, Fortune’s technology newsletter.

Facebook’s decision to break out its artificial intelligence research in this way is somewhat unusual among its peers.

For example, Microsoft


, which is home to a large artificial intelligence research group that is part of its Microsoft Research unit, does not turn over its efforts to a commercialization team that then turns it into a product for internal consumption. Instead, a researcher might work directly with a person on the product team to build a tool or new service using machine learning.

Externally, Microsoft is trying to build a platform of services for machine learning and offer them to customers through Azure, its cloud computing platform, says Peter Lee, the head of Microsoft Research.

Still, Lee is in agreement with Facebook’s Schroepfer that machine learning and AI are enabling companies to build new products that were just too time-consuming or resource-heavy in years past.

Candela, who came to Facebook from Microsoft, says he intentionally tried to create a different structure within Facebook because he felt that good ideas couldn’t move quickly across the organization when he was at Microsoft. Each innovation or new artificial intelligence algorithm was locked into one team. Facebook is trying to resist that, he says.

But Andrew Moore, dean of computer science at Carnegie Mellon University, is skeptical that an artificial intelligence platform like FBLearner Flow can be used broadly across an organization. Most machine learning models can’t be generalized, he says.

“For machine learning, there’s one trap, and I don’t think I know of large company that hasn’t fallen into this trap,” he says. “It seems like a very useful thing to build a platform to support [machine learning algorithms] but you discover that each application that uses machine learning needs a different application to use it. So there is sometimes a disconnect between the the creators of a machine learning platform and those trying to build a product.”

For now, Facebook is happy with its efforts to date, and they seem to be paying off in its new products. There are still plenty of things the company must get right as it hands more decisions to algorithms, but the overarching project has changed the way the company measures its success.

For example, Facebook’s launch of a “real names policy,” which required people to use their actual names on the site, upset transgender users (who may not identify with their given name),users of Native American descent (whose names don’t neatly fit into Western formats),and victims of abuse (who sought additional privacy). But Facebook’s algorithms couldn’t easily parse these names.

Today, Facebook segments its data differently to ensure that smaller populations don’t get lost in the averaging process, Schroepfer says. It also conducts qualitative reviews of new products with focus groups and direct user feedback. All of this has helped prevent “rocky” product launches, he says. “Now it’s pretty rare for us to launch something where we didn’t understand how [the change] was better for people.”

It’s an early step on what amounts to be a very long road. Artificial intelligence technologies are inarguably making computer processing more efficient and allowing us to build systems at a scale never before seen. They are helping Facebook expand the reach and capabilities of its social network without eroding the profits it generates. With a little luck, they’ll help us better learn how to live with machines, too.

Inside Facebook’s Biggest Artificial Intelligence Project Ever – Fortune