These days, nearly every company is using artificial intelligence and machine learning to help make critical business decisions. But few companies have an idea how the systems they employ actually work.
As an engineering manager working on Facebook's News Feed for nearly two years, Krishna Gade got a firsthand look at how difficult it can be for companies to understand their own machine learning models.
"It became very difficult for even people within Facebook to be able to reason about 'Hey, how is News Feed working?' because there is an ensemble of machine learning models at play," Gade told CNBC.
"We needed to build tools and platforms to unlock this thing and provide those insights to an engineer all the way to an executive within Facebook."
Amit Paka, a former colleague and classmate of Gade's, had a similar experience when he was with Samsung, where he worked on the company's shopping apps. The apps relied on machine learning models to recommend products to users.
"Some of the challenges there were measuring the return on investment," Paka said. "How do you compare performance of an old model with a new model?"
After working on these problems, Gade and Paka realized companies might want tools to help understand their artificial intelligence technology. In October, the pair quit their companies and co-founded Fiddler Labs, a Mountain View, California, start-up that is building an explainable AI engine.
"When you build models you don't know really intrinsically what features are impacting the models. It's pretty much a black box," said Gade, Fiddler Labs' CEO. "What we're providing is a model that unlocks the black box."
Fiddler Labs' arrival comes at a crossroads for the world of AI. In the past two years, numerous research papers have been published about how to build AI models that can be explained. The company is using methods outlined in that research to build its products.
At the same, the demand for explainable AI is on the rise. Last year, the European Union implemented the General Data Protection Regulation. Article 22 of GDPR states that Europeans have the right to obtain an explanation for how automated decisions that have a significant impact on their lives are reached.
"Models increasingly are going to be important in the way we make a bunch of decisions, so asking for some level of accountability just makes a lot of sense," said James Cham, a partner at Bloomberg Beta, which has invested in the company. "I don't think GDPR is a one-off thing, but rather part of a larger trend as countries all over the world try to figure this out."
To kick off, Fiddler Labs has raised $3 million in a seed round led by Lightspeed Venture Partners with participation by Bloomberg Beta and Haystack, and it plans to launch its explainable AI engine in mid-2019, Gade and Paka said.
For now, the company is focused on hiring and expanding its team. It is also amassing a pool of design partners who can test their product and provide feedback. The idea is Fiddler Labs' explainable AI engine will allow companies to analyze, manage and deploy their machine learning models at scale. The goal is for these tools to provide simplicity so that anyone from a data scientist to a company executive can understand their AI technology.
"The long-term vision is to help enterprises build trustworthy AI experiences for their customers," Paka said.