Oath, the Verizon-owned parent company of Yahoo, is releasing for free some of its most important internal software, which the company has long used to make recommendations, target ads and execute searches.
The Vespa software solves a common but surprisingly difficult problem: quickly figuring out what to show a user in response to input, like when they type text into a box. Oath uses it in around 150 applications, including Flickr, Yahoo Mail and the main Yahoo search engine (specifically for components like entities, local results, images and answers to questions). It handles 3 billion native ad requests every day.
"The typical case is you don't know what you want to serve, but you have 20 billion pictures and you want to find the right ones," Jon Bratseth, a distinguished architect at Yahoo who led Vespa's development, told CNBC in an interview.
Vespa, which is now live on GitHub with an Apache 2.0 open-source license, can easily be added to different applications, making it suitable for use at big companies like Amazon, Facebook and Google that need to do different kinds of processing on different sets of data.
The release is the most important for Yahoo since it open-sourced the code for the Hadoop big data software in 2006. Hadoop has since come to be at the center of two public companies, Cloudera and Yahoo spin-off Hortonworks. Today people at lots of companies can contribute to technology that's still widely used at Yahoo, and build their own systems using Hadoop.