Cloudera and Confluent Introduce New Open Source API Compatibility Testing for Apache Kafka

Strata+Hadoop London, May 6, 2015 (GLOBE NEWSWIRE) -- Cloudera, the leader in enterprise analytic data management powered by Apache Hadoop™, together with Confluent, is leading an initiative to create an open source API compatibility testing framework for advancing adoption of Apache Kafka™ by users, independent software vendors (ISVs), original equipment manufacturers (OEMs), and solution providers. The challenge for today's developers is the incremental risk, costs, and time associated with building products and applications that rely on the Kafka APIs and protocols, and making sure that the underlying platform implements those interfaces correctly. By having access to free tools for validating compatibility with those specs, development can be significantly streamlined.

This framework will be released in open source as part of the Apache Kafka project.

"Our intention with this new compatibility testing framework is to ensure that any distribution including Kafka can be certified as compatible with the releases made by the Apache Software Foundation," said Jay Kreps, CEO at Confluent. "While Kafka has already gained significant traction in the developer community, leading this joint testing framework offering with Cloudera will help to accelerate its value to enterprises building critical systems around Kafka."

"This initiative, in conjunction with others we are bringing to market, reflects our commitment at Cloudera to preserve and accelerate the adoption of open source and open standards that create the foundation for the big data platform of the future," said Mike Olson, Chief Strategy Officer at Cloudera. "Customers benefit greatly from open standards such as Kafka, which allow for more data to be consumed, and at a faster rate, into next-generation workloads, thus transforming data into business value more quickly than legacy applications. Being able to automatically validate compatibility will help ensure that the work vendors do never interferes with compatibility with the open source project."

Kafka, currently maintained at the Apache Software Foundation and initially designed and largely developed by Confluent employees, acts as a real-time, fault tolerant, highly scalable messaging system. Kafka exploits new data infrastructures made possible by massively parallel commodity clusters. Its fundamental ability to make high volume data available as a real-time stream for consumption is extremely beneficial for systems with very different requirements--from batch systems like Hadoop, to real-time systems that require low-latency access, and to stream-processing engines that transform the data streams as they arrive.

Supporting Organizations:



Celer Technologies




Dell Computer

E8 Security



NetFlow Logic













Supporting Quotes:


"Dedicated to helping clients pursue a data-driven digital transformation, Accenture is committed to building its technology ecosystem with the tools clients need to turn data into an asset that drives real-time decisions and outcomes. In June 2014, Accenture and Cloudera established a formal alliance to accelerate the speed in which joint-clients in all industries can intake, access, and analyze data. Cloudera's new open source compatibility testing framework can help developers to more-swiftly and effectively build solutions on Apache Kafka for clients."

-- Narendra Mulani, Senior Managing Director, Accenture Analytics


"Most big data projects include a real-time decision component, and more and more open source technology such as Kafka, Sqoop, or Flume and of course APIs are key components. More than ever, the market demands a shorter time to market. Using an open source compatibility testing framework will dramatically improve the time to market and the reliability of these business critical functions."

-- Manuel Sevilla, CTO Global Insights & Data Practice


"As a provider of network data analytics for IT operational intelligence, Corvil is asked to integrate with many big data solutions. Standards such as Sqoop and Flume greatly reduce the required development burden and allow us to respond to market demands with greater agility. It is hugely advantageous to see such standards being more broadly updated, and to know that implementations are validated by Cloudera."

-- Donal O'Sullivan, VP Products


"As an open source NoSQL vendor, we strongly support any effort that makes it easier for enterprises and developers to build on open technologies. Apache Kafka is used by many Couchbase customers, including PayPal and LivePerson, who regularly adopt open source technology for innovative big data solutions. Having a free, open test framework will make it faster and easier for enterprises to deploy open source technology needed to support mission critical applications."

-- Bob Wiederhold, CEO


"As an early-adopter of open-source technologies, Fortscale believes that Cloudera's efforts to accelerate the adoption of Apache Kafka will enable faster growth of innovative solutions that can scale effectively, thereby generating better value for customers. This was demonstrated numerous times when Fortscale successfully delivered products built using Hadoop, Kafka, and Flume as key building blocks."

-- Guy Mordecai, Director of Product Management

Intel Corporation

"Enterprises like Intel are increasingly investing in big data analytics solutions that require the collection and processing of streaming as well as bulk data. Platforms based on open standards, open source software, and open APIs can significantly reduce the cost and complexity of application development and deployment. Intel has a long-standing commitment to standards-based innovation and supports this initiative to enable greater compatibility between key components of the Apache Hadoop platform."

-- Ron Kasabian, VP Data Center Group and General Manager, Big Data Solutions

NetFlow Logic

"Furthering the adoption of open standards such as Kafka, Flume, and Sqoop in the next-generation big data platform benefits the entire community and allows users to accelerate the development and adoption of next-generation applications."

-- Damian Miller, VP Customer Success


"Today, it is more critical than ever to provide organizations with capabilities to achieve greater context around the massive amounts of data they are collecting to extract real intelligence. The work that Cloudera and their partners are doing around open standards is a significant industry milestone that we are excited to support and leverage to help organizations make better security decisions."

-- David Corlette, Sr. Product Manager, NetIQ security portfolio of Micro Focus


"It's great that the Apache community is investing in testable APIs. This is will be a tremendous benefit to ISVs such as Rocana who are building on top of a broad range of projects."

-- Eric Sammer, CTO


"We expect the adoption of open standards like Apache Kafka, Sqoop, and Flume to set the foundation for tomorrow's analytic, big data workloads. Giving developers free and open access to tools fosters innovation and collaboration across the data science community, but also helps data-intensive industries re-imagine the role of high performance computing across the enterprise."

-- Bob Braham, CMO


"StreamSets, like Cloudera, is completely dedicated to furthering the adoption of open, multi-vendor standards as important building blocks in tomorrow's big data platform. Apache Kafka, which is an important component of the StreamSets offering, enables the ingestion of more data, more rapidly, into next-generation analytic workloads, so we support this Open Source Compatibility Framework initiative as a means of accelerating its uptake across the ecosystem."

-- Arvind Prabhakar, CTO


"We are excited to participate in Cloudera and Confluent's open source API compatibility initiative which will help accelerate vendor participation and customer adoption."

-- Tendu Yogurtcu, General Manager for Big Data


"In our client engagements, we see that the success of impactful big data systems critically depend on the efficiency of data ingestion. This ranges from traditional batch-oriented systems to the exciting new streaming applications necessitated by the Internet of Things era. We believe that open source standards such as Kafka, Flume, and Sqoop and the associated test frameworks are central to the rapid adoption of big data applications"

-- Dr. Satya Ramaswamy, Global Head of TCS Digital Enterprise


"Teradata supports the Apache Kafka compatibility testing framework initiative. Working together, we can streamline the advancement of this open source technology, which offers our customers the choice of an additional set of tools. Open source technologies, like Kafka, are an important addition to the existing analytical ecosystem."

-- Chris Twogood, VP Products and Services


"TIBCO is committed to providing our customers with the benefits that open standards bring. Any effort to further the adoption of open standards such as Kafka, Flume, and Sqoop in the next-generation big data platform is a positive development for customers, the industry, and the ecosystem."

-- Karl Van den Bergh, VP Product and Cloud

About Cloudera

Cloudera is revolutionizing enterprise data management by offering the first unified Platform for big data, an enterprise data hub built on Apache Hadoop. Cloudera offers enterprises one place to store, access, process, secure, and analyze all their data, empowering them to extend the value of existing investments while enabling fundamental new ways to derive value from their data. Cloudera's open source big data platform is the most widely adopted in the world, and Cloudera is the most prolific contributor to the open source Hadoop ecosystem. As the leading educator of Hadoop professionals, Cloudera has trained over 30,000 individuals worldwide. Over 1,450 partners and a seasoned professional services team help deliver greater time to value. Finally, only Cloudera provides proactive and predictive support to run an enterprise data hub with confidence. Leading organizations in every industry plus top public sector organizations globally run Cloudera in production.

Connect with Cloudera

Read our blogs: and

Follow us on Twitter:

Visit us on Facebook:

Join the Cloudera Community:

Cloudera, Cloudera's Platform for Big Data, Cloudera Enterprise Data Hub Edition, Cloudera Enterprise Flex Edition, Cloudera Enterprise Basic Edition and CDH are trademarks or registered trademarks of Cloudera Inc. in the United States, and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.


CONTACT: Deborah Wiltshire Cloudera +1 (650) 644-3900 ext. 5907

Source: Cloudera