HP, Cloudera to join forces on big data appliance

08 Aug 2012

An end-to-end hardware/software solution for Hadoop big data approaches is a great idea – but not a unique one, says Jessica Twentyman

In late July, Hewlett-Packard and Cloudera strengthened their existing relationship with a reseller agreement under which HP will distribute Cloudera Enterprise. More intriguing, however, are their plans to launch an HP Hadoop appliance at the end of 2012, based on Cloudera’s software, HP Proliant x86 servers and HP storage and networking hardware.

Basically, it’s a preconfigured solution that combines all the software and hardware that companies need to embark on their big data journeys. As Merv Adrian, an analyst at Gartner, has put it: “Providing complete end-to-end solutios for Apache Hadoop deployments will be key to significantly speeding up its adoption within mainstream enterprise environments.”

That’s because, when it comes to unleashing the power of big data, Apache Hadoop has fast become the tool of choice for companies that are already tackling the challenge of ingesting and transforming vast torrents of information. By dividing workloads into many smaller fragments and distributing them across clusters of commodity servers, it’s a highly effective way to get numbers crunched. But it can be difficult to deploy and hard to use.

Apache Hadoop, moreover, is open source, which is where Cloudera comes in: it distributes Hadoop and related Apache Project tools, as well as Cloudera Enterprise, a package that bundles these open source solutions with the company’s own tools for running Hadoop in production environments. In addition, the company offers support, consulting services and training around these products. That, according to Cloudera execs, gives enterprise customers the confidence they need to use an essentially ‘free’, but highly complex product, created and continually developed by a disparate community of developers.

There’s clearly a market for products that enable companies to deal with the many infrastructure challenges involved in creating a big data production environment. After all, many struggle with the deployment and management of big data solutions, and a good deal of specialised knowledge is often required – knowledge that is hard to come by and comes at a premium in salary terms.

But the concept of a Hadoop ‘appliance’ that bundles hardware and software is hardely a new one. EMC, for example, partners closely with MapR (another Hadoop distributor and Cloudera rival) for its own Hadoop ‘appliance’. And since August 2011, Cloudera has been partnering with Dell on a joint solution that bundles Dell hardware (including Dell PowerEdge C2100 servers and PowerConnect switches) with Cloudera’s Hadoop distribution, Cloudera Enteprise and support services.

But, at the very least, the HP/Cloudera announcement means more choice for customers, especially IT decision-makers at companies currently wrestling with big data infrastructure decisions.