Big Data: Success Stories Beyond Hadoop

Author

Bernard Marr, Contributor

November 26, 2015

John Schroeder is the co-founder and CEO of MapR, one of the big names of the Big Data revolution and a key provider and enabler of many of its biggest success stories. I talked to Schroeder about the big data industry, their business, the most interesting use cases and his views on the future.

MapR was founded in 2009 and quickly established itself as a key vendor of Hadoop – the open source data and analytics framework responsible for powering much of Web 2.0. Today, the company provides Big Data applications across many industries from healthcare to finance.

MapR is doing well, it has just reported a year-over-year expansion rate of 157%. MapR was responsible for developing Amazon’s Elastic Map Reduce service and is a contributor to many open source projects including Apache Pig, Hive, HBase and Zookeeper. As a result it has deployed storage and analytical solutions widely across government and industry. I asked Schroeder which current ones he is particularly excited about.

He said “The interesting use cases are those which are moving towards real time operational uses.”

One example he mentions is Aadhar, the Unique Identifying Authority of India, which is the world’s largest national identification system and stores biometric identification details on over 750 million citizens used to administer taxes and distribute welfare. It is seen as essential in a country where infrastructure is underdeveloped in many areas and over $1bn a year is lost to embezzlement.

Aadhar has a huge database with very important records, which can’t be lost or corrupted, as well as big analytics demands.

Schroeder says: “You have to make sure someone isn’t assembling a fake ID by using your fingerprints and my iris scan and someone else’s photo – and you need to be able to authenticate that in milliseconds.” He goes on to say that “With 750 million people in the system, the system can be authenticating identities four or five million times per minute during peak time.

Another of MapR’s large scale deployments is at American Express where Hadoop, machine learning algorithms and other big data tools are deployed to protect them and their millions of card holders against fraudulent transactions.  The multi-petabyte system is able to analyze transactions and pick up potentially fraudulent patterns in less than 2 milliseconds, which allows approvals to be made in rear real-time, at the point of sale.

Finally, Schroeder flags up a use case that comes from the healthcare sector, where MapR was working with a major cancer center to improve patient treatment using Big Data to provide personalized treatment programs.

Schroeder says “If you think about when you go to your doctor, they’re relying on their experience, and the white papers they’ve read and the research they’re doing, to stay on top of current treatment options. But they’re not very personalized to you. You’ve got a certain genetic make up and you’re a certain age with a certain BMI, you’ve potentially got other maladies such as diabetes or high blood pressure.”

Big data is enabling the medical profession to take into account all these different factors to provide the most effective treatment for individual patients.

The push towards real time analytics within Big Data projects is one of the reasons why MapR is looking at other technology beyond Hadoop (I have written about the limitations of Hadoop when it comes to real time data processing before, which you can read here). Schroeder tells me that the market is developing very fast and technology for storing and analyzing data is constantly changing. Schroeder says “We’re contributing to projects like Drill, and we are early supporters of Spark, but there are also things we can’t get done in open source.” What MapR has been successful at is extending the open source software with a platform layer that provides value to the customers.

Customers – MapR has over 700 now, compared to 300 at the start of 2014 – are the key, Schroeder said, when I asked him what the secret was to the company’s success.

“Hands down it’s been our focus on customer,” he tells me. “We really, really listened carefully – Srivas [co-founder and CTO of MapR] has a great vision for architecture and I’ve been able to identify and define products which have been successful in the market over the years, but it’s all been based on listening to our customers and making them successful with our product. So while we drive a lot of innovation, we’re doing that by being hand in hand with our customers and understanding what their challenges and use cases are.”

MapR has just appointed a new president – former Oracle SVP Matt Mills – and is gearing up for an IPO later this year. I asked Schroeder about the recent appointment of Matt Mills as company president, and what he will bring to the company.

Schroeder said: “We [Mills and M C Srivas] really felt Matt was someone we could partner with. Despite being at Oracle, he has very different views on company culture than someone you would associate with Oracle – much more in line with my views.

“The way we’ve set it up, he’s got most of the go-to market facing responsibilities – sales, sales engineering, customer support and services – and I keep product management, engineering and so on. We see taking on Matt as additive – it’s a growth for the company to have that sort of resource.”

I am excited to see more success stories and use cases coming out of MapR and look forward to catching up with Schroeder in the not to distant future.

This article was written by Bernard Marr from Forbes and was legally licensed through the NewsCred publisher network.


Comment this article

Great ! Thanks for your subscription !

You will soon receive the first Content Loop Newsletter