Tuesday, February 24, 2015

EMC's DataLake Foundation


On February 23rd 2015, EMC announced the "Data Lake Foundation" - which is suite of EMC products and solutions to build a rock solid Data Lake - which is the foundation the supports all big data analytics.

The rise of big data and the demand for real-time information is putting more pressure than ever on enterprise storage.

Big data analytics needs & creates massive volumes of data, This unprecedented data growth - which can quickly overwhelm existing storage systems. Over last one year, EMC has been building storage systems to address the specific needs  of big data.

In 2015, EMC announced  Data Lake Foundation strategy - which is based in products like EMC Isilon and EMC ECS (Elastic Cloud Storage). These storage systems are designed to work with HDFS (Hadoop Files Systems) and is easily integrated with Pivotal, Cloudera & Hortonworks stack data analytics tools - thus make it simple to store and  analyze massive volumes of data.

EMC has certified DataLake Foundation to work with the rich analytics tools from vendors: Pivotal, Cloudera and Hortonworks provide. Pivotal and EMC have worked together to test, benchmark and size the Data Lake Apache Hadoop solution.

Isilon OneFS 7.2 OS will support newer and more current versions of Hadoop protocols including HDFS 2.3 and HDFS 2.4 delivering faster time to insights.  It will also support for OpenStack Swift to support both file and object – the unstructured data types that are growing the fastest.

EMC's DataLake Foundation makes it easy for enterprises to run their analytics tools; Helps eliminate storage silos and provide simpler ways to store and manage data so they can focus efforts more toward gaining insights and value from their data.

Here's what the DataLake Foundation brings to the enterprise:


  1. Efficient Storage: Eliminates storage silos, simplifies management, and improves utilization.
  2. Massive Scalability: Built from scale-out architectures that are massively scalable and simple to manage.
  3. Increased Operational Flexibility: Multi-protocol and next-generation access capabilities support traditional and emerging applications.
  4. Enterprise Attributes: Protects data with efficient and resilient backup, disaster recovery and security options. Enterprise class data protection to maximize availability and security options to meet business requirements.
  5. In-Place Big Data Analytics: Leverages shared storage and support for protocols such as HDFS to deliver cost-efficient, in-place analytics with faster time to results.


Two products from EMC portfolio that form the Data Lake foundation are EMC Isilon and EMC Elastic Cloud Storage (ECS).

EMC Isilon provides an enterprise-scale, file-based Data Lake Foundation with the ability to run traditional and next-gen workloads. Starting at 2PB, scaling upto 50PB per cluster, Isilon provides a great balance of performance and capacity for analytics workloads.

EMC ECS is the scalable Object storage for next generation of modern applications. ECS delivers geo-distributed high availability, nearly infinite capacity for big data analytics - provided on commodity storage.

With ECS and the new Isilon platform and features, customers have everything they need to store, protect, secure, manage and analyze all unstructured data now and is built to scale out for all the future needs.

Business Benefits


EMC DataLake Foundation is replicating VBLOCK strategy, where all the components needed for Bigdata analytics is pre-configured, and comes with Pivotal HAWQ subscriptions and Pivotal HD.

This simplifies deployment of all BigData analytics programs, while providing EMC's enterprise grade support, nearly infinite scalablility, and data security.

Resources:



  1. IDC Lab Validation Brief.  
  2. Pivotal Blog: Pivotal and EMC Come Together To Shore Up The Data Lake 
  3. EMC Isilon x410 release 
  4. Pivotal HD 
  5. Pivotal Big Data Suite  
  6. Pivotal webinar: Querying External Data Sources with Hadoop 
  7. Hadoop for the Enterprise 

1 comment:

david said...

EMC's DataLake Foundation is really a big achievement for EMC Partner. Now we can facilitate our customer in a big way.