An expert advice on what you need from big data analytics, and how to get there.
Business and technology leaders in most organizations understand the power of big data analytics—but few are able to harness that power in the way they want. The challenges are complex, and so are the technologies. Identifying and investing in key principles will help you navigate that complexity to find the right way to tap the growing pools of information available to your organization.
There are six main factors required to get a big data analytics platform right. Lets take a look at each one of them and explain how companies can get their big data right.
1. Blazing speed
Expectations around data are higher than ever. Business users and customers demand results almost instantly, but meeting those expectations can be challenging, especially with legacy systems. Speed is not the only factor in implementing a big data analytics strategy, but it's top of the list. Customers typically need to run queries on data sets that are 10 terabyte or larger and want results in few minutes.
Typical Business warehouse solutions would take 48 hours or more, In today's high speed business world, results after 48 hours is almost useless.
Time to insight is a top priority for any new analytics platform. Companies need to invest in High Performance Computing (HPC) to get the results in few minutes. With newer in-memory analytics systems - such as SPARK or SAP HANA, the wait times can shrink to less than a second!
New solutions are fully optimized, enabling it to provide insights on time to fuel bottom-line results.
2. Scalable capacity
Today, It's a given that any big data analytics solution must accommodate huge quantities of data, but it also needs to grow organically with data volumes. Analytics solution must be able to grow in scale as the data size increases. Today, customers can't afford a "rip and replace" options when the database sizes grow.
Business needs a system that can handle all the data growth in a way that is transparent to the data consumer or analyst - with very little downtime, if any at all. Capacity and computer expansion must all happen in the background.
3. Intelligent integration of legacy tools
Big Data Analytics must work with legacy tools so that business management can have seemless continuity. But it is also important to know which tools must be replaced and when.
Businesses have made investments in these older tools - such as Business Warehouse, Databases, ETL tools etc. Top management is comfortable with these legacy tools. But as data size grows newer data analysis tools will be needed and these new tools will have to work along with legacy tools.
4. Must play well with Hadoop
Big data and Hadoop has almost become synonymous with big data analytics. But Hadoop alone is not enough.While Hadoop is well known, it is built on generic low cost servers, it is also slow.
Hadoop, an open source big data framework, is a batch processing system, meaning that when a job is launched to analyze data, it goes into a queue, and it finishes when it finishes - i.e., users have to wait for results.
Today, Big data analysis needs to be fast - we are talking about high-concurrency in-memory analytics. Companies will still use Hadoop - but find newer ways to run Hadoop without incurring the performance penalties. Newer implementations of Hadoop (ver 2.7.x) and Spark will allow both systems to run in parallel.
5. Invest in data scientists
Organizations must build teams of data analytics experts. Not just hire data scientists, but also invest in tools that allow them to conduct more robust analyses on larger sets of data.
The key to move forward with best possible data analysis solution is to enable data scientists work with actual data sets and not a sample subset. The data analytics development environment must have the scale and size needed to work on actual data sizes, else the answers can go wrong and also leads to longer and more iterative development process.
6. Advanced analytics capabilities
Data Analytics tools and capabilities are rapidly evolving. Newer analytical tools use Artificial Intelligence (AI) tools as businesses move toward predictive analytics.
Big data has moved beyond reporting. Big data analytics is being used to answer very complex questions based on the data in your database. Analytics are now being more predictive, geospatial, and sentiment focused.
The shift toward predictive analysis and other advanced analysis has started. Organizations now—with the way data science has become more and more a corporate asset—there's definitely greater interest in becoming more predictive and more data-science savvy in nature.
Closing Thoughts
Globally, data is growing at a very rapid rate: 40-50 percent per year. In this environment, every business is going to struggle against an overwhelming volume of data. New technologies are there that can help manage data at that speed and scale.
But having a right big data strategy is vital for success. As new tools, technologies emerge, it becomes critical to have the right strategy to incorporate them into the existing eco-system in a seemless non-disruptive way.
In this blog, I have highlighted 6 main aspects of a big data strategy that helps organizations to get its big data strategy right.
Business and technology leaders in most organizations understand the power of big data analytics—but few are able to harness that power in the way they want. The challenges are complex, and so are the technologies. Identifying and investing in key principles will help you navigate that complexity to find the right way to tap the growing pools of information available to your organization.
There are six main factors required to get a big data analytics platform right. Lets take a look at each one of them and explain how companies can get their big data right.
1. Blazing speed
Expectations around data are higher than ever. Business users and customers demand results almost instantly, but meeting those expectations can be challenging, especially with legacy systems. Speed is not the only factor in implementing a big data analytics strategy, but it's top of the list. Customers typically need to run queries on data sets that are 10 terabyte or larger and want results in few minutes.
Typical Business warehouse solutions would take 48 hours or more, In today's high speed business world, results after 48 hours is almost useless.
Time to insight is a top priority for any new analytics platform. Companies need to invest in High Performance Computing (HPC) to get the results in few minutes. With newer in-memory analytics systems - such as SPARK or SAP HANA, the wait times can shrink to less than a second!
New solutions are fully optimized, enabling it to provide insights on time to fuel bottom-line results.
2. Scalable capacity
Today, It's a given that any big data analytics solution must accommodate huge quantities of data, but it also needs to grow organically with data volumes. Analytics solution must be able to grow in scale as the data size increases. Today, customers can't afford a "rip and replace" options when the database sizes grow.
Business needs a system that can handle all the data growth in a way that is transparent to the data consumer or analyst - with very little downtime, if any at all. Capacity and computer expansion must all happen in the background.
3. Intelligent integration of legacy tools
Big Data Analytics must work with legacy tools so that business management can have seemless continuity. But it is also important to know which tools must be replaced and when.
Businesses have made investments in these older tools - such as Business Warehouse, Databases, ETL tools etc. Top management is comfortable with these legacy tools. But as data size grows newer data analysis tools will be needed and these new tools will have to work along with legacy tools.
4. Must play well with Hadoop
Big data and Hadoop has almost become synonymous with big data analytics. But Hadoop alone is not enough.While Hadoop is well known, it is built on generic low cost servers, it is also slow.
Hadoop, an open source big data framework, is a batch processing system, meaning that when a job is launched to analyze data, it goes into a queue, and it finishes when it finishes - i.e., users have to wait for results.
Today, Big data analysis needs to be fast - we are talking about high-concurrency in-memory analytics. Companies will still use Hadoop - but find newer ways to run Hadoop without incurring the performance penalties. Newer implementations of Hadoop (ver 2.7.x) and Spark will allow both systems to run in parallel.
5. Invest in data scientists
Organizations must build teams of data analytics experts. Not just hire data scientists, but also invest in tools that allow them to conduct more robust analyses on larger sets of data.
The key to move forward with best possible data analysis solution is to enable data scientists work with actual data sets and not a sample subset. The data analytics development environment must have the scale and size needed to work on actual data sizes, else the answers can go wrong and also leads to longer and more iterative development process.
6. Advanced analytics capabilities
Data Analytics tools and capabilities are rapidly evolving. Newer analytical tools use Artificial Intelligence (AI) tools as businesses move toward predictive analytics.
Big data has moved beyond reporting. Big data analytics is being used to answer very complex questions based on the data in your database. Analytics are now being more predictive, geospatial, and sentiment focused.
The shift toward predictive analysis and other advanced analysis has started. Organizations now—with the way data science has become more and more a corporate asset—there's definitely greater interest in becoming more predictive and more data-science savvy in nature.
Closing Thoughts
Globally, data is growing at a very rapid rate: 40-50 percent per year. In this environment, every business is going to struggle against an overwhelming volume of data. New technologies are there that can help manage data at that speed and scale.
But having a right big data strategy is vital for success. As new tools, technologies emerge, it becomes critical to have the right strategy to incorporate them into the existing eco-system in a seemless non-disruptive way.
In this blog, I have highlighted 6 main aspects of a big data strategy that helps organizations to get its big data strategy right.
No comments:
Post a Comment