Big Data as an Organizational Center of Excellence

Introduction - Managing Data Across Large Enterprise

Today, enterprises are looking for Integrating Data to Support Analytics Based Decision Making and are still facing massive challenges. The biggest challenges they have is that data is located in silos, and yet large volumes of data are being generated are still managed in silos.

Thanks to new technologies such as IoT, there is an abundance of data being generated. Enterprise information systems, Networks, Applications and Devices that churn out huge volumes of information are awash in Big Data.

But enterprises are unable to make best use of this data as their internal organization — the network of people & business process are operating in isolation and many analytics efforts that only take into account information from a single silo - that delivers results in a vacuum - thus prevent them from making better business decisions.

The best way to lower costs of managing big data and leveraging this data for actionable insights - is to have a  pan-enterprise Big Data strategy.  (Also see Getting Your Big Data Strategy Right)

The best way to solve this problem is to create Big Data as an organizational center of excellence. This special group that can cut across silos, take ownership of all data and create new opportunities for operational excellence with Big Data.

Big Data as an Organizational Center of Excellence

Managing all data across the entire organization will improve efficiencies and services while lowering costs of mining this data.

Organizational BU's can now use this centralized data for actionable insights that can help them make better business decisions.

As business leaders recognize a pan-enterprise Big Data effort provides much more meaningful insights because it's based on an integrated view of the business. Yet they are faced with the challenges of siloed information systems. The current IT implementations and business process prevent data sharing and access to data.

In this article, I will present a solution to address the challenges of data silos and data hoarding.  As a enterprise wide solution, I shall present a new CoE for Big Data - a solution for breaking down data silos and discuss the key benefits of the solution.

Big Data CoE will take ownership of all data and present an integrated data for data analysis improves performance across the global enterprise. Big Data CoE can thus deliver long-term return on investment (ROI), enabling business leaders to develop a solid data for making better Analytics Based Decisions.

Understanding Data Silos and the Pitfalls of Data Hoarding 

Before I dive into the details of Big Data CoE, let take a quick look at data silos and dangers of data hoarding.

In large, global enterprises, individual business units (often referred to as BU's) are notorious and pervasive. Data silos are born in this toxic environment - where individual BU budgets procurement processes create data silos.

Group IT resources which are often scarce and are often designated for specific BU functions.   Each individual BU has no financial incentive to share data and IT resources. As a result, data collect by each BU becomes siloed and data is hoarded in each BU's IT systems and is not shared.

This results in organizational deficiencies where the entire organization suffer from redundant systems and inefficient decision-making. Because enterprise information systems remain segregated, data is walled up in departmental databases and applications. With valuable assets trapped in silos, BU's are unable to leverage data to improve processes, workflow or service delivery.

While data silos are created by operational and technical challenges, data hoarding is a result of insular agency cultures that encourage autonomy and self-reliance, as well as stringent compliance mandates for securing data, especially in this era of risks from data leaks/breaches and liability lawsuits.

In this environment, "business data" becomes "OUR DATA." Data hoarding trumps openness and sharing.

The impact of data silos and data hoarding is quite devastating. Without data sharing across BU's, each BU maintains its own view of business and there is no holistic view of a consistent global business. There is no integration of relevant data and this leads to missed opportunities and wrongful expenses and wastage, delays in discovering fraud, waste & misuse of money.

In addition, critical decisions are made with partial data - that leads to unproductive staff and duplicated efforts - which leads to wastage. Budgets are drained by the cost of managing and maintaining complex and redundant information systems, applications and system interfaces.

Finally, data silos and data hoarding weaken security & compliance efforts. It's harder to ensure the security and privacy of information as it moves among computer systems and databases, and can lead to noncompliance with critical regulations. (PCI-DSS, SOX, etc)

A Holistic Model: Big Data CoE 

Envision a pan-enterprise model for managing Big Data as an organizational center of excellence, A competency center whose core focus is to manage Big Data across the organization and provide right set of tools & infrastructure for business analytics.

Big Data CoE is created with a common focus to manage data and develop new technologies and architectures to analyze big data for making better business decisions

When Big DataCoE model is applied to the enterprise, we can instantly see the following benefits:

• Data is treated as an organizational asset.
  Treating data as an organizational asset, CoE develops and fosters a collaborative environment for users across BU's to meet and exchange ideas, discuss new projects and share best practices.
  
   
• Data is managed separately from IT in terms of strategy, organization, resources, purchasing and deployment. This frees up enterprise IT from handling the challenges of Big Data. Data can reside in-house systems or on public clouds.
  
   
• Distinct processes are developed for collecting, consolidating, managing, linking, securing, sharing, analyzing, archiving, publishing and governing data.
  
   
• Analytical expertise is shared among individual departments, which relieves them of the burden of independently recruiting their own talent and developing unique solutions.
  
   
• Data is aggregated, shared and analyzed using a single, enterprise-wide data platform. A unified system of enterprise data management and analytics tools ensures seamless integration of data repositories with analytic tools, provides a common user experience with access to all types of enterprise data and builds end user engagement in data-driven decision-making.

Unlike siloed data, an enterprise wide approach to data provides all BU's with a single version of the truth. With this integrated, holistic view, decision-making involves all relevant, consistent data, regardless of data ownership.

Creation of Big Data CoE is Transformative

The biggest benefit of Big Data CoE is that it transforms business operations.

Big Data CoE leads to business transformation:

1. More efficient Enterprise. When used at the enterprise level, Big Data can reduce fraud, waste and misuse of funds, Enhance communication and coordination between BU's, Improve management of Big Data, and Identify key business trends across the entire enterprise.
    
2. Faster & Better Decision Making. A complete view of each BU's data, CoE can offer the most appropriate data management & analytical services by identifying patterns that might otherwise be missed. They can also eliminate data processing errors and duplicative data entry. Provide consistent procedures and processes eliminates waste of valuable time and speed up decision making.
    
3. Stronger compliance efforts. When data management is integrated, it makes it easier to implement data compliance mandates across organization. Entire enterprise data can be made secure and compliant even when it is shared across BU's.
    
4. Cost reduction. More efficient data management, analytics & workflows, compliance, security and  service delivery - all lead to cost reductions. By consolidating data analytics efforts under a single CoE, additional savings can be realized because departments don't have to procure and manage their own systems or hire department-specific data scientists and analysts.

To realize the benefits of an enterprise approach to Big Data, Enterprises must adopt a comprehensive approach that leverages appropriate tools and techniques.

Closing Thoughts  

Big Data can provide tremendous business advantages by for improving business productivity,  business decision making and service delivery. But Big Data can only live up to its true potential only if analytics programs are implemented thoughtfully and skillfully.

The strategic use of Big Data and data analytics technologies and tools requires considerable innovation, creative thinking and leadership.

The "silo mentality" has to be broken up and data needs to be shared across the enterprise as a common asset. Having a CoE manage all of Big Data allows enterprises to holistically manage, share and leverage data for faster decision making and service delivery.

Big Data CoE helps enterprises and its BU's to rethink and retool the way they collect, manage, archive and use data. CoE will enable Bus to work together and share information - this leads to better decision-making, faster service delivery and develop an enterprise wide approach to managing and using Big Data.