Challenges of Big Data

Today, all most all marketing magazines, journals, seminars, blogs etc are talking about the benefits of big data. While Big Data promises the gold for a marketer - in terms of having real time customer insights and the ability to influence customer's decision, it is not easy. There are major challenges in implementing really good technology platform needed for Big data analytics.

Getting the vital customer insights is more like gold mining - you need to process 3-4 tons of ore and dirt to extract 5 grams of gold. Big Data analytics will also have to run through tones of data first - except that with advancing technology, it is possible to get that insights fast and cheaper than ever before. Companies must work out the major challenges of Big data first - before they can make use of the customer insights.

Challenge-1: Get the right data source

Today, in most organizations, data is distributed in multiple databases. It is common to see 50-60 different databases. In larger companies, it is common to see data scattered across 100+ databases. So identifying the right data sources for the big data anlytics is the first big challenge. Given the dynamic nature of business today - especially in high-tech world, with mergers and acquisitions, the number of data sources just keeps increasing.

Integrating multiple sources of data is still a big technical challenge and is yet to be solved by IT.

Challenge-2: Choose the right technology.

Another challenge is to select the right technology platform. Today there are several big data technologies in the market: Hadoop, Mongo, MIKE 2.0, NoSQL etc. These new technologies offer an opportunity to analyze very large amounts of unstructured data at a scale and speed that was simply not possible just a few years ago - but selecting the right technology platform is based on the type of analytics you need - which depends on the data sources (see challenge-1)

Challenge-3: Get the right storage IT infrastructure

Big Data analytics needs big data storage and networks to move the big data around. Since the data exists in multiple databases, consolidating data will require a much bigger storage capability and a faster network to move the big volumes of data around the corporate network. In many cases, the current IT infrastructure is inadequate to meet the demands of big data. Data storage & network bandwidth management is still a big challenge for most companies and this has to be resolved before companies can benefit from big data analytics.

Challenge-4: Have an action plan

Big data analytics is useful only if one can act on the analytics. Since most of the real time analytics need real time action, there must be a plan on what to do with the analytics. Today, Big Data is more like a technology hype than a complete business solution with a clear path to business value. So before companies can start on their big data project,  they have to separate the hype from reality and have a clear cut plan on what analytics to run and what to do with it.
Today, most companies don't even act on the customer feedback that is readily available. Marketers don't look at basic Web analytics to act on the insights that have been at their fingertips for years. In such case, should the company be willing to act on the insights you could generate from a Big Data initiative! And for that there must have a plan.

Challenge-5: Avoid the data trap!

Data analytics is not everything. Customers have emotional reactions too and in many cases, customers do not really know what they really need. Henry Ford once said "If I had asked people what they wanted, they would have said faster horses." Customer insights and marketing plans must not rely solely on analytics!

Closing Thoughts

Big Data analytics is a key resource for any marketer and can be a competitive advantage. There are several success stories of businesses using big data to improve the business. However, most companies will hit road blocks on the journey to this land of gold. Today, Big Data is more a piece of the technology than a complete solution with a clear path to business value. Only big technology companies have been able to exploit the benefits of Big Data.

Customer behavior is complex and customers are emotional. So no matter how robust the analytics  is, it can provide only a part of the picture. It takes human intuition - tuned by big data analytics to really make a difference and create a success story.

Why Small Companies are more Innovative?

In my article Agile Innovation, I had mentioned how companies can increase their innovation capabilities, but this is easier said than done. Large companies cannot change their organizational structure to implement agile innovation. Big companies use acquisitions as path to innovation. This is very well explained by Steve Blank in his article titled: Why big companies cannot Innovate. In the article, he rightly observes that "It's not surprising that younger entrepreneurial firms are considered more innovative."

As an innovation coach, I have always looked at innovative organizations and observed that there are few things small and entrepreneurial firms do right - which makes them innovative. Essentially, startups have an ecosystem which encourages collaboration and innovation. In my previous article: Agile Innovation, I had mentioned that the first step is to create an Agile ecosystem for innovation. But then how do companies create an innovative ecosystem? Or what are the characteristics of an innovative ecosystem?

This article will highlight six aspects which create an ecosystem that fosters Innovation.

1. No organizational Silos
2. Value Diversity
3. Challenge existing Assumptions
4. Everyone will contribute
5. Identify, Nourish and exploit valuable resources
6. Hammer the message home

If one were to walk into any of the startups anywhere in the world - be it in Silicon Valley or Israel or Bangalore, you will see these six aspects.

No organizational Silos

Startups are often have flat organizations and do not have organizational silos. The lack of organizational structure creates a perfect ground for people to collaborate and innovate. Flat organizations also helps in eliminating beauracracy. With little organization hierarchy, ideas are measured on the inherent value. This eliminates organizational politics and faster innovation.

Value Skilled People & Diversity

Startups value skilled people. In places like silicon valley - there is wide diversity as people come from different parts of the world. So when you look at most startups, you will find people from different backgrounds, ethnic cultures and skill sets. This diversity creates a fertile ground for new ideas and challenging current assumptions. Diversity also provides different view points to the same problem and avoids group think.

Challenge existing Assumptions

Startups are often founded on the notion of an opportunity - which in many ways challenges the current status-quo. Challenging the currently held belief or assumptions will require new thinking and ideas. Successful innovation emerge with the older assumptions are proven wrong and new opportunities are identified. Founders of startups are also have rebel streak in them, and they tend to attract similar people so that they all can effectively challenge the current status-quo.

Everyone will contribute

In Smaller organizations everyone knows everyone else. So there is no room for slackers or freeloaders. The transparency in the organization, the collective work environment and a common message will make everyone contribute towards innovation and all contributors will be rewarded.

Identify, Nourish and utilize creative talent  

Small organizations often value of talent more than larger organizations. As individual's skills are easily known in a smaller organization, the leadership or founders are able to identify, nourish and utilize the creative talent of its people. As the individual's skills are better recognized, people are more motivated to do more and be more creative.

Hammer the message home

Founders at a startup are easily accessible and are constantly talking about the innovation objectives, the market plans and how their small company can win. Here all employees are constantly reminded on the importance of innovation and have targets to meet. The spirit of innovation is everywhere and employees become truly innovative.

In large companies, employees are mainly working on their main operations task & innovation becomes a part of annual leadership speech. Employees in large organization are often told to be innovative and work on innovation projects in addition to their regular day jobs. And just does not work.

Closing Thoughts

Smaller organizations are inherently more creative and innovative than larger organizations. This implies that large companies must reorganize into smaller innovation groups and that can help speed up innovation - particularly in Agile Innovation. Alternatively, big companies can buy innovation by acquiring the small, innovative startups and then provide the benefits of large organization scale and reach. Companies such as Cisco, EMC, Apple, Microsoft, IBM etc. have created special groups that are skilled in identifying and acquiring small and innovative companies.

Challenges with Electric Cars - Tesla continues to struggle

In my previous Blog: Challenges with Electric Cars, I had outlined the main problems with electric cars an year ago. Even now, most of those problems exist - and can be seen with the troubles at Tesla Cars.

The fourth quarter results of Tesla motors showed a bigger loss - $89.9 Million and the new car sales are still not enough to break-even. Though Elon Musk gave a rosy picture on the fourth-quarter results - talking about record bookings and increased production & revenues, the troubles seems to be deep rooted and persistent.

To begin with, the company's financial position is messy, piling losses has converted Tesla into a  cash burning machine. Tesla has been racking up losses and burning through cash so quickly that it was forced to issue a secondary stock offering in the hopes of drumming up a rescue operation

Another problem, a deeper and more dangerous technical problems with battery technology still remains. Tesla cars use the same Lithium-Ion battery technology as Boeing used in 787 Dreamliner. Since the batteries are extensively used in commercial airlines, the battery problems were exposed faster.

The Lithium-Ion batteries have another major technical problem - popularly called as "Brick". Once the battery is ever totally discharged, the car becomes a "brick": a completely immobile vehicle that cannot be started or even pushed down the street. The only known remedy is for the owner to pay Tesla approximately $40,000 to replace the entire battery.

See: Tesla Motors' Devastating Design Problem

Given the high cost of  replacing battery pack, not all first time customers are likely to become second time customers and not all bookings are being converted into actual sales.

There are two other hidden challenges: Tesla plans to expand its network of super-fast, free charging stations along highways. The next charging stations will appear on the East Coast and in Texas, Washington, and Illinois. Building this network of charging stations will be a big drain on the company's financial position.

Another challenge for Tesla is the falling crude prices. With USA, Iraq, & Libya increasing crude oil production - the average crude oil prices are dropping. This coupled with countries dropping subsidy to solar power generation, implies that cost of green technologies will remain high and noncompetitive for years to come. So beyond the initial hype of a fast electric "green" car, Tesla will have no markets - which implies that the proposed increase in sales volume is unlikely to happen.

With increasing costs and lower sales, Tesla will eventually have to visit Chapter-11!

Also see: Challenges with Electric Cars

Agile Innovation

Continuing on the product review of Blackberry 10, and the possibility of Blackberry regaining its market leadership. It is clear today that even in the best case scenario Blackberry will have to compete for the third place! Even the race for third place will be tough - with Microsoft/Nokia, LG, Sony & HTC competing for the same third place. As the first & second place positions are already decided, the winner for the third place position will have to bleed in the brutal price war with HTC & LG, so even winning the third place will be a hollow victory. But then, Blackberry 10 and BES 10 are truly innovative products. So what went wrong? How can an innovative company introduce a truly innovative product and still fail?

The answers to these questions is: Speed of Innovation, or the lack of speed in innovation. By the time Blackberry developed Blackberry 10 OS, BES 10 and Z10 handset, Samsung released close to 30+ models of Smart phones and 10+ models of tablets. Apple released four new versions of iPhone and Tablets, developed Siri and iCloud and expanded its retail network globally. In other words, Blackberry was too slow for the market.

Today, Samsung has set the a scorching speed for introducing new products and Apple has set the high bar for quality and innovation. So for Blackberry to regain its market dominance, it must beat both Samsung in terms of new product launches or beat Apple in Total Customer Experience. Which is a tall order for any company, and hence Blackberry is forced to fight for third place.

Using Blackberry as an example, I want explain one of the principles of success with innovation called Agile Innovation.

What is Agile Innovation?

In a simplest terms Agile Innovation is innovating faster than the competition. But this is a lot harder and lot more complex than it sounds.

Innovation is an outcome of creative thinking (1% inspiration) and hard work (99% perspiration). So there is a built-in paradox with Agile Innovation! One cannot be fast and be creative at the same time. For example, if you try to write a creative poem in 30 minutes - I am sure, even with your best effort, it will be a lousy poem (or a copy of a popular poem/song).

The speed is spurred by competition and ambition to succeed, it is extremely difficult - except by sheer chance - to come up with a truly creative new idea under pressure.

Creative work needs time! Period.

But the market demands new & innovative products to be introduced faster than ever before.

So enter the world of Agile Innovation. A whole new process centric approach to innovation.

Agile Innovation is process of releasing new and innovative products at a regular intervals - while each release of a product has an incremental innovation over the previous release.

The concept of Agile Innovation has its roots in "Kaizen" the Japanese principle of continuous improvement and Agile product development.

With Agile Product development, companies can introduce new products at regular intervals and with Kaizen, the product is improved in each release. So Agile Innovation is method that ensures that each release of the product has innovations and it is possible to make course corrections - if the innovation is not well received. So instead of having one big-bang innovation - Agile Innovation splits it up into a number of smaller innovations spread over time, which enables market corrections when needed.

Another way to look at Agile innovation is to compare it with building a BIG Pyramid Vs moving one stone at a time. Each stone helps build the final pyramid - and in a phased manner - with the option of changing the plan if needed.

Need for Agile Innovation

Today, the markets and investors demand higher earnings and faster innovation. Companies have to cut the lead time required for innovation by 50% or more and offer differentiated products. Throughout business history, it has been noted that the first movers get the advantage of higher market share and higher returns. Also having multiple product releases leads to faster feedback from customers and enhances the learning cycle. Finally, Agile Innovation increases the capacity for innovation.

Agile Innovation is best suited for new product development - particularly in areas that has higher levels of uncertainty. Be it totally new technologies such as Cloud computing or renewable power etc. or in areas which is seeing massive growth and intense competition - such as cell phones.

For products that have long development cycles,  agile innovation is risky. Therefore it is best to try to break the long development cycle into shorter cycles - with the first cycle delivering the minimum features needed to release the product.

For stable & mature product areas - where there is little new development and products are mostly in sustenance mode, then Agile innovation is not the right choice - as the cost of release (& development) is often much higher then the incremental revenue.

How to implement Agile Innovation? 

As it becomes clear for companies that Agile innovation is the way to go, it raises the question: How to implement Agile Innovation. Implementing Agile Innovation is not easy. It takes a strong commitment from the entire organization - from the board of directors to front-line employees. The top management must commit to the required investments and organization changes. The front-line teams must be trained in agile methodologies and empowered to act independently. Top management must create adequate financial structures to closely monitor the success/failures of agile product releases. In short, the entire organization must be geared up for the agile world.

Its almost impossible to list and document all the steps needed, but here are the major ones needed.

Step-1: Create an agile ecosystem

Agile innovation relies on active interaction between users and developer. The first step is to create an  environment that allows for free interactions. Empowerment and collaboration are an absolute must. Rigid process or hierarchy does not help.

Creative ideas do not come from a lone inventor. Most of the creative ideas come constant interactions between creative people. Ideas take time to mature and evolve from a vague concept to something concrete that can be implemented.

Agile Innovation requires responsiveness to rapidly changing business environments and customer needs and so empower front-line product development staff to collaborate across organizations silos - this will encourage meeting the business objectives with a skilled, motivated, and self-organizing teams.

Step-2: Create an Innovation Pipeline

The long development cycle and long product life cycle requires  arduous research and development cycles which increases risks. With Agile Innovations, firms must shift their strategies to a more rapid understanding of customer needs and quick delivery of new products and services.

This means that firms must have more products and services in the innovation and development pipeline than ever before, and must launch more new products and services to keep pace with demand and competition.

With Agile Innovation, greater profits is possible with a shorter period of time as companies can respond faster to competition and deal with faster obsolescence of products. This, in turn, reinforces the need for the efficient use of ideas, which will drive new product and service creation.

Step-3: Focus on a single user persona

Since the basic principle of Agile Innovation is to opt for a larger number of releases instead of one big release, each release will have to concentrate on a small set of features. So each release must concentrate on meeting the needs of a particular user group. The first release concentrates on early adopters, the subsequent releases then concentrate on widening the market base by concentrating on features/innovations needed to please this new market segment. The second release concentrates on customers in the adjacent markets while retaining the early adopters and so on.

Main benefit of focusing on a single user persona is that the scope and hence scale of each release is minimized - which in turn lowers the risks involved and maximize profits. In many cases the exact requirements of a particular user group may not be well understood or known - say for example while the company is developing the product in the US, the target customer is in Japan. In such cases the new product releases may have to be tweaked in subsequent releases through focused experiments.

Step-4: Develop Focused Experiments

New product development is inherently risky. So when the user requirements are now clear or when the core technologies are not well developed, it is better to have some focused experimental products to test the market or core technologies. This way, the main product line is not impacted by a failure of an experiment. For example, Samsung released Galaxy Note - to experiment the market reaction to phablets - the hybrid between smart phone and Tablets. Developing a separate product and separating it from the main Samsung Galaxy S series was a great way to experiment.

Step-5: Iterate, Iterate & Iterate

Traditional product development tends spend a lot of time in defining a perfect product and then spend a whole lot of time and money executing it. By the time, the fist customer views the finished product - a while lot of time and money would have been spent. Like in case of Blackberry - which took about 2 years to develop. And by then, the market requirement could have changed or the competition could have changed the market landscape.

Agile Innovation calls for having multiple releases - releasing in shorter time and with each release learning from customer feedback - which is then applied to the next version of the product. This ensures that the product is at all times kept close to the needs of the customer. Which makes for a better product.

Agile innovation allows for several iterations to run in parallel - which in turn speeds up innovation. New technologies developed in each iteration can be applied in subsequent iterations. The parallel development increases the speed to innovation - while minimizing risks to other projects.

Agile Innovation is characterized by experimentation, and rapid prototyping. Each release is like a beta release which is used to test the product and improve on it. This approach allows the companies to experiment a lot and scale up on things that work.  Agile Innovation is inherently adaptive and allows for incremental development that minimizes risk, and keeps better control on costs.

Closing Thoughts

Agile Innovation has its roots in Agile project management - but it is not processes. It is a compete redesign of innovation, designed to help organizations react better and faster to changing market landscape.

Agile Innovation focuses more on outputs, outlining a product vision that has a strong customer focus  and minimizing the risks.

As a shining example of Agile Innovation, Samsung, Today, sells more mobile phones, and more smart phones, than any other company. It competes in all markets, from the high-end down, and is pouring its record profits into expansion and advertising. Samsung is now among the most valuable brands on the planet.

Samsung rose to prominence by out-grinding rivals in commodity markets, and it approaches phones and tablets the same way, by quickly pumping out handsets with incremental tweaks and improvements.

We are in the middle of a paradigm shift towards Agile Innovation. Any company that still relies on traditional innovation process will suffer. Companies will have to tweak its existing business models and methods to support agile innovation.  Now is the time to determine new business models, use of customer insights and feedback and to create new products and become more nimble, more agile and more innovative with Agile Innovation.

=======================

Trademark Disclaimer

Product names, logos, brands and other trademarks referred to in this article are the property of their respective trademark holders. These trademark holders are not affiliated with this blog or website. They do not sponsor or endorse this article.   

Yahoo! - All Hands on the Deck

Recent memo from Yahoo CEO Marissa Mayer's - banning work-from-home and calling all employees to be in office has come under sharp criticism from media and experts.  I was initially surprised and dismayed by the news - but then I realized that it was the right think to do and I have full support to Marissa Mayer's decision. Over next few days I read lot of criticism on her decision. The worst one came in Networkworld.com's blog By Zeus Kerravala:  Marissa Mayer: CEO of the year 1955 

While experts and mavens criticize Marissa Mayer's decision, the same people will agree on the following:

1. Nature of Internet is changing from PC to Mobile world.

2. Yahoo is not a leader in Mobile Internet.

3. Yahoos's current revenue is tied to PC based Internet

4. Yahoo is at the risk of becoming irrelevant.

By the end of 2013, majority of Internet traffic will be from mobile devices - Smart Phones and tablets. In this new world, Yahoo faces several challenges. Today Yahoo does not feature in top ten Apps for iPhone or Android or Windows 8.0 or Blackberry. In short, people who use Mobile Internet do not need Yahoo! Thus making Yahoo irrelevant to the Internet.

Yahoo is also becoming irrelevant to the PC based Internet. Taking my life as an example, I have been using Internet ever since the days of Archie & Gropher. Initially, when World-Wide-Web was introduced, I was among the earliest to have a Web email - Rocketmail & Hotmail. Rocketmail was taken over by Yahoo. I also became a heavy user of Yahoo search (replaced by Google), Yahoo Photos (replaced by Piccasa), Yahoo Groups (replaced by Google Groups), Yahoo Finance (replaced by Google Finance), & Geocities (replaced by Blogspot). But then Yahoo decided to shoot itself on its feet  and killed Rocketmail, Yahoo Photos, & Geocities.  As a result, today, I do not use any of the Yahoo Internet properties at all. So for me Yahoo is irrelevant.

Yahoo's slow slide to irrelevance is seen in its decline of market share and revenue, and Marissa Mayer is acutely aware of this. She has been asked by the Yahoo board to turn things around and make Yahoo a leader of Internet once again.

In order to achieve such a challenging goal, Marissa has to navigate rough seas and take tough measures to transform Yahoo. As a leader of Yahoo, she has just called for "All hands on the Deck" - calling all Yahoo employees to act with deep sense of urgency.

Marissa has already laid out the next steps in the transformation of Yahoo.

Step-1: Create a sense of urgency

While there is no doubt that Yahoo is in a crisis. Yahoo needs to transform quickly to stay relevant and protect its revenues. Marissa is also leading the charge to create a bigger sense of danger within its employees. The recent announcement of plans to kill old products is the best way to kill any sense of complacency and create a sense of urgency. I would compare this to a old military strategy: "Burning your bridges to win the war"

See: Yahoo's new strategy - Kill Old Products (http://www.businessinsider.com/yahoos-new-strategy-kill-old-products-2013-3)

Once a sense of urgency is created, Marissa Mayer should ensure that all hands agree and work towards the transformation .

Step-2: Have a strong Moral purpose with empathy

Any change is hard, Marissa Mayer is driven by resolute purpose. She has a deep change insight - vision of what the future of Yahoo will be, and she must not let that insight stay with her - it must be shared by the entire organization. To succeed, she must be persistent with her vision and ideas - a leaders simply do not, and would never, give up in face of opposition to those ideas.

This requires empathy and respect to deep human values. In order to win over others, leaders have to possess the impressive empathy that enables them to understand why people disagree and then figure out how to relate to the objections and make them see things in they leaders see it.

So far, Marissa Mayer has been able to explain the real reasons for calling all employees to office and taking away the work-from-home option. I am sure, Marissa Mayer has articulated the reasons for her actions and have won over the internal critics. There are stories on web - about employees agreeing with her and some being delighted with this decision.

A leader cannot call for "All Hands on the Deck" without a strong purpose.  In case of Yahoo, the purpose is very clear and articulated in Yahoo's product strategy - A 21 page product plan for next three years which can be summarized into five main action points:

1. Infuse deep personalization using science and data into every consumer and advertising experience we build.
2. Delight our customers with best-in-class products, iterating frequently for constant improvement.
3. Build for connected devices first with localized, in-context, multi-screen experiences in mind.
4. Power real social relationships with features that enable conversations around content.
5. Build a digital media ecosystem that creates a premium marketplace for advertising and content and distributes Yahoo! experiences across the Web.

- See more at: http://allthingsd.com/20120927/what-will-marissa-do-heres-yahoos-2011-three-year-21-page-product-strategy-plan-that-reads-a-lot-like-mayers-new-vision/#sthash.d8z77ehG.dpuf

Once the purpose is articulated, the strategy is defined, the next step is to execute on it. It is crucial for Yahoo to stay on course against all odds and in early stages of execution, display empathy to opposition and win them over. Leaders in successful organizations focus on a small number of core priorities, stay consistent on the message, and develop people who convert the vision into reality, all the while making course corrections as necessary.

Step-3: Motivate People to Collaborate 

Once all hands are on the deck and their purpose is well understood, Leaders must motivate people to collaborate and be more creative to achieve greater results.

Marissa seems to have brought some ideas from Google. People tend to be more creative and successful when the collaborate with others. Once all employees are meeting face to face in offices, they can bounce off ideas from each other, feed on other people's ideas and create new things faster. Google spends enormous amounts of money to create the best workplace for its employees  - which is collaborative workplace and the results are obvious. I am sure that in next few months, we will see a rapid transformation in work place at Yahoo and soon Yahoo will be ranked as one of the best places to work.

Closing Thoughts

"Wisdom is the ability to act with knowledge while doubting what you know" - Pfeffer and Sutton

Leaders must always be confident of the outcomes. As a change leader, Marissa Mayer has displayed a strong confidence in her actions. She understands that failure is not an option for Yahoo and to succeed she must act wisely. Everyone at Yahoo knows that the future is unpredictable and yet Yahoo must hurl itself into this great unknown, solve complex problems in the process while exude confidence in the end result.

I would like to compare the situation at Yahoo with the original Star Trek's tag line - "Space: the final frontier. These are the voyages of the starship Enterprise. Its five-year mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no man has gone before."