4 Stages of Developing a Data Lake

Companies generally go through the following four stages of development when building a data lake:

Why Edge Computing is critical for IoT success?

Edge computing is the practice of processing data near the edge of your network, where the data is being generated, instead of in a centralised data-processing warehouse.

Edge computing is a distributed, open IT architecture that features decentralised processing power, enabling mobile computing and Internet of Things (IoT) technologies. In edge computing, data is processed by the device itself or by a local computer or server, rather than being transmitted to a data centre.

Edge computing enables data-stream acceleration, including real-time data processing without latency. It allows smart applications and devices to respond to data almost instantaneously, as its being created, eliminating lag time. This is critical for technologies such as self-driving cars, and has equally important benefits for business.

Edge computing allows for efficient data processing in that large amounts of data can be processed near the source, reducing Internet bandwidth usage. This both eliminates costs and ensures that applications can be used effectively in remote locations. In addition, the ability to process data without ever putting it into a public cloud adds a useful layer of security for sensitive data.

8 Key Points in a Product Plan

Developing a great product is not an accident. It takes careful planning upfront in developing a Product Requirement Document (PRD).

A good PRD addresses 8 main points which are listed here. This document defines what the product will be, what problem it solves, when it will be ready and how much it will cost. There is no limitation on number of pages the document contain, but it could be comprehensive & concise.

The key to building a great product is to keep this PRD document true to its core intentions. This implies a lot of upfront work, but is absolutely essential for success. If the product is well planned, then only one can build a great product. Oftentimes, it makes sense to develop a user guide as part of the proposed solution – as this helps in developing the product. The amount of work that needs be done upfront is huge – but it aids in every step of product development. Some companies even go into great depths of defining each small step in the project plan with weekly timelines.

"One can achieve greatness with 10000 small steps!"

5 Pillars of Data Management for Data Analytics

Basic Data Management Principles for Data Analytics

Data is the lifeblood for Big data analytics and all the AI/ML solutions built on top.

Here are 5 basic data management principles that must never be broken.

1. Secure Data at Rest

• Most of the data is stored in storage systems which must be secured.
• All data in storage must be encrypted  

2. Fast & Secure Data Access 

• Fast access to data from databases, storage systems. This implies using fast storage servers and FC SAN networks.  
• Strong access control & authentication is essential

3. Manage Networks for Data in Transit

• This involves building fast networks - a 40Gb Ethernet for compute clusters and 100Gb FC SAN networks
• Fast SD-WAN technologies ensure that globally distributed data can be used for data analytics.

4. Secure IoT Data Stream

• IoT endpoints are often in remote locations and have to be secured.
• Corrupt data from IoT will break Analytics.
• Having Intelligent Edge helps in preprocessing IoT data - for data quality & security

5. Rock Solid Data backup and recovery

• Accidents & Disasters do happen. Protect from data loss & data unavailability with a rock solid data backup solutions.
• Robust disaster recovery solutions can give zero RTO/RPO.

Business Success with Data Analytics

Data and advanced analytics have arrived. Data is becoming ubiquitous but several  organizations are struggling to use data analytics in everyday business process. Companies who adapt data analytics in the truest and deepest levels will have a significant competitive advantage, ; those who fall behind risk becoming irrelevant. Analytics has the potential to upend the prevailing business models in many industries, and CEOs are struggling to understand how analytics can help.

Here are 10 key points that must be followed to succeed.

1. Understand how Analytics can disrupt your industry
2. Define ways in which Analytics can create value & new opportunities
3. Top managers should learn to love metrics and measurements
4. Change Organizational structures to enable analytics based decision making
5. Experiment with data driven, test-n-learn decision making processes
6. Data Ownership must be well defined & Data Access must be made easier
7. Invest in data management, data Security & analytics tools
8. Invest in training & hiring people to drive analytics 
9. Establish Organizational Benchmarks for data analytics
10. Layout a long term road map for business success with Analytics

5 AI uses in Banks Today

1. Fraud Detection
Artificial intelligence tools improve defense against fraudsters and allowing banks to increase efficiency, reduce headcount in compliance and provide a better customer experience.
For example, if a huge transaction is initiated from an account with an history of minimal transactions – AI can shop the transactions until it is verified by a human.

2. Chatbots
Intelligent chatbots can engage users and improve customer service. AI  chatbot brings a human touch, have human voice nuances and even understand the context of the conversation.
Recently Google demonstrated its  AI chatbot that could make table reservation at a restaurant.

3. Marketing & Support
AI tools have the ability to analyze past behavior to optimize future campaign. By learning from prospect’s past behavior, AI tools automatically select & place ads or collateral for digital marketing. This helps craft directed marketing campaigns
Also see: https://www.techaspect.com/the-ai-revolution-marketing-automation-ebook-techaspect/

4. Risk Management
Real time transactions data analysis when used with AI tools can identify potential risks in offering credit. Today, banks have access to lots of transactional data – via open banking, and this data needs to be analyzed to understand micro activities and access the behavior of parties to correctly identify risks. Say for example, if the customer has borrowed money from a large number of other banks in recent times.

5. Algorithmic Trading
AI takes data analytics to the next level. Getting real time market data/news from live feeds such as Thomson Reuters Enterprise Platform, Bloomberg Terminal etc., and AI tools can use this information to understand investor sentiments and take real-time decisions on trading. This eliminates the time gap between insights & action.

Importance of Fintech to India

On 8 November 2016, the Government of India announced the demonetization of all ₹500 and ₹1000 banknotes, it set off a wave of Fintech growth in India. Fintech is now mainstream and a critical segment for the future of India's economic growth.

Here are 10 reasons why Fintech is very important to India.

1. Economic Growth
The payment segment has been a major enabler of economic growth. Electronic payments systems added $300B to GDP in 70 countries between 2011-2015, which resulted in ~2.6Million Jobs/Yr
Each 1% increase in electronic payment produces ~$104 B in consumption of goods & services

2. Financial Inclusion
Fintech opens up opportunities for the previously unbanked population to access modern financial instruments. For people living in poverty or at the fringes of the economy, Fintech lowers costs of Financial transactions: Lower cost of credit and other banking services.

3. Speed & Quality of Innovation
Fintech drives improvements in traditional financial services – which will replace legacy systems. Eg: Peer-to-peer lending, Robo advisors, Hi-frequency trading

4. Business Sustainability & Scalability
Fintech has made businesses sustainable & Scalable. The entire e-commerce economy was built on e-payment systems and new business models such as Ridesharing: OLA, UBER, Metro Bikes, etc were developed on Fintech e-payment systems – which allows these businesses to scale and grow rapidly

5. Transparency & Audits
All digital transactions are inherently auditable hence bringing greater transparency into the system. Data sharing in real-time across banks & financial institution reduce fraud risks and reduces the cost of regulatory processes.

6. New Value Streams
New fintech technologies are creating new business opportunities. Bitcoin & other cryptocurrencies have spawned whole new businesses.

7. Market Curation & Structural Transformation 
Fintech technologies are transforming other industries. For example, healthcare record management, Real estate, land registration with Blockchain, etc. This is bringing structural reforms to businesses that were on the fringes of the regulated economy into the mainstream economy.

8. Collaborative Culture
New Fintech businesses are built in collaboration with other businesses. For example, Blockchain is based on open collaboration between members who host the shared ledger.

9. The Scale of the Industry
Fintech has grown from being a niche to mainstream. Today Fintech companies are collectively worth more than $500Billion and directly employ millions of men.

10. Borderless Innovation
Technological innovations in Fintech can be quickly adapted across borders, creating new competition and new opportunities for existing players. This rapid innovation is bringing whole new financial hubs and opening new markets.

Skills Needed To Be A Successful Data Scientist

Data Scientist, the most demanded job of 21st century, requires multidisciplinary skills – mix of Math, Statistics, Computer Science, Communication & Business Acumen.

Top Challenges Facing AI Projects in Legacy Companies

Legacy companies which have been around for more than 20 years have been always slow to embrace new technologies & the case is also very true with embracing AI technologies.

Companies relutcantly start few AI projects - only to abandon them.

Here are are the top 7 challenges AI projects face in legacy companies:

1. Management Reluctance
Fear of Exacerbating asymmetrical power of AI
Need to Protect their domains
Pressure to maintain statusquo

2. Ensuring Corporate Accountability 
Internal Fissures
Legacy Processes hinder accountability on AI systems

3. Copyrights  & Legal Compliance 

• Inability to agree on data copyrights
• Legacy Processes hinder compliance when new AI systems are implemented

4. Lack of Strategic Vision

• Top management lacksstrategic vision on AI
• Leaders are unaware of AI's potential
• AI projects are not fully funded 

5. Data Authenticity

• Lack of tools to verify data Authenticity
• Multiple data sources
• Duplicate Data 
• Incomplete Data

6. Understanding Unstructured Data

• Lack of tools to analyze Unstructured data
• Middle management does not understand value of information in unstructured data
• Incomplete data for AI tools

7. Data Availability

• Lack of tools to consolidate data 
• Lack of knowledge on sources of data
• Legacy systems that prevent data sharing 

Benefits of Aadhaar Virtual ID

Use Aadhaar Virtual ID to Secure your Aadhaar Details

Considering the privacy of the personal data including the demographic and biometric information mentioned on the Aadhaar card, UIDAI has recently decided to come up with a unique feature, termed as Aadhaar Virtual ID.

The Aadhaar Virtual ID offers limited KYC access providing only that much information which is required for verification rather than offering complete details of an individual's Aadhaar card.

What is an Aadhaar Virtual ID?

The Aadhaar Virtual ID consists of 16-digit random numbers that is mapped to an individual's Aadhaar card at the back end. An Aadhaar card holder using the virtual ID need not submit his Aadhaar number every time for verification purpose, instead he can generate a Virtual ID and use it for various verification purposes like mobile number, bank and other financial documents.

The Aadhaar Virtual ID gives access to the biometric information of an Aadhaar card holder along with the basic details like name, address and photograph that are sufficient for the e-KYC. Unlike in the past, the agency will not know the 12-digit Aadhaar number and other personal details.

Benefits of Aadhaar Virtual ID

1. Complete Privacy of personal Data
   eKYC can now be done without sharing Aadhaar number
   All private information: biometric, DOB, address are private
    
2. User has complete control on sharing Aadhaar ID details
   Only the Aadhaar card holder can generate virtual ID
   Only the Aadhaar card holder can share virtual ID
   Aadhaar Virtual ID expires after a pre-set time, preventing misuse
    
3. Automates all eKYC verification process in the backend
   Simplifies agencies task of individually verifying KYC data
   Web Based verification system is fast and reliable for real time business applications

Big Data Analytics for Digital Banking

Big Data has a huge impact on banking, especially in the era of digital banking.

Here are six main benefits for data analytics for banks.

1. Customer Insights

Banks can follow customer's social media & gain valuable insights on customer behavior patterns
Social media analysis gives a more accurate insights than traditional customer surveys
Social media analysis can be near real time, thus helping understand customer needs better

2. Customer Service

Big data analysis based on customer's historical data, current web data can be used to identify customer issues proactively and resolve them even before customer complains
Eg: Analyzing customers geographical data can help banks optimize ATM locations

3. Customer Experience  

Banks can use big data analytics to customize website in real time - to enhance customer experience.
Banks can use analytics to send real time messages/communications regarding account status etc.,
With Big Data analytics, Banks can be proactive to enhance custoemr service.

4. Boosting Sales

Social media analysis gives a more accurate insights into customer's needs and help promote the right banking products to customers. For e.g., customers looking at housing advertisements and discussing housing finance in social media - are most likely in need of a housing loan.
Data analytics can accurately acess customer's needs & banks can promote right types of solutions.

5. Fraud Detection

Big Data analysis can detect fraud in real time and prevent it
Data from third parties and banking networks holds valuable information about customer interactions.

6. New Product Introduction

Big Data analysis can identify new needs and develop products that meet those needs
Eg: Mobile Payment services, Open Bank APIs, ERP Integration gateways, International currency exchange services etc are all based on data analytics