March 2, 2010

The New Monster


The Road goes ever on and on
Down from the door where it began.
Now far ahead the Road has gone,
And I must follow, if I can,
Pursuing it with eager feet,
Until it joins some larger way
Where many paths and errands meet.
And whither then? I cannot say.
-          Lord of the Rings – J.R.R Tolkein
The road today for most businesses is murky and unclear. While most of them are trying to figure out the new monster sleeping in their organization, some of they are well on their way to maximize their gains using the new power of this monster.
Analytics today is the buzzword in most businesses and this post is a humble effort to breakdown this monster into small bits that can be digested by most of us. I shall attempt to make the posts here MECE but please do pardon me if I were to walk down different paths, that meet elsewhere in the future.
The problem today is an overabundance of data .Data today is flowing in at a rate at which most firms today are not able to store, let alone analyze and use. International Data Corporation (IDC), in a market study in 2008 estimated that around 1,200 exabytes (1.2 x 1012 GB) of digital data will be generated this year. Wal-Mart handles over 200 Million customer transactions in a week! 
The new road
Analytics is simply defined as the ‘science of Analysis’.  Irrespective of domain, field, technique and any other category that one can think of, everyone of us has used ‘Analytics’ at some point in our life. As a child one learns to identify different building blocks and stack them together, which is a basic example of pattern based analytics. We recognize the smell of wet earth before it begins to rain, which is an example of predictive analytics.
Analysts today uses various statistical and stochastic techniques to define, measure, analyze, implement and control (DMAIC from Six Sigma) various business processes, which broadly defines analytics as a process. While Analytics, Data mining & Business Intelligence are terms that have been used interchangeably in the past, it is important to understand the subtle difference between them if one wants to attain nirvanaJ.
Data mining is traditionally defined as the process of extracting patterns from data and understanding them. Typically classification, clustering and rule based learning are most common application of data mining. Business Intelligence traditionally comprises of all the components of a business that go into supporting decision making, which could includes data mining and analytics.
Analytics, in today’s parlance, comprises of data mining tools and techniques as well as business intelligence reports, systems and process, which result in actionable insights to answer business problems. While insights are the key to easy decision making, actionable insights can provide businesses a sustainable advantage, which can take them right to the top.
The next generation
While most professionals today struggle to understand and interpret data, the next generation will come well prepared to steamroll the archaic systems of ‘gut feel’ and ‘experience’ using smart, trendy and dynamic tools that can give you both a Birdseye view as well as a drilled down report on what’s right and what’s not. In short, they can provide business leaders with the ‘One ring to rule them all’ and answer most questions based on data.
Time to stop the gyan and provide a glimpse of what I have been ranting about. Take a look at the graph below:
The graph provides the trend of daily unique visitors going to Pagalguy and Careerlauncher websites in the last 30 days.
Table 1 provides regionwise distribution of these unique visitors along with a comparison between the two respective websites.
 Table 2 provides a list of websites that were also visited by people who visited the above websites. It also provides a comparison of how many people who visited Pagalguy also visited xyz website as compared to those who visited Careerlauncher. For example, almost 75% of people who visited pagalguy also visited clexam.com while almost all people who visited Careerlauncher visited clexam.com.
 Table 3 provides a list of keywords that were searched by visitors who landed on the two websites under consideration.
 Using the three tables and the graph above, one can easily understand the visit trend and behavior of people visiting the two sites. For example, Careerlauncher has a lot of visitors from the Delhi/NCR region while the number of visits from the Southern region is almost negligible. Whereas pagalguy has a decent amount of visit even from the rest of india as compared to Delhi/NCR region.
While this might not be a case of apple to apple comparison, it does help understand the behavior of the target market, which in this case are students/professionals who are appearing for various B school entrances and other competitive exams.
The above graph and tables were generated using Google trends for websites and to know more about the same please visit Avinash kaushik’s post on Competitive Intelligence Analysis.
If I have managed to even remotely interest you, then I am sure you will sit up after read the next section.
 The below graphs and tables are generated using Google Insights for Search.
 The graph above provides comparative ranking of number of times a search has been done for a particular term, relative to the total number of searches done on Google for the time period under consideration. Simply put, it gives a comparative ranking (post normalization) of the various keywords you entered with a common base (which is total number of searches on Google).
The above graph is for 4 keywords – Dell, HP, Acer & Lenovo in the last 90 days in India.

Table 4 provides a region wise breakdown of search for each term. Mouse over on a region to view the average percentage split for the region.
 Table 5 provides the top search phrases with ‘the most significant level of interest’ for Dell and also provides a list of rising searches. The percentage figure associated is computed using a base period of the same duration as that entered by you. In this case, the time period for the search if for 90 days (Dec, Jan & Feb), which implies that the rising search % is calculated based on the search volume for Sep, Oct and Nov for the same phrases.
Table 6 provides the same results as those of table 5 but the search term is HP.
 As a Brand/Marketing manager, this will help me understand if my strategy of buying a very expensive hoarding in the busiest part of city xyz has helped increase awareness for my latest product in the city or not.  It will also provide me with a very rough understanding of what my competition is focusing on and what I could do to improve my product awareness.
It is important to understand that the above tables/graphs are a snap shot based on a time frame and if one were to analyze various timeframes, it would provide us with a real picture of how what works and what doesn’t.
Consider the below data, which is the normalized number for each of the four brands for Karnataka region over a period of time.

While it is important to understand that the above data is based on normalized search numbers, using some basic assumptions, it can easily be used to gain insights into search term performance. For example, if I were to assume a 10% increase in number of searches on Google YoY, then I can clearly see that search for HP has significantly reduced in 2009 as compared to 2008, while Dell’s share of search has increased. Based on this insight, I can then deep dive into specifics of advertising or other marketing activity which might have led to this increase in search share for dell and take corrective measures to prevent HP’s share from declining.
Bottom-line
The bottom line is the where it all ends both for businesses and this post. Analytics is going to be the new driver for increased revenue as well as profitability and businesses today are waking up the sleeping data monster. While it is important to leverage the insights and drivers using analytics, it is equally important to understand that every business is unique and there can be no solution that fits all.
Though the roads are broad, I keep marching,
Searching for the tree of light.
Waking up the slumbering giant,
On whose shoulder I shall ride….