It used to be ‘Show me your friends, and I’ll tell you who you are’, now it’s the ads targeted to your demographic, your location and your recent status updates that tell most about you. Big data is the DNA of consumer behaviour. We look behind the hype for what it means to your business.
WHAT IS BIG DATA?
First, big data is big numbers. The numbers that describe how much data we are creating and storing are almost unimaginable. Last year alone we created enough data to fill the Library of Congress 60,000 times. Now, 60,000 is a lot. But try 2.5 quintillion. A quintillion is a billion billion. We create 2.5 quintillion bytes of data every day, texting, posting, tweeting, typing word docs, taking pictures and videos, making calls, sending GPS signals, leaving purchase records – right up the chain to climate information gathered by satellites, says IBM. Your every tweet, every text, every impulsive online purchase or Facebook ‘like’ is leaving a digital trace.
But numbers are only part of the story. What takes all this data from isolated bits of information to insightful, actionable knowledge are the algorithms and software that are being developed to search this enormous resource. They link unstructured information to spot patterns of behaviour and ultimately to predict what is likely to happen next.
Coal was a dirty black rock until we learned how to use it to power the industrial revolution. Oil was black sludge until the internal combustion engine revolutionised transport and brought the world closer together.
Data is bits and bytes until they can be sewn together to build a comprehensive, real -time profile of who and where you are, what you like, how you’re likely to vote, shop and react in a crisis.
Imagine combining what you buy at the supermarket, the size of your tax liability, where you travel on holiday with your medical records. It’s the DNA of consumer behaviour.
International Data Corporation recently forecast that the big data technology and services market will grow to about $17 billion by 2015. Big data has become a whole new asset class.
What makes big data so compelling for business is its promise as a game changer. A recent report from the McKinsey Global Institute, ‘Big data: The next frontier for innovation, competition, and productivity’, says by harnessing big data effectively, companies can lift productivity by a staggering 65%.
Dr Liz Ferrier, Senior Lecturer in Marketing at UQ Business School, is interested in how the collection and analysis of customer information is reshaping marketing–customer interaction – not just by small degrees, but by leaps and bounds – so significant that we can’t yet fully see the outcome.
“By gleaning rich data about consumers, companies are able to anticipate customer preferences and interests; check customers are happy with their product or service; engage them in dialogue and activities that improve the value of services; forecast trends, sales and stock needs more accurately; make better purchase decisions; and respond quickly when they see demand change or new opportunities arise,” says Dr Ferrier. But, she warns, “how to analyse the available data and develop strategy that is informed by it for competitive advantage remains a challenge. Some businesses are missing huge opportunities.”
WHAT’S NEW?
The more you know about your customers, the better you can meet their needs. It’s ancient business wisdom that’s been with us since the local butcher first asked Mrs Smith about her dicky knee as he wrapped her pound of sausages.More recently, data mining, business intelligence and business analytics have become familiar tools contributing to core business functions, from strategy and marketing to product development, supply chain management and budgeting.
Now, big data describes the surge of new data from web traffic as well as the jump in e-commerce information such as purchase records, digital images, IT logs, GPS signal data and videos. But it also describes the ‘old’ transactional data like store point-of-sale information, bank transactions, mobile phone records and billing data, stored invoices and tables of sales numbers.
“If ads are popping up on your Facebook page offering love matches or wrinkle tricks, the chances are that these things have recently crossed your mind. If you’ve bought a book on Amazon.com by clicking ‘people who bought this also enjoyed…’ it’s big data in action.”
HOW DO YOU DO IT
The magic of big data lies in combining and analysing both types of data to more easily predict and interpret people’s choices and actions.
Big data is more than simply collecting and storing information. Although the advent of cloud computing has made this far more affordable, the benefits come from what companies do with what they know.It will mean a whole new set of technologies. Dr Sophie Cockcroft, Lecturer in Business Information Systems at UQ Business School, says current data management systems won’t cope with big data challenges, and businesses will have to jump on the technology bandwagon.
The biggest issue is that the old ‘transactional’ data and the new ‘observational’ data behave differently. Says Ian Bertram, Global Manager of the Analytics and Business Intelligence Research and Head of Research for Asia/Pacific at Gartner: “The former is stored in a warehouse to be brought into company resource planning systems later on. It’s structured.”
But you can’t fit Word documents, PDFs, e-mails, voice mails, video, tweets and Facebook ‘likes’ into a warehouse. Nor, can web traffic be handled by the usual business analysis systems. “It sits out there. You bring it in, act on it and then get rid of it,” he says.
Nor will it simply be a question of running the data in the hope of gaining something valuable.“To mine big data takes the right tools, plus creativity and asking the right questions,” says Dr Cockcroft.
Bertram agrees: “Put simply, to make sense of big data, you have to mash all the data sources together and let them either reinforce or refute each other. In short, reinforcing data means executives can be confident in whatever they are doing. If data sets don’t gel then they need to ask why.”
Cockcroft sounds another note of caution: more data does not necessarily mean greater accuracy. In some cases, more data can introduce a cacophony of noise or chatter that can drown out something significant. “Do you really want to get to that invisible layer of gossip by including tweets, for example?” she asks. “Yes, you would like to know what your customers are saying, but filtering out the junk is a challenge.”
Companies need to monitor data and work out how to change mainstream processes before they can take advantage of the high end anticipating and predicting that big data analysis facilitates. Bertrand sees companies mistakenly leap into predicting and optimising without mastering simple monitoring. “If you toss in some of those tools before the company is mature enough to use them properly, the result will look like another failed IT project.”
It’s about companies taking baby steps to use all the information held in separate parts of the organisation.
The big opportunities won’t come just from technology, warns Cockcroft. “Identifying the opportunities is still up to the people who make business decisions. Relying on data alone could lead a company down the wrong path.”
GARTNER SAYS…
Gartner estimates that enterprise data will grow by 650% in the next five years. A solid 80% of that will be in unstructured or observational data from sources such as emails, smart phones and social media sites.
However, it’s not easy pickings. “The hype factor is huge. The question is: now that companies have access to a truckload of data, what are they doing with it? Until that becomes clear, there are some ludicrously inflated expectations,” says Ian Bertram, Global Manager of the Analytics and Business Intelligence Research. He believes that many companies are struggling to make sense of what they have. “The opportunity is there: what will organisations make of it?” he asks.
“We’ve seen insurance companies reduce investigations by two-thirds which is a massive cost saving.
In healthcare, we’ve seen certain drugs pulled off the shelves because the data showed a correlation with harmful behavior”.The future lies in a company’s ability to capture diverse data types and manage them to uncover what was previously unseen.
GEARING UP YOUR BUSINESS
As data analysis uncovers new kinds of opportunities, new skill sets and business structures will emerge to take advantage of them.
The first challenge, says Dr Liz Ferrier, is developing the skill set. “There’s a shortage of managers who understand the value of data and have the relevant skills: managers who are able to work with data analysts, and to develop business strategies that are based on science – what the data reveals. The field of data mining is so vast, subject to rapid change, and characterised by so many different and emergent methods for visualizing, modeling, analyzing and ‘monetising’ data, that it will take time for many industry sectors to develop preferred standards, metrics and currencies relating to data, its value and use.
”It’s also about organisational structure. Traditionally, companies have had teams that have focused on structured data-polishing and cleaning warehouse data – and a different team looking at enterprise content management. It’s time to bring the skills and practices of these teams together. “Successful data mining is also about combining marketing insights with targeted sales activities to leverage those insights,” says Dr Ferrier.
Companies need to focus on developing the backroom analytical talent and linking it with frontline personnel who are capable of taking the analytics and building the tools and the strategies to make them count.
BIG DATA FOR THE BETTER WORLD
The World Economic Forum identifies big data as an opportunity to improve services, avoid crises and better target needs such as health care and education among vulnerable populations.
With the proliferation of mobile handsets across remote regions, health care and mobile banking are now available in places where hospitals and bricks and mortar infrastructure are in short supply. This is creating a torrent of data about populations that have been less visible until recently.
Understanding population health, watching the impact of a natural disaster, targeting services where they are needed more efficiently are all outcomes that the World Economic Forum identify in their report ‘Big data, Big Impact: New Possibilities for International Development’.
THE BANKS
HOW CAN BIG DATA HELP?
- Understand online customer behaviour and interactions
- Identify trends and topics in social media sentiment analytics
- Micro-target advertising campaigns and special offers
- Identify opportunity maps for sales campaigns
- Track product status (e.g. alert the manufacturer when a car is due for a service)
- Track how customer behaviour can impact on a product (e.g. a safe driving style could lower insurance premiums)
- Track patterns in population health (e.g. identifying the next flu outbreak)
- Identify financial fraud
- There is tangible evidence pointing to a real competitive advantage for banks that are collecting and using big data to drive new customer related insights, says Stuart Scoular, Banking and Capital Markets Leader for PricewaterhouseCoopers.
How banks respond to the new onerous regulations, fast-moving technological changes and shifting demographics is key to their futures but in a social media world, how they engage with their customers will make or break them.“To this end, banks are taking the biggest leap forward seen in the customer data space for years,” says Scoular.
However, strategies are shrouded in confidentiality agreements. Critically, growth can only come by carefully targeting customers at the micro level, selling them more varied products, and hoping they won’t walk.
RETAIL
Retailers are grappling with the same questions: How do I get a presence, how do I manage my brand? says Hugo Driemeyer, a director at Private Equity Gateway Group. The uncertainty, he says, “is driving a whole new wave of investment into technology platforms like Hearis, providing management and monetising tools for retailers and multi-site corporations who have to increase and sensitively manage their online presence or be left behind.”
MANUFACTURING
Manufacturers of a range of products can use data capture and analysis to track use and state of products. Some car manufacturers including BMW use sensor-data to tell customers when their cars need to be serviced, for example.
GETTING CREATIVE
Two McGill University computer scientists have come up with a creative way of wading through the tsunami of DNA data to cross match genes across species. Jerome Waldispühl and Mathieu Blanchette have developed Phylo, a game that contributes to mapping diseases within human DNA. Finding patterns in the DNA is something computers don’t always do well without eating up a lot of computing time. Humans can often do a better job, and more quickly. And that’s where the game Phylo comes in.
About 500 people a day are playing it from all over the world, according to the scientists.
GOVERNMENT
Data captured by health workers in the field can be a tool to map health trends or monitor virus outbreaks. When collected in the context of individual electronic health records, this data will improve care for the individual, but it can be used to create datasets with which treatments and outcomes can be compared in an efficient and cost effective manner, suggests the World Economic Forum.
DR SOPHIE COCKCROFT
Sophie specialises in information systems, including the analysis and design of electronic commerce and web applications and security. Her research interests include data quality and medical information systems.
DR LIZ FERRIER
Liz is a Senior Lecturer in Marketing whose research interests focus on advertising and communications. Her research projects have included online communities, amateur content creation on the web, and television and advertising audiences.
LEARN MORE
If you would like to learn more about the research in this article, then take a look at:
“Big data: The next frontier for innovation, competition, and productivity”,
Report from the McKinsey Global Institute, 2011
“Big Data, Big Impact: New Possibilities for International Development”,
Report from the World Economic Forum, 2012