The Future of Big Data

Share the knowledge

What is the future of so-called ‘Big Data’?

Big Data is characterized by a large volume of information collected and analyzed by any number of software programs to attempt to understand human behavioral patterns. More often than not, the behaviors that are being observed and studied are related to either spending or voting.

I spent over a decade analyzing data so I’m interested in what the future of Big Data will look like. Let’s take a look into the future to see how we might use all of this data.

 #5 Special Prices

Have you ever struck up a conversation with a stranger on an airplane? If so, then at some point, you might have compared what you paid for airfare. Someone paid  $250 while someone else paid $450. And yet someone else paid using frequent flyer miles obtained via flights that cost $1,000 or as low as $100.

What’s going on here?

Everybody on the airplane is subject to what’s called first degree price discrimination, which, according to Adam Ozimek of Forbes, “involves charging every individual customer a price based on their individual willingness to pay.”

Now, you probably would have preferred paying $250 over $450, particularly when it seems that your seatmate’s experience is identical to your own. But you might not have been given the opportunity to do so. Or maybe you were, either by the time you were booking, or where you were booking from (either your IP address or where you surfed in from), and you even didn’t know it at the time.

But here we are, ten years from now. And guess what? For you and your prospects, first degree price discrimination is the rule, and not the exception.

You go to buy groceries. Green bananas cost more than the yellow ones because you can store them longer. A mixed salad costs more than the fixings not only because of the labor involved in putting it together, but also because you are willing to pay extra for the convenience. Prices are dependent not only on what you paid last week, but also on your spending habits. Are you more likely to cook Italian or Chinese style foods? That will also determine which prices you’re offered, as will the supermarket’s stock and the expiration dates for the sauces.

You can really throw a monkey wrench into things if you step out of character and throw a party, and shop for it. Suddenly the system might think you have a dozen teenagers, based on all the pizza and chips you bought.

In some ways, it’s the electronic equivalent of an outdoor market. But instead of people haggling over rugs or spices, it’s the use of big data, as the supermarket attempts to predict what you’ll pay, what you’ll buy, and what will keep you coming back.

How do you beat it? Current conventional wisdom is to clear cookies, surf privately, be patient and watch for changes. But what if you need it now? And what if this is all happening in the grocery aisles or at the checkout counter, when you’re in a rush or at least nowhere in the vicinity of a calm, quiet place to think about things? About the only things you can do are to pay in cash or put your purchases back, thereby opting out completely.

#4 Turn Big Data On Its Head

As prospects show their preferences through their demographics, their search histories, and where they click, retargeting allows for personalization in how often ads are delivered, what they say, and how they relate to where a prospect is in the sales funnel. This turns Big Data on its head, as the trends are used not to toss off ‘one size fits all’ predictions, but instead to reach prospects directly and present them with information they really need.

In the future, you’ll be using Big Data as a guideline while, at the same time, you use personalization via demographics, search histories, and more. With ad retargeting, you’ll be judicious about Big Data but less reliant on it.

#3 A Different Kind of Unfairness

In the future there will be even more social stratification. The rich are richer. The poor haven’t budged much. The middle class is even more squeezed. Why?

This is another issue with Big Data – biases. So much attention is paid to the quantity of data that its quality can sometimes be overlooked, as can its relevance, or the reason for the quantity. As marketers, we look at the quantity of information and at the popularity of certain pieces of it, and we might, consciously or unconsciously, add more weight to it.

It’s a bit of a selection bias, too, as we and our prospects might select or spread information about a particular news story or product because we’ve heard of it, and then information about the product is spread even more. It’s a self-fulfilling prophecy that the information will continued to be spread at a more rapid rate than news of something else.

In the future, we might not even notice the selection biases going on around us, or that we ourselves have made, either in our personal lives or in business. After all, we’ve told Facebook or its successor, and all news outlets, that news about, say, dolphins, is important to us. Hence we are served up more and more tales of dolphins, whereas stories of elections or the like aren’t served up quite as quickly as we, and a statistically significant portion of our peers, continue to choose fluff pieces and familiar storylines over hard news, particularly if it’s about faraway unfamiliar places or causes that don’t resonate with us.

#2 Automated Advertising

For prospects, ad retargeting gives them what they are looking for without turning their experience into just another echo chamber. Automated advertising allows you to speak the prospect’s language. Not by telling them what they want to hear, but by giving them what they need. And you can take advantage of this bias in favor of familiarity by simply becoming more familiar to your prospects.

#1 More Objectivity and Granularity

As more and more data is gathered, trends will continue to emerge. Big Data will continue to record as many objective measurements as possible, and analysts and marketers will be there to interpret it. If the data show that your prospects go to LinkedIn and download a white paper 45% of the time, a reasonable conclusion might be that the white paper is of interest but could stand to be improved, or that it just isn’t reaching people for some reason. Or maybe 45% is adequate, and the time and financial budgets would be better suited toward something else.

But that’s all futuristic. Let’s look for reasons for data quantity and popularity that go beyond numbers and really try to understand our prospects, what they want and how best to provide it for them.

Join Our Email List

Photo Credit: Ed Schipul CC