Science, Technology, and Real Estate

The targeted advertising landscape is starting to go another through a major overhaul, and it’s certainly no stranger to change. By utilizing computers, statistical analysis, deep packet capture, cross-user correlation, and fast computation technology, companies are poised to cause consumers and advertisers alike to change the way products are sold. Sales will become more efficient (less cost-per-conversion), consumers won’t see ads for things they wouldn’t be interested in, and markets will take another step toward ultimate Adam Smith style efficiency.

Background
Modern targeted advertising started in print media; advertisers placed banking ads in the business section of the newspaper and shoe ads in the fashion section. I’ll call this model “old media targeted advertising.” This style of advertising has a place in the traditional media, but has serious drawbacks, inasmuch as it doesn’t target an individual consumer’s tastes but rather takes broad strikes at the entire landscape, hoping to hit a few interested parties.

The first major overhaul of targeted advertising happened with the inception of keyword based internet advertising. Early pioneers like DoubleClick (1996) and Yahoo shaped the second generation of ad delivery. Now instead of simply placing a Financial Services ad in the printed Dow Jones Industrial charts, a Stock Broker could place their ad in-line with an internet article that talks about selecting a Stock Broker. Not only was this more efficient in terms of targeting the buying consumer, but ad-space became much cheaper — a print advertising campaign that cost several thousand dollars and hit many unrelated parties became an online campaign that cost a few hundred dollars.

DoubleClick in particular tried to bridge the gap between the keyword based online advertising and the new, more intelligent history based advertising. What does that mean, exactly? DoubleClick advertises in many places and they collect data each time they serve an advertisement, regardless of the site. They take this data, and try and correlate many events to a single user; they try and track you through each site you visit in your day-to-day life in order to build an advertising profile, then they use this profile to serve you more targeted ads.

As a concrete example, a consumer (let’s call him Joe) visits Expedia, Delta Airlines, and Kayak, presumably looking for airline tickets. Assuming DoubleClick advertises on both Expedia and Kayak, they can try and put two-and-two together to determine that Joe is planning a trip, and they can serve travel related advertising to Joe on any site that he visits whether or not the content on the site is related to travel.

By making correlations between Joe’s site visits and the detected patterns, DoubleClick has been able to increase the effectiveness of their advertising campaigns. There are, however, some serious limitations to the DoubleClick model. 3rd generation advertising attempts to overcome these limitations.

Present Limitations

• Present advertising software lacks a comprehensive view of a user’s actions. They see only what they see, and that’s a very small picture.

When DoubleClick’s software realized that Joe was going to take a trip, they used every opportunity available to try and present him with ads for travel. Unfortunately, DoubleClick’s internet coverage is quite small; fewer than 1 in 5,000 sites are reachable by DoubleClick. This means that while DoubleClick may have properly guessed that Joe intended to take a trip, they can’t see when Joe visits Delta’s website directly (which doesn’t have DoubleClick ads) and purchases his tickets. Not only is DoubleClick’s ad wasted since Joe doesn’t need travel arrangements, they’ve also wasted their opportunity to present him with another more appropriate ad. No one benefits from this situation.

• Keyword driven advertising misses the actual content.

Perhaps the most well known keyword based advertising system on the internet is Google’s AdSense. AdSense has built an advertising network spanning many sites (incidently this network is intimately related to DoubleClick’s network as Google purchased DoubleClick in 2007 and has begun to integrate its products with DoubleClick). These sites are corporate and personal alike, with all types of content and all motivations. When a content provider signs up for AdSense, they integrate Google advertising into their own content and are paid based on readers’ clicks on the advertising.

When Google delivers advertising based on keywords found on a content providers’ site, they are doing so with blinders on. For instance, a web page with investment advice may mention the term “Real Estate” tens of times, but not fundamentally talk about buying or selling a home. Nonetheless, AdSense will spot the many uses of the term “Real Estate”, and probably serve an ad for a local REALTOR. This ad doesn’t really benefit anyone involved, and again the opportunity to serve a better, more targeted ad is lost. This is the reason that the clicks-per-view for Google AdSense are quite low (0.05 clicks per impression).

• Conversions aren’t used to target other advertising.

When a user is served an online advertisement and then clicks on the ad to buy a product, everyone benefits. The advertiser has made a sale, the advertisement content provider (like Google’s AdSense) has generated revenue from the click, and the consumer gets something they’re after. There is unrealized potential for more benefit from this transaction for every party, but the data regarding the transaction ends at a sale. In the next, or 3rd, generation of targeted advertising, data regarding a sale is sacred and potentially much more valuable than the keywords on the website in which the ad was served.

To explain how this data is valuable, consider a poker tournament. A good poker player will find a “tell” in all of her opponents. A tell is a twitch, an involuntary twitter, a change in breathing patterns, or something else discernible when a player is bluffing, or playing as if their cards were better than they actually are. By finding and learning to watch for a tell like this, a good player has given herself an advantage in the game by being able to recognize when the odds for a win are in their favor.  Compare this with the data collected in a sale — the sale is a consumer tell. Advertisers should be able to categorize, classify, and manage the events and content leading up to the click and sale, perhaps even going back weeks or months in browsing history.  Once this tell is discovered, the advertiser can look for the same tell in another consumer’s patterns to try and predict another purchase, or apply this tell to other consumers that behave similarly.

The 3rd Generation of Targeted Advertising

Advertisers and marketers alike are advancing the advertising landscape to overcome these limitations and soon will take advantage of the tells ready to be found.  Deep packet capture, or the practice of capturing and storing all network traffic for a period of time, is becoming a standard component in the IT management community and has already hit mainstream in the intelligence community.  The idea is simple:  Store everything that happens on a network because you probably won’t know until long after the traffic has passed which parts are interesting.

The benefits of this data retention to government intelligence agencies should be obvious.  If the FBI arrests a suspected criminal, they could potentially obtain a warrant and see all of the suspect’s past network traffic which includes website visits, IM conversations, email, and other sensitive data.  The CIA and the NSA may use the same data as a basis for analysis; more on that later.

For IT managers, the benefit of data retention probably isn’t as immediately obvious, though the benefits are still quite important and ultimately valuable.  Foremost, a company involved in a lawsuit with an employee over wrongful termination would no-doubt like to have a complete history of all network traffic the litigious employee originated to use as evidence in a courtroom.  Similarly, the same company might be a defendant in another lawsuit and could produce network traffic during discovery or as evidence while defending itself in court.

From the technology development and maintenance standpoint, data retention through deep packet capture can provide an IT staff with a forensics toolkit for finding slow points in network infrastructure.  The staff may observe that packet transit times on a particular network are 5 times slower than other networks and use this data to find a faulty switch.  At the application level, the IT staff might develop a model for what standard network traffic looks like and apply the model to new traffic to quickly find problems and security breaches.

But back to the main question, how does this apply to targeted advertising?  The answer is that deep packet capture provides a history and dataset upon which “tell tracking” can be built — that is, it gives enough information to advertisers to fundamentally change the advertising landscape. By installing a capture box at the internet service provider (ISP) level, a 3rd generation advertising company would see all network traffic originating from or destined to that internet provider.  Every single instant message a consumer sends and every web page they visit can be used to build a profile which goes leaps beyond what DoubleClick has done.  Because the advertising company could see all traffic, with and without advertising alike, the amount of data available to find a consumer tell grows exponentially, as does the ability to convert adspace into sales.

Thus it becomes clear that packet capture is a required basis for 3rd generation advertising.  This basis isn’t sufficient to propel advertising into new efficiencies and conversion ratios though.  There is another critical component to be built on top of the packet capture, in the same way that flour is a critically important component in the final product when baking bread: statistics.

Statistics are the “how” for tell tracking. By building statistical profiles for all consumer traffic at once, advertisers can search for (and find) consumer behavior patterns which lead to sales.  Every iota of every data packet will be analyzed by a statistics engine, and each of these iotas will be analyzed in aggregate.  The statistical information generated from this process will be applied to other consumers data, and a spider web of interconnected behavior and patterns will be built.

At this point it’s easy to see why the government is interested in installing packet capture devices wherever possible and how they might analyze that collected information.  Law enforcement will be made more efficient as will traditional intelligence gathering and anti-terrorist monitoring.  Cyber-warfare attacks will be more easily recognized, predicted, and prepared for.  The same style software will also be a boon to advertisers.

This spider web will allow advertisers to assign probabilities to all types of behavior, the interesting behavors include being able to tell with a 78.3% confidence that Bob Richards is going to buy a new car in the next 3 days and selling Bob’s information and statistical profile to Ford or GM for hundreds of dollars.  The best part is that as time progresses and more data is collected, the spider web will grow bigger and become more accurate, possibly even to the point of realizing that Bob Richards is going to buy a new car before Bob himself knows it.  And because the advertisers aren’t looking at keywords to target their advertising, they could give Bob a car ad on a furniture restoration website and have higher conversion rates than placing an ad on Edmunds today, hoping to find a car buyer.  Markets will take the next step toward ultimate efficiency, advertising will become cheaper on a cost-per-lead basis, and consumers will see more ads that are relevent to them.

So why hasn’t this happened before now?  Simply put, computers weren’t fast enough, storage wasn’t cheap enough, and the industry tends to move in small baby steps without making the giant leap to the next revolution.  The gap is still wide, and an enterprising company stands to create and own a new market which uses statistics to push targeted advertising into its third generation.