Online measurement: analytics, metrics and methodologies
Measurement
As marketing dollars have become more scarce, the importance of measuring ROI and building a business case to support investment has become paramount. For those seeking to better understand this subject, here’s a “simple” methodology for quantifying value. In this case, we’ll look at ROS (return on spend = expected revenue divided by cost of online media) and ROI (expected net present value divided by total investment) from online campaigns. If you find it to be of value, or if you have additional questions, please comment below.
Steps to Calculating Value
1. Determine the Value metric (Revenue, Margin, NPV) of a customer.
Some companies look at value of a transaction, annual revenue per customer or lifetime value (profit) of a customer. Some assign higher values for new customers vs. a new sale to an existing customer. You need to determine what is best for your organization (hint: choose the metric that is most used by your executives). For this example, let’s assume your average sale is $1,000 and that the lifetime value of a customer is $5,000.
2. Assign conversion rates to approximate close rates.
Let’s assume 3% of site visitors request more information (inquiries) and that 30% of inquiries complete a purchase. If you’ve done online campaigns before, you should have a basis for inquiry rates. Hopefully your VP-Sales know how many leads convert to a transaction. If 3% of visitors become leads, and 30% of leads are closed, 0.9% of visitors will become customers.
3. Determine what your cost or investment will be.
Let’s assume you will spend $10,000 in online advertising (display, search, email, etc.) this month.
4. Do the math to calculate ROS and ROI:
Assuming your efforts drive 2,000 incremental visitors to your site (cost: $5 each) you should see 60 new leads (3% conversion rate) and 18 new customers (30% close rate) worth $18,000 in revenue or $1.80 direct ROS ($1.80 in revenue for every $1 spent).
The Net Present Value of the 18 customers is $90,000 ($5,000 each) yielding a Return On Investment of 900%.
If you present these types of results to your CFO, you’ll quickly find a lot of interest (and dollars) in online marketing.
Another Metric: Value per Engagement
Another way to measure results is calculating value per engagement (visit, inquiry, etc.). In the example above, each visit is worth $9 in revenue ($18,000 divided by 2,000 visits), whereas each inquiry is worth $300 ($18k divided by 60), compared to a cost per visit of $5 and a cost per inquiry of $166.67.
Remember Your Margins
While revenue is an easy metric to measure, margins are much more important. Assuming your gross margin is 60%, you are making profit as long as your cost per visit is less than $5.40 or cost per inquiry is less than $180.
Caveat Emptor!
Please use good judgment when applying these methodologies to your own business. Again, these are not a panacea for every situation. But hopefully, they will give you some building blocks for quantifying the impact of your interactive marketing program. If you have specific questions, please leave them here. I can’t promise I’ll know the answer, but I’ll do my best to help you figure it out. Happy number crunching!
Please feel free to COMMENT, SHARE with others and SUBSCRIBE to our blog. We look forward to your feedback!
Steve Latham (follow me on Twitter)
I came across a recent Forrester post on Attribution and felt the need to comment… I’ll be short and to the point!
I agree the concept of attribution is not new but unfortunately there are still many issues that need to be addressed, such as…
1. Ad servers reliance on their tag to be served on the last visit preceding an action. Unless I’m mistaken, ad servers above can only attribute credit for prior engagements if the last click preceding the conversion is goes through their server. Unfortunately most conversions are preceded by visits from direct navigation and/or natural search. So unless the ad server integrates with site analytics data, they can’t attribute credit for a majority of online conversions.
2. Lack of an agreed upon methodology for recasting the cost per action across the touch-points that played a supporting role. How far back do you go? How many impressions are worth one click? How do you split the credit across different types of media? We have our views and am sure others have theirs. And most are probably based on sound logic.
3. Acknowledgment that our ability to measure impact is severely limited by increasing use of multiple devices (work, home, mobile) and cookie deletion. We’ve seen for years that users often browse at work and buy at home. Now they are relying more and more on their mobile devices for browsing, making it pretty tough to figure out how and where they are becoming engaged and interested in our offer. For every action we can measure via cookies, there must be 3-4 that we can’t measure.
To sum it up, engagement mapping and attributing credit across touch-points is an important and useful approach. But it alone will not tell the whole story. Market testing and surveys should also be included in your toolkit for determining what works in online media.
Related articles and presentations:
Online Demand Generation: Strategy and Metrics
Making Sense of Online Campaign Results: Part 1
Making Sense of Online Campaign Results: Part 2
I hope you find this helpful or at least thought-provoking. Feel free to share with your colleagues, clients and propellerheads who are into web analytics and media modeling!
Steve Latham
Follow us on Twitter!
Find us on Facebook!
Contact us for more information
I recently posted a question on LinkedIn Answers about the quality of competitive web site data you can find at the free sites like Compete, Alexa and Quantcast. I’ve worked with Quantcast and Compete but I hadn’t heard of Alexa for quite some time (it was quite popular in the early days of the Web, but has not been nearly as visible in recent years). But due to recent events (described below) I had to quickly learn about Alexa so I posted the question to see if others had insights they could share. It didn’t take long until I was overwhelmed with responses. I was surprised by how strongly some felt about the various vendors and thought they would make for an interesting post.
Overall, most feel the info you get from Compete and Quantcast is pretty solid, but not entirely accurate. While it may not be 99% accurate as to the amount of traffic your competitors’ sites are receiving, it is consistent in its methods of measuring activity, so you can have a high level of confidence as to the relative difference in traffic and page views between your site and those of your competitors.
On the other hand, most felt the data from Alexa was very suspect and easily gamed. Because Alexa relies on browser plug-ins on individual computers to capture information, the results are reported to be somewhat easy to influence. One marketer noted that the only thing Alexa is good for is to manufacture metrics you can use to show a client how successful you were in marketing their site. Others were less critical but most felt the data was skewed and unreliable. See the screen shot below for actual comments.
Back to the reason I started down this path… I recently came across a situation where an agency used the Alexa ranking of a brand new site (vs. that of its peers) as the single metric for success. Not surprisingly, they achieved their goal of achieving a superior Alexa ranking in less than a month. Yet the same site doesn’t even register on Compete.com or Quantcast, and it has a Google Page Rank of 1. You can draw your own conclusions…
I decided to do my own test for my agency’s site and asked some of my team members to download the Alexa plug-in and visit our site each day for a few weeks. When we started the test on July 15 our site was ranked 990,000 out of 30 million. Just 2 weeks later, we are now at 730,000. At this rate we’ll be in the 600,000 range by August 15. If this plays out, it’s a pretty clear indicator that Alexa rankings are pretty easy to manipulate.
Here is a screenshot of some of the responses (sorry if it’s hard to read). Muy interesante!
Online Media Funnel
Last week I spoke at the Online Marketing Summit’s tour stop in Houston on Demand Generation. I was scheduled to speak in Dallas and Austin as well, but an unexpected foot injury / surgery sidelined me from travel.
At OMS I unveiled a new presentation that addresses the #1 objective of most marketers: generating leads, sales and other measurable results from online media. The presentation “Online Demand Generation: Strategy and Metrics” is embedded below for your viewing pleasure; you can also find it on slideshare. I started by defining “demand generation” (broader and more upscale than “lead gen”), the components of a demand generation program and various roles of online media. I also introduced engagement paths and the importance of defining the right metrics for success.
Also included is a practical methodology for measuring ROI and indexing performance against the market. As a bonus, I also included my view of the 10 worst and best practices for managing campaigns (would really like your feedback on these!)
Demand Generation Online Marketing Steve Latham 2009
View more Microsoft Word documents from Steve Latham.
I hope you’ll take this information and use the insights to take your business or agency to the next level. And as always, comments are welcome!
Steve Latham
http://twitter.com/stevelatham
Social media is hot. Everyone’s doing it and everyone wants it. But how many marketers have figured out how to use social media to build their brand and drive revenue? Unfortunately, not nearly enough. I believe one of the hurdles to pursuing social media as a marketing program is the challenge of creating a compelling business case that frees up the resources (budget) needed to fund it.
I recently spoke to a group of business executives about how companies are using (or planning to use) social media, and how to build a business case for it. In my presentation I also included some new data on how the Inc. 500 is using social media, 5 reasons to pursue it, and a methodology for measuring ROI.
You can view the presentation below or find it at slideshare (note: sorry for some of the formatting issues caused by slideshare conversion).
I hope it’s helpful and that you’ll provide some feedback for improving it. And if you have any good data points to support the case, please send them my way!
For more info you can use, view our blog. And for updates follow me on Twitter!
If you are reading this, you’re probably expecting to another pundit to start bashing display ads. Sorry to disappoint you but I’m actually going to defend the proverbial step-child of online media (while 3rd party email as the proverbial adopted child). If you are a step (as I am) or adopted (as my sister is) don’t take it personally. This is just a metaphor…
Now back to my rant… with the meltdown in the economy and paralysis that has gripped consumers, display ads are taking a beating due to their perceived lack of effectiveness. According to AdWeek, “Forrester Research expects display ads to come under the scrutiny of tight-fisted marketers uncertain of their effectiveness.” IMHO, the experts are taking a myopic view of the value of display.
I am not proposing that you invest heavily in display as your first buy. Your first online ad dollars should go to paid search; that’s where you’ll get the biggest bang for you buck. But if you are in a limited category or geographic area, Search alone may not help you make your revenue goals. There are only so many searches every day. And these days there are fewer than there used to be.
This is where Display ads can work very well. As we’ve seen firsthand, adding display to your mix, after optimizing paid search, is an effective way to increase awareness and create demand that eventually results in more site traffic, leads and sales. But unlike Search, you probably won’t see the direct link via click-thrus and conversions. Just as billboards (though we may hate them) create awareness, so do banner ads (when properly targeted. While Display ads may create awareness, they usually produce poor click-thru rates and even lousier conversion rates. Most often, the impact of a good display campaign will show up in the form of a lift in branded searches, SEM click-thru rates and direct visits. So you have to take a holistic view. Here is a chart (from a 1/09 client report) that demonstrates this concept:
For this campaign we quickly learned that search impressions were very limited. So to supplement search we started running display ads (4 weeks ago). While some ads had decent CTRs, most of the increase in traffic came from Direct navigation, branded search and paid search. As shown, the increase in impressions had a direct impact on site traffic. As long as conversion rates hold up, we’ll continue to invest in display. And given that Display Ad prices are falling faster than Wal-Mart closeout prices, this should become an even more attractive opportunity over time.
Caveat Emptor! While Display does have a place in the mix, you have to make smart buys. You need to target (demo, geo, behavioral, contextual, etc.), cap frequency and daily impressions, specify where they will (and will NOT) be served and have a good ad serving / web analytics system for reporting. If not planned and executed well, it can be a waste of time and money. But if done correctly, you can expand your category, increase awareness and preference, and extend ROI from your scarce marketing budget.
If you’d like to discuss or debate, comment below, contact me or look me up on Facebook or Twitter.
Peace!
Earlier this week I posted the first of two articles on the challenges of making sense of online campaign results. The first posting addressed the shortcomings of relying on cookie-based tracking to quantify results. This posting addresses the supporting role of display and email ads, and explains why Google gets more credit than it should for your online marketing success. I use an analogy we can all relate to… the proverbial Wingman.
The Wingman Effect
Another impediment to measurement is that most analytics platforms (including Google Analytics) were designed to attribute credit for an action to the last medium clicked. As noted above, user engagement typically entails multiple touch-points, both online and offline. Especially true with considered purchases, customers often make multiple visits, by way of multiple paths (e.g. display ad, search, direct nav), before taking action. Even if all of the visits are done on the same computer (and you COULD track using cookies), assigning credit for the lead or sale to the last click provides only part of the picture, and it generally rewards Search at the expense of Display, Email, Social and other media. 
Using a crude analogy, consider the proverbial Wingman. Picture two guys out at dinner and they see an attractive woman. You know the routine – one guy is the leader; the other is his Wingman. The job of the Wingman is to support his pal. He is often the one who initiates conversation and breaks the ice so his friend can move in for the kill. If the lead guy gets a phone number or email address, he knows he has the Wingman to thank for the assist.
In online marketing, Display is often the Wingman, the one who starts the conversation, while Search is the guy who gets most of the dates. But unlike the example above, Search gets all the credit and the Wingman’s contribution goes unnoticed. Moreover, because his contribution is not noticed, he may not be invited the next time they go out. Media planners often cut ad buys because they can’t see the supporting role they play in the engagement cycle. If they don’t drive directly actions, based on last-click analysis, conventional wisdom dictates you stop buying them. This ultimately works against you and will result in fewer conversations that lead to your desired results.
The takeaway for marketers is that you need to track interaction through the engagement cycle. Recently coined “Engagement Mapping” by Atlas (now part of Microsoft), savvy marketers are now tracking the first, middle and last clicks to determine how each media unit impacts results.
As we’ve seen firsthand on numerous campaigns, for every lead or sale you can directly attribute to an ad unit, there are 0.5 to 2.0 actions that are not traceable due to the reasons cited above.
While these are formidable challenges, do not despair – there are affordable, proven methods for overcoming both of them. Since this is how we make a living, I can’t share all the secrets with you. But I can provide some general recommendations.
First, you need to take a strategic approach to engagement mapping that will shed light on the various contributions (lead, supporting, etc.) your various online media units play in the engagement cycle. Once you understand which units create awareness, and which ones close the deal, you can produce smarter media plans.
Second, you should treat every campaign as a learning experience and make systematic media testing an ongoing program. By varying flight dates of various media, you can will gain better insights into the performance and contribution of your online media mix.
Lastly, you have to look at the overall lift in site traffic and activity, not just the visits that are directly attributable to specific ads. Don’t underestimate the tendency for people to take action on their 2nd, 3rd or 4th visits. Take a holistic view and you’ll see a much clearer picture.
As always, your comments are welcome, so make your voice heard. And if you think this is great – please share with your colleagues!
When I talk to marketers these days, one of the most frequently cited needs I hear about is the ability 
to measure (and make sense of) results from online campaigns. More specifically, the gap in most organizations is their ability to trace results (e.g. engagement, leads or sales) back to individual ads, email blasts or search phrases that drove the desired results. They have data but it often doesn’t make sense. While many have figured out how to use tools such as Google Analytics for measuring conversions and attributing those to the ad unit (e.g. display ad or search phrase) that preceded the conversion, the results often do not make sense, or are contradictory to what their media plan said should happen.
If you are in this category, don’t despair – you are in good company. My personal research indicates that 94.59% of marketers are struggling with the same issues. I believe the challenge of measuring results that make sense is two-fold: 1) shortcomings of cookie-based tracking, and 2) the fallacy of last-click analysis, which I refer to as the “Wingman effect”.
Crumbling Cookies
As consumers become smarter, savvier online users, they are becoming more deliberate and less 
impulsive in their decision process. With so many more options at their fingertips they can easily do research and comparison shopping before buying that new camera or requesting information on your services. For big purchases, they often confer with others, e.g. sending a link to that new road bike to their friend or spouse to get their input). Another issue is the growing trend of “surf at work, buy at home”, where they do research on one computer and take action on another. Since we rely on cookies to track actions for each individual, these issues impact our ability to measure results. Here’s an example:
Let’s say you are at work, and just as you are thinking about how much you need a vacation, you see a VacationsToGo ad for a Caribbean cruise on your MyYahoo home page.You click through and like what you see but you have a lot to do and cubicles don’t offer much privacy for vacation shopping. Later that day you tell your spouse about the trip and tell him to go to VacationsToGo.com to learn more. He Googles it and finds it through a paid search listing. Later that evening, in the safety and comfort of your home, you jump on your personal computer and navigate directly to VacationsToGo.com to book the trip. Five minutes later, you are thinking about where you’ll eat in Cozumel.
Since the site sees that your home computer does not have its cookie, it assumes you have not visited before. Consequently, it views you as a 1st visit buyer, and does not know that you responded to the MyYahoo ad (or that your spouse found the site through a paid search ad). Both of the prior visits will appear to be a waste of ad spend. The poor analyst who has to measure performance of ad buys has no clue that the Yahoo ad started the engagement and that the paid listing contributed to the process. He will only see that at 7pm a 1st time visitor booked a cruise.
The example above illustrates how multiple visits and machines reduce the effectiveness of cookie-based tracking. The increasing use of cookie-cleaning tools adds to the dilemma. If, as widely reported, 40% of 3rd party cookies are either not accepted or deleted within 30 days, we’re blind to what impacts a significant portion of our results. Consequently, it’s difficult to take an accurate measure of which media buys are performing, and which are not.
So with that we conclude the first part of this subject. In my next post I will introduce the Wingman and give new hope to display media salespeople. But for the mean time, please tell me what you think! If you found this to be of value, please comment below.
Later!