I would like to thank Google’s AdWords for this title. You will see why in a bit.
One of the key things you can do to dramatically improve your marketing is to target your messaging. Whether you are marketing a consumer web service or an enterprise hardware appliance, good messaging can make a big difference. In large organizations, most messages are shaped by someone in the marketing department — they’re in marketing so they should know, right? In many startups messaging is an afterthought — something some engineer figures out just before shipping product. In both cases, messaging is treated as art (or annoyance) rather than science. While there is a place for art, there is also a place for science…
One of my favorite tools for doing fast and cheap empirical message testing is Google AdWords. For example, I created an AdWords campaign in about 10 minutes to A/B test eight titles for this post.
First I inserted the description, the display URL, and the target URL. Since we are A/B testing the title of this post, everything else was held constant. You can get into testing multiple variables at once, but let’s keep it simple for now.
I also went into “Edit Settings” and changed ad serving to “rotate” from “optimize” — we are simply trying to test messages here, so an equal messages distribution is ideal here.
I then wrote this post and went to sleep. When I got up in the morning, this is what I found.
It cost us $10.87, 15 minutes of work, and less than 10 hours of testing (much of it while I was sleeping) to run this test. You should clearly spend more time than I did in crafting your messages. You should run the test for a longer period of time and test more variations on the messaging. And you should be thoughtful about the keywords you select to test them. This need not be the only source of data in making your decision about messaging and positioning.
The goal of this experiment was to show you how to leverage the hundreds of millions of Google search consumers and hundreds of millions of engineering investment by Google in Search and AdWords to run a very cheap market positioning and messaging experiments. Thanks GOOG!
*Note, November 9, 2008: I just found mention of a similar test that was done using AdWords to pick the title of a book — Super Crunchers (page 60 of the paperback).
13 responses so far ↓
Peldi // July 16, 2008 at 7:32 pm |
Thanks for the tip Mike, I will try it!
sabyberry // July 16, 2008 at 8:07 pm |
Great post Mike!
damian // July 16, 2008 at 8:09 pm |
Great tip. This would have saved us a few hours a few weeks ago.
zooie // July 16, 2008 at 9:17 pm |
nice!
it would be interesting to also try this experiment on something like mechanical turk. ask each worker to choose the best title from a randomized list (should be small) of candidates. after k users have voted, take the most popular one as your answer.
would be interesting to compare the results, costs, and time it takes to reach some metric of statistical significance on each platform.
– vik
Stanislav Shalunov // July 16, 2008 at 10:57 pm |
Testing messaging is a great idea.
In this case, however, all the CTRs are very low — perhaps because the keywords were insufficiently relevant. This is the sort of CTR you might get on a social network, not on a search engine.
When using AdWords for message testing, one would need to control for position. Position is determined by max bid and quality score. I presume you used the same max bid. Quality score has a zillion factors, but one is simply the occurrence of keywords in the ad. So, if you bid on keywords that like startup, marketing, and advice, the ad would have higher QS and higher position and higher CTR just because it has these words.
One could use position preference to try to put the ads in the same position. Position preference, however, is unreliable and starts working only after a few days. Further, if the position you choose is 1-3, quality score can still shift the ads between top and right, where CTRs are different regardless of the ad text. And if you choose a lower position, it’ll take ages to collect enough clicks for statistically significant results. (The test in this post would need to run longer.)
Note that Facebook’s social ad platform has none of the problems: you don’t need to choose keywords, you can control ad position, and you don’t get rewarded or penalized for having the keywords in the ad (as there are no keywords). You can also run the experiments faster.
Facebook ad platform gotchas: disable social actions; fix position on the side; choose a demographic you actually care about.
dave mcclure // July 27, 2008 at 7:11 pm |
excellent suggestion… however, let me one up this a bit further.
before you test conversion / CTR on your messaging, simply identifying relevant keyword volume (& cost) is quite helpful for many startups.
process i would suggest:
1) start by building a list of 5-25 keywords you think are revelant to you & your audience (brands, products, competitors, problems, solutions, etc for your market).
2) use Google Adwords tools to identify high-volume keywords & semantic equivalents.
3) prioritize keywords by volume & cost, and THEN start working on ad copy, campaign concepts, graphics for landing pages.
4) consider using the keywords not just for SEM, but also for SEO — you know have a sense for what keywords you should be targeting to rank for on SEO, and you can vary landing pages / home page graphics by checking for referral keywords on organic traffic.
5) use the keyword list for your blogging. post every day with some variation of keywords in the post headline, and develop content around your keywords / market. you’ll probably start ranking for traffic quickly, unless it’s a highly competitive market. even just a few hundred readers can start getting you some decent organic traffic to test market with.
6) consider using your keywords for email copy, if you collect email addresses. test market the same copy you use for adwords in the email subject lines.
in summary: understanding keyword vocabulary for your market, and the volume / cost / conversion of those keywords in various areas of your product & marketing functions is incredibly valuable.
Orange County Viral Marketing // July 31, 2008 at 8:23 pm |
ive seen this done in my previous job. this is definitely a great tip. thanks
Ms. Single Mama // August 4, 2008 at 2:03 am |
Stanislav –
But what about the poor performance of Facebook ads, something the social network is struggling to fight right now. No one clicks on them because they don’t want to leave. Banner ads have a 2-5% click rate, 5% is being generous.
If I were a small biz I would never put my money in Facebook. It costs SO much. Organic search is a much more economical way, especially when you can send your message directly to Facebook users by creating a viral buzz.
Startup Marketing Advice from Balsamiq Studios | The Balsamiq Blog // August 5, 2008 at 8:19 am |
[...] of this post is a bit pretentious for my taste, but I wanted to pay homage to Mike Speiser’s excellent post on A/B testing using AdWords, and take advantage of his $10.87 investment while I was at it!
. Mike is one of the brightest [...]
Vasco // August 5, 2008 at 1:04 pm |
Ok Mike, you made my day with one. Thx a million for this convenient exercise.
kareem // August 5, 2008 at 9:24 pm |
we did this at edufire when choosing a name and i’ve used this to make decisions on a couple of other experimental ideas i’ve been playing with. it’s a great technique to gather real data quickly.
more research & stuff « NEW MEDIA GROUP 13 // October 30, 2008 at 12:54 am |
[...] http://laserlike.com/2008/07/16/startup-marketing-advice/ [...]
Revue de presse | Simple Entrepreneur // December 26, 2008 at 5:52 am |
[...] Startup marketing advice Comment trouver l’accroche qui convertira le plus de visiteurs en membres payants ? Comment sélectionner le message le plus percutant (et surtout le plus efficace) ? Voici une petite astuce basée sur les possibilités offertes par Google Adwords. [...]