Laserlike

Inside Steve’s Brain and “Vectors going in time.”

I read Inside Steve’s Brain this weekend.  It was a quick read, although the book isn’t one that I would recommend to most of my friends (except the extreme Mac-heads out there).  However, there was one point that I wanted to explore further — Kahney discusses how Steve Jobs pays keen attention to the convergence of various technology trends (which Jobs calls “vectors going in time”) to do things that were not possible previously.  

The iPod was made possible by cheap, small storage (1.8 inch drives from Toshiba), the proliferation of digital music (ubiquitous PC CD burners helped to make that possible), broadband penetration in the home, and advances in the manufacture of plastics and later metals.  YouTube was made possible by ubiquitous availability of Flash in the browser, cheap storage and bandwidth, and broadband penetration in the home and workplace.  

So what are some interesting “vectors going in time”?

There are a number of interesting trends that have the potential for major disruptions.  Here are a few that come to mind (please contribute your thoughts in the comments of this post):

+ Cheap and portable storage.  

Flash memory is going to change the game — by both reducing the space required for storage and because flash memory is so durable.  What devices can be re-invented by leveraging Flash memory?  On the other end of the spectrum, storage capacity (of the spinning disc variety) is growing at such a rapid pace that it’s just about free.  What can you do with infinite storage capacity?

+ General purpose Graphics Processing Units (GPGPU).

GPUs are brilliant at offloading (from the CPU) highly intensive graphics tasks.  Graphics is an inherently parallel task (every pixel is separate), so using many GPUs to handle computation is a superior solution to jamming everything through the CPU.  With data volumes exploding, are we finally ready for parallel computing?  Are there more tasks ideally suited for this technology?  Will something emerge that offers a layer of abstraction that allows the massive investment in existing programming techniques to be sent to a cloud of GPGPUs?

+ Distributed file systems + MapReduce.

Projects like Google’s GFS/MapReduce and the open source Hadoop are frameworks for running applications on highly distributed commodity hardware.  Google’s entire search index was rebuilt on GFS/MapReduce as soon as the software was ready.  These frameworks are critical in order to process petabytes of data.  They are also relatively new and improving at a rapid clip.  How can we take advantage of Hadoop to do cool stuff with huge data volumes?

+ Machine learning.

With infinite storage capacity and compute cycles, what can we do with all of this data?  Kevin Kelly wrote a great piece called The Google Way of Science.  Machine learning is going to evolve beyond it’s limited use today in bio-informatics, web search, and anti-virus products.  What really hard problems can we attack today with machine learning which were previously impossible?

+ Ubiquitous wireless access + AJAX.

Just as fixed-line broadband created many new opportunities, so too will the ubiquitous availability of wireless — 3G, GPRS, CDMA/GSM, WiMAX, Wifi, or Bluetooth.  This plus the move to AJAX (asynchronous Javascript and XML) will finally allow server-side computing to own the planet.  What applications benefit most from mobility?  

+ GPS everywhere.

GPS will soon be a default in all phones.  The cost of GPS will eventually be low enough to embed it in all devices.  With wireless everywhere, devices will both be able to tell us where they (and we) are AND communicate that information to server-side services real-time.  What services would be dramatically better by having locational information?

+ New human-computer interfaces.

A friend of ours picked up a new iPhone — her four year old daughter picked up the phone when she wasn’t looking and started using it.  Not the way a four year old uses a Blackberry, but really using it like an expert.  The touch screen interface is such a natural way for humans to interact with machines.  

The Wii has taken the lead in the console wars with inferior graphics processing capabilities — the reason because Nintendo is innovating on a different vector — human-console interactions.  The Wii controller’s gyroscopic interactions with the console and Wii Fit’s accelerometer has a nailed virtual reality.  How can we use these emerging interfaces to build better web experiences?  The iPhone also has an accelerometer.  Hmm….

Our post on the Optimal startup burn rate and the Kelly criterion was extremely popular as it provided an actual model to help people think through startup burn rates.  Hopefully a post with an actual model to help people think through viral marketing will be useful, too.

First, go here and download “viral_tuning.xls.”  Click the down arrow sign on the iDisk folder (you will need Safari on Mac or IE on Windows as Firefox does not seem to be playing nice with iDisk), which will download the file for you.  Make sure that you have enabled macros (from the Excel menu:  Tools -> Macro -> Security; the latest version of Excel for Mac does not support Macros).  For those of you who don’t have Excel, you can find a neutered version of the model on Zoho here (no Macros).

The BLUE cells are inputs (can be edited).  The BLACK cells are formulas (don’t touch them, unless you are checking the formulas for accuracy).  Play around with the BLUE cells to see what it takes for a service to be viral.  Then click one of the gray “SOLVE” buttons above any of the steps in order to optimize virality for just that step [assumes the other variables are held at a user-defined constant].  In this example, I have taken the registration flow of a typical social networking site. 

Step 1:  A user shows up at your site.  What percent complete registration?

Step 2:  You offer the user the opportunity to invite friends.  What percent use the import tool (tool to grab contacts from Gmail, Yahoo! Mail, Hotmail, whatever social network, etc.)?

Step 3:  How many friends on average do users invite?  The emails go out to that many users and then you go back to Step 1. 

The purpose of this spreadsheet is to help you think through just how hard it is to make a viral product.  You should lay out the flow a user goes through when interacting with your product in steps just as I have done here.  Try to understand the key drivers that encourage people to get their friends to visit your site — and optimize around those by developing and testing hypotheses in order to experience viral growth.  The steps need not happen online (the same underlying principle applies for offline retail), but when the entire flow is online it’s obviously easier to measure.

The flow outlined in this model was key to building a number of social products on the web today — but consumers are tired of this strategy.  Still, it should help you think through what virality means and just how hard it is to achieve a truly viral product.

I’m sick and tired of spam and even more ticked off about false positives.  Machine learning (ML) researchers keep on telling me that ML is going to make the spam problem go away.  Yet, for this user, it’s worse than ever.

Anti-spam technology.

Don’t get me wrong — huge strides in anti-spam technology have been made.  Paul Graham’s essay A Plan for Spam discusses a now popular technology – Bayesian filtering.  According to Sophos 92.8% of all mail sent in the first quarter of 2008 was spam.  Wow.  So spam technology is taking care of most of the problem, but that’s just not good enough. 

Amazon’s Mechanical Turk.

The idea of artificial artificial intelligence (AAI) isn’t new, but the term AAI comes from Amazon’s Jeff Bezos.  From Wikipedia:

Artificial artificial intelligence (AAI) is a term coined by Jeff Bezos. Certain computational tasks, such as identifying whether a person in a photograph is male or female, are tasks that humans still do better and faster than computers. If perfect artificial intelligence systems existed, computer programs could complete those tasks. The idea of artificial artificial intelligence is to outsource those parts of a computer program to humans.  AAI is the underlying principle behind Amazon Mechanical Turk.

Amazon’s Mechanical Turk outsources human labor at the task level.  You get paid in HITs (human intelligence tasks) for doing tasks that companies, individuals, and organizations pay for on a per HIT basis.  

AAI + Spam = Nirvana

We should turn down the intensity of the anti-spam filters to avoid false-positives (I have several important messages — on the sending and receiving side — go into spam EVERY WEEK).  We should then supplement anti-spam models by having real humans evaluate messages which fall in the gray area.  

One option is to literally use Mechanical Turk as part of an anti-spam solution.  Take the messages that are clearly spam according to your model and toss it.  Let the kosher mail pass without harassment.  But then send questionable mail to the Mechanical Turk to have humans help you figure out what to do.  Another solution is to build a fully-integrated Anti-spam Turk.  

There are clearly privacy issues which need to be managed carefully.  But I’m confident that those issues can be solved with technology better than relying on AI to understand the difference between spam and important mail.  Until SMTP is updated to include a much higher level of authentication/security, spam will be an issue.  We either need to leverage some cost-based model like bonds, or we need to supplement AI with a little AAI.

In my last post, The Product, Part II: Technical architecture and the innovator’s paradox I talked about the importance of staying in the game and linked to a Wikipedia article on the Kelly criterion.  In the comments, entrepreneur and physicist Max Skibinsky took the idea literally and used the Kelly criterion to calculate the optimal burn of a startup.  I was so impressed with Max’s comment that I imported his Google spreadsheet into Excel and played around with it:  here is an editable copy of [my updated version] Max’s spreadsheet.

Max’s math:

p = probability of success
b = payout odds per kelly
F = funding
V = valuation
M = valuation multiplier on “win”
R = burn rate per time frame
T = time frame units to develop and prove

What we can learn about optimal burn from the Kelly criterion.

You obviously shouldn’t take this too literally.  But I do find that it is a very interesting reality check.  

Assumptions in the spreadsheet:

+ Capital raised to $2MM on a $6MM post money valuation.  

+ 15% chance of any experiment returning a 10x increase in valuation.

+ 9 months to build and test each experiment in the market.

+ $100,000 per head (includes salary, benefits, rent, computers, marketing).

Using Max’s spreadsheet which is based on the Kelly criterion’s probability of maximizing long-term returns, the optimal monthly burn is $32K, which would cover 4 heads.  This would give you capital for 7 experiments.

A few brief thoughts:

For any hit driven (or wildly innovative) business, you should assume that [at least] your first experiment will fail.  This will remove pressure and allow for maximum flexibility.  It also drives how you should build your product and manage your finances.  It also drives the following recommendations:

1.  Keep burn very low until you have proof of traction.  

Everyone knows this intuitively, but the vast majority of startups spend an order of magnitude greater than their target Kelly burn.  You can reduce burn by hiring fewer people, keeping salaries low, working long hours, and hiring very productive people.  Most people focus on keeping salaries low bit, but my experience is that hiring a few exceptional people at higher salaries is cheaper than hiring more [less productive] people at lower salaries.

2.  Raise more money than you need.

Easier said than done — but if you have the opportunity to raise a bunch of capital, you should seriously consider doing so.  Figure out the optimal number of people needed to run an experiment and use the Kelly burn spreadsheet to impute required capital.  The cost of giving up more equity early on is often more than offset by the increased flexibility to take chances.  There is obviously some equilibrium point in there between loss of present value as a result of taking too much equity capital (for the entrepreneur) versus loss of present value as a result of taking too little capital and putting too much capital into a single bet or few bets.  Many entrepreneurs can’t raise more capital, but those who can should.  

3.  Increase the probability of success on each experiment.

This is clearly the highest leverage point in the model.  You can increase your odds of success by (a) picking a big existing market (rather than trying to invent a market, reinvent an existing one); (b) recruiting a killer team; and (c) picking great investors.

You can also increase your odds of success by building and shipping product quickly, by instrumenting your site / product so that you can run tests and make data-driven decisions, and by killing failed experiments quickly.   

For an entertaining history on mathematics, information theory, economics, gambling, and the mob check out Fortune’s Formula.

I would like to thank Jeff Cordova for his extensive feedback on a draft of this post.  

For consumer web products, my rule of thumb is that it should take 90 days to get from IDEA and TEAM to your first external [closed] Alpha.  In Part I of The Product, I suggested leading with design.  I do want to be clear that: (1) during the period of time when your designer is working on mockups, the rest of the team should be busy working on key technology choices, and; (2) that the entire 90 day period [and beyond] is about constant iteration.  

Hopefully the entire team is sitting near each other, because as soon as you divide up to work on your own task problems that you hadn’t considered will quickly surface.  By having everyone there together, you can knock those problems down quickly and move on.  As every entrepreneur quickly learns, where you end up is a very different place from where you started.  As Jeff Cordova reminded me in his feedback on this post, “nobody has ever built a *world changing* product using top-down thinking. Instead, it requires dozens or even hundreds of product iterations to get there.”

Make a priority ordered list from 1 to 10 with the key features that you want to build.  Push yourself to draw hard lines between each feature?  Is Feature #1 by itself an interesting product?  For a web product, that should be your goal — adding features is easy, but taking them away is hard.  Your mocks should focus only on what you will ship in 90 days; focus is key in the early days of a new product.

The architecture.

All new web applications should follow the model view controller (MVC) pattern and employ a standard framework.  Examples of standard frameworks include: Rails (for Ruby) and Spring (for Java).  More than the language choice, these frameworks provide enormous leverage.  They will allow you to move unbelievably quickly — an unfair advantage startups [can] have over larger companies which always devolve into a congealed mass of heterogenous technology crap. 

The innovator’s paradox.

There is a paradox here:  innovative people question everything, but success usually requires that you focus your innovation on the one thing that will make a big difference.  Something that will either provide dramatically better value for consumers than the existing alternatives or something that will lower the cost of doing business dramatically (in a segment where lower costs are a key success factor).  When these innovative people invent stuff that doesn’t need inventing (or at least that isn’t a major contributor to your core area of focus), they create unnecessary risk and lower the probability of success.  

In investing there is a saying, “stay in the game.”  The magic of compounding growth means that the number one priority of an investor should be to avoid taking a risk of going to zero.  Entrepreneurs should heed this advice — make smart technology choices that will keep you in the game.  On every technology choice you face, ask yourself if this architectural innovation is key to your success?  If not, go with standard technologies and focus your engineering cycles on building a great product rather than innovating on architecture.

If your engineering team is strong, they will figure this out.  If they are young, they will take more risks.  This risk taking will increase innovation on your core product (good), but it will add unnecessary risk elsewhere (bad).  Make sure that everyone understands what the potential rewards of taking those risks are before you decide to innovate everywhere.

It’s time to start writing code.

Set some goal about when you are going to pull your first internal alpha together — six weeks is half way to your external alpha, so that’s a good target.  There will be some setup and startup time, but you should move to weekly sprints as soon as you can (Friday late afternoons are good as consumer web usage is low then — in the early days you will take the site down for pushes, but you will eventually move to hot rolls).  Of course some projects will take longer than one week, but setting a release cadence will help the team stay focused.  And nothing energizes a team like seeing a product come together.  Once you are public, each release will also make your users feel like they are part of the innovation process.  

You have the IDEA and you have the core TEAM, which hopefully includes a great designer (at least as a contractor, but preferably a co-founder or first hire).  So now what?

BusinessWeek had a great post on Apple’s design process a few months ago.  Here is a snippet from the interview:

10 to 3 to 1
Apple designers come up with 10 entirely different mock ups of any new feature. Not, Lopp said, “seven in order to make three look good”, which seems to be a fairly standard practice elsewhere. They’ll take ten, and give themselves room to design without restriction. Later they whittle that number to three, spend more months on those three and then finally end up with one strong decision.

People wonder why Apple continuously builds such great products.  How is it possible?  A big part of the answer is that Apple starts product development with pixel perfect mockups.  Many would argue that the 10 to 3 to 1 process which lasts months is too cumbersome for your average startup.  However, the likelihood that a random design will hit the mark is near-zero.  Isn’t spending several years of your team’s time and [and possibly, your investor's money] on a crappy product quite a bit more cumbersome?  As Microsoft has learned, it’s incredibly hard to fix bad interaction design after the fact.  

You needn’t copy Apple’s approach “pixel for pixel.”  But doing several rounds of INTERACTION DESIGN and VISUAL DESIGN mockups is well worth the cost.  Just make sure that you hire a great designer who gets interactions.  Interaction design does not equal visual design — but you can find individuals who get both.

Visual design is important, but the best overall design is the one that goes with the grain of human nature — that’s interaction design.  And very few product managers and engineers get interaction design.  

Once you have click-able (HTML mockups) ready, run a quick and dirty usability test.  Invite friends over and ask them to do a few key tasks.  Videotape their behavior.  How many clicks did it take for them to accomplish your desired task?  How can you improve your product by cutting clicks to achieve a task in half?  How long did it take them to find the link they wanted on a page?  How can you improve that?  Then iterate at least once more…

Once you have something that you are proud of, you are ready to START writing code.  

While it’s hard to test business people, that’s not the case with design and engineering.  If you have already worked with someone before, you’re golden.  But if you haven’t, there are some simple things that you can do to increase the odds of success.

1.  Tests.

At Bix Jeff Cordova put new candidates through code tests.  He had a candidate write an application over a few hours and knew at the end of the meeting the candidate’s strengths and weaknesses with incredible precision.  This was far better than reviewing a resume line item by line item.  

You can also put a designer through a test by asking her to design something with you real-time.  When you get to the point where you need product management, “working” on a product management specification for some simple hypothetical product will teach you much more about the candidate than the schools the person attended.

2.  The cultural fit.

Tests should reduce the probability of hiring a poor performer to near zero.  But a bad cultural fit can ruin your mojo.  So after someone has passed the code or design test, have a plan for determining cultural fit.  We had a few folks who were really good (and very focused) on assessing fit, so they interviewed every candidate.  Doing something social with potential co-founders or candidates is also important.  Are these people you want to hang out with?

3.  The sell… and anti-sell.

Once you know you want someone, do whatever it takes to get her.  Sell her on the future potential of the business.  Help her see how your project has the potential to do great things, not just make money. But before she signs up, make sure that you do the anti-sell.  Tell her all of the things that suck about the job.  Be honest.  Actually, over do it.  If you turn her off, she was wrong for the job.  But if she takes the position, she will be ready for the downside (i.e., long-hours), but will be pleasantly surprised that you exaggerated the negatives.

So now that you have your idea, you can go one of two directions:

1. Build a prototype yourself.

If you’re an engineer and the project is something that you can build and launch in your free time or you can cover your own expenses for some period, that may be a good approach.

2. Build a small team.

If you’re not a great engineer, you must go down path two. Another benefit of this path is that good ideas become great ideas through debate and iteration. But it’s often harder to coordinate the work of more than one person in your free time. So this path, for all but the independently wealthy, will likely require that you find a source of funds sooner than path one.

Let’s assume that you go down path two, because sooner or later you will likely need to recruit a team anyway. So who do you need?

For a web-based business, my take on the perfect team includes:

+ 1 designer (interaction design, visual design, and CSS).

I’m a big fan of starting with very detailed interaction flows AND visual design before you write one line of code.  This is not the standard design approach for the vast majority of products, which is one reason so many products suck.  I’ve made this mistake too many times myself.  Spend the time upfront to get the user experience right.  It’s worth it.

You don’t hire a designer to do your logo, although she can do that.  You don’t hire her to do the visual design, although she should be able to do that too.  And you don’t hire her because she is a black-belt in CSS, although that’s a big plus.  You make a designer one of the first recruits because a great designer will help you get your interactions right.  And it’s much better to do that from the get-go, rather than to try to retrofit design after the fact.  

+ 2/3 engineers.

It depends on what you’re building, but it’s always nice to have more than one engineer to discuss ideas. If what you are doing requires a good bit of Javascript or Flash, you likely want to make one of your engineers a front-end engineer. If not, good core engineers can get the basic Javascript / Flash stuff done.  

Don’t hire the perfect VP of engineering.  Get doers.  It’s easier for doers to learn how to manage than it is for managers to learn how to do.  Find extremely smart people who can help you think about and build the product.  Engineering is not manufacturing.  Engineering is about problem solving.  Your idea articulates the problem and a high-level solution hypothesis.  The entire team needs to work together to solve the problem.  

In the early days, all engineers should be general purpose — everyone writes code, does their own QA, and someone on the team takes the lead on operations (most web businesses should outsource their servers in the early days).   

That’s it.  I believe that a 3-5 person team can build v1.0 of just about anything on the web.  If you’re not the designer or one of the three engineers, you’re overhead.  Which means that you either shouldn’t be on the team or that you are the CEO and that you should be doing everything else, including getting space, ordering equipment, raising capital, negotiating deals (content acquisition and/or distribution), and doing the dishes.  

I usually start with a consumer problem, often one that I have experienced.  For example, how do I invest my money and beat other investment alternatives?  How can we improve the “user experience” of airlines (a thought I have every time I fly)?  How can we improve our government?  How can I collect data from other sources to harden my assumptions in a model?  I try to come up with at least one problem each day, which isn’t hard since I question everything.  I then use one or more of the methods below to solve the problem/s I have identified.  

1.  The world view.

This approach is based on some deeply held belief that you have about some under-appreciated truth.  For example, I believe that problems involving common knowledge (Karl Popper’s term) are ideally suited for distributed solutions (e.g., Wikipedia, markets) while those involving scientific knowledge (Popper, again) are best solved by extremely small groups of brilliant people.  

Steven Landsburg shares his world view in The Armchair Economist.

The world abounds with inefficiency, and to the untrained eye much of it seems to be the result of “cutthorat competition” or “markets run amok.”  But the Invisible Hand Theorem tells us that if we seek the source of inefficiency, we should look for markets that are missing, not for markets that exist.

We already do this implicitly when we think about problems.  I’m suggesting here that you explicitly develop a point of view and use it to solve problems you have identified.

2.  The logical extreme.

I typically take an economic or technical trend to an extreme and ask, “what if.”  For example, storage costs are decreasing faster than Moore’s Law.  What if storage is free?  If you could provide infinite storage at no cost, how would that change things?

What if GPS chips are embedded in every phone, computer, and car in the future?  What if those devices could pinpoint the location of every consumer on the planet at anytime?

China has moved 250MM people into urban areas over the past few decades.  What if they move another 300MM into urban areas?  How does that change things?

3.  Less is more.

This approach has been popularized by 37 Signals and Evan Williams.  College students are using social networks.  What if you just provided a social network for them?  Blogging is crazy popular, what if you had a 140 character blog?  What could you do?  

The idea here is to look at bloated products and pick off one small user segment or feature and do it really, really well.

Next up, The Team, Part I:  Who do you need, and when do you need them?

The July 2006 issue of Wired had an article entitled What Kind of Genius are You.  The short summary is that there are two types of innovators –- those who peak early (CONCEPTUALISTS) and those who bloom late (EXPERIMENTALISTS).   I don’t buy this argument, nor do I agree with those that argue “you better innovate by some young age or else”. 

Here is an alternative innovation hypothesis:

The human brain’s compression algorithm is pattern recognition.  It’s a shortcut whereby the use of patterns allows us to rapidly make decisions rather than storing every bit of data and running a complete analytical process for every decision.  When confronted with new data, our brain checks for patterns and fits new data into existing patterns.  New data often feels like old data, because our brain processes patterns so quickly. 

This process allows us to efficiently go about our daily business.  Unfortunately, bad patterns get stored alongside good patterns.  To paraphrase Richard Feynman, innovation is identifying bad patterns and replacing them with good patterns through the scientific process. 

We acquire wisdom (patterns) with age.  Young people lack the wealth of wisdom that they will have in old age, but they also lack the biases of years of accumulated bad patterns.   So while I believe that anyone can innovate, you must have an open mind and question everything at any age.  Ignoring patterns, therefore, is easier to do when you don’t have as many (when you’re young), so it’s no surprise in a probabilistic sense that young people are innovators more often than older people.

But innovation is possible at any age.  Question everything.  And then question everything again.  It’s not easy, but it’s the key to innovation.

Next up, The Idea, Part II: Flint for the brain.

I had lunch with Venture Hacks founder Babak Nivi this week – he suggested that I add more execution posts to the mix of IDEAS and EXECUTION.  Nivi’s recommendation sounds like a great idea, so I am going to spend the next few weeks on EXECUTION.

While I come up with new ideas everyday, my thoughts on execution are experiential and sequential.  Consequently, I will start the execution posts at the beginning (ideation), then work through company formation, building a team, and conclude with thoughts on exits (in particular M&A, because that is where I have had the most experience as a seller and as a buyer).

Hopefully other entrepreneurs will offer alternatives to my suggestions in the comment section just as they have on the idea-oriented posts.

Up next The Idea Part I:  Question Everything. 

In Shadow Market:  Money management by the masses, we discussed the idea of distributing the management of a mutual fund to consumer traders.  The popularity of this post got me thinking, “can we help the U.S. Congress do a better job managing tax revenue?”  

Town Hall:  Providing unsolicited “guidance” to our elected representatives.

1.  Start with the 2007 US buget as our “stram-man.”

Build a taxonomy of every policy issue based on the US budget.  Allow anyone to navigate the budget from top to bottom or from bottom to top.  

2.  Give every voter her “fair share” of budget dollars to allocate.

Take the entire U.S. budget and divide by registered voters.  

Each voter gets to allocate his share of the US budget as she sees fit.  We will develop an algorithm that uses the votes to date as a sample for the overall population.  In addition to all existing line items, “give me back my money” will be an available option.  At any point in time, votes can see how the “crowd” would allocate the US budget versus how our government is actually spending our money.

3.  Develop discussion forum around on every line item.

This could become the basis of a Politics portal and could really change the way the government spends our money.  And I’m sure that we could do a better job of managing our money than our representatives.  If the Founders had the internet in 1776, this is the way they would have wanted it

I recently picked a friend of mine up for lunch at Logitech’s Squeezebox offices in Mountain View, CA.  He gave me a tour of the latest Squeezebox products, which are a devices that play digital music though an 802.11 connection.  You can put a Squeezebox in any room in your house and get high-quality wireless music.  However, since all of my music was on iTunes, I decided instead to pick up an AirPort Express, which extended the reach of my AirPort and let me play iTunes music from my Macbook Pro in my bedroom on my stereo in the family room through AirTunes.

Then another friend of mine raved about the Amazon Kindle.  It has changed his life, and this is only version 1.0!  Perhaps I will pick one up at v2.0.  Finally, another friend told me about the Chumby.  The Chumby is a multi-purpose home appliance that serves as an alarm, a music player, and thermometer.  It is a cool looking device, and it’s only $180.  Sweet.

Apple has announced its 3G iPhone, which will finally make surfing the web with a phone something that doesn’t make me want to pull out my hair (what’s left of it, at least).  The alphabet soup of Wi-Fi, RFID, GPS, 3G, GPRS, CDMA/GSM, and WiMAX is converging into a wireless melting pot of data and location goodness.  And the Squeezebox, AirPort Express/AirTunes, Kindle, Chumby, and iPhone are just touching the tip of the proverbial iceberg.

Try this though experiment — what could you do if you embeded the ability to data (sending and receiving) or broadcast location into every device in your life (home, shopping, work).  Chumby did just this with the alarm clock.  I will touch on a few ideas (the result of 10 minutes of brainstorming).  Would love to hear your crazy ideas, too.

Ideas.

1.  The wireless shopping cart.

While much has been written about putting RFID tags on every piece of food to improve the efficiency of the supply chain, there is a much simpler place to start.  By simply putting the same cheap tags on each shopping cart, a retailer could map consumer flows through the store.  They could match time spent in certain locations with actual purchase data and run A/B tests to determine appropriate pricing and product placement.  

Once all products are tagged, the shopping cart could add a simple screen to show the buyer a list of what’s in the cart and the current balance.  It would also be great to get some navigation assistance based on things like “The South Beach Diet,” “organic”, etc.

2.  Eye in your pocket — wireless casino chips.

Casinos talk about the eye in the sky (closed circuit cameras put in place to monitor gamblers).  A buddy of mine, Terry Noonan, suggested this idea to me several years ago (on a business trip to Las Vegas, it turns out) — what if you put an RFID chip in casino chips?  You could actually track the flow of money throughout your casino.  It would be a nice addition to having pit bosses use back-of-the-envelope estimates about the average size and duration of your betting — particularly when gamblers get up and move around.  Casinos would also have a definitive estimate of chips leaving the casino, which would limit fraud.

3.  The friend navigation system.

You know the navigation system in your car?  The one that helps you find places?  I want one of those for my friends.  Someone should use Facebook Connect and the GPS in the new iPhone to help me get directions to where my friends currently are.  Of course, this doesn’t work great until all of your friends have GPS-enabled phones.  But that’s not too far away.

4.  The really smart fridge.

It would be nice for manufacturers to include expiration data in the RFID tags for food.  They should also work with appliance vendors so that consumers get a benefit, beyond lower prices due to efficiency gains from better supply chain management.

It would be really great if my fridge could tell me when food items have gone bad and need to be replaced.  I would also love to see a mashup of what’s in my fridge with a “what can you make with the ingredients you have in your home” utilities on the web.  So use RFID to know what I have and use an XML data feed from some food site to do a mash-up on my refrigerator screen about the recipes that fit what’s in the cupboard and fridge.

A friend of mine sent me an email entitled Four Major Transformations by Herb Meyer.  The email said the paper was presented at Davos.  I cannot confirm that this is by Herb Meyer or that the contents of the argument are accurate, but I found a copy of the email I received here.  I assume that the details of Herb’s paper are accurate, but even if they aren’t I think the point of this post is still applicable.

Here is a snippet of the paper that I would like to discuss:

The Emergence of China

In the last 20 years, China has moved 250 million people from the farms and villages into the cities. Their plan is to move another 300 million in the next 20 years.  When you put that many people into the cities, you have to find work for them.  That’s why China is addicted to manufacturing; they have to put all the relocated people to work.  When we decide to manufacture something in the U.S. it’s based on market needs and the opportunity to make a profit. In China, they make the decision because they want the jobs, which is a very different calculation.
  
While China is addicted to manufacturing, Americans are addicted to low prices.  As a result, a unique kind of economic codependency has developed between the two countries.  If we ever stop buying from China, they will explode politically.  If China stops selling to us, our economy will take a huge hit because prices will jump.  We are subsidizing their economic development; they are subsidizing our economic growth.   

The capacity model and enterprise sales.

Enterprise sales professionals frequently use a revenue prediction and management tool called the capacity model.  The capacity model assumes infinite demand.  So rather than building a bottoms-up demand model, the capacity model simply focuses on the supply side of the equation — the number of quota carrying sales representatives.  Assume that each sales rep can sell x million of product per year.  Assume that you need y number of people supporting each rep.  Assume that each rep attains 25% percent of her quota during the first quarter, 50% percent during the second full quarter, and 100% by her third quarter on the job.  It’s a bit more complicated than this, but not much.  And it works really well… until demand becomes an issue (e.g., competition, market maturity, recession).  Then management often assumes that sales leadership needs to be replaced due to “bad execution” when the reality is that it may be time to change the overall corporate strategy (enter new markets, change pricing model, exit market, use M&A to consolidate market).  It’s really hard to predict demand, so most large companies rely on some type of supply driven model far too long.

China and the capacity model.

China is pursuing the capacity model writ large.  And as long as supply outstrips demand, it will work.  But at some point the combination of a doubling of Chinese labor working on manufacturing (increasing supply), the large number of people moving into manufacturing from India and Eastern Europe (increasing supply), increases in productivity (increasing supply), and the certainty of a global economic downturn at some point in the future (lowering demand) will lead to a collapse in the prices of manufactured things.

There are other constraints that may temporarily slow things down — the availability of energy and other non-labor inputs required for manufacturing could act as a short or medium term throttle on supply.  But these issues will be resolved.  And then what happens?  Economic theory argues that, in an efficient market, prices will drop to near marginal cost.  We know that the marginal cost of bits is near zero and that pricing for many of those things is FREE.  But what happens when the marginal cost of atoms approaches zero?  

Razor and blades.

Might the cost of many manufactured things get close to zero?  Is FREE the future of bits and atoms [I haven't read Chris Anderson's new book -- perhaps he has the answer]?  Don’t roll your eyes, it’s possible.  We may see the razor and blades business model much more frequently in the future.

It is already happening:

+ Free atoms can increase barriers to entry, allowing providers to charge for services.  For example, the recent move by Apple to allow carriers to subsidize their phone was a brilliant move by the carriers to keep those consumers addicted to the crack of “cheap” phones.  And they already do offer some phones for “free” with an agreement to use their services for 1 or 2 years.

+ Cable vendors use this model with set-top boxes and cable service.  TiVO makes a better product, but why buy when you get a DVR for free?

But these cases are examples where average revenue per user (ARPU) is extremely high.  With the marginal cost of manufactured products dropping dramatically, service providers will be able to offer free products based on much lower ARPU.

What’s next?

+ Free computers, pay for data plans?

+ Free cars, pay for the gas

What can you give away for free and then make money on the services?

VERITAS Software and The Gone List.

Early in the Summer of 2005 a colleague of mine at VERITAS Software started calling me with daily information about who left the company and which groups were adding new hires most quickly.  He was an individual contributor engineer without access to senior executives, yet he had more of this type of information than anyone at the company including the head of Human Resources!  How was that possible?  So I took him out for coffee to find out the source of his information.  

What I found out was amazing (remember, it was 2005).  An ingenious engineer realized that all of this data was available to every employee in the company — we all had access to Vdir (the VERITAS directory) through Vnet (the VERITAS intranet) and LDAP.  Unfortunately, it was impossible to turn that raw data into information just by looking at a bunch of pages.  Humans are notoriously bad at preserving the fidelity of massive quantities of atomic bits of data and then scanning for changes over slices of time.  But it turns out that computers are extremely good at this sort of task.  So this engineer wrote a program that produced an UNOFFICIAL daily summary of all changes to Vdir and sent the results (called the Gone List) to any VERITAS employee who wanted to subscribe via email.  

The email had: total current employee count including contractors (a number that many executives have trouble getting just prior to conference calls), employees removed from the company database (all departures, listed by office location and function), employees added to the company database (new hires, listed by office location and function), and existing employees with new departments, locations, and names (who got married).  Just after I found out about the list, it was shut down.  Rumor has it that HR didn’t like the the Gone List and shut it down. 

Facebook introduces the News Feed and the Mini-Feed.

On September 5, 2006 Facebook introduced the News Feed and the Mini-Feed.  Here is my edited summary of the post (note the bit about Mark adding Britney Spears — you’ve got to love a company where a PM makes fun of the co-founder & CEO in a press release about a new feature):  

“News Feed highlights what’s happening in your social circles on Facebook. It updates a personalized list of news stories throughout the day, so you’ll know when Mark adds Britney Spears to his Favorites or when your crush is single again. Now, whenever you log in, you’ll get the latest headlines generated by the activity of your friends and social groups…News Feed and Mini-Feed are a different way of looking at the news about your friends, but they do not give out any information that wasn’t already visible…These features are not only different from anything we’ve had on Facebook before, but they’re quite unlike anything you can find on the web. We hope these changes help you stay more up to date on your friends’ lives.”

News Feed is the Gone List for the Facebook community.  And I agree with Ruchi Sanghvi (the product manager on these features), that this is unlike anything else on the web (circa September 2006).  There is a good chance that when history is written, Facebook’s greatest contribution to the web will have been News Feed.

Vitality everywhere.

According to Webster’s, the definition of vitality is, “the peculiarity distinguishing the living from the nonliving.”

What the Gone List and News Feed did was bring distributed bits of “nonliving” data to life by aggregation around an ontology (directories, in both cases).  Then they simply developed a model about what database changes would be interesting (who’s gone with the Gone List, who’s single now with News Feed) and presented the information to users in once place (email with the Gone List, Facebook homepage with News Feed).  

To be clear, the underlying concept goes back a long, long time.  Anything that pushes changes out instantaneously plays on this idea.  Stock price changes, RSS feeds, and “news” alerts all play on the same concept.  But understanding that any database change [linked to some ontology] could result in interesting information was revolutionary.  While a number of firms have figured out just how powerful this metaphor can be (LinkedIn, Yahoo’s MyBlogLog, FriendFeed, Plaxo), the vast majority of the world has yet to embrace the revolution.  

Therein lies the opportunity.

Idea:  Autofrais —  News Feed for the rest of us.  

1.  Develop platform that accepts inputs in an open format (XML?) with a set of user-defined attributes (e.g., author from directory + location, group, keyword).  Encourage “developers” to attach as much ontological information as possible (e.g., a corporate directory).

2.  Allow developer to describe rules / model through which the data will be filtered.  For example, an in-house corporate IT developer could build a model to automatically share all employee departures and additions with everyone in the group from which they departed / joined on the homepage of the corporate intranet everyday (now that’s information that would get me coming back).  Or a user-generated web site could munge their [tagged] content with the Facebook social graph through Autofrais + Facebook Connect as their new homepage.

3.  Build A/B testing platform to allow third-party developers to optimize content based on end-user click-throughs or some other objective function.  This isn’t required with small data volumes, but with growing data volumes some prioritization system will be required to separate signal from noise.

4.  Develop simple templates to allow developers to deploy a solution with minimal input (e.g., News Feed wizard which walks users through simple creation process where I can pick from UI templates, colors, etc).

In 1965 H. A. Simon said that ”machines will be capable, within twenty years, of doing any work a man can do.”  Unfortunately, over forty years later his prediction is likely quite far away (if we ever get there).  Artificial intelligence has a long, long history.  Some people have tried to reverse-engineer the human brain while others have used brute force compute power and predictive algorithms in an attempt to ride Moore’s law to enlightenment.  It hasn’t been the complete failure many would have you believe, but it’s far from H. A. Simon’s prediction.

More recently, the W3C has pursued the Semantic Web — I love the way Tim Berners-Lee expressed the vision:

“I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web – the content, links, and transactions between people and computers. A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The ‘intelligent agents’ people have touted for ages will finally materialize.”

– Tim Berners-Lee, 1999

Don’t make computers smart, make humans more precise.

The W3C’s efforts with the Semantic Web are extremely ambitious.  My summary of the Semantic Web is that humans need to help machines get smart.  There is a huge amount of data on the web which machines cannot understand, but humans can.  If we can classify that data in a language that machines can understand, machines will serve us much, much better.  

Resource Description Format sets out to do just that.  Here is an example from the Wikipedia page on RDF, as I cannot think of an easier way to make the point.

Example: The postal abbreviation for New York

Certain concepts in RDF are taken from logic and linguistics, where subject-predicate and subject-predicate-object structures have meanings similar to, yet distinct from, the uses of those terms in RDF. This example demonstrates:

In the English language statement ‘New York has the postal abbreviation NY’ , ‘New York’ would be the subject, ‘has the postal abbreviation’ the predicate and‘NY’ the object.

Encoded as an RDF triple, the subject and predicate would have to be resources named by URIs. The object could be a resource or literal element. For example, in the Notation 3 form of RDF, the statement might look like:

<urn:states:New%20York> <http://purl.org/dc/terms/alternative> "NY"

In this example, “urn:states:New%20York” is the URI for a resource that denotes the U.S. state New York, “http://purl.org/dc/terms/alternative” is the URI for a predicate (whose human-readable definition can be found at [1]), and “NY” is a literal string. Note that the URIs chosen here are not standard, and don’t need to be, as long as their meaning is known to whatever is reading them.

N-Triples is just one of several standard serialization formats for RDF. The triple above can also be equivalently represented in the standard RDF/XML format as:

<rdf:RDF
  xmlns:rdf=“http://www.w3.org/1999/02/22-rdf-syntax-ns#”
  xmlns:terms=“http://purl.org/dc/terms/”>
        <rdf:Description rdf:about=“urn:x-states:New%20York”>
                        <terms:alternative>NY</terms:alternative>
        </rdf:Description>
</rdf:RDF>

If humans encode objects with machine-readable tags (RDF example above) and humans create ontologies (open directory project, social graph), machines can then use inference to better serve humans (see Guha’s example below).  There are two major categories of projects here, so today I will focus on the first:  encoding entities on the web with machine-readable tags.  I will [try to] tackle the ontology bit in a post later this week.

The challenge with the W3C’s current approach is that it will take an inordinate amount of time to get the key players to adopt the standards required to make the Semantic Web a reality.  This includes not only agreeing to standards, but training thousands, if not millions, of people to leverage these standards.

How does the consumer benefit from all of this?

R.V. Guha developed meta-content format (MCF) at Apple Computer and later Netscape in the mid-1990’s.  MCF was a predecessor to RDF.  Over a decade ago, Guha used this example to demonstrate the power of machine understandable tags.

“Simple lexical word occurrence based searching is by far the most prevalent way of searching for information today. One of the shortcomings of this approach is its inability to distinguish between different word senses. 
 
Example : The user is using one of the WWW search engines (such as Lycos) to search for pages about lions - the animal - not Lion King or Red Lion Hotels or Lions Club. Since all that the search engine is looking for are occurrences of the four characters “lion”, there is no way in which it can distinguish between these different uses of the word “lion”. 
 
How is one to recognize the word sense of a particular occurrence of “lion” without solving the natural language understanding problem? One way of identifying occurrences of “lion” that have a significantly greater likelihood of referring to the animal is to use a subject categorization of the WWW pages. Yahoo, for example, contains a category corresponding to “Animals & Pets”. Pages that occur under this category that use word “lion” are more likely to be using it to refer to the animal. Unfortunately, Yahoo does not index the words that occur in the content of pages. But our program can issue a query to a search engine such as Lycos, translate (”lift”) the answers into a common meta-content language, filter out those pages that don’t occur under the “Animals & Pets” part of the Yahoo hierarchy and give us a small set of pages all of which are most likely about lion the animal.”

Wikipedia, mankind’s greatest achievement?

A few years ago John Beatty argued [to me over coffee] that Wikipedia is one of mankind’s greatest achievements.  Here is some data that supports John’s argument: “currently, the English Wikipedia alone has over 2,404,773 articles of any length, and the combined Wikipedias for all other languages greatly exceeded the English Wikipedia in size, giving a combined total of more than 1.74 billion words in 9.25 million articles in approximately 250 languages. The English Wikipedia alone has over 1 billion words, over 25 times as many as the next largest English-language encyclopedia, Encyclopædia Britannica…”

And here is the competition. I’m with John.

Idea — Wikimachina:  Wikipedia for machines.

Wikipedia has assembled the world’s greatest collection of encyclopedic knowledge with [according to an article on Wikipedia, of course] 4,000 editors who represent 64,567,607 total edits, with an average of 16,141 edits per editor. This accounts for 32.8% of the 196,705,582 total edits made to the English Wikipedia.  The idea behind Wikimachina is to have a relatively small group of humans (like Wikipedia) help tag all content on the web with metadata so that developers can build much smarter applications for humans.

1.  Seed the system with as much tagged data as possible.

Leverage APIs, scrape, and do deals to get data from any service that already has good tagging data — StumbleUpon, Delicious, Digg, Reddit, Flickr and so on… 

2.  Develop peer-to-peer browser toolbar for the community.

Develop a toolbar that allows the Wikimachina community to tag any entity on the web with machine readable tags.  All edits would be instantly sent to the distributed P2P index.  So when a Wikimachina community member visits a web page that has been tagged by a fellow community member, the latest change would be visible to him.

3.  Build community, develop community tools.

If we could get 4,000 editors to produce 16,141 tags per editor, the web would be a very different place.  Editors would need to learn some basic rules about our tagging system, but it wouldn’t be any harder than HTML.  

A system which separates the creation of these incremental metadata tags from the creation of content has some very positive characteristics.  Distributed systems are prone to spam, which is exactly what happened with HTML tags in search (because incentives for content owners to mislead search engines with metadata tags are so high).  So giving a loyal group of semantic “taggers” the tools to use their judgement would likely avoid many of the perils of collocating the creation of content with the creation of semantic tags.

4.  Offer access to the index through web services.

Just as with Wikipedia, anyone would be able to view the index — which would also be available through a set of web services.  Any third party developer could leverage these tags to develop or improve their service (search, for example).  In addition to machine understandable tags, our service would allow for bulk download of data — required for machines, and prevented in today’s tagging systems developed with humans in mind.

You have got to hand it to Jeff Bezos.  First he reinvented the book publishing industry.  Then he reinvented the overall ecommerce market — Amazon’s 2006 online revenue was more than Staples and Office Depot combined (the number 2 and 3 ecommerce players according to Morgan Stanley; page 29). Finally, he realized that he can more effectively utilize his fixed assets by layering web services on top and he opened his technology platform to the world.  Jeff Bezos is one entrepreneur who isn’t lucky.  He’s good.  Really good.

Amazon Web Services.

In the average data center, servers and storage are massively underutilized — estimates are all over the place and utilization of 15% is not uncommon while something between 25%-50% is probably close to average.  And these are numbers from large data centers.  Small and medium size enterprises are likely not even close to as efficient as large enterprises.  

The idea of utility computing was that customers would rent compute cycles and storage space on a pay as you go basis.  Only pay for what you need, when you need it.  Cloud computing is finally happening, and Amazon is leading the way rather than the traditional enterprise hardware and software vendors who predicted this evolution many years ago.  Amazon Elastic Compute Cloud (EC2), SimpleDB, Simple Storage Service (S3), and Simple Queue Service (SQS) are delivering on the utility computing dream.  

The amazing thing about this is the starting point — while the technology vendors started with a theoretical premise, Amazon started from a very simple place (at least, this is my impression of how it happened).  We have all of this unused capacity which is wasteful.  Our ability to deliver the best experience to our customers includes being able to offer excellent service AND low prices.  So how can we lower per unit costs?  Increase utilization.  And there you have it.  

Idea – Amazon Web Services for everything else.

This got me thinking, “how many other business can take their inefficiently utilized existing assets and dramatically increase leverage by layering on web services?”  Build software to automate the control, sharing, and billing of fixed assets.  Make the entire process self-service.  And, like Amazon Web Services, focus on web developers.  Imagine the Cambrian explosion once the world’s fixed assets can be rented on a pay-as-you go basis by any entrepreneur with a good idea.  As always, there are many hurdles.  But suspend disbelief for a minute and think about the possibilities.

1.  Manufacturing Web Services.  

A metals machine shop opens its infrastructure to third party developers.  A smart web developer builds a simple AJAX CAD application that allows users to quickly design a part, get an electronic estimate of costs, and submit an order with a credit card — all self-service.

A textile manufacturer with extra capacity opens up with web services and an enterprising developer builds an Flash-based fashion design application that allows fashion designers to design new items (e.g., purses, dresses, suits, comforters), get a quote, produce a prototype, and submit a credit card.

A cabinet manufacturer layers on web services and an entrepreneur builds a drag-and-drop application for consumers to design their own furniture — and this is as easy as configuring your new car on an online car site (e.g., pick the color, wood type, style).

The list goes on and on and on…

2.  Distribution Web Services.

Now that you have your virtual manufacturing facility in place, you need a distribution plan.  The metals shop is in the US Midwest, the textile manufacturer is in China, and the cabinet maker is up north in Canda.   What to do?

A logistics company which already has the rail, trucking, shipping, and air freight relationships in place could expose excess capacity through web services.  Developers could then pull these services into their overall offering.  So perhaps the manufactured party or small quantity of dresses can be shipped through Federal Express, but the cabinets need to be shipped by train.

3.  Communications Web Services.

Apple Computer and AT&T proved that this is feasible.  iTunes activation of my wife’s iPhone was the single best experience I have had turning on any communications service.  And there wasn’t even anyone from Apple or AT&T involved.  

AT&T and every mobile vendor should open up their system to allow third parties to buy and allocate accounts — similar to the notion of drop-shipping in manufacturing.  This would allow for add-on services.  

For example, imagine a service that allows a corporation to issues thousands of mobile phone numbers at the click of a button.  Like what corporations do today with a fixed-line PBX.  A third party developer could offer a [virtual] mobile PBX systems with group voice mail and integration of voice mail and email systems like Apple’s visual voice mail (for ease of use and for corporate compliance).  Today, you cannot configure a pool of mobile phones like a fixed-line pool with a PBX.  Why not? 

4.  Retail Web Services.

Many retail stores carry goods from various manufacturers (e.g., Wal-Mart).  Even those manufacturers who push mostly their own goods, often have complimentary goods from third parties (e.g., Apple has third party iPod add-ons and Macintosh software).

So rather than have merchandisers (central planners) use their overly mathematical models (e.g., economic order quantity), why not auction off shelf space?  Create web services which select shelf-space for auction and allow third party services to tap into the inventory through an auction.  

Retailers measure their productivity by sales per square foot — I will bet you cash money that this system will dramatically increase sales productivity.

5.  Finance Web Services.

When you buy a car, the dealer always pushes financing on you.  This third party coverage isn’t from the manufacturer, but it’s integrated into the sales process.  Why not enable the online equivalent for any ecommerce store?  Just before you consummate the purchase of a new Sony PlayStation 3 at Best Buy, the salesman asks you if you would like to buy a service plan.  Why not open up the system to third party developers?  Allow any online vendor to offer a service plan, insurance, or financing?  

Call it the virtualization of fixed-assets.  And while the virtualization of compute cycles and storage is great fun, the opportunity is so MUCH bigger.  I’m sure that we would all be amazed at the ingenuity of entrepreneurs who had the flexibility to experiment and strong economic incentives to motivate them.  It will take a few more Jeff Bezos’ to pull this off.  And, as was the case with Amazon, there is great opportunity for new entrants.  There is also a huge opportunity for incumbents who will wake up once others show them the way.

Three things are certain in life:  death, taxes, and advertising.

You can add advertising to Benjamin Franklin’s quip that “in this world nothing is certain but death and taxes.”  The conventional wisdom about consumer web firms is that there are basically two ways to build a big business:  (1) attract a massive audience and make a small amount of revenue on a per unit basis through advertising, or (2) attract a smaller audience and make a large amount of revenue through ecommerce.  Put more succinctly, you either need to have a massive audience to whom you can advertise or you need to be an ecommerce company, or you are irrelevant.

Online advertising:  a concentrated market.

A recent Morgan Stanley report on Internet Trends has some very interesting data.  The first striking fact is that Google and Yahoo! revenue accounted for a whopping 61% of U.S. online advertising revenue in the Fourth quarter of 2007 (page 38).  While that number is clearly heavily weighted by search advertising (which is a Goolge / Yahoo! oligopoly), it’s still a striking number.  

Striking, but not surprising.  A concentrated number of suppliers is one way to slice and dice audience for targeting, to limit the number of interfaces with which advertisers and agencies must work (both the online UI and sales representative coverage), and to provide an end-to-end analytics infrastructure that helps me to manage return on investment.  

The report also notes that Facebook + YouTube has more page views than Google or Yahoo! (page 6).   The slide concludes that “Massive Transition in Available Ad Units & Supply > Demand.”  Of course, in information products-based products, supply and demand are easily brought back into balance through pricing.  The likely continued growth in the “supply” of page views (or whatever metric for attention comes next) combined with anemic US economic growth means that ad pricing will likely take a hit. 

The legions of entrepreneurs out there should appreciate just how MASSIVE you need to come in order to build a big advertising-based business.  And to make matter worse, building a content-based business is getting cheaper and cheaper.  The probability of wining this game is extremely low and will only get lower in the coming years, but the rewards are extremely handsome.  Of course, another advertising-based strategy is to keep your costs extremely low and be happy with a niche business or some revenue on the side (if this is a hobby, like blogging or a project with a few buddies outside of work).

Ecommerce: dominated by offline players.

While it is true that ecommerce is a more fragmented market and that you can do extremely well with a fraction of the audience of an advertising-based business (the #1 online retailer, Amazon, isn’t in the Top 10 most visited sites), there is a catch.  Of the Top 15 online retailers, only 2 (Amazon.com and Newegg.com) are pure-play online retailers (Page 29 of the Morgan Stanley report).

While you don’t need the same size audience, you will compete with the entire world of retail — not just pure-play online ecommerce sites.  And many retailers have clearly figured out how to leverage the web.  Having said that, there will likely be more interesting startups in ecommerce than in content-advertising — interesting in that there will be more winners (but the advertising-based winners will likely still be bigger).

Beyond advertising and ecommerce…

There are two other categories of revenue generating opportunities which it seems have been written-off — premium services and virtual transactions.  While both have had limited success in the past, there is reason to be optimistic.  Virtual transactions in social networking (Cyworld) and gaming (WoW) have had success is some applications.  Consumers pay prices that exceed marginal cost frequently (according to conventional economic thinking, defying economic gravity).  Why do people pay a big premium on a ring from Tiffany or a watch from Rolex?  While you may or may not do so yourself, you know the answer to that question.  And if consumers perceive value, there is value —  it makes no difference that it is made from bits rather than atoms (what is the intrinsic value of a $100 US bill?).

Premium services have also enjoyed some, though limited, success.  The Wall Street Journal online is still charging a fee despite over a decade of naysayers arguing that they would need to go free.  Yahoo’s Mail and Flickr properties make revenue on premium services, too.

The internet tax — advertising and registration.

For most consumers, advertising is a usage tax.  It offers little value, adds latency and cognitive overhead to the user experience.  Since firms make so little on advertising on a per-customer basis, customer support is limited.  But because signing up for premium services (and virtual currency) on a site by site basis costs money and takes time, most consumers choose not to do so.  

Another usage tax for just about every web site is registration.  Why do sites require registration?   There are two reasons, one of which is legitimate and the other is not:  (1) so that they can target advertising and content to you, and (2) so they can authenticate that you are a real person, not a bot.

The promise of registration is that I will get better content by giving the site information about myself and allowing it to track me — with few exceptions (Amazon.com), sites fail on the first reason.  On authentication, sites do need to make sure that I’m a real person so as to limit the chances of several million daily Gmail registrations by one person and a bot for the purposes of flooding the web with SPAM.  The problem with this is that SPAMMERS are very good at breaking most authentication schmes  And the harder you make it for spammers, the harder you make it for consumers.  This has lead to a number of actions which hurt user experience (e.g., web-based mail limits on messages sent per day, attachment size).

Idea:  Unifi — Cross-firm Premium Services Packaging and Virtual Currency

1.  Offer universal registration.

By cutting deals with content sites throughout the internet, you can solve both registration objectives through centralization.  Since you will be charging every customer for premium services (see point 2), you will have their credit card on file.  This, along with other one-time registration information, will allow for world class authentication.  And since consumers are paying for a “premium services” package, you have no need to target ads and special offers to them.  

2.  Would you like the basic package, the Gold Package, or the Platinum Package*?

Cable packages allow you to select the package of channels you would like in exchange for a monthly fee.  We will offer basic services (no advertising), a Gold package (add IMAP mail, no limit on attachment size, high-quality photos in your photo sharing, etc), and the Platinum Package (how about customer service that actually works?).

 So why will this work across sites when it has more or less failed on a one-off basis?  Imagine ordering cable by negotiating a deal with every channel?  Bundling has huge advantages for all involved.  The challenge would be figuring out how to share revenue with partner sites.  

You could do it on a page view basis or some other equally distributed attention metric.  Or, you could get really fancy and allow consumers to pick the services they value most to create their own bundles (e.g., “I will take one Gmail no attachment limits, one Flickr pro account, and one five free downloads from iTunes each month”).  Service providers would accept our virtual currency as payment and convert our currency into real revenue each month.

3.  Virtual currency, across the web.

Like premium services, virtual currency is something that benefits from bundling.  If I can earn currency in a game on Facebook and spend it on a game on MySpace (or premium services at Yahoo!), virtual currency might actually work.  

Work a deal with American Express to have an exchange rate between our currency and airline miles to turn virtual currency into real currency.

*Note:  You may argue that this idea is inconsistent with my argument in a post a few weeks ago on cable.  As  the great Ralph Waldo Emerson said, “A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines.”

The preface to Karl Popper’s The Logic of Scientific Discovery has a very simple framework for thinking about knowledge.  “The problem of epistemology may be approached from two sides: (1) as the problem of ordinary or common-sense knowledge, or (2) as the problem of scientific knowledge.”  My summary of Popper’s distinction is that common-sense knowledge is knowledge that already exists (though it may not be universally or even widely known).  Scientific knowledge is about the growth of knowledge — that is, the creation of new knowledge.  Here is a link to the actual text.

It is important to understand the difference, as it is my belief that understanding the difference between “common-sense knowledge” and “scientific knowledge” holds the key to knowing whether or not to offshore.

A few years ago, everyone in Silicon Valley decided that every startup had to have an offshore strategy.  Lower labor costs and an infinite labor pool in India, China, Russia, and elsewhere in Eastern Europe had joined forces with open source software to turn engineering into a commodity, the conventional wisdom held (and still holds in many quarters). “Take that overpaid Silicon Valley engineering talent,” said the bean counters.  Whether you needed customer support or an agile team of algorithm black belts, offshoring was the answer.  

Now people aren’t so sure.  

So does offshoring make sense?

Back to our friend, Karl Popper.  If you have a repetitive task which requires virtually no INNOVATION and which can operate relatively independently from other core groups, having that task separated geographically may work.  In Popper’s terminology, common-sense knowledge projects are a natural fit for distributed work (offshoring, outsourcing, open source software).  For example, if you decide that you want to have all support manuals scanned and uploaded into a system for universal access it may make sense to offshore that effort.  Or if you have years to replicate an existing technology, having geographically distributed teams can work (e.g., many open source software projects are highly distributed).

However, if the focus of the function which you are thinking about offshoring requires the creation of new knowledge — scientific knowledge — you would be well served collocating all professionals and teams required to accomplish your objective.  For example, having a management team, product management team, and your engineering architects in the US and offshoring coding work (to, let’s say India) is a very bad idea.

Innovation requires limited latency — anything that introduces latency decreases the probability of success.  Offshoring introduces massive latency.  And so do large teams, which often accompany offshoring because it’s “cheap.”  If you are doing anything that requires innovation, working with large offshore groups is far from cheap.  

Please note that I am not arguing that innovation can’t exist in traditional offshore destinations like India, China, and Eastern Europe.  Rather, I am arguing that a U.S.-based business that wants innovation and has an engineering team in India, should move product, design, and management to India or move engineering to the US.  The key is for the innovation team to be together — where that is matters less than having people together. 

Update:  June 5, 2008 2:48 GMT

I removed the two references (with strike-through text above) to open source software and distributed work.  It’s a red herring, out of place, and consequently takes away from the point of the post.   

I recently attended a talk by Louise Richardson summarizing the key themes of her recent book, What Terrorists Want: Understanding the Enemy, Containing the Threat.  Richardson argued that we would be far better off identifying and exploiting the key vulnerabilities of our enemies rather than using raw military force in our efforts to destroy terrorism.  She wasn’t saying that using the military was inherently wrong, but rather that HOW we are using it is ineffective.

She used the story of the Peruvian government’s battle with the Shining Path as an example of how we can more intelligently fight our battles.  After tens of thousands of deaths in battles between the Peru military and Shining Path rebels, the government of Peru set up a 70 person intelligent unit focused on bringing down the Shining Path.  The intelligence unit concluded that the weakness of Shining Path was its highly centralized organizational structure under leader Abimael Guzmán.  The unit set off to decapitate the leadership with the assumption that after it did, the movement would collapse.  After much research on Guzman, it was discovered that he had psori