Laserlike

The experience curve describes cases where the cost of doing something decreases the more often the task is done.  It is a cousin of the learning curve and has been a cornerstone of corporate strategy since the late 1960’s when Bruce Henderson of the Boston Consulting Group (BCG) applied cumulative learning gains to corporate strategy.  An early mover in a market characterized by a significant experience curve should pass on cost savings to consumers with the objective of gaining share.  A positive feedback loop is then triggered leading to a potential monopoly.

Brief Summary of the Experience Curve (from Wikipedia)

The experience curve… states that the more often a task is performed the lower will be the cost of doing it. The task can be the production of any good or service. Each time cumulative volume doubles, value added costs (including administration, marketing, distribution, and manufacturing) fall by a constant and predictable percentage.

These effects are often expressed graphically. The curve is plotted with cumulative units produced on the horizontal axis and unit cost on the vertical axis. A curve that depicts a 15% cost reduction for every doubling of output is called an “85% experience curve”, indicating that unit costs drop to 85% of their original level.

Mathematically the experience curve is described by a power law function sometimes referred to as Henderson’s Law:

• ^[4]

where

•  is the cost of the first unit of production
•  is the cost of the nth unit of production
•  is the cumulative volume of production
•  is the elasticity of cost with regard to output

I have a hypothesis that there is also a learning curve for machines — I have yet to formally test this hypothesis and would be interested to hear from you about whether you think this makes sense and to see if anyone has heard of any tests that would support or refute this argument.

The Machine Learning Curve is likely limited to particular “industries,” just as is the case with the Experience Curve in manufacturing.  So the ML Curve for Search may have little to do with the ML Curve in some other market — for example, genetic testing.  In other markets I suspect that we will find out just how closely the two are linked — for example, behavioral targeting of text and display ads.

Arguments for the likely existence of a Machine Learning Curve.

1.  More data = better hypotheses.   The availability of massive stores of data to both regression algorithms and to human beings allows for broader investigation into potential relationships between variables that were not previously known.  Furthermore, experience in finding these relationships will improve the ability to spot *potential* patterns in the future.  For example, a group may learn that a set of actions may proceed another set of actions by 30 days (e.g. reading user reviews on cars prior to purchase).  This time-based delay is a potential candidate for future experiments — from targeting content on autos to understanding the consideration period for other items.  This could have a cumulative impact, as every relationship builds on previous relationships.

2.  Larger audience = more experiments.  When it is time to conduct randomized tests, a larger audience will allow for more granular testing.  Let’s say a test requires that we put something in front of 1,000 users.  We need at least 2,000 users as we need a test group and a control group.  So the number of experiments that can be run is (audience-1,000)/1,000.  If this sort of process is well executed, one would assume that the early mover would have the largest audience.

3.  Time-based effects.  While you could make the argument that a new player could buy a ton of equipment and spend a huge sum on marketing to neutralize an incumbent on arguments 1 & 2 above, there are significant time-based effects that make such a strategy an unlikely winner.  It takes time to measure certain effects.  For example, the consideration period for buying expensive products like cars can only be measured over a period of time that is commensurate with [at least] the consideration period.  The probability that a particular user is a real person and has a worthy reputation grows with time.  The half-life of some data (e.g., news) has a very different value with respect to time than other data (e.g., historic financial results).  By the time a new player figures these things out, the incumbent has grown her lead.

Just as with the Experience Curve, the most likely way that the Machine Learning Curve loses momentum is a discontinuity in the market.  As a Stanford professor of computer science recently told me, machine learning is not a product but rather a way to take a product to the next level.  I agree and would add that the ML Curve may increase the odds that a market leader who aggressively adopts machine learning has the potential to lock out the competition.