Laserlike

The End of Theory?

Chris Anderson’s Wired cover story The End of Theory has many in the blogosphere up in arms.  But rather than wade into this epistemological brouhaha, I will instead make the argument that Chris Anderson should have made — that artificial intelligence (AI) is here today.  If you haven’t read The End of Theory, you should read it and check out Kevin Kelly’s The Google Way of Science while you are at it.  

Artificial intelligence has negative connotations because previous attempts pursued an overly ambitious top-down approach — reverse-engineer the human brain and then build a model in silicon.  Top-down approaches to solving problems of the complex systems variety usually don’t work.  There are a number of factors that have enabled the bottoms-up approach that we have today:  massive volumes of publicly available data thanks to the web, cheap compute cycles thanks to commodity hardware and open source software, and improved programming models for the processing large data sets like MapReduce and advances in machine learning algorithms. 

In addition to these enablers, there is a point that is often overlooked which I believe is of critical import — that the web is the greatest empirical testing platform in the history of the world.  Most authors have focused on the value of HISTORIC data sets for the identification of correlated objects.  If the story ended there, we would have Jorge Borges’ Library of Babel — the story of a library with every pair of letters and, consequently, the explanation for why the library exists and how to use it.  Correlation without an empirical feedback loop is worth as much as a Zimbabwe dollar.

Fortunately, it doesn’t end there — those correlated data sets are put in front of hundreds of millions of users.  Let’s examine a snippet from Kelly’s post:

For instance, take Google’s spell checker. When you misspell a word when googling, Google suggests the proper spelling. How does it know this? How does it predict the correctly spelled word? It is not because it has a theory of good spelling, or has mastered spelling rules. In fact Google knows nothing about spelling rules at all.

Instead Google operates a very large dataset of observations which show that for any given spelling of a word, x number of people say “yes” when asked if they meant to spell word “y.” Google’s spelling engine consists entirely of these datapoints, rather than any notion of what correct English spelling is. That is why the same system can correct spelling in any language.

Kelly touches on what I believe is the key point — that “x number of people say ‘yes’ when asked if they meant to spell word ‘y.’”  Every time a user clicks the suggested spelling the system learns.  Every time the user DOESN’T click the suggested spelling the system learns.  Machine learning is a bit of a misnomer – humans are programming the machines and many more humans are providing continuous feedback to probabilistically accept or reject atomic-level hypotheses (in this case, the spelling of a word).  

This isn’t a battle between machines with data and the scientific method, but rather a bottoms-up approach to AI that mimics the way most humans learn most of what we know.  In The Stuff of Thought, Steven Pinker examines language to gain insight into the workings of the human brain.  

Put yourself in the booties of a child who is in the midst of figuring out how to speak the language as it is spoken by parents, friends, and siblings… As you continue to hoover up verbs over the months and years…they appear in two synonymous constructions but differ in whether it is the content or the container that shows up as the direct object…

Smear grease on the axle.

Smear the axle with grease.

…When the locative rule is applied willy-nilly, it cranks out many errors…

Pour water into the glass.

Pour the glass with water.

What’s the difference… think about the physics.  In the first list, the agent applies force to the substance and the surface simultaneously, by pushing one against the other.  In the second, the agent allows gravity to do the work.  It’s the difference between causing and letting, between acting on something directly and acting on it via an intermediary force, between expecting something to change as one is doing something in real time and expecting it to change shortly after one has done something….

When I read Pinker’s argument, I was fascinated.  Brilliant!  Yet I have managed to get through my first 37 years without understanding the role physics plays in the use of verbs in the English language.  As a child I learned a few words by association.  I then learned verbs and how to construct sentences.  As I experimented with various constructions, my parents and teachers rewarded my experiments with “well done” or helped me to identify my failed experiments by correcting me.  And I am fairly certain that they based their corrections on “what sounded right” rather than a deep appreciation for the role of the physical sciences in verb use.

So what’s the opportunity?  

As Pinker tells us, language acquisition is an example of the problem of induction — making generalizations about the future from historic data.  Both humans and increasingly “machines” run continuous experiments in order to adjust their internal models.  

A good place to start would be to look for systems which have evolved over long periods of time where most people learn through some type of induction plus real-world feedback.  Spelling and translation are the most frequently cited examples of Google “science,” and they both fit the bill.  

Another good candidate is medicine – most physicians rely heavily on identifying patterns from previous cases and applying them to the current case.  ”The last 100 times I saw symptoms x and y it was disease z.”  Doctors proceed with caution, as they are aware of the limitation of their heuristic approach.  They propose various treatments or medications as small experiments in the hope that they can confirm their [inferred] hunch.   

Surely we can create a better system for some set of medical problems?  What are some other areas where induction is the primary form of human learning?