“Science is not a body of facts. Science is a state of mind. It is a way of viewing the world, of facing reality square on but taking nothing on its face.” -Natalie Angier, The Canon
About a week ago I came home from work and found my kids perplexed with our TV. The volume wasn’t working and they didn’t know what to do. My first grader knew about the “volume” and “mute” buttons on the remote, but neither worked for her. This was clearly a family crises, so I made correcting this situation my top priority.
First I turned the amplifier mute on and then off. Nope. Then the cable receiver mute on and then off. No again. Next I rebooted the cable box. Didn’t work. But when I tried the mute button on the cable receiver again, voila! We have all used a simple sequential problem solving process like this for issues with a TV, computer, or some other home appliance. By developing a clear hypothesis, isolating potential solutions, and testing these solutions one at a time, we are implicitly using the scientific method. The feedback loop on whether our experiments worked or not is abundantly clear — the volume is either on or off.
Then we go to work. And despite a degree from XYZ university in engineering or business, we often apply less scientific rigor to product development than we do to fixing the volume on our televisions at home. Think about it. Product releases are often packed with features. Why? The product development leadership has decided that those features are what the customer wants, right? But how will they know if their hypotheses are right if they are simultaneously running a large number of experiments (features)? Sure they can develop statistical inference models to isolate certain variables, but why go there? It’s like trying to solve a home PC internet connectivity problem by trying every potential solution simultaneously. That’s a terrible algorithm, yet it dominates product development.
Less is More?
There is a fashionable design philosophy in product development circles these days — let’s call it less is more. 37signals has been a champion of this approach, but there are many others. A key tenet of the philosophy is that uncluttered products with fewer, better features are preferred to similar products with more features. I agree with the less is more product development approach, but for a different reason.
The reason I like less is more as an approach is that it allows for a more scientific approach to product development. By starting a new product off with as few features as possible (1?), you can be incredibly scientific. With 10 features in a single release, you may spend more time trying to figure out what is working and what isn’t working than it took to build the thing in the first place. As you incrementally experiment with your product, you can observe the impact of a particular feature one at a time and adjust accordingly.
Scientific product development.
So this leads us to the following approach to developing new products.
Step 1. Have a very clear idea of the problem you are trying to solve.
Step 2. Develop a hypothesis about the minimal feature set that will address the problem (ideally just one thing). You can do research or just have a gut feeling about the answer. A good “product picker” offers significant leverage in this step of the process.
Step 3. Test your hypothesis by shipping product quickly. A killer engineering team provides massive leverage in this step of the process.
Step 4. Observe the results of the experiment. Did the results of your test match what you expected? If not, kill the feature and start over at Step 2. If things worked, continue feature development by starting over at Step 1 again.
Most experiments fail. Many teams do a relatively good job on Steps 1 through 3, but forget the importance of Step 4. Instead of killing bad features, they simply add more hoping that feature 10 will somehow make crappy features 1 through 9 better.
By embracing a scientific approach to product development, not only will your business have a much higher probability of success, but it will also be a more fun and creative place to work. Nothing kills innovation like the fear of failure. And nothing leads to failure like a process that resembles astrology more than it does astronomy.
7 Comments
September 22, 2008 at 1:11 pm
Excellent points, and all very near to my heart as you know.
September 23, 2008 at 9:39 pm
“Nothing kills innovation like the fear of failure. And nothing leads to failure like a process that resembles astrology more than it does astronomy.”
I love this quote. I’m a big believer in a scientific approach to product development. It definitely makes the job more fun and allows the team to learn from our mistakes and misconceptions.
Question, how do you use the scientific approach to kill an entire product? This has always been a tough issue with me.
September 28, 2008 at 2:48 am
I like this train of thought. I generally agree that fear of failure is a real common cause of failed execution of great ideas.
I’d also suggest that engineers often try to cram in as many features as possible without taking a step back and asking themselves if the feature make sense. Instead of doing a few (and richer) features + doing them really well, we tend to get many features. End result: few of the features are really well done.
I believe that a scientific approach isn’t going to be the only thing that solves this problem. I really believe there has to be “heart and passion” into the process and a charismatic and passionate driving force is as much or perhaps even more essential to driving towards result on the product side. I look at companies that constantly re-invent themselves and their industries by saying “this should be done differently” and i think they do it by challenging common wisdom. I think apple is a good example of what happens when a company decides to break the rules of how things should work to force innovation. And i think there are many others out there doing the same (but without the resources).
I’m still surprised that developers haven’t eliminated the “save” button in the word processor and other personal productivity software packages. There is absolutely no technical reason for it, our computers are certainly fast enough to continuously save everything we type at the speed we type it. yet its still there.
Perhaps it’s time to go and challenge common beliefs and aged wisdoms…
September 28, 2008 at 9:18 pm
Steve Blank has written a book on this scientific approach to new product introduction called “Four Steps to the Epiphany.” If you are not familiar with it it’s definitely worth reading: he offers a method for hypothesis testing he calls “customer development” that matches steps in “product development.” It’s available on Amazon here
http://www.amazon.com/Four-Steps-Epiphany-Steven-Blank/dp/0976470705
and about $10 cheaper on CafePress here:
http://www.cafepress.com/kandsranch
We’ve blogged about it a couple of times:
http://www.skmurphy.com/services/consulting/customer-development/
http://www.skmurphy.com/blog/2008/01/22/steve-blank-on-customer-development-process-for-startups/
http://www.skmurphy.com/blog/2007/07/31/steve-blank-speaking-at-tie-on-wed-aug-15-07/
October 1, 2008 at 7:53 pm
[...] friends know I’m a fan of using the scientific method to improve web sites: iterate fast, disprove invalid assumptions quickly, etc. I believe [...]
October 20, 2008 at 1:08 pm
[...] your decisions to empirical reality and letting the data guide your intuition. Mike Speiser wrote about this last September and it’s a good phrase, so I’m going to steal [...]
October 22, 2008 at 6:43 pm
Mike, very cool post. Had two comments.
If I’d need to name the one, most important advantage of building web apps compared to building usual software systems, it will be this fact, that you can be ‘released’ from the notion of a release but rather incrementally add features to your product and test them one by one. The second most important feature is that you can base the “success” of a feature on real hard data - not focus groups or complaint emails but actual monitoring of how that feature affected the user experience, what’s its “CTR” etc. That’s why, on average a smart web app company will have the upper hand over an equally smart software company.
Now, as for the scientific process, sorry to burst the bubble but that’s hardly the case. It may seem, from the outside, that science is a very ordered discipline, but in fact its the other way around. Usually science is moving forward through leaps of intuition and faith that are later rationalized by data and experiments. If you’re doubtful, remember that scientific data is public domain. That means that potentially all of the scientists have the same data, and still it is just the few that have the creativity and intuition to push us forward. It was Newton that inferred the laws of mechanics and Einstein that built the theory of relativity. In fact, neither of those had much data to base their leaps on. Einstein was quoted when asked once before a big validation experiment, what if the experiment will go wrong saying something like “Then God is wrong, because this theory is too beautiful to not be true”…
Leave a Reply