How to Design Smart Business Experiments - http://hbr.org/2009/02/how-to-design-smart-business-experiments/ar/1
You or someone on your team is suggesting a change that just might work. But why act on a hunch when you can hold out for evidence? According to the author, the best way to support decision making on potential innovations is to...
- Design an experiment.
Start with a hypothesis about how the change will help the business. If it’s a good one, you’ll learn as much by disproving it as you would by proving it. Put it to the test by measuring what happens in a test group versus a control group. From the outset, be clear on what you need to measure to produce a decisive result—and whether that’s a metric you even have the capability to track.
- Act on the facts.
Nothing but a success in a testing environment should be rolled out more broadly. But neither should failures simply be scrapped. Refine the hypothesis on the basis of the results, and consider testing a variation. Most important, capture what’s been learned, and make it available to others in the organization through a “learning library,” so resources aren’t wasted proving the same thing again.
Example: Marketers at the Subway restaurant chain wanted to drum up business by putting foot-long subs on sale for only $5, but franchise owners worried that the promotion would lure existing customers away from higher-priced menu items. An experiment pitting test sites against control sites proved that the promotion would pay off—which it subsequently did.
- Make testing the norm.
Create the training and infrastructure that will enable nonexperts in statistics to oversee rigorous experiments. Off-the-shelf software can walk them through the steps and help them analyze results. A core group of experts can lend resources and expertise and maintain the learning library. Leadership must cultivate a test-and-learn culture, in part by penalizing those who act without sufficient evidence.
As your managers become more comfortable with testing, they’ll discover that it paves the way for, rather than throwing up barriers to, promising new ideas.
New framework for Stage 5: autonomous analytics Autonomous Actor + Data Pipeline + In-memory + Reactive + Functor + Deep Learning
