Many of us know that in order for a Web application to improve, it’s often a good idea to run A/B experiments: Given enough visitors to a site, you can present different versions to users, and see which one is most effective. This technique is just one way in which you can conduct online experiments. How can we control these experiments, and thus learn more from the results? What sorts of experiments can we run? What sorts of experiments have companies successfully used in the last few years? In this talk, Ronny Kohavi describes the history of controlled experiments, and of online controlled experiments, and provides examples of how they have helped to improve a number of businesses. He also gives us hints for how to create our own experiments, and how to make those provide us with powerful and useful results.