As a coach, I use the word experiment a lot. I use it to reduce resistance from a client in trying a new idea. I find the barrier to entry is lowered greatly when I suggest that we try an experiment with a fixed group and a fixed time frame over a major change.
The use of the phrase experiment has been hugely valuable in my practice. When I suggest to a team or an organization that we experiment with “mob programming as a way to increase productivity and learning” or “Automation as a way to decrease lead time” are these actually experiments in name only?
After nearly three hours “researching” on the internet and going down the rabbit hole of the scientific method, field experiments, variables, laboratory experiments, quasi-experiments, and behavioral economics, left me with a resounding feeling that the types of experiments I am conducting with my clients are not scientific in the least, but do they have to be. If yes, what is the minimum level of rigor I need to apply to conduct a useful experiment?
The types of scientific experiments are largely divided by the context in which they are run: lab or real world and how the variables are controlled.
The types of experiments I run with my clients are all in the “real-world”. Where they largely fall short is (3-6) in the scientific method.
Make observations.
Formulate a hypothesis.
Design and conduct an experiment to test the hypothesis.
Evaluate the results of the experiment.
Accept or reject the hypothesis.
If necessary, make and test a new hypothesis.
How we can elevate our organizational experiments to something that resembles the scientific method?
“Teams seem to approach this aspect of continuous learning rather sloppily. Even when I try to institute an improvement experiment into the retrospective, rarely do they remember to be explicit about exactly what problem they are addressing, and (especially) what measure they use to observe an effect.” -Tim Born
Elevating our experiments
Harvard professors John Beshears and Francesca Gino suggest a simple framework for organizational experiments that I find useful as a starting point.
Identify the target measurable outcome
Articulate what exactly your proposed change will involve
Introduce the change in some places in the organization (the “treatment group”) but not in others (the “control group”)
More on this in future posts.
