As a coach, I use the word experiment a lot. I use it to reduce resistance from a client in trying a new idea. I find the barrier to entry is lowered greatly when I suggest that we try an experiment with a fixed group and a fixed time frame over a major change.

The use of the phrase experiment has been hugely valuable in my practice. When I suggest to a team or an organization that we experiment with “mob programming as a way to increase productivity and learning” or “Automation as a way to decrease lead time” are these actually experiments in name only?

After nearly three hours “researching” on the internet and going down the rabbit hole of the scientific method, field experiments, variables, laboratory experiments, quasi-experiments, and behavioral economics, left me with a resounding feeling that the types of experiments I am conducting with my clients are not scientific in the least, but do they have to be. If yes, what is the minimum level of rigor I need to apply to conduct a useful experiment?

The types of scientific experiments are largely divided by the context in which they are run: lab or real world and how the variables are controlled.

The types of experiments I run with my clients are all in the “real-world”. Where they largely fall short is (3-6) in the scientific method.

Make observations.
Formulate a hypothesis.
Design and conduct an experiment to test the hypothesis.
Evaluate the results of the experiment.
Accept or reject the hypothesis.
If necessary, make and test a new hypothesis.

How we can elevate our organizational experiments to something that resembles the scientific method?

“Teams seem to approach this aspect of continuous learning rather sloppily. Even when I try to institute an improvement experiment into the retrospective, rarely do they remember to be explicit about exactly what problem they are addressing, and (especially) what measure they use to observe an effect.” -Tim Born

Elevating our experiments

Harvard professors John Beshears and Francesca Gino suggest a simple framework for organizational experiments that I find useful as a starting point.

Identify the target measurable outcome
Articulate what exactly your proposed change will involve
Introduce the change in some places in the organization (the “treatment group”) but not in others (the “control group”)

More on this in future posts.

Charles Duhigg describes the habit loop as

Triger
Routine
Reward

This loop that turns behaviors into habits. He asserts that the key to changing a habit is not changing either the trigger or the reward but the routine.

For example, the trigger could be work-related stress, the routine undeserved (etc. a cigarette, snacking), and the reward: a dopamine hit. Duhigg suggests only changing the routine, not the trigger or reward. Over time, that new routine will become a habit. For example, walking 1k steps.

There is a similar idea in programming. Refactoring modifies the code of a function or subroutine without changing the inputs or outputs, to avoid regression impact on the larger system. This is a laborious process, often overlooked due to the fact the eliminated technical debt is very difficult and is easier left ignored.

Can organizations be refactored?

Where the inputs and outputs remain the same but the undesired routines are changed.

	Trigger: Client escalation or production outage

Routine: Assert pressure on staff
  
 Reward: Temporary spike in productivity

Can this be refactored to:

	Trigger: Input client escalation

Routine: Remove waste from the systems
  
Output: Increase in productivity

What is the major difference? The difference is felt over time. The former’s quick fix provides an immediate improvement with long term degradation, while the latter is the opposite.