How to Experiment with Experimenting
Is it just me?
I hate running experiments.
I know that I’m not supposed to say that, especially because I tell other people that they absolutely must run experiments.
After all, experiments are an essential part of the innovation process. There are some things that you absolutely cannot learn unless you go out into the world, interact with other humans, and operate in a real-world setting.
But run an experiment, especially if it’s on something I created? No, thank you. I’ll just sit right here in the safety of my office and polish my idea to within an inch of its theoretical life.
It’s all of us
To understand why running experiments can be stress-inducing, itself helpful to understand what happens in our brains when we’re conducting them.
No matter the type of experiment you’re running, the context in which you’re experimenting, or what you’re trying to learn, simply by running an experiment you are saying to the world (or your team, or your boss, or you peers), “I’m not entirely certain that I’m right. I may be wrong.” That is a very uncomfortable, even vulnerable, position to be in and our brains, always on the look-out for danger, activate our fight-or-flight response, resulting in something that feels a lot like fear.
According to Scott Steinberg, bestselling author of Make Change Work for You, the seven most common fears people report feeling in the workplace are:
- Fear of Failure
- Embarrassment
- Underperformance
- Rejection
- Change and Uncertainty
- Confrontation
- Isolation
When you run an experiment, especially if you’re in a function/company/culture where experimentation is not the norm, you are at risk of experiencing at least one, and often all, of the fears listed.
Don’t believe me?
The experiment that never happened
Imagine that you work for a big multi-national food company and you’ve been tasked with creating a new line of snacks targeting Baby Boomers and offering functional benefits like improved version, greater stability, better muscle tone. You’ve done the research and you know the ingredients that can deliver each of the desired benefits. You’ve done some small scale taste testing and you’re confident that the products taste good and have the right texture. You’ve even checked with Legal and they’re confident that you can make the claims you want to make.
The last thing you need to test is that people will pay what you need them to pay for the new products.
You could ask people what they’re willing to pay but you know you’ll get better data if you actually sell the new products to people, asking them to exchange their hard earned money for a bag of nuts. You decide that the fastest, easiest, and cheapest way to do this is to set up a little stand in the company cafeteria, sell your new products, and see what happens.
That’s when things come to a grinding halt.
“What if we don’t sell as much as we need to?” one person asks (fear of failure and underperformance).
“What if people don’t like the products?” says another (fear of embarrassment)
“What if no one buys?” says a third (fear of rejection)
“There is no way that Legal will let us do this. No one has ever done this before. Plus, we’re talking about selling something that contains allergens.” says a fourth (fear of uncertainty and confrontation)
“You’re right,” you finally agree. “It’s better than we keep this work on the down-low. If people find out what we’re doing, they may feel like we’re stepping on their toes and we’ll get shut down.” (fear of isolation).
True story (but I bet you already knew that).
I’ve seen the same thing happen at other clients and in other industries. People want to experiment. They know they need to experiment. But they don’t experiment. Instead, they come up with all sorts of reasons, some very real and true and some greatly exaggerated, why they can’t.
Experiment with experimenting
First, let’s be clear — there are are LOTS of good reasons why some experiments can not or should not happen. If you know for a fact that an experiment is going to break a law, put someone at risk, or potentially result in an unrecoverable financial loss, do not do it! Common sense, people.
But fears are not facts. They are feelings. Feelings that can be acknowledged, addressed, and overcome.
So how do you overcome the feelings that can get in the way of experiments?
Start with a Plan B.
I don’t know about you, but I always feel better when I have a fall-back plan. Many people decide on the type of experiment they need to run and then go for it. Until they hit a roadblock. Then they stop….not just the experiment but ALL experiments. This type of “all or nothing” mindset — if we can’t do this experiment we can’t experiment at all — creates incredible stress because it makes the mere act of experimenting super high stakes.
Instead, start by getting very clear and very specific on what you need to learn — not “Will people pay for this?” but “Will people pay $4.99 for an 8oz bag of nuts.” Then, brainstorm all the ways you could learn what you need to learn, without worrying about the quality of the data you’ll get. Once you have a long list of different possible experiments, assess how complex it would be to run each (high/medium/low) and the reliability/quality of the data you’ll get (high/medium/low). You now have a list of different experiments to run and a sense of the trade-offs inherent in each one. As a result, you should feel more confident in choosing an experiment to pursue because you already have your Plan B (and C and D) identified in case you discover a legal, regulatory, or financial reason that your stops you from pursuing your original experiment.
Change how you talk about experiments.
When people ask we need to run experiments, we often answer with things like, “because I don’t know if…” or “because we need to if….” There’s nothing wrong with those answers per se but admitting that you don’t know something or are unsure of something can feel really really risky. After all, uncertainty can be perceived as risky, dangerous, and even a sign of incompetence.
So try phrasing answers in a way to show that you are somewhere good and the experiment will move you somewhere better. For example, “We know that we need to sell an 8oz bag for $4.99 in order to make our numbers and based on your benchmarks we’re confident that we can do that, but we want proof and this experiment is the best way to get it.” Of course you don’t want to lie but you also shouldn’t discount all the work and learning you’ve done to get to this point. Share that knowledge, get credit for the work, and explain the benefit of continuing the journey.
Turn around your thinking.
Ok, this is going to sound a bit “woo” but, if you’re like me and tend to focus on the worst-case scenario (because then you’re prepared for it!), you need to get out of your head, see your feelings for what they are, and get focused back on the facts before you can move forward. One technique for this is a series of questions known as “The Turnaround”in which you ask (and answer) a series of 5 questions. The questions below are how they would have been phrased in the example above, the root of the question (the part that doesn’t change) is underlined:
- Is it true that we will get shut down if we run this experiment? (Maybe)
- Can I absolutely know whether it is true before I run the experiment? (No)
- What happens when I believe that it is true that we will be shut down? (I don’t want to run the experiment)
- What would I do if I didn’t believe that we’ll be shut down? (I would run the experiment)
- What are 3 examples of when this thought in this context was not true (1. When our first concept got terrible results, 2. when our initial financial estimates didn’t achieve the required profit margin, 3. when the first taste tests got mediocre results)
By the end of this, you now have 3 facts that refute your 1 feeling. And as I like to say, “don’t bring feelings to a fact party ’cause we ain’t got time for that.”
In conclusion
Everyone gets uncomfortable at some point in the innovation process. For some people it’s the uncertainty at the start, for others it’s the process of setting KPIs without knowing whether you can hit them or not, and for others (like me) it’s the moment when you have to take your beautiful precious idea out into the world and face the fact that it might fail.
By experimenting with the way we approach planning for experiments, talking about them, and even thinking about them, both my clients and I have been able to lessen the fear and anxiety we feel when we don’t know what will happen next and find the courage to move forward, make things happen, and learn what needs to be learned.