The Trap of Innovation Metrics (and How to Avoid It)
Of all the facets of innovation, innovation metrics may be the most requested, studied, and debated.
This is not surprising given that companies need to justify the billions of dollars they spend every year on “innovation” and the best way to do that is through a sound set of metrics with proven predictive power and relevant benchmarks.
However, despite the need, and decades of work to address it, a satisfying answer to “what innovation metrics should we use?” is as elusive as ever.
The key word there is “satisfying”
Innovation metrics exist. There are probably hundreds of them.
Innovation metrics are in use. Maybe even at your company.
But one universal set of metrics with proven predictive power and relevant benchmarks? Nope, that doesn’t exist.
And it shouldn’t exist.
What innovation is, why it is pursued, and the investment of resources and time required varies from industry to industry and company to company. So, it makes no sense to apply a standard set of metrics to a custom approach and activity.
The trap of “satisfying” metrics
“What gets measured gets managed” is a trap (and also not true and Peter Drucker never said it).
It’s a trap because, as noted by monetary theorist Charles Goodhart noted, “Any statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”
Or as us common folk (and anthropologist Marilyn Strathern) say, “When a measure becomes a target, it ceases to be a good measure.”
It is not satisfying when ideas become cobras
Although Goodhart’s Law was first articulated in 1975, it’s been in action for centuries. Consider the story of the British government’s bounty on cobras back when India was still a colony – to reduce the cobra population, the British government offered to pay people for every dead cobra they brought in. The money was good and eventually, people started raising cobras for the sole purpose of killing them to collect the county.
This is exactly what happens with most innovation metrics.
Because companies are so keen to measure and track progress, they set metrics they can measure. Those measurable metrics quickly become targets.
For example, one of my clients was eager to “fill their innovation funnel” and so they set a metric for the number of new ideas submitted each quarter. It didn’t take long before the innovation team spent most of the last day of each quarter frantically submitting “new ideas” so that they could hit the target.
The next day, they would go into the system and reject all the frantically submitted ideas because the ideas didn’t meet established qualitative criteria for strategic fit, scalability, or business potential.
At the end of the year, management was dumbfounded – despite getting thousands of ideas, the team had yet to pilot an idea, let alone launch a new business and generate revenue.
Metrics aren’t satisfying if they aren’t effective in achieving your ultimate goal
Was killing lots of cobras Britain’s ultimate goal? No.
They wanted to decrease the cobra population.
But killing cobras was a key step in decreasing the population and it was easier and faster to measure and track the number of cobras killed.
Was getting lots of ideas my client’s ultimate goal? No.
They needed to increase revenue by 50% in the next 5 years.
But getting ideas was a key step in creating a new stream of revenue and it was easier and faster to measure and track the number of ideas submitted.
How to Create Effective Innovation Metrics
State the ONE Why.
This is the hardest part of setting metrics because there are lots of reasons to pursue innovation. But we have to remember that innovation is a means to an end and effective metrics focus on the end, not the means.
I like to use the 5 Whys to get to the ultimate goal. Here’s a rough approximation (with completely made-up numbers) of the conversation I had with the client mentioned above:
- ME: Why do you want # of new ideas in the funnel each quarter?
- Client: Because we need that many to get 4 new products to pilot
- Me: Why do you need to pilot 4 new products?
- Client: Because we have a 50% pilot success rate
- Me: OK, so you need to launch 2 new products. Why two?
- Client: Because our new products generate $100M in Yr 1 revenue
- Me: Why do you need $200M in new revenue each year?
- Client: Because we have a $1B gap between what we promised to shareholders and what we’re on track to deliver in 5 years
Their ultimate goal was to close a $1B revenue gap in 5 years, not generate # new ideas per year.
Set interim milestones.
Achieving an ultimate goal takes years of hard work and you may need to pivot or readjust expectations along the way. For that reason, it’s important to set interim milestones, not as decision points but as markers for when to pull your head out of the day-to-day details and assess whether or not you are on track and, if not, what you need to change.
Using the same client as an example, we could have decided that $1B in new revenue in 5 years meant that we needed to generate $200M in new revenue every year. But, as we talked through things, we realized that it would take time to fill the innovation funnel AND build the organizational capability to develop, test, and launch new products. As a result, we should set revenue goals that increased each year, reflecting the organization’s increasing innovation effectiveness.
Explore the MANY Hows.
Innovation may be one way to achieve your ultimate goal but there are lots of others and it’s important to stay open to all of them.
For my client, we quickly realized that it wasn’t reasonable to rely solely on organic innovation to close the $1B revenue gap. Instead, we needed to explore multiple approaches from organic innovation to M&A and everything in between.
In closing…
It’s easy to fall into the trap of “satisfying” metrics that measure activity.
It’s a bit harder to evade the trap by identifying effective metrics that measure progress to an ultimate goal.
But, I can assure you, achieving that ultimate goal is way beyond satisfying.
10 Times When Innovation is NOT The Answer
We all love Innovation but there are times when it isn’t the answer. In fact, there are times when it is the absolute last thing a business should do.
Do NOT innovate if:
- Your current business is declining
- You need immediate results
- You want to play it safe and follow rather than lead
- You’re afraid of losing customers
- You’re afraid of falling behind the competition
- You’re getting pressure from shareholders
- You want to hang out with famous CEOs/be on TV and the cover of magazines
- You read an article/book/had a conversation and it doesn’t seem that hard
- You want to do something fun/different/exciting/noteworthy
- You can
Luckily, there is a corresponding list of times when Innovation is the answer.
DO innovate when:
- Your current business is solid
- You need a pipeline to deliver new revenue now AND in 3, 5, 10 years
- You are willing to take smart risks so you can lead instead of follow
- You want to better serve your customers
- You are confident in the business fundamentals and its future potential
- You believe in investing and building for the long-term
- You care about the long-term health of the business and your people
- You are committed to learning and building an organizational capability
- You are willing to work hard for a long time to do the impossible
- You will
Did you notice some themes in those two lists?
In the first list, you’re reacting. The beliefs, biases, and prejudices of your unconscious mind are controlling and driving you. An event occurred in your environment and now you’re afraid of losing something – great business results, competitive advantage, stakeholder support, your reputation. You’re afraid of losing something so you’re looking for something to save you and innovation is a bright shiny object that everyone loves, and no one will fault you for pursuing it.
In the second list, you’re responding. Your conscious and unconscious minds are working together to take in information, considering the well-being of those around you, and factoring in your beliefs and values. An event occurred in your environment that presents new information. Instinctively you perceive it as a threat but then reframe it as an opportunity to create, change, or improve the status quo. As a sign of your commitment and belief in the possibilities and potential, you use innovation as a tool to drive long-lasting impact.
The next time you consider starting an innovation project, hosting an innovation event, or staffing up an innovation team, pause to consider why you’re doing it. Is the answer on the first list or the second?
After all, just because you can, doesn’t mean you should.
Why Falling in Love with Your Ideas Kills Innovation
“Fall in the love the problem, not the solution.”
Although not as well-known as other innovation mantras like “Fail Fast” and “See around corners,” this mantra is, in my experience, truer than any of them. It is also harder to follow.
It’s hard to fall in love with a problem.
By the time most of us enter the workforce, we are Do-ers. Our bosses train us to bring solutions, not problems. Our employers reward us for showing initiative, taking action, and resolving issues. We celebrate when we find the answer, crack the case, and drive the change.
We have stopped being Questioners, Wonderers, and Explorers. We stopped asking “Why?” and “What if?” and “Why not.” We stopped wondering why or where or when or how things might be or once were or could be. We stopped exploring the unknown, the uncertain, and the unexplained.
Falling in love with a problem requires us to ask why it exists and why it hasn’t been solved yet. It requires us to wonder about why things that look like solutions aren’t. It requires us to explore and look for root causes and to welcome surprises.
Falling in love with a solution is what we are trained to do.
But falling in love with a solution, not the problem it solves, locks us into a course of action that is often all or nothing.
Falling in love with the problem is what we need to do.
Falling in love with a problem gives us a place of certainty to return to. The problem is evidence of market opportunity, we simply need to find the solution that unlocks it.
Consider these 3 examples:
An R&D team develops a breakthrough new technology but producing it at scale would make it cost 10x the current solution.
- Fall in love with the Solution: Management shuts down the project due to lack of a viable business
- Fall in love with the Problem: The team updates management that the new technology is not economically viable and asks for an additional 3 months and $100,000 to develop a different solution. Management agrees.
A product team develops a new product, complete with new branding, positioning, and Direct to Consumer go to market strategy. Sales expresses concern that existing retail customers will view this as a threat.
- Fall in love with the Solution: Management cancels the project launch out of fear that retailers will retaliate by discontinuing all of the manufacturer’s products and heavily promoting competitors’ brands.
- Fall in love with the Problem: The team revisits the Go to Market strategy, questioning the importance of Direct to Consumer in solving the problem. The team realizes that, while it is important, it does not need to be the only distribution channel at launch, ultimately recommending that it be used to seed the market before distribution through traditional retailers. Sales positions pre-launch DTC as a net benefit for retailers and management approves the launch.
An innovation team develops a category-creating product that receives overwhelming acceptance in concept tests and receives top scores when evaluated against its nearest competitors. However, when launched into test markets, it fails miserably.
- Fall in love with the Solution – Option 1: Management ends the target markets and cancels the project
- Fall in love with the Solution – Option 2: Management continues to make incremental changes to the product, its positioning and marketing, and its pricing, ultimately spending millions over multiple years with little to no change in product adoption.
- Fall in love with the Problem – Team talks to their target consumers, including the people who gave the product high marks in concept tests, and learns that the problem the product solves isn’t a terribly painful or important problem and, as a result, consumers aren’t willing to spend money or change behavior to adopt a better solution. The team shelves the category-creating product to focus on creating a new solution to the real problem.
Did you see the pattern there?
When you fall in love with the solution and the solution fails, the project stops.
When you fall in love with the problem and the solution fails, you circle back to the problem and try again.
And if anyone would know the importance of being able to circle back, it’s the man credited with coining the “Fall in love with the problem, not the solution” mantra – Uri Levine. The founder of Waze.
Why Companies Need Salmon Ladders to Achieve Their Growth Goals
“You are a salmon.”
This was not the career advice I hoped for.
After months of tests, quizzes, career counseling sessions, and peer support groups, I was sitting in the expansive wood-paneled office of Harvard Business School’s Head of Career Services, eagerly anticipating learning what my ideal career would be. I did not expect to be told that I was a fish.
“You are only happy when you are rebelling against something. You are a salmon. Whatever you choose to do, you must swim upstream.”
This did not sound like a recipe for the peaceful, easy, frictionless career that I envisioned. This sounded like a career of frustration, struggle, and just generally being annoyed most of the time.
It was 100% accurate.
Innovators are salmon
Quick tutorial on salmon – they hatch in freshwater, swim out to the ocean and live in the saltwater before returning to freshwater to spawn and die (assuming the don’t get caught by a bear or fisherman first).
Quick tutorial on innovators – they are born (like everyone else) with creativity and a sense of unlimited possibility, they go out into the world and face the sting of reality before returning to their natural creative state to ask questions, challenge the status quo, drive growth, and (like everyone else) die.
Swimming upstream is exhausting for fish and for people. That’s why most don’t do it. That’s why most fish and people make the sensible choice to go with the flow, work within the system, and find better ways to do the same things.
But innovators, like salmon, instinctively know that to create, they must follow a different and much harder path. Instead of going with the flow, they must question the status quo. Instead of finding a better way to do the same things, they question whether those things even need to be done and create new ways to do new things.
Companies recognize that they need innovators. They need people who think differently and aren’t afraid to take smart risks because these are the people that will create new revenue streams and drive companies’ long-term growth. But, like the river, the company can’t change how it operates.
Companies need salmon ladders
Quick tutorial on salmon ladders – they are structures on or around natural barriers like waterfalls and dams, that have a high enough water velocity to attract the fish but not so much as to exhaust them, and often have very low steps that the salmon can leap up to get to their destination
Quick tutorial on innovation processes – they are structures separate from companies’ existing processes (and the barriers that often slow progress), that allow enough freedom for creativity and experimentation but not so much as to put the company at serious risk, and often have steps or stages that innovators and their projects must leap up in order to launch a scalable business.
Salmon ladders help salmon progress more rapidly and efficiently, decreasing the odds that they die from exhaustion before reaching their destination. Innovation processes help innovators progress ideas more rapidly and efficiently through an organization, decreasing the odds that they die (or quit, or get fired) before reaching their destination.
How to build a salmon ladder for innovators
Google “innovation process” and you will get 1.8 BILLION results in 0.59 seconds. That’s because there is no one magic innovation process that works for everyone.
But as varied as effective innovation processes are, they share common characteristics.
- People over process. If you announce the implementation of a new process, send out a user manual, and then walk away, the process will fail. If you involve people in designing and pressure testing the process, support them in the first 6-12 months of use, and they experience the value of the process for themselves, the process will succeed.
- Goals are good. Constraints are better. Innovation is an investment, and the company expects a return on that investment. Be clear about those expectations from the start. Innovation is not “anything goes!” Be clear about what the company is and is not willing to support before your start.
- Guidance, not direction. Innovation is filled with known unknowns and unknown unknowns and, for that reason, the path to launch isn’t a straight line. But it also can’t be an infinite loop. Innovation processes should set guardrails for the work to be done but shouldn’t be overly prescriptive.
- Insights, not information. Innovation processes should enable learning and encourage its sharing. Building in time for teams to reflect on their experience and write down their insights is critical for continued growth. Weighing the team down with detailed analyses, complicated templates, and constant update presentations doesn’t enable and encourage learning, it slows it down.
- Celebrate milestones, not just launches. More innovation projects will be stopped than will be launched. The stopped projects aren’t failures, they’re lessons learned and steppingstones to the next big launch. Celebrating the milestones that projects achieve, even the milestone of being stopped, will keep innovators energized and swimming upstream far longer than celebrating only the launches.
Innovators do the hard work of swimming upstream.
Help them out by building a process to guide and propel them.
After all, as one of my favorite poems points out,
If you move with the crowd, you’ll get no further than the crowd
…
Simply swimming with the tide leaves you nowhere.
So if you believe in something that’s good, honest, and bright – stand up for it.
Maybe your peers will get smart and drift your way