Cost Benefit Analysis of Failure And Innovation
We tend to focus on just one side of that equation
A discussion this week with the members of the Institute of Innovation in Large Organizations (ILO), focused on how to lower the cost of failure when working on innovative projects that by nature would carry more risk than merely maintaining ongoing practices. I have thoughts.
The discussion did a pretty solid job at defining what we mean by “costs”; typically dollars and time but also brand reputation and customer relationships. Those are all valid costs to consider and should be weighed on a case-by-case basis depending on the scope and scale of the innovation work. At Amazon, we referred to the tension here as being between a one way door and a two way door. Two-way doors are those that are easy to come back through - they carry little time/dollar cost, and have low potential to impact the brand rep or customer relationships. One-way doors, are just the opposite - they carry the potential of high cost in all those areas.
What’s interesting to me here, is how easy it is and how quickly we can define the costs that we’re looking to lower but defining the benefits often takes longer or is less expansive. Tells me that we’ve been taught to focus on the costs to a degree that we can ignore the variety of benefits that can accrue so I’ll just talk about one example - running an enterprise-wide innovation/idea generation program at Amazon and the benefits that I saw come out of that.
“Think Big” is one of the Amazon Leadership Principles. It’s also the name we gave to the program that we ran that solicited ideas from Amazonians anywhere in the world. The most obvious benefit that people will ask about or look for is how many of those ideas made it into production and resulted either in an increase in sales or internal savings. That’s also going to be the smallest fractional benefit that will come from a program like that. To be sure, there were some outstanding ideas and they got briefed to senior leaders who could take them all the way into production but not many made it that far. When we went back out and talked to people about the benefits they found in the program, one benefit that loomed large in their minds was the experience gained from participating in a process in which you had to master the format and process for writing up your idea and then getting feedback on how you did - both on the idea and on process. We heard a lot about how those were opportunities to learn and grow in those areas that they would not have received any other way (especially from junior folks). So maybe raising the bar for the entire organization’s ability to think about innovative ideas and build those thoughts into cogent presentations is a benefit we could put on that side of the ledger.
The team we had to manage the Think Big program was small (but mighty) and we realized quickly that we would not be able to judge all the submitted ideas ourselves so we’d need to figure out how to crowdsource that process. I’m proud of the fact that we were committed to ensuring that every idea would receive feedback (since we’d heard from folks that they considered that feedback on their idea as basically a prize) but that meant we’d need judges, lots of judges. It also meant that we’d have to provide some kind of training to ensure that we had a consistent quality level from the judges. We did all that - people were super supportive in signing up to judge and completing the training. One interesting dynamic we found was that some of our best judges (looking at the most ideas and providing the most in-depth feedback) were people who had never submitted an idea in any of the years we ran the program. It really hits you, when you see the numbers, that some of the best people at assessing ideas in your organization and folks who would never think of submitting their own ideas but they are a critical part of the innovation ecosystem. I’d count this as another benefit - identifying the segment of your organization that can assess ideas well and then in turn, providing training to that segment to again raise the bar across this group in terms of quality.
So we’ve added increasing the organization’s overall ability to surface and present ideas and identifying and valuing a population of the org that is great at assessing those ideas to the positive side of the ledger - I’ve got one more. You can think about incubating ideas but you can also think about incubating people (no, not like in the Matrix or if you’re old enough, Soylent Green). Innovation programs allow you to watch people in real time, in a non-production environment, manage people, budgets, schedules, presentations to leadership, etc - all in a way that probably doesn’t exist anywhere else in the org. You run some pilots and maybe they don’t work out - that possibility of failure is just table stakes for pilots - but you surface the next generation of senior leaders by watching how they deal with the pressures of these pilots, how they deal with them when they succeed and more importantly, how they deal with them when they fail. That’s not an inconsequential benefit.
The point of all this is that maybe if we learn to think more expansively about the benefits that can come from running innovation programs at scale, instead of simply focusing on how many ideas make it into production (still a reasonable criteria but maybe it shouldn’t be the only one), then our cost/benefit analysis might look a little different.