Volume 2, #3, March 2004
Making Innovation Count: A Framework for Measuring the Creative Contribution to Product Development
"Innovation is the specific instrument of entrepreneurship. the act that endows resources with a new capacity to create wealth."
-- Peter Drucker, Innovation and Entrepreneurship, quoted in Harvard Business Review May/June 1986
A quick search on Google for "history of innovation" yields nearly 7,000 results, many of which come from corporations using the phrase to describe their own activities. Given this, you'd think that most companies understand the process of innovation and can use it effectively to provide their businesses "with a new capacity to create wealth," as Peter Drucker says.
You'd be wrong. Many companies view innovation as a necessary evil arising spontaneously from an ill-understood confluence of circumstances, and allow it to run along its own track with little management oversight. Even acknowledged leaders in innovation, such as 3M and IBM, struggle with how to best harness innovation in the context of a profit-making enterprise.
As with any business process, the first step in exploiting innovation to serve your company's goals is to measure it -- an inherently difficult undertaking. Innovation is a creative process, which, by its very nature, involves doing things that have never been done before. Yet, if you don't measure it, you can't manage it, and if you can't manage it, you can't control it, and if you can't control it, it can sneak up and sabotage your business before you even know there's a problem.
The message of this article is that you can measure and optimize innovation. Read on to find out how.
Innovation in context -- where does it fit?
In general, the innovation process precedes the more structured product development process. Innovation, as we define it (see sidebar), might begin with an employee's "aha!" moment in the shower before coming to work. The employee might be inspired with an idea for a totally new technology, a completely new way of providing a service, or even a new way of selling, as in the case of Dell Computer, which pioneered the idea of customers specifying build-to-order systems online.
The inspiration then enters two distinct phases. The first, which we refer to as ideation, is the process of filtering the idea. The company examines the shower inspiration to see if it makes sense and to determine whether it's a viable strategic and market fit that the company wants to back with an investment. During the second phase -- incubation, in our parlance -- the company begins to spend money to determine whether the idea is feasible. Both phases occur prior to the idea's incorporation into a specific product. After incubation, the idea moves into whatever process the company uses for product development, and the innovation phase ends.
What's wrong with current ways of measuring innovation?
The problem is not that companies don't measure innovation today. Every company does, in one way or another. Most innovation metrics, however, are immature. Either they are takeoffs on regular product development metrics or are based on a single company's past experience. Since neither approach uses objective facts about what succeeds and what doesn't, you might as well close your eyes and throw a dart at the wall to determine how to encourage innovation that leads to successful products.
A robust innovation maturity model needs to address both the ideation and incubation phases, as well as a critical third piece: the management system that enables innovation to be successful. Without the right controls, funding, and processes to bring an idea from the employee's shower to the customer, a company may fail even if the rest of the innovation system is strong.
As with any metric, it's important to understand that the act of measuring doesn't actually solve problems. It does, however, provide information so leaders can make more informed decisions, giving everyone not only an accurate view of where they stand, but what they have to do to improve.
Who needs an innovation framework?
Companies whose bottom lines are driven by innovation are good targets for applying an innovation framework such as the one developed by IBM and PDC (more on this to follow). For example, in the heyday of deregulation, power companies were looking for innovative ways to differentiate themselves and increase profitability. Now that deregulation has receded as a business force in the power industry, power companies have returned to core businesses, and probably would not be good candidates for applying an innovation framework. In contrast, many of the technology and service companies that weathered the economic downturn by cutting R&D budgets now have far fewer products in the pipeline than is healthy. They need to pump up the volume, but how? Using an innovation framework would allow them to get more products into the pipeline not simply by throwing money at the problem but by spending that money in ways that make sense and minimize risk.
IBM decided to tackle the problem of measuring innovation directly. Through work with its Emerging Business Opportunities (EBO), IBM realized it couldn't treat the development of completely new products and technologies the same way it treated ongoing product development. The EBO initiative introduced a different set of rules. For example, it's impossible to do a clear business case or an accurate market projection before the breadth and depth of any technology's potential use is known. So best practices in the EBO phase supplement traditional business analysis and market projections with efforts like technology diffusion studies. One of the goals of IBM's EBO phase is to grow revenue by finding the potential breadth and depth of a new technology's application. That information, combined with the current market size of the potential applications, gives insights into the relative value a new EBO.
Still, even an innovation leader like IBM, which realized the right way for EBOs to go, had no objective way to measure the many processes involved. That's when the IBM corporate EBO process architects contacted PDC. With 14 years of client work and a database of innovation best practices, we were able to provide the hard data to help IBM sort out what works and what doesn't. What are the key elements of innovation? What differentiates success from failure?
Although many people believe that real breakthroughs come from the lone entrepreneur working in a garage, PDC's research shows that larger companies have a greater need to control the innovation process. In IBM's case, for example, the company had no trouble generating ideas. The challenge was getting the right ideas into the system and then rapidly turning them into profitable products -- finding the elusive control we mentioned earlier.
One of the ways companies are successful in innovation, of course, is to purchase smaller companies, and the model addresses this as well.
The nuts and bolts of the model
PDC developed the innovation maturity model in conjunction with IBM during 2003, using both proprietary and public research. The model is divided into five parts and covers 30 aspects of innovation. Why 30? We wanted enough to provide meaningful information, but not so many points as to make the model cumbersome. According to Dave Coughlin, Executive Consultant for IBM's integrated product development team, "We had lots of discussion about what were the key elements that the model needed to cover. Originally, we started with 80, then cut it to 60. We still felt that was too many."
People simply won't use any model that requires special training or setup, so we kept the model simple. "We spent time taking surveys from other organizations," says Tom Luin, Business Transformation Architect and another key team member. "They took an hour or two to complete. We knew most executives wouldn't put in that much time. Ours takes about 15 or 20 minutes, and that has proven very useful. Almost everyone agrees to take that amount of time to answer 30 questions with multiple choice answers."
The model is supported with an Excel spreadsheet including dropdown answers. It begins with an introductory section that summarizes the company or business unit being studied and asks about its relative success in innovation. This information is used to sort the data and statistically correlate individual elements to success. Each of the next three sections asks ten questions, to which there are five possible answers. These sections cover management system, ideation, and incubation. The final section is an automatically generated report showing, in chart form, where a company stands in relationship to the rest of the company and to other companies.
The five potential answers to each question in the management system, ideation, and incubation sections describe five specific items that characterize a level of maturity. It's important to note that all the answers are based on real and demonstrated practices at companies. This means that although you might look at the highest level and say, we could do better than that, you'd be attempting something that no company has achieved in the real world. Likewise, the lowest level may not be as bad as it could get, but it represents the worst that any company is doing in the real world. The key to establishing a relevant baseline is using objectivity in answering the question.
As with any other benchmarking scenario, no single company will achieve the highest level in every area. That would be an unrealistic goal, like taking the gold in every Olympic event. The genius of benchmarking is that it points out where your weaknesses lie, so you can take steps to address them. Using the reporting section at the end of the tool to map your company against the database for all other companies can be particularly revealing.
Using the tool
There are a couple of steps to using the tool. First, someone with a fairly high-level view of an organization or project fills out the questionnaire, which usually takes 15 to 20 minutes. Then the company goes through a process called calibration. This involves a small group of people, usually between four and 15 -- reviewing the answers to achieve consensus. Calibration is a significant element of the process, because it ensures that everyone who answers the questions understands them in the same way and ensures that everyone comparing himself or herself to the database is looking at a single version of the truth.
Interesting -- and potentially distorting -- things can happen if a company skips the calibration process. In PDC's work with clients, we have discovered that approximately 20 percent of the answers change in some way as a result of the calibration process (which sheds a less-than-flattering light on traditional benchmarking questionnaires without calibration built in). Tom Luin of IBM agrees. "Using individual answers, we can come up with something that's fairly close to accurate. The collaboration step allows everyone to come to a consensus about what the answers are for their business unit. You eliminate the extreme viewpoints. It's a team-building exercise as well. Everybody gets to say what's important to him or her. It's a very useful organizational construct." We also have found that simply knowing others will review their answers causes people to answer questions more thoughtfully, although of course there is no way to measure the effect of this.
Let's look specifically at one area of the ideation section, portfolio management, to see how the process works. The following five answers to the question about how you handle innovation portfolio management, as with any other section, are based on real company experience and range from poor to excellent. Here's the basic idea, with answers ranging from worst to best:
- Funding is ad hoc or "creative." There is no central portfolio management.
- One or two people do portfolio management. It is centralized, but proposals must be justified and rejustified as funding often moves from one project to another based on short-term goals.
- Portfolio management follows a process, usually annual, but data about potential projects is either incomplete or not credible.
- Portfolio management is process- and data-driven, but adjustments are made based on short-term events.
- Portfolio management is process- and data-driven and regularly adjusted, with clear tie-ins to business and technology strategy.
In general, most companies will probably find themselves starting at level 3. They have some sort of a process in place, but individual projects offer wild projections about potential success, showing off the proverbial hockey stick in management presentations. This makes executives highly skeptical of the data, which makes it hard for them to justify decisions. The benchmarking exercise has revealed that the company needs better data to advance to the next level.
It's usually best to target improvements one level at a time. However, the process can also reveal that you might want to jump ahead faster than that -- if, say, your company were at a level 2 and others in your industry were at level 4. Again, measurement doesn't solve the problem for you, but it tells you the size of the problem and how much to worry about it, allowing you to put resources where they can do you the most good in your marketplace.
The process also can be used within a company. Interestingly, one company we worked with allowed different internal research organizations to compete for dollars. If a particular group were mature in an area, it would get more money. This made for a healthy internal competition. Once you can measure how well a research organization does research, then you can give people incentives to improve.
A work in progress
At IBM, Luin and Coughlin's group intends to use the framework internally to evaluate implementation of an innovation management system, and externally to compare IBM's activities with other companies. "We've done some of both already," Luin says. They have taken it to six or seven companies externally, and they have asked a number of internal EBO groups to use the framework. "They found it interesting, but we haven't determined how to turn interest into action." Part of that is a question of priorities, and part is related to the challenges of trying to change management culture (see the accompanying interview.) "The idea of talking about and managing EBOs is a culture-shift from IBM's traditional management style and executive experience."
While IBM already has gained from using this model, there is more work to be done. According to Luin, the philosophical feedback received from executives in EBO areas was very positive. "Almost all of them said, 'This is good stuff, it should work; it's a different way of thinking about innovation but it shows promise. We need to pilot it and demonstrate its usefulness.' We are still very much in the middle of that process."
One thing that would advance the process is a bigger database of participating companies, which would help refine the results and ensure they are statistically significant. We at PDC would like to correlate individual answers with levels of success to determine which areas are most critical. That said, this is still one of the only research- and data-driven methods for managing innovation and a good place for any company to begin. "We planted the seed and it's starting to grow," Luin says. Coughlin adds, "We're incubating the idea."
What's in it for you?
To figure out whether your company could benefit from innovation measurement, you first need to determine the strategic importance of innovation to your company. Is innovation a central competitive factor in your industry, or is the market driven by other factors? If innovation is significant, are you satisfied with the efficiency of your R&D efforts, or are you wasting R&D dollars chasing the wrong innovations? Finally, take a hard look at your current innovation process and how well it measures the critical elements. If your process has room for improvement, you may benefit from a formal assessment of your company's innovation maturity. This isn't an easy road, but it can transform your business from one in which innovation is just a marketing buzzword to one in which innovation drives revenue growth.
Links
Articles by Robert Tucker on Innovation Resource
Interview with Robert Tucker on Innovation Metrics
The House of Innovation - Weblog on innovation, thread on metrics
Innovation Tools - Weblog on innovation, thread on metrics
Management of Innovation and New Technology (MINT) Research Centre at McMaster University
Read the Customer Commentary
