Author's Blog: "The Four Principles of Enduring Success"? Not Very Likely!

The July-August 2007 issue of Harvard Business Review has a lead article titled “The Four Principles of Enduring Success” by Christian Stalder of Innsbruck University. It's the latest effort by business researchers to identify the principles of high performance in business, and it's worthy of examination on a website that is interested in clear thinking about business performance.

First, a personal note. I met Mr. Stalder at the meetings of the Strategic Management Society last autumn in Vienna, where he heard me discuss some of the themes in The Halo Effect. He is a smart and well-intentioned man. Recently, Mr. Stalder kindly sent me a link about the article, and since then we have corresponded about the methodology he used in his study. In addition to the HBR article, there is expected to be a book coming out in the future. So perhaps there is a way for my comments, below, to sharpen the study and improve the validity of its claims.

Here's a summary of the article, as sent by Mr. Stalder:

“The Four Principles of Enduring Success

“When your company is doing well, and money is pouring in, how do you know if it could be doing better? How can you tell which management practices are making the difference—and which are merely not doing obvious harm?

“To find out, Christian Stadler and a team at Innsbruck University’s business school conducted a massive benchmarking study comparing nine pairs of European companies over 50 years. Each pair was from the same industry (and, preferably, the same country) and included one exceptional performer and one less impressive, but solid performer.

"The project yielded four main findings, which Stadler calls the four principles of enduring success:

“Exploit before you explore. Great companies don’t innovate their way to growth—they grow by efficiently exploiting the fullest potential of existing innovations.

“Diversify your business portfolio. Good companies, conscious of the dangers of irrational conglomeration, tend to stick to their knitting. But the great companies know when to diversify, and they remain resilient by maintaining a wide range of suppliers and a broad base of customers.

“Remember your mistakes. Good companies tell stories of success, but great companies also tell stories of past failures to avoid repeating them.

“Be conservative about change. Great companies very seldom make radical changes—and take great care in their planning and implementation.

“How much difference do these principles make? An investment of $1 in 1953 in the group of companies in the study that consistently applied them — insurers Allianz, Legal & General, and Munich Re; financial services firm HSBC; building materials maker Lafarge; high-tech firms Nokia and Siemens; oil giant Shell; and pharmaceutical firm GlaxoSmithKline — would be worth $4,077 today. A $1 investment in the comparison companies — Aachener und Münchener, Prudential Limited, and Cologne Re; Standard Chartered; Ciments Français; Ericsson and AEG; BP; and Wellcome — would have yielded $713.”

What we have here is essentially a European replication of Jim Collins and Jerry Porras's 1994 best-seller, Built to Last – with, in my view, the same appeals and the same weaknesses. The critique I made about Built to Last, in chapter 6 of my book, applies here as well. For all the appearance of rigor – lots of data, gathered over several years – there are some basic problems that undermine its claims.

Here's what I think Mr. Stalder and his team of researchers have done. First, they selected their sample based on the dependent variable, which is some index of performance. Second, they identified top performers (gold medal) and comparison companies (silver medal). So far, that's fine. The problem is not one of dependent variable or design. The problem, instead, is about the data they relied upon for their independent variables, that is, those things they use to determine what sets the best performers apart from the others.

As Mr. Stalder explained in a subsequent correspondence, they drew on three sources of data: publicly available data, archival data, and interviews. There is a claim that by using three sources, they have “triangulated” – that is, not relied on any one, but sought a truth that emerges by drawing on a three sources. But triangulation is not achieved by drawing on three sources, for all three may be biased. Triangulation results from using one or more valid sources to help improve the validity of data from other sources that may be prone to bias. For example, using the transcripts of a meeting may help guide the retrospective recollections of participants at the meeting, and ensure that their memories are grounded in fact. The result may not be perfect recollection, but more accurate than in the absence of no transcript.

Unfortunately, in Mr. Stalder’s description of method there is no mention of data validity or independence at all, any more than was present in the work of Collins and Porras. There is, instead, mention of the “massive benchmarking study” with vast amounts of data gathered, as if data quantity by itself were important. This is what I call the Delusion of Rigorous Research, an attempt to impress (or even overwhelm) readers by emphasizing the quantity of data, when of course quantity hardly matters if the quality is poor.

Some of these data are probably valid and unbiased -- namely financial statements, stock market performance, and perhaps some measures of operational performance that come from objectively recorded sources. But much of publicly available data, much of archival material, and most retrospective and even contemporaneous interviews are likely to be shaped by what is known about performance. The researchers have not met the most basic standard of ensuring that independent variables are unbiased by the very thing they are seeking to explain, namely firm performance. They have not shown what leads to high performance; they have shown how high performers tend to be dexcribed.

As a result, the conclusions may seem reasonable, and no doubt they are in part true. But they are not obviously different than retrospective sense-making. In fact, some of the four points are, in my view, lacking in any intellectual underpinning at all. Take the first two. What they say, in combination, is: exploit but also diversify. Don't innovate too much, but also don't stay too focused. Well, if we select our sample based on performance and gather data retrospectively, we can always say that the winners got this balance exactly right, and that the silver medalists were too far on one side or the other. But that is just retrospective sense-making. To have any merit, we would need to be able to show, ex ante, what a company ought to do in the future in order to strike the balance. As for remembering mistakes, this analysis of Shell and BP is not only superficial and based on questionable sources, it is also highly dependent on the precise moment at which you determined who was more successful than the other. It is easy to criticize BP today, but had they conducted their analysis at a slightly different time, the "stories" that were told might have been rather different.

No doubt this book will have some appeal to managers who want to believe that they can achieve long-term success by being conservative, exploit, diversify moderately, and tell stories about failures. But do we really think that a company which earnestly strives to do these four things will achieve lasting success? I think the story is more complex than this -- the reliance on flawed data has led you to a series of simplistic (although perhaps appealing) results.

As noted above, Christian Stalder is a serious scholar and is interested in improving the validity of his study. He would not have contacted me and asked for my input if that were not the case. I wish him well – and hope he can improve the validity of his claims. What the Innsbruck researchers should do is this: set aside all data from sources that have questionable data independence, leave in the data set only those that are valid and not biased, and run the finding again. If they would be willing to do that, I would love to see the results – but I suspect they will not claim with confidence that following these four principles leads to enduring and significant differences in performance.

For its part, Harvard Business Review (which favorably reviewed The Halo Effect in March and seemed to approve of my main points) should improve the standards by which it edits articles and accepts them for publication. Lead articles that claim to offer “four principles for enduring success” should be based on more than an abundance of suspect data.

Comments

Spot-on! I read the HBR article and was surprised that Mr. Stadler's work was included for publication--but then again, HBR has been slipping for quite some time. Kudos to professor Rosensweig for pointing out the many flaws in this type of business "research".

Posted by: tj | February 10, 2008 at 12:13 AM

cialis soft generic cialis soft order cialis soft tab description cialis soft tab india cialis soft tablets cialis soft tabs 10 mg cialis soft tabs bestseller cialis soft tabs online cialis soft top cialis softabs cialis softabs generic cialis softtab how works cialis softtabs online

Posted by: RX-order | November 21, 2010 at 01:25 AM