The Reuters article by Helen Chernikoff, published on June 7, is to my knowledge the first time that a reporter contacted for comment the authors of some of these books I critique. It’s most interesting to see the replies!
Here’s what Ms. Chernikoff wrote:
“Peters, one of the authors of In Search of Excellence, said that while his book does contain a list of recommendations, ‘It is not prescriptive in the sense that it says if you do this, you alone will have a great company.’
“Porras, who said he has read part of The Halo Effect, notes that Built to Last explicitly warns readers against drawing conclusions about cause and effect from its examples.
“Collins declined to comment on The Halo Effect.”
As for Tom Peters’s reply, readers of my book will know that I am fairly charitable about In Search of Excellence. For all the
errors that Peters and Waterman made – first, errors of design, as they selected a sample of companies based on outcome, then compounded this mistake by errors of data, relying on data that were undermined by the halo effect – at least they did not make extravagent claims about scientific rigor. In Search of Excellence provides appealing stories, but never claimed to do a great deal more.
The same cannot be said of Built to Last or of Good to Great, both of which claimed to have conducted rigorous, exhaustive research, and to have found the reasons why some companies achieved high performance while others did not. Both books are explicit: If you learn from these successful companies, you can be successful, too. It's hard to see how this claim does not suggest cause and effect!
In the case of Built to Last, there is an explicit claim to have found why a set of 18 firms (the so-called Visionary companies) achieved lasting success, while a group of Comparison companies did not. Their findings rest on a large set of data, but much of it is biased and not valid because the observations are not independent of performance itself – that’s a central idea in The Halo Effect. That critique holds whether the authors claim their findings offer a narrow explanation of retrospective performance only, or whether they make the claim of prospective performance. (To my reading, the book, which says it offers a “blueprint” for lasting success, makes the latter claim of predictive causality, but even if we accept Porras’s disclaimer, the book is explicit that it has explained why the Visionary companies did better than Comparison companies.) Either way, their findings are flawed since they relied on a great deal of invalid data. As I write in Chapter 6, Collins and Porras do not offer an adequate explain of why some companies achieved enduring performance while others did not – rather, they showed how enduring companies tended to be described -- which is an entirely different matter. (Which also helps explains, by the way, why so many of these enduring faltered as soon as the period of study ended!)
As for Good to Great, we only need to read the full title: Good to Great: Why Some Companies Make the Leap ... and Others Don’t. There it is: a definitive statement that the book can explain why some companies achieved high performance while others did not. This claim is reinforced by the repeated and explicit mentions of “the enduring physics of great organizations,” the “timeless principles” that are said to “remain true and relevant” over time, and the “immutable laws of organized human performance.” How can these statements be construed as anything other than a promise of causality? Why does Jim Collins refer so explicitly to the world of science if he does not mean to suggest that he has shown causal predictability? Much of the appeal of Good to Great is precisely its claim of improved performance for those companies that follow its advice. The authors cannot claim to have it both ways.
None of this suggests that Collins and Porras’s work is entirely without merit. I mention in several places that Collins and Porras are wonderful storytellers. Their books inspire managers and point them in generally useful directions – who can deny the importance of such things as caring for customers, having strong values, and being persistent in the face of adversity? Those are all helpful concepts, and to the extent that managers come away inspired to do good things, that’s not bad at all. But a good story is a long way from a claim of solid research or scientific rigor – with is the claim made repeatedly throughout both of these books.
By relying on data that are contaminated by the halo effect, Collins and Porras have produced work that does not remotely meet the most basic test of good research. Simply stated, much of what they use for their independent variables are not independent of the thing they are trying to explain, namely firm performance. And as a result, they divert attention from some of the most important elements that drive performance, namely the need to take strategic decisions that will make a company different from its rivals, yet which are made under conditions of uncertainty and therefore involve risks. The emphasis on formulas, with the strong suggestion of predictable high performance, is not only wrong, but is disingenuous. A more accurate understanding of business performance needs to shift away from the easy formulas of these books, and embrace a recognition of choice, risk, and uncertainty.
Trouble on the horizon. Christian Stradler, an associate professor at Innsbruck University School of Management in Austria, has a forthcoming book titled, "Enduring Success: What We Can Learn from the History of Outstanding European Corporations." He previews the book in the July-August 2007 issue of Harvard Business Review.
If the title raises suspicions of the same sort of methodological flaws that plagued Collins' books, the intro to the HBR article will quickly give you a sinking feeling. Prof. Stradler writes: "The seminal work of Jim Collins and Jerry Porras ..." Uh-oh. Here we go again.
Sure enough, the book is the European version of "Built to Last," comparison companies and all. Prof.Stradler’s research consisted of:
* Initial company selection based on which companies were featured in the Fortune Global 500
* Further selection based on which companies outperformed the major market indexes (Dow, DAX, FTSE) by at least 15 over the chosen period
* Surveys and interviews of financial analysts and business scholars
* Years studying corporate histories and collecting and coding articles and other archival materials (HSBC's corporate history "comprised four volumes totaling 3,114 pages!" he adds breathlessly.)
* Interviews with active and retired executives of the companies
Contrary to a lengthy recounting of this exhaustive research in the HBR article, and the title of his book, Prof. Stradler adds this caveat: "Our intention is merely to contribute to the ongoing discussion about what really works -- not to claim discovery of the ultimate truth."
Posted by: Jordan Pine | June 26, 2007 at 12:14 AM
There are some wonderful posts here! The Halo effect is something very interesting but if you want to see something different you have to visit the Halo Bungie site.
Posted by: michael jones | April 16, 2008 at 11:25 PM