10 July 2006

They’re my statistics and I’m taking them home with me….

Lighthouse Analyst Relations MD, Duncan Chapple, is not a man with whom to trifle.

Lighthouse’s annual survey of vendor opinion on analyst services concluded that vendors find – in order – Forrester, Gartner, Ovum, Aberdeen and Burton the most influential (yes, that really is Aberdeen in fourth place…).

This clearly piqued the head of research at one analyst firm, presumably not one from Forrester. He asserted that the central hypothesis contained a scientific flaw and sent a terse note to Lighthouse, before crying into his hot milk before bedtime.

Full of academic outrage, Duncan threw on his smoking jacket and trod downstairs heavily to the study in order to author the type of riled riposte that only an analyst could, including the succinct killer paragraph:

“A small firm probably won't win as much influence in a year as a large firm. That is real. In our opinion, it's the right way to show the data. There is an option, which we dismissed. The option is that instead at looking at the changes from year to year, we could look at the rate of change. For example: firm A goes from influencing 5% of firms to influencing 10% of firms; but firm B goes from influencing 0.05% of firms to influencing 0.15%. Our current method makes it look as if as firm A has increased the most. We think that's right. The option would be to focus on the fact that Firm B has had a larger percentage growth, but it's still tiny. We think it would be much worse to say that firm B has increased its influence the most. That's our approach, and we stick by it.”


Come on lads, get over it. Statistics are rubbish, and are only of use to simplify a technology story for national media. Even then you generally have to use leading questions and favourable analysis for them to serve their purpose. So stop pretending any of the twisted number crunching you do actually means anything, and if you have a problem with each other just have a fight in a car park like real men.

1 comment:

Duncan Chapple said...

I appreciate that many PR people feel that statistics are only used as a rust nail to hang bad stories on but, in fact, the rest of the world has found that statistics are a necessary pillar of industrial society. Our research isn't used for media work, but to guide managers' decisions.