In Development

This is a guest post by Hadyn Green. Hadyn Green is a Senior Analyst in the Ministry of Education’s Research Division. His main area of work is student assessment and he works with data on a daily basis.

At the Open Govt BarCamp held recently, one group was asked to raise their hands if they believed what they read in the newspaper. I was one of only two people who did so. I’m not entirely gullible, but if the paper says “Mr Heatley announced a 20,000 tonne increase in the total allowable catch for Hoki, New Zealand’s largest finfish export” then I believe it. And why not?

Should the paper then report something about research into fish populations showing a reversal of downward trends, then I will be a little more critical.

A while ago came across an article in the Australian Dental Journal on the role of alcohol in oral carcinogenesis with particular reference to alcohol-containing mouthwashes. Or basically, will mouthwash give you oral cancer? With an abstract like that it was bound to end up in the newspaper.

But sadly this was a clear case of poor research (bold mine):

Professor Laurence Walsh, head of the School of Dentistry at the University of Queensland has rejected the claim and said there was no established link between mouthwash and oral cancer. In a letter to the editors of the journal, Professor Walsh criticised the authors of the paper for drawing on a “small and selective group of studies”. “A wide range of critical and systematic reviews over many years have failed to show any statistically significant association between mouthwash use and oral cancer,” he said. “There is certainly nothing in the current paper to change our thinking in that regard.”

I used this example to start a discussion about good research versus bad research called “Mouthwash vs Deathwash”, because, naturally, this is not an isolated case. These Deathwash research reports are constantly found in major newspapers.

But it seems futile to rail against Deathwash research. After all, there will always be “quirky” research reports or “interesting” secondary results discovered in the primary investigation. Moreover there seems to be a poor public perception of research; that any research that shows the status quo is incorrect tends to be more accepted. I call it the “Iguanodon effect” (research is only right until something else comes along, despite whether that’s correct or not).

So if it’s the case now that we see reported poorly researched, non-peer reviewed analysis of information in the media, what’s going to happen if we open up all government information? I suspect you know the answer. And what do we, as public servants, do when the inevitable happens?

We tend to have to be reactionary. To release a statement like “there is no established link between mouthwash and oral cancer” is silly, because nobody would’ve contemplated that there was. But we feel that we have to be ready with that statement ready to go lest the opposite is reported. As such our risk management processes are finely tuned.

With any luck open government data will help. I don’t expect the ratio of Mouthwash to Deathwash analysis to change, but the subsequent critique of them will be easier and faster and done with more regularity. With open data “the truth” can travel around the world just as fast as “the lie”. Of course the truth still takes baggage with it, but it’ll be packed and ready to go.

Technorati Tags: data, dentistry, mouthwash, opengovt, truth