A new standard for policy-relevant science?
The increasing concern about unreliability in scientific literature1, 2 is a problem for people like me — I am the science adviser to DEFRA, the UK government department for environment, food and rural affairs. To counsel politicians, I must recognize systematic bias in research. Bias is cryptic enough in individual studies, let alone in whole bodies of literature that contain important inaccuracies2, 3.
It worries me that because of bias, some parts of the published scientific literature, such as studies on the safety of genetically modified (GM) organisms and pesticides, or trends in biodiversity measurements, might have only limited use in policy-making.
To mitigate this problem, policy-makers should consider holding published scientific evidence to an audited standard that can be replicated and is robust to variations in assessor competence. A weighting factor, or ‘kite mark’, applied to journals or individual articles, could help policy-makers to assess the robustness of studies for use in particular applications. Similar methods established by non-profit standards associations are used in research to certify laboratory practice and in engineering to certify building standards.
The quality of research results fluctuates because of varying tractability in the problems being probed1, 4. For example, it is easy to judge the efficacy of an experiment to engineer a tomato to produce the pigment anthocyanin5, because if it succeeds, that tomato is the colour of a ripe plum. It is much harder to judge the reliability of a study investigating whether a GM crop is toxic to animals6. The latter situation is much more susceptible to inaccuracy and interpretation.
Click here to read more from this September 11, 2013 Nature article by Ian Boyd.