EuroEpi thoughts

Posted on by
Reply

One thing that has struck me listening to talks at the European Congress of Epidemiology is the incredible weight given to the phrase “statistically significant”. This is an old chestnut among theoreticians in the area, so my surprise perhaps indicates more about my selective contact with epidemiology to date than anything else. It is nonetheless interesting to see the work this strange concept does.

The most striking example was in an interesting talk on risk factors for colorectal cancers. A slide was displayed showing results of a case control study. For every one of the 8 or so risk factors, incidence among cases was higher than controls. However, the speaker pointed out that only some of these differences were statistically significant.

This struck me as very strange. The level of statistical significance is more or less arbitrary – perhaps not entirely, but arbitrary in the same way as specifying a certain height for “short”. In this context, that means that the choice of risk factors to ignore was also, in the same sense, arbitrary. Moreover, the fact that the difference was the same way in all the risk factors (ie higher exposure in cases than controls) also seemed, to my untutored eye, to be the sort of unlikely coincidence one might wish to investigate further.

In a way, that is exactly what came next. One of the “insignificant” factors turned out – and I confess I did not follow how – to interact significantly with another (the two being fibre and calcium intake).

I am not sure that any of this is problematic, but it is certainly puzzling. The pattern is not unique to this talk. I have seen more than one table presented of variables potentially associated with an outcome, with the non significant ones then being excluded. On many occasions this must surely be a good, quick way to proceed. It seems like a strange exercise, to my untutored eye, if some non significant differences are studied further anyway. But that is surely an artefact of my lack of understanding.

I am less sure that my lack of understanding is to blame for other doubts, however. Where a number of risk factors are aligned, it seems arbitrary to ignore the ones that fail a certain level of statistical significance. The fact of alignment is itself some evidence of a non chance phenomenon of some kind. And, of course, the alignment might indicate something important, for example an as yet unthought of causal factor. The non significant factors could be as useful as the significant ones in detecting such a factor, by providing further means of triangulation.

The Myth of Translation

Posted on by
Reply

Next week I am part of a symposium at EuroEpi in Porto, Portugal with the title Achieving More Effective Translation of Epidemiologic Findings into Policy when Facts are not the Whole Story.

My presentation is called “The Myth of Translation” and the central thesis is, as you would guess, that talk of “translating” data into policy, discoveries into applications, and so forth is unhelpful and inaccurate. Instead, I am arguing that the major challenge facing epidemiological research is assuring non-epidemiologists who might want to rely on those results that they are stable, meaning that they are not likely to be reversed in the near future.

I expect my claim to be provocative in two ways. First, the most obvious reasons I can think of for the popularity of the “translation” metaphor, given its clear inappropriateness (which I have not argued here but which I argue in the presentation), are unpleasant ones: claiming of scientific authority for dearly-held policy objectives; or blaming some sort of translational failing for what are actually shortcomings (or, perhaps, over-ambitious claims) in epidemiological research. This point is not, however, something I intend to emphasize; nor am I sure it is particularly important. Second, the claim that epidemiological results are reasonably regarded by non-epidemiologists as too unstable to be useful might be expected to raise a bit of resistance at an epidemiology conference.

Given the possibility that what I have to say will be provocative, I thought I would try my central positive argument out here.

(1) It is hard to use results which one reasonably suspects might soon be found incorrect.

(2) Often, epidemiological results are such that a prospective user reasonably suspects that they will soon be found incorrect.

(3) Therefore, often, it is hard to use epidemiological results.

I think this argument is valid, or close enough for these purposes. I think that (1) does not need supporting: it is obviously true (or obviously enough for these purposes). The weight is on (2), and my argument for (2) is that from the outside, it is simply too hard to tell whether a given issue – for example, the effect of HRT on heart disease, or the effect of acetaminophen (paracetamol) on asthma – is still part of an ongoing debate, or can reasonably be regarded as settled. The problem infects even results that epidemiologists would widely regard as settled: the credibility of the evidence on the effect of smoking on lung cancer is not helped by reversals over HRT, for example, because from the outside, it is not unreasonable to wonder what the relevant difference is between the pronouncements on HRT and the pronouncements on lung cancer and smoking. There is a difference: my point is that epidemiology lacks a clear framework for saying what it is.

My claim, then, is that the main challenge facing the use of epidemiological results is not “translation” in any sense, but stability; and that devising a framework for expressing to non-epidemiologists (“users”, if you like) how stable a given result is, given best available current knowledge, is where efforts currently being directed at “translation” would be better spent.

Comments on this line of thought would be very welcome. I am happy to share the slides for my talk with anyone who might be interested.

Book manuscript – comments invited

Posted on by
1

If anyone is interested in looking at a book manuscript on philosophy of epidemiology, or on any parts thereof, please get in touch. The manuscript is under contract and has been delivered to the publisher, so this is a good time for comments and criticism. Table of contents can be accessed here:

2012-06-29 contents

…with apologies for the “undefined bookmarks”. The publishers have already indicated that the title should be “Philosophy of Epidemiology” or something similarly descriptive – opinions on this also welcome.

Apologies for the recent inactivity on this blog – the book is a big part of the explanation.

Musings on laws of nature

Posted on by
4

I am always puzzled by philosophical talk of laws of nature. The terms “law”, “govern”, and so forth amount to an extended metaphor drawn from human affairs, and thoroughly unnatural ones at that. It is a recurrent philosophical mistake to suppose that the most fundamental thing about the universe can only be treated with philosophical precision through a human metaphor. Without “laws”, and the corresponding but evidently false idea that it is possible to break them, contemporary metaphysics would look quite different. If you ask a philosopher whether it is possible to walk through a wall, she will say “Yes”, because only the laws “prohibit” it – even though everyone knows you can’t walk through a wall.

Post-doc: Philosophy of Science, University of Johannesburg

Posted on by
Reply

The University of Johannesburg seeks to appoint a Postdoctoral Research Fellow in Philosophy. Any research area will be considered, but it is hoped that the Fellow will work with Dr Alex Broadbent on issues in the philosophy of science broadly construed, and in particular issues related to epidemiology, public health, causation, explanation or prediction (although unconceived alternatives will be considered). The Fellow will be asked to teach one Honours course for one term, out of four in our academic calendar. (This is a light commitment: Honours is a small group (5-10) course for first year postgraduates, whose content usually reflects the interests of the person teaching it.) Applicants should email two pieces of written work of around 8-10k words (e.g. publications, work under review, thesis chapters) along with a CV and covering letter to Dr Alex Broadbent at abbroadbent@uj.ac.uk by 30 June 2012. Doctorate must be in hand at time of commencement. Duration is one year in the first instance, with the possibility of another year, depending on publication performance.

Acetaminophen (paracetamol), Asthma, and the Causal Fallacy

Posted on by
2

In November 2011, a senior American pediatrician suggested that there was enough evidence to warrant restricting acetaminophen (paracetamol) use among children at risk of asthma, despite inadequate evidence for a causal inference. His argument was based on an ethical principle. However neither his argument nor the evidence he surveys are sufficient to warrant the recommendation, which therefore has the status, not of a sensible precaution, but a stab in the dark. I have written to the editors of Pediatrics to explain why – the link is here:

http://pediatrics.aappublications.org/content/128/6/1181.full/reply#pediatrics_el_53669

The theoretical point underlying this is one under-emphasized in both philosophical and epidemiological thinking, namely, that causal inference is something rather different from making a prediction based on the causal knowledge so obtained. The temptation to suppose that we have even a hunch what we happen when we restrict acetaminophen use on the basis that we have a hunch that it causes asthma is fallacious. It all depends on what consequences the non-use of acetaminophen has, and that in turn depends on the form that non-use takes. The point is familiar from philosophical studies of counterfactuals, but those studies arguably either do not offer much of practical use for epidemiology or else have not received an epidemiological audience. (I favour the former option, although I realise many philosophers will disagree.)

The result is a common fallacy of reasoning which we might call The Causal Fallacy: epidemiologists, policy makers, and probably the public assume that because we have causal knowledge, we have knowledge of what will happen when we manipulate those causes. In practice we do not. (This under-appreciated point has been emphasized by Sander Greenland among epidemiologists and Nancy Cartwright among philosophers, and as I see it tells heavily against the programme of manipulationist or interventionist theories of causation.) Establishing whether an exposure such as acetaminophen is a cause of an outcome such as asthma is not sufficient to predict the outcome of a given recommendation on the use of acetaminophen, for the simple reason that more than one such policy is possible, and each may in principle have a different outcome.

Taubes’ Tautology

Posted on by
Reply

In the once fertile garden of epidemiology, all is not well, according to some commentators. The low-hanging fruit has been plucked, and the epidemiological ladder is not long enough to bring the remainder within reach. Possibly the most famous expression of this dissatisfaction is a report by a journalist writing in Science in 1995 called “Epidemiology Faces Its Limits”. Gary Taubes cites a number of contrary findings, where exposures have been found to be harmful and then safe (or vice versa) in different studies, or harmful in different ways, or harmful when studied using one study design but not when using another. He interviews a number of eminent epidemiologists and reaches a simple diagnosis: epidemiology has spotted the big effects already, and is now scrabbling around trying to identify small ones. These are much harder to distinguish from biases or chance effects. Indeed, he hypothesizes that epidemiological methods may be unable to tell the difference at all, in some cases. In this sense, Taubes suggests, epidemiology is facing its limits.

The epidemiological garden is still growing nearly two decades later. Either the gardeners did not listen, and continued to tend fruitless trees, or Taube’s diagnosis was wrong. But epidemiologists did listen: the piece is well-known. Moreover epidemiologists are among the most methodologically reflective and self-critical of scientists, which is evident from the fact that most of Taubes’ criticism is drawn directly from interviews with epidemiologists (and which is one reason epidemiologists are such a pleasure to engage with philosophically). The implication is that Taubes’ low-hanging fruit hypothesis is mistaken.

Taubes’ hypothesis is tempting because it is true that big discoveries lie in the past. It is, however, a fallacy to suppose that this means no big discoveries lie in the future. On inspection, the tempting hypothesis reveals itself as an instance of a very common theme: that we are nearing the end of what inquiry can tell us. This has been said before, most famously in physics shortly before Einstein’s impact. If the history of science tells us anything it is that this claim is always false. We know more about the past than the future, and so we know what the big discoveries of the past are, but not the big discoveries of the future. If there were low-hanging fruit that epidemiology has not yet plucked, then we would not know it, even if they were going to be plucked tomorrow afternoon. More is needed to prove that epidemiology faces its limits than that the tautologous claim that its most striking discoveries to date lie in the past.