ML in epidemiology: thoughts on Keil and Edwards 2018

Found a great paper (that I should already have known about, of course): Keil, A.P., Edwards, J.K. You are smarter than you think: (super) machine learning in context. Eur J Epidemiol 33, 437–440 (2018). https://doi.org/10.1007/s10654-018-0405-9

Here are some brief thoughts on this really enjoyable article, which I would recommend to philosophers of science, medicine, and epidemiology looking for interesting leads on the interaction between epidemiology and ML – as well as to the target audience, epidemiologists.

Here are some very brief, unfiltered thoughts.

  1. Keil and Edwards discuss an approach, “super learning”, that assembles the results of a bundle of different methods and returns the best (as defined by a user-specified, but objective, measure). In an example, they show how adding a method to that bundle can result in a worse result. Philosophically, this resonates with familiar facts about non-deductive reasoning, namely that as you add information, you can “break” and inference, whereas adding information to the premise set of a deductive argument does not invalidate the inference provided the additional information is consistent with what’s already there. Not sure what to make of the resonance yet, but it reminds me of counterexamples to deductive-nomological explanation – which is like ML in being formal.
  2. They point out that errors like this are reasonably easy for humans to spot, and conclude: “We should be cautious, however, that the billions of years of evolution and experience leading up to current levels of human intelligence is not ignored in the context of advances to computing in the last 30 years.” I suppose my question would be whether all such errors are easy for humans to spot, or whether only the ones we spot are easy to spot. Here, there is a connection with the general intellectual milieu around Kahneman and Tversky’s work on biases. We are indeed honed by evolution, but this leads us to error outside of our specific domain, and statistical reasoning is one well-documented error zone for intuitive reasoning. I’m definitely not disagreeing with their scepticism about formal approaches, but I’m urging some even-handed scepticism about our intuitions. Where the machine and the human disagree, it seems to me a toss-up who, if either, is right.
  3. The assimilation of causal inference to a prediction problem is very useful and one I’ve also explored. It deserves wider appreciation among just about everyone. What would be nice is to see more discussion about prediction under intervention, which, according to some, are categorically different from other kinds. Will machine learning prove capable of making predictions about what will happen under interventions? If so, will this yield causal knowledge as a matter of definition, or could the resulting predictions be generated in a way that is epistemically opaque? Interventionism in philosophy, causal inference in epidemiology, and the “new science of cause and effect” might just see their ideas put to empirical test, if epidemiology picks up ML approaches in coming years. An intervention-supporting predictive algorithm that does not admit of a ready causal interpretation would force a number of books to be rewritten. Of course, according to those books, it should be impossible; but the potency of a priori reasoning about causation is, to say the least, disputed.

Spectator podcast Women with Balls, with Oxford epidemiologist Sunetra Gupta

https://www.spectator.co.uk/podcast/with-oxford-epidemiologist-sunetra-gupta

Katy Balls talks to Sunetra Gupta, Professor of Theoretical Epidemiology at the University of Oxford. An expert in the fight against infectious diseases, she is the lead scientist behind the Oxford study that disputed Imperial College’s dire coronavirus predictions. She is also a novelist and translator. On the podcast, she talks to Katy about her writing and how it was inspired by her intellectual father; her dispute with the mentor of Imperial College’s Neil Ferguson; and how she has found being in the public eye.

Just out, Better the drug you know: commentary on Daughton 2020, Natural Experiment Concept to Accelerate the Re-purposing of Existing Therapeutics for Covid-19 https://www.sciencedirect.com/science/article/pii/S2590113320300110

Broadbent A. 2020. Better the drug you know: commentary on Daughton 2020, Natural Experiment Concept to Accelerate the Re-purposing of Existing Therapeutics for Covid-19. Global Epidemiology 2(10027):1-2. https://doi.org/10.1016/j.gloepi.2020.100027

This is a (positive) commentary on what I thought was a really useful idea for accelerating research into anti-COVID drugs, which I shared previously and which you can (and should) read here:

Daughton CG. 2020. Natural experiment concept to accelerate the re-purposing of existingtherapeutics for Covid-19. Global Epidemiology 2(100026):1–6.66. https://doi.org/10.1016/j.gloepi.2020.100026

And the author Christian Daughton posted a reply to my commentary here:

Daughton CG. 2020. Response to: Broadbent 2020, Better the drug you know: Commentary on “Daughton 2020, Natural experiment concept to accelerate the re-purposing of existing therapeutics for Covid-19”. Global Epidemiology 2(100028):1-2. https://doi.org/10.1016/j.gloepi.2020.100028

 

Impressive that the @FT is featuring a prominent discussion of Chesterton’s Fence

https://fs.blog/2020/03/chestertons-fence/

“There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.”

GK Chesterton 1929, The Thing

Great piece from @JonathanJFuller ‘What’s Missing in Pandemic Models: Philosophy is needed to put the science of COVID-19 in perspective.’ In @NautilusMag #epitwitter

http://nautil.us/issue/84/outbreak/whats-missing-in-pandemic-models

Jonathan Fuller writes: “In the COVID-19 pandemic, numerous models are being used to predict the future. But as helpful as they are, they cannot make sense of themselves. They rely on epidemiologists and other modelers to interpret them. Trouble is, making predictions in a pandemic is also a philosophical exercise. We need to think about hypothetical worlds, causation, evidence, and the relationship between models and reality.”

Read more…

Boston Review: ‘COVID-19 has revealed a contest between two competing philosophies of scientific knowledge. To manage the crisis, we must draw on both’ says @JonathanJFuller #epitwitter

http://bostonreview.net/science-nature/jonathan-fuller-models-v-evidence

‘How do the coronavirus models generating these hypothetical curves square with the evidence? What roles do models and evidence play in a pandemic? Answering these questions requires reconciling two competing philosophies in the science of COVID-19.’ Great piece which will still be interesting a week, month, year and decade from now, unusually at present.

The role of philosophers in the coronavirus pandemic

What is the point of philosophy? That’s a question many philosophers struggle with, not just because it is difficult to answer. That goes for many academic disciplines, including “hard” sciences and applied disciplines like economics. However, unlike physicists and economists, philosophers ought to be able to answer this question, in the perception of many. And many of us can’t, at least to our own satisfaction.

I’ve written some opinion pieces (1,2) and given some interviews during this period, and I know of a handful of other philosophers who have done so (like Benjamin Smart, Arthur Caplan, and Stefano Canali). However, I also know of philosophers who have expressed frustration at the “uselessness” of philosophy in times like these. At the same time, I’ve seen an opinion piece by a computer scientist, whose expert contribution is confined to the nature of exponential growth: something that all of us with a basic mathematical education have studied, and which anyone subject to a compound interest rate, for example through a mortgage, will have directly experienced.

Yet computer science hasn’t covered itself in glory in this epidemic. Machine learning publications claiming to be able to arrive at predictive models in a matter of weeks have been notably lacking in this episode, confirming, for me, the view that machine learning and epidemiology have yet to interact meaningfully. Why do computer scientists (only one, admittedly; most of them are surely more sensible) and philosophers have such different levels of confidence at pronouncing on matters beyond their expertise?

There are no experts on the COVID-19 pandemic

This pandemic is subject to nobody’s expertise. It’s a novel situation, and expertise is remarkably useless when things change, as economists discovered in 2008 and pollsters in 2016.

Of course, parts of the current situation fall within the domains of various experts. Infectious disease epidemiologists can predict its spread. But there is considerably more to this pandemic than predicting its spread. In particular, the prediction of the difference that interventions make requires a grasp of causal inference that is a distinct skill set from that of the prediction of a trend, as proponents of the potential outcomes approach have correctly pointed out. Likewise, the attribution, after the fact, of a certain outcome to an intervention only makes good sense when we know what course of action we are comparing that intervention with; and this may be underspecified, because the “would have died otherwise” trend is so hard to establish.

Non-infectious-disease epidemiologists may understand the conceptual framework, methodology, terminology and pitfalls of the current research on the pandemic, but they do not necessarily have better subject-specific expertise than many in public health, the medical field, or others with a grasp on epidemiological principles. Scientists from other disciplines may be worse than the layperson because, like the computer scientist just mentioned, they wrongly assume that their expertise is relevant, and in doing so either simplify the issue to a childish extent, or make pronouncements that are plain wrong. (Epidemiology is, in my view, widely under-respected by other scientists.)

Turning to economics and politics, economists can predict the outcome of a pandemic or of measures to control it only if they have input from infectious disease epidemiologists on the predictive claims whose impacts they are seeking to assess.

Moreover, the health impact of economic policies are well-studied by epidemiologists, and to some extent by health economists; but these are not typically knowledgeable about the epidemiology of infectious disease outbreaks of this nature.

Jobs for philosophers

In this situation, my opinion is that philosophers can contribute substantially. My own thinking has been around cost-benefit analysis of public health interventions, and especially the neglect of the health impact – especially in very different global locations – of boilerplate measures being recommended to combat the health impact of the virus. This is obviously a lacuna, and especially pressing for me as I sit writing this in my nice study in Johannesburg, where most people do not have a nice study. Africa is always flirting with famine (there are people who will regard this as an insult; it is not). Goldman Sachs is predicting a 24% decline in US GDP next quarter.

If this does not cost lives in Africa, that would be remarkable. It might even cost more lives than the virus would, in a region where only 3% are over 65 (and there’s no evidence that HIV status makes a difference to outcomes of COVID-19). South Africa is weeks into the epidemic and saw its first two deaths just today.

Yet the epidemiological community (at least on my Twitter feed) has entirely ignored either the consequences of interventions on health, merely pointing out that the virus will have its own economic impact even without interventions, which is like justifying the Bay of Pigs by pointing out that Castro would have killed people even without the attempted invasion. And context is nearly totally ignored. The discipline appears mostly to have fallen behind the view that the stronger the measure, the more laudable. Weirdly, those who usually press for more consideration of social angles seem no less in favour, despite the fact that they spend most of the rest of their time arguing that poverty is wrongly neglected as a cause of ill-health.

Do I sound disappointed in the science that I’m usually so enthusiastic about, and that shares with philosophy the critical study of the unknown? Here we have a virus that may well claim a larger death toll in richer countries with older populations, and a set of measures that are designed by and for those countries, and a total lack of consideration of local context. Isn’t this remarkable?

There is more to say, and many objections; I’ll write this up in an academically rigorous way as soon as I can. Meanwhile, I’ll continue to publish opinion pieces, where I think it’s useful. Right now, my point is that there’s a lot for philosophers to dissect here. I don’t mean in this particular problem, but in the pandemic as a whole. And the points don’t have to be rocket science. They can be as simple as recommending that a ban on sale of cigarettes be lifted.

What is required for us to be useful, however, is that we apply our critical thinking skills to the issue at hand. Falling in with common political groupings adds nothing unique and requires the suspension of the same critical faculties that we philosophers pride ourselves on in other contexts. This is a situation where nearly all the information on which decisions are being made is publicly available, where none of it is the exclusive preserve of a single discipline, and where fear clouds rational thought. Expert analyses of specific technical problems are also readily available. These are ideal conditions for someone trained to apply analytic skills in a relatively domain-free manner to contribute usefully.

Off the top of my head, here are a handful topic ideas:

  • How to circumscribe the consequences of COVID-19 that we are interested in when devising our measures of intervention (this is an ethical spin on the issue I’m interested in above)
  • The nature of good prediction (which I’ve worked on in the public health context – but there is so much more to say)
  • The epistemology of testimony, especially concerning expertise, in a context of minimal information (to get us past the “trust the scientists FFS” dogma – that’s an actual quote from Twitter)
  • The weighing of the rights of different groups, given the trade off between young and old deaths (COVID-19 kills almost no children, while they will die in droves in a famine)

One’s own expertise will suggest other topics, provided that the effort is to think critically rather than simply identify people with whom one agrees. I very much hope that we will not see a straightforward application of existing topics: inductive risk and coronavirus; definition of health and coronavirus; rights and coronavirus; etc. To be clear, I’m not saying that no treatment of coronavirus can mention inductive risk, definition of health, or rights; just that the treatment must start with Coronavirus. My motto in working on the philosophy of epidemiology is that my work is philosophical in character but epidemiological in subject: it is philosophical work about epidemiology. Where it suggests modifications to existing debates in philosophy, as does happen, that is great, but it’s not the purpose. The idea is to identify new problems, not to cast old ones in a new light. Perhaps there are no such things as new philosophical problems; but then again, perhaps it’s only by trying to identify new problems that we can cast new light on old ones.

Call to arms

The skill of philosophers, and the value in philosophy, does not lie in our knowledge of debates that we have had with each other. It lies in our ability to think fruitfully about the unfamiliar, the disturbing, the challenging, and even the abhorrent. The coronavirus pandemic is all these things. Let’s get stuck in.

JOB: Post Doc in Philosophy of Medicine and/or Epidemiology, Johannesburg

The African Centre for Epistemology and Philosophy of Science in the Department of Philosophy at the University of Johannesburg seeks applications for postdoctoral fellowships in the philosophy of medicine and/or the philosophy of epidemiology. The successful candidate(s) will be expected (a) to pursue his/her own course of research in these fields and (b) to work with Prof Alex Broadbent, Dr Ben Smart, Ms Zinhle Mncube, Mr Chad Harris, and the rest of the ACEPS team, especially in relation to the project Health and Medicine in Africa. Start date negotiable, stipend is ZAR 220 000 per annum and is tax free.

Deadline 31 October 2017

Application form: https://goo.gl/1DJkDg

Submission link: https://goo.gl/mVJq59

 

More about ACEPS:

https://www.uj.ac.za/faculties/humanities/aceps/

http://africancentre-epistemology-phil-sci.weebly.com/

 

More about UJ Department of Philosophy:

https://www.uj.ac.za/faculties/humanities/Department%20of%20Philosophy/

 

More about UJ:

https://www.uj.ac.za/

 

 

Call for Papers: Philosophy of Epidemiology, a Special Issue of Synthese

 

Guest Editors: Sean A. Valles (Michigan State University, USA) and Jonathan Kaplan (Oregon State University, USA)

Special Issue Description: Philosophy of epidemiology is a burgeoning subfield within the philosophy of science and medicine. This special issue will provide philosophy of epidemiology with a forum to develop this area and expand its boundaries. The guest editors seek both to help develop philosophy of epidemiology’s existing lines of research (e.g., models of causal analysis) and expand philosophy of epidemiology to include a broader community of contributors (e.g., philosophers of race) and a wider array of lines of research (e.g., concepts of epidemiological risk and human-ecosystem dynamics).

Appropriate topics for submission include, but are not limited to: the role(s) of values in epidemiology; the role(s) of formal models in epidemiology; concepts of risk in epidemiology; the relationship between philosophy of epidemiology and philosophy of ecology (and other branches of philosophy); the metaphysical and causal repercussions of epidemiological data on the environmental and social determinants of health.

For further information, please contact the guest editors:

  • valles@msu.edu
  • kaplanj@oregonstate.edu

Deadline for submissions is: October 9, 2017