Misleading COVID statistics, or why you should not eat data raw
I was looking into the origins of a strange claim by anti-vax conspiracy theorists, which is that those who are vaccinated are actually more likely to be sick with COVID than those who are unvaccinated.
Looking into those claims turns out to illustrate some of the difficulties we are facing in this pandemic.
I traced a possible source in the UK vaccine surveillance reports, reporting unadjusted rates of COVID infection, hospitalisation, and death in vaccinated and unvaccinated populations in the UK. The following table is taken from page 47 of their latest report (week 5, 2022).
It turns out that it is actually true that unadjusted rates of infection are higher among vaccinated people. The footnotes make it however quite clear that this is the result of differences in the two populations. The UKHSA wrote a clarifying blog post about this.
Quite conveniently, the anti-vax ignore the data that shows hospitalization and mortality is much lower for vaccinated people. As the UKHSA explain, this data is much less subject to bias; indeed, you can choose whether to get tested, but you do not choose whether to die. Death is the great equalizer…
This data and rumors emerging from it provide a real good example of the many difficulties in scientific communication in this crisis.
- One difficulty is statistical sophistication. People have to understand that raw data does not tell the whole story and has to be adjusted to take account of differences in the population. For example, unvaccinated people are less likely to go get tested if they are sick. But the raw data is easy to understand, and those who do not trust scientists will prefer to rely on it rather than on processed data. This is because they have difficulties understanding why and how it has been processed, and they do not trust the people who do the processing. This is a bit the same logic as preferring to cook oneself from raw ingredients when traveling in a strange country…
- A second difficulty is how policies affect beliefs which in turn affect behavior. A large part of the issue IMHO is that vaccinated people have been made to believe that they have nothing to fear anymore, and this belief is reinforced when governments let them access all social activities without restrictions, while keeping unvaccinated people away. Because the vaccine actually does not protect well against infection, especially against Omicron, giving those privileges to vaccinated people may turn this into a pandemic of the vaccinated, rather than the advertised pandemic of the unvaccinated. The issue however is that it is difficult to change course. As Macron famously said, restrictions are a way to encourage vaccination by making life difficult for the unvaccinated.
- A third difficulty is the government’s lack of trust in people‘s ability to decide for themselves whether to get vaccinated or not. On average, of course, it is a very bad decision not to get vaccinated, as the likelihood to get hospitalized and die is then much higher, as shown in the data. However, there can be good reasons to self-select into not being vaccinated. As the UKHSA indicates, the unvaccinated may have different occupations, family situations, or health issues, or they may unofficially know they have been infected recently, or they are more careful in their social interactions, etc. Many of those things cannot be readily assessed and taken into account with a broad-brush policy that says everyone has to get vaccinated, whether they want it or not.
- A fourth difficulty is conspiracy thinking, that is, preferring the worst possible, but simple, interpretation of the data rather than the more benign, but complicated explanation. In summary, what the anti-vax are saying is that the vaccine is making people sick. This corresponds to their view of the world, whereby the government and “those behind it” is out there to get them (No matter that even the most amoral State would actually want its population to be healthy, if only because healthy people are less costly and pay more taxes…) Unlike often said, conspiracy thinking is not necessarily complicated; you simply have to think some malign force is acting behind the scenes, and that you are cleverer than others, who for some reason are sheep. This explains why the anti-vax do not stop to wonder why the data we see here is given for all to see, when obviously it does not serve the interests of the “conspiracy”. No, they think they found a chink in the armor…
All of this from some numbers in a table. :) In summary, do not eat data raw, let statisticians cook it first. ;)
PS: French statistics show that the likelihood to be tested positive is the same, rather than lower, among the unvaccinated, see graphique 1 in this DREES report. France seems to do a better job testing the unvaccinated population.
Behavioral public policy 