SINGAPORE: The term “scientific evidence” has become part of our everyday vernacular since the beginning of the COVID-19 pandemic.
Policymakers rely on it when making any decision – whether to promote vaccination amongst older and high-risk individuals, tighten or ease COVID-19 safety measures or impose requirements on international travellers. The public also makes informed decisions based on it.
But this evidence often comes in the form of numbers, which can be confusing if we don’t fully understand the context and research methodologies. For example, how the data is collected, the sample size and representativeness will affect findings.
Failing to sift through the complexities in data presentation and interpretation may create confusion and give rise to potentially dangerous misinformation.
Up until recent format changes, the Ministry of Health (MOH)’s daily COVID-19 update seemed at first glance to indicate that about half the patients in serious condition – who required oxygen supplementation and ICU care – were fully vaccinated.
If we only look at these numbers, we might be tempted to conclude that the chance of serious illness for vaccinated individuals is about the same as getting a heads or tails in a coin flip.
Then it should not come as a surprise that some people are confused about the value of vaccination for avoiding serious COVID-19 complications.
UNDERSTANDING REAL RISK REQUIRES A FULLER PICTURE
However, to compare the real risk of a bad outcome between the vaccinated group and the unvaccinated, we must know how many people were exposed to risk in each group – in this case, this refers to how many people are vaccinated and how many are not.
A picture may help. What we observed in daily updates only captures the top half: The absolute number of people requiring ICU care or requiring oxygen supplementation.
The bottom half is the rest of the story: The number of people, vaccinated and unvaccinated, exposed to COVID-19 infection.
At the moment, 85 per cent of the local population have completed the full vaccination regimen. That means that for every 100 people, about 85 are vaccinated and 15 are not.
For illustrative purposes, imagine the four people in red experience bad outcomes: Two vaccinated and two unvaccinated. To be clear, the actual rate in Singapore is much lower.
So, if we only think about the absolute numbers in the top part of figure, we could fairly wonder if vaccination is such a great idea.
But from the entire picture, we see the true story: Vaccinated people with bad outcomes come from a big pool of people while unvaccinated people with bad outcomes from a small pool.
With the full picture, it becomes clear that vaccinated people are at much lower risk than unvaccinated people.
Already thinking whether you will vaccinate your younger children? Listen to experts and a parent discuss concerns over side effects and other risks on CNA’s Heart of the Matter:
BEWARE OF BIASES IN DATA INTERPRETATION
When judging risks, the confusion illustrated here is actually quite common and even has a name: Availability bias – the tendency of humans to make judgements based on information that comes readily to mind. And bad outcomes are the most memorable.
At the most extreme, where everyone is vaccinated, any bad outcomes would only occur amongst those who are vaccinated. That is, unless the vaccine is perfect, in which case no one would have a bad outcome and we wouldn’t be having this discussion.
Look a little further and we see the whole story is even more nuanced. Older people are more likely to be vaccinated than children but are at higher risk of having breakthrough COVID-19 infection with the possibility of a bad outcome, given their underlying conditions.
So, if we compare an older group of vaccinated people with a younger group of unvaccinated people, the vaccinated people will have worse outcomes because they are older, not because they are vaccinated. Figuring this out is why we have statisticians.
SIMPLER INDICATORS CAN HELP PUBLIC UNDERSTAND COMPLEX INFORMATION
Public confusion over the COVID-19 data shows that the way the data was presented was playing into this cognitive bias.
To its credit, MOH modified its communication by highlighting the rate of fully vaccinated and non-fully vaccinated cases in ICU per 100,000 population.
The daily update dated Nov 24 indicated that the proportion of fully vaccinated and non-fully vaccinated cases who are critically ill in the ICU are at 0.5 and 4.6 per 100,000 population respectively over the past seven days. Presented this way, the benefit is apparent — COVID-19 vaccines are highly effective in preventing severe illness.
More recently, MOH also introduced a number of simple indicators to provide greater clarity on the local COVID-19 situation and guide the public’s expectation of the changes in safety measures.
Among them, the weekly infection growth rate is a key parameter monitored by the Government to assess the speed of spread of the virus in the community.
After this number went below one – meaning the number of weekly cases is decreasing – and the hospital and intensive care unit situation appeared to have stabilised, the Government announced a relaxing of rules on Saturday (Nov 20).
It is now easier for the public to access and follow data that is more reflective of what is important for decision making.
Ample data is always welcome and should be read with care and necessary skills.
Even with these new indicators the virus may fool us and make a mess of the best plans. But at least we are fighting the virus rather than confusion.
Dr David Matchar is a Professor at the Health Services and Systems Research Programme at Duke-NUS Medical School, Singapore.