Where are the numbers? by Norman Fenton and Martin Neil

The very best of Cheap Trick.....

Our team first exposed the practice of deliberate miscategorisation of vaccine status on December 3rd, 2021, in a tweet that got 3 million impressions…

Read more

3 days ago · 298 likes · 118 comments · Martin Neil and Norman Fenton

‘

Our team first exposed the practice of deliberate miscategorisation of vaccine status on December 3rd, 2021, in a tweet that got 3 million impressions:

The tweet was based on our paper lodged on the ResearchGate archive, here (this and an earlier version got approximately half a million reads).

The ‘cheap trick’ is simple: categorise those who are vaccinated as unvaccinated up until some arbitrarily defined time period after vaccination takes place. The time period might be 7, 14 or 21 days. The supposed justification for this practice being that the benefits of the vaccine do not accrue until it has had time to ‘kick in’. And before it becomes effective on day seven, fourteen, or whatever, the recipient is considered to be unvaccinated.

At the time of our original work, we were not at all sure how globally widespread this selection bias was. Recently the Dark Horse podcast covered the issue and shortly after Bret Weinstein asked me if there was a comprehensive list of studies that had deliberately committed this cheap trick.

This article is our first attempt to provide such a list. It contains a mix of observational and other studies that have employed the cheap trick when assessing vaccine effectiveness for either infection, hospitalisation or death.

Note that as well as numerous and varying time periods that the trick might be employed, there are four ways in which this kind of selection bias might take place:

1. Miscategorised: Vaccinated are categorised as unvaccinated (or twice vaccinated categorised as single vaccinated Etc.)

2. Unverified: Those who are vaccinated but cannot be verified as such are categorised as unvaccinated.

3. Excluded: Those who are vaccinated but are infected before 14 days (or whatever) are allocated to neither the unvaccinated nor vaccinated categories but are instead simply removed from the analysis.

4. Undefined: The definitions of vaccinated and unvaccinated are left intentionally undefined.

This third version of the trick - exclusion - has tended to have been overlooked.

To compile this list, we looked through our own records and also consulted this Lancet meta-analysis on vaccine effectiveness.

The list is organised by authors and publication venue (Lancet, New England Journal of Medicine (NEJM), Journal of Vaccines Etc). Click on the authors name to get the web link to the paper.

Andrews et al UK study - NEJM

“Duration of Protection against Mild and Severe Disease by Covid-19 Vaccines”

Only "those at least 28 days after first dose were considered vaccinated:

Vaccine effectiveness was assessed for each vaccine separately and according to intervals after vaccination of at least 28 days after the first dose and at least 14 days after the second dose.

Baum et al Finland study - BMC Infection Diseases

“High vaccine effectiveness against severe COVID-19 in the elderly in Finland before and after the emergence of Omicron”

Definitions are vague but it appears anybody with first dose within 21 days was classified as unvaccinated.

Buchan et al Ontario study - JAMA

“Estimated Effectiveness of COVID-19 Vaccines Against Omicron or Delta Symptomatic Infection and Severe Outcomes”

We excluded …. individuals who had received only 1 dose or 4 doses of COVID-19 vaccine or who had received a second dose less than 7 days before being tested.

Corazo et al Quebec study - Lancet

“Protection against omicron (B.1.1.529) BA.2 reinfection conferred by primary omicron BA.1 or pre-omicron SARS-CoV-2 infection among health-care workers with and without mRNA vaccination: a test-negative case-control study”.

We defined mRNA vaccination as receipt of one dose at least 14 days before specimen collection or two or three doses at least 7 days before specimen collection, with the first and second doses administered at least 21 days apart. People who had a SARS-CoV-2 test within these intervals, who had an invalid vaccination date, or who received a non-mRNA vaccine were excluded.

Chung et al Ontario study - Open Forum Infections diseases

“Effectiveness of COVID-19 Vaccines Over Time Prior to Omicron Emergence in Ontario, Canada: Test-Negative Design Study”

We excluded tests from individuals who, on the testing date, had previously tested positive for SARS-CoV-2 or had received only a single vaccine dose, had received 2 doses but were <7 days from the second dose, had received any non–Health Canada–approved vaccines (including the Johnson & Johnson/Janssen Ad26.COV2.S vaccine, which was approved but rarely used in Ontario), or had received 3 doses.

Pálinkás and Sándor Hungary study - Vaccines

This was a study into covid-19 vaccine effectiveness in Hungary.

They admitted they simply excluded those people who were ‘partially’ vaccinated:

Partially vaccinated people who received the vaccine during the follow-up period but did not become protected during the study period (did not receive the booster or died before the 7th day after complete vaccination) were excluded from the investigation.

Dagan et al Israel study - Lancet

This study by Dagan et al from April 2021 in the NEJM looked at Covid-19 vaccine effectiveness in Israel.

In the paper the authors said:

“We considered three periods: days 14 through 20 after the first dose of vaccine, days 21 through 27 after the first dose (administration of the second dose was scheduled to occur on day 21 after the first dose), and day 7 after the second dose until the end of the follow-up….The period immediately after the first dose, when immunity is gradually building, was excluded in the main analyses”.

Mark Reeder in his paper on the Dagan study carefully documented this and other sources of selection bias.

Haas et al Israel study - Lancet

The Hass et al study, published in the Lancet has been the subject of a long-running saga here on our substack. It looked at effectiveness in those who were twice vaccinated (after 7 days had passed) versus the unvaccinated:

….COVID-19-related hospitalisation, severe or critical hospitalisation, and death was calculated on the basis of incidence rates in fully vaccinated individuals (defined as those for whom 7 days had passed since receiving the second dose of vaccine) compared with rates in unvaccinated individuals (who had not received any doses of the vaccine).

Ferdinands et al USA study - BMJ

“Waning of vaccine effectiveness against moderate and severe covid-19 among adults in the US from the VISION network: test negative, case-control study”.

We excluded individuals who… received only one dose of an mRNA vaccine less than 14 days before.

Lyngse et al Denmark Delta study - Nature Communications

This study from Denmark only considered ‘fully vaccinated’ against ‘unvaccinated’ with neither term defined anywhere in the paper. The authors also admit that that the ‘unvaccinated’ were likely to have been subject to more frequent PCR tests than the ‘fully vaccinated’.

Nordström et al Swedish study - Lancet

“Risk of infection, hospitalisation, and death up to 9 months after a second dose of COVID-19 vaccine: a retrospective, total population cohort study in Sweden”

Compares only those “Vaccinated” which is defined simply as ‘two doses’ against ‘unvaccinated’. No definition of either is given anywhere in the paper.

Rosenberg et al USA study - NEJM

Very high efficacy rates were reported in this paper but “vaccinated” here refers to:

fully vaccinated (≥14 days after receipt of the final dose)

while “unvaccinated” refers to anybody who

did not have a matching Covid-19 vaccine record according to the NYSIIS and CIR databases.

Petráš et al Czech study - Vaccines

“The Effectiveness of Post-Vaccination and Post-Infection Protection in the Hospital Staff of Three Prague Hospitals: A Cohort Study of 8-Month Follow-Up from the Start of the COVID-19 Vaccination Campaign (COVANESS)”

This paper is remarkable in that it considers three categories of vaccination status (unvaccinated, partially vaccinated, full vaccinated) without providing any definition of what any of these categories mean.

Thomas et al International RCT - NEJM

This was a placebo-controlled trial, but efficacy was only assessed based on cases at least 7 days after the second dose:

BNT162b2 efficacy against laboratory-confirmed Covid-19 with an onset of 7 days or more after the second dose was assessed.

Australia New South Wales - official study

“Vaccination among COVID-19 cases in the NSW Delta outbreak Reporting period: 16 June to 7 October 2021”

Unvaccinated cases were defined as either those with a) no vaccine record in NCIMS and unable to link to a record of any COVID-19 vaccination in AIR, or b) within 21 days of first dose.

Double vaccinated cases were defined as those as least 14 days after second dose and a minimum of 14 days between doses.

Robles et al Puerto Rico study - Lancet

This appallingly biased study was actually used to impose vaccine mandates in Puerto Rico. The authors’ state:

The decision to impose mandates, as well as current Department of Health recommendations related to boosters, were guided by the data and the effectiveness estimates presented here.

But the very high efficacy rates published were based on comparing ‘fully vaccinated’ with everybody else:

We denoted an individual as fully vaccinated two weeks after the date they received the final dose in the COVID-19 vaccine series.

So, anybody not ‘fully vaccinated’ was essentially classified as unvaccinated.

Swedish Ministry of Health

The Swedish doctors appeal uncovered the scandal that the Swedish ministry of health had miscategorised over 900 deaths:

The Swedish Public Health Agency has misled the public about the benefits of vaccination against Covid-19 by recording Covid-19 deaths amongst the vaccinated as unvaccinated. Over 900 deaths with Covid-19 have been misrepresented as occurring in the unvaccinated, thereby greatly distorting the effectiveness of the vaccination programme, for which the Swedish Public Health Agency is responsible.

USA CDC definition

The US CDC defines covid-19 cases by vaccine status using a two-week rule:

Vaccinated case with a primary series: SARS-CoV-2 RNA or antigen detected in a respiratory specimen collected ≥14 days after verifiably completing the primary series of an FDA-authorized or approved COVID-19 vaccine.

Vaccinated case with a monovalent booster: SARS-CoV-2 RNA or antigen detected in a respiratory specimen collected in a person verified to have received a primary series of an FDA-authorized/approved vaccine and ≥14 days after receipt of at least one additional dose of any monovalent FDA-authorized/approved COVID-19 vaccine ….

Vaccinated case with a bivalent booster: SARS-CoV-2 RNA or antigen detected in a respiratory specimen collected in a person verified to have received a primary series and ≥14 days after receipt of one additional dose of any bivalent FDA-authorized/approved COVID-19 vaccine …..

Partially vaccinated case: SARS-CoV-2 RNA or antigen detected in a respiratory specimen collected from a person who received at least one FDA-authorized or approved vaccine dose but did not complete a primary series ≥14 days before collection of a respiratory specimen with SARS-CoV-2 RNA or antigen detected.

The CDC classify the unvaccinated as:

Unvaccinated case: SARS-CoV-2 RNA or antigen detected in a respiratory specimen from a person who has not been verified to have received any COVID-19 vaccine doses before the specimen collection date.

The crucial thing about the unvaccinated category is that it hinges on verification. So, if someone cannot have their vaccination status verified, they will likely be listed as a an unvaccinated covid-19 case/death despite being vaccinated.

Also note that those who are categorised as ≥14 days might not be classed as unvaccinated (or as partially vaccinated if they have taken a second dose) but might instead simply be excluded from any statistics or efficiency calculations altogether.

Bermingham et al UK ONS study - official report

Estimates of the risk of hospital admission for coronavirus (COVID-19) and death involving COVID-19 by vaccination status, overall and by age group, using anonymised linked data from Census 2021:

We calculated vaccine effectiveness for different doses (first, second and third dose or booster) and time since dose, to observe how the effectiveness changes over time. The vaccination statuses used were:

• unvaccinated (those with no vaccination or who were vaccinated with a first dose less than 21 days ago)

Subscribe

UKHSA - official response

In response to a FOI request made by Joel Smalley the UK Health Security Agency (UKHSA) admitted to a similar practice when recording those who had received two vaccine doses as having received a single dose.

Individuals who have received 2 doses but less than 14 days before the specimen date of their positive COVID-19 test are included in the ‘Received 1 dose ≥ 21 days before specimen date’ category.

Pilishvili et al USA study - NEJM

This paper in the NEJM on the vaccination of US Healthcare personnel and employed the trick there:

“The effectiveness of a single vaccine dose was measured from 14 days after receipt of the first dose through 6 days after receipt of the second dose (partially vaccinated).”

Bermingham et al UK ONS study - preprint

The observational study by Bermingham et al looked at vaccine effectiveness against hospitalisation and death using the ONS data set. The pre-print is available on medrixv.

Whilst the paper defines exposure as first dose, second dose and third dose Covid-19 vaccination the unvaccinated category is conspicuous by its absence of any definition.

However, perhaps partly in reaction to our past criticisms the authors did perform a sensitivity analysis by omitting those vaccinated ‘within 21 days’ from the unvaccinated category:

The impact of including the first 21 days after a first dose of a vaccine in the ‘unvaccinated’ category was investigated by omitting these days from the ‘unvaccinated category’ (treating the 21 days after first dose as a separate vaccination status).

Clearly the default categorisation is that 21 days after first dose is counted as being unvaccinated and removing this group from the unvaccinated category is treated as a deviation from normal practice This is confirmed elsewhere by the ONS’s own admitted practice.

In the paper’s discussion section, they then make a Freudian slip by referring to the ‘unvaccinated vaccination’ category!

In addition, we also show in a sensitivity test that including the first 21 days after the first dose, when protection is rising, in the unvaccinated vaccination produces results consistent with the main analysis.

Angel et al Israel study - JAMA

The main results stated (which equated to a VE ratio of 83%) were based only on “fully vaccinated” against never vaccinated where:

Full vaccination was defined as more than 7 days after receipt of the second vaccine dose.

Arbel et al Israel study - NEJM

In September 2022 the NEJM published a study by Arbel et al, who investigated nirmatrelvir as a treatment for covid-19. Subsequently Hoeg et al submitted a letter to the NEJM editor complaining about confounding and healthy vaccine bias in the data.

However, unfortunately Hoeg et al failed to pick up on the obvious miscategorisation bias in the Arbel et al study. This was instead reported by Retsef Levi on twitter:

Subsequently the authors of the study responded to the letter from Hoeg et al, casually admitting the miscategorisation:

The authors and colleagues reply: With regard to a potential healthy vaccinee bias, all participants in our study started at an “unboosted” status, which was changed to a “boosted” status 7 days after vaccination….

Lyngse et al Denmark Omicron study - Nature Communications

This is similar to the other Lyngse et al paper except it focuses on the omicron variant.

We classified individuals into three groups: (i) unvaccinated; (ii) fully vaccinated; or (iii) booster vaccinated. The definition of fully vaccinated included individuals that had been infected more than 14 days previously, but was otherwise defined according to the vaccine used as follows: 7 days after second dose of Comirnaty (Pfizer/BioNTech); 15 days after second dose of Vaxzevria (AstraZeneca); 14 days after second dose of Spikevax (Moderna); 14 days after vaccination with Janssen (Johnson & Johnson); 14 days after the second dose for cross vaccinated. Booster-vaccinated was defined as 7 days following the booster vaccination.

Paternina-Caicedo et al Colombian study - Lancet

Paternina-Caicedo et al published this study about vaccine effectiveness in Colombia. Mark Mead sent us this communication about the study:

As an aside, I recently confirmed this misclassification practice with Dr. Angel Paternina-Caicedo, lead author of a cohort study in Colombia (n=720K). He openly admitted to me that this is what they were doing. In their study, the magnitude of the claimed Pfizer vaxx “death vaccine effectiveness" is so high 93% (men) and 94% (women), that if these values were true, the Pfizer jabbed countries (e.g., Israel) would have Covid deaths rates that should have stopped in their tracks - 93% reduction. So it simply cannot be true! It's like running a clinical trial where everyone is in the control group, then changing groups conditional on surviving 2 weeks after people taking a drug. I have many thoughts about this study but won’t go into them.’