Steve Kirsch is an interesting character in this whole vaccine safety debate. On the one hand, he’s loud and vocal and willing to put his money where his mouth is. On the other, he likes to grab onto any shred of data and trumpet it as the biggest revelation since sliced bread and the information that will finally turn the tide. He also really likes publicity stunts.
I am very much not like him. However, I see the value in what he does, I just wish he was more careful and methodical.
That being said, he just released this substack which I’m sure many of you have seen
It took me awhile to understand what he was doing, so I thought I’d walk through it with you and then give my thoughts as to whether this works or not for what he’s suggesting. I will note that Dr. Fenton who he shared this idea with is a much smarter man than I am and has a much greater grasp of statistics than I do. I’m just a guy. He liked this approach and thought it was statistically sound.
Unfortunately, Steve starts his article with so much hyperbolic psych up wording, I almost gave up. I won’t do the same. Here is the idea:
Steve took a survey (without announcing why) for people to report deaths of people they knew, who died between January of 2021 and December of 2022, a 2 year period, regardless of vaccine status. If they KNEW the vaccine status and especially if they knew the last date of vaccination, then they were to report that information. In reality, this health data is available in most single payer systems (the UK ONS should definitely have it) and it could be applied to the entire population.
Why did Steve take this survey? Because he wanted to see, for any individual to die within those two years, how are those deaths distributed within the timeline. This is the crux but also the hardest part to understand. It’s also pretty clever because it overcomes several other data issues.
For each individual, he calculates a total possible days of life in the time period. For an unvaccinated person, this is always 2 years or 730 days. So the total possible days of life is the unvaccinated population * 730 days. So if he adds up all the number of days each unvaccinated person actually lived and divides by the total number of possible days lived (# people * 730). This gives a value, that in a perfect world should be 0.5. A normal distribution of deaths occurring mostly in the mid range with a few both very early and very late.
For the vaccinated, he starts the “days of life” counter at the date of the last vaccine dose. So for someone who received their last dose on September 1st of 2022, they would have roughly 90 days of life max available to them in the time period. Someone who had their last dose in February of 2021 would have nearly the entire time period. So for the vaccinated population, he adds up the possible days per person, adds up the ACTUAL days per person and does the same calculation. Actual Days/Possible Days. Again, this number should average out to roughly 0.5. He does break this out by age brackets, which makes sense to do to find any age specific trends, but it isn’t strictly necessary.
So why is this clever? First it’s clever because these dates are objective. A person died on X date and was or was not vaccinated, and if they were, the date of last vaccine was Y date. Selection bias doesn’t appear to apply. Even if everyone who reported results ONLY reported vaccinated deaths, the vaccinated deaths should still average to ~0.5 if it’s a safe intervention. The data isn’t being taken in relation to the other population, it’s being take in relation to itself.
Second it’s clever because the data is self normalizing. Yes, the vaccinated will have fewer days in their total calendar, but since deaths are normalized by maximum possible days and the last vaccine dose is used to start the clock, even someone who is vaccinated near the very end of the time period is (theoretically) just as likely to die on any of those remaining days. If they regularly die earlier, then it indicates at a minimum that vaccination is pulling deaths forward, even if the numbers of total deaths were the same.
Steve actually spends some time going through various biases that could still be in the data (like recall bias - the idea that more recent deaths were more likely to be the ones reported and they are closer to the end of the time period, so we would expect them to push averages higher rather than lower.
What did he find? He found that vaccination had an average of 0.31 and unvaccinated had an average of 0.58. The vaccinated value was quite consistent (when he had 10 or more records) but the unvaccinated value was actually pretty variable ranging from 0.52-0.68 which is quite a range and most likely does indicate a definite bias as being unvaccinated should not have an impact on when you die in a positive way in THIS test.
Okay, so is this a bullet-proof result? I’m not sure. I’ve come up with 2 reasons to believe it might be overstating the harms on the vaccine side. Steve has a “double your money” bet, but only if you can show that the data shows the opposite of what he says. I don’t think my objections would rise to that level, but here are my concerns.
Steve handwaves over the seasonal variation in expected mortality. He says it won’t have much impact, but I’m not so sure. Winter mortality ranges from 10-30% higher than the rest of the year, especially among older age cohorts. With many countries pushing booster campaigns in the fall, as well as the original vaccine campaigns being in the Winter in many countries (with the US contributing to 70% of his data), it is possible that his mortality measurement is a result of seasonal mortality and clustered vaccine dates. This issue does not exist in the unvaccinated cohort because there isn’t any possible seasonality to a null intervention. I think some clustering analysis needs to be done on the death dates (are the majority in winter? How do the vaccine dates cluster?) or possibly a cohort matching where an unvaccinated person is picked to count days starting at the same date as someone else’s last date of vaccination. With a large enough sample, one should be able to sample last vaccine doses across time to get a sample which is less prone to this potential bias.
The second issue is a kind of recall bias in the vaccinated cohort. Steve asked for a SINGLE death of the person you were closest to. But it is equally possible that people reported the vaccinated deaths they were most certain were related to the vaccine. Those deaths are likely to be temporally coherent to the known last dose. While that doesn’t stop the data from being objective - the date of vaccination and date of death a still true - I think the survey design potentially needs to be redesigned to avoid this confounder.
Steve has the data up an available, but doesn’t seem to have made any attempt to lock down the data they used for analysis, so I’m not certain that the copy I made is the same one he ran his tests against. That being said, I do plan on spending some time seeing if I can do more than come up with potential confounders and actually see if the confounding I suspect might be there, is there. (Well at least with regards to seasonality, I’m not sure how to deal with the second one).
So where does that leave us? It leaves us really wanting someone like the UK ONS or Denmark to run this analysis with the data they have. They will literally be doing some simple database work without any kind of recall or sampling bias and will be in the best spot to identify any clustering of the results.
In the meantime, I’m certainly open to suggestions as to why I’m wrong, right, or terminally confused.