As Hurricane Maria crippled the power grid and left millions of Puerto Ricans without access to clean water in 2017, the cash-strapped island wasn’t prepared to capture the disaster’s catastrophic impact.
Initial reports concluded only 64 people had died in the category 4 storm — a number that seemed far too low to surviving witnesses. A year later, as the island’s only morgue was still filled with bodies, researchers estimated the true death toll at nearly 3,000 people.
Puerto Rico’s governor said mistakes were made and was forced to acknowledge a much greater loss of life. The world’s understanding of Hurricane Maria and its devastating consequences was permanently changed, thanks to a common calculation known as excess deaths.
Epidemiologists have used this calculation for more than a century to estimate the death toll of hurricanes, heat waves, virus outbreaks and other disasters. So it’s no surprise that when our newsroom began reporting on COVID-19 deaths in San Diego County, researchers quickly suggested we measure the excess deaths in the area since the pandemic began.
But the county pushed back when we shared what we had done.
RELATED: Pandemic’s Death Toll 66% Higher Than Official Counts, Inewsource Finds
At the onset of our work, we consulted with a handful of epidemiologists who talked us through the calculation: Take the number of deaths observed during the pandemic and subtract what you would have expected the loss of life to be in a normal year. That leaves you with an estimate of how many deaths were caused by the pandemic.
Our analysis showed an increase of more than 1,000 deaths during the first six months of the pandemic, which was roughly 66% higher than official COVID-19 death totals kept by the local public health office. Plus, the excess deaths were heavily concentrated among Hispanic people.
With the calculations complete, we sent our results to the county’s Health and Human Services Agency to hear their thoughts.
In an email, county spokesperson Sarah Sweeney called the analysis “premature” and cautioned against calculating excess deaths while the pandemic is ongoing.
When we spoke with a staff member from the county public health department, we were warned that these calculations should be saved for acute disasters like hurricanes and wildfires and to wait until after a crisis is over to analyze mortality data this way.
After the conversation, we called Andrew Noymer, an epidemiologist at UC Irvine, to evaluate the county’s objections.
“That is absolutely wrong,” he said.
Noymer, who studies pandemic diseases, has performed his own COVID-19 excess death estimates in Orange County.
He pointed out what we had heard from other experts: Excess deaths is a valuable measure of the true death toll of disasters — pandemics included — because it factors in the many people who public health departments fail to capture when a crisis hits.
In San Diego County, the public health office only includes residents with positive COVID-19 tests in the tally of pandemic deaths. But our analysis includes everyone who died as a result of the pandemic, directly or indirectly.
That means we captured people who contracted COVID-19 but were never tested. Our findings also include people who never got the virus but died from the pandemic anyway, like someone who died at home from a stroke because they were too scared of the coronavirus to go to a hospital.
“San Diegans deserve the whole picture, Californians deserve the whole picture, Americans deserve the whole picture,” Noymer said.
This is, after all, the 5th largest county in the U.S. with 3.4 million residents, a population larger than that of 20 states.
“I think the people have a right to know,” he added.
We talked through the county’s concerns with four epidemiologists, and they each refuted what the public health office told us.
“There’s no intellectual basis for them saying what they said,” said David Leon, an epidemiologist at the London School of Hygiene & Tropical Medicine.
How We Calculated Excess Deaths
Excess deaths is a calculation used by researchers to determine how many deaths have occurred beyond what is expected in normal circumstances.
To calculate excess deaths during the pandemic, inewsource gathered monthly mortality data from 2018 through 2020 from the county’s public health agency, which provided deaths of local residents, and from the state’s public health department, which provided all deaths in San Diego County regardless of residency.
For each month this year from March through August, we subtracted the average number of deaths that occurred in the same month from 2017 through 2019, giving us the excess deaths for that month. Then we added up the excess deaths from each month to get the total excess since the pandemic began.
The same method was used to calculate the number of excess deaths among county residents broken down by racial and ethnic group, cause of death, and age group.
The county did not provide resident-only data broken down by death location type (for example, whether someone died at home or the hospital), so we used the state’s data for these breakdowns, which includes non-county residents.
To measure whether the rise in deaths was a result of normal year-to-year variation, we calculated the standard deviation of the deaths that occurred each month from 2017 through 2019. The increase in deaths seen each month from April through August this year was more than two standard deviations above the mean, indicating the rise in deaths is highly unusual.
For comparison, we pulled the regularly updated list of official COVID-19 victims from the county’s public health office and broke down the data by month, racial and ethnic group, and age group. This list only includes county residents. We subtracted the number of people on the list from the number of excess deaths occurring during the same period to estimate how many people are not captured in the county’s COVID-19 death total.
September’s data wasn’t included in the analysis because the state hasn’t yet released it.
The 2020 data used in this analysis is provisional. The number of deaths on record for the first six months of the pandemic — and the number directly attributed to COVID-19 infections — will likely increase over time.
We consulted with eight epidemiologists and mortality researchers for this story and its methodology. Our estimates don’t take into account population changes, which can affect death counts, but experts said those changes are small in a four-year time span and unlikely to majorly affect the results.
In April, Leon and an international team of researchers published a paper on the importance of calculating excess deaths during the COVID-19 pandemic.
He said he’s never heard an epidemiologist express the concerns we heard from the county and told us that conducting these calculations in real-time — while the pandemic is ongoing — allows governments to make better public policy decisions.
After Hurricane Maria, having more accurate death totals led Congress to sign off on a $43 billion funding package to aid in the territory’s reconstruction. As for COVID-19, these numbers can help officials decide whether to impose tighter restrictions on a region or how much disaster relief money a community may need.
And while excess deaths estimates will become more accurate the longer into the pandemic we wait, the experts said it’s still important to make the public aware of the data now — something the Centers for Disease Control and Prevention is already doing at the state and national level.
“It’s a sensible thing to do,” Leon said.
County officials also told us they were concerned that the rise in deaths this year identified by our analysis is just part of normal year-to-year variation.
We took the advice of Cécile Viboud, a senior researcher at the National Institutes of Health, who has reported on excess mortality for 15 years. We calculated standard deviations in our data, which showed a less than 5% chance the rise in deaths was due to random variation.
When you find results that far away from the norm, Viboud said, you know “something really unusual is happening.”
We did take one piece of advice from the county: only including San Diego County residents in our estimates. The public health office said this would allow for a more accurate comparison to prior years, because an influx of COVID-19 hospital patients from nearby areas could skew our numbers.
Ultimately, experts told us, the calculations could be done either way, and the results only slightly changed when we excluded non-residents.
Dr. Steven Woolf, an epidemiologist at Virginia Commonwealth University, called the debate over our methodology “missing the forest through the trees.”
“The larger point is that the number of deaths we’re hearing about on the news is an undercount,” Woolf said. “And then the question becomes, what policy changes are important?”
Woolf recently co-authored a study that found more than 226,000 pandemic-related excess deaths through Aug. 1. About 67% of them had already been attributed to COVID-19.
A fellow researcher wrote into the journal calling the findings “sufficiently mortifying and motivating,” even knowing that the exact numbers would undoubtedly change over time.
The preliminary results, he said, already tell us what we need to know: “An intense, persistent, multipronged, and coherent response must be the order of the day and an urgent priority for the nation.”
