The Pandemic Tracking Collective is working with The Rockefeller Foundation’s Pandemic Prevention Initiative to survey U.S. vaccine breakthrough infection (BI) data by U.S. states and territories with over 500,000 people. Our scorecard defines and measures best practices for complete and meaningful reporting of Covid-19 vaccine BI data. We assign states letter grades based on their alignment with these best practices. You can read more about why we selected these best practices in our announcement blog post and in this best practices document we sent to states.
This scorecard does not evaluate how well state health officials are responding to Covid-19 overall, but instead specifically reflects the completeness of each state’s publicly available vaccine breakthrough data reporting and presentation, with a focus on data transparency and interpretability. Additionally, many health departments that do not share BI data publicly face insurmountable technical obstacles to collecting data caused by decades of chronic underfunding. Bear this in mind for states receiving especially low grades.
Our scorecard focuses on the U.S. since states began reporting BI data when global reporting was still scarce; we hope to draw out lessons in best practices that could apply both in the U.S. and globally.
Scorecard and Criteria
Our state scoring system uses 29 criteria to calculate the grade for each state and territory we track. (We use “state” throughout to refer to states, territories, and the District of Columbia.)
Data Reporting
These criteria measure whether a state is publishing complete breakthrough data, as well as non breakthrough data during the same time period. You can see the exact metrics and evaluation criteria in the Data Dictionary tab of the spreadsheet above.
Cumulative counts
- Cumulative counts of breakthrough infection (BI) and non-breakthrough infection (non-BI) cases
- Cumulative counts of BI and non-BI hospitalizations
- Cumulative counts of BI and non-BI deaths
Percentages
- Cases as a percent of fully vaccinated individuals
- Hospitalizations as a percent of fully vaccinated individuals
- Deaths as a percent of fully vaccinated individuals
Recent Rates
- Recent rates of Covid-19 cases, hospitalizations and/or deaths by vaccination status
- Recent rate ratios of Covid-19 cases, hospitalizations and/or deaths by vaccination status
Time series
- Covid-19 outcomes over time as counts and, rates per capita by vaccination status
- Covid-19 cases by vaccination status
- Covid-19 hospitalizations by vaccination status
- Covid-19 deaths by vaccination status
- Population fully vaccinated and not fully vaccinated over time
Booster Status
- Covid-19 outcomes in boosted individuals
- Cumulative outcomes by vaccination status, including in boosted individuals
- Time series of outcomes among all vaccination statuses, including in boosted individuals
- Other breakthrough infection data by booster status
Data Presentation
These criteria evaluate how well states contextualize, format and publish their Covid-19 data.
Definitions
- BI Cases
- BI Hospitalizations
- BI Deaths
- Vaccination Status
- Rate methodology
Data Availability
- Data available to download in a machine readable format
- Data updated regularly (at least monthly)
Related Updates
Tracking State Reporting of Covid-19 Vaccine Breakthrough Infections
Today, we are releasing a reporting scorecard that evaluates U.S. state data reporting for Covid-19 vaccine breakthrough infections (BI) against best practices we have identified. We assign states letter grades based on their alignment with these best practices for reporting BI data and will update the grades monthly to monitor progress. The scorecard with states’ […]
More