Editor’s Note: Jeff Asher is co-founder of AH Datalytics. A nationally recognized data analyst with expertise in evaluating criminal justice data, he has worked as an analyst for the CIA, the US Department of Defense and the New Orleans Police Department. His work has appeared in FiveThirtyEight, the New York Times, Slate and other publications. Follow him on Twitter @crimealytics. The views expressed here are his own. Read more opinion on CNN.
Imagine a world where the World Series winner is not announced until nine months after it has taken place; a world where teams collect only the most rudimentary statistics, and then decide to report only a handful of them to the public.
It would be lunacy for a major sports league to collect and report data in this way, but it is essentially how Americans are supposed to respond to the largest one-year increase in murder ever recorded. Murder rose nearly 30% nationally in 2020, and the available data suggests a smaller percentage increase occurred last year. But what’s still missing from the picture is a national data infrastructure which could illuminate why things are getting better or worse.
Understanding why murder rose so much in 2020 – and why murder may be falling in some places but not others – is literally a life-or-death problem which demands better data delivered faster. Learning why murder becomes more or less prevalent could help officials make better decisions about deploying police, regulating guns and designing, financing and providing social services. Instead, we get mostly anecdotes and informed opinions when explaining why murder is going up or down.
For example, while the number of murders has increased nationwide, both Dallas and St. Louis saw declines in 2021 – by 13% and 25%, respectively – following large increases in both cities in 2020. Police departments in both cities pointed to smarter policing as the primary cause for the drop. Local community leaders and activists in St. Louis, by contrast, credited “a renewed focus at the grassroots level to address and combat the issue,” rather than policing.
Either explanation is certainly plausible, but neither argument has a clear evidentiary base supporting it yet. Murder often goes up or down in a city from one year to the next for complex reasons that lack obvious explanations. Sometimes murder goes up or down because of randomness in the share of shootings that proved to be fatal. Murder rose in both Atlanta and Hartford, Connecticut, for example, but the number of people who were shot in each city fell last year relative to 2020.
Many of the challenges to gaining better insight into murder trends stem from a national crime data reporting system which is imprecise and woefully slow. Through the FBI’s Uniform Crime Report (UCR) program, local law enforcement agencies report data to state agencies which report it up to the FBI. The FBI collects the data, ensures it has been reported properly, and reports it to the public by the end of September for the most recently completed year.
The FBI does not require law enforcement agencies to collect and report data specifically on shootings where a person is injured but not killed, despite a record 77% of murders having been committed with a firearm in 2020, according to the FBI. This information is critical for evaluating the degree to which policing strategies, community interventions, or just plain luck contributed to changes in gun violence Some cities do publicly report shooting data, though the lack of standardized data collection on shootings at the national level is a major hurdle to a more informed understanding of changing murder patterns in most cities.
That national crime data is reported in September of the following year also makes it too late to catch trends for the country as a whole as they happen. The FBI attempted to address this problem in 2020 by reporting national crime estimates quarterly, providing an important first glimpse at the breadth of 2020’s murder rise.
In 2021, however, the FBI has not published quarterly crime estimates because too few local agencies are regularly reporting data to federal law enforcement. The problem appears to stem from the FBI’s switch in 2021 to the National Incident Based Reporting System (NIBRS). On the surface, NIBRS would seem like the program we need to get access to better data, and with that a better understanding of violent crime patterns. It’s designed to provide more specificity about crimes and arrests reported by law enforcement compared to the legacy system (known as the Summary Reporting System). NIBRS provides more nuanced details of reported crimes, though unfortunately this does not include specifically counting shootings as such.
Agencies were required to submit data to the FBI only via NIBRS starting January 1, 2021, but many agencies have not yet transitioned to this new system. Nearly 40% of law enforcement agencies that reported data to the FBI in 2020 did so via the legacy system, and only 53% of agencies enrolled in UCR used NIBRS in 2020. No agencies in California and only a small minority of agencies in Illinois, New York, Maryland, and Pennsylvania reported data via NIBRS in 2020.
It’s not yet clear how many agencies submitted data to the FBI through NIBRS in 2021, but the FBI has set a threshold of 60% of eligible agencies for making quarterly crime estimates and it has so far fallen short of this threshold in each quarter of 2021. As a result, each quarterly data release of 2021 from the FBI has come with a frustrating disclaimer stating, “due to participation under the 60% threshold, data trends by region and aggregate population group will not be available.”
The available evidence points to a significant gap between the numbers of agencies that will report data in 2021 and the number of agencies that have reported data in previous years. Under 10,000 out of the roughly 16,000 local law enforcement agencies that submit crime data to the FBI each year reported data via NIBRS in 2020 leaving over 6,000 agencies were still using the legacy reporting system. Fewer than 1,500 local agencies switched to NIBRS from the legacy system in 2020, so there is a strong chance that there will be thousands of local agencies that previously reported crime data to the FBI but did not in 2021. Fewer agencies reporting data means that when the FBI makes its year end crime estimates, it will do so using several thousand fewer data points than in previous years which could decrease confidence in those estimates.
One possible workaround for this data desert would be for more law enforcement agencies to embrace 21st century data reporting systems. Many – though not all – large agencies collect shooting data, but only a handful of cities – like Cincinnati, Philadelphia, and Rochester, New York, – publish victim-level data in near real-time. Other agencies track it only internally or report aggregate shooting data, either of which could conceivably be reported publicly.
Get our free weekly newsletter
Dozens of cities publishing detailed data on gun violence in near real-time would overcome some of the deficiencies in the FBI’s data collection and reporting system. Interested citizens, government analysts and academic researchers would be able to identify emerging trends, evaluate community programs or changes in policing strategies, and compare gun violence patterns across numerous American cities.
Murder rose at a historic pace in 2020 and the available evidence points to a smaller increase in 2021. Where we go from here though is a mystery, and improved data collection and reporting practices could be an important tool for identifying what works and embracing proven strategies.
