This paper surveys six attempts to estimate the annual level of killings by the police in the United States by describing the three federal reporting programs and three crowdsourced websites’ efforts to estimate the volume of killings.

 I find that the actual totals of killings by police in 2014 and 2015 are around 1000—almost exactly double the officially reported numbers that have been circulating for decades. The creative analysis by Bureau of Justice Statistics consultants shows conclusively that the existing federal systems miss about half of police killings and confirms that crowdsourced estimates close to 1000 are correct.

 

TABLE OF CONTENTS

I. The Official Stories

The federal government collects a comprehensive account of what are called the Vital Statistics of all who reside in the nation.7 This official count of births and deaths is part of an international system of reporting and classifying Vital Statistics to learn about trends over time and country-to-country variations in health statistics.8 Specifically, the system of Vital Statistics tries to compile comprehensive data on the number of deaths, causes of death, and the demographic characteristics of those who die.9 County-level health departments provide Vital Statistics with data on deaths and births.10 The system is almost completely accurate in determining the number of Americans who die each year and good, if not perfect, in determining the immediate causes of death.11 The Vital Statistics therefore contain trustworthy counts on the number of citizens killed by various forms of weapons, including deaths caused by firearms. But can they also determine how many of the persons killed by gunfire were shot by police? The good news on this question is that there is a special category, which the system calls “legal interventions” for such cases.12 But for many years, the number of killings by police has been substantially underreported because county coroners did not identify many killings as caused by police and thus while the report of a death went into the system, it was not listed in the legal intervention category.13

The likely cause of the problem is innocent oversight, because police killings are a tiny category in total deaths (less than a tenth of one percent) and these cases have never been regarded as an extremely important aspect of the Vital Statistics reports.14 But the failure to notice and fix the undercount is also another robust indicator that police killings were not an important national statistic in the United States for a long time.

Whatever the reason for the undercount of legal intervention killings, there was no basis in the Vital Statistics to estimate the magnitude of uncounted killings and thus no basis for using the known volume of citizen deaths from legal interventions as a foundation for a guess about the true volume of such deaths. Studies by Colin Loftin and his associates demonstrated that the totals for legal intervention killings were consistently lower in the years 1976–1998 than the volume of killings reported by the FBI in its supplemental homicide reports.15 But while the number of killings in the FBI series was somewhat higher than the Vital Statistics, it was also an undercount, as the next subsection will demonstrate, and putting these two incomplete measures together could not provide a plausible projection of the true volume of killings by police. Rates of these two killings provide a minimum estimate of the true volume, but no clear indication of how much less than the true volume is being reported. Figure 1 shows the trends from the time series and illustrates again the maxim that “two wrongs don’t make a right.”

 

Figure 1. Annual Numbers of Justifiable Homicides Committed by Police Officers, United States, 1976–1998.16 [[{"fid":"521","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 1: Annual Numbers of Justifiable Homicides Committed by Police Officers, United States 1976--1998","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"6":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 1: Annual Numbers of Justifiable Homicides Committed by Police Officers, United States 1976--1998","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"title":"Figure 1: Annual Numbers of Justifiable Homicides Committed by Police Officers, United States 1976--1998","class":"media-element file-full","data-delta":"6"}}]]

Indeed, the Loftin group’s study demonstrates substantial gaps in the ages and demographic details of the victims reported in the two data sets. This is a clear demonstration that the true volume of killings could be substantially greater than either of the official reports because so many cases were apparently reported in one system but not the other.17

More recent information from the FBI and the Vital Statistics show a shift from the FBI total having the higher numbers. Figure 2 compares the fatalities reported by the two systems for the five years beginning in 2008.

Figure 2. Fatalities Reported by Vital Statistics and the FBI, 2008–2012.18

[[{"fid":"531","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 2: Fatalities Reported by Vital Statistics and the FBI, 2008--2012","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"7":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 2: Fatalities Reported by Vital Statistics and the FBI, 2008--2012","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"title":"Figure 2: Fatalities Reported by Vital Statistics and the FBI, 2008--2012","class":"media-element file-full","data-delta":"7"}}]]

The two systems report very similar numbers in 2008 through 2011, but then the Vital Statistics death count opens a substantial lead with twenty-three percent more deaths in 2011 and twenty-nine percent more deaths in 2012.19 The current advantage of the Vital Statistics count may be a result of an effort to create a national violent death tracking system that currently has sixteen state systems enrolled. This wide of a swing in reporting is yet more evidence that there has been substantial undercounting in the estimation of killings by the police in the United States.

A.    Supplemental Homicide Reports to the FBI

The second national reporting system that classifies killings by the police as “justifiable homicide by police” is part of a larger registry of supplemental homicide reports created by local police departments (and sheriffs) at the request of the Uniform Crime Reporting Program (UCRP) of the Federal Bureau of Investigation in 1976.20 This program has been collecting information on violent killings for four decades. This “supplemental” program for homicides differs from other data on crime sent in to the FBI by police departments, because data on other crimes are reported only as statistical summaries (e.g., forty-eight armed robberies in the first quarter of 2015), which the FBI then cumulates into a national statistical aggregate. By contrast, each violent killing reported to the SHR by an agency becomes an individual event narrated in a brief summary with the month’s other killings in the supplemental reporting system.21 Just as killings by the police are only a small part of a larger death statistics system in the Vital Statistics program (several hundred killings in a total population of millions of deaths), the justifiable homicides by police are only one type of the violent killings reported in the Supplemental Homicide Program (several hundred cases a year in a population of several thousand violent deaths). But while “legal intervention” deaths are a tiny fraction of total deaths in the Vital Statistics, the 400 or so police killings each year are a much larger fraction of the homicides in the supplemental reports, and the weapons, causes of death, and victim demography of the police killings are much more similar to the other deaths in the FBI series than are the “legal intervention” deaths in Vital Statistics when compared to the patterns, causes, and populations of greatest risk for nonviolent deaths.

So the FBI’s program has some advantages over the Vital Statistics in that hundreds of police killings are less likely to go unnoticed. But there are also profound limits in the way in which killings by the police are documented and reported. One major problem is that participation in the reporting program is voluntary, and some police agencies do not participate.22 With a loophole that large, it is rather surprising that the FBI program consistently reported a greater fraction of killings by police than the Vital Statistics program found for legal interventions in the 1980s and 1990s. We know that many law enforcement agencies do not send in reports,23 but there is no precise estimate of what proportion of killings are from cities and county law enforcement agencies who do not participate in the program.

The second major problem with the FBI SHR is that there is no auditing of the accuracy of the descriptions of these killing events. While there is some auditing of the statistics on what are called “index crimes” sent to the FBI’s Uniform Crime Statistics Program, there is no quality control auditing of the supplemental homicide descriptions, which the local police departments provide. This is a particular problem because the reporting police department has direct interests in how such cases get classified, and even a pecuniary interest in the legality of a particular killing.24 So police reports have incentives to find every killing by an officer to be justified.

The voluntary nature of the reporting system means that significant numbers of killings by police do not get included in the official FBI numbers. The absence of auditing means that agencies with clear pecuniary interests in justifying cases are the only source of information available to the reporting system.

B.  Arrest-Related Deaths in the Bureau of Justice Statistics

There is a third program of data collection and reporting in the federal government that is concerned with killings by police. The BJS is an office in the Department of Justice that gathers and analyzes information on the operation of criminal justice agencies and actors. The Bureau was given responsibility to gather data about deaths that happened while citizens were involved in custodial relationships with criminal justice agencies by Congress in 2000 when the Death in Custody Reporting Act25 was passed. The BJS created three separate data gathering programs under the authority of the Act. The first was a survey of deaths by persons in prisons in the United States.26 The second was a survey of deaths in jails and similar holding facilities.27 The third program, started in 2003, was called the Arrest-Related Deaths (ARD) Program.28 The BJS data is gathered from accounts of deaths that law enforcement agencies provide, but each state has a state-level program coordinator to engineer compliance by the reporting state and local agencies.29

The ARD Program seeks information on all deaths that happen when law enforcement officers are interacting with citizens, including encounters that do not involve arrests or criminal investigation.30 The term “arrest-related” was used to create continuity with the jail and prison death programs where custody is a necessary element of the event. For deaths related to police and policing, there is no such restriction.

The “arrest-related deaths” reported in the BJS program include many deaths where a police officer does not injure the decedent and in that sense did not cause the death—suicides, natural causes, and accidents also count as “arrest-related deaths,” while both the “legal intervention” deaths in the Vital Statistics and the “justifiable homicides” in the FBI Supplemental Homicides are restricted to deaths caused by police use of force.31

Table 1 shows the distribution of the arrest-related deaths covering 2003–2009 by the categories of causes used in BJS reporting.

 

Table 1. Percent Distribution of Causes of Death Reported by Agency for Arrest-Related Deaths, BJS
2003–2009.32

Law Enforcement Homicides:

60.9%

Suicide:

11.2%

Intoxication:

10.9%

Accident:

5.7%

Natural Causes:

5%

Unknown:

5.7%

 

The six out of ten “arrest-related deaths” that are classified as homicides by the police are the comparable category to the justified homicides in the UCR, and the legal interventions in the Vital Statistics.

Figure 3 shows the numbers of reported homicides by law enforcement generated by the Arrest-Related Deaths (ARD) Program for the seven consecutive years 2003–2009.

Figure 3. Number of Reported Homicides by Law Enforcement, Arrest-Related Deaths Program: BJS,
2003–2009.33

[[{"fid":"541","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 3: Number of Reported Homicides by Law Enforcement, Arrest-Related Deaths Program: BJS, 2003--2009","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"8":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 3: Number of Reported Homicides by Law Enforcement, Arrest-Related Deaths Program: BJS, 2003--2009","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"title":"Figure 3: Number of Reported Homicides by Law Enforcement, Arrest-Related Deaths Program: BJS, 2003--2009","class":"media-element file-full","data-delta":"8"}}]]

The volume of police homicides reported in the ARD program starts quite close to the volume of the SHR for the first three years of the BJS reports, and then trends modestly higher for the years after 2005.34 These arrest-related death reports came from the departments that experienced a death during the year, so that the factual details about how the death occurred come from the law enforcement agency. In this respect, the same problems of unaudited information from agencies with personal interests in the lawfulness of the killing that happens in the FBI program are found as well in the case history details of the Arrest-Related Deaths reports.

The agency collecting the ARD report did, however, make a more sustained effort than the FBI to remedy the undercount of arrest-related deaths when it resumed collecting data for 2011. When the data from the agencies’ survey was compared to the cases of killings by police submitted to the Supplemental Homicide programs, BJS estimated that the true population of police killings for 2011 was about 930 cases.35 Because all of the official estimates were much lower than this, the Bureau did not publish its 2011 survey number and has focused instead on designing a study of the volume of killings by police that will use “crowdsourced” reports of arrest-related deaths to create a multi-method estimate of total killings.36

The data that the ARD requests on the circumstances of deaths is more detailed than either the SHR or the “legal intervention” cases in the Vital Statistics. The 2013 ARD questionnaire has an incident report with details on the victims, the precipitating circumstances, and the use of weapons and injury to persons inflicted by the eventual decedent.37 While the weapons that caused death are requested, the level of force—particularly the number of shots fired—is not addressed in the survey nor is the number of officers present or firing weapons requested by the incident report. The name and therefore presumably the contact information of the person filling out the report are requested in the incident report so that some fact checking or auditing of the accuracy of the data is possible, either by the “state reporting coordinators” at the state level or by central staff at BJS or its agents. Whether auditing had been attempted is not known.

The research efforts to date of the BJS program have demonstrated that existing reporting systems undercount the actual total of killings by police by about half. The sources of these gaps have not been clearly identified, and the design of more accurate reporting systems is just beginning.

The BJS analysis has, in essence, deconstructed the current official sources of information on killings by police; what now remains is to design a comprehensive count of police killings and an accurate and detailed description of the circumstances of killings by police. One foundation for building such a system would be the current reporting programs in the federal government. A second source is crowdsourced reports on the Internet, as covered in the next section.

II. Crowdsourcing in the Age of the Internet

While the official statistics on killings by police depend on reporting by government agencies—either coroners or police—a second and somewhat independent source of information can be produced by surveying and analyzing reports of killings that are from media or word-of-mouth accounts and are collected at websites established for that purpose. Because any violent killing is important local news, very few killings by police officers go unmentioned in local media.38 Once specialized websites are established and used, unofficial aggregations can provide estimates of the volume of killings by police and accounts of the circumstances of police killings that do not depend on official agencies.

There are a number of websites that aggregate reports of killings by police. Wikipedia has one of the most venerable websites.39 Facebook now has a “Fatal Encounters” page and a number of less institutional collections exist with somewhat more dramatic titles (see, for example, “DeadSpin” and “KilledbyPolice.net”).40

These aggregations of hearsay accounts are, of course, far from perfect measures of the true volume of killings by police. A website might undercount the actual volume of police killings if some killings go unreported by media or undisclosed by police. This is probably not a common occurrence with killings by police but is probably much more common when police inflict non-fatal wounds. A second problem with decentralized aggregation of media reports is the possibility of double counting. The same event might be given different data and victim names and counted twice in an estimate of total volume.

There are also manifold sources of factual errors in crowdsourced reports. The media or the police might get their facts wrong. Or a media report may simply reproduce a police account of the facts that is inaccurate. Some media reports may trust accounts given by critics of the police account without justification. There may even be situations where nonfatal injuries get reported as fatalities. Although some of the problems of factual accuracy in such reports cannot be corrected, many problems can be discovered and corrected through careful analysis of aggregated reports. If the name, date, and place identifications are accurate, a careful observer can eliminate double-counting.

Such careful analysis of crowdsourced accounts of killings by police can improve knowledge of police killings in three ways. First, the media accounts can provide a more accurate estimate of the volume of killings than any of the existing official aggregations. While official numbers from the FBI and Vital Statistics cluster under 500 per year, the crowdsourced estimate total rates are roughly double the official totals, and the BJS-sponsored “Monte Carlo” analysis of the official estimates suggest that the larger number is more likely accurate.41

Because the cases reported in the crowdsourced aggregations are a much larger fraction of the true number of police killings, these larger groups of cases are also a better sample of events to find out the demographic profile of victims and the type of incidents that lead to police killings. If the official statistics are based on only fifty percent samples of the true volume of cases, they may very well be a biased sample of victims and situations.

There is one further advantage to accounts from the media on killings by police. Official reports filled out by police will generally report only the police department’s version of events. Media accounts may provide indications of conflicting versions of the events leading up to a killing. Even when the conflict cannot be resolved by further analysis, knowing how many conflicts exist and what issues are the subjects of the conflict can be important elements of the story.

While reports of killings by police have been gathered in sites like Wikipedia for some time, the analysis of case descriptions to eliminate double-counting and exclude cases that should not be classified as killings by police is a relatively recent phenomenon. The data-analysis website FiveThirtyEight.com compiled a count of police killings for the calendar year 2014, estimating a total of 1100 different officer-caused deaths.42 The estimate was based on a reduction of seven percent from the total cases counted.43 This adjustment is based on the percentage of error discovered when two analysts made a detailed investigation of a ten percent sample of the cases in the 2014 crowdsourced entries.44 The volume of police killings FiveThirtyEight.com estimated for 2014 was 1100—more than twice the volume of annual cases for 2011 and 2012 in the Supplemental Homicide program and in the “legal intervention” estimates in the Vital Statistics.45

Two newspapers provided detailed accounts of each case of police killings they found on the web with credible factual details. Both The Washington Post and The Guardian have tried to capture all cases that meet their criteria for all of calendar 2015. The Washington Post gathers data on all police killings by shootings.46 The Guardian included all killings by police by all weapons and also includes deaths caused by Tasers and other mechanisms not typically regarded as lethal weapons.47 The Post’s “shootings only” deaths totaled 990 for 2015.48 The disadvantage of the “shootings only” restriction is that a number of deaths from police use of force may be omitted. The broader inclusions of all deaths caused by police may mix different types of attacks together in potentially misleading ways. It is not likely that police using Tasers intend to kill or realize that death might result. That said, the broad inclusion in The Guardian data set is probably the better strategy, because cases with usually nonlethal effects can be separately analyzed or excluded for some purposes. The cases that never enter the Post’s pool of killings are not available for later analyses. Figure 4 compares estimated annual totals from the three crowdsourced surveys now available for 2014 or 2015.

Figure 4. Estimates of Annual Killings, Three Crowdsourced Analyses versus Recent Official Reports.49

 

[[{"fid":"551","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 4: Estimates of Annual Killings, Three Crowdsourced Analyses versus Recent Official Reports","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"9":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 4: Estimates of Annual Killings, Three Crowdsourced Analyses versus Recent Official Reports","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"title":"Figure 4: Estimates of Annual Killings, Three Crowdsourced Analyses versus Recent Official Reports","class":"media-element file-full","data-delta":"9"}}]]

 

While the definitions and periods covered vary, all three analyses produce estimates of the same magnitude, at least double the levels reported in the SHR and Vital Statistics categories, as well as the most recently published ARD estimates for 2009. While the serious analysis of crowdsourced data is only beginning (like all manifestations of the public importance of the topic in 2014 and 2015), it has already changed the best estimate of the size of the phenomenon in the United States and questioned the validity of every official measure of police killings in national statistics. The next section discusses the state of knowledge of police killings in the United States in 2015 and the central puzzles about such killings that must be resolved in the near future.

III. Known and Unknown Dimension of Police Killings—A 2015 Analysis

Part I of this article did not mention the most important analysis conducted by a federal agency: the analysis and estimates generated by the BJS’s 2011 survey of their earlier reports of police killings.50 They decided not to publish their results.51 The BJS program had reported police killings over a seven-year period from 2003 to 2009.52 One reason this program did not continue on a routine basis was that the legislation and appropriation that had created the entire deaths in custody program in 2000 was not reauthorized when the initial authorization expired in 2006. So the BJS data collection was an orphan program that was continued at lower visibility and effort after 2006.

But the BJS did conduct a survey of 2011 and it also commissioned a study by the Research Triangle Institute (RTI), its contractor in the ARD program, to compare the coverage of cases that had been reported by the ARD program and those reported by the Supplementary Homicide Program for the years 2003–2009 and 2011.53 While the two programs reported approximately the same volume of killings for most years (around 400), an attempt to match the cases without unique identifiers suggested that the cases reported by the FBI only matched individuals reported in cases in the ARD survey about half the time.54 Just as many of the FBI’s cases were not reported in the ARD report for the same year. The capacity of the FBI sample to report all the cases found in the ARD was even worse. More than half the cases reported by the ARD were not reported in the SHR for the same year. Figure 5, taken from the BJS technical report, shows the estimated overlap of the two federal statistical programs.

Figure 5. Estimated Proportions of Law Enforcement Homicide Universe Covered by Each Source, with No Agency Adjustment, 2003–2009 and 2011.55

[[{"fid":"561","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 5: Estimated Proportions of Law Enforcement Homicide Universe Covered by Each Source, with No Agency Adjustment, 2003--2009 and 2011","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"10":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":"Figure 5: Estimated Proportions of Law Enforcement Homicide Universe Covered by Each Source, with No Agency Adjustment, 2003--2009 and 2011","field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"title":"Figure 5: Estimated Proportions of Law Enforcement Homicide Universe Covered by Each Source, with No Agency Adjustment, 2003--2009 and 2011","class":"media-element file-full","data-delta":"10"}}]]

The senior authors of the analysis that was published describe the logic of their methodology as follows:

BJS conducted a capture/recapture-based analysis to estimate the number of law enforcement homicides in the United States and the extent to which the ARD program and the SHR captured those homicides. Capture-recapture analysis uses the overlap between two or more lists to estimate the total size of the population from which the lists were taken. Simply stated, as the amount of overlap increases, the estimated underlying population size converges on the size of the lists.56 A number of analogies can be used to illustrate this. Consider two people randomly and independently selecting objects from a container, for example. When the first person selects his objects, he marks them and returns them to the container. The second person then makes her selection, taking note of how many of her selections have been marked by the first person. If the first person selected and marked 10 objects and the second person also draws 10, with 9 being marked by the first person, we can see that the population of objects must be larger than 10, as they have observed 11 between them. Since the overlap between the two draws was very high, it is also intuitive to think that both people drew nearly the entire population (i.e., the population size is fairly close to the size of [ ] each independent lists or draw). Conversely, if the second person draws 10 [ ] objects and only 1 of those has been marked by the first person, it is likely that neither individual observed close to the full population of objects.

While law enforcement homicides are believed to be a relatively rare event, there are more than 18,000 law enforcement agencies in the United States, and approximately 1,200 of those agencies reported a law enforcement homicide to the ARD program during the years studied (2003-2009 and 2011) and less than 1,000 reported to the SHR. The number of law enforcement homicides in the non-reporting agencies is unknown, so BJS conducted two sets of capture-recapture analyses to account for this missing information. Together, these two approaches provide a range for the best and worst case scenarios for under-reporting in the ARD.57

The fact that thousands of agencies did not report to either the ADR or SHR program made any single estimate of the actual volume of killings each year by police depend on whether and to what extent the analysis assumed that these uncounted agencies had killings during the period under study. Rather than a single guess on this issue, the analysts prepared two separate estimates.

For their first analysis, the authors assumed that the agencies that reported to either program accounted for all the police killings in the eight years (2003–2009 and 2011) when data from both programs was available. On this assumption, they estimate that a total of 7427 killings actually occurred during the period, an average of 928 each year when the aggregate is divided by the eight years covered.58 If, however, there were killings by officers as well in the agencies that did not report to either program, then the eight-year estimate of total killings is adjusted upward to 9937 killings, an annual total of 1242.59 Without better data on the missing agencies, it is prudent to estimate 928 deaths per year as a lower bound and 1242 as an upper bound of the mean annual average of deaths for the eight years.

The RTI analysis has two important impacts on knowledge about policing killings in the United States. First, the lower bound of plausible estimates (928) rather clearly shows that all the numbers from official sources for the years studied should be discounted.60 The analysis impeaches the validity of all the annual volume reports of all the federal reporting programs, not least the ARD’s arrest-related deaths totals that were published for 2003–2009. Instead, the minimum average annual volume that the analysis produced—928—is in fact the best current minimum estimate of volume of police killings, which should be based on the two federal reporting programs that were used to produce it. The second clear result from the RTI analysis is that it independently establishes an annual deaths estimate rather close to those being generated by the crowdsourced data sets analyzed by the media efforts. By confirming the population parameters of the crowdsourced analyses, the RTI analysis also makes the case for using these Internet-based data sets to answer questions about the nature and circumstances of killings by police.

We will need all the data sets we can get because the proof of such partial coverage in the RTI analysis undermines the existing federal sources as information about the character of police killings, the victims, and the location. It was known that the SHR did not include Florida,61 New York City, and other departments.62 But the RTI analysis suggests that fewer than half of all police killings are covered—and that is much more of a gap than the known non-inclusions would predict. We do not know why the SHR reports are so incomplete, and the almost as great undercount in the arrest-related death statistics is even more of a mystery.

So each of the national statistical systems reports no more than half of the true volume of cases. The samples that get reported to the FBI or BJS may be biased as well as incomplete. This mysterious loss of half of all cases only compounds the problems that arise because only the police version of most killings is the basis for information on the case. This double disadvantage—the lack of fact-checking on the one-sided accounts provided by police and the fact that half of all cases are missing from the surveys—caused me to rely on cases found by The Guardian and on facts available in news links on those cases when my book addressed the circumstances and justification for killings by police. The crowdsourced data sets contain a far larger proportion of police killings and there is no reason to suppose that the factual accounts in such records are worse than in the unaudited accounts the police provide to the FBI and police departments give to state coordinators in the ARD program.

For this reason, the plan of the Bureau of Justice Statistics to use accounts on the Internet to enrich efforts to build a foundation for better reporting is fully justified.63 But the capacity of the larger number of crowdsourced reports of killings to help us estimate the true volume of police killings and to determine the circumstances and the victims of such killings is prospective only. The true volume of killings in 2014 and 2015 is within reach; however, official estimates for earlier years were quite low. But the real trends over recent years cannot be measured with confidence. We can compare the demographic characteristic of decedents in the official statistics with those found in the news accounts. If there are no major discrepancies, some reassurance will be provided that the undercounting was not also a distorted sample of police killings. But any discrepancies between the official profile of such killings and the broader sample now in hand cannot be easily resolved for earlier periods. This absence of accurate data from the government sources on killings by police in the United State has been both a serious gap in American public health statistics, and also a symptom of the failure of government at all levels to acknowledge a serious national problem.

  • 7About the National Vital Statistics System, Ctrs. for Disease Control & Prevention, http://www.cdc.gov/nchs/nvss/about_nvss.htm (https://perma.cc/M6PP-FKZA).
  • 8Id.
  • 9Deaths Final Data for 2008, supra note 3, at 1.
  • 10Colin Loftin et al., Underreporting of Justifiable Homicides Committed by Police Officers in the United States, 1976–1998, 93 Am. J. Pub. Health 1117, 1117–18 (2003).
  • 11Id. at 1118.
  • 12Id. (Legal intervention deaths were defined as “injuries inflicted by the police or other law enforcement agents, including military on duty, in the course of arresting or attempting to arrest law breakers, suppressing disturbances, maintaining order, and other legal action.”).
  • 13Id. at 1120.
  • 14See Deaths Final Data for 2008, supra note 3, at 36 (estimating that legal interventions accounted for only 381 out of 2.4 million deaths); Loftin et al., supra note 10, at 1120 (claiming that it is unknown why certifiers fail to mention police involvement).
  • 15Loftin et al., supra note 10, at 1119.
  • 16Id.
  • 17Id. at 1120.
  • 18Deaths Final Data for 2008, supra note 3, at 36; Kenneth D. Kochanek et al., Ctrs. for Disease Control & Prevention, Deaths Final Data for 2009, 60 Nat’l Vital Statistics Reports 1, 39 (2011) [hereinafter Deaths Final Data for 2009]; Sherry L. Murphy et al., Ctrs. for Disease Control & Prevention, Deaths Final Data for 2010, 61 Nat’l Vital Statistics Reports 1, 41 (2013) [hereinafter Deaths Final Data for 2010]; Kenneth D. Kochanek, Sherry L. Murphy & Jiaquan Xu, Ctrs. for Disease Control & Prevention, Deaths Final Data for 2011, 63 Nat’l Vital Statistics Reports 1, 42 (2015) [hereinafter Deaths Final Data for 2011]; Sherry L. Murphy et al., Ctrs. for Disease Control & Prevention, Deaths Final Data for 2012, 63 Nat’l Vital Statistics Reports 1, 41 (2015) [hereinafter Deaths Final Data for 2012]; Fed. Bureau of Investigation, 2008–2012 Expanded Homicide Data Table 14 (2012), https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2012/crime-in-the-u.s.-2012/offenses-known-to-law-enforcement/expanded-homicide/expanded_homicide_data_table_14_justifiable_homicide_by_weapon_law_enforcement_2008-2012.xls (https://perma.cc/L2BP-3MCX).
  • 19Deaths Final Data for 2008, supra note 3, at 36; Deaths Final Data for 2009, supra note 18, at 39; Deaths Final Data for 2010, supra note 18, at 41; Deaths Final Data for 2011, supra note 18, at 42; Deaths Final Data for 2012, supra note 18, at 41; 2008–2012 Expanded Homicide Data Table 14, supra note 18.
  • 202008–2012 Expanded Homicide Data Table 14, supra note 18 (Justifiable homicide is defined as “[t]he killing of a felon by a law enforcement officer in the line of duty.”).
  • 21Loftin et al., supra note 10, at 1117.
  • 22Id. at 1117–18.
  • 23Id. at 1118.
  • 24Whether the department or the municipal government might have to compensate the relatives of victims depends on whether the person was wrongfully killed.
  • 25Death in Custody Reporting Act of 2000, Pub. L. No. 106-297, 114 Stat. 1045 (2000).
  • 26Duren Banks et al., Bureau of Just. Statistics, Arrest-Related Deaths Program Assessment Technical Report 1, 3 (2015).
  • 27Id.
  • 28Id.
  • 29Id.
  • 30Arrest-Related Deaths Topic, supra note 6.
  • 31Id.
  • 32Andrea M. Burch, Bureau of Just. Statistics, Arrest-Related Deaths 2003–2009 - Statistical Tables 4 (2011).
  • 33Id. at 4.
  • 34Burch, supra note 32, at 4; 2003–2007 Expanded Homicide Data Table 13, supra note 5; Fed. Bureau of Investigation, 2007–2011 Expanded Homicide Data Table 14 (2011), https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2011/crime-in-the-u.s.-2011/tables/expanded-homicide-data-table-14 (https://perma.cc/2HT8-RFEH).
  • 35Banks et al., supra note 26, at 13.
  • 36Part III discusses that research project after Part II describes the “crowdsourcing” collection of data on killings by police.
  • 37See Bureau of Just. Statistics, Arrest-Related Death Incident Report 2013, CJ-11A Question by Question Guide (2013).
  • 38See About the Counted, supra note 1.
  • 39See Franklin E. Zimring & Brittany Arsiniega, Trends in Killings of and by Police: A Preliminary Analysis, 13 Ohio St. J. Crim. L. 247, 257–58 (2015).
  • 40Fatal Encounters, Facebook (Feb. 5, 2016, 12:00 AM), https://www.facebook.com/fatal
    encounters.US/ (https://perma.cc/P4JF-G2Z2]; U.S. Police Shootings Data (Responses), DeadSpin (Feb. 5, 2016, 12:10 AM), https://docs.google.com/spreadsheets/d/1cEGQ3eAFKpFBVq1k2mZIy5mBPxC6nBTJHzuSWtZQSVw/edit#gid=1144428085 (https://perma-archives.org/warc/QXR5-TC6W]; Killed By Police 2015, KilledByPolice.net (Feb. 5, 2016, 12:05 AM), http://killedbypolice.net/ (https://perma.cc/R3S9-BYUG).
  • 41Banks et al., supra note 26, at 8, 13.
  • 42Reuben Fischer-Baum & Al Johri, Another (Much Higher) Count of Homicides by Police, FiveThirtyEight (Aug. 25, 2014), http://fivethirtyeight.com/datalab/another-much-higher-count-of-police-homicides/ (https://perma.cc/KW58-8XVT). The FiveThirtyEight.com estimate is based on their seven percent deduction for accidental deaths and includes other arrest-related deaths. It is best compared with The Guardian total of killings.
  • 43Id.
  • 44Id.
  • 45Compare id., with 2008–2012 Expanded Homicide Data Table 14, supra note 18, Deaths Final Data 2011, supra note 18, at 42, and Deaths Final Data 2012, supra note 18, at 41.
  • 46How The Washington Post is Examining Police Shootings in the U.S., Wash. Post (June 30, 2015), https://www.washingtonpost.com/national/how-the-washington-post-is-examining-police-shootings-in-the-us/2015/06/29/f42c10b2-151b-11e5-9518-f9e0a8959f32_story.html (https://perma.cc/RP7X-ACJ9).
  • 47See About the Counted, supra note 1.
  • 48Investigation: People Shot and Killed by Police This Year, Wash. Post (Feb. 4, 2016, 12:00 AM), https://www.washingtonpost.com/graphics/national/police-shootings/ (https://perma.cc/WJ6W
    -JZVM] [hereinafter Washington Post Investigation).
  • 49Deaths Final Data for 2012, supra note 18, at 41; 2008–2012 Expanded Homicide Data Table 14, supra note 18; Burch, supra note 32, at 4; Washington Post Investigation, supra note 48; The Counted: People Killed by Police in the United States in 2015, Guardian (Feb. 4, 2016, 12:10 AM), http://www.theguardian.com/us-news/ng-interactive/2015/jun/01/the-counted-police-killings-us-database (https://perma.cc/WU5F-2ZYE]; Reuben Fischer-Baum & Al Johri, Another (Much Higher) Count of Homicides by Police, supra note 42.
  • 50See Banks et al., supra note 26.
  • 51Bureau of Just. Statistics, Arrest-Related Deaths Topic, http://www.bjs.gov/index.cfm?ty=tp&tid=82 (http://perma.cc/8F6W-YXZN).
  • 52Id.
  • 53Banks et al., supra note 26, at 1–3.
  • 54Id. at 12.
  • 55Id. at 15.
  • 56“Capture-recapture analysis relies on a number of assumptions, including the ability to match cases across lists, that the lists are limited to cases that meet the definition of law enforcement homicides, and that inclusion on what list is independent from inclusion on the other. Many of these assumptions were met, but the nature of the ARD program and SHR necessitated the violation of other assumptions (including independence across lists, as some ARD SRCs relied on the same reporting mechanisms as those that inform the SHR). RTI implemented a number of adjustments to account for these violations[.]” Memorandum from Duren Banks & Lance Couzens, RTI Intl. to Franklin Zimring (Dec. 27, 2015) (on file with author) (citing id. at 8–12).
  • 57Id. at 1–2.
  • 58Banks et al., supra note 26, at 13.
  • 59Memorandum from Duren Banks & Lance Couzens to Franklin Zimring, supra note 56,
    at 2.
  • 60Banks et al., supra note 26, at 13.
  • 61Id. at 4.
  • 62Id. at 25 (“Rhode Island, Vermont, Delaware, and Wyoming did not report any law enforcement homicides to either system in 2011.”).
  • 63Tom McCarthy, US Government Database Hopes to Tell ‘Whole Story’ of Police Killings After Year of Guardian Count, Guardian (Dec. 13, 2015), http://www.theguardian.com/us-news/2015/dec/13/justice-department-database-police-killings-counted-statistics (https://perma.cc/7QFR-7FQF).