The Crime Reporting Dilemma

Many forces are converging to call upon the nation to develop innovative solutions for data driven policing where the collection and analysis of crime data serves strategic, tactical and operational purposes.

Why crime reporting is not working

 

There are no official national statistics to even estimate the number of incidents where force is used by police or describing the potential provocation for doing so.  Similarly, we have no current national data sufficient to explain why many cities are experiencing a spike in violent crime.  Without such data, available in a timely way,  the discussion of how to address these issues is at best, as former FBI Director Comey has put it, uninformed.  It is certainly difficult to imagine how to shape new policies, programs, methodologies, and training efforts with no data to measure improvements or formulate tactical strategies.

Consequently, the Office of Management and Budget and many other federal, state, and local entities have struggled for years to develop good metrics for managing performance of agencies and organizations.  In the law enforcement world, the public and policy makers hold crime numbers as the standard of measurement.  Chief executives are often blamed for increases in crime rates and sometimes credited for crime rate reductions.

The dilemma facing law enforcement and government in general is that the system in which we invested our efforts to collect crime data and generate statistics is simply not up to the task.  Since 1929 when it was formed, the Uniform Crime Reports (UCR) program has kept the same foundation.  Local law enforcement agencies are asked, on a voluntary basis, to submit a summary count of the Part I (or Index) crimes known to the police to their states where the data are aggregated and submitted to the Federal Bureau of Investigation (FBI) to generate a national crime statistics report which is published some months after the year ends.  Originally, the Crime Index consisted of 7 crimes—murder, rape, robbery, aggravated assault, burglary, larceny, and motor vehicle theft; in 1978 arson was added to the Crime index.  In its summary form, UCR only counts the most serious offense in an incident so that if a perpetrator enters a bar, shoots the bartender, commits 3 rapes, and steals everyone’s wallet, there is still only one crime.  This underreporting of offenses committed has been the practice for 85 years.

This oversimplified counting system does not help much with actual performance management.  The Summary UCR program does not account for many of the crimes that keep police executives up at night, such as cybercrime, gang violence or international narcotics trafficking.  Terrorism is not even mentioned.

The summary count also does not help us understand crime in its context, or allow us to gain insight into the trends in crime types in various victim categories such as children and the elderly or understand the extent of domestic violence and the conditions under which these crimes occur.   For that we need more detailed data and we also need to collect the data using standards for the definition of terms and the ways we count occurrences.

Law enforcement agencies also need to share detailed information on criminal incidents.  Police want to share data on crimes within their own department but also with neighboring jurisdictions to compare data on serial criminals and help solve cases.  The operational use of the crime data requires both an extensive amount of detail and a high level of data quality and accuracy; particularly as other agencies take action based on the data.  To share information on crime across state boundaries, it is very important that there be standards in place to ensure that the meaning of terms like the categories of crime are mutually understood and consistently applied.

Crime information is shared by local agencies with state repositories so that states may assess crime trends and problems requiring new programs or evaluate the results of innovative projects in reducing crime.  In order to do this, the data collected must have the kind of quality that statistical reliability and accuracy demands.  As the data are then aggregated across the states to present national estimates of crime trends, the statistical purposes can only be fulfilled with data meeting the quality requirements of valid statistical systems.

So the dilemma we still face is that either for sharing crime information with others for tactical purposes or for the generation of meaningful and useful statistics that will enable us to measure performance or to evaluate programs, it is essential that we collect detailed data about crimes that meets both statistical and operational standards for quality and accuracy.

 

What Needs to Change

 

While the strength of the UCR lies in its concept of uniformity, which originated from a penal-code based approach to classifying crimes into the 8 categories, and theoretically allowed for comparisons of changes in crime rates over time within and between places, the classification system has not been revised since its inception; the offense categories are broad enough to allow dissimilar events to be classified together, and new and important crime types do not appear in the Crime Index.

Major city and county law enforcement executives find this summary reporting not to be very helpful in developing strategies for responding to specific categories of crime or in making comparisons with other jurisdictions.  Because it is only a summary count, there are no data to help drill down on the parameters of various crime categories and victim groups to either determine what is happening to specific classes of victims or to evaluate risk mitigation programs.  Worse, by just using this group of 8 indicators, we have no national statistics on important issues such as intimate partner and domestic violence, human trafficking, cybercrime, or transnational organized crime just to name a few.  In addition, over time as state statutes changed, the UCR’s penal code based approach to classification approached its limits and the ostensibly homogeneous classes of crime became more heterogeneous.  For example, a home invasion could be classified as a robbery in one jurisdiction but as a burglary in another.  For crimes such as domestic violence, the assaultive nature of the crime can be ignored in favor of the domestic component and the crime would not be classified as a Part 1 crime; rather, it would fall into the simple assault category.

Given the complexity of crime events, a better method for counting and classifying is needed to meet the information demands of 21st Century Policing.  A move away from a strict penal code based classification to an attribute-based classification system is necessary to describe criminal events, victims, and offenders more fully.  In incidents involving multiple offenses, each offense needs to be counted separately, or at a minimum, the event needs to be described as one involving multiple offenses.

Additionally, there is a pressing need to improve the data quality and quantity in crime reporting as law enforcement and intelligence agencies continue to rely on the basic data about events to share knowledge about the potential of further crime or terrorist acts.  Building a stronger, standards based accumulation of data is essential to a more effective analysis of crime including terrorism, and to support the emerging concepts of proactive policing.  Gathering detailed data of high quality about suspicious behaviors in incidents that might lead to the prevention of crime or terrorism is an essential component of our crime reduction and counter-terrorism strategies.

It is clear that the requirements for data quality and completeness in a robust system of data collection and information sharing are essential for operational purposes of analysis and investigation, but also for the generation of statistics that provide insight on a local state, and national level about crimes including terrorism.  In effect, the rigor and discipline needed in data collection in order to support the generation of proper statistics is exactly what is needed to support operational decision making by creating data that is both complete and accurate.

Furthermore, without complete compilations of criminal incidents and offenses, and without a fuller description of the attributes of the criminal incidents, there is no satisfactory way to measure the performance or success of programs and projects designed to reduce or prevent the occurrence of these crimes.  For example, using the current UCR Summary Program, it is virtually impossible to assess the impact of funding to prevent crimes of violence against women.  As a result, neither funding agencies nor the public can evaluate expenditures aimed at preventing these kinds of crimes.

This shortcoming becomes very important as government agencies at all levels embrace evidence-based practices as a way of improving the operation of the criminal justice system.  Without disciplined and sufficient data collection, gathering evidence requires expensive and often duplicative data collection processes, and the kind of funding to measure the performance of all the promising strategies for system improvement is simply not available.

The federal government in its execution of the mission of  creating national statistics relies on the voluntary contribution of data from 18,000 law enforcement agencies and approximately 800,000 police officers and sheriff’s deputies.  While to the Federal agencies there is no apparent cost for the data collection effort, there most certainly is a substantial cost to the state and local agencies that are so engaged.  It is essential that we take whatever steps are necessary to use the incident data for multiple purposes and make it as accurate and complete as reasonably possible.  It is also imperative that we honor the trust that state and local agencies put in the federal government by the contribution of this data.

A Promising Solution

 

Part of the solution to this problem was pioneered in the 1980’s when the FBI introduced the National Incident Based Reporting System (NIBRS) as a way to generate meaningful and useful data and statistics about crimes known to the police.  NIBRS calls for law enforcement agencies to submit incident data to their state UCR Programs with 58 separate data elements on each incident including multiple offenses, victims and offenders. The resulting robust database enables analysis of crimes by type and victim group, and sets the stage for measuring the outcomes of programs, thereby addressing the performance management issues previously discussed.  There are 47 offense categories in NIBRS rather than the 8 offenses that comprise UCR Part I summary statistics.  In addition, the attributes of these offenses are recorded, so that it is possible to describe the victim-offender relationships, the injury and losses to victims, and the severity of incidents within crime classes.  For example, the degree of injury in an assault can be measured and if the perpetrator was an intimate partner, the assault can be so-classified.

However, NIBRS is still a voluntary program, and only about one third of all police agencies (about 6,600) report in NIBRS formats. Because most larger police departments are not included, the existing reporting only accounts for about 30% of the population of the U.S., and it is not possible to generate an estimate of national crime statistics from such a skewed sample.  In response to the pervasive need for better insight into crime and its prevention, there is growing widespread support for moving the nation to fully deploy NIBRS as the standard for reporting crime.  As an initial step in this direction, the Bureau of Justice Statistics (BJS) and the FBI have embarked on an effort to expand participation in NIBRS to add 400 strategically selected agencies to those already reporting NIBRS data. When combined with the existing NIBRS agencies, the dataset will be a sufficient sample to be nationally representative.  This sample-based approach to developing a national system of  incident-level crime statistics from local police agencies’ operational information systems is seen as an key intermediate step in obtaining complete NIBRS participation from the Nation’s roughly 18,000 law enforcement agencies.

BJS and the FBI call the program to expand NIBRS to the 400 sample agencies the National Crime Statistics-Exchange (or NCS-X) Program.  It presents a clear goal—the creation of a nationally representative system of incident-based crime statistics—that reflects the interests and priorities of both agencies.  In parallel with the NCS-X effort, BJS and the FBI have funded a panel at the National Academy of Sciences on modernizing crime statistics that is contributing to the basis for moving toward incident based reporting and further useful enhancements.

In support of this progress, the major law enforcement organizations, including the International Association of Chiefs of Police, the National Sheriff’s Association, the Major Cities Chiefs Association, and the Major County Sheriff’s Association have co-signed an endorsement of the FBI’s plan to move the country to full NIBRS reporting by January 1, 2021, subject to annual evaluations of progress. With these expressions of support, the Advisory Policy Board for the FBI’s Criminal Justice Information Systems Division has recommended to the FBI Director that the FBI move forward with this plan.  All of this support gives a mandate to the FBI to make this happen.

The need for information alone would not necessarily make NIBRS expansion feasible.  However, the current effort to expand NIBRS rests upon a second foundation: As technology has become more prevalent in law enforcement agencies, most medium and large agencies (and many smaller agencies) have implemented automated records management systems containing much of the data called for in the NIBRS standards.

As these systems have matured, it has become increasingly clear that the statistical objectives for crime reporting should be met by a system that is first designed to meet operational and investigative needs, generating data that can produce the statistical results as a by-product of operations.  In general, the current versions of commercial software available today for law enforcement records management systems support the collection and storage of most of the data required to meet the NIBRS standards.  As long as these systems are held to the data quality control standards called for by NIBRS, they can be the source of local data to be reported to the NIBRS system through the states.   Some of the existing automated systems in place have to be upgraded to capture the full data in accordance with the NIBRS data quality rules.  Further, the state reporting programs in states that have not yet adopted NIBRS have to be modified to implement a new reporting and quality control system based on full incident data.

To assist in increasing participation in NIBRS, BJS has made grants to states to either undertake a planning study to define how states will create or improve their NIBRS programs and reporting, or to request funds for implementation or improvement of existing NIBRS programs.  This represents the NCS-X program’s first step towards building a nationally representative system of incident-based crime statistics derived from the operational records management systems of local law enforcement agencies.

Scripting the future

 

The combination of the effort to move the nation to incident based reporting within 5 years and the advancing information technology being implemented in policing agencies will solve the dilemma we have described.

The ability to collect the kind of detailed data called for in the NIBRS program is essential to measuring the outcomes of innovative programs and projects. When this data is available at the local, state, and national levels, it will be possible to disaggregate the data in order to examine the efficacy of specific programs, and to generate evidence that can shape program policy. It is also essential that a modernized crime reporting system produce the evidence supporting the performance analysis of strategies in those areas not previously examined because of the lack of data collection, such as the critical crime categories of cybercrime and white-collar crime including financial crimes and corruption.

One of the most valuable initiatives that could speed development and enhancement of both NIBRS and related systems is the Standards Coordinating Council (SCC) that has been created to engage industry in the development, adoption, and implementation of standards to ensure sustainable interoperability.  This organization engages the standards development organizations and other advisory bodies at the table to severely cut the time to market for innovations in implementation of information sharing frameworks.  The full implementation of NIBRS and the further expansion of N-DEx are fully dependent on the active engagement of industry in this field, and the SCC is a vehicle to contribute to that end.

The very concept of a state or regional information sharing environment makes it clear that the precept of a distributed, decentralized but coordinated approach to information is built on local autonomy in defining crimes and responses to crime at the state and local level rather than Federal.  In support of this approach, NIBRS has been designed to support local control over data collection while providing an aggregation capability at the state level and from the states to the national collection of statistical information at the FBI, entirely consistent with the implementation of state information sharing environments.   In many ways, incident based reporting of crimes is the cornerstone of an information sharing environment.

The building of state based incident reporting systems on strong national standards and frameworks would enable these work efforts to generate a national crime statistics reporting system that far exceeds our current national capability to understand crime and establish proof of the efficacy of crime reduction methods.  By creating a full capacity to generate national crime statistics in a way that makes them useful at local and state levels as well, we will be producing tools for better management and an improved quality of justice in the nation.

 

5 thoughts on “The Crime Reporting Dilemma”

  1. Paul,
    You have done a nice job on a complex process. I am not sure that everyone will pick up on all of the complexities that you have brought up. I see attribute based crime classification, incident based reporting, increases in uniformity and quality control as a result of incident based reporting,and increases in the complexity of performance indicators. It is a useful announcement of what is going on and that we all should get behind. I hope the BJS/FBI folks are planning to keep constituents aware of the progress of NCS-X.

    Always amazed at your ability to describe complexity.

    Jim

  2. Paul
    I wish I could improve on what you write above but I cannot. It is so elegantly written and forces everyone to ask “well how can we not have been collecting such data much better?” For all the reasons you note and others we are so behind that I fear our ability to catch up in a way that all persons who feed into systems their data and the various constituencies who use them for different but equally relevant reasons will be a much longer way off than we need. Still perhaps baby steps are what we can ask for at this time.

    1. You may be right that baby steps are all we can take, but I think we must at the same time begin to shape a more transformative vision that may allow us to accelerate our path to better and more useful data.

      Paul

  3. Paul,

    You make some excellent points. We all assume that technology is the answer and forget that process analysis is also a factor in the equation. Crime in America is a complex intellectual and emotional subject. A national move to #NIBRS is definitely long overdue.

Leave a Reply

Your email address will not be published. Required fields are marked *