Are Your Criminal Justice Laws Working? Here’s How to Tell.

Are Your Criminal Justice Laws Working? Here's How to Tell.

It starts by involving researchers, data and evidence early in the legislative process.

The variables inherent in data-oriented research can muddy the waters, for example, when evaluating the effectiveness of police programs in low-income, high-crime areas. (MattGush/Getty Images)

How do lawmakers know if their criminal justice laws are working?

It can be a challenge, experts told a session at this year's NCSL Base Camp.

"We want to accurately assess either the cost-benefit of a program or just the overall impact on the outcomes that we're interested in," says Matthew Ross, associate professor of public policy and economics at Northeastern University. "The most important thing is, we want this to be a really rigorous, clean, concise, accurately estimated impact of the program."

But variables can muddy the waters when, for example, evaluating the effectiveness of police programs in low-income, high-crime areas.

"The poorest neighborhoods where there's a lot of crime often receive more police," he says. "So, are we really finding the effect of police on crime-or police selecting into more dangerous neighborhoods which have more crime? That makes it very challenging, particularly in the criminal justice and policing space."

Ross says causal research initiatives are best set up early. "If you're implementing a pilot program or phasing in a particular policy, it is actually oftentimes quite easy, if you have a researcher involved in the early stages, to set things up in a way that replicates a randomized control trial. That makes evaluating the outcomes and the cost-benefit at the end really pretty bulletproof. You're going to get an unbiased estimate if it's set up correctly."

Perspective From the Beehive State

"Our office is around to help organizations improve," says Kade Minchey, legislative auditor general with the Utah Legislature. "There's a policy made, or there's an intervention of some kind-how do we know that it had the right impact?"

Minchey cites a case study on the initial stages of Utah's implementation in 2020 of the federal Justice Reinvestment Initiative, a data-driven program to help states improve their criminal justice systems by reducing crime and incarceration rates. "Part of it was implemented, but not all of it," he says. "The question was: Is it working?"

His office encountered a roadblock. "This idea to have a data-driven criminal justice system-it just wasn't there. It was really hard for us to answer a lot of questions, to get the information. A lot of that was at the county level."

Some areas were more easily parsed. The initiative involved more parole and probation for nonviolent crimes instead of incarceration. "Utah did achieve its goal to reduce the prison population," Minchey says. "That was one of the goals. We wanted more prison space for violent offenders."

Minchey says his office also has analyzed behavioral health as it relates to the criminal justice system. One audit examined several questions regarding Utah's behavioral health workforce: "Do we have the proper workforce in the state of Utah to address the behavioral health needs? Where's the funding going? Where's all the money? How's it being delivered, and what's the oversight of this funding? Do we have the right metrics in place to measure if we're getting the most efficiency and effectiveness out of these funds?"

The primary finding: "It's fragmented," he says. To improve the state's behavioral health system, "There needs to be more coordination and more review, and that was our recommendation. We believe, as we improve this, we can also help improve the criminal justice system."

Strategies for Staffers

Carrington Skinner, program principal for NCSL's Center for Results-Driven Governing, says existing research and data can provide guidance, and there are opportunities for states to generate their own evidence and data.

"But there's often a disconnect between these two sides, between evidence, research and data on the one hand, and then legislative decision-making on the other," he says. "Our team focuses on this very issue about how to bring the best available data and evidence into this process, which we refer to as evidence-informed, or evidence-based, policymaking."

Strategies include new offices and positions, data sharing and expanded IT infrastructure. An example of data sharing is Indiana's Management Performance Hub, which helps the work to integrate, share and use data in the decision-making process. Washington state took a novel approach by enacting legislation to study the feasibility of providing state employees access to peer-reviewed scientific journals.

Performance metrics can also be useful in the legislative process, Skinner says. "The collection and reporting of performance information is done in virtually every state, while 36 states and Washington, D.C., have statutory requirements to do so. This information can also help decision-makers to understand how programs are working, especially against set goals or targets over an extended period of time."

The NCSL-supported National Legislative Program Evaluation Society offers an opportunity for staffers to network and share best practices.

"This evaluation work can look very different from one state to another or one project to another," Skinner says. "It requires lots of different resources and time, depending on whether states are conducting a randomized control trial or quasi-experimental study or some other kind of assessment of program implementation."

Eric Peterson is a Denver-based freelance writer.