Problem
The Problem:
The study aimed to address the accuracy and validity of pretrial risk assessment tools (PSC and VPRAI) used in Oregon counties to predict failure to appear (FTA) and re-arrest rates during pretrial release.
Impact on the system and/or public:
Pretrial detention practices directly impact community safety, fairness, and judicial efficiency. Validating these tools is critical to reduce racial/ethnic disparities and ensure just decision-making in release and detention.
Research Questions:
- How accurate are the pretrial risk tools in predicting failure to appear and re-arrest?
- How can these tools or protocols be improved for better predictive accuracy?
Method and Analysis
Program evaluated or gaps addressed:
The study focused on evaluating the predictive accuracy of PSC and VPRAI, addressing gaps in tool validation and calibration across jurisdictions.
Data and Sample Size:
- Data were compiled from Oregon Judicial Department (OJD) and Law Enforcement Data System (LEDS).
- Final sample: 24,438 cases from nine counties, representing diverse demographics.
Analysis Used:
- Sensitivity and specificity analysis using Area Under the Curve (AUC) metrics to determine predictive validity.
- Logistic regression models to control for external factors influencing outcomes.
- Evaluation of racial/ethnic disparities in tool performance.
Outcome
Key Findings:
- The PSC tools generally performed better than the non-validated VPRAI in predicting outcomes, particularly in identifying low-risk defendants.
- VPRAI showed weak to poor performance in Oregon counties, likely due to uncalibrated use.
- Variability in AUC values across counties highlighted the need for local calibration of risk tools.
- Discrepancies in predictive accuracy for racial/ethnic groups indicate potential bias in the tools.
Implications or Recommendations:
- Validation and Calibration: Pretrial tools should be validated and recalibrated for local populations to improve accuracy and fairness.
- Policy Alignment: Tools must align with judicial policies, including efforts to minimize racial and systemic biases.
- Improved Data Integration: Enhance data systems to reduce sample loss and improve analysis reliability.
- Periodic Review: Tools should be reviewed and revalidated regularly, as required by Oregon's Chief Justice Order.
- Training and Awareness: Judicial and law enforcement personnel should receive training on the limitations and appropriate use of risk tools..
This research underscores the importance of data-driven, equitable pretrial decision-making processes and highlights the critical role of ongoing validation and recalibration of risk assessment tools.