How Well Do Juvenile Risk Assessments Measure Factors to Target in Treatment? Examining Construct Validity
There has been a surge of interest in using one type of risk assessment instrument to tailor treatment to juveniles to reduce recidivism. Unlike prediction-oriented instruments, these reduction-oriented instruments explicitly measure variable risk factors as “needs” to be addressed in treatment. There is little evidence, however, that the instruments accurately measure specific risk factors. Based on a sample of 237 serious juvenile offenders (M age=18, SD=1.5), we tested whether California Youth Assessment Inventory (CA-YASI) scores validly assess the risk factors they purport to assess. Youth were assessed by practitioners with good interrater reliability on the CA-YASI, and by research staff on a battery of validated, multi-method criterion measures of target constructs. We meta-analytically tested whether each CA-YASI risk domain score (e.g., Attitudes) related more strongly to scores on convergent measures of theoretically similar constructs (e.g., criminal thinking styles) than to scores on discriminant measures of theoretically distinct constructs (e.g., intelligence, somatization, pubertal status). CA-YASI risk domain scores with the strongest validity support were those that assess criminal history. The only variable CA-YASI risk domain score that correlated more strongly with convergent (Zr=.35) than discriminant (Zr=.07) measures was Substance Use. There was little support for the construct validity of the remaining six variable CA-YASI risk domains — including those that ostensibly assess strong risk factors (e.g., “Attitudes,” “Social Influence”). Our findings emphasize the need to test the construct validity of reduction-oriented instruments — and refine instruments to precisely measure their targets so they can truly inform risk reduction.
Public Significance: This study suggests that scores on a juvenile risk assessment instrument provide little direction for targeting treatment to reduce young people’s risk of reoffending. With the exception of “Substance Abuse,” we found little evidence that scores on ostensibly treatment-relevant scales measure the risk factors they purport to measure. The instrument must be refined, to support valid interpretations about risk factors to target in treatment.