A new study conducted by researchers from UCLA Health reveals that current methods of storing and tracking data on children receiving emergency care often fail to identify those at risk of self-harm. The study found that several machine learning models designed by the researchers were more effective at detecting children who may exhibit self-injurious thoughts or behaviors. This breakthrough comes at a time when mental health providers are striving to improve their ability to identify at-risk children and intervene earlier.
The study highlights the limitations of current risk-prediction models, which are primarily based on incomplete data due to health systems’ inadequate understanding of who is seeking care for self-injurious thoughts or behaviors. The most common methods used to flag at-risk patients, such as International Classification of Diseases (ICD-10) codes and chief complaints, often exclude children who may have underlying mental health issues, such as depression or anxiety, but have not explicitly reported self-harm.
To gain insight into the effectiveness of ICD codes and chief complaints, experts reviewed the clinical notes of 600 emergency department visits made by children aged 10 to 17. The analysis revealed that ICD codes missed 29% of children who sought emergency care for self-injurious thoughts or behaviors, while the chief complaint missed over half (54%) of these cases. Even when used together, these methods still failed to identify approximately 22% of at-risk children.
Moreover, screening methods based on ICD codes and chief complaints were found to be less effective in flagging male children, as well as preteens compared to teenagers. There were also indications that Black and Latino youth were disproportionately underrepresented in risk prediction models, raising concerns about the potential biases in these systems.
To address these limitations, researchers developed three different machine learning models, each aimed at improving the identification of children at risk of self-harm. The most comprehensive model incorporated 84 data points from a patient’s electronic record, including previous medical care, medications, demographic information, and socioeconomic factors. Another model utilized all diagnostic codes for mental health, while a third examined additional indicators such as medications and lab tests.
All three machine learning models outperformed the traditional methods of using ICD codes and chief complaints alone. Interestingly, no single machine learning model demonstrated significantly better performance than the others. This suggests that health systems can enhance their ability to flag at-risk patients without the need for overly complex models.
While it is acknowledged that these machine learning models may generate more false positives, the benefits of using sensitive screening tools outweigh the drawbacks. The ability to identify potential cases of self-harm prompts further scrutiny by medical records analysts, ensuring that crucial cases are not missed.
The lead author of the study, Dr. Juliet Edgcomb, emphasizes the importance of improving youth suicide risk prediction models, particularly for elementary-school age children, which have been largely overlooked.
The study, published in JMIR Mental Health on July 21, 2023, sheds light on the significant hidden risks of self-harm in children. It serves as a call to action for health systems to reevaluate their data storage and tracking methods, seeking more comprehensive approaches that leverage machine learning to improve the detection of at-risk children. By addressing these hidden risks, mental health providers can intervene earlier and prevent tragic outcomes.
