The detection of changes in mental states such as those caused by psychoactive drugs relies on cl... more The detection of changes in mental states such as those caused by psychoactive drugs relies on clinical assessments that are inherently subjective. Automated speech analysis may represent a novel method to detect objective markers, which could help improve the characterization of these mental states. In this study, we employed computer-extracted speech features from multiple domains (acoustic, semantic, and psycholinguistic) to assess mental states after controlled administration of 3,4methylenedioxymethamphetamine (MDMA) and intranasal oxytocin. The training/validation set comprised within-participants data from 31 healthy adults who, over four sessions, were administered MDMA (0.75, 1.5 mg/kg), oxytocin (20 IU), and placebo in randomized, double-blind fashion. Participants completed two 5-min speech tasks during peak drug effects. Analyses included group-level comparisons of drug conditions and estimation of classification at the individual level within this dataset and on two independent datasets. Promising classification results were obtained to detect drug conditions, achieving cross-validated accuracies of up to 87% in training/validation and 92% in the independent datasets, suggesting that the detected patterns of speech variability are associated with drug consumption. Specifically, we found that oxytocin seems to be mostly driven by changes in emotion and prosody, which are mainly captured by acoustic features. In contrast, mental states driven by MDMA consumption appear to manifest in multiple domains of speech. Furthermore, we find that the experimental task has an effect on the speech response within these mental states, which can be attributed to presence or absence of an interaction with another individual. These results represent a proof-of-concept application of the potential of speech to provide an objective measurement of mental states elicited during intoxication.
Preliminary Evaluation of Phenotypic Features from Clinical Interactions
The Journal of Pain, 2021
Clinical interactions are not commonly scored and most result in a clinical progress note. Intera... more Clinical interactions are not commonly scored and most result in a clinical progress note. Interactions quantified by instrument scores are liable to training biases, lack of granularity, and enforced phenotypic dimensionality. Measurements derived from speech and facial features have been previously demonstrated to correlate with clinical assessment. Here we evaluate face, gaze, head, and speech features recorded during routine clinical interactions for useful disease state phenotypic information. Clinical interactions were voluntarily recorded from eight inpatient participants over 48 sessions as they progressed from admission to discharge. We used a Zoom Q8 camera and two channel audio. Face action units, head pose, and gaze features were calculated using the OpenFace library1,. Acoustic, prosodic, and linguistic features were extracted using common software tools (e.g. Praat, Opensmile, GloVe, LIWC). We evaluated the results of four algorithms (Support vector, Lasso, Ridge, and Linear regression) to infer several Brief Psychiatric Rating Scale (BPRS) sub-scores using a 10-fold cross-validation approach. In addition, we visually explored the feature space for phenotypic subtypes and potential response to treatment. From face, gaze, and head movement features we predicted non-zero BPRS depressive mood scores with accuracy of .93 and inferred BPRS depressive mood, blunted affect and motor retardation subscores with Pearson coefficients of (.78, .82, and .76). Participant feature data plotted longitudinally exposes subtle changes in objective measures humans would find difficult to identify from memory. We demonstrate the range of objective digital features that can be generated in the normal course of a relatively brief, unstructured clinical interactions and correlate these objective features with subjective clinical ratings and patient trajectories. (DB) NIH T32 MH019961, (DB) R25 MH071584.
Analyzing progression of motor and speech impairment in ALS
2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2019
Amyotrophic lateral sclerosis (ALS) is a degenerative disease which causes death of neurons contr... more Amyotrophic lateral sclerosis (ALS) is a degenerative disease which causes death of neurons controlling voluntary muscles. It is currently assessed with subjective clinical measurements, but it would benefit from alternative surrogate biomarkers that can better estimate disease progression. This work analyzes speech and fine motor coordination of subjects recruited by the Answer ALS foundation using data from a mobile app. In addition, clinical variables such as speech, writing and total ALSFRS-R scores are also acquired along with forced and slow vital capacity. Cross-sectional and longitudinal analyses were performed using speech and fine motor features. Results show that both types of features are useful to infer clinical variables especially for males (R2=0.79 for ALSFRS-R total score), but their initial values are not helpful to predict speech and motor decline. However, we found that longitudinal progression for bulbar and spinal ALS onset are different and they can be identified with high accuracy by the extracted features.
Clinical utilization of automated image analysis software for improving retinal reader's performance
2016 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI), 2016
The incidence of cardiovascular disease (CVD) is on the rise and reported to be the world's l... more The incidence of cardiovascular disease (CVD) is on the rise and reported to be the world's leading cause of death. Retinal vascular abnormalities and lesions of hypertensive retinopathy (HR) have been shown to be highly predictive risk factors for CVD. However, the inter-reader agreement on detection of HR abnormalities by trained retinal readers is only fair to moderate, such as κ=0.56 for artery-venous nicking and κ=0.42 for arterial narrowing, indicating a significant inconsistency. We have developed a system for automated analysis of retinal photographs for HR abnormalities, which can assist a retinal reader in the grading process and provide additional insight into CVD risk. This includes a set of algorithms for retinal vessel network analysis and detection of HR abnormalities. Three retinal readers graded a set of 120 retinal images with and without the assistance of the system. Assistance resulted in an average 30% improvement in the reader's sensitivity to retinopathy detection, with a 32% reduction in average reading time. The inter-reader agreement using this system increased by an average of 54%, which indicates improvement in grading standardization. The system increases the efficiency of the grading process, and demonstrates increased reproducibility of the retinal grading that can standardize the diagnosis.
Vessel discoloration detection in malarial retinopathy
Medical Imaging 2016: Computer-Aided Diagnosis, 2016
Cerebral malaria (CM) is a life-threatening clinical syndrome associated with malarial infection.... more Cerebral malaria (CM) is a life-threatening clinical syndrome associated with malarial infection. It affects approximately 200 million people, mostly sub-Saharan African children under five years of age. Malarial retinopathy (MR) is a condition in which lesions such as whitening and vessel discoloration that are highly specific to CM appear in the retina. Other unrelated diseases can present with symptoms similar to CM, therefore the exact nature of the clinical symptoms must be ascertained in order to avoid misdiagnosis, which can lead to inappropriate treatment and, potentially, death. In this paper we outline the first system to detect the presence of discolored vessels associated with MR as a means to improve the CM diagnosis. We modified and improved our previous vessel segmentation algorithm by incorporating the ‘a’ channel of the CIELab color space and noise reduction. We then divided the segmented vasculature into vessel segments and extracted features at the wall and in the centerline of the segment. Finally, we used a regression classifier to sort the segments into discolored and not-discolored vessel classes. By counting the abnormal vessel segments in each image, we were able to divide the analyzed images into two groups: normal and presence of vessel discoloration due to MR. We achieved an accuracy of 85% with sensitivity of 94% and specificity of 67%. In clinical practice, this algorithm would be combined with other MR retinal pathology detection algorithms. Therefore, a high specificity can be achieved. By choosing a different operating point in the ROC curve, our system achieved sensitivity of 67% with specificity of 100%.
Background and Hypothesis Disturbances in self-experience are a central feature of schizophrenia ... more Background and Hypothesis Disturbances in self-experience are a central feature of schizophrenia and its study can enhance phenomenological understanding and inform mechanisms underlying clinical symptoms. Self-experience involves the sense of self-presence, of being the subject of one’s own experiences and agent of one’s own actions, and of being distinct from others. Self-experience is traditionally assessed by manual rating of interviews; however, natural language processing (NLP) offers automated approach that can augment manual ratings by rapid and reliable analysis of text. Study Design We elicited autobiographical narratives from 167 patients with schizophrenia or schizoaffective disorder (SZ) and 90 healthy controls (HC), amounting to 490 000 words and 26 000 sentences. We used NLP techniques to examine transcripts for language related to self-experience, machine learning to validate group differences in language, and canonical correlation analysis to examine the relationshi...
2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)
The diagnosis and treatment of psychiatric disorders depends on the analysis of behavior through ... more The diagnosis and treatment of psychiatric disorders depends on the analysis of behavior through language by a clinical specialist. This analysis is subjective in nature and could benefit from automated, objective acoustic and linguistic processing methods. This integrated approach would convey a richer representation of patient speech, particularly for expression of emotion. In this work, we explore the potential of acoustic and prosodic metrics to infer clinical variables and predict psychosis, a condition which produces measurable derailment and tangentiality in patient language. To that purpose, we analyzed the recordings of 32 young patients at high risk of developing clinical psychosis.
BACKGROUND In contrast to all other areas of medicine, psychiatry is still nearly entirely relian... more BACKGROUND In contrast to all other areas of medicine, psychiatry is still nearly entirely reliant on subjective assessments such as patient self-report and clinical observation. The lack of objective information on which to base clinical decisions can contribute to reduced quality of care. Behavioral health clinicians need objective and reliable patient data to support effective targeted interventions. OBJECTIVE We aimed to investigate whether reliable inferences—psychiatric signs, symptoms, and diagnoses—can be extracted from audiovisual patterns in recorded evaluation interviews of participants with schizophrenia spectrum disorders and bipolar disorder. METHODS We obtained audiovisual data from 89 participants (mean age 25.3 years; male: 48/89, 53.9%; female: 41/89, 46.1%): individuals with schizophrenia spectrum disorders (n=41), individuals with bipolar disorder (n=21), and healthy volunteers (n=27). We developed machine learning models based on acoustic and facial movement fea...
We conducted a feasibility analysis to determine the quality of data that could be collected ambi... more We conducted a feasibility analysis to determine the quality of data that could be collected ambiently during routine clinical conversations. We used inexpensive, consumer-grade hardware to record unstructured dialogue and open-source software tools to quantify and model face, voice (acoustic and language) and movement features. We used an external validation set to perform proof-of-concept predictive analyses and show that clinically relevant measures can be produced without a restrictive protocol. 2 Barron et al.
Predictive Linguistic Markers of Suicidality in Poets
2018 IEEE 12th International Conference on Semantic Computing (ICSC), 2018
Suicidal rates have been increasing since 2000 according the latest report of Centers for disease... more Suicidal rates have been increasing since 2000 according the latest report of Centers for disease control and prevention. Today internet opens a channel where people can communicate and information remains registered, making acoustic, semantic and syntactic analyses especially appealing to find hidden cues that can be used to detect signs of different mental conditions. Here we analyze poems from poets who committed suicide to develop a method to detect suicidal signs. We use bipartite graph matching algorithms after data retrieval to assure our results are not susceptible to bias created by variation in poem sample size among poets, and focus on linguistic content (e.g. similarity to specific concepts) and structure (e.g. density of ideas). Our results using different classifiers yield accuracy rates of up to 86% to discriminate suicidal from non-suicidal poets.
ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019
One of the main foci of addiction research is the delineation of markers that track the propensit... more One of the main foci of addiction research is the delineation of markers that track the propensity of relapse. Speech analysis can provide an unbiased assessment that can be deployed outside the lab, enabling objective measurements and relapse susceptibility tracking. This work is the first attempt to study unscripted speech markers in cocaine users. We analyzed 23 subjects performing two tasks: describing the positive consequences (PC) of abstinence and the negative consequences (NC) of using cocaine. We perform two main experiments: first, we analyzed whether acoustic and semantic features can infer clinical variables such as the Cocaine Selective Severity Assessment; then, we analyzed the main problem of interest: to see if these features are powerful enough to infer if the subjects remains abstinent. Our results show that speech features have potential to be used as a proxy to monitor cocaine users under treatment to recover from their addiction.
Uploads
Papers by Carla Agurto