Brian Liu: Predicting Patient Success in Digital Mental Healthcare

SilverCloud by Amwell is one of the organizations taking part in the MIT Sloan HSI Lab for Population Health’s research on workplace wellness. Specifically, SilverCloud aims to deliver digital mental health services through its online platform. Patient success is evaluated by the increase in their scores on standardized tests that measure anxiety and depression.

Brian Liu, a PhD student supervised by Rahul Mazumder, collaborated with SilverCloud on research aimed at understanding how to improve patient success rates. Using anonymized data from the company, Liu’s work recently won two awards from the Institute for Operations Research and the Management Sciences and the American Statistical Association for his research focused on developing interpretable machine learning algorithms that can be applied to understand which patients' characteristics and interactions may facilitate better clinical outcomes.

How SilverCloud Works
SilverCloud by Amwell describes itself as a digital, on-demand platform designed to strengthen the mental health and well-being of users. It offers evidence-based Cognitive Behavioral Therapy (CBT)-based content, interactive tools, and relatable user videos, all available anytime, anywhere on a user’s smartphone, tablet, or computer. Much like online classes, patients are assigned modules to complete. A key aspect of the platform is its support system, where each patient is assigned a human supporter, who functions similarly to a therapist within this online learning environment.
 
Supporters are assigned randomly to patients, and a single supporter typically manages a large number of clients. Communication with the supporter is primarily asynchronous, occurring through messages rather than real-time conversations. While patients usually have the same supporter throughout their engagement, the relationship is more focused on check-ins rather than intensive therapeutic interaction.

Upon joining SilverCloud, patients complete initial anxiety and depression surveys, such as the GAD-7, which are self-assessment questionnaires used in traditional in-person therapy settings. These surveys are repeated approximately every two weeks to monitor the patient's progress.

Award-Winning Research
The primary objective of this research is to understand and predict patient success within the SilverCloud platform. Liu and Mazumder are attempting to identify factors that contribute to positive outcomes and develop models that can reliably predict whether a patient will experience a favorable outcome.
 
Developing these predictive models is challenging due to the noisy nature of mental healthcare data. This noisiness is partly attributed to the fact that much of the data are self-reported, which can be inherently subjective. Additionally, the data have many dimensions, as researchers initially avoided preconceived notions and included a wide range of factors in their analysis. A significant part of the research involves not just building accurate models but also making them understandable and useful for practitioners. This consists of taking complex, high-performing models and distilling them into smaller, more interpretable models.

Findings
A key conclusion emerging from the research is the importance of early and consistent patient engagement as a predictor of successful mental health outcomes. Research has proven the intuitive notion that patients who use the program more frequently initially and continue to use it tend to achieve better results. Specifically, engagement during the first two weeks of using the platform is a critical indicator of overall improvement, as it is directly linked to the level of activity patients exhibit during this initial period.
 
Furthermore, an improvement in anxiety and depression survey scores within these first two weeks is also a strong sign of likely overall success.
 
Current research efforts are focused on gaining a more granular understanding of this early engagement, analyzing how long patients spend on the platform in each session and the time they dedicate to specific learning modules. The goal is to identify potential relationships between engagement with particular content and patient outcomes, which could then be used to nudge patients toward more beneficial courses.
 
Interestingly, initial findings suggest that factors such as gender, age, and initial diagnosis do not significantly impact outcomes within SilverCloud’s asynchronous environment. This research holds considerable potential for improving the delivery of mental health services, particularly in addressing the high demand and limited availability of traditional therapy. The focus on identifying early predictors and understanding engagement patterns can inform strategies to support patients better and enhance the effectiveness of SilverCloud’s platform.
 
Additional papers by Liu and Mazumder, focusing on making models interpretable, can be found here and here.