HSI Fall 2021 Research Updates

The project team has reached their participation goal of 500 participants and completed enrollment in the RCT. They will follow the participants for one year. Analyses of weekly and monthly data pulls will ensure that data quality remains high and may suggest further research questions.

Professor Doyle’s successful track record with the collaborator prompted that healthcare organization to agree to further analytics research with HSI. A second project, also with tremendous potential public health impact, studies the efficacy of text message campaigns to encourage flu vaccination.

Specifically, if patients who are identified by computer algorithm or medical records as high-risk for flu are told of their risk status, the researchers want to understand if that message makes the patients more likely to get vaccinated. On a more granular level, if patients are told that they are at risk along with an explanation for that determination, does giving patients that context affect their behavior?

Professor Farias’ research continues to investigate innovative methods to analyze very large highly complex, yet noisy and incomplete data sets. His focus is on observational data related to proteomics, that is the study of proteins in a cell with the ultimate goal of diagnosing illness and discovering new therapeutics.

Farias’s methods enable researchers to pinpoint a subset of data they are able to parse that is representative of the larger complex set, thereby giving the researchers an understanding of the entire data set from this specifically chosen sample. The method additionally offers indications that will lead to approaches to understand causal mechanisms.

There are both theoretical and practical successes from this work. The team’s theoretical work on causal analysis was recently accepted by Neural Information Processing Systems (NeurIPS) for their upcoming conference, one of only 55 papers accepted for full presentation out of more than 10,000 submissions.

On the practical side, the team, in collaboration with the Broad Institute tested their methods on a large-scale data set and successfully demonstrated its power to overcome some of the limitations in collecting proteomic data.

Farias and his team have recently applied for funding for a follow-on project under the auspices of the MIT-Takeda Alliance.

Jónas Jónasson, David Rand, and Erez Yoeli: Combining Machine Learning with Behavioral Insights to Provide Differentiated Digital Adherence Support

Since the previous update, the research team have been writing up results for publication and thinking about follow-on research to investigate questions sparked by the original project. The team continues to partner with digital health start-up Keheala, which provides the mobile phone platform. 

This project’s goal is to use machine learning to enable Keheala and its field partners in Kenya to provide differentiated care to provide a proof-of-concept for others. The hope is to be able to prioritize access to treatment before patients are enrolled. That is, to triage those patients who would benefit the most. Second, after patients are enrolled and taking the medication, the researchers want to be able to identify at-risk patients so they can be flagged for engagement to encourage them to be compliant. 

After analysis of the pre-enrollment data, the researchers report that the intervention has a meaningful positive effect for the patients. The greatest benefit was seen by those patients who, according to pre-enrollment data, were most likely to fail to complete treatment. This is a gratifying result since it suggests that the researchers’ model can predict which patients will need to most engagement to successfully complete treatment. They don’t have to wait to react  after a patient starts to fail.  

They also analyzed the intervention’s average treatment effects from using the Keheala digital platform. Keheala is associated with a roughly one-third reduction in treatment non-completion, and a roughly two-thirds reduction in daily non-adherence, as measured by random urine testing. 

The researchers are considering three avenues for future research: 

• Does support by Keheala make patients less likely to need to return for treatment for TB? Recurrence of TB is common, so it would be helpful to understand if using this innovation reduces the reinfection rate. 
• Currently, at-risk patients do not interact with the same support sponsor consistently. Is there a benefit to ensuring an ongoing relationship between sponsor and patient? 
• There are different types of support that patients can receive, and some patients may respond better to a specific type or support rather than another. The team is considering a project to try to design an algorithm for deciding which support services to provide to which patients on each day. 

Professors Jónas Jónasson and Nikos Trichakis are continuing their collaboration with industry partner, The Staten Island Performing Provider System (SIPPS) researching methods and models to predict an individual’s risk for any adverse opioid-related event and to assess how providers can use these models in intervene with patients. 

At this point, they have tested their model and come to four main findings:  

1. They have found a way to train their model to predict an outcome that is relatively rare, specifically fatal overdose. Their method outperforms others already in use. 
2. Rather than develop several models, one for each possible outcome, the research team’s approach led to a single model for predicting different outcomes. In the field, this results in simpler implementation and maintenance of the tool for providers. 
3. Their model can identify the small number of patients with whom to intervene that will result in preventing a large fraction of opioid-related harm. Specifically, they can identify the 1% of patients who would account for 68% of all opioid-related adverse events. 
4. The model is sufficiently robust that its predictive performance is not affected by data delays in reporting or extending the forecast in days for an adverse event. 

This project resulted in a model and implementation that could help the SIPPS better serve their at-risk patients. A team of MIT researchers has been formed to support this implementation and to conduct impact evaluation of the intervention system (a combination of predictive analytics and outreach program).

Professor Kelly and her team investigated whether workers in fulfillment centers that establish a Health and Well-being Committee (HaWC) composed of frontline workers and middle managers who work together to address workplace concerns fare better than their counterparts in other fulfillment centers in the same company. 

Since the previous update, the research team has continued to support the pilot site. The initial insights indicate that the intervention shows promise. The leads of HaWCs are still performing that role and have come up with methods to keep middle managers informed while maintaining facility workers’ participation. Based on these encouraging signs, they have begun to conduct baseline surveys of the next group of sites and to facilitate setting up of additional HaWCs.

Going forward, Kelly will be collaborating with faculty from the Harvard School of Public Health, as the next stages of this project are now funded by the National Institute for Occupational Safety and Health and tied to the Harvard Center for Work, Health & Well-being.

Georgia Perakis: Improving Hospital Operations through Predictive-Prescriptive Analytics and Behavioral Models

Perakis and her team are investigating new ways to allocate resources for emergency department (ED) patients that would be more effective and efficient than the current methods, resulting in better patient outcomes. Learnings from this area of the hospital can then be adapted to better hospital operations more generally.

The team built and compared models to predict patient’s length of stay (LOS) in the ED. From this work, they learned that dividing LOS into three components (arrival-to-room; room-to-disposition; disposition-to-discharge) improved predictive accuracy of the models because each component has different drivers. And, this granular detail also enabled the team to glean operational insights that wouldn’t be available otherwise.  

In the process of model building, the team observed that real hospital data shows evidence of inconsistencies and biases. Going forward, they are doing research to gain a deeper understanding of how these issues factor into decisions made on in the ED. They are working to extend their model to correct for such biases.

Aspects of this study, for example dividing LOS into three distinct time periods and combining that with resource and capacity factors into one predictive ED model –– are novel approaches that have not been researched before. This project has tremendous opportunities for benefiting patient care in the ED as well as offering recommendations for operations process improvements throughout the hospital.

Since Professor David Rand and his collaborators initially submitted their proposal in March 2021, the research team expanded the scope of their project. Originally, they planned to explore the connections between cultural beliefs and practices; and COVID-19 vaccine hesitancy among Black Americans. Following an understanding of this dynamic, the team wanted to develop interventions to encourage vaccination. 

Now, the team broadened the scope to include, more generally, how perceptions influence health behaviors with pathways known to impact health disparities (e.g., COVID-19 vaccination, flu vaccination, medication behavior, etc.),

The team has run six online surveys. They have received thousands of responses that are a nationally representative sample. Their high-level findings include: 

• There is substantial variation among Black Americans in their attitudes towards the COVID-19 vaccine.  
• Similarly, Black Americans have a wide range of reasons for being hesitant to get a COVID-19 vaccine. 
• Acculturation - the extent to which Black Americans are aligned with African American versus White culture - is a strong predictor of attitudes toward COVID-19 vaccination; it is substantially stronger, for example, than education or income. 

Following these results, the team decided to collect qualitative data to understand the reasons behind their high-level findings. Their preliminary results suggest that for those who did get the vaccine, government regulation was the motivation. For those who did not, the justifications include propaganda, mistrust and safety concerns. 

Rand and his team now plan to collect additional qualitative and quantitative data to better understand Black Americans’ attitudes toward vaccines. Once they believe they have a solid understanding of the reasoning, they will design and test messaging to learn which of these interventions are most effective at changing attitudes.

COVID-19 policy, in part, relies on COVID-19 infections data that is gathered from state databases. Professor Tucker’s research shows that not accounting for delays in reporting leads to misguided policy decisions that may have a detrimental effect on public health. 

There are several possibilities for reporting delays, but Tucker and her student Yifei Wang focus their research on delays caused by states’ reliance on outdated fax-based reporting, which leads to severe delay in reported actual number of cases and short-term fluctuations in reported numbers. The delay matters because, as Tucker shows, reporting delays lead to incorrect, ineffective policy decisions to deal appropriately with the pandemic. She concludes that if adjustments were made to correct for reporting delays, several major policies would be different.

Tucker focuses on six different policies: 

1. stay-at-home/shelter-in-place orders 
2. mandating face mask use by all individuals in public spaces 
3. closing non-essential businesses 
4. closing K-12 schools 
5. closing restaurants except take-out 
6. closing bars

As restaurants and bars are regarded as essential businesses in many states and therefore subject to separate closing and reopening schedules from non-essential businesses, the researchers treat them as separate policy variables from closing non-essential businesses orders in their analysis.

If policy makers assume no delays in reported numbers, the data analysis would lead to recommendations to shut down restaurants, open bars, and lift stay-at-home orders as soon as possible. They should also relax policies such as mask mandates and closing businesses. It seems that assuming some delay makes sense since assuming no delay leads to somewhat strange policy recommendations. 

If policy makers assume that reporting delays are the same in every state, the analysis would lead to a strong recommendation for wearing masks and closing non-essential businesses., policies two and three in the researchers list of policies under review.

However, states use different methods and processes for testing and reporting, so it is more accurate to assume that the reporting delays are different for different states. With that more accurate assumption, the data analysis still would recommend mask mandates and closing non-essential business, but now the protective effect is even stronger than in the scenario above, where reporting delays are the same in every state. Furthermore, under these assumptions, there is also evidence of a large beneficial effect of closing K-12 schools. 

On the other hand, policies such as stay-at-home orders and closing restaurants and bars are consistently estimated to have insignificant effects no matter how Tucker adjusted for reporting delay. 

Tucker’s research clearly demonstrates the need to take reporting delays into considerations when analyzing COVID-19 data. Without doing so, policy makers may not recommend the most beneficial interventions. More generally, this research may lead to questioning the effectiveness of current data-management and reporting practices and the myriad differences among the states.