New MIT Sloan study discovers “widespread and repeated” retroactive changes to ESG scores

CAMBRIDGE, Mass., June 28, 2022 – The explosion in environmental, social, and corporate governance (ESG) investing has led to strong reliance on ESG ratings providers—and questions about the reliability of those ratings. This is not surprising given that trillions of dollars are at stake in ESG investing. A new research paper by MIT Sloan School of Management research associate Florian Berg and Kornelia Fabisik and Zacharias Sautner of the Frankfurt School of Finance and Management, validates these concerns, as they discovered “widespread and repeated” changes to the historical ESG scores by a leading vendor of this data.

“Is history repeating itself? The (un)repeatable past of ESG ratings” won the John L. Weinberg/IRRCi Research Award from the Weinberg Center at the University of Delaware, which was presented at the 2022 Corporate Governance Symposium by the European Corporate Governance Institute (ECGI).

Berg says, “The incredible growth of sustainable finance has created a billion-dollar market for ESG data. Yet, we found that the data is not reliable or consistent. The changes made in ESG scores at any particular time in history are massive.”

He explains that the data for any specific point in time should remain the same for a firm unless there is a documented reason for a retroactive change. However, their study revealed significant unannounced and unexplained changes to the data provided by Refinitive ESG, which was previously owned by Thomson Reuters. For example, looking at two versions of the same Refinitive ESG data for identical firm years – one from September 2018 and the other from September 2020 – the median overall scores in the rewritten data were 18% lower than in the initial data.

“The score rewriting leads to large changes in what are deemed to be high- or low-ESG firms. This is important because the classification of firms is widely used in ESG research and the investment industry,” says Berg, cofounder and research associate of the MIT Sloan Sustainability Initiative’s Aggregate Confusion Project.

In their paper, the researchers highlight how firms that performed better in a given year experienced upgrades in their E and S scores for that year through the data rewriting. Using predictive regressions, they showed that investing in firms with higher ESG scores in the initial data would not have led to economically or statistically significant performance gains. Yet, in the rewritten data, they found economically large, statistically significant positive effects of the E&S score on the firm’s future stock returns.

“These large differences matter because this performance would not have been possible with the data available to investors when forming their investment strategies,” says Berg.

He notes that the data rewriting occurs on an ongoing basis without any public announcements. To show this, the researchers compared ESG scores from February 9 and March 23, 2021 – just six weeks apart – and found that 85% of firms’ scores changed. While the score changes were mostly small in magnitude, the ongoing retroactive changes affected the classification of firms and the link between ESG scores and returns.

Berg says, “Our study highlights the incentive of the data provider to introduce a positive relationship between ESG scores and returns in the data to demonstrate that their ESG scores are useful for data users developing ESG-related investing strategies. Investors should always beware and conduct due diligence, but this is particularly critical with ESG ratings.”

About the MIT Sloan School of Management

The MIT Sloan School of Management is where smart, independent leaders come together to solve problems, create new organizations, and improve the world. Learn more at mitsloan.mit.edu.