Generative AI and Worker Productivity

How does access to a generative AI tool affect work in a call center?

That was a research question addressed by MIT Sloan Professor Danielle Li at a recent session of the MIT Institute for Work and Employment Research (IWER) weekly seminar series.

On April 9, Li gave a presentation based on her National Bureau of Economic Research (NBER) working paper “Generative AI at Work,” coauthored with Erik Brynjolfsson, the Jerry Yang and Akiko Yamazaki Professor and Senior Fellow at the Stanford Institute for Human-Centered AI, and Lindsey Raymond, a doctoral candidate at MIT Sloan. Li is an Associate Professor of  Technological Innovation, Entrepreneurship, and Strategic Management at the MIT Sloan School of Management, as well as a Faculty Research Fellow at NBER.

Li, Brynjolfsson, and Raymond studied the rollout of an AI conversational support tool for workers providing technical customer support via chat for a Fortune 500 company that sells software products to small businesses. The tool was rolled out in the workplace in 2020 and 2021, and the researchers were able to analyze data from 3 million chats involving 5179 customer support workers, including 1.2 million chats involving 1636 workers after they had access to the AI tool.

This AI tool offered conversational suggestions and links to resources to the workers providing customer support, but the workers had the choice about whether or not to adopt the tool’s suggestions. Li explained that not all of the tool’s suggestions were great, so workers didn’t blindly follow its advice; in fact, they adhered to only about 38% of the tool’s suggestions, on average.

Overall, access to the new tool increased worker productivity by about 14% on average, with the productivity gains concentrated among lower-skill, newer workers. For the most skilled workers, the efficiency gains from the tool were close to zero. By using the tool, “newer workers move down the experience curve faster,” Li explained, noting that such generative AI tools learn from the behavior of better workers and disseminate those best practices to lower-skilled workers.

There were also some signs that tips from the AI tool made a difficult job more pleasant: Customers swore less at workers and were less likely to ask to speak to a manager, and worker turnover decreased. With the AI tool’s assistance, newer workers get yelled at less frequently by unhappy customers, Li said.

Read more about this research in the working paper “Generative AI at Work.”