AI has high data center energy costs — but there are solutions

Surging demand for artificial intelligence has had a significant environmental impact, especially when it comes to data center use. The International Energy Agency has estimated that global electricity demand from data centers could double between 2022 and 2026, fueled in part by AI adoption.

“As we move from text to video to image, these AI models are growing larger and larger, and so is their energy impact,” said Vijay Gadepally, a senior scientist and principal investigator at MIT Lincoln Laboratory, where he leads the Supercomputing Center’s research initiatives. “This is going to grow into a pretty sizable amount of energy use and a growing contributor to emissions across the world.”

What’s needed are strategies that boost data center sustainability. Organizations can counter rising energy demands and costs with several foundational practices, Gadepally said at the MIT Sustainability Conference in October, from rethinking AI model training to investing in more efficient hardware.

The MIT Lincoln Laboratory Supercomputing Center’s efforts to reduce its own data center energy consumption show that it’s possible for organizations to pair sustainability practices with cost savings, tackling both fiscal and sustainability issues at the same time, Gadepally said.

AI’s energy impact

AI models — particularly generative AI models like GPT-4 — are becoming exponentially larger, which means that more data center energy is being used to train them and to process data.

Consider a “token,” a unit of text that a generative AI model uses to process inputs and generate outputs. Processing a million tokens, constituting a dollar’s worth of compute time, emits an amount of carbon similar to that produced by a gas-powered vehicle driven five to 20 miles, Gadepally said. Creating an image with generative AI uses the energy equivalent of fully charging a smartphone.

Already, data centers account for 1% to 2% of overall global energy demand, similar to what experts estimate for the airline industry, Gadepally said. That figure is poised to skyrocket, given rising AI demands, potentially hitting 21% by 2030, when costs related to delivering AI to consumers are factored in.

Water needed for cooling is another factor in data center sustainability calculations. As more data center equipment is squeezed into tighter physical quarters, it increases the requirement for aggressive cooling technologies, many of which draw from already stressed watershed areas, Gadepally said.

A playbook for reducing emissions

Gadepally said that taking some simple steps can make a significant dent in AI data center emissions — potentially shaving 10% to 20% off global data center electricity demand. The good news is that companies’ desire to do what’s right for the environment can align with their financial goals. “There isn’t a huge [capital expenditure] investment you need to make to cut down on energy emissions,” Gadepally said. “You can employ some of these techniques and cut your operating expenses.”

The Supercomputing Center has employed several strategies to reduce its own data center footprint and cut back on energy emissions. Gadepally’s recommendations, based on his research, include the following:

Limit the amount of power available. Just as organizations can choose to replace traditional light bulbs with more efficient LED bulbs, they can make more energy-efficient choices about data center gear. The obvious approach, Gadepally said, is opting for more efficient hardware whenever possible. His team has also experimented with “power capping,” or limiting the amount of power feeding the processors and graphics processing units that their supercomputer foundation comprises. “Rather than letting them go to 100%, we limit usage to 150 or 250 watts [about 60% to 80% of their total power] depending on which processor we’re using,” he said. “We’ve applied this to both training and inferencing workloads, and it not only reduces the overall power and energy consumption of the workloads; it reduces operating temperatures as well.”

Rethink model training. Another approach is to lean into cheaper, less-robust AI models for training. For example, Gadepally’s team decided to forgo the usual routine of training thousands of models to completion for a drug discovery application, given that most of training data would ultimately never be used. Instead, the team built a training speed estimation tool that tracks the loss curve of training models, allowing them to predict end-state accuracy after 20% of a computation is complete. “That allows us to quickly shave off about 80% of the compute, with no impact to the end model,” Gadepally said.

Make software AI- and carbon-aware. Software can be designed to automatically adjust for variations in carbon emissions and carbon impact throughout the day. By employing intelligent energy-reduction strategies as part of scheduling systems, AI workloads that aren’t time-sensitive can be automatically shifted to run at different times or in different geographic zones to address peak energy usage periods and achieve optimal energy savings.

For example, MIT, in collaboration with Northeastern University, built a software tool called Clover that makes carbon intensity a parameter so it can recognize peak energy periods and automatically make the proper adjustments, including using a lower-quality model or opting for lower-performing compute horsepower. “With this experiment, we reduced the carbon intensity for different types of operations by about 80% to 90%,” Gadepally said.

While the MIT Lincoln Laboratory Supercomputing Center continues to use these energy-efficient AI initiatives, there’s more work to be done, Gadepally said, including working with the broader community to collect data, build benchmarks, and rethink the current mindset that casts bigger AI models and data as better. “We need to think more about how we can get to the same answer but add a bit of intelligence to make AI processing more energy efficient,” he said.