Hugging Face Says AI Models With Reasoning Use 30x More Energy on Average

It's not news to anyone that there are concerns about AI’s rising energy bill. But a new analysis shows the latest reasoning models are substantially more energy intensive than previous generations, raising the prospect that AI’s energy requirements and carbon footprint could grow faster than expected.

As AI tools become an ever more common fixture in our lives, concerns are growing about the amount of electricity required to run them. While worries first focused on the huge costs of training large models, today much of the sector’s energy demand is from responding to users’ queries.

And a new analysis from researchers at Hugging Face and Salesforce suggests that the latest generation of models, which “think” through problems step by step before providing an answer, use considerably more power than older models. They found that some models used 700 times more energy when their “reasoning” modes were activated.

“We should be smarter about the way that we use AI,” Hugging Face research scientist and project co-lead Sasha Luccioni told Bloomberg. “Choosing the right model for the right task is important.”

The new study is part of the AI Energy Score project, which aims to provide a standardized way to measure AI energy efficiency. Each model is subjected to 10 tasks using custom datasets and the latest generation of GPUs. The researchers then measure the number of watt-hours the models use to answer 1,000 queries.

The group assigns each model a star rating out of five, much like the energy efficiency ratings found on consumer goods in many countries. But the benchmark can only be applied to open or partially open models, so leading closed models from major AI labs can’t be tested.

In this latest update to the project’s leaderboard, the researchers studied reasoning models for the first time. They found these models use, on average, 30 times more energy than models without reasoning capabilities or with their reasoning modes turned off, but the worst offenders used hundreds of times more.

The researchers say that this is largely due to the way AI reasoning works. These models are fundamentally text generators, and each chunk of text they output requires energy to produce. Rather than just providing an answer, reasoning models essentially “think aloud,” generating text that is supposed to correspond to some kind of inner monologue as they work through a problem.