# Measuring the energy consumption of your local LLM setup

[Skip to content](#lm-inhoud)Network/[NL](/en/lokale-llm-energieverbruik-meten)EN[Hubhub.llmnet.nlCompare models on task, language, cost and license.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs in production: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlRolling out AI in an organization, pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, on your own tasks.](https://benchmark.llmnet.nl/en/)[Careersvacatures.llmnet.nlAI roles, salaries and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRun AI privately on your own Mac, PC, NAS or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/en/)[](https://x.com/intent/post?url=https%3A%2F%2Fgids.llmnet.nl%2Fen%2Flokale-llm-energieverbruik-meten&text=Measuring%20the%20energy%20consumption%20of%20your%20local%20LLM%20setup)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fgids.llmnet.nl%2Fen%2Flokale-llm-energieverbruik-meten)[](https://www.reddit.com/submit?url=https%3A%2F%2Fgids.llmnet.nl%2Fen%2Flokale-llm-energieverbruik-meten&title=Measuring%20the%20energy%20consumption%20of%20your%20local%20LLM%20setup)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fgids.llmnet.nl%2Fen%2Flokale-llm-energieverbruik-meten&text=Measuring%20the%20energy%20consumption%20of%20your%20local%20LLM%20setup)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fgids.llmnet.nl%2Fen%2Flokale-llm-energieverbruik-meten)[](https://www.reddit.com/submit?url=https%3A%2F%2Fgids.llmnet.nl%2Fen%2Flokale-llm-energieverbruik-meten&title=Measuring%20the%20energy%20consumption%20of%20your%20local%20LLM%20setup)[](#)

 [gids.llmnet.nl](https://gids.llmnet.nl/en/)
 
# Measuring the energy consumption of your local LLM setup

 By Ivo Donker - 6 August 2026

 
 
 Running large language models (LLMs) locally offers immense advantages in privacy, control and latency. Anyone running inference with heavy models on powerful graphics cards or professional server hardware, however, quickly notices that power consumption can climb considerably. Measuring energy consumption accurately is useful not only for predicting the electricity bill but also for determining which quantization or model type delivers the best energy efficiency per generated token.

 In this guide we work through all the ways to map the energy consumption of a local AI setup: from hardware measurements at the wall socket to software telemetry at GPU and CPU level.

 

 
 
## Why measure energy consumption with local LLMs?

 With traditional CPU workloads, power consumption fluctuates strongly depending on short peaks. LLM inference and training, however, place a continuous, heavy load on the hardware for extended periods. Various factors determine how much energy is ultimately consumed:

 
 
- Idle vs. active (inference): A system merely waiting for instructions consumes considerably less power than one actively generating tokens. The ratio between this idle consumption and peak consumption determines efficiency.
 
- Model size and quantization: A 70B parameter model requires more memory bandwidth and compute than an 8B model. Choosing quantization wisely can drastically lower energy consumption per token. Consult the guide on [quantization explained](/en/kwantisatie-uitgelegd) for more background.
 
- Power supply efficiency (PSU): A computer power supply loses energy as heat. The degree to which alternating current (AC) from the socket is converted to direct current (DC) affects total power demand.
 
 

 
 
## Method 1: Measuring at the wall socket (hardware level)

 The most reliable method of measuring total power consumption is from the wall. Software tools, after all, only see the components themselves and ignore the efficiency losses of the power supply, the power consumption of cooling fans, water cooling and the motherboard.

 
### Inline meters and smart plugs

 For a quick measurement, a simple plug-in energy meter suffices. But if you want to monitor consumption continuously and link it to your LLM benchmarks, choose a smart plug with an open API or integration options (such as Zigbee, Tasmota or Home Assistant).

 
 
 
 Type of meter | 
 Accuracy | 
 Data export | 
 Application | 
 

 
 
 
 Basic meter (plug-in) | 
 Moderate to good | 
 None (display only) | 
 Occasional spot check | 
 

 
 WiFi/Zigbee smart plug | 
 Good | 
 Through API / Home Assistant | 
 Long-term monitoring & automation | 
 

 
 Professional PDU (rack) | 
 Very high | 
 SNMP / Prometheus | 
 Multi-GPU servers & homelabs | 
 

 
 

 
 Note: Make sure the smart plug can handle your setup's peak power. A workstation with two high-end GPUs can briefly draw a high current spike during startup or heavy load.

 
 

 
 
## Method 2: Software telemetry through the CLI

 To know exactly which part of your setup (GPU, CPU or memory) draws the most power, you can use built-in command-line tools.

 
### Measuring NVIDIA GPUs with nvidia-smi

 If you use an NVIDIA graphics card, nvidia-smi is the standard solution. The command below queries current power consumption in watts every second:

 nvidia-smi --query-gpu=timestamp,name,power.draw,temperature.gpu --format=csv -l 1

 Want to log power consumption straight to a file for later analysis? Then run the following:

 nvidia-smi --query-gpu=timestamp,power.draw --format=csv -l 1 > gpu_power_log.csv

 
### Measuring AMD GPUs with rocm-smi

 On Linux systems with AMD hardware and the ROCm stack, use rocm-smi to query power:

 rocm-smi --showpower

 
### Measuring Apple Silicon (Mac) with powermetrics

 Running LLMs locally on a Mac with Apple Silicon (M1/M2/M3/M4)? Then the CPU and GPU share the same unified memory. The power consumption of the SoC (system on chip) can be read on macOS through the terminal with root privileges:

 sudo powermetrics -i 1000 --samplers cpu_power

 This gives you a detailed overview every 1,000 milliseconds of consumption by the CPU cores and the GPU cluster in milliwatts.

 

 
 
## Calculating energy efficiency: joules per token

 Looking only at the number of watts gives an incomplete picture. A model consuming 300 watts while generating 60 tokens per second is more efficient than one consuming 150 watts but reaching only 10 tokens per second.

 To compute real efficiency we use the measure joules per token (J/tok). One watt equals one joule per second ($1\text{ Watt} = 1\text{ Joule/second}$).

 The formula for energy consumption per token is:

 Energie per token (J/tok) = Totaal vermogen (Watt) / Gegenereerde tokens per seconde (tok/s)

 
### Worked example:

 
 
- Scenario A: System consumes 250 watts and reaches 50 tok/s. 
 $$\frac{250\text{ W}}{50\text{ tok/s}} = 5.0\text{ Joules per token}$$
 
- Scenario B: System consumes 120 watts and reaches 15 tok/s. 
 $$\frac{120\text{ W}}{15\text{ tok/s}} = 8.0\text{ Joules per token}$$
 

 Although scenario B draws less power, scenario A is 37.5% more energy-efficient per generated token because the task finishes faster.

 

 
 
## Tips for lowering your LLM setup's power consumption

 Once you have mapped consumption, you can apply targeted optimizations to cut energy use without sacrificing performance drastically:

 
 
- Set power limiting: NVIDIA GPUs can be throttled very efficiently. You can often lower maximum permitted power by 20-30% through nvidia-smi -pl [limiet_in_watts], while the performance loss on LLM inference stays limited to a few percent.
 
- Choose the right quantization: Switching from 8-bit to 4-bit quantization halves the VRAM required. That means less memory bandwidth intensity and therefore lower power consumption. Read more on the page about [power consumption of local AI](/en/stroomverbruik-lokale-ai).
 
- Optimize your hardware configuration: Ensure an efficient power supply (80 Plus Gold or Platinum rating) and check whether you made the right choices through the guide to [hardware for local LLMs](/en/hardware-voor-lokale-llm).
 
- Avoid needless idling: Running your models in containers? Make sure the GPU drops back to its lowest power state when no requests come in. See also the article on [LLM optimization on hardware](/en/llm-op-oude-hardware-optimaliseren).
 
 

 
 
## Continuing the conversation and comparing benchmarks

 Want to compare your measurements with the community, or curious which hardware scores best on joules per token? Join the discussions on the [llmnet community](https://community.llmnet.nl/en/) or see independent performance tests on the knowledge network.

 
 

 © 2026 llmnet.nl - Knowledge network for local AI and LLMs