Green Meter

Energy & carbon metering plugin for vLLM serving. It samples GPU power inside the inference process, attributes each scheduler step's energy to the requests that ran in it (conserving the total under concurrency), converts energy to operational carbon, and exposes everything over a small HTTP API plus a zero-build dashboard. It is a standalone, general-purpose plugin; NewAPI is one supported adaptation target (see adapters/newapi/), not a dependency.

Quickstart

pip install -e "./vllm-green-meter[nvml]"

export VLLM_GREEN_METER=1                      # master switch (default off)
export VLLM_GREEN_METER_MODEL=qwen3-4b-instruct # energy profile key
export VLLM_GREEN_METER_DIR=/tmp/vllm-green-meter  # JSONL ledger output
export VLLM_GREEN_METER_PORT=9109              # HTTP API port

vllm serve Qwen/Qwen3-4B-Instruct-2507

Then open http://127.0.0.1:9109/ for the dashboard.

Other environment variables (all optional): VLLM_GREEN_METER_HOST, VLLM_GREEN_METER_GPU (GPU index), VLLM_GREEN_METER_SAMPLE_MS (power sampling period, default 50), VLLM_GREEN_METER_CARBON_FACTOR (kg CO2e/kWh, default 0.5777), VLLM_GREEN_METER_WINDOW_S (GPU time-series window, default 5), VLLM_GREEN_METER_DEGRADED_AFTER (missed steps before the hook is declared degraded, default 50).

Architecture

any OpenAI-compatible client / NewAPI
        |  (direct to vLLM, or forwarded by NewAPI)
        v
  vLLM + green-meter plugin (vllm.general_plugins entry point)
        |  monkey-patches V1 Scheduler.schedule / EngineCore.step
        |  NVML 50 ms power sampling -> trapezoid integral per step
        |  energy-counter reconciliation + per-request conservation ledger
        v
  :9109  HTTP API  (/summary /requests /tasks /recent /metrics /health)
        |
        v
  single-file dashboard (served at /)  <== iframe-embeddable into NewAPI

Attribution rule: one GPU time slice is measured once. Serial requests get their slice directly; concurrent requests split the measured slice by calibrated token-work weights, so allocations always sum to the measurement — never double-counted. When NVML coverage of a step is insufficient, the plugin falls back to the versioned estimator profile (energy_source in the ledger says which). When the scheduler hook itself cannot capture batches (incompatible vLLM internals), the plugin degrades to whole-GPU windowed energy (confidence: "gpu-only") and logs a loud warning instead of failing silently.

HTTP API (:9109)

Endpoint	Method	Description
/	GET	Self-hosted dashboard (single-file HTML, no build)
/summary	GET	Totals: J / kWh / gCO2e, hook status, conservation_error_pct, power series
/requests	GET	Per-request ledger (top 100 by recency)
/tasks	GET	Per-task aggregates: energy, kWh, carbon, requests, tokens, first/last seen
/tasks/assign	POST	Retag ledger entries: {"task_id": str, "request_ids": [str]}
/ingest/step	POST	Internal: engine-worker step forwarding (multiprocess topology)
/recent	GET	Recent step records
/metrics	GET	Prometheus text format
/health	GET	hook_status, vLLM version, sampler liveness, conservation error, forwarding stats

Compatibility

vLLM version	Status
0.9.2 (V1 engine)	Verified baseline — hook, attribution, reconciliation all active
0.8.5 (V1, RTX 4090)	Verified 2026-08: hook, forwarding, reconciliation all active
other versions	Capability probe runs at load: if Scheduler.schedule, EngineCore.step or the SchedulerOutput fields are missing, the patch is not applied, hook_status becomes disabled with a loud warning, and GPU-only metering continues

Multiprocess topology. vLLM runs the engine core in a separate process by default. The plugin handles this without any configuration: whichever process binds the API port (normally the API server) owns accounting, and engine workers forward captured steps to it over loopback HTTP (/ingest/step), with a durable local copy in steps-raw.jsonl before every forward so nothing is lost if the API is unreachable. /health exposes api_owned, forwarded_steps and forward_failures. Single-process deployments (VLLM_ENABLE_V1_MULTIPROCESSING=0) account in-process and need no forwarding.

Hook lifecycle is observable: disabled (probe failed / not applied) -> degraded (patch applied, no batch captured yet or capture failing) -> active (batches flowing). Check GET /health or the dashboard status line. Energy-counter reconciliation (conservation_error_pct) uses nvmlDeviceGetTotalEnergyConsumption where the card supports it (e.g. RTX 4090); on cards without the counter the field is null and energy_counter_note explains why.

Method boundaries

• Scope: GPU operational energy only, from NVML. No CPU/RAPL, no host DRAM, no embodied/manufacturing carbon, no datacenter overhead (PUE) unless you fold it into the carbon factor yourself.
• Carbon is an engineering estimate: energy x a configurable grid factor. It is not a certified measurement.
• Estimator profiles are fitted on RTX 3090 / BF16 / vLLM 0.9.2; on other hardware they are approximations. Measured (NVML) attribution is preferred whenever available.

Repository layout

• vllm-green-meter/ — the plugin (Python package vllm_green_meter)
• adapters/newapi/ — integration contract for NewAPI deployments
• LICENSE — MIT

Development

cd vllm-green-meter
python -m pip install -e . --no-deps
python -m unittest discover -s tests

All tests are GPU-free (pynvml is imported lazily and is optional).