View on GitHub

Open benchmark · Autonomous AI · Energy systems

Evaluating autonomous AI agents in dynamic energy systems.

Agents can reason, plan and act. But what happens when the environment changes, information is incomplete, or the consequences of an action are not obvious?

GridActionBench is an open benchmark for evaluating how autonomous AI agents perceive, decide and act in simulated energy environments.

It tests not only whether an agent reaches the right outcome, but how it responds when conditions change — including when to act, hold, adapt or escalate.

Explore the benchmarkView on GitHub
Reliability evaluation pipelineEnvironment state flows to ground truth, the actual state of the system, and to an observation, which is what the agent actually receives and can be complete, missing, stale, conflicting or adversarial. The agent decides to act, hold, adapt or escalate. Its action becomes a simulated state change, checked against ground truth, and produces an evaluation.Environment stateGround truthSTATE OF THE SYSTEMObservationWHAT THE AGENT GETScompletemissingstaleconflictingadversarialAgent decisionactholdadaptescalateSimulated state changeEvaluationGROUND TRUTH AND THE OBSERVATION ARE NEVER THE SAME OBJECT

How it works

From observation to action

An agent does not see the environment as it actually is. It receives an observation, interprets it, and decides what to do.

  1. 01Ground truthThe actual state of the system
  2. 02ObservationThe information available to the agent
  3. 03Agent decisionWhat should happen next?
  4. 04ActionAct · Hold · Adapt · Escalate
  5. 05EnvironmentThe system changes
  6. 06EvaluationWas the response appropriate?

The separation between ground truth and observation allows the benchmark to test how agents behave when their view of the system is incomplete.

Capabilities

What we test

  • 01

    Perceive

    Can the agent correctly interpret the information available to it?

  • 02

    Decide

    Can it select an appropriate response given the current conditions?

  • 03

    Act

    Can it take an action without violating system constraints?

  • 04

    Adapt

    Can it respond when the situation changes?

  • 05

    Escalate

    Can it recognise when it should stop and involve a human?

Reliability

Reliability under changing conditions

Agent behaviour can appear reliable when conditions remain stable. GridActionBench tests what happens when those conditions change.

Complexity

More steps, dependencies and decisions.

Uncertainty

Incomplete, conflicting or changing information.

Autonomy

More freedom to decide and act.

Scenarios

Scenarios introduce changes such as

  1. 01Incomplete or conflicting information
  2. 02Changing system conditions
  3. 03Operational constraints
  4. 04Multi-step decisions
  5. 05Consequences that emerge over time
  6. 06Situations where escalation may be appropriate

ObservableReproducibleComparable

Environment

Current environment

GB-BESS v0.1

The first GridActionBench environment is a simulated grid-connected battery energy storage system.

It provides a controlled setting for testing agents against changing operating conditions, incomplete information and system constraints.

The initial environment is intentionally narrow.

The benchmark can expand to other energy assets and more complex environments as the methodology develops.

Explore GB-BESS

Contribute

Built to be open

GridActionBench is open source.

The scenarios, evaluation logic and tooling are intended to be inspectable, reproducible and open to contribution.

We’re interested in feedback from people working across:

  • 01

    Energy systems

    Power, batteries, networks and system operations.

  • 02

    AI evaluation

    Reliability, robustness and agent evaluation.

  • 03

    Agent development

    Testing different models and agent architectures.

  • 04

    Research

    Designing scenarios, metrics and experiments.

Join the team

Want to help build this?

Tell us a bit about yourself and how you’d like to contribute. This opens your email client, addressed to hello@gridactionbench.org.

View on GitHubRead the contribution guide