GitHub

Getting started

Installation

using Pkg
Pkg.add("DecisionRules")

DecisionRules.jl builds policies with Flux.jl and subproblems with JuMP; training requires a DiffOpt-compatible solver for the stage problems (Ipopt for smooth NLPs, HiGHS for LPs/MIPs). For GPU-accelerated training of large NLPs, install the companion package DecisionRulesExa.jl (see GPU Acceleration with DecisionRulesExa.jl).

Anatomy of a training run

Every TS-DDR training run assembles the same five ingredients:

  1. Stage subproblems — one JuMP model per stage, wrapped with DiffOpt.diff_optimizer so Lagrange duals and sensitivities are available. The incoming state, the uncertainty, and the policy's target state enter as parameters.
  2. A policy — a Flux model mapping $[w_t;\, x_{t-1}]$ to a target state $\hat{x}_t$ (see Target-state policies).
  3. An uncertainty sampler — how trajectories $w_{1:T}$ are drawn; the three supported formats (independent pools, joint-scenario pools, trajectory samplers) are the subject of Uncertainty Sampling.
  4. A training formulation — deterministic equivalent, stage-wise, multiple shooting, or strict (see Three training formulations and Strict mode: penalty-free gradient signal).
  5. An evaluation protocol — out-of-sample stage-wise rollout via RolloutEvaluation, with target- or realized-state feedback (see Evaluation semantics).

Quick start

using DecisionRules, JuMP, DiffOpt, Flux, Ipopt

# Build per-stage subproblems in JuMP (DiffOpt-enabled)
# subproblems, state_params_in, state_params_out, uncertainty_samples, initial_state = ...

# Define a policy: maps [uncertainty; state] → target state
policy = Chain(
    Dense(policy_input_dim(num_uncertainties, num_states), 64, relu),
    Dense(64, num_states),
)

# Train via stage-wise decomposition
train_multistage(
    policy, initial_state, subproblems,
    state_params_in, state_params_out, uncertainty_samples;
    num_batches=100, optimizer=Flux.Adam(1e-3),
)

Choosing a training formulation

FormulationHorizon couplingGradient source
Deterministic EquivalentFull horizon, one large NLPDuals on the coupled problem
Stage-wise (single shooting)Sequential rolloutDuals + DiffOpt per stage
Multiple ShootingWindowed sub-horizonsDiffOpt per window, continuity penalties
Strict subproblemsSequential rollout, no slackPure shadow-price duals

As a rule of thumb:

  • start with stage-wise training — closed-loop, smallest solves, fewest assumptions;
  • move to the deterministic equivalent (or its GPU implementation) when the per-stage solves are large and horizon-coupled gradient signal pays off;
  • use multiple shooting as the middle ground on long horizons;
  • switch to strict mode whenever you can construct a feasibility-guaranteeing policy — one that bounds its output to the one-stage reachable set $R(x, w)$ of the dynamics. Constructing $R$ is problem-specific (cheap for resource-balance dynamics with box bounds). The battery-storage AC-OPF specification shows both the battery-dynamic interval and the additional network-feasibility gate needed before strict mode is accepted. Strict mode eliminates the target-slack penalty and its tuning entirely, and the dual $\lambda_t$ becomes the pure shadow price.

Robustness and hardware

  • Solver or differentiation failures during training are handled by the pluggable gradient fallback system — by default a failed iteration logs a warning and is skipped.
  • Problems with large stage NLPs train an order of magnitude faster on GPU through DecisionRulesExa.jl, which implements the strict deterministic equivalent with ExaModels + MadNLP/cuDSS.

Where to go next