API Reference

Public API

DecisionRules.AbstractGradientFallback — Type
AbstractGradientFallback

Abstract type governing what happens when a solver or differentiation error occurs during training.

DecisionRules ships two concrete subtypes:

TypeBehavior
ZeroGradientFallbackLog a warning, return zero gradients, continue training
ErrorGradientFallbackRe-throw the error (useful in tests)

Extending

Implement your own subtype to customize recovery:

struct MyFallback <: DecisionRules.AbstractGradientFallback end

function DecisionRules.handle_gradient_error(::MyFallback, e, n_state_in, n_state_out)
    # e is the caught exception
    # Return a tuple of cotangents (same shape as the rrule pullback) or rethrow
    @error "Custom handler" exception=e
    return DecisionRules._zero_cotangents(n_state_in, n_state_out)
end

function DecisionRules.handle_training_error(::MyFallback, e, iter)
    # Return true to skip this iteration, false to rethrow
    @error "Custom training handler" exception=e
    return true  # skip
end

function DecisionRules.handle_rollout_error(::MyFallback, e, iter)
    # Return true to skip this scenario, false to rethrow
    return true
end

Then pass gradient_fallback=MyFallback() to train_multistage or train_multiple_shooting.

source
DecisionRules.AbstractIntegerStrategy — Type
AbstractIntegerStrategy

Abstract supertype for strategies that prepare a JuMP model before reading duals or solver sensitivities.

Arguments

This abstract type has no fields. Concrete subtypes are passed as the integer_strategy::AbstractIntegerStrategy keyword to simulation and training functions.

Examples

simulate_multistage(
    subproblems,
    state_params_in,
    state_params_out,
    initial_state,
    uncertainties,
    policy;
    integer_strategy = FixedDiscreteIntegerStrategy(),
)
source
DecisionRules.ContextualPolicy — Type
ContextualPolicy(policy, context)

Wrap a stage policy so each call receives known exogenous context before the usual policy input. The wrapped policy is called as policy(vcat(context_at(context, t), input)), where t is advanced once per call and reset by Flux.reset!.

This keeps training and rollout loops unchanged: context is data attached to the trajectory, while the inner policy remains the only trainable component. Matrix contexts are interpreted as d_context x T; function contexts are called as context(t).

source
DecisionRules.ContinuousRelaxationIntegerStrategy — Type
ContinuousRelaxationIntegerStrategy()

Relax all binary/integer constraints to continuous bounds (binary → [0,1]), solve the resulting LP, and read duals in that relaxed state.

Mathematically, this replaces $z \in \{0,1\}$ or integer restrictions with continuous bounds before solving. The derivative signal belongs to the relaxed problem, not to the original MIP.

Compared to FixedDiscreteIntegerStrategy:

  • Faster: one LP solve instead of MIP + LP.
  • Smoother gradients: no integer fixing means no zero-gradient dead zones.
  • Less accurate: the LP solution may have fractional integer variables, so the gradient does not correspond to any feasible integer assignment.

A practical pattern is to train with ContinuousRelaxationIntegerStrategy during warmup (smooth landscape for initial learning) and switch to FixedDiscreteIntegerStrategy later (integer-accurate gradients for fine-tuning).

Arguments

This type has no fields.

Examples

strategy = ContinuousRelaxationIntegerStrategy()
source
DecisionRules.ErrorGradientFallback — Type
ErrorGradientFallback()

Strict fallback: re-throw any solver or differentiation error. Use this in tests to ensure that controlled problems never silently produce zero gradients.

source
DecisionRules.FixedDiscreteIntegerStrategy — Type
FixedDiscreteIntegerStrategy()

Solve a mixed-integer model, fix discrete variables to their incumbent values, relax integrality, re-solve, and read duals or sensitivities from the fixed continuous model.

If $z^*$ is the incumbent binary/integer solution, this strategy reads derivative-like information from the continuous problem

\[\min_x f(x, z^*) \quad \text{subject to} \quad g(x, z^*) \le 0.\]

The result is local to the incumbent integer assignment. It is not a differentiable MIP method.

Arguments

This type has no fields.

Examples

strategy = FixedDiscreteIntegerStrategy()
source
DecisionRules.NoIntegerStrategy — Type
NoIntegerStrategy()

Solve the model exactly as written before reading duals or sensitivities.

Use this for continuous LP, conic, or nonlinear models whose derivative information is available directly from the solved model.

Arguments

This type has no fields.

Examples

strategy = NoIntegerStrategy()
source
DecisionRules.RolloutEvaluation — Type
RolloutEvaluation(subproblems, state_params_in, state_params_out, initial_state,
                  scenarios; stride=1, policy_state=:realized)

Evaluation helper that assesses the policy with a stage-wise rollout (the deployment semantics of a target-trajectory policy) on a fixed held-out scenario set. Deterministic-equivalent evaluation re-optimizes all stages jointly and can absorb stage-wise-unfollowable targets through the slack penalty, silently overstating policy quality; the rollout metric is the guard that detects this.

policy_state controls what state is passed back into the policy:

  • :realized pipes the previous realized state into the policy. This is the deployment/closed-loop rollout semantics.
  • :target pipes the previous target state into the policy, matching the deterministic-equivalent target-generation semantics from simulate_states while still solving the stage subproblems sequentially.

scenarios must be a vector of materialized scenarios, sampled once before training (e.g. [DecisionRules.sample(uncertainty_samples) for _ in 1:n]), so every evaluation uses the same fixed set. subproblems may be the training subproblems (all stage parameters are rewritten on every solve) or a separately built copy; when training on a deterministic equivalent, pass the stage-wise subproblems here.

Call evaluation(iter, model), e.g. from within a record callback. Every stride calls it rolls the policy out over the fixed set and reports:

  • metrics/rollout_objective_no_deficit: the rollout objective excluding the target-slack penalty term (the operational cost), and
  • metrics/rollout_target_violation_share: the realized slack penalty divided by the full rollout objective (NaN when undefined).

Policy comparisons should only be trusted when the violation share is small (≤ ~0.05); a larger share means the policy's targets are not followable stage by stage and the reported cost is not what deployment would realize. The latest values are kept in last_objective_no_deficit / last_violation_share for custom logging. Calls on batches that are not a multiple of stride are a no-op and leave the cached values unchanged.

source
DecisionRules.RolloutEvaluation — Method
(evaluation::RolloutEvaluation)(iter, model) -> Nothing

Evaluate the policy on the held-out scenario set every stride iterations.

On active iterations (iter % stride == 0), rolls model out over all fixed scenarios using either closed-loop (:realized) or target-feedback (:target) semantics. Prints metrics/rollout_objective_no_deficit and metrics/rollout_target_violation_share, and caches the values in evaluation.last_objective_no_deficit / evaluation.last_violation_share. Scenarios that fail to solve are skipped with a warning if all fail.

source
DecisionRules.SampleLog — Type
SampleLog(; on_sample=(s, models, sample_log) -> nothing,
          objective_no_deficit_fn=get_objective_no_target_deficit)

Per-sample logger with local cache state for the training loops, patterned after SaveBest. During each batch the training loop calls sample_log(s, det_equivalent_or_subproblems) right after sample s has been simulated (successful solves only; a failed solve throws exactly as before). The default behavior caches, per sample, the full objective (objective_value) and the objective excluding the target-slack penalty term (objective_no_deficit_fn). The cache is cleared at the start of every batch and handed to the per-batch record(sample_log, iter, model) callback.

on_sample is an optional hook called as on_sample(s, models, sample_log) after the default caching. It receives the live JuMP model(s), so it can inspect termination statuses or dump the details of a suspicious sample for debugging — without paying any per-sample logging cost in the default configuration.

source
DecisionRules.SaveBest — Type
SaveBest(best_loss::Float64, model_path::String)

Callback that saves the best policy state seen during training.

SaveBest is a small callable object used as a training callback. When called as callback(iter, model, loss), it compares loss with the best loss stored so far. If the new loss is smaller, it copies model to CPU, normalizes any recurrent layer state, and writes the Flux state to model_path with JLD2. It returns false, so it records checkpoints without stopping training.

Arguments

  • best_loss::Float64: incumbent loss. Use Inf to save the first observed model.
  • model_path::String: path of the JLD2 file that receives the best model state.

Examples

callback = SaveBest(Inf, "best_policy.jld2")
train_multistage(policy, x0, subproblems, state_in, state_out, sampler;
    record = (log, iter, model) -> callback(iter, model, mean(log.losses)))
source
DecisionRules.SaveBest — Method
(callback::SaveBest)(iter, model, loss) -> Bool

Compare loss against the incumbent callback.best_loss. When loss is strictly smaller, copy model to CPU, normalize recurrent state via normalize_recur_state, and write the Flux state to callback.model_path with JLD2. Always returns false (never stops training).

source
DecisionRules.ScoreFunctionConfig — Type
ScoreFunctionConfig(
    subproblems::AbstractVector{<:JuMP.Model},
    state_params_in::AbstractVector,
    state_params_out::AbstractVector;
    dual_weight::Real = 0.5,
    perturbation_std::Real = 1.0,
    num_rollouts::Integer = 8,
    baseline::Symbol = :mean,
)

Configure the score-function correction used by train_multistage.

The deterministic-equivalent training path differentiates the target policy through dual information. For mixed-integer subproblems, those duals are local to a fixed integer assignment. This configuration adds a REINFORCE-style correction estimated from stage-wise rollouts with perturbed targets.

The rollout models are solved exactly as they are built. If subproblems contain binary variables, the score-function rollouts solve MIPs. If they contain relaxed variables, the score-function rollouts solve the relaxation. This is intentionally separate from the integer_strategy keyword of train_multistage, which controls only how the differentiable dual-gradient path reads local sensitivity information from the deterministic equivalent.

If $\hat{x}_{t+1}(\theta)$ is the target emitted by the policy and $\delta_t \sim \mathcal{N}(0, \sigma^2 I)$, the perturbed rollout solves with target $\hat{x}_{t+1}(\theta) + \delta_t$. The score-function surrogate loss is

\[L_{sf}(\theta) = \frac{1}{M} \sum_{m=1}^{M} (R_m - b) \sum_{t=1}^{T} \left\langle \frac{\delta_{m,t}}{\sigma^2}, \hat{x}_{t+1}(\theta) \right\rangle ,\]

and the mixed gradient is

\[\nabla L = \alpha \nabla L_{dual} + (1 - \alpha) \nabla L_{sf}.\]

Arguments

  • subproblems::AbstractVector{<:JuMP.Model}: stage-wise rollout models used to estimate realized costs under perturbed targets.
  • state_params_in::AbstractVector: stage input-state parameters.
  • state_params_out::AbstractVector: pairs (target_parameter, state_variable) for every stage output state.

Keywords

  • dual_weight::Real: mixing weight $\alpha$ on the dual-gradient term.
  • perturbation_std::Real: Gaussian standard deviation $\sigma$.
  • num_rollouts::Integer: number of perturbed rollouts $M$ per sample.
  • baseline::Symbol: either :mean for mean-centering costs or :none. Mean-centering reduces variance but, because the baseline is computed from the same $M$ rollouts, it introduces a small $O(1/M)$ bias (effectively scaling the estimator by $(M-1)/M$) that vanishes as num_rollouts grows; a leave-one-out baseline would be exactly unbiased.

Examples

score_function = ScoreFunctionConfig(
    rollout_subproblems,
    state_params_in,
    state_params_out;
    dual_weight = 0.5,
    perturbation_std = 1.0,
    num_rollouts = 8,
)

train_multistage(
    policy,
    initial_state,
    det_equivalent,
    state_params_in,
    state_params_out,
    uncertainty_sampler;
    score_function,
)
source
DecisionRules.ScoreFunctionSchedule — Type
ScoreFunctionSchedule(config::ScoreFunctionConfig; <keyword arguments>)

Ramp a ScoreFunctionConfig into training after a pure-dual warmup.

The schedule delays score-function rollouts until sf_start, then linearly increases the score-function weight, perturbation scale, and rollout count until the final values stored in config are reached.

Let $k$ be the current iteration and $\rho_k = \operatorname{clip}((k - k_0) / r, 0, 1)$. The effective score-function weight is $\rho_k (1 - \alpha)$. The effective dual weight is one minus that value.

Arguments

  • config::ScoreFunctionConfig: final score-function configuration.

Keywords

  • sf_start::Integer: first iteration at which score-function rollouts are considered.
  • ramp_batches::Integer: number of iterations in the linear ramp.
  • perturbation_std_initial::Real: initial $\sigma$ at ramp start.
  • num_rollouts_initial::Integer: initial rollout count at ramp start.

Examples

schedule = ScoreFunctionSchedule(
    score_function;
    sf_start = 200,
    ramp_batches = 300,
    perturbation_std_initial = 0.1,
    num_rollouts_initial = 2,
)
source
DecisionRules.StallingCriterium — Type
StallingCriterium(patience::Int, best_loss::Float64, stall_count::Int)

Early-stopping callback that halts training when the loss stalls.

Tracks the number of consecutive iterations without improvement. When stall_count reaches patience, the callback returns true to signal that training should stop. Use best_loss = Inf and stall_count = 0 for a fresh start.

Arguments

  • patience::Int: maximum consecutive non-improving iterations before stopping.
  • best_loss::Float64: incumbent best loss. Use Inf to accept the first value.
  • stall_count::Int: current stall counter (typically initialized to 0).
source
DecisionRules.StallingCriterium — Method
(callback::StallingCriterium)(iter, model, loss) -> Bool

Update the stall counter and return true when stall_count >= patience.

If loss < best_loss, reset the counter to zero and update the incumbent. Otherwise increment stall_count. Returns true (stop training) once the patience budget is exhausted; false otherwise.

source
DecisionRules.StateConditionedPolicy — Type
StateConditionedPolicy

A policy architecture that separates temporal encoding from state conditioning.

The encoder is a recurrent cell (LSTMCell/GRUCell/RNNCell, or a Chain of cells) that processes only the uncertainty sequence to capture temporal dependencies. The combiner is a feed-forward target head that merges the encoder output with the previous state to predict the next state.

Given uncertainty $w_t$ and previous state $x_{t-1}$, the forward pass is

\[h_t, s_t = \text{LSTM}(w_t, s_{t-1}) \\ \hat{x}_t = f_{\text{combine}}([h_t;\; x_{t-1}])\]

where $s_t$ is the hidden recurrent state carried across stages and $f_{\text{combine}}$ is a Dense layer with activation $\sigma$.

Flux's recurrent cells are stateless (Flux >= 0.16): each call returns (output, new_state) instead of mutating an internal Recur. StateConditionedPolicy therefore carries the encoder's recurrent state itself in state, threading it through one call per stage. Call Flux.reset! to clear it (back to Flux.initialstates) at the start of a rollout.

Fields

  • encoder::E: recurrent cell or Chain of cells encoding uncertainty.
  • combiner::C: feed-forward head mapping $[h_t;\; x_{t-1}]$ to $\hat{x}_t$.
  • state::S: current recurrent state $s_t$, carried across calls.
  • n_uncertainty::Int: dimensionality of $w_t$.
  • n_state::Int: dimensionality of $x_{t-1}$.

Input format: [uncertainty..., previous_state...].

source
DecisionRules.StateConditionedPolicy — Method
(m::StateConditionedPolicy)(x) -> AbstractVector

Execute the forward pass of the state-conditioned policy.

The input x is split into the uncertainty portion $w_t$ and the previous state $x_{t-1}$. The forward pass computes

\[h_t, s_t = \text{encoder}(w_t, s_{t-1}) \\ \hat{x}_t = f_{\text{combine}}([h_t;\; x_{t-1}])\]

where $s_t$ is the updated recurrent state (stored in m.state for the next call) and $f_{\text{combine}}$ is a Dense layer.

Arguments

  • x::AbstractVector: concatenated input [w_t..., x_{t-1}...] of length m.n_uncertainty + m.n_state.

Returns

  • AbstractVector: predicted next state $\hat{x}_t$.
source
DecisionRules.WindowData — Type
WindowData

One multiple-shooting window: the JuMP model of stages stage_range solved together, and handles to its parameters.

Fields

  • model::JuMP.Model: the window's optimization model.
  • state_in_params: parameters fixing the state entering the window, $x_{t_0-1}$.
  • state_out_params: for each stage of the window, the pairs (target_param, realized_var) linking the policy target $\hat x_t$ to the realized state $x_t$; the parameter duals of target_param are the policy gradient signal.
  • uncertainty_params: for each stage of the window, the parameters holding $w_t$.
  • stage_range::UnitRange{Int}: the global stages $t_0, \dots, t_1$ this window covers.
source
DecisionRules.ZeroGradientFallback — Type
ZeroGradientFallback()

Default fallback: log a warning and return zero gradients when the solver or DiffOpt differentiation fails. Training continues with a skipped update for that iteration.

source
DecisionRules.compute_parameter_dual — Method
compute_parameter_dual(model::JuMP.Model, param::JuMP.VariableRef)

Compute the dual value (sensitivity) of a parameter in a solved JuMP model.

The parameter dual represents ∂(objective)/∂(parameter_value) and is computed by:

  1. Finding all constraints where the parameter appears
  2. For each constraint, computing: -coefficient * constraint_dual
  3. For the objective, adding the coefficient (or negative for maximization)
  4. Summing all contributions

This works for any solved model, not just convex ones, as long as dual values are available.

Arguments

  • model: A solved JuMP model
  • param: A parameter variable (created with @variable(model, p in MOI.Parameter(value)))

Returns

  • The dual value (sensitivity) of the parameter

Example

model = Model(HiGHS.Optimizer)
@variable(model, x >= 0)
@variable(model, p in MOI.Parameter(1.0))
@constraint(model, con, x >= 2 * p)
@objective(model, Min, 3 * x + p)
optimize!(model)
dual_p = compute_parameter_dual(model, p)  # = 2 * dual(con) + 1  (constraint x - 2p >= 0 has coef -2 on p, contribution -coef*dual)
# Hand-check: at the optimum x* = 2p the objective is 7p, so d(obj)/dp = 7 = 2*3 + 1 with dual(con) = 3.

Limitations

Only affine, quadratic, and vector-affine constraint functions are inspected: if the parameter appears in any other constraint function type (e.g. a nonlinear ScalarNonlinearFunction constraint), that contribution is skipped with a one-time warning and is NOT captured in the returned sensitivity.

The constraint-dual formula is validated for MIN_SENSE objectives; for MAX_SENSE models JuMP's dual sign conventions differ and this function's constraint contribution has not been validated.

source
DecisionRules.create_deficit! — Method
create_deficit!(model::JuMP.Model, len::Int; penalty_l1=nothing, penalty_l2=nothing, penalty=nothing)

Create deficit variables to penalize state deviations in a JuMP model.

Supports three modes controlled by the penalty keywords. Let $d \in \mathbb{R}^n$ be the deficit vector (len = n).

L1 norm only (default when no penalty keyword is given, or penalty_l1 alone):

\[\text{norm\_deficit} \geq \| d \|_1 = \sum_{i=1}^{n} |d_i|, \quad \text{objective} \mathrel{+}= \lambda_1 \cdot \text{norm\_deficit}.\]

Implemented via MOI.NormOneCone(1 + n).

L2 squared norm only (penalty_l2 alone):

\[\text{norm\_deficit} \geq \| d \|_2^2 = \sum_{i=1}^{n} d_i^2, \quad \text{objective} \mathrel{+}= \lambda_2 \cdot \text{norm\_deficit}.\]

Both norms (penalty_l1 and penalty_l2):

\[\text{norm\_deficit} \geq \lambda_1 \| d \|_1 + \lambda_2 \| d \|_2^2, \quad \text{objective} \mathrel{+}= 1 \cdot \text{norm\_deficit}.\]

Arguments

  • model: The JuMP model to add deficit variables to.
  • len: Number of deficit variables (typically dimension of state).
  • penalty_l1: Penalty coefficient $\lambda_1$ for the L1 norm (NormOneCone). Pass :auto to use max |objective coefficients|.
  • penalty_l2: Penalty coefficient $\lambda_2$ for the L2 squared norm (sum of squares). Pass :auto to use max |objective coefficients|.
  • penalty: Legacy argument. If provided and penalty_l1/penalty_l2 are both nothing, uses this for L1 norm only.

Returns

  • norm_deficit: Single variable representing total penalized deviation (for logging compatibility).
  • _deficit: Vector of deficit variables for each state dimension.

Examples

# L1 norm only (default behavior, backwards compatible)
norm_deficit, _deficit = create_deficit!(model, 3; penalty=1000.0)

# L2 norm only
norm_deficit, _deficit = create_deficit!(model, 3; penalty_l2=1000.0)

# Both L1 and L2 norms
norm_deficit, _deficit = create_deficit!(model, 3; penalty_l1=1000.0, penalty_l2=500.0)
source
DecisionRules.default_annealed_schedule — Method
default_annealed_schedule(num_batches::Int)

Build the default annealed target-penalty schedule over num_batches training batches: multipliers 0.1 -> 1.0 -> 10.0 -> 30.0 with phase lengths proportional to 2/2/4/16 of the horizon (the last phase takes the remainder; every phase keeps at least one batch). For num_batches < 4 the last num_batches multipliers are used, one batch each, so the run always ends at the strong-penalty phase.

Returns a Vector{Tuple{Int,Int,Float64}} of (first_batch, last_batch, multiplier) entries suitable for the penalty_schedule keyword of train_multistage and train_multiple_shooting. Multipliers are applied relative to the penalty the model was built with (the objective coefficient of the norm_deficit variables created by create_deficit!), so with penalty=:auto the effective penalty is multiplier * max |objective coefficient|.

source
DecisionRules.default_record — Method
default_record(sample_log, iter, model)

Default per-batch recording callback: prints the same two per-batch lines as the historical record_loss default (metrics/loss = mean objective excluding the target-slack penalty, then metrics/training_loss = mean full objective) and returns false (training continues). Return true from a custom record to stop training.

source
DecisionRules.dense_multilayer_nn — Method
dense_multilayer_nn(num_inputs, num_outputs, layers; activation=Flux.relu, dense=Dense)

Create a multi-layer neural network with the specified architecture.

Given hidden layer widths $(h_1, \dots, h_L)$ and activation $\sigma$, the resulting Chain computes

\[f(x) = W_{L+1} \, \sigma\bigl(W_L \cdots \sigma(W_1 x + b_1) \cdots + b_L\bigr) + b_{L+1},\]

where $W_k \in \mathbb{R}^{h_k \times h_{k-1}}$, $h_0 = \text{num\_inputs}$, and the final layer $W_{L+1} \in \mathbb{R}^{\text{num\_outputs} \times h_L}$ has no activation. When layers is empty, there is no hidden/output distinction: the returned single dense layer uses activation, preserving the same constructor semantics as Dense(num_inputs, num_outputs, activation).

If dense is a recurrent type (LSTM, GRU, RNN), pair notation in => out is used instead of (in, out, σ), and the last layer omits the activation so it can act as a linear projection.

Arguments

  • num_inputs::Int: number of input features $h_0$.
  • num_outputs::Int: number of output features.
  • layers::Vector{Int}: hidden layer widths $(h_1, \dots, h_L)$.
  • activation: element-wise activation $\sigma$ (default: Flux.relu).
  • dense: layer constructor (Dense, LSTM, GRU, RNN).

Returns

  • Chain (or a single layer when layers is empty).

Examples

mlp = dense_multilayer_nn(10, 3, [64, 32])       # 10 → 64 → 32 → 3
rnn = dense_multilayer_nn(5, 2, [16]; dense=LSTM) # 5 → 16 → 2 (LSTM)
source
DecisionRules.find_variables — Method
find_variables(model::JuMP.Model, variable_name_parts::Vector{<:AbstractString})

Return variables from model whose JuMP name contains all substrings in variable_name_parts. When the initial filter yields more than one variable, results are reordered by matching "<first_part>[i]" for i = 1, 2, ... to produce a consistently indexed vector.

source
DecisionRules.materialize_tangent — Method
materialize_tangent(t::ChainRulesCore.MutableTangent) -> NamedTuple

Unwrap a MutableTangent (produced by Zygote for mutable structs such as Flux.Recur) by extracting its backing NamedTuple and recursing.

source
DecisionRules.normalize_recur_state — Method
normalize_recur_state(state)

Return a copy of a Flux.state object where any Recur-like nodes have their state field set to cell.state0. This avoids Flux.loadmodel! tie errors when loading into freshly constructed recurrent layers.

source
DecisionRules.policy_input_dim — Method
policy_input_dim(num_uncertainties, num_states)

Compute the input dimension for a policy network.

Policy networks receive $[w_t;\; x_{t-1}]$ as input, so the required dimension is

\[d_{\text{in}} = \dim(w_t) + \dim(x_{t-1}).\]

This format is consistent between subproblem and deterministic-equivalent formulations, enabling warm-starting policies trained with det_eq for use with subproblems.

Arguments

  • num_uncertainties::Int: dimensionality of the stage uncertainty $w_t$.
  • num_states::Int: dimensionality of the state $x_{t-1}$.

Returns

  • Int: total input dimension $d_{\text{in}}$.

Examples

d = policy_input_dim(5, 3)  # 8
source
DecisionRules.policy_input_dim — Method
policy_input_dim(uncertainty_samples, initial_state)

Compute the input dimension for a policy network from problem data.

Infers $\dim(w_t)$ from the first uncertainty sample and $\dim(x)$ from the initial state vector, then delegates to policy_input_dim(::Int, ::Int).

Arguments

  • uncertainty_samples::Vector: uncertainty samples; length(uncertainty_samples[1]) gives $\dim(w_t)$.
  • initial_state::Vector: initial state vector; length(initial_state) gives $\dim(x)$.

Returns

  • Int: total input dimension $d_{\text{in}}$.

Examples

d = policy_input_dim(uncertainty_samples, x0)
source
DecisionRules.predict_window_targets — Method
predict_window_targets(decision_rule, s_in, uncertainties_vec)

Predict one target per stage in a window. This is an AD-friendly scan: target1 = π([u1; sin]) target2 = π([u2; target1]) ...

source
DecisionRules.sample — Method
sample(sampler::Function, T::Int)

Draw a full trajectory using a callable trajectory sampler with temporal dependence.

sampler(t, past) receives the current stage t and a vector of all previously realized samples past[1:t-1], and returns the realized sample for stage t.

This enables autoregressive, Markovian, or any custom temporal correlation between stages — something the data-based pool formats cannot express.

See sample for the full API and examples.

source
DecisionRules.sample — Method
sample(sampler::Function)

Call a zero-argument trajectory sampler that returns a complete trajectory.

This is the dispatch used by train_multistage and train_multiple_shooting when uncertainty_sampler is a callable. Wrap a trajectory sampler as:

uncertainty_sampler = () -> sample(my_stage_sampler, T)
source
DecisionRules.sample — Method
sample(uncertainty_pool) -> Vector{Vector{Tuple{VariableRef, T}}}

Draw one full uncertainty trajectory from a DecisionRules uncertainty pool.

The returned trajectory is a length-$T$ vector where each element is Vector{Tuple{VariableRef, Float64}} — one realized value per uncertain parameter for that stage. This is the format consumed by simulate_multistage, train_multistage, and all other training/evaluation functions.

Three pool formats are supported, offering increasing levels of correlation:

1. Independent sampling (per-unit pools)

Each uncertain parameter has its own finite support; sampling draws independently from each support at each stage.

sample(multistage_pool::Vector{Vector{Tuple{VariableRef, Vector{T}}}})

multistage_pool[t] is [(param₁, [v₁₁, v₁₂, …]), (param₂, [v₂₁, v₂₂, …]), …]. Each parameter picks one value uniformly at random from its own support. No spatial or temporal correlation is preserved.

2. Joint-scenario sampling (spatial correlation)

Scenarios are pre-defined joint realizations across all parameters at each stage. Sampling picks one complete scenario per stage uniformly, preserving cross-parameter correlations (e.g., spatially correlated inflows across hydro reservoirs). Stages are still drawn independently.

sample(multistage_joint::Vector{Vector{Vector{Tuple{VariableRef, T}}}})

multistage_joint[t] is [scenario₁, scenario₂, …] where each scenario is [(param₁, val₁), (param₂, val₂), …].

3. Trajectory sampler (spatial + temporal correlation)

A callable sampler(t, past) -> Vector{Tuple{VariableRef, T}} that generates stage t's realization given the realized values from stages 1:t-1. This enables autoregressive, Markovian, or any custom temporal dependence.

sample(sampler::Function, T::Int)

The callable receives:

  • t::Int — the current stage (1-indexed)
  • past::Vector{Vector{Tuple{VariableRef, T}}} — realized samples from stages 1:t-1 (empty vector for t=1)

and must return Vector{Tuple{VariableRef, T}} — the realized sample for stage t.

Output format

All three methods return Vector{Vector{Tuple{VariableRef, T}}} — a length-$T$ vector of per-stage realized samples. This is the universal input to simulate_multistage, train_multistage, simulate_multiple_shooting, and all evaluation functions.

Examples

# 1. Independent sampling (each unit draws independently):
independent_pool = [
    [(inflow_1, [10.0, 15.0, 12.0]), (inflow_2, [8.0, 12.0, 9.0])],
    [(inflow_1, [11.0, 14.0, 13.0]), (inflow_2, [7.0, 11.0, 10.0])],
]
path = sample(independent_pool)

# 2. Joint-scenario sampling (preserves spatial correlation):
joint_pool = [
    [[(inflow_1, 10.0), (inflow_2, 8.0)],   # scenario 1
     [(inflow_1, 15.0), (inflow_2, 12.0)]],  # scenario 2 — stage 1
    [[(inflow_1, 11.0), (inflow_2, 7.0)],
     [(inflow_1, 14.0), (inflow_2, 11.0)]],  # stage 2
]
path = sample(joint_pool)

# 3. Trajectory sampler (preserves temporal + spatial correlation):
function my_sampler(t, past)
    if t == 1
        ω = rand(1:nScenarios)
        return [(inflow_params[t][r], data[r][t, ω]) for r in 1:nHyd]
    else
        # AR(1): next inflow depends on previous realized inflow
        prev_values = [pair[2] for pair in past[end]]
        noise = randn(nHyd) .* σ
        return [(inflow_params[t][r], ρ * prev_values[r] + noise[r]) for r in 1:nHyd]
    end
end
path = sample(my_sampler, T)

See the Uncertainty Sampling documentation page for a complete guide.

source
DecisionRules.setup_shooting_windows — Method
setup_shooting_windows(subproblems, state_params_in, state_params_out, initial_state,
                       uncertainties; window_size, model_factory=() -> JuMP.Model())

Build window models for multiple shooting.

Notes:

  • We store only the uncertainty PARAMETER refs (not sample sets) in WindowData.
source
DecisionRules.sf_params — Method
sf_params(config::ScoreFunctionConfig, iteration::Integer)
sf_params(schedule::ScoreFunctionSchedule, iteration::Integer)

Return the effective score-function parameters for iteration.

Arguments

  • config::ScoreFunctionConfig: unscheduled score-function configuration.
  • schedule::ScoreFunctionSchedule: scheduled score-function configuration.
  • iteration::Integer: one-based training iteration.

Returns

A named tuple with fields:

  • alpha::Float64: weight on the dual-gradient term.
  • score_weight::Float64: weight on the score-function term.
  • perturbation_std::Float64: Gaussian standard deviation $\sigma$.
  • num_rollouts::Int: number of perturbed rollouts.
  • active::Bool: whether rollout estimation should run.

Examples

params = sf_params(schedule, 250)
params.active && @show params.score_weight
source
DecisionRules.simulate_multiple_shooting — Method
simulate_multiple_shooting(windows, decision_rule, initial_state, uncertainty_sample, uncertainties_vec)
  • uncertainty_sample: per-stage sampled tuples (param, value) matching your existing sampler output: Vector{Vector{Tuple{VariableRef,<:Real}}}
  • uncertainties_vec: per-stage vectors (Float32) used as policy inputs

Returns total objective across windows. Gradients flow through:

  • targets within each window (via solve_window rrule)
  • realized end state between windows (via solve_window rrule seeding on end vars)
source
DecisionRules.simulate_multistage — Method
simulate_multistage(det_equivalent::JuMP.Model, state_params_in, state_params_out,
                    initial_state, uncertainties, decision_rules) -> Float64

Convenience overload: rolls out decision_rules to produce target states, then calls the deterministic-equivalent simulate_multistage to solve the coupled problem.

source
DecisionRules.simulate_multistage — Method
simulate_multistage(subproblems, state_params_in, state_params_out,
                    initial_state, uncertainties, decision_rules) -> Float64

Stage-wise (single shooting) forward simulation. Rolls decision_rules over uncertainties, solving one subproblem per stage. The realized state from each stage feeds the next via get_next_state. Returns the total objective across all stages (Extension §2, Eq. 2.1–2.4).

source
DecisionRules.simulate_multistage — Method
simulate_multistage(det_equivalent, state_params_in, state_params_out,
                    uncertainties, states) -> Float64

Deterministic-equivalent (direct transcription) forward pass. Sets all parameter values from states and uncertainties into the coupled det_equivalent model, solves it, and returns the objective value (Extension §1, Eq. 1.1).

source
DecisionRules.simulate_stage — Method
simulate_stage(subproblem, state_param_in, state_param_out, uncertainty,
               state_in, state_out_target) -> Float64

Set parameter values on subproblem (incoming state, outgoing target, uncertainty), solve it, and return the objective value. Used as the inner solve in single-shooting rollouts (Extension §2, Eq. 2.1).

source
DecisionRules.simulate_states — Method
simulate_states(initial_state, uncertainties, decision_rule) -> Vector{Vector}

Roll out decision_rule over uncertainties to produce a target-state trajectory. At each stage the policy receives [uncertainty..., previous_state...] and outputs the next target state. Returns a length-(T+1) vector of states starting with initial_state.

source
DecisionRules.solve_window — Method
solve_window(window_model, window_state_in_params, window_state_out_params,
             s_in, targets)

Solve a deterministic-equivalent window model.

Arguments

  • window_model: JuMP model (DiffOpt-enabled) for the window
  • window_state_in_params: Vector of MOI.Parameter vars for window initial state
  • window_state_out_params: per-stage vector of tuples (targetparam, realizedvar)
  • s_in: numeric initial state
  • targets: vector of numeric targets, one per stage in the window

Returns

  • (objective, s_out): objective value, realized end state (Float32 vector)
source
DecisionRules.stage_phase_context — Method
stage_phase_context(T; period, include_progress=true)

Build a d x T context matrix encoding known calendar/stage information. Rows are sin(2*pi*t/period), cos(2*pi*t/period), and, when include_progress=true, t/T.

The sine/cosine pair represents a cyclic process without a discontinuity between the final and first period positions. A raw index would put those neighbors far apart; one-hot period features would be exact but high dimensional and would not encode adjacency. The optional progress feature has a different purpose: it tells the policy how close it is to the optimization horizon endpoint.

source
DecisionRules.state_conditioned_policy — Method
state_conditioned_policy(n_uncertainty, n_state, n_output, layers;
                         activation=Flux.relu, encoder_type=Flux.LSTM,
                         combiner_layers=Int[])

Create a StateConditionedPolicy with the specified architecture.

The resulting policy computes

\[h_t, s_t = \text{encoder}(w_t, s_{t-1}) \\ \hat{x}_t = \sigma\bigl(W [h_t;\; x_{t-1}] + b\bigr)\]

where the encoder is a stack of recurrent cells of width $(l_1, \dots, l_L)$ and the combiner has input dimension $l_L + n_{\text{state}}$. combiner_layers adds optional hidden layers to this nonrecurrent combiner, making the state-to-target map nonlinear while keeping recurrence confined to the uncertainty sequence. This distinction is important for TS-DDR hydro experiments: inflow history is recurrent, but the current reservoir state is used only as contemporaneous conditioning.

Arguments

  • n_uncertainty::Int: dimensionality of the uncertainty input $w_t$.
  • n_state::Int: dimensionality of the state $x_{t-1}$ (both input and output).
  • n_output::Int: output dimension (typically equal to n_state).
  • layers::Vector{Int}: hidden layer widths $(l_1, \dots, l_L)$ for the encoder.
  • activation: activation $\sigma$ for the combiner Dense layer (default: Flux.relu).
  • encoder_type: recurrent layer/cell constructor (LSTM, GRU, RNN, or their *Cell variants; default: Flux.LSTM). Must support Flux.initialstates and the stateful (x, state) -> (output, new_state) interface (Flux >= 0.16).
  • combiner_layers::Vector{Int}: hidden widths for the state-conditioned target head. The default Int[] preserves the original single Dense head.

Returns

  • StateConditionedPolicy: ready-to-use policy with initialized recurrent state.

Examples

policy = state_conditioned_policy(5, 3, 3, [16, 16])
x = randn(Float32, 8)  # [uncertainty(5); state(3)]
y = policy(x)           # predicted next state (length 3)

deep_head = state_conditioned_policy(
    5, 3, 3, [16, 16];
    activation = sigmoid,
    combiner_layers = [32, 32],
)
source
DecisionRules.train_multiple_shooting — Method
train_multiple_shooting(model, initial_state, windows, uncertainty_sampler; ...)

Train a target-state policy with multiple-shooting decomposition (windowed).

uncertainty_sampler controls how trajectories are drawn at each SGD step. Three formats are accepted (same API as train_multistage):

  1. Per-unit pool (Vector{Vector{Tuple{VariableRef, Vector{T}}}}): independent sampling per parameter per stage.
  2. Joint-scenario pool (Vector{Vector{Vector{Tuple{VariableRef, T}}}}): one scenario drawn per stage, preserving spatial correlation.
  3. Callable (() -> Vector{Vector{Tuple{VariableRef, T}}}): a zero-arg function returning a realized trajectory. Use this for temporal correlation; see sample.

See the Uncertainty Sampling documentation for details.

source
DecisionRules.train_multistage — Method
train_multistage(model, initial_state, det_equivalent::JuMP.Model,
                 state_params_in, state_params_out, uncertainty_sampler;
                 score_function=nothing, kwargs...)

Train a target-state policy with a deterministic equivalent (direct transcription).

For one sampled trajectory $w_{1:T}$, the policy first produces the full target trajectory

\[\hat{x}_{1:T}(\theta) = \pi_\theta(w_{1:T}, x_0).\]

The coupled implementation problem is

\[\begin{aligned} Q(w; \theta) = \min_{\{x_t, y_t, \delta_t\}_{t=1}^{T}} \quad & \sum_{t=1}^{T} f_t(x_t, y_t) + C_\delta \sum_{t=1}^{T} \|\delta_t\| \\ \text{s.t.}\quad & x_t = T_t(w_t, y_t, x_{t-1}) && t=1,\ldots,T, \\ & x_t + \delta_t = \hat{x}_t(\theta) && : \lambda_t,\quad t=1,\ldots,T, \\ & h_t(x_t, y_t) \ge 0 && t=1,\ldots,T . \end{aligned}\]

The target trajectory appears as right-hand-side parameters. If $\lambda_t$ is the dual multiplier of the target constraint, the envelope gradient used by this overload is

\[\nabla_\theta \mathbb{E}[Q(w; \theta)] \approx \frac{1}{S} \sum_{s=1}^{S} \sum_{t=1}^{T} \lambda_t^s \odot \nabla_\theta \hat{x}_t^s(\theta),\]

where $S$ is num_train_per_batch and $\odot$ denotes componentwise multiplication.

Pass a ScoreFunctionConfig or ScoreFunctionSchedule via score_function to mix the dual gradient with a REINFORCE correction estimated from rollouts under perturbed targets.

When score_function is used, there are two separate solve paths:

  1. integer_strategy applies to det_equivalent and controls how local dual information is read for the differentiable dual-gradient term.
  2. score_function owns separate rollout subproblems. Those models are solved exactly as they are built, and their realized costs define the Monte Carlo score-function term.

For a mixed-integer model, this usually means integer_strategy = FixedDiscreteIntegerStrategy() for the dual path and MIP rollout subproblems inside ScoreFunctionConfig for the score-function path.

Arguments

  • model: differentiable Flux-compatible policy. It is rolled forward over uncertainty values to produce $\hat{x}_{1:T}$.
  • initial_state::AbstractVector{<:Real}: state $x_0$.
  • det_equivalent::JuMP.Model: full-horizon JuMP model for one sampled trajectory.
  • state_params_in: input-state parameters in the deterministic equivalent.
  • state_params_out: (target_parameter, realized_state_variable) pairs for each target state.
  • uncertainty_sampler: source of uncertainty trajectories, passed to sample. Three formats are accepted:
    1. Per-unit pool (Vector{Vector{Tuple{VariableRef, Vector{T}}}}): independent sampling per parameter per stage.
    2. Joint-scenario pool (Vector{Vector{Vector{Tuple{VariableRef, T}}}}): one scenario drawn per stage, preserving spatial correlation.
    3. Callable (() -> Vector{Vector{Tuple{VariableRef, T}}}): a zero-arg function returning a full trajectory. Use this for temporal correlation; see sample.

Keywords

  • num_batches::Integer: number of SGD batches.
  • num_train_per_batch::Integer: sampled trajectories per batch $S$.
  • optimizer: Flux optimizer used to update model.
  • adjust_hyperparameters::Function: optional hook returning the batch size for the current iteration.
  • record_loss: legacy logging callback.
  • sample_log::SampleLog: per-batch objective cache.
  • record::Function: callback called as record(sample_log, iter, model).
  • penalty_schedule: optional multiplier schedule for target-penalty terms.
  • integer_strategy::AbstractIntegerStrategy: strategy used to read local dual information from det_equivalent when it has discrete variables.
  • score_function: optional ScoreFunctionConfig or ScoreFunctionSchedule for mixed dual/score-function gradients.

Examples

train_multistage(
    policy,
    initial_state,
    det_equivalent,
    state_params_in,
    state_params_out,
    uncertainty_sampler;
    num_batches = 200,
    num_train_per_batch = 16,
    optimizer = Flux.Adam(1.0e-3),
    integer_strategy = FixedDiscreteIntegerStrategy(),
    score_function = nothing,
)
source
DecisionRules.train_multistage — Method
train_multistage(model, initial_state, subproblems::Vector{JuMP.Model},
                 state_params_in, state_params_out, uncertainty_sampler;
                 kwargs...)

Train a target-state policy with stage-wise decomposition (single shooting).

For one sampled uncertainty trajectory $w_{1:T}$, this overload solves one optimization problem per stage. At stage $t$, given the realized incoming state $x_{t-1}$, the policy predicts a target $\hat{x}_t = \pi_\theta(w_t, x_{t-1})$ and the stage problem is

\[\begin{aligned} q_t(x_{t-1}, w_t; \hat{x}_t) = \min_{x_t, y_t, \delta_t} \quad & f_t(x_t, y_t) + C_\delta \|\delta_t\| \\ \text{s.t.}\quad & x_t = T_t(w_t, y_t, x_{t-1}) && : \mu_t, \\ & x_t + \delta_t = \hat{x}_t && : \lambda_t, \\ & h_t(x_t, y_t) \ge 0 . \end{aligned}\]

The rollout objective is the sum of stage values,

\[Q(\theta; w) = \sum_{t=1}^{T} q_t(x_{t-1}, w_t; \hat{x}_t),\]

where each realized $x_t$ is read from the previous stage solve. In this single-shooting rollout $\theta$ influences later stage costs through two couplings: the realized-state chain (the solver map $x_t = X_t(x_{t-1}, \hat{x}_t; w_t)$ propagates earlier targets forward), and the policy-feedback path (the policy input at stage $t$ is the realized state $x_{t-1}$, which itself depends on earlier targets). The exact gradient is the total-derivative recursion

\[\frac{d Q}{d \theta} = \sum_{t=1}^{T} \left[ \frac{\partial q_t}{\partial \hat{x}_t} \frac{d \hat{x}_t}{d \theta} + \frac{\partial q_t}{\partial x_{t-1}} \frac{d x_{t-1}}{d \theta} \right],\]

with the coupled state and target recursions (and $d x_0 / d\theta = 0$)

\[\frac{d x_t}{d \theta} = \frac{\partial X_t}{\partial x_{t-1}} \frac{d x_{t-1}}{d \theta} + \frac{\partial X_t}{\partial \hat{x}_t} \frac{d \hat{x}_t}{d \theta}, \qquad \frac{d \hat{x}_t}{d \theta} = \nabla_\theta \pi_\theta(w_t, x_{t-1}) + \frac{\partial \pi_\theta}{\partial x_{t-1}} \frac{d x_{t-1}}{d \theta}.\]

All derivative products must be read as total derivatives that include the policy-feedback composition: a term such as $\partial \pi_\theta / \partial x_{k-1} \cdot d x_{k-1} / d\theta$ is present at every stage, so $\theta$ reaches stage $k$ not only through the realized-state chain $\partial x_j / \partial x_{j-1}$ but also through the policy input $x_{k-1}$. Reverse-mode AD (Zygote through the stage rrules) computes exactly this full chain: the dual terms from target and transition constraints supply $\partial q_t / \partial \hat{x}_t$ and $\partial q_t / \partial x_{t-1}$, while the state sensitivities are computed through DiffOpt in the rrules for simulate_stage and get_next_state.

Arguments

  • model: differentiable Flux-compatible policy. It receives vcat(stage_uncertainty, realized_state) and returns the next target state.
  • initial_state::AbstractVector{<:Real}: state $x_0$ entering stage 1.
  • subproblems::Vector{JuMP.Model}: one JuMP model per stage.
  • state_params_in: stage input-state parameters.
  • state_params_out: (target_parameter, realized_state_variable) pairs for each stage output state.
  • uncertainty_sampler: source of uncertainty trajectories, passed to sample. Three formats are accepted:
    1. Per-unit pool (Vector{Vector{Tuple{VariableRef, Vector{T}}}}): independent sampling per parameter per stage.
    2. Joint-scenario pool (Vector{Vector{Vector{Tuple{VariableRef, T}}}}): one scenario drawn per stage, preserving spatial correlation.
    3. Callable (() -> Vector{Vector{Tuple{VariableRef, T}}}): a zero-arg function returning a full trajectory. Use this for temporal correlation by wrapping a trajectory sampler: () -> sample(my_stage_sampler, T) where my_stage_sampler(t, past) generates stage t conditioned on past realizations.

Keywords

  • num_batches::Integer: number of SGD batches.
  • num_train_per_batch::Integer: sampled trajectories per batch.
  • optimizer: Flux optimizer used to update model.
  • adjust_hyperparameters::Function: optional hook returning the batch size for the current iteration.
  • record_loss: legacy logging callback.
  • sample_log::SampleLog: per-batch objective cache.
  • record::Function: callback called as record(sample_log, iter, model).
  • penalty_schedule: optional multiplier schedule for target-penalty terms.
  • integer_strategy::AbstractIntegerStrategy: strategy used when a stage model has discrete variables and derivative information must be read.

Examples

# With data pool (independent or joint):
train_multistage(
    policy, initial_state, subproblems,
    state_params_in, state_params_out, uncertainty_pool;
    num_batches=200, optimizer=Flux.Adam(1e-3),
)

# With trajectory sampler (temporal correlation):
ar_sampler(t, past) = my_ar1_model(t, past, inflow_params)
train_multistage(
    policy, initial_state, subproblems,
    state_params_in, state_params_out,
    () -> sample(ar_sampler, T);
    num_batches=200, optimizer=Flux.Adam(1e-3),
)
source
DecisionRules.variable_to_parameter — Method
variable_to_parameter(model, variable; initial_value=0.0, deficit=nothing)

Replace a decision variable with an MOI.Parameter and bind them via an equality constraint. When deficit is provided, the constraint becomes variable + deficit == parameter and both the parameter and the deficit variable are returned.

source
DecisionRules.vcat_contexts — Method
vcat_contexts(a, b, ...)

Vertically concatenate context matrices after checking that they cover the same number of stages.

source

Internal Functions

DecisionRules.STRICT_GRADIENTS — Constant
STRICT_GRADIENTS

Global flag controlling gradient fallback behavior in rrules.

When false (default), rrule pullbacks return zero gradients with a warning when the solver terminates unsuccessfully — this keeps training alive when a few samples hit numerical trouble.

When true, the same situation throws an error instead. Enable this in tests to verify that controlled test cases never silently fall through to zero gradients:

DecisionRules.STRICT_GRADIENTS[] = true
Note

This flag controls the rrule-level fallback for bad solver status. For the training-loop-level fallback (DiffOpt assertion errors, etc.), use the gradient_fallback keyword in train_multistage and train_multiple_shooting.

source
ChainRulesCore.rrule — Method

ChainRulesCore.rrule for getlastrealized_state

Computes gradients w.r.t.:

  • s_in (window start numeric state)
  • targets (all target vectors in the window)

Given cotangents (Δs_out), we:

  • seed reverse variables (realized end state vars) with Δs_out
  • reverse_differentiate!
  • read reverse sensitivities w.r.t. windowstatein_params and all target params
source
ChainRulesCore.rrule — Method
ChainRulesCore.rrule(get_next_state, subproblem, state_param_in, state_param_out, state_in, state_out_target)

Correct reverse-mode rule using DiffOpt:

  • Seeds reverse on realized output variables with Δstate_out
  • Calls DiffOpt.reverse_differentiate!
  • Reads sensitivities wrt parameter vars (stateparamin, stateparamout parameters)
  • Returns VJP wrt the numeric inputs state_in and state_out_target

Assumptions:

  • subproblem is a JuMP.Model constructed with Model(() -> DiffOpt.diff_optimizer(...))
  • state_param_in::Vector{JuMP.VariableRef} are JuMP Parameter variables (incoming state parameters)
  • state_param_out::Vector{Tuple{JuMP.VariableRef,JuMP.VariableRef}} holds (target-Parameter variable, realized-state Variable) per component
  • get_next_state(...) updates parameter values, optimize!s, and returns a Vector matching the realized-state variables
source
ChainRulesCore.rrule — Method
ChainRulesCore.rrule(::typeof(simulate_multistage), det_equivalent, state_params_in,
                     state_params_out, uncertainties, states)

Reverse-mode rule for the deterministic-equivalent (full-horizon) solve.

Mathematical basis (TS-DDR, arXiv:2405.14973, Eq. 1.2; Extension §1)

For the coupled problem $Q(w;θ) = \min \sum_t f_t + C_δ\|δ_t\|$ the gradient estimator is:

∇_θ E[Q] ≈ (1/S) Σ_s  λ^s ⊙ ∇_θ π(·; θ)

where $λ_t$ is the dual of the target constraint $x_t + δ_t = \hat{x}_t$. The pullback returns $Δ_{states}$ such that $Δ_{states}[1]$ holds the parameter duals of the initial-state parameters and $Δ_{states}[t+1]$ holds the target-constraint duals $λ_t$ for each stage.

Fallback strategy

Same as simulate_stage: tries compute_parameter_dual first, falls back to DiffOpt reverse differentiation if pdual raises. Solver failure (bad termination status) returns zero gradients or throws depending on STRICT_GRADIENTS.

source
ChainRulesCore.rrule — Method
ChainRulesCore.rrule(::typeof(simulate_stage), subproblem, state_param_in,
                     state_param_out, uncertainty, state_in, state_out)

Reverse-mode rule for a single-stage subproblem solve.

Mathematical basis (TS-DDR, arXiv:2405.14973; Extension §2 Eq. 2.5)

For stage problem $q_t(x_{t-1}, w_t; \hat{x}_t)$, the sensitivities are:

∂q_t/∂(state_in)  = μ_t    (dual of dynamics constraint w.r.t. incoming state)
∂q_t/∂(target)     = λ_t    (dual of target constraint w.r.t. target x̂_t)

These are the Lagrange multipliers that compute_parameter_dual (pdual) extracts from the solved model. This is the preferred path: closed-form and exact whenever the solver exposes constraint duals.

Fallback strategy

  1. pdual (parameter duals) — tried first.
  2. DiffOpt reverse differentiation — if pdual raises (e.g. the optimizer wrapper does not expose conic duals). Computes the same sensitivities via implicit differentiation of the KKT system.
  3. Zero gradients — only when the solver terminated with an unsuccessful status (not OPTIMAL / ALMOSTOPTIMAL / LOCALLYSOLVED). A warning is emitted. Set DecisionRules.STRICT_GRADIENTS[] = true to throw instead.
source
ChainRulesCore.rrule — Method

ChainRulesCore.rrule for solve_window

Computes gradients w.r.t.:

  • s_in (window start numeric state)
  • targets (all target vectors in the window)

Given cotangents (Δobj_val), we:

  • seed reverse variables (objective and realized end state vars) with Δobj_val
  • reverse_differentiate!
  • read reverse sensitivities w.r.t. windowstatein_params and all target params
source
ChainRulesCore.rrule — Method

ChainRulesCore.rrule(::typeof(setwindowuncertainties!), window::WindowData, uncertainty_sample)

Declare setwindowuncertainties! as non-differentiable (mutates solver state).

source
DecisionRules._apply_deficit_penalty_multiplier! — Method
_apply_deficit_penalty_multiplier!(model::JuMP.Model, bases::Dict{VariableRef,Float64}, multiplier::Real) -> JuMP.Model
_apply_deficit_penalty_multiplier!(models::Vector{JuMP.Model}, bases::Vector, multiplier::Real) -> Vector{JuMP.Model}

Mutate model (or each model in models) in place, setting every deficit variable's objective coefficient to multiplier * base using the bases from _deficit_penalty_bases. Return the mutated model(s).

source
DecisionRules._as_float64_vec — Method
_as_float64_vec(val) -> Vector{Float64}

Coerce val to a Vector{Float64}. If val is already a Vector{Float64}, return it as-is; if it is another AbstractVector, convert element-wise; if it is a scalar, wrap it in a single-element vector.

source
DecisionRules._assert_successful_solve — Method
_assert_successful_solve(model::JuMP.Model; context::AbstractString = "solve")

Throw an error unless model terminated with an accepted success status.

Arguments

  • model::JuMP.Model: model whose termination status is checked.
  • context::AbstractString: human-readable phrase included in the error.

Examples

DecisionRules._assert_successful_solve(model; context = "fixed LP solve")
source
DecisionRules._center_rollout_costs — Method
_center_rollout_costs(costs::AbstractVector{<:Real}, baseline::Symbol)

Convert rollout costs into score-function advantages.

Arguments

  • costs::AbstractVector{<:Real}: operational costs from perturbed rollouts.
  • baseline::Symbol: either :mean or :none.

Examples

advantages = DecisionRules._center_rollout_costs([10.0, 12.0], :mean)
source
DecisionRules._create_like_variable — Method
_create_like_variable(m::JuMP.Model, src::JuMP.VariableRef, t::Int;
                      force_parameter=false) -> JuMP.VariableRef

Create a new variable in m that mirrors src. If src is a JuMP parameter (or force_parameter is true), the new variable is created as an MOI.Parameter initialized to _param_init_value(src); otherwise a free decision variable is created. The variable is named via var_set_name! with stage suffix t.

source
DecisionRules._deficit_penalty_bases — Method
_deficit_penalty_bases(model::JuMP.Model; deficit_name="norm_deficit") -> Dict{VariableRef,Float64}
_deficit_penalty_bases(models::Vector{JuMP.Model}; deficit_name="norm_deficit") -> Vector{Dict{VariableRef,Float64}}

Capture the current objective coefficient of every deficit variable as the multiplier base for _apply_deficit_penalty_multiplier!. A variable counts as a deficit variable if deficit_name occurs in its name and its linear objective coefficient (see _linear_objective_coefficient) is nonzero.

Must be called before any penalty_schedule multiplier is applied, so the captured coefficients reflect the as-built penalties.

source
DecisionRules._get_dual_from_objective — Method
_get_dual_from_objective(model::JuMP.Model, param::JuMP.VariableRef)

Get the dual contribution from the objective function. If parameter appears in the objective with coefficient c, contribution is:

  • +c for minimization
  • +c for maximization
source
DecisionRules._get_dual_from_quadratic_constraints — Method
_get_dual_from_quadratic_constraints(model::JuMP.Model, param::JuMP.VariableRef, S)

Get dual contribution from quadratic constraints of set type S. Handles both affine terms (parameter appears linearly) and quadratic terms (parameter appears in products pv or pp).

source
DecisionRules._get_parameter_coefficient_from_quadratic — Method
_get_parameter_coefficient_from_quadratic(expr::JuMP.GenericQuadExpr, param::JuMP.VariableRef)

Get the effective coefficient of a parameter from quadratic terms. For terms like coef * p * v, the effective coefficient is coef * value(v). For terms like coef * p * p, the effective coefficient is 2 * coef * value(p).

source
DecisionRules._linear_objective_coefficient — Method
_linear_objective_coefficient(model::JuMP.Model, variable::VariableRef) -> Float64

Return variable's linear coefficient in model's objective, or 0.0 if variable does not appear in it. Supports affine and quadratic objectives (quadratic terms are ignored); throw ArgumentError for any other objective type.

source
DecisionRules._param_init_value — Method
_param_init_value(src::JuMP.VariableRef) -> Float64

Return the current parameter value of src if it is a JuMP MOI.Parameter; otherwise return 0.0. Silently returns 0.0 if parameter_value throws (e.g., when the parameter has been deleted from the model).

source
DecisionRules._penalty_multiplier_for — Method
_penalty_multiplier_for(schedule, iter::Int) -> Float64

Return the multiplier of the phase containing batch iter. If iter is past the schedule's last phase, return that phase's multiplier (hold the final value steady).

source
DecisionRules._print_window_status_and_params — Method
_print_window_status_and_params(window, status; context="")

Print a diagnostic dump for a window solve that did not reach an optimal status.

Outputs: a primal feasibility report for window.model, the solver status, the window.stage_range, and a sorted listing of every JuMP parameter in the window model with its current value. The optional context string is included in the header for traceability.

source
DecisionRules._record_loss_adapter — Method
_record_loss_adapter(record_loss)

Adapt the deprecated 4-argument record_loss(iter, model, loss, tag) callback to the record(sample_log, iter, model) interface, reproducing the historical two-call contract: record_loss is called first with tag = "metrics/loss", and only if that returns false is it called again with tag = "metrics/training_loss". Return the result of whichever call is made last.

source
DecisionRules._remap_uncertainties — Method
_remap_uncertainties(uncertainties, var_src_to_dest, cons_to_cons)

Replace source-model VariableRef keys in an uncertainty pool with their destination-model counterparts (using the variable or constraint mapping built by deterministic_equivalent!).

Two methods dispatch on the pool format:

  • Per-unit pools (Vector{Vector{Tuple{VariableRef, Vector{T}}}}): each stage maps [(param₁, [v₁, …]), …] independently.
  • Joint-scenario pools (Vector{Vector{Vector{Tuple{VariableRef, T}}}}): each stage maps [[scenario₁…], [scenario₂…], …] preserving the grouped structure.

This is an internal helper; users interact with it indirectly through deterministic_equivalent!.

source
DecisionRules._reset_sample_log! — Method
_reset_sample_log!(sample_log::SampleLog) -> SampleLog
_reset_sample_log!(sample_log) -> typeof(sample_log)

Clear the per-batch cache of a SampleLog before the next training batch. For a SampleLog, delegates to Base.empty!; for any other type the call is a no-op and returns sample_log unchanged.

source
DecisionRules._resolve_penalty_schedule — Method
_resolve_penalty_schedule(penalty_schedule, num_batches::Int)

Resolve the penalty_schedule keyword of train_multistage and train_multiple_shooting into a Vector{Tuple{Int,Int,Float64}} of (first_batch, last_batch, multiplier) phases, or nothing if penalty scaling is disabled:

source
DecisionRules._resolve_record — Method
_resolve_record(record, record_loss)

Resolve the record/record_loss keywords of the training loops into a single per-batch callback. Return record unchanged if record_loss is nothing; otherwise require record === default_record and return _record_loss_adapter(record_loss). Throw ArgumentError if both a custom record and a record_loss are given.

source
DecisionRules._sample_target_perturbations — Method
_sample_target_perturbations(num_stages::Integer, state_dimension::Integer, sigma::Real)

Draw Gaussian target perturbations for one score-function rollout.

Arguments

  • num_stages::Integer: number of stage targets to perturb.
  • state_dimension::Integer: length of each target state vector.
  • sigma::Real: Gaussian standard deviation $\sigma$.

Examples

perturbations = DecisionRules._sample_target_perturbations(3, 2, 0.5)
source
DecisionRules._score_function_rollouts — Method
_score_function_rollouts(
    config::ScoreFunctionConfig,
    initial_state::AbstractVector,
    uncertainties,
    targets;
    perturbation_std = config.perturbation_std,
    num_rollouts = config.num_rollouts,
) -> (advantages, perturbations)

Estimate rollout advantages for the score-function term.

Arguments

  • config::ScoreFunctionConfig: score-function rollout configuration.
  • initial_state::AbstractVector: state entering stage 1.
  • uncertainties: sampled uncertainty trajectory.
  • targets: target trajectory, including the initial state.
  • perturbation_std::Real: Gaussian standard deviation $\sigma$.
  • num_rollouts::Integer: number of perturbed rollouts to sample.

Examples

advantages, perturbations = DecisionRules._score_function_rollouts(
    score_function,
    initial_state,
    uncertainty_sample,
    targets;
    perturbation_std = 0.5,
    num_rollouts = 4,
)
source
DecisionRules._score_function_surrogate — Method
_score_function_surrogate(
    advantage::Real,
    perturbations,
    targets,
    perturbation_std::Real,
) -> Real

Build the differentiable scalar whose gradient is the Gaussian score estimate.

For fixed rollout cost advantage $A$ and perturbations $\delta_t$, the surrogate is

\[A \sum_t \left\langle \delta_t / \sigma^2, \hat{x}_{t+1}(\theta) \right\rangle .\]

Arguments

  • advantage::Real: centered rollout cost $R - b$.
  • perturbations: stage perturbations $\delta_t$.
  • targets: differentiable target trajectory produced by the policy.
  • perturbation_std::Real: Gaussian standard deviation $\sigma$.

Examples

loss = DecisionRules._score_function_surrogate(
    3.0,
    perturbations,
    targets,
    0.5,
)
source
DecisionRules._sensitivity_forward_status — Method
_sensitivity_forward_status(model::JuMP.Model, strategy) -> MOI.TerminationStatusCode

Return the termination status that an rrule should use for gradient fallback.

Arguments

  • model::JuMP.Model: model inspected after the forward pass.
  • strategy::AbstractIntegerStrategy: integer strategy used for the solve.

Examples

status = DecisionRules._sensitivity_forward_status(model, strategy)
source
DecisionRules._set_multistage_parameters! — Method
_set_multistage_parameters!(state_params_in, state_params_out,
                            uncertainties, states) -> Nothing

Write MOI parameter values into a deterministic-equivalent JuMP model across all $T$ stages before solving.

For each stage $t = 1, \ldots, T$:

  • Initial state ($t = 1$ only): state_params_in[1] receives states[1] (the initial state $x_0$).
  • Uncertainty $w_t$: each (parameter, value) pair in uncertainties[t] is written.
  • Target $\hat{x}_t$: the target parameters in state_params_out[t] receive states[t + 1].

Note that state_params_in[t] for $t > 1$ is NOT set here because in the deterministic equivalent the incoming state is an internal variable linked by constraints, not a parameter.

Arguments

  • state_params_in: per-stage vectors of incoming-state parameters.
  • state_params_out: per-stage vectors of (target_parameter, state_variable).
  • uncertainties: per-stage (parameter, value) pairs for $w_t$.
  • states: length-$(T+1)$ target trajectory $[x_0, \hat{x}_1, \ldots, \hat{x}_T]$.
source
DecisionRules._set_score_function_stage_parameters! — Method
_set_score_function_stage_parameters!(
    state_params_in,
    state_params_out,
    uncertainties,
    state,
    target,
) -> Nothing

Set the JuMP parameters needed for one perturbed rollout stage.

Arguments

  • state_params_in::AbstractVector: parameters receiving the current state.
  • state_params_out::AbstractVector: (target_parameter, state_variable) pairs receiving the target state.
  • uncertainties::AbstractVector: (parameter, value) pairs for stage uncertainty.
  • state::AbstractVector{<:Real}: realized state entering this stage.
  • target::AbstractVector{<:Real}: perturbed target for the output state.

Examples

DecisionRules._set_score_function_stage_parameters!(
    spi[t],
    spo[t],
    uncertainty_sample[t],
    state,
    target,
)
source
DecisionRules._set_stage_parameters! — Method
_set_stage_parameters!(state_param_in, state_param_out, uncertainty,
                       state_in, state_out_target) -> Nothing

Write MOI parameter values into a single-stage JuMP subproblem before solving.

Three groups of parameters are set:

  1. Incoming state $x_{t-1}$: each element of state_param_in receives the corresponding entry of state_in.
  2. Uncertainty $w_t$: each (parameter, value) pair in uncertainty is written directly.
  3. Outgoing target $\hat{x}_t$: the first element of each tuple in state_param_out (the target parameter) receives the corresponding entry of state_out_target.

After this call the subproblem is ready for optimize!.

Arguments

  • state_param_in: JuMP parameter variables for the incoming state.
  • state_param_out: (target_parameter, realized_state_variable) pairs.
  • uncertainty: (parameter, value) pairs for stage uncertainty $w_t$.
  • state_in::AbstractVector{<:Real}: realized incoming state $x_{t-1}$.
  • state_out_target::AbstractVector{<:Real}: policy target $\hat{x}_t$.
source
DecisionRules._set_window_parameters! — Method
_set_window_parameters!(window_state_in_params, window_state_out_params,
                        s_in, targets) -> Nothing

Write numeric initial-state and target values into the JuMP MOI.Parameter variables of a window model before solving. s_in is broadcast into window_state_in_params; each element of targets is broadcast into the corresponding stage's target parameters in window_state_out_params.

source
DecisionRules._sf_config — Method
_sf_config(score_function) -> Union{Nothing,ScoreFunctionConfig}

Extract the underlying ScoreFunctionConfig, if one exists.

Arguments

  • score_function::Nothing: score-function correction is disabled.
  • score_function::ScoreFunctionConfig: returned as-is.
  • score_function::ScoreFunctionSchedule: unwraps score_function.config.

Examples

config = DecisionRules._sf_config(score_function)
source
DecisionRules._simulate_multistage_det — Method
_simulate_multistage_det(det_equivalent, state_params_in, state_params_out,
                         uncertainties, states, integer_strategy) -> Float64

Solve the deterministic-equivalent model for a single uncertainty trajectory and return the optimal objective.

Sets all parameters via _set_multistage_parameters!, solves the coupled full-horizon problem $Q(w; \theta)$ through with_sensitivity_solution, and caches both the full objective and the deficit-free operational cost in det_equivalent.ext for logging.

Arguments

  • det_equivalent::JuMP.Model: full-horizon coupled JuMP model.
  • state_params_in: per-stage incoming-state parameters.
  • state_params_out: per-stage (target_parameter, state_variable) pairs.
  • uncertainties: per-stage (parameter, value) pairs for $w_t$.
  • states: length-$(T+1)$ target trajectory from the policy.
  • integer_strategy::AbstractIntegerStrategy: discrete-variable strategy.
source
DecisionRules._simulate_multistage_det_with_parameter_duals — Method
_simulate_multistage_det_with_parameter_duals(det_equivalent, state_params_in,
                                               state_params_out, uncertainties,
                                               states, integer_strategy)
    -> (objective, Δ_states)

Solve the deterministic equivalent and extract parameter duals for the rrule pullback.

Like _simulate_multistage_det, this sets parameters and solves the full-horizon problem. In addition it reads the dual sensitivities via pdual for each stage:

\[\Delta_{\text{states}}[1] = \frac{\partial Q}{\partial x_0}, \qquad \Delta_{\text{states}}[t+1] = \lambda_t = \frac{\partial Q}{\partial \hat{x}_t}, \quad t = 1, \ldots, T.\]

These $\lambda_t$ duals are the envelope-theorem gradient used in the TS-DDR training objective (arXiv:2405.14973, Eq. 1.2).

Arguments

  • det_equivalent: full-horizon coupled JuMP model.
  • state_params_in: per-stage incoming-state parameters.
  • state_params_out: per-stage (target_parameter, state_variable) pairs.
  • uncertainties: per-stage (parameter, value) pairs for $w_t$.
  • states: length-$(T+1)$ target trajectory from the policy.
  • integer_strategy::AbstractIntegerStrategy: discrete-variable strategy.

Returns

  • objective::Float64: optimal coupled objective $Q(w; \theta)$.
  • Δ_states::Vector{Vector{Float64}}: length-$(T+1)$ vector of parameter duals; Δ_states[1] holds $\partial Q / \partial x_0$ and Δ_states[t+1] holds $\lambda_t$.
source
DecisionRules._simulate_multistage_target_feedback — Function
_simulate_multistage_target_feedback(subproblems, state_params_in,
    state_params_out, initial_state, uncertainties, decision_rules,
    integer_strategy) -> Float64

Run a stage-wise rollout where the target state (not the realized state) is fed back into the policy at each stage, matching the deterministic-equivalent target-generation semantics from simulate_states. Each stage subproblem is solved sequentially; the accumulated objective value (including target-deficit penalties) is returned.

Used by RolloutEvaluation when policy_state == :target.

source
DecisionRules._simulate_stage — Method
_simulate_stage(subproblem, state_param_in, state_param_out, uncertainty,
                state_in, state_out_target, integer_strategy) -> Float64

Forward-solve one stage of the multistage problem and return the objective.

Sets all parameters via _set_stage_parameters!, then solves subproblem through with_sensitivity_solution (which applies the integer_strategy for models with discrete variables). Returns the scalar optimal objective value $q_t(x_{t-1}, w_t; \hat{x}_t)$.

Arguments

  • subproblem::JuMP.Model: stage-$t$ JuMP model.
  • state_param_in: incoming-state parameters.
  • state_param_out: (target_parameter, realized_state_variable) pairs.
  • uncertainty: (parameter, value) pairs for $w_t$.
  • state_in: realized incoming state $x_{t-1}$.
  • state_out_target: policy target $\hat{x}_t$.
  • integer_strategy::AbstractIntegerStrategy: controls how discrete variables are handled during the solve.
source
DecisionRules._simulate_stage_with_parameter_duals — Method
_simulate_stage_with_parameter_duals(subproblem, state_param_in, state_param_out,
                                     uncertainty, state_in, state_out_target,
                                     integer_strategy)
    -> (objective, d_state_in, d_state_out_target)

Forward-solve one stage and extract parameter duals for the rrule pullback.

Like _simulate_stage, this sets parameters and solves the stage problem. In addition it reads the dual sensitivities via pdual:

\[\mu_t = \frac{\partial q_t}{\partial x_{t-1}}, \qquad \lambda_t = \frac{\partial q_t}{\partial \hat{x}_t}.\]

These duals are the preferred (closed-form) gradient path used by the simulate_stage rrule when parameter duals are available from the solver.

Arguments

  • subproblem: stage-$t$ JuMP model.
  • state_param_in: incoming-state parameters (yields $\mu_t$).
  • state_param_out: target parameters (yields $\lambda_t$).
  • uncertainty: (parameter, value) pairs for $w_t$.
  • state_in: realized incoming state $x_{t-1}$.
  • state_out_target: policy target $\hat{x}_t$.
  • integer_strategy::AbstractIntegerStrategy: discrete-variable strategy.

Returns

  • objective::Float64: optimal stage cost $q_t$.
  • d_state_in::Vector{Float64}: $\mu_t$ (sensitivities w.r.t. incoming state).
  • d_state_out_target::Vector{Float64}: $\lambda_t$ (sensitivities w.r.t. target).
source
DecisionRules._state_eltype — Method
_state_eltype(state) -> Type

Return the scalar element type of a recurrent state.

For nested tuple states (e.g. LSTM's (h, c) or a Chain's tuple of per-layer states) this recurses into the first element until it reaches an AbstractVector, then returns eltype(v). The result is used to cast inputs to the encoder's precision before each step.

Arguments

  • state::Tuple: nested recurrent state.
  • v::AbstractVector: leaf state vector.

Returns

  • Type: the scalar element type (e.g. Float32).
source
DecisionRules._step_encoder_layers — Method
_step_encoder_layers(layers, x, states) -> (output, new_states)

Recursively advance a tuple of recurrent layers by one time step.

Each layer receives the output of the previous layer as input and its own independent recurrent state. The base case (layers == ()) returns the input unchanged with an empty state tuple.

Arguments

  • layers::Tuple: remaining recurrent cells to evaluate.
  • x: current input (or output of the prior layer).
  • states::Tuple: per-layer recurrent states, same length as layers.

Returns

  • output: output of the last layer in layers.
  • new_states::Tuple: updated recurrent states, one per layer.
source
DecisionRules._target_deficit_penalty_value — Method
_target_deficit_penalty_value(obj, norm_deficit) -> Float64

Return the part of a JuMP objective expression that is attributed to target deficit variables. A variable is treated as a target-deficit variable when its JuMP name contains norm_deficit.

Proof sketch for the affine case: if the solved objective is q(x) + sum_i c_i d_i, and d_i are exactly the matched target-deficit variables, then the operational cost is the solved objective value minus sum_i c_i value(d_i).

Quadratic terms involving target-deficit variables are deliberately rejected. For such an objective, subtracting only the affine coefficient would not remove the whole penalty and would produce a silently biased operational cost.

source
DecisionRules._target_violation_share — Method
_target_violation_share(objective::Real, objective_no_deficit::Real) -> Float64

Return the target-violation share, (objective - objective_no_deficit) / objective. Return NaN if either input is nonfinite or abs(objective) <= 1e-12 (share undefined).

source
DecisionRules._total_objective_value — Method
_total_objective_value(model::JuMP.Model) -> Float64
_total_objective_value(models::Vector{JuMP.Model}) -> Float64

Return the objective value of model, falling back to the cached value model.ext[:_last_obj] (default 0.0) when the model is dirty or objective_value throws. The multi-model method sums across all models.

source
DecisionRules._validate_penalty_schedule — Method
_validate_penalty_schedule(schedule) -> typeof(schedule)

Validate an explicit penalty_schedule, a Vector of (first_batch, last_batch, multiplier) tuples: phases must be non-empty, contiguous, start at batch 1, satisfy first_batch <= last_batch, and have finite positive multipliers. Return schedule unchanged, or throw ArgumentError describing the first violation found.

source
DecisionRules._with_current_or_sensitivity_solution — Method
_with_current_or_sensitivity_solution(f, model, integer_strategy)

Run f(model) directly for continuous models and through with_sensitivity_solution for integer strategies.

Arguments

  • f::Function: callback that reads values, duals, or sensitivities.
  • model::JuMP.Model: model to inspect.
  • integer_strategy::AbstractIntegerStrategy: current integer strategy.

Examples

value = DecisionRules._with_current_or_sensitivity_solution(
    m -> JuMP.objective_value(m),
    model,
    strategy,
)
source
DecisionRules._zero_cotangents — Method
_zero_cotangents(n_in, n_out)

Create a tuple of zero/no tangents compatible with the get_next_state rrule pullback signature.

Used by handle_gradient_error to produce a safe, neutral gradient when the solver or DiffOpt differentiation fails. The returned tuple matches the cotangent layout expected by ChainRulesCore.rrule for get_next_state: four NoTangent() entries (for the function itself and non-differentiable arguments), followed by dense zero vectors for the state-in and state-out dimensions, and a trailing NoTangent().

Arguments

  • n_in::Int: dimension of the incoming state vector.
  • n_out::Int: dimension of the outgoing state vector.

Returns

A Tuple of NoTangent and Vector{Float64} elements that Zygote can propagate without error.

source
DecisionRules.add_child_model_exps! — Method
add_child_model_exps!(model, subproblem, var_src_to_dest, skip_parameter_refs,
                      state_params_out, state_params_in, t) -> Dict

Copy all constraints and the objective contribution from stage-t subproblem into the deterministic-equivalent model, remapping variables through var_src_to_dest via create_constraint and copy_and_replace_variables. Constraint-based state parameters and input parameters at t == 1 are updated to point to the new model's constraint refs. Parameter constraints for incoming states that are linked to the previous realized state are skipped via skip_parameter_refs; all other parameter constraints are copied. The subproblem objective is added to model's existing objective. Returns a Dict mapping source ConstraintRef to destination ConstraintRef.

source
DecisionRules.add_child_model_vars! — Method
add_child_model_vars!(model, subproblem, t, state_params_in, state_params_out,
                      initial_state, var_src_to_dest, skip_parameter_refs)
                      -> Dict{VariableRef,VariableRef}

Copy decision variables from stage-t subproblem into the deterministic- equivalent model, populating the source-to-destination mapping var_src_to_dest. State-coupling variables (incoming parameters and outgoing realized-state/target pairs) are handled specially: at t == 1 they become fresh parameters in model; at t > 1 incoming state parameters are linked to the previous stage's realized state variables. Each copied variable is renamed via var_set_name! with a #t suffix. Mutates state_params_in, state_params_out, var_src_to_dest, and skip_parameter_refs in place; returns var_src_to_dest.

source
DecisionRules.apply_rule — Method
apply_rule(stage::Int, decision_rule, uncertainty, state_in) -> Vector

Apply a single (shared) policy to produce the target state for stage stage.

The policy receives a concatenated input vector $[w_t^{(1)}, \ldots, w_t^{(n_w)}, x_{t-1}^{(1)}, \ldots, x_{t-1}^{(n_x)}]$ and returns the next target $\hat{x}_t = \pi_\theta(w_t, x_{t-1})$.

Arguments

  • stage::Int: current stage index (unused when a single rule is shared).
  • decision_rule: callable policy $\pi_\theta$.
  • uncertainty: (parameter, value) pairs for $w_t$; values are extracted.
  • state_in: realized incoming state $x_{t-1}$.
source
DecisionRules.apply_rule — Method
apply_rule(stage::Int, decision_rules::Vector, uncertainty, state_in) -> Vector

Apply a stage-specific policy from a vector of per-stage decision rules.

Dispatches to apply_rule(stage, decision_rules[stage], uncertainty, state_in), selecting the rule at index stage.

Arguments

  • stage::Int: current stage index, used to select decision_rules[stage].
  • decision_rules::Vector: one callable policy per stage.
  • uncertainty: (parameter, value) pairs for $w_t$.
  • state_in: realized incoming state $x_{t-1}$.
source
DecisionRules.copy_and_replace_variables — Method
copy_and_replace_variables(src, map::Dict{VariableRef,VariableRef})

Deep-copy a JuMP expression src, substituting every VariableRef key in map with its destination value. Dispatches on the concrete expression type:

  • Vector: element-wise recursive call.
  • Real: returned as-is (no variables to replace).
  • VariableRef: direct lookup in map.
  • GenericAffExpr: rebuild with remapped variable keys and same coefficients.
  • GenericQuadExpr: rebuild affine part and remapped UnorderedPair keys.
  • GenericNonlinearExpr: recursively remap arguments, then reconstruct via @expression.

Throws an error for unrecognized expression types.

source
DecisionRules.create_constraint — Method
create_constraint(model, obj, var_src_to_dest) -> ConstraintRef

Add a constraint to model whose function is obj.func with all VariableRef keys replaced via var_src_to_dest (see copy_and_replace_variables).

Four methods handle different constraint types:

  • Generic ScalarConstraint: uses @constraint(model, new_func in obj.set).
  • ScalarConstraint{NonlinearExpr, MOI.EqualTo}: new_func == obj.set.value.
  • ScalarConstraint{NonlinearExpr, MOI.LessThan}: new_func <= obj.set.upper.
  • ScalarConstraint{NonlinearExpr, MOI.GreaterThan}: new_func >= obj.set.lower.
source
DecisionRules.deterministic_equivalent! — Method
deterministic_equivalent!(model, subproblems, state_params_in, state_params_out,
                          initial_state, uncertainties)

Build the deterministic-equivalent (direct transcription) JuMP model by copying all stage subproblems into model. Variables are renamed with a #t suffix to avoid conflicts. Stage coupling is enforced by identifying the realized state variable of stage t with the incoming state parameter of stage t+1.

uncertainties accepts both sampling formats (see sample):

  • Per-unit pools: Vector{Vector{Tuple{VariableRef, Vector{T}}}} — one pool per parameter, drawing independently per parameter.
  • Joint-scenario pools: Vector{Vector{Vector{Tuple{VariableRef, T}}}} — pre-built joint scenarios preserving cross-parameter correlations.

Returns (model, uncertainties_new) where uncertainties_new has the same format as the input but with variable refs remapped to the deterministic-equivalent model. This function also remaps state_params_in and state_params_out in place. Copy those arrays before calling if you need to keep the original stage-wise references for rollout evaluation or later model construction.

source
DecisionRules.discrete_variables — Method
discrete_variables(model::JuMP.Model)

Return the binary or integer variables in model.

Arguments

  • model::JuMP.Model: model to inspect.

Examples

vars = DecisionRules.discrete_variables(model)
source
DecisionRules.extract_uncertainty_params — Method
extract_uncertainty_params(window_uncertainties) -> Vector{Vector{VariableRef}}

Extract the JuMP parameter VariableRefs from each stage of an uncertainty pool.

Handles three possible input shapes (automatically detected):

  • Already-extracted Vector{Vector{VariableRef}} — returned as-is.
  • Per-unit pool Vector{Vector{Tuple{VariableRef, Vector}}} — extracts the first element of each tuple.
  • Joint-scenario pool Vector{Vector{Vector{Tuple{VariableRef, T}}}} — extracts params from the first scenario of each stage (all scenarios share the same params).
source
DecisionRules.get_last_realized_state — Method
get_last_realized_state(window_model, window_state_in_params, window_state_out_params,
                        s_in, targets)

Get the realized end state from the window model after solving.

source
DecisionRules.get_next_state — Method
get_next_state(subproblem, state_param_in, state_param_out, state_in,
               state_out_target) -> Vector

Return the realized state from the most recent solve of subproblem by reading the values of the realized-state variables in state_param_out.

source
DecisionRules.get_objective_no_target_deficit — Method
get_objective_no_target_deficit(subproblem::JuMP.Model;
                               norm_deficit="norm_deficit") -> Float64

Compute the operational cost of a solved subproblem, excluding the target-deficit penalty.

The full objective includes a penalty $C_\delta \|\delta_t\|$ that penalizes deviations between realized and target states. This function strips those terms so that logged costs reflect true operational cost:

\[\text{cost}_t = q_t - \sum_{j \in \mathcal{D}} c_j \, \delta_j,\]

where $\mathcal{D}$ is the set of variables whose names contain norm_deficit.

If the model is dirty (parameters changed since last solve), returns the cached value from a previous successful call (stored in subproblem.ext[:_last_obj_no_deficit]). If no such value exists, or if the objective shape is unsupported, throws instead of inventing a cost.

Arguments

  • subproblem::JuMP.Model: a solved JuMP model.

Keywords

  • norm_deficit::AbstractString: substring matched against variable names to identify deficit-penalty terms.
source
DecisionRules.get_objective_no_target_deficit — Method
get_objective_no_target_deficit(subproblems::Vector{JuMP.Model};
                               norm_deficit="norm_deficit") -> Float64

Sum the deficit-free operational costs across all stage subproblems.

Calls the single-model get_objective_no_target_deficit on each element and returns the total.

Arguments

  • subproblems::Vector{JuMP.Model}: one solved JuMP model per stage.

Keywords

  • norm_deficit::AbstractString: substring matched against variable names to identify deficit-penalty terms.
source
DecisionRules.handle_gradient_error — Method
handle_gradient_error(fallback::AbstractGradientFallback, e, n_state_in, n_state_out)

Handle an exception raised inside the get_next_state rrule pullback.

This is the rrule-level extension point: it is called when the backward pass through a single stage fails (e.g., the solver returned an infeasible status and DiffOpt cannot differentiate). Concrete methods decide whether to absorb the error or propagate it.

Arguments

  • fallback::AbstractGradientFallback: dispatch tag controlling recovery behavior.
  • e: the caught exception.
  • n_state_in::Int: dimension of the incoming state vector (needed to build zero cotangents).
  • n_state_out::Int: dimension of the outgoing state vector.

Returns

A cotangent tuple (same layout as _zero_cotangents) when the error is absorbed, or does not return (re-throws) when the error is propagated.

See AbstractGradientFallback for how to implement custom subtypes.

source
DecisionRules.handle_rollout_error — Method
handle_rollout_error(fallback::AbstractGradientFallback, e, iter)

Handle an exception raised during a rollout evaluation scenario.

This is the rollout-level extension point: it is called when a single out-of-sample scenario fails during RolloutEvaluation (e.g., solver infeasibility on an unseen uncertainty sample). Absorbing the error skips that scenario and lets the evaluation continue with the remaining samples.

Arguments

  • fallback::AbstractGradientFallback: dispatch tag controlling recovery behavior.
  • e: the caught exception.
  • iter::Int: scenario index within the rollout batch.

Returns

  • true to skip this scenario and continue the rollout.
  • Does not return (re-throws) when the error should propagate.
source
DecisionRules.handle_training_error — Method
handle_training_error(fallback::AbstractGradientFallback, e, iter)

Handle an exception raised during a full training iteration (gradient computation and parameter update).

This is the training-loop-level extension point: it is called when Zygote.gradient or the subsequent optimizer update throws (e.g., a DiffOpt assertion error or a numerical issue in the loss computation). Unlike handle_gradient_error, which operates inside a single-stage rrule, this handler wraps the entire forward-backward pass for one iteration.

Arguments

  • fallback::AbstractGradientFallback: dispatch tag controlling recovery behavior.
  • e: the caught exception.
  • iter::Int: current training iteration index (used in log messages).

Returns

  • true to skip this iteration and continue training.
  • Does not return (re-throws) when the error should propagate.
source
DecisionRules.has_discrete_variables — Method
has_discrete_variables(model::JuMP.Model) -> Bool

Return whether model contains at least one binary or integer variable.

Arguments

  • model::JuMP.Model: model to inspect.

Examples

if DecisionRules.has_discrete_variables(model)
    @info "MIP model"
end
source
DecisionRules.pdual — Method
pdual(v::VariableRef) -> Float64

Compute $∂Q/∂p$ for a JuMP parameter variable $p$ in a solved model, where $Q$ is the optimal objective value. By the envelope theorem / Lagrangian duality this equals the sum of $-\text{coef} \times \text{dual}$ over all constraints where $p$ appears, plus the objective coefficient of $p$.

This is the key quantity in TS-DDR (arXiv:2405.14973): the dual $λ_t$ of the target constraint gives the sensitivity $∂Q/∂\hat{x}_t$ used in Eq. 1.2.

source
DecisionRules.rollout_with_perturbation — Method
rollout_with_perturbation(
    config::ScoreFunctionConfig,
    initial_state::AbstractVector,
    uncertainties,
    targets,
    perturbations,
) -> Float64

Run one stage-wise rollout with fixed target perturbations.

The rollout target at stage t is targets[t + 1] + perturbations[t]. The returned cost excludes the target-deficit penalty so the score-function signal estimates operational cost rather than target-following slack.

Arguments

  • config::ScoreFunctionConfig: rollout models and parameter mappings.
  • initial_state::AbstractVector: state entering stage 1.
  • uncertainties: sampled uncertainty trajectory.
  • targets: target trajectory, including targets[1] == initial_state.
  • perturbations: one perturbation vector for each stage target.

Examples

cost = DecisionRules.rollout_with_perturbation(
    score_function,
    initial_state,
    uncertainty_sample,
    targets,
    perturbations,
)
source
DecisionRules.set_window_uncertainties! — Method
set_window_uncertainties!(window, uncertainty_sample)

Set sampled (realized) uncertainty values into the window model's JuMP parameters.

uncertainty_sample is a realized trajectory (output of sample), so each stage is Vector{Tuple{VariableRef, Float64}} regardless of whether the original pool used independent or joint-scenario format.

source
DecisionRules.var_set_name! — Method
var_set_name!(src::JuMP.VariableRef, dest::JuMP.VariableRef, t::Int) -> Nothing

Name dest after src with a #t stage suffix. If src has a JuMP name, the result is "<name>#<t>"; otherwise the MOI variable index is used as fallback, producing "_[<index>]#<t>".

source
DecisionRules.windows_equivalent! — Method
windows_equivalent!(model, subproblems, state_params_in, state_params_out,
                     initial_state, uncertainties)

Build a coupled JuMP model for a contiguous window of stages by copying all variables, constraints, and objectives from subproblems into model. Stage coupling is enforced by identifying each stage's realized state variable with the next stage's incoming state parameter (same approach as deterministic_equivalent!, but scoped to a window).

uncertainties accepts both per-unit and joint-scenario pool formats (see sample). The returned uncertainties_new preserves the input format with variable refs remapped to the window model.

Returns (model, state_params_in_new, state_params_out_new, uncertainties_new).

source
DecisionRules.with_sensitivity_solution — Method
with_sensitivity_solution(f, model, integer_strategy)

Run f(model) while model is in a state suitable for reading duals or DiffOpt sensitivities.

Arguments

  • f::Function: callback that reads values, duals, or sensitivities.
  • model::JuMP.Model: model to solve and inspect.
  • integer_strategy::AbstractIntegerStrategy: strategy used to prepare models with binary or integer variables.

Examples

objective = with_sensitivity_solution(model, FixedDiscreteIntegerStrategy()) do m
    JuMP.objective_value(m)
end
source
Flux.reset! — Method
Flux.reset!(m::StateConditionedPolicy)

Reset the encoder's recurrent state to Flux.initialstates, e.g. before starting a new rollout.

source