API Reference
Public API
DecisionRules.AbstractGradientFallback — Type
AbstractGradientFallbackAbstract type governing what happens when a solver or differentiation error occurs during training.
DecisionRules ships two concrete subtypes:
| Type | Behavior |
|---|---|
ZeroGradientFallback | Log a warning, return zero gradients, continue training |
ErrorGradientFallback | Re-throw the error (useful in tests) |
Extending
Implement your own subtype to customize recovery:
struct MyFallback <: DecisionRules.AbstractGradientFallback end
function DecisionRules.handle_gradient_error(::MyFallback, e, n_state_in, n_state_out)
# e is the caught exception
# Return a tuple of cotangents (same shape as the rrule pullback) or rethrow
@error "Custom handler" exception=e
return DecisionRules._zero_cotangents(n_state_in, n_state_out)
end
function DecisionRules.handle_training_error(::MyFallback, e, iter)
# Return true to skip this iteration, false to rethrow
@error "Custom training handler" exception=e
return true # skip
end
function DecisionRules.handle_rollout_error(::MyFallback, e, iter)
# Return true to skip this scenario, false to rethrow
return true
endThen pass gradient_fallback=MyFallback() to train_multistage or train_multiple_shooting.
DecisionRules.AbstractIntegerStrategy — Type
AbstractIntegerStrategyAbstract supertype for strategies that prepare a JuMP model before reading duals or solver sensitivities.
Arguments
This abstract type has no fields. Concrete subtypes are passed as the integer_strategy::AbstractIntegerStrategy keyword to simulation and training functions.
Examples
simulate_multistage(
subproblems,
state_params_in,
state_params_out,
initial_state,
uncertainties,
policy;
integer_strategy = FixedDiscreteIntegerStrategy(),
)DecisionRules.ContextualPolicy — Type
ContextualPolicy(policy, context)Wrap a stage policy so each call receives known exogenous context before the usual policy input. The wrapped policy is called as policy(vcat(context_at(context, t), input)), where t is advanced once per call and reset by Flux.reset!.
This keeps training and rollout loops unchanged: context is data attached to the trajectory, while the inner policy remains the only trainable component. Matrix contexts are interpreted as d_context x T; function contexts are called as context(t).
DecisionRules.ContinuousRelaxationIntegerStrategy — Type
ContinuousRelaxationIntegerStrategy()Relax all binary/integer constraints to continuous bounds (binary → [0,1]), solve the resulting LP, and read duals in that relaxed state.
Mathematically, this replaces $z \in \{0,1\}$ or integer restrictions with continuous bounds before solving. The derivative signal belongs to the relaxed problem, not to the original MIP.
Compared to FixedDiscreteIntegerStrategy:
- Faster: one LP solve instead of MIP + LP.
- Smoother gradients: no integer fixing means no zero-gradient dead zones.
- Less accurate: the LP solution may have fractional integer variables, so the gradient does not correspond to any feasible integer assignment.
A practical pattern is to train with ContinuousRelaxationIntegerStrategy during warmup (smooth landscape for initial learning) and switch to FixedDiscreteIntegerStrategy later (integer-accurate gradients for fine-tuning).
Arguments
This type has no fields.
Examples
strategy = ContinuousRelaxationIntegerStrategy()DecisionRules.ErrorGradientFallback — Type
ErrorGradientFallback()Strict fallback: re-throw any solver or differentiation error. Use this in tests to ensure that controlled problems never silently produce zero gradients.
DecisionRules.FixedDiscreteIntegerStrategy — Type
FixedDiscreteIntegerStrategy()Solve a mixed-integer model, fix discrete variables to their incumbent values, relax integrality, re-solve, and read duals or sensitivities from the fixed continuous model.
If $z^*$ is the incumbent binary/integer solution, this strategy reads derivative-like information from the continuous problem
\[\min_x f(x, z^*) \quad \text{subject to} \quad g(x, z^*) \le 0.\]
The result is local to the incumbent integer assignment. It is not a differentiable MIP method.
Arguments
This type has no fields.
Examples
strategy = FixedDiscreteIntegerStrategy()DecisionRules.NoIntegerStrategy — Type
NoIntegerStrategy()Solve the model exactly as written before reading duals or sensitivities.
Use this for continuous LP, conic, or nonlinear models whose derivative information is available directly from the solved model.
Arguments
This type has no fields.
Examples
strategy = NoIntegerStrategy()DecisionRules.RolloutEvaluation — Type
RolloutEvaluation(subproblems, state_params_in, state_params_out, initial_state,
scenarios; stride=1, policy_state=:realized)Evaluation helper that assesses the policy with a stage-wise rollout (the deployment semantics of a target-trajectory policy) on a fixed held-out scenario set. Deterministic-equivalent evaluation re-optimizes all stages jointly and can absorb stage-wise-unfollowable targets through the slack penalty, silently overstating policy quality; the rollout metric is the guard that detects this.
policy_state controls what state is passed back into the policy:
:realizedpipes the previous realized state into the policy. This is the deployment/closed-loop rollout semantics.:targetpipes the previous target state into the policy, matching the deterministic-equivalent target-generation semantics fromsimulate_stateswhile still solving the stage subproblems sequentially.
scenarios must be a vector of materialized scenarios, sampled once before training (e.g. [DecisionRules.sample(uncertainty_samples) for _ in 1:n]), so every evaluation uses the same fixed set. subproblems may be the training subproblems (all stage parameters are rewritten on every solve) or a separately built copy; when training on a deterministic equivalent, pass the stage-wise subproblems here.
Call evaluation(iter, model), e.g. from within a record callback. Every stride calls it rolls the policy out over the fixed set and reports:
metrics/rollout_objective_no_deficit: the rollout objective excluding the target-slack penalty term (the operational cost), andmetrics/rollout_target_violation_share: the realized slack penalty divided by the full rollout objective (NaNwhen undefined).
Policy comparisons should only be trusted when the violation share is small (≤ ~0.05); a larger share means the policy's targets are not followable stage by stage and the reported cost is not what deployment would realize. The latest values are kept in last_objective_no_deficit / last_violation_share for custom logging. Calls on batches that are not a multiple of stride are a no-op and leave the cached values unchanged.
DecisionRules.RolloutEvaluation — Method
(evaluation::RolloutEvaluation)(iter, model) -> NothingEvaluate the policy on the held-out scenario set every stride iterations.
On active iterations (iter % stride == 0), rolls model out over all fixed scenarios using either closed-loop (:realized) or target-feedback (:target) semantics. Prints metrics/rollout_objective_no_deficit and metrics/rollout_target_violation_share, and caches the values in evaluation.last_objective_no_deficit / evaluation.last_violation_share. Scenarios that fail to solve are skipped with a warning if all fail.
DecisionRules.SampleLog — Type
SampleLog(; on_sample=(s, models, sample_log) -> nothing,
objective_no_deficit_fn=get_objective_no_target_deficit)Per-sample logger with local cache state for the training loops, patterned after SaveBest. During each batch the training loop calls sample_log(s, det_equivalent_or_subproblems) right after sample s has been simulated (successful solves only; a failed solve throws exactly as before). The default behavior caches, per sample, the full objective (objective_value) and the objective excluding the target-slack penalty term (objective_no_deficit_fn). The cache is cleared at the start of every batch and handed to the per-batch record(sample_log, iter, model) callback.
on_sample is an optional hook called as on_sample(s, models, sample_log) after the default caching. It receives the live JuMP model(s), so it can inspect termination statuses or dump the details of a suspicious sample for debugging — without paying any per-sample logging cost in the default configuration.
DecisionRules.SaveBest — Type
SaveBest(best_loss::Float64, model_path::String)Callback that saves the best policy state seen during training.
SaveBest is a small callable object used as a training callback. When called as callback(iter, model, loss), it compares loss with the best loss stored so far. If the new loss is smaller, it copies model to CPU, normalizes any recurrent layer state, and writes the Flux state to model_path with JLD2. It returns false, so it records checkpoints without stopping training.
Arguments
best_loss::Float64: incumbent loss. UseInfto save the first observed model.model_path::String: path of the JLD2 file that receives the best model state.
Examples
callback = SaveBest(Inf, "best_policy.jld2")
train_multistage(policy, x0, subproblems, state_in, state_out, sampler;
record = (log, iter, model) -> callback(iter, model, mean(log.losses)))DecisionRules.SaveBest — Method
(callback::SaveBest)(iter, model, loss) -> BoolCompare loss against the incumbent callback.best_loss. When loss is strictly smaller, copy model to CPU, normalize recurrent state via normalize_recur_state, and write the Flux state to callback.model_path with JLD2. Always returns false (never stops training).
DecisionRules.ScoreFunctionConfig — Type
ScoreFunctionConfig(
subproblems::AbstractVector{<:JuMP.Model},
state_params_in::AbstractVector,
state_params_out::AbstractVector;
dual_weight::Real = 0.5,
perturbation_std::Real = 1.0,
num_rollouts::Integer = 8,
baseline::Symbol = :mean,
)Configure the score-function correction used by train_multistage.
The deterministic-equivalent training path differentiates the target policy through dual information. For mixed-integer subproblems, those duals are local to a fixed integer assignment. This configuration adds a REINFORCE-style correction estimated from stage-wise rollouts with perturbed targets.
The rollout models are solved exactly as they are built. If subproblems contain binary variables, the score-function rollouts solve MIPs. If they contain relaxed variables, the score-function rollouts solve the relaxation. This is intentionally separate from the integer_strategy keyword of train_multistage, which controls only how the differentiable dual-gradient path reads local sensitivity information from the deterministic equivalent.
If $\hat{x}_{t+1}(\theta)$ is the target emitted by the policy and $\delta_t \sim \mathcal{N}(0, \sigma^2 I)$, the perturbed rollout solves with target $\hat{x}_{t+1}(\theta) + \delta_t$. The score-function surrogate loss is
\[L_{sf}(\theta) = \frac{1}{M} \sum_{m=1}^{M} (R_m - b) \sum_{t=1}^{T} \left\langle \frac{\delta_{m,t}}{\sigma^2}, \hat{x}_{t+1}(\theta) \right\rangle ,\]
and the mixed gradient is
\[\nabla L = \alpha \nabla L_{dual} + (1 - \alpha) \nabla L_{sf}.\]
Arguments
subproblems::AbstractVector{<:JuMP.Model}: stage-wise rollout models used to estimate realized costs under perturbed targets.state_params_in::AbstractVector: stage input-state parameters.state_params_out::AbstractVector: pairs(target_parameter, state_variable)for every stage output state.
Keywords
dual_weight::Real: mixing weight $\alpha$ on the dual-gradient term.perturbation_std::Real: Gaussian standard deviation $\sigma$.num_rollouts::Integer: number of perturbed rollouts $M$ per sample.baseline::Symbol: either:meanfor mean-centering costs or:none. Mean-centering reduces variance but, because the baseline is computed from the same $M$ rollouts, it introduces a small $O(1/M)$ bias (effectively scaling the estimator by $(M-1)/M$) that vanishes asnum_rolloutsgrows; a leave-one-out baseline would be exactly unbiased.
Examples
score_function = ScoreFunctionConfig(
rollout_subproblems,
state_params_in,
state_params_out;
dual_weight = 0.5,
perturbation_std = 1.0,
num_rollouts = 8,
)
train_multistage(
policy,
initial_state,
det_equivalent,
state_params_in,
state_params_out,
uncertainty_sampler;
score_function,
)DecisionRules.ScoreFunctionSchedule — Type
ScoreFunctionSchedule(config::ScoreFunctionConfig; <keyword arguments>)Ramp a ScoreFunctionConfig into training after a pure-dual warmup.
The schedule delays score-function rollouts until sf_start, then linearly increases the score-function weight, perturbation scale, and rollout count until the final values stored in config are reached.
Let $k$ be the current iteration and $\rho_k = \operatorname{clip}((k - k_0) / r, 0, 1)$. The effective score-function weight is $\rho_k (1 - \alpha)$. The effective dual weight is one minus that value.
Arguments
config::ScoreFunctionConfig: final score-function configuration.
Keywords
sf_start::Integer: first iteration at which score-function rollouts are considered.ramp_batches::Integer: number of iterations in the linear ramp.perturbation_std_initial::Real: initial $\sigma$ at ramp start.num_rollouts_initial::Integer: initial rollout count at ramp start.
Examples
schedule = ScoreFunctionSchedule(
score_function;
sf_start = 200,
ramp_batches = 300,
perturbation_std_initial = 0.1,
num_rollouts_initial = 2,
)DecisionRules.StallingCriterium — Type
StallingCriterium(patience::Int, best_loss::Float64, stall_count::Int)Early-stopping callback that halts training when the loss stalls.
Tracks the number of consecutive iterations without improvement. When stall_count reaches patience, the callback returns true to signal that training should stop. Use best_loss = Inf and stall_count = 0 for a fresh start.
Arguments
patience::Int: maximum consecutive non-improving iterations before stopping.best_loss::Float64: incumbent best loss. UseInfto accept the first value.stall_count::Int: current stall counter (typically initialized to0).
DecisionRules.StallingCriterium — Method
(callback::StallingCriterium)(iter, model, loss) -> BoolUpdate the stall counter and return true when stall_count >= patience.
If loss < best_loss, reset the counter to zero and update the incumbent. Otherwise increment stall_count. Returns true (stop training) once the patience budget is exhausted; false otherwise.
DecisionRules.StateConditionedPolicy — Type
StateConditionedPolicyA policy architecture that separates temporal encoding from state conditioning.
The encoder is a recurrent cell (LSTMCell/GRUCell/RNNCell, or a Chain of cells) that processes only the uncertainty sequence to capture temporal dependencies. The combiner is a feed-forward target head that merges the encoder output with the previous state to predict the next state.
Given uncertainty $w_t$ and previous state $x_{t-1}$, the forward pass is
\[h_t, s_t = \text{LSTM}(w_t, s_{t-1}) \\ \hat{x}_t = f_{\text{combine}}([h_t;\; x_{t-1}])\]
where $s_t$ is the hidden recurrent state carried across stages and $f_{\text{combine}}$ is a Dense layer with activation $\sigma$.
Flux's recurrent cells are stateless (Flux >= 0.16): each call returns (output, new_state) instead of mutating an internal Recur. StateConditionedPolicy therefore carries the encoder's recurrent state itself in state, threading it through one call per stage. Call Flux.reset! to clear it (back to Flux.initialstates) at the start of a rollout.
Fields
encoder::E: recurrent cell orChainof cells encoding uncertainty.combiner::C: feed-forward head mapping $[h_t;\; x_{t-1}]$ to $\hat{x}_t$.state::S: current recurrent state $s_t$, carried across calls.n_uncertainty::Int: dimensionality of $w_t$.n_state::Int: dimensionality of $x_{t-1}$.
Input format: [uncertainty..., previous_state...].
DecisionRules.StateConditionedPolicy — Method
(m::StateConditionedPolicy)(x) -> AbstractVectorExecute the forward pass of the state-conditioned policy.
The input x is split into the uncertainty portion $w_t$ and the previous state $x_{t-1}$. The forward pass computes
\[h_t, s_t = \text{encoder}(w_t, s_{t-1}) \\ \hat{x}_t = f_{\text{combine}}([h_t;\; x_{t-1}])\]
where $s_t$ is the updated recurrent state (stored in m.state for the next call) and $f_{\text{combine}}$ is a Dense layer.
Arguments
x::AbstractVector: concatenated input[w_t..., x_{t-1}...]of lengthm.n_uncertainty + m.n_state.
Returns
AbstractVector: predicted next state $\hat{x}_t$.
DecisionRules.WindowData — Type
WindowDataOne multiple-shooting window: the JuMP model of stages stage_range solved together, and handles to its parameters.
Fields
model::JuMP.Model: the window's optimization model.state_in_params: parameters fixing the state entering the window, $x_{t_0-1}$.state_out_params: for each stage of the window, the pairs(target_param, realized_var)linking the policy target $\hat x_t$ to the realized state $x_t$; the parameter duals oftarget_paramare the policy gradient signal.uncertainty_params: for each stage of the window, the parameters holding $w_t$.stage_range::UnitRange{Int}: the global stages $t_0, \dots, t_1$ this window covers.
DecisionRules.ZeroGradientFallback — Type
ZeroGradientFallback()Default fallback: log a warning and return zero gradients when the solver or DiffOpt differentiation fails. Training continues with a skipped update for that iteration.
DecisionRules.compute_parameter_dual — Method
compute_parameter_dual(model::JuMP.Model, param::JuMP.VariableRef)Compute the dual value (sensitivity) of a parameter in a solved JuMP model.
The parameter dual represents ∂(objective)/∂(parameter_value) and is computed by:
- Finding all constraints where the parameter appears
- For each constraint, computing: -coefficient * constraint_dual
- For the objective, adding the coefficient (or negative for maximization)
- Summing all contributions
This works for any solved model, not just convex ones, as long as dual values are available.
Arguments
model: A solved JuMP modelparam: A parameter variable (created with@variable(model, p in MOI.Parameter(value)))
Returns
- The dual value (sensitivity) of the parameter
Example
model = Model(HiGHS.Optimizer)
@variable(model, x >= 0)
@variable(model, p in MOI.Parameter(1.0))
@constraint(model, con, x >= 2 * p)
@objective(model, Min, 3 * x + p)
optimize!(model)
dual_p = compute_parameter_dual(model, p) # = 2 * dual(con) + 1 (constraint x - 2p >= 0 has coef -2 on p, contribution -coef*dual)
# Hand-check: at the optimum x* = 2p the objective is 7p, so d(obj)/dp = 7 = 2*3 + 1 with dual(con) = 3.Limitations
Only affine, quadratic, and vector-affine constraint functions are inspected: if the parameter appears in any other constraint function type (e.g. a nonlinear ScalarNonlinearFunction constraint), that contribution is skipped with a one-time warning and is NOT captured in the returned sensitivity.
The constraint-dual formula is validated for MIN_SENSE objectives; for MAX_SENSE models JuMP's dual sign conventions differ and this function's constraint contribution has not been validated.
DecisionRules.context_at — Method
context_at(context, t)Return the context vector for one-based stage t.
DecisionRules.create_deficit! — Method
create_deficit!(model::JuMP.Model, len::Int; penalty_l1=nothing, penalty_l2=nothing, penalty=nothing)Create deficit variables to penalize state deviations in a JuMP model.
Supports three modes controlled by the penalty keywords. Let $d \in \mathbb{R}^n$ be the deficit vector (len = n).
L1 norm only (default when no penalty keyword is given, or penalty_l1 alone):
\[\text{norm\_deficit} \geq \| d \|_1 = \sum_{i=1}^{n} |d_i|, \quad \text{objective} \mathrel{+}= \lambda_1 \cdot \text{norm\_deficit}.\]
Implemented via MOI.NormOneCone(1 + n).
L2 squared norm only (penalty_l2 alone):
\[\text{norm\_deficit} \geq \| d \|_2^2 = \sum_{i=1}^{n} d_i^2, \quad \text{objective} \mathrel{+}= \lambda_2 \cdot \text{norm\_deficit}.\]
Both norms (penalty_l1 and penalty_l2):
\[\text{norm\_deficit} \geq \lambda_1 \| d \|_1 + \lambda_2 \| d \|_2^2, \quad \text{objective} \mathrel{+}= 1 \cdot \text{norm\_deficit}.\]
Arguments
model: The JuMP model to add deficit variables to.len: Number of deficit variables (typically dimension of state).penalty_l1: Penalty coefficient $\lambda_1$ for the L1 norm (NormOneCone). Pass:autoto usemax |objective coefficients|.penalty_l2: Penalty coefficient $\lambda_2$ for the L2 squared norm (sum of squares). Pass:autoto usemax |objective coefficients|.penalty: Legacy argument. If provided andpenalty_l1/penalty_l2are bothnothing, uses this for L1 norm only.
Returns
norm_deficit: Single variable representing total penalized deviation (for logging compatibility)._deficit: Vector of deficit variables for each state dimension.
Examples
# L1 norm only (default behavior, backwards compatible)
norm_deficit, _deficit = create_deficit!(model, 3; penalty=1000.0)
# L2 norm only
norm_deficit, _deficit = create_deficit!(model, 3; penalty_l2=1000.0)
# Both L1 and L2 norms
norm_deficit, _deficit = create_deficit!(model, 3; penalty_l1=1000.0, penalty_l2=500.0)DecisionRules.default_annealed_schedule — Method
default_annealed_schedule(num_batches::Int)Build the default annealed target-penalty schedule over num_batches training batches: multipliers 0.1 -> 1.0 -> 10.0 -> 30.0 with phase lengths proportional to 2/2/4/16 of the horizon (the last phase takes the remainder; every phase keeps at least one batch). For num_batches < 4 the last num_batches multipliers are used, one batch each, so the run always ends at the strong-penalty phase.
Returns a Vector{Tuple{Int,Int,Float64}} of (first_batch, last_batch, multiplier) entries suitable for the penalty_schedule keyword of train_multistage and train_multiple_shooting. Multipliers are applied relative to the penalty the model was built with (the objective coefficient of the norm_deficit variables created by create_deficit!), so with penalty=:auto the effective penalty is multiplier * max |objective coefficient|.
DecisionRules.default_record — Method
default_record(sample_log, iter, model)Default per-batch recording callback: prints the same two per-batch lines as the historical record_loss default (metrics/loss = mean objective excluding the target-slack penalty, then metrics/training_loss = mean full objective) and returns false (training continues). Return true from a custom record to stop training.
DecisionRules.dense_multilayer_nn — Method
dense_multilayer_nn(num_inputs, num_outputs, layers; activation=Flux.relu, dense=Dense)Create a multi-layer neural network with the specified architecture.
Given hidden layer widths $(h_1, \dots, h_L)$ and activation $\sigma$, the resulting Chain computes
\[f(x) = W_{L+1} \, \sigma\bigl(W_L \cdots \sigma(W_1 x + b_1) \cdots + b_L\bigr) + b_{L+1},\]
where $W_k \in \mathbb{R}^{h_k \times h_{k-1}}$, $h_0 = \text{num\_inputs}$, and the final layer $W_{L+1} \in \mathbb{R}^{\text{num\_outputs} \times h_L}$ has no activation. When layers is empty, there is no hidden/output distinction: the returned single dense layer uses activation, preserving the same constructor semantics as Dense(num_inputs, num_outputs, activation).
If dense is a recurrent type (LSTM, GRU, RNN), pair notation in => out is used instead of (in, out, σ), and the last layer omits the activation so it can act as a linear projection.
Arguments
num_inputs::Int: number of input features $h_0$.num_outputs::Int: number of output features.layers::Vector{Int}: hidden layer widths $(h_1, \dots, h_L)$.activation: element-wise activation $\sigma$ (default:Flux.relu).dense: layer constructor (Dense,LSTM,GRU,RNN).
Returns
Chain(or a single layer whenlayersis empty).
Examples
mlp = dense_multilayer_nn(10, 3, [64, 32]) # 10 → 64 → 32 → 3
rnn = dense_multilayer_nn(5, 2, [16]; dense=LSTM) # 5 → 16 → 2 (LSTM)DecisionRules.find_variables — Method
find_variables(model::JuMP.Model, variable_name_parts::Vector{<:AbstractString})Return variables from model whose JuMP name contains all substrings in variable_name_parts. When the initial filter yields more than one variable, results are reordered by matching "<first_part>[i]" for i = 1, 2, ... to produce a consistently indexed vector.
DecisionRules.materialize_tangent — Method
materialize_tangent(x::AbstractArray) -> AbstractArrayReturn array tangents unchanged (arrays are already Flux.update!-compatible).
DecisionRules.materialize_tangent — Method
materialize_tangent(ref::Base.RefValue)Dereference a RefValue wrapper (used inside MutableTangent fields) and recurse on the contained value.
DecisionRules.materialize_tangent — Method
materialize_tangent(t::ChainRulesCore.MutableTangent) -> NamedTupleUnwrap a MutableTangent (produced by Zygote for mutable structs such as Flux.Recur) by extracting its backing NamedTuple and recursing.
DecisionRules.materialize_tangent — Method
materialize_tangent(::ChainRulesCore.NoTangent) -> NothingMap NoTangent (structural zeros for non-differentiable fields) to nothing.
DecisionRules.materialize_tangent — Method
materialize_tangent(t::ChainRulesCore.Tangent) -> NamedTupleUnwrap an immutable Tangent by extracting its backing and recursing.
DecisionRules.materialize_tangent — Method
materialize_tangent(::ChainRulesCore.ZeroTangent) -> NothingMap ZeroTangent (additive identity in the tangent space) to nothing.
DecisionRules.materialize_tangent — Method
materialize_tangent(::Nothing) -> NothingMap nothing tangents (unused parameters) to nothing.
DecisionRules.materialize_tangent — Method
materialize_tangent(x::Number) -> NumberReturn numeric tangents unchanged (leaf values are already plain scalars).
DecisionRules.materialize_tangent — Method
materialize_tangent(t::Tuple) -> TupleRecurse through every element of a Tuple, materializing each value.
DecisionRules.materialize_tangent — Method
materialize_tangent(nt::NamedTuple{K}) -> NamedTuple{K}Recurse through every field of a NamedTuple, materializing each value.
DecisionRules.normalize_recur_state — Method
normalize_recur_state(state)Return a copy of a Flux.state object where any Recur-like nodes have their state field set to cell.state0. This avoids Flux.loadmodel! tie errors when loading into freshly constructed recurrent layers.
DecisionRules.policy_input_dim — Method
policy_input_dim(num_uncertainties, num_states)Compute the input dimension for a policy network.
Policy networks receive $[w_t;\; x_{t-1}]$ as input, so the required dimension is
\[d_{\text{in}} = \dim(w_t) + \dim(x_{t-1}).\]
This format is consistent between subproblem and deterministic-equivalent formulations, enabling warm-starting policies trained with det_eq for use with subproblems.
Arguments
num_uncertainties::Int: dimensionality of the stage uncertainty $w_t$.num_states::Int: dimensionality of the state $x_{t-1}$.
Returns
Int: total input dimension $d_{\text{in}}$.
Examples
d = policy_input_dim(5, 3) # 8DecisionRules.policy_input_dim — Method
policy_input_dim(uncertainty_samples, initial_state)Compute the input dimension for a policy network from problem data.
Infers $\dim(w_t)$ from the first uncertainty sample and $\dim(x)$ from the initial state vector, then delegates to policy_input_dim(::Int, ::Int).
Arguments
uncertainty_samples::Vector: uncertainty samples;length(uncertainty_samples[1])gives $\dim(w_t)$.initial_state::Vector: initial state vector;length(initial_state)gives $\dim(x)$.
Returns
Int: total input dimension $d_{\text{in}}$.
Examples
d = policy_input_dim(uncertainty_samples, x0)DecisionRules.predict_window_targets — Method
predict_window_targets(decision_rule, s_in, uncertainties_vec)Predict one target per stage in a window. This is an AD-friendly scan: target1 = π([u1; sin]) target2 = π([u2; target1]) ...
DecisionRules.sample — Method
sample(sampler::Function, T::Int)Draw a full trajectory using a callable trajectory sampler with temporal dependence.
sampler(t, past) receives the current stage t and a vector of all previously realized samples past[1:t-1], and returns the realized sample for stage t.
This enables autoregressive, Markovian, or any custom temporal correlation between stages — something the data-based pool formats cannot express.
See sample for the full API and examples.
DecisionRules.sample — Method
sample(sampler::Function)Call a zero-argument trajectory sampler that returns a complete trajectory.
This is the dispatch used by train_multistage and train_multiple_shooting when uncertainty_sampler is a callable. Wrap a trajectory sampler as:
uncertainty_sampler = () -> sample(my_stage_sampler, T)DecisionRules.sample — Method
sample(uncertainty_pool) -> Vector{Vector{Tuple{VariableRef, T}}}Draw one full uncertainty trajectory from a DecisionRules uncertainty pool.
The returned trajectory is a length-$T$ vector where each element is Vector{Tuple{VariableRef, Float64}} — one realized value per uncertain parameter for that stage. This is the format consumed by simulate_multistage, train_multistage, and all other training/evaluation functions.
Three pool formats are supported, offering increasing levels of correlation:
1. Independent sampling (per-unit pools)
Each uncertain parameter has its own finite support; sampling draws independently from each support at each stage.
sample(multistage_pool::Vector{Vector{Tuple{VariableRef, Vector{T}}}})multistage_pool[t] is [(param₁, [v₁₁, v₁₂, …]), (param₂, [v₂₁, v₂₂, …]), …]. Each parameter picks one value uniformly at random from its own support. No spatial or temporal correlation is preserved.
2. Joint-scenario sampling (spatial correlation)
Scenarios are pre-defined joint realizations across all parameters at each stage. Sampling picks one complete scenario per stage uniformly, preserving cross-parameter correlations (e.g., spatially correlated inflows across hydro reservoirs). Stages are still drawn independently.
sample(multistage_joint::Vector{Vector{Vector{Tuple{VariableRef, T}}}})multistage_joint[t] is [scenario₁, scenario₂, …] where each scenario is [(param₁, val₁), (param₂, val₂), …].
3. Trajectory sampler (spatial + temporal correlation)
A callable sampler(t, past) -> Vector{Tuple{VariableRef, T}} that generates stage t's realization given the realized values from stages 1:t-1. This enables autoregressive, Markovian, or any custom temporal dependence.
sample(sampler::Function, T::Int)The callable receives:
t::Int— the current stage (1-indexed)past::Vector{Vector{Tuple{VariableRef, T}}}— realized samples from stages1:t-1(empty vector fort=1)
and must return Vector{Tuple{VariableRef, T}} — the realized sample for stage t.
Output format
All three methods return Vector{Vector{Tuple{VariableRef, T}}} — a length-$T$ vector of per-stage realized samples. This is the universal input to simulate_multistage, train_multistage, simulate_multiple_shooting, and all evaluation functions.
Examples
# 1. Independent sampling (each unit draws independently):
independent_pool = [
[(inflow_1, [10.0, 15.0, 12.0]), (inflow_2, [8.0, 12.0, 9.0])],
[(inflow_1, [11.0, 14.0, 13.0]), (inflow_2, [7.0, 11.0, 10.0])],
]
path = sample(independent_pool)
# 2. Joint-scenario sampling (preserves spatial correlation):
joint_pool = [
[[(inflow_1, 10.0), (inflow_2, 8.0)], # scenario 1
[(inflow_1, 15.0), (inflow_2, 12.0)]], # scenario 2 — stage 1
[[(inflow_1, 11.0), (inflow_2, 7.0)],
[(inflow_1, 14.0), (inflow_2, 11.0)]], # stage 2
]
path = sample(joint_pool)
# 3. Trajectory sampler (preserves temporal + spatial correlation):
function my_sampler(t, past)
if t == 1
ω = rand(1:nScenarios)
return [(inflow_params[t][r], data[r][t, ω]) for r in 1:nHyd]
else
# AR(1): next inflow depends on previous realized inflow
prev_values = [pair[2] for pair in past[end]]
noise = randn(nHyd) .* σ
return [(inflow_params[t][r], ρ * prev_values[r] + noise[r]) for r in 1:nHyd]
end
end
path = sample(my_sampler, T)See the Uncertainty Sampling documentation page for a complete guide.
DecisionRules.setup_shooting_windows — Method
setup_shooting_windows(subproblems, state_params_in, state_params_out, initial_state,
uncertainties; window_size, model_factory=() -> JuMP.Model())Build window models for multiple shooting.
Notes:
- We store only the uncertainty PARAMETER refs (not sample sets) in WindowData.
DecisionRules.sf_params — Method
sf_params(config::ScoreFunctionConfig, iteration::Integer)
sf_params(schedule::ScoreFunctionSchedule, iteration::Integer)Return the effective score-function parameters for iteration.
Arguments
config::ScoreFunctionConfig: unscheduled score-function configuration.schedule::ScoreFunctionSchedule: scheduled score-function configuration.iteration::Integer: one-based training iteration.
Returns
A named tuple with fields:
alpha::Float64: weight on the dual-gradient term.score_weight::Float64: weight on the score-function term.perturbation_std::Float64: Gaussian standard deviation $\sigma$.num_rollouts::Int: number of perturbed rollouts.active::Bool: whether rollout estimation should run.
Examples
params = sf_params(schedule, 250)
params.active && @show params.score_weightDecisionRules.simulate_multiple_shooting — Method
simulate_multiple_shooting(windows, decision_rule, initial_state, uncertainty_sample, uncertainties_vec)uncertainty_sample: per-stage sampled tuples (param, value) matching your existing sampler output: Vector{Vector{Tuple{VariableRef,<:Real}}}uncertainties_vec: per-stage vectors (Float32) used as policy inputs
Returns total objective across windows. Gradients flow through:
- targets within each window (via solve_window rrule)
- realized end state between windows (via solve_window rrule seeding on end vars)
DecisionRules.simulate_multistage — Method
simulate_multistage(det_equivalent::JuMP.Model, state_params_in, state_params_out,
initial_state, uncertainties, decision_rules) -> Float64Convenience overload: rolls out decision_rules to produce target states, then calls the deterministic-equivalent simulate_multistage to solve the coupled problem.
DecisionRules.simulate_multistage — Method
simulate_multistage(subproblems, state_params_in, state_params_out,
initial_state, uncertainties, decision_rules) -> Float64Stage-wise (single shooting) forward simulation. Rolls decision_rules over uncertainties, solving one subproblem per stage. The realized state from each stage feeds the next via get_next_state. Returns the total objective across all stages (Extension §2, Eq. 2.1–2.4).
DecisionRules.simulate_multistage — Method
simulate_multistage(det_equivalent, state_params_in, state_params_out,
uncertainties, states) -> Float64Deterministic-equivalent (direct transcription) forward pass. Sets all parameter values from states and uncertainties into the coupled det_equivalent model, solves it, and returns the objective value (Extension §1, Eq. 1.1).
DecisionRules.simulate_stage — Method
simulate_stage(subproblem, state_param_in, state_param_out, uncertainty,
state_in, state_out_target) -> Float64Set parameter values on subproblem (incoming state, outgoing target, uncertainty), solve it, and return the objective value. Used as the inner solve in single-shooting rollouts (Extension §2, Eq. 2.1).
DecisionRules.simulate_states — Method
simulate_states(initial_state, uncertainties, decision_rule) -> Vector{Vector}Roll out decision_rule over uncertainties to produce a target-state trajectory. At each stage the policy receives [uncertainty..., previous_state...] and outputs the next target state. Returns a length-(T+1) vector of states starting with initial_state.
DecisionRules.solve_window — Method
solve_window(window_model, window_state_in_params, window_state_out_params,
s_in, targets)Solve a deterministic-equivalent window model.
Arguments
window_model: JuMP model (DiffOpt-enabled) for the windowwindow_state_in_params: Vector of MOI.Parameter vars for window initial statewindow_state_out_params: per-stage vector of tuples (targetparam, realizedvar)s_in: numeric initial statetargets: vector of numeric targets, one per stage in the window
Returns
- (objective, s_out): objective value, realized end state (Float32 vector)
DecisionRules.stage_phase_context — Method
stage_phase_context(T; period, include_progress=true)Build a d x T context matrix encoding known calendar/stage information. Rows are sin(2*pi*t/period), cos(2*pi*t/period), and, when include_progress=true, t/T.
The sine/cosine pair represents a cyclic process without a discontinuity between the final and first period positions. A raw index would put those neighbors far apart; one-hot period features would be exact but high dimensional and would not encode adjacency. The optional progress feature has a different purpose: it tells the policy how close it is to the optimization horizon endpoint.
DecisionRules.state_conditioned_policy — Method
state_conditioned_policy(n_uncertainty, n_state, n_output, layers;
activation=Flux.relu, encoder_type=Flux.LSTM,
combiner_layers=Int[])Create a StateConditionedPolicy with the specified architecture.
The resulting policy computes
\[h_t, s_t = \text{encoder}(w_t, s_{t-1}) \\ \hat{x}_t = \sigma\bigl(W [h_t;\; x_{t-1}] + b\bigr)\]
where the encoder is a stack of recurrent cells of width $(l_1, \dots, l_L)$ and the combiner has input dimension $l_L + n_{\text{state}}$. combiner_layers adds optional hidden layers to this nonrecurrent combiner, making the state-to-target map nonlinear while keeping recurrence confined to the uncertainty sequence. This distinction is important for TS-DDR hydro experiments: inflow history is recurrent, but the current reservoir state is used only as contemporaneous conditioning.
Arguments
n_uncertainty::Int: dimensionality of the uncertainty input $w_t$.n_state::Int: dimensionality of the state $x_{t-1}$ (both input and output).n_output::Int: output dimension (typically equal ton_state).layers::Vector{Int}: hidden layer widths $(l_1, \dots, l_L)$ for the encoder.activation: activation $\sigma$ for the combinerDenselayer (default:Flux.relu).encoder_type: recurrent layer/cell constructor (LSTM,GRU,RNN, or their*Cellvariants; default:Flux.LSTM). Must supportFlux.initialstatesand the stateful(x, state) -> (output, new_state)interface (Flux >= 0.16).combiner_layers::Vector{Int}: hidden widths for the state-conditioned target head. The defaultInt[]preserves the original single Dense head.
Returns
StateConditionedPolicy: ready-to-use policy with initialized recurrent state.
Examples
policy = state_conditioned_policy(5, 3, 3, [16, 16])
x = randn(Float32, 8) # [uncertainty(5); state(3)]
y = policy(x) # predicted next state (length 3)
deep_head = state_conditioned_policy(
5, 3, 3, [16, 16];
activation = sigmoid,
combiner_layers = [32, 32],
)DecisionRules.train_multiple_shooting — Method
train_multiple_shooting(model, initial_state, windows, uncertainty_sampler; ...)Train a target-state policy with multiple-shooting decomposition (windowed).
uncertainty_sampler controls how trajectories are drawn at each SGD step. Three formats are accepted (same API as train_multistage):
- Per-unit pool (
Vector{Vector{Tuple{VariableRef, Vector{T}}}}): independent sampling per parameter per stage. - Joint-scenario pool (
Vector{Vector{Vector{Tuple{VariableRef, T}}}}): one scenario drawn per stage, preserving spatial correlation. - Callable (
() -> Vector{Vector{Tuple{VariableRef, T}}}): a zero-arg function returning a realized trajectory. Use this for temporal correlation; seesample.
See the Uncertainty Sampling documentation for details.
DecisionRules.train_multistage — Method
train_multistage(model, initial_state, det_equivalent::JuMP.Model,
state_params_in, state_params_out, uncertainty_sampler;
score_function=nothing, kwargs...)Train a target-state policy with a deterministic equivalent (direct transcription).
For one sampled trajectory $w_{1:T}$, the policy first produces the full target trajectory
\[\hat{x}_{1:T}(\theta) = \pi_\theta(w_{1:T}, x_0).\]
The coupled implementation problem is
\[\begin{aligned} Q(w; \theta) = \min_{\{x_t, y_t, \delta_t\}_{t=1}^{T}} \quad & \sum_{t=1}^{T} f_t(x_t, y_t) + C_\delta \sum_{t=1}^{T} \|\delta_t\| \\ \text{s.t.}\quad & x_t = T_t(w_t, y_t, x_{t-1}) && t=1,\ldots,T, \\ & x_t + \delta_t = \hat{x}_t(\theta) && : \lambda_t,\quad t=1,\ldots,T, \\ & h_t(x_t, y_t) \ge 0 && t=1,\ldots,T . \end{aligned}\]
The target trajectory appears as right-hand-side parameters. If $\lambda_t$ is the dual multiplier of the target constraint, the envelope gradient used by this overload is
\[\nabla_\theta \mathbb{E}[Q(w; \theta)] \approx \frac{1}{S} \sum_{s=1}^{S} \sum_{t=1}^{T} \lambda_t^s \odot \nabla_\theta \hat{x}_t^s(\theta),\]
where $S$ is num_train_per_batch and $\odot$ denotes componentwise multiplication.
Pass a ScoreFunctionConfig or ScoreFunctionSchedule via score_function to mix the dual gradient with a REINFORCE correction estimated from rollouts under perturbed targets.
When score_function is used, there are two separate solve paths:
integer_strategyapplies todet_equivalentand controls how local dual information is read for the differentiable dual-gradient term.score_functionowns separate rollout subproblems. Those models are solved exactly as they are built, and their realized costs define the Monte Carlo score-function term.
For a mixed-integer model, this usually means integer_strategy = FixedDiscreteIntegerStrategy() for the dual path and MIP rollout subproblems inside ScoreFunctionConfig for the score-function path.
Arguments
model: differentiable Flux-compatible policy. It is rolled forward over uncertainty values to produce $\hat{x}_{1:T}$.initial_state::AbstractVector{<:Real}: state $x_0$.det_equivalent::JuMP.Model: full-horizon JuMP model for one sampled trajectory.state_params_in: input-state parameters in the deterministic equivalent.state_params_out:(target_parameter, realized_state_variable)pairs for each target state.uncertainty_sampler: source of uncertainty trajectories, passed tosample. Three formats are accepted:- Per-unit pool (
Vector{Vector{Tuple{VariableRef, Vector{T}}}}): independent sampling per parameter per stage. - Joint-scenario pool (
Vector{Vector{Vector{Tuple{VariableRef, T}}}}): one scenario drawn per stage, preserving spatial correlation. - Callable (
() -> Vector{Vector{Tuple{VariableRef, T}}}): a zero-arg function returning a full trajectory. Use this for temporal correlation; seesample.
- Per-unit pool (
Keywords
num_batches::Integer: number of SGD batches.num_train_per_batch::Integer: sampled trajectories per batch $S$.optimizer: Flux optimizer used to updatemodel.adjust_hyperparameters::Function: optional hook returning the batch size for the current iteration.record_loss: legacy logging callback.sample_log::SampleLog: per-batch objective cache.record::Function: callback called asrecord(sample_log, iter, model).penalty_schedule: optional multiplier schedule for target-penalty terms.integer_strategy::AbstractIntegerStrategy: strategy used to read local dual information fromdet_equivalentwhen it has discrete variables.score_function: optionalScoreFunctionConfigorScoreFunctionSchedulefor mixed dual/score-function gradients.
Examples
train_multistage(
policy,
initial_state,
det_equivalent,
state_params_in,
state_params_out,
uncertainty_sampler;
num_batches = 200,
num_train_per_batch = 16,
optimizer = Flux.Adam(1.0e-3),
integer_strategy = FixedDiscreteIntegerStrategy(),
score_function = nothing,
)DecisionRules.train_multistage — Method
train_multistage(model, initial_state, subproblems::Vector{JuMP.Model},
state_params_in, state_params_out, uncertainty_sampler;
kwargs...)Train a target-state policy with stage-wise decomposition (single shooting).
For one sampled uncertainty trajectory $w_{1:T}$, this overload solves one optimization problem per stage. At stage $t$, given the realized incoming state $x_{t-1}$, the policy predicts a target $\hat{x}_t = \pi_\theta(w_t, x_{t-1})$ and the stage problem is
\[\begin{aligned} q_t(x_{t-1}, w_t; \hat{x}_t) = \min_{x_t, y_t, \delta_t} \quad & f_t(x_t, y_t) + C_\delta \|\delta_t\| \\ \text{s.t.}\quad & x_t = T_t(w_t, y_t, x_{t-1}) && : \mu_t, \\ & x_t + \delta_t = \hat{x}_t && : \lambda_t, \\ & h_t(x_t, y_t) \ge 0 . \end{aligned}\]
The rollout objective is the sum of stage values,
\[Q(\theta; w) = \sum_{t=1}^{T} q_t(x_{t-1}, w_t; \hat{x}_t),\]
where each realized $x_t$ is read from the previous stage solve. In this single-shooting rollout $\theta$ influences later stage costs through two couplings: the realized-state chain (the solver map $x_t = X_t(x_{t-1}, \hat{x}_t; w_t)$ propagates earlier targets forward), and the policy-feedback path (the policy input at stage $t$ is the realized state $x_{t-1}$, which itself depends on earlier targets). The exact gradient is the total-derivative recursion
\[\frac{d Q}{d \theta} = \sum_{t=1}^{T} \left[ \frac{\partial q_t}{\partial \hat{x}_t} \frac{d \hat{x}_t}{d \theta} + \frac{\partial q_t}{\partial x_{t-1}} \frac{d x_{t-1}}{d \theta} \right],\]
with the coupled state and target recursions (and $d x_0 / d\theta = 0$)
\[\frac{d x_t}{d \theta} = \frac{\partial X_t}{\partial x_{t-1}} \frac{d x_{t-1}}{d \theta} + \frac{\partial X_t}{\partial \hat{x}_t} \frac{d \hat{x}_t}{d \theta}, \qquad \frac{d \hat{x}_t}{d \theta} = \nabla_\theta \pi_\theta(w_t, x_{t-1}) + \frac{\partial \pi_\theta}{\partial x_{t-1}} \frac{d x_{t-1}}{d \theta}.\]
All derivative products must be read as total derivatives that include the policy-feedback composition: a term such as $\partial \pi_\theta / \partial x_{k-1} \cdot d x_{k-1} / d\theta$ is present at every stage, so $\theta$ reaches stage $k$ not only through the realized-state chain $\partial x_j / \partial x_{j-1}$ but also through the policy input $x_{k-1}$. Reverse-mode AD (Zygote through the stage rrules) computes exactly this full chain: the dual terms from target and transition constraints supply $\partial q_t / \partial \hat{x}_t$ and $\partial q_t / \partial x_{t-1}$, while the state sensitivities are computed through DiffOpt in the rrules for simulate_stage and get_next_state.
Arguments
model: differentiable Flux-compatible policy. It receivesvcat(stage_uncertainty, realized_state)and returns the next target state.initial_state::AbstractVector{<:Real}: state $x_0$ entering stage 1.subproblems::Vector{JuMP.Model}: one JuMP model per stage.state_params_in: stage input-state parameters.state_params_out:(target_parameter, realized_state_variable)pairs for each stage output state.uncertainty_sampler: source of uncertainty trajectories, passed tosample. Three formats are accepted:- Per-unit pool (
Vector{Vector{Tuple{VariableRef, Vector{T}}}}): independent sampling per parameter per stage. - Joint-scenario pool (
Vector{Vector{Vector{Tuple{VariableRef, T}}}}): one scenario drawn per stage, preserving spatial correlation. - Callable (
() -> Vector{Vector{Tuple{VariableRef, T}}}): a zero-arg function returning a full trajectory. Use this for temporal correlation by wrapping a trajectory sampler:() -> sample(my_stage_sampler, T)wheremy_stage_sampler(t, past)generates stagetconditioned on past realizations.
- Per-unit pool (
Keywords
num_batches::Integer: number of SGD batches.num_train_per_batch::Integer: sampled trajectories per batch.optimizer: Flux optimizer used to updatemodel.adjust_hyperparameters::Function: optional hook returning the batch size for the current iteration.record_loss: legacy logging callback.sample_log::SampleLog: per-batch objective cache.record::Function: callback called asrecord(sample_log, iter, model).penalty_schedule: optional multiplier schedule for target-penalty terms.integer_strategy::AbstractIntegerStrategy: strategy used when a stage model has discrete variables and derivative information must be read.
Examples
# With data pool (independent or joint):
train_multistage(
policy, initial_state, subproblems,
state_params_in, state_params_out, uncertainty_pool;
num_batches=200, optimizer=Flux.Adam(1e-3),
)
# With trajectory sampler (temporal correlation):
ar_sampler(t, past) = my_ar1_model(t, past, inflow_params)
train_multistage(
policy, initial_state, subproblems,
state_params_in, state_params_out,
() -> sample(ar_sampler, T);
num_batches=200, optimizer=Flux.Adam(1e-3),
)DecisionRules.variable_to_parameter — Method
variable_to_parameter(model, variable; initial_value=0.0, deficit=nothing)Replace a decision variable with an MOI.Parameter and bind them via an equality constraint. When deficit is provided, the constraint becomes variable + deficit == parameter and both the parameter and the deficit variable are returned.
DecisionRules.vcat_contexts — Method
vcat_contexts(a, b, ...)Vertically concatenate context matrices after checking that they cover the same number of stages.
Internal Functions
DecisionRules.STRICT_GRADIENTS — Constant
STRICT_GRADIENTSGlobal flag controlling gradient fallback behavior in rrules.
When false (default), rrule pullbacks return zero gradients with a warning when the solver terminates unsuccessfully — this keeps training alive when a few samples hit numerical trouble.
When true, the same situation throws an error instead. Enable this in tests to verify that controlled test cases never silently fall through to zero gradients:
DecisionRules.STRICT_GRADIENTS[] = trueThis flag controls the rrule-level fallback for bad solver status. For the training-loop-level fallback (DiffOpt assertion errors, etc.), use the gradient_fallback keyword in train_multistage and train_multiple_shooting.
ChainRulesCore.rrule — Method
ChainRulesCore.rrule for getlastrealized_state
Computes gradients w.r.t.:
- s_in (window start numeric state)
- targets (all target vectors in the window)
Given cotangents (Δs_out), we:
- seed reverse variables (realized end state vars) with Δs_out
- reverse_differentiate!
- read reverse sensitivities w.r.t. windowstatein_params and all target params
ChainRulesCore.rrule — Method
ChainRulesCore.rrule(get_next_state, subproblem, state_param_in, state_param_out, state_in, state_out_target)Correct reverse-mode rule using DiffOpt:
- Seeds reverse on realized output variables with Δstate_out
- Calls
DiffOpt.reverse_differentiate! - Reads sensitivities wrt parameter vars (stateparamin, stateparamout parameters)
- Returns VJP wrt the numeric inputs
state_inandstate_out_target
Assumptions:
subproblemis a JuMP.Model constructed withModel(() -> DiffOpt.diff_optimizer(...))state_param_in::Vector{JuMP.VariableRef}are JuMP Parameter variables (incoming state parameters)state_param_out::Vector{Tuple{JuMP.VariableRef,JuMP.VariableRef}}holds (target-Parameter variable, realized-state Variable) per componentget_next_state(...)updates parameter values,optimize!s, and returns a Vector matching the realized-state variables
ChainRulesCore.rrule — Method
ChainRulesCore.rrule(::typeof(simulate_multistage), det_equivalent, state_params_in,
state_params_out, uncertainties, states)Reverse-mode rule for the deterministic-equivalent (full-horizon) solve.
Mathematical basis (TS-DDR, arXiv:2405.14973, Eq. 1.2; Extension §1)
For the coupled problem $Q(w;θ) = \min \sum_t f_t + C_δ\|δ_t\|$ the gradient estimator is:
∇_θ E[Q] ≈ (1/S) Σ_s λ^s ⊙ ∇_θ π(·; θ)where $λ_t$ is the dual of the target constraint $x_t + δ_t = \hat{x}_t$. The pullback returns $Δ_{states}$ such that $Δ_{states}[1]$ holds the parameter duals of the initial-state parameters and $Δ_{states}[t+1]$ holds the target-constraint duals $λ_t$ for each stage.
Fallback strategy
Same as simulate_stage: tries compute_parameter_dual first, falls back to DiffOpt reverse differentiation if pdual raises. Solver failure (bad termination status) returns zero gradients or throws depending on STRICT_GRADIENTS.
ChainRulesCore.rrule — Method
ChainRulesCore.rrule(::typeof(simulate_stage), subproblem, state_param_in,
state_param_out, uncertainty, state_in, state_out)Reverse-mode rule for a single-stage subproblem solve.
Mathematical basis (TS-DDR, arXiv:2405.14973; Extension §2 Eq. 2.5)
For stage problem $q_t(x_{t-1}, w_t; \hat{x}_t)$, the sensitivities are:
∂q_t/∂(state_in) = μ_t (dual of dynamics constraint w.r.t. incoming state)
∂q_t/∂(target) = λ_t (dual of target constraint w.r.t. target x̂_t)These are the Lagrange multipliers that compute_parameter_dual (pdual) extracts from the solved model. This is the preferred path: closed-form and exact whenever the solver exposes constraint duals.
Fallback strategy
- pdual (parameter duals) — tried first.
- DiffOpt reverse differentiation — if pdual raises (e.g. the optimizer wrapper does not expose conic duals). Computes the same sensitivities via implicit differentiation of the KKT system.
- Zero gradients — only when the solver terminated with an unsuccessful status (not OPTIMAL / ALMOSTOPTIMAL / LOCALLYSOLVED). A warning is emitted. Set
DecisionRules.STRICT_GRADIENTS[] = trueto throw instead.
ChainRulesCore.rrule — Method
ChainRulesCore.rrule for solve_window
Computes gradients w.r.t.:
- s_in (window start numeric state)
- targets (all target vectors in the window)
Given cotangents (Δobj_val), we:
- seed reverse variables (objective and realized end state vars) with Δobj_val
- reverse_differentiate!
- read reverse sensitivities w.r.t. windowstatein_params and all target params
ChainRulesCore.rrule — Method
ChainRulesCore.rrule(::typeof(setwindowuncertainties!), window::WindowData, uncertainty_sample)
Declare setwindowuncertainties! as non-differentiable (mutates solver state).
DecisionRules._apply_deficit_penalty_multiplier! — Method
_apply_deficit_penalty_multiplier!(model::JuMP.Model, bases::Dict{VariableRef,Float64}, multiplier::Real) -> JuMP.Model
_apply_deficit_penalty_multiplier!(models::Vector{JuMP.Model}, bases::Vector, multiplier::Real) -> Vector{JuMP.Model}Mutate model (or each model in models) in place, setting every deficit variable's objective coefficient to multiplier * base using the bases from _deficit_penalty_bases. Return the mutated model(s).
DecisionRules._as_float64_vec — Method
_as_float64_vec(val) -> Vector{Float64}Coerce val to a Vector{Float64}. If val is already a Vector{Float64}, return it as-is; if it is another AbstractVector, convert element-wise; if it is a scalar, wrap it in a single-element vector.
DecisionRules._assert_successful_solve — Method
_assert_successful_solve(model::JuMP.Model; context::AbstractString = "solve")Throw an error unless model terminated with an accepted success status.
Arguments
model::JuMP.Model: model whose termination status is checked.context::AbstractString: human-readable phrase included in the error.
Examples
DecisionRules._assert_successful_solve(model; context = "fixed LP solve")DecisionRules._center_rollout_costs — Method
_center_rollout_costs(costs::AbstractVector{<:Real}, baseline::Symbol)Convert rollout costs into score-function advantages.
Arguments
costs::AbstractVector{<:Real}: operational costs from perturbed rollouts.baseline::Symbol: either:meanor:none.
Examples
advantages = DecisionRules._center_rollout_costs([10.0, 12.0], :mean)DecisionRules._check_deficit_penalty_bases — Method
_check_deficit_penalty_bases(bases) -> typeof(bases)Return bases (from _deficit_penalty_bases) unchanged if it has at least one entry; otherwise throw ArgumentError, since a penalty_schedule would then have nothing to scale.
DecisionRules._create_like_variable — Method
_create_like_variable(m::JuMP.Model, src::JuMP.VariableRef, t::Int;
force_parameter=false) -> JuMP.VariableRefCreate a new variable in m that mirrors src. If src is a JuMP parameter (or force_parameter is true), the new variable is created as an MOI.Parameter initialized to _param_init_value(src); otherwise a free decision variable is created. The variable is named via var_set_name! with stage suffix t.
DecisionRules._deficit_penalty_bases — Method
_deficit_penalty_bases(model::JuMP.Model; deficit_name="norm_deficit") -> Dict{VariableRef,Float64}
_deficit_penalty_bases(models::Vector{JuMP.Model}; deficit_name="norm_deficit") -> Vector{Dict{VariableRef,Float64}}Capture the current objective coefficient of every deficit variable as the multiplier base for _apply_deficit_penalty_multiplier!. A variable counts as a deficit variable if deficit_name occurs in its name and its linear objective coefficient (see _linear_objective_coefficient) is nonzero.
Must be called before any penalty_schedule multiplier is applied, so the captured coefficients reflect the as-built penalties.
DecisionRules._get_dual_from_affine_constraints — Method
_get_dual_from_affine_constraints(model::JuMP.Model, param::JuMP.VariableRef, S)Get dual contribution from scalar affine constraints of set type S.
DecisionRules._get_dual_from_constraints — Method
_get_dual_from_constraints(model::JuMP.Model, param::JuMP.VariableRef)Compute the dual contribution from all constraints containing the parameter.
DecisionRules._get_dual_from_objective — Method
_get_dual_from_objective(model::JuMP.Model, param::JuMP.VariableRef)Get the dual contribution from the objective function. If parameter appears in the objective with coefficient c, contribution is:
- +c for minimization
- +c for maximization
DecisionRules._get_dual_from_quadratic_constraints — Method
_get_dual_from_quadratic_constraints(model::JuMP.Model, param::JuMP.VariableRef, S)Get dual contribution from quadratic constraints of set type S. Handles both affine terms (parameter appears linearly) and quadratic terms (parameter appears in products pv or pp).
DecisionRules._get_dual_from_vector_affine_constraints — Method
_get_dual_from_vector_affine_constraints(model::JuMP.Model, param::JuMP.VariableRef, F, S)Get dual contribution from vector affine constraints (like conic constraints).
DecisionRules._get_objective_parameter_coefficient — Method
_get_objective_parameter_coefficient(obj, param::JuMP.VariableRef)Get the coefficient of a parameter in the objective function.
DecisionRules._get_parameter_coefficient — Method
_get_parameter_coefficient(expr::JuMP.GenericAffExpr, param::JuMP.VariableRef)Get the coefficient of a parameter in an affine expression.
DecisionRules._get_parameter_coefficient_from_affine — Method
_get_parameter_coefficient_from_affine(expr::JuMP.GenericQuadExpr, param::JuMP.VariableRef)Get the coefficient of a parameter from the affine part of a quadratic expression.
DecisionRules._get_parameter_coefficient_from_quadratic — Method
_get_parameter_coefficient_from_quadratic(expr::JuMP.GenericQuadExpr, param::JuMP.VariableRef)Get the effective coefficient of a parameter from quadratic terms. For terms like coef * p * v, the effective coefficient is coef * value(v). For terms like coef * p * p, the effective coefficient is 2 * coef * value(p).
DecisionRules._linear_objective_coefficient — Method
_linear_objective_coefficient(model::JuMP.Model, variable::VariableRef) -> Float64Return variable's linear coefficient in model's objective, or 0.0 if variable does not appear in it. Supports affine and quadratic objectives (quadratic terms are ignored); throw ArgumentError for any other objective type.
DecisionRules._param_init_value — Method
_param_init_value(src::JuMP.VariableRef) -> Float64Return the current parameter value of src if it is a JuMP MOI.Parameter; otherwise return 0.0. Silently returns 0.0 if parameter_value throws (e.g., when the parameter has been deleted from the model).
DecisionRules._penalty_multiplier_for — Method
_penalty_multiplier_for(schedule, iter::Int) -> Float64Return the multiplier of the phase containing batch iter. If iter is past the schedule's last phase, return that phase's multiplier (hold the final value steady).
DecisionRules._print_window_status_and_params — Method
_print_window_status_and_params(window, status; context="")Print a diagnostic dump for a window solve that did not reach an optimal status.
Outputs: a primal feasibility report for window.model, the solver status, the window.stage_range, and a sorted listing of every JuMP parameter in the window model with its current value. The optional context string is included in the header for traceability.
DecisionRules._record_loss_adapter — Method
_record_loss_adapter(record_loss)Adapt the deprecated 4-argument record_loss(iter, model, loss, tag) callback to the record(sample_log, iter, model) interface, reproducing the historical two-call contract: record_loss is called first with tag = "metrics/loss", and only if that returns false is it called again with tag = "metrics/training_loss". Return the result of whichever call is made last.
DecisionRules._remap_uncertainties — Method
_remap_uncertainties(uncertainties, var_src_to_dest, cons_to_cons)Replace source-model VariableRef keys in an uncertainty pool with their destination-model counterparts (using the variable or constraint mapping built by deterministic_equivalent!).
Two methods dispatch on the pool format:
- Per-unit pools (
Vector{Vector{Tuple{VariableRef, Vector{T}}}}): each stage maps[(param₁, [v₁, …]), …]independently. - Joint-scenario pools (
Vector{Vector{Vector{Tuple{VariableRef, T}}}}): each stage maps[[scenario₁…], [scenario₂…], …]preserving the grouped structure.
This is an internal helper; users interact with it indirectly through deterministic_equivalent!.
DecisionRules._reset_sample_log! — Method
_reset_sample_log!(sample_log::SampleLog) -> SampleLog
_reset_sample_log!(sample_log) -> typeof(sample_log)Clear the per-batch cache of a SampleLog before the next training batch. For a SampleLog, delegates to Base.empty!; for any other type the call is a no-op and returns sample_log unchanged.
DecisionRules._resolve_penalty_schedule — Method
_resolve_penalty_schedule(penalty_schedule, num_batches::Int)Resolve the penalty_schedule keyword of train_multistage and train_multiple_shooting into a Vector{Tuple{Int,Int,Float64}} of (first_batch, last_batch, multiplier) phases, or nothing if penalty scaling is disabled:
nothingreturnsnothing(no scaling);:default_annealedreturnsdefault_annealed_schedule(num_batches);- any other value is checked with
_validate_penalty_scheduleand returned as-is.
DecisionRules._resolve_record — Method
_resolve_record(record, record_loss)Resolve the record/record_loss keywords of the training loops into a single per-batch callback. Return record unchanged if record_loss is nothing; otherwise require record === default_record and return _record_loss_adapter(record_loss). Throw ArgumentError if both a custom record and a record_loss are given.
DecisionRules._sample_target_perturbations — Method
_sample_target_perturbations(num_stages::Integer, state_dimension::Integer, sigma::Real)Draw Gaussian target perturbations for one score-function rollout.
Arguments
num_stages::Integer: number of stage targets to perturb.state_dimension::Integer: length of each target state vector.sigma::Real: Gaussian standard deviation $\sigma$.
Examples
perturbations = DecisionRules._sample_target_perturbations(3, 2, 0.5)DecisionRules._score_function_rollouts — Method
_score_function_rollouts(
config::ScoreFunctionConfig,
initial_state::AbstractVector,
uncertainties,
targets;
perturbation_std = config.perturbation_std,
num_rollouts = config.num_rollouts,
) -> (advantages, perturbations)Estimate rollout advantages for the score-function term.
Arguments
config::ScoreFunctionConfig: score-function rollout configuration.initial_state::AbstractVector: state entering stage 1.uncertainties: sampled uncertainty trajectory.targets: target trajectory, including the initial state.perturbation_std::Real: Gaussian standard deviation $\sigma$.num_rollouts::Integer: number of perturbed rollouts to sample.
Examples
advantages, perturbations = DecisionRules._score_function_rollouts(
score_function,
initial_state,
uncertainty_sample,
targets;
perturbation_std = 0.5,
num_rollouts = 4,
)DecisionRules._score_function_surrogate — Method
_score_function_surrogate(
advantage::Real,
perturbations,
targets,
perturbation_std::Real,
) -> RealBuild the differentiable scalar whose gradient is the Gaussian score estimate.
For fixed rollout cost advantage $A$ and perturbations $\delta_t$, the surrogate is
\[A \sum_t \left\langle \delta_t / \sigma^2, \hat{x}_{t+1}(\theta) \right\rangle .\]
Arguments
advantage::Real: centered rollout cost $R - b$.perturbations: stage perturbations $\delta_t$.targets: differentiable target trajectory produced by the policy.perturbation_std::Real: Gaussian standard deviation $\sigma$.
Examples
loss = DecisionRules._score_function_surrogate(
3.0,
perturbations,
targets,
0.5,
)DecisionRules._sensitivity_forward_status — Method
_sensitivity_forward_status(model::JuMP.Model, strategy) -> MOI.TerminationStatusCodeReturn the termination status that an rrule should use for gradient fallback.
Arguments
model::JuMP.Model: model inspected after the forward pass.strategy::AbstractIntegerStrategy: integer strategy used for the solve.
Examples
status = DecisionRules._sensitivity_forward_status(model, strategy)DecisionRules._set_multistage_parameters! — Method
_set_multistage_parameters!(state_params_in, state_params_out,
uncertainties, states) -> NothingWrite MOI parameter values into a deterministic-equivalent JuMP model across all $T$ stages before solving.
For each stage $t = 1, \ldots, T$:
- Initial state ($t = 1$ only):
state_params_in[1]receivesstates[1](the initial state $x_0$). - Uncertainty $w_t$: each
(parameter, value)pair inuncertainties[t]is written. - Target $\hat{x}_t$: the target parameters in
state_params_out[t]receivestates[t + 1].
Note that state_params_in[t] for $t > 1$ is NOT set here because in the deterministic equivalent the incoming state is an internal variable linked by constraints, not a parameter.
Arguments
state_params_in: per-stage vectors of incoming-state parameters.state_params_out: per-stage vectors of(target_parameter, state_variable).uncertainties: per-stage(parameter, value)pairs for $w_t$.states: length-$(T+1)$ target trajectory $[x_0, \hat{x}_1, \ldots, \hat{x}_T]$.
DecisionRules._set_score_function_stage_parameters! — Method
_set_score_function_stage_parameters!(
state_params_in,
state_params_out,
uncertainties,
state,
target,
) -> NothingSet the JuMP parameters needed for one perturbed rollout stage.
Arguments
state_params_in::AbstractVector: parameters receiving the current state.state_params_out::AbstractVector:(target_parameter, state_variable)pairs receiving the target state.uncertainties::AbstractVector:(parameter, value)pairs for stage uncertainty.state::AbstractVector{<:Real}: realized state entering this stage.target::AbstractVector{<:Real}: perturbed target for the output state.
Examples
DecisionRules._set_score_function_stage_parameters!(
spi[t],
spo[t],
uncertainty_sample[t],
state,
target,
)DecisionRules._set_stage_parameters! — Method
_set_stage_parameters!(state_param_in, state_param_out, uncertainty,
state_in, state_out_target) -> NothingWrite MOI parameter values into a single-stage JuMP subproblem before solving.
Three groups of parameters are set:
- Incoming state $x_{t-1}$: each element of
state_param_inreceives the corresponding entry ofstate_in. - Uncertainty $w_t$: each
(parameter, value)pair inuncertaintyis written directly. - Outgoing target $\hat{x}_t$: the first element of each tuple in
state_param_out(the target parameter) receives the corresponding entry ofstate_out_target.
After this call the subproblem is ready for optimize!.
Arguments
state_param_in: JuMP parameter variables for the incoming state.state_param_out:(target_parameter, realized_state_variable)pairs.uncertainty:(parameter, value)pairs for stage uncertainty $w_t$.state_in::AbstractVector{<:Real}: realized incoming state $x_{t-1}$.state_out_target::AbstractVector{<:Real}: policy target $\hat{x}_t$.
DecisionRules._set_window_parameters! — Method
_set_window_parameters!(window_state_in_params, window_state_out_params,
s_in, targets) -> NothingWrite numeric initial-state and target values into the JuMP MOI.Parameter variables of a window model before solving. s_in is broadcast into window_state_in_params; each element of targets is broadcast into the corresponding stage's target parameters in window_state_out_params.
DecisionRules._sf_config — Method
_sf_config(score_function) -> Union{Nothing,ScoreFunctionConfig}Extract the underlying ScoreFunctionConfig, if one exists.
Arguments
score_function::Nothing: score-function correction is disabled.score_function::ScoreFunctionConfig: returned as-is.score_function::ScoreFunctionSchedule: unwrapsscore_function.config.
Examples
config = DecisionRules._sf_config(score_function)DecisionRules._simulate_multistage_det — Method
_simulate_multistage_det(det_equivalent, state_params_in, state_params_out,
uncertainties, states, integer_strategy) -> Float64Solve the deterministic-equivalent model for a single uncertainty trajectory and return the optimal objective.
Sets all parameters via _set_multistage_parameters!, solves the coupled full-horizon problem $Q(w; \theta)$ through with_sensitivity_solution, and caches both the full objective and the deficit-free operational cost in det_equivalent.ext for logging.
Arguments
det_equivalent::JuMP.Model: full-horizon coupled JuMP model.state_params_in: per-stage incoming-state parameters.state_params_out: per-stage(target_parameter, state_variable)pairs.uncertainties: per-stage(parameter, value)pairs for $w_t$.states: length-$(T+1)$ target trajectory from the policy.integer_strategy::AbstractIntegerStrategy: discrete-variable strategy.
DecisionRules._simulate_multistage_det_with_parameter_duals — Method
_simulate_multistage_det_with_parameter_duals(det_equivalent, state_params_in,
state_params_out, uncertainties,
states, integer_strategy)
-> (objective, Δ_states)Solve the deterministic equivalent and extract parameter duals for the rrule pullback.
Like _simulate_multistage_det, this sets parameters and solves the full-horizon problem. In addition it reads the dual sensitivities via pdual for each stage:
\[\Delta_{\text{states}}[1] = \frac{\partial Q}{\partial x_0}, \qquad \Delta_{\text{states}}[t+1] = \lambda_t = \frac{\partial Q}{\partial \hat{x}_t}, \quad t = 1, \ldots, T.\]
These $\lambda_t$ duals are the envelope-theorem gradient used in the TS-DDR training objective (arXiv:2405.14973, Eq. 1.2).
Arguments
det_equivalent: full-horizon coupled JuMP model.state_params_in: per-stage incoming-state parameters.state_params_out: per-stage(target_parameter, state_variable)pairs.uncertainties: per-stage(parameter, value)pairs for $w_t$.states: length-$(T+1)$ target trajectory from the policy.integer_strategy::AbstractIntegerStrategy: discrete-variable strategy.
Returns
objective::Float64: optimal coupled objective $Q(w; \theta)$.Δ_states::Vector{Vector{Float64}}: length-$(T+1)$ vector of parameter duals;Δ_states[1]holds $\partial Q / \partial x_0$ andΔ_states[t+1]holds $\lambda_t$.
DecisionRules._simulate_multistage_target_feedback — Function
_simulate_multistage_target_feedback(subproblems, state_params_in,
state_params_out, initial_state, uncertainties, decision_rules,
integer_strategy) -> Float64Run a stage-wise rollout where the target state (not the realized state) is fed back into the policy at each stage, matching the deterministic-equivalent target-generation semantics from simulate_states. Each stage subproblem is solved sequentially; the accumulated objective value (including target-deficit penalties) is returned.
Used by RolloutEvaluation when policy_state == :target.
DecisionRules._simulate_stage — Method
_simulate_stage(subproblem, state_param_in, state_param_out, uncertainty,
state_in, state_out_target, integer_strategy) -> Float64Forward-solve one stage of the multistage problem and return the objective.
Sets all parameters via _set_stage_parameters!, then solves subproblem through with_sensitivity_solution (which applies the integer_strategy for models with discrete variables). Returns the scalar optimal objective value $q_t(x_{t-1}, w_t; \hat{x}_t)$.
Arguments
subproblem::JuMP.Model: stage-$t$ JuMP model.state_param_in: incoming-state parameters.state_param_out:(target_parameter, realized_state_variable)pairs.uncertainty:(parameter, value)pairs for $w_t$.state_in: realized incoming state $x_{t-1}$.state_out_target: policy target $\hat{x}_t$.integer_strategy::AbstractIntegerStrategy: controls how discrete variables are handled during the solve.
DecisionRules._simulate_stage_with_parameter_duals — Method
_simulate_stage_with_parameter_duals(subproblem, state_param_in, state_param_out,
uncertainty, state_in, state_out_target,
integer_strategy)
-> (objective, d_state_in, d_state_out_target)Forward-solve one stage and extract parameter duals for the rrule pullback.
Like _simulate_stage, this sets parameters and solves the stage problem. In addition it reads the dual sensitivities via pdual:
\[\mu_t = \frac{\partial q_t}{\partial x_{t-1}}, \qquad \lambda_t = \frac{\partial q_t}{\partial \hat{x}_t}.\]
These duals are the preferred (closed-form) gradient path used by the simulate_stage rrule when parameter duals are available from the solver.
Arguments
subproblem: stage-$t$ JuMP model.state_param_in: incoming-state parameters (yields $\mu_t$).state_param_out: target parameters (yields $\lambda_t$).uncertainty:(parameter, value)pairs for $w_t$.state_in: realized incoming state $x_{t-1}$.state_out_target: policy target $\hat{x}_t$.integer_strategy::AbstractIntegerStrategy: discrete-variable strategy.
Returns
objective::Float64: optimal stage cost $q_t$.d_state_in::Vector{Float64}: $\mu_t$ (sensitivities w.r.t. incoming state).d_state_out_target::Vector{Float64}: $\lambda_t$ (sensitivities w.r.t. target).
DecisionRules._state_eltype — Method
_state_eltype(state) -> TypeReturn the scalar element type of a recurrent state.
For nested tuple states (e.g. LSTM's (h, c) or a Chain's tuple of per-layer states) this recurses into the first element until it reaches an AbstractVector, then returns eltype(v). The result is used to cast inputs to the encoder's precision before each step.
Arguments
state::Tuple: nested recurrent state.v::AbstractVector: leaf state vector.
Returns
Type: the scalar element type (e.g.Float32).
DecisionRules._step_encoder_layers — Method
_step_encoder_layers(layers, x, states) -> (output, new_states)Recursively advance a tuple of recurrent layers by one time step.
Each layer receives the output of the previous layer as input and its own independent recurrent state. The base case (layers == ()) returns the input unchanged with an empty state tuple.
Arguments
layers::Tuple: remaining recurrent cells to evaluate.x: current input (or output of the prior layer).states::Tuple: per-layer recurrent states, same length aslayers.
Returns
output: output of the last layer inlayers.new_states::Tuple: updated recurrent states, one per layer.
DecisionRules._target_deficit_penalty_value — Method
_target_deficit_penalty_value(obj, norm_deficit) -> Float64Return the part of a JuMP objective expression that is attributed to target deficit variables. A variable is treated as a target-deficit variable when its JuMP name contains norm_deficit.
Proof sketch for the affine case: if the solved objective is q(x) + sum_i c_i d_i, and d_i are exactly the matched target-deficit variables, then the operational cost is the solved objective value minus sum_i c_i value(d_i).
Quadratic terms involving target-deficit variables are deliberately rejected. For such an objective, subtracting only the affine coefficient would not remove the whole penalty and would produce a silently biased operational cost.
DecisionRules._target_violation_share — Method
_target_violation_share(objective::Real, objective_no_deficit::Real) -> Float64Return the target-violation share, (objective - objective_no_deficit) / objective. Return NaN if either input is nonfinite or abs(objective) <= 1e-12 (share undefined).
DecisionRules._total_objective_value — Method
_total_objective_value(model::JuMP.Model) -> Float64
_total_objective_value(models::Vector{JuMP.Model}) -> Float64Return the objective value of model, falling back to the cached value model.ext[:_last_obj] (default 0.0) when the model is dirty or objective_value throws. The multi-model method sums across all models.
DecisionRules._validate_penalty_schedule — Method
_validate_penalty_schedule(schedule) -> typeof(schedule)Validate an explicit penalty_schedule, a Vector of (first_batch, last_batch, multiplier) tuples: phases must be non-empty, contiguous, start at batch 1, satisfy first_batch <= last_batch, and have finite positive multipliers. Return schedule unchanged, or throw ArgumentError describing the first violation found.
DecisionRules._with_current_or_sensitivity_solution — Method
_with_current_or_sensitivity_solution(f, model, integer_strategy)Run f(model) directly for continuous models and through with_sensitivity_solution for integer strategies.
Arguments
f::Function: callback that reads values, duals, or sensitivities.model::JuMP.Model: model to inspect.integer_strategy::AbstractIntegerStrategy: current integer strategy.
Examples
value = DecisionRules._with_current_or_sensitivity_solution(
m -> JuMP.objective_value(m),
model,
strategy,
)DecisionRules._zero_cotangents — Method
_zero_cotangents(n_in, n_out)Create a tuple of zero/no tangents compatible with the get_next_state rrule pullback signature.
Used by handle_gradient_error to produce a safe, neutral gradient when the solver or DiffOpt differentiation fails. The returned tuple matches the cotangent layout expected by ChainRulesCore.rrule for get_next_state: four NoTangent() entries (for the function itself and non-differentiable arguments), followed by dense zero vectors for the state-in and state-out dimensions, and a trailing NoTangent().
Arguments
n_in::Int: dimension of the incoming state vector.n_out::Int: dimension of the outgoing state vector.
Returns
A Tuple of NoTangent and Vector{Float64} elements that Zygote can propagate without error.
DecisionRules.add_child_model_exps! — Method
add_child_model_exps!(model, subproblem, var_src_to_dest, skip_parameter_refs,
state_params_out, state_params_in, t) -> DictCopy all constraints and the objective contribution from stage-t subproblem into the deterministic-equivalent model, remapping variables through var_src_to_dest via create_constraint and copy_and_replace_variables. Constraint-based state parameters and input parameters at t == 1 are updated to point to the new model's constraint refs. Parameter constraints for incoming states that are linked to the previous realized state are skipped via skip_parameter_refs; all other parameter constraints are copied. The subproblem objective is added to model's existing objective. Returns a Dict mapping source ConstraintRef to destination ConstraintRef.
DecisionRules.add_child_model_vars! — Method
add_child_model_vars!(model, subproblem, t, state_params_in, state_params_out,
initial_state, var_src_to_dest, skip_parameter_refs)
-> Dict{VariableRef,VariableRef}Copy decision variables from stage-t subproblem into the deterministic- equivalent model, populating the source-to-destination mapping var_src_to_dest. State-coupling variables (incoming parameters and outgoing realized-state/target pairs) are handled specially: at t == 1 they become fresh parameters in model; at t > 1 incoming state parameters are linked to the previous stage's realized state variables. Each copied variable is renamed via var_set_name! with a #t suffix. Mutates state_params_in, state_params_out, var_src_to_dest, and skip_parameter_refs in place; returns var_src_to_dest.
DecisionRules.apply_rule — Method
apply_rule(stage::Int, decision_rule, uncertainty, state_in) -> VectorApply a single (shared) policy to produce the target state for stage stage.
The policy receives a concatenated input vector $[w_t^{(1)}, \ldots, w_t^{(n_w)}, x_{t-1}^{(1)}, \ldots, x_{t-1}^{(n_x)}]$ and returns the next target $\hat{x}_t = \pi_\theta(w_t, x_{t-1})$.
Arguments
stage::Int: current stage index (unused when a single rule is shared).decision_rule: callable policy $\pi_\theta$.uncertainty:(parameter, value)pairs for $w_t$; values are extracted.state_in: realized incoming state $x_{t-1}$.
DecisionRules.apply_rule — Method
apply_rule(stage::Int, decision_rules::Vector, uncertainty, state_in) -> VectorApply a stage-specific policy from a vector of per-stage decision rules.
Dispatches to apply_rule(stage, decision_rules[stage], uncertainty, state_in), selecting the rule at index stage.
Arguments
stage::Int: current stage index, used to selectdecision_rules[stage].decision_rules::Vector: one callable policy per stage.uncertainty:(parameter, value)pairs for $w_t$.state_in: realized incoming state $x_{t-1}$.
DecisionRules.copy_and_replace_variables — Method
copy_and_replace_variables(src, map::Dict{VariableRef,VariableRef})Deep-copy a JuMP expression src, substituting every VariableRef key in map with its destination value. Dispatches on the concrete expression type:
Vector: element-wise recursive call.Real: returned as-is (no variables to replace).VariableRef: direct lookup inmap.GenericAffExpr: rebuild with remapped variable keys and same coefficients.GenericQuadExpr: rebuild affine part and remappedUnorderedPairkeys.GenericNonlinearExpr: recursively remap arguments, then reconstruct via@expression.
Throws an error for unrecognized expression types.
DecisionRules.create_constraint — Method
create_constraint(model, obj, var_src_to_dest) -> ConstraintRefAdd a constraint to model whose function is obj.func with all VariableRef keys replaced via var_src_to_dest (see copy_and_replace_variables).
Four methods handle different constraint types:
- Generic
ScalarConstraint: uses@constraint(model, new_func in obj.set). ScalarConstraint{NonlinearExpr, MOI.EqualTo}:new_func == obj.set.value.ScalarConstraint{NonlinearExpr, MOI.LessThan}:new_func <= obj.set.upper.ScalarConstraint{NonlinearExpr, MOI.GreaterThan}:new_func >= obj.set.lower.
DecisionRules.deterministic_equivalent! — Method
deterministic_equivalent!(model, subproblems, state_params_in, state_params_out,
initial_state, uncertainties)Build the deterministic-equivalent (direct transcription) JuMP model by copying all stage subproblems into model. Variables are renamed with a #t suffix to avoid conflicts. Stage coupling is enforced by identifying the realized state variable of stage t with the incoming state parameter of stage t+1.
uncertainties accepts both sampling formats (see sample):
- Per-unit pools:
Vector{Vector{Tuple{VariableRef, Vector{T}}}}— one pool per parameter, drawing independently per parameter. - Joint-scenario pools:
Vector{Vector{Vector{Tuple{VariableRef, T}}}}— pre-built joint scenarios preserving cross-parameter correlations.
Returns (model, uncertainties_new) where uncertainties_new has the same format as the input but with variable refs remapped to the deterministic-equivalent model. This function also remaps state_params_in and state_params_out in place. Copy those arrays before calling if you need to keep the original stage-wise references for rollout evaluation or later model construction.
DecisionRules.discrete_variables — Method
discrete_variables(model::JuMP.Model)Return the binary or integer variables in model.
Arguments
model::JuMP.Model: model to inspect.
Examples
vars = DecisionRules.discrete_variables(model)DecisionRules.extract_uncertainty_params — Method
extract_uncertainty_params(window_uncertainties) -> Vector{Vector{VariableRef}}Extract the JuMP parameter VariableRefs from each stage of an uncertainty pool.
Handles three possible input shapes (automatically detected):
- Already-extracted
Vector{Vector{VariableRef}}— returned as-is. - Per-unit pool
Vector{Vector{Tuple{VariableRef, Vector}}}— extracts the first element of each tuple. - Joint-scenario pool
Vector{Vector{Vector{Tuple{VariableRef, T}}}}— extracts params from the first scenario of each stage (all scenarios share the same params).
DecisionRules.get_last_realized_state — Method
get_last_realized_state(window_model, window_state_in_params, window_state_out_params,
s_in, targets)Get the realized end state from the window model after solving.
DecisionRules.get_next_state — Method
get_next_state(subproblem, state_param_in, state_param_out, state_in,
state_out_target) -> VectorReturn the realized state from the most recent solve of subproblem by reading the values of the realized-state variables in state_param_out.
DecisionRules.get_objective_no_target_deficit — Method
get_objective_no_target_deficit(subproblem::JuMP.Model;
norm_deficit="norm_deficit") -> Float64Compute the operational cost of a solved subproblem, excluding the target-deficit penalty.
The full objective includes a penalty $C_\delta \|\delta_t\|$ that penalizes deviations between realized and target states. This function strips those terms so that logged costs reflect true operational cost:
\[\text{cost}_t = q_t - \sum_{j \in \mathcal{D}} c_j \, \delta_j,\]
where $\mathcal{D}$ is the set of variables whose names contain norm_deficit.
If the model is dirty (parameters changed since last solve), returns the cached value from a previous successful call (stored in subproblem.ext[:_last_obj_no_deficit]). If no such value exists, or if the objective shape is unsupported, throws instead of inventing a cost.
Arguments
subproblem::JuMP.Model: a solved JuMP model.
Keywords
norm_deficit::AbstractString: substring matched against variable names to identify deficit-penalty terms.
DecisionRules.get_objective_no_target_deficit — Method
get_objective_no_target_deficit(subproblems::Vector{JuMP.Model};
norm_deficit="norm_deficit") -> Float64Sum the deficit-free operational costs across all stage subproblems.
Calls the single-model get_objective_no_target_deficit on each element and returns the total.
Arguments
subproblems::Vector{JuMP.Model}: one solved JuMP model per stage.
Keywords
norm_deficit::AbstractString: substring matched against variable names to identify deficit-penalty terms.
DecisionRules.handle_gradient_error — Method
handle_gradient_error(fallback::AbstractGradientFallback, e, n_state_in, n_state_out)Handle an exception raised inside the get_next_state rrule pullback.
This is the rrule-level extension point: it is called when the backward pass through a single stage fails (e.g., the solver returned an infeasible status and DiffOpt cannot differentiate). Concrete methods decide whether to absorb the error or propagate it.
Arguments
fallback::AbstractGradientFallback: dispatch tag controlling recovery behavior.e: the caught exception.n_state_in::Int: dimension of the incoming state vector (needed to build zero cotangents).n_state_out::Int: dimension of the outgoing state vector.
Returns
A cotangent tuple (same layout as _zero_cotangents) when the error is absorbed, or does not return (re-throws) when the error is propagated.
See AbstractGradientFallback for how to implement custom subtypes.
DecisionRules.handle_rollout_error — Method
handle_rollout_error(fallback::AbstractGradientFallback, e, iter)Handle an exception raised during a rollout evaluation scenario.
This is the rollout-level extension point: it is called when a single out-of-sample scenario fails during RolloutEvaluation (e.g., solver infeasibility on an unseen uncertainty sample). Absorbing the error skips that scenario and lets the evaluation continue with the remaining samples.
Arguments
fallback::AbstractGradientFallback: dispatch tag controlling recovery behavior.e: the caught exception.iter::Int: scenario index within the rollout batch.
Returns
trueto skip this scenario and continue the rollout.- Does not return (re-throws) when the error should propagate.
DecisionRules.handle_training_error — Method
handle_training_error(fallback::AbstractGradientFallback, e, iter)Handle an exception raised during a full training iteration (gradient computation and parameter update).
This is the training-loop-level extension point: it is called when Zygote.gradient or the subsequent optimizer update throws (e.g., a DiffOpt assertion error or a numerical issue in the loss computation). Unlike handle_gradient_error, which operates inside a single-stage rrule, this handler wraps the entire forward-backward pass for one iteration.
Arguments
fallback::AbstractGradientFallback: dispatch tag controlling recovery behavior.e: the caught exception.iter::Int: current training iteration index (used in log messages).
Returns
trueto skip this iteration and continue training.- Does not return (re-throws) when the error should propagate.
DecisionRules.has_discrete_variables — Method
has_discrete_variables(model::JuMP.Model) -> BoolReturn whether model contains at least one binary or integer variable.
Arguments
model::JuMP.Model: model to inspect.
Examples
if DecisionRules.has_discrete_variables(model)
@info "MIP model"
endDecisionRules.pdual — Method
pdual(v::VariableRef) -> Float64Compute $∂Q/∂p$ for a JuMP parameter variable $p$ in a solved model, where $Q$ is the optimal objective value. By the envelope theorem / Lagrangian duality this equals the sum of $-\text{coef} \times \text{dual}$ over all constraints where $p$ appears, plus the objective coefficient of $p$.
This is the key quantity in TS-DDR (arXiv:2405.14973): the dual $λ_t$ of the target constraint gives the sensitivity $∂Q/∂\hat{x}_t$ used in Eq. 1.2.
DecisionRules.rollout_with_perturbation — Method
rollout_with_perturbation(
config::ScoreFunctionConfig,
initial_state::AbstractVector,
uncertainties,
targets,
perturbations,
) -> Float64Run one stage-wise rollout with fixed target perturbations.
The rollout target at stage t is targets[t + 1] + perturbations[t]. The returned cost excludes the target-deficit penalty so the score-function signal estimates operational cost rather than target-following slack.
Arguments
config::ScoreFunctionConfig: rollout models and parameter mappings.initial_state::AbstractVector: state entering stage 1.uncertainties: sampled uncertainty trajectory.targets: target trajectory, includingtargets[1] == initial_state.perturbations: one perturbation vector for each stage target.
Examples
cost = DecisionRules.rollout_with_perturbation(
score_function,
initial_state,
uncertainty_sample,
targets,
perturbations,
)DecisionRules.set_window_uncertainties! — Method
set_window_uncertainties!(window, uncertainty_sample)Set sampled (realized) uncertainty values into the window model's JuMP parameters.
uncertainty_sample is a realized trajectory (output of sample), so each stage is Vector{Tuple{VariableRef, Float64}} regardless of whether the original pool used independent or joint-scenario format.
DecisionRules.var_set_name! — Method
var_set_name!(src::JuMP.VariableRef, dest::JuMP.VariableRef, t::Int) -> NothingName dest after src with a #t stage suffix. If src has a JuMP name, the result is "<name>#<t>"; otherwise the MOI variable index is used as fallback, producing "_[<index>]#<t>".
DecisionRules.windows_equivalent! — Method
windows_equivalent!(model, subproblems, state_params_in, state_params_out,
initial_state, uncertainties)Build a coupled JuMP model for a contiguous window of stages by copying all variables, constraints, and objectives from subproblems into model. Stage coupling is enforced by identifying each stage's realized state variable with the next stage's incoming state parameter (same approach as deterministic_equivalent!, but scoped to a window).
uncertainties accepts both per-unit and joint-scenario pool formats (see sample). The returned uncertainties_new preserves the input format with variable refs remapped to the window model.
Returns (model, state_params_in_new, state_params_out_new, uncertainties_new).
DecisionRules.with_sensitivity_solution — Method
with_sensitivity_solution(f, model, integer_strategy)Run f(model) while model is in a state suitable for reading duals or DiffOpt sensitivities.
Arguments
f::Function: callback that reads values, duals, or sensitivities.model::JuMP.Model: model to solve and inspect.integer_strategy::AbstractIntegerStrategy: strategy used to prepare models with binary or integer variables.
Examples
objective = with_sensitivity_solution(model, FixedDiscreteIntegerStrategy()) do m
JuMP.objective_value(m)
endFlux.reset! — Method
Flux.reset!(m::StateConditionedPolicy)Reset the encoder's recurrent state to Flux.initialstates, e.g. before starting a new rollout.