Binding cheat sheet

ReactantNitro connects its pieces by name. A batch field reaches the hook that declares a keyword of that name. A schedule key names the slot it drives. A run keyword names the accessor it overrides, which names the field it reads. This page lists each of those connections with its rule.

The three markers: where a field goes

@experiment struct MnistMLP    width::GraphConst{Int} = 128          # baked into the compiled graph as the literal 128;                                          # in the compile key, so a new value is a new program    smoothing::Device{Float32} = 0.05f0   # a device scalar, a traced INPUT to every program;                                          # not in the key: sweep it, schedule it, set_device! it    images::Matrix{Float32} = load_mnist() # Host (unmarked): stays on the driver;                                          # the trace sees StrippedHost{:images}(), nothing walkedend

What each hook sees when it reads e.<field>:

Unmarked is Host. Device is for a number you might change without recompiling. GraphConst is for a value that changes the graph.

The batch: fields to hooks, by keyword name

batch = (; img, label, idx)          # what the loader yields; the batch dimension is LAST in eachforward(e, model, ps, st; img)       # declares `img`, receives `img`loss(e, outputs; label)              # declares `label`, receives `label`metrics(e, outputs; label)           # the same                                     # `idx`: no hook declares it, so it never leaves the host

• A hook's keyword arguments are the batch fields it wants, by name. The framework resolves the method once at setup and passes exactly that subset.
• Only declared fields are transferred to the device. A field no hook declares stays on the host.
• A required keyword the batch lacks raises on the hook's first call, naming both sides. A keyword with a default is optional. A method ending in kwargs... receives the whole batch.
• forward declares inputs and never targets, which is why predict works on a batch with no labels.

The outputs: forward to everything downstream

forward(e, model, ps, st; img) returns (outputs, st_new). The framework threads st_new back into the layer state, discarding it in eval mode, and hands outputs on exactly as returned, as one positional argument:

• Whatever forward puts in the first slot is what loss receives in its second. Nothing is splatted, unpacked, or renamed in between.
• Do not unpack outputs a second time. If forward returns a bare array, first(outputs) is its first element, every later operation still broadcasts, and the run trains on one number without an error.
• Several outputs: return a NamedTuple and destructure by name downstream. Every array leaf keeps the batch dimension last, and nothing already reduced over the batch belongs in outputs.

Metrics: the pair, and where each hook runs

metrics(e, outputs; label) = (;    acc = sum(correct) => size(label, 2),   # per image: summed, then divided by the summed count    confusion = cm => :sum,                 # summed across the split, never divided    ranking = (; score = p, label) => :concat,  # every sample of the split, aligned)finalize_metrics(e, acc, split)             # acc.acc is a mean; acc.confusion is the split's matrix

metrics_residency(e, hook) flips either. No metrics method at all substitutes loss with a count of 1.

Schedules: a key to its slot

schedules(e) = (;    eta = total -> OneCycle(total, 1f-3),   # a rule field: `eta` of every group without a path key    smoothing = _ -> t -> 0.1f0 * t / 5000, # a Device field: e.smoothing, written each step    opt = (; encoder = (; eta = ...)),      # path-bound: the :encoder group's chain only    device = (; lambda = _ -> t -> 0.5f0),  # qualified: e.lambda; `lambda` is a rule field too)

• A bare key resolves against two namespaces: the optimizer rules' field names (opt) and the experiment's Device fields (device). One match binds; two is an ambiguity error that shows the qualified forms; none is an error naming the fix.
• Every entry is a factory of the horizon: called once with the total optimizer steps, then once per step with the upcoming step, counting from 1. A bare Number is a constant.
• A GraphConst cannot be scheduled: its value is the program. A Host field is in neither namespace.
• For each group, a path key (opt.encoder.eta) overrides the bare key (eta or opt.eta), and a group with neither uses its learning_rate accessor. Either key's eta is multiplied by the group's ratio, learning_rate(e, Val(g)) / learning_rate(e), so one bare curve moves the groups together.

Parameters: leaves to groups to one buffer each

param_group(e, keypath)         # each leaf of `ps` to a group: :encoder, :default, ...                                # one flat buffer and one rule chain per grouplearning_rate(e, Val(g))        # a RATIO against learning_rate(e); a schedule scales the baselambda(e, Val(g))               # decoupled decay toward decay_anchor(e, Val(g)): :zero, :w0, arrayno_decay(e, keypath, x)         # per leaf: excluded from decay; the default excludes 1-D leavesno_perturb(e, keypath, x)       # per leaf: excluded from SAM's perturbation; the default excludes noneadaptive_perturb(e, keypath, x) # per leaf: ASAM scales by |w| + eta; the default scales rank 2 and up

The optimizer program applies each chain once per group, not once per array, which keeps its compile small. gradient_clip_norm(e) is a global norm over the whole accumulated gradient, applied before any group, and is not per group or schedulable.

Run keywords: keyword to accessor to field

Nitro(e; max_epochs = 60)   # the keyword wins for THIS run;max_epochs(e)               # else the accessor: define a method for the experiment's own default;e.max_epochs                # else a field of the same name, when the struct declares one;1                           # else the framework's value

Sixteen keywords follow this chain: seed, run_dir, n_devs, accum, max_epochs, schedules, gradient_clip_norm, check_divergence, autolog_schedules, log_every_n_steps, logger, checkpointer, early_stop, monitor, eval_loss, ema. schedules is the one whose accessor reads no field. Four keywords name a fact about this construction and have no accessor: data, weights, resume, run_ref. w0, restore_optimizer, restore_best, eval_weights and load_ema are more of that kind, and checkpoint is the deprecated spelling of weights = path.

Weights: the ways in

weights and resume name a record the same way: a path, or run => checkpoint with the run :latest, a run id, :all or another Nitro, and the checkpoint :best or :latest (weights = :latest => :best, or Best(1) and Last(1)). weights also takes an average of several, run => Average(Last(10)). Checkpoints covers the grammar, retention and averaging.

restore_best = true changes where a handle ends rather than where it starts: train! finishes by loading the run's own best checkpoint into ps and st. restore_best = Average(Last(10)) ends it holding the average of the run's own latest 10 instead.

w0, the anchor for decay_anchor = :w0, is build_model's init unless w0 = :weights or a tree says otherwise.

What recompiles

A Nitro is a fixed point: none of these land on an existing handle. Build a new one and the module-level cache recompiles only what moved. ReactantNitro.cache_stats() is the check.

Where to read what bound

show(nitro) prints the handle and, under it, the bindings: each split with its batch count and prefetch path, the clip threshold and every schedule key with its source, one row per parameter group with rate, ratio, anchor, decay and rule, and for a Level 2 optimizer, the rule field each scheduled value landed in. The same text goes to the logger at setup as binding_report, so the run's record carries it. After training, history(nitro) is the run as a table.