Export API
Every exported name in ReactantServerExport that carries a docstring: the offline export tooling that turns a model into a bundle. See the API overview for the other packages and the unexported internals the guides refer to.
ReactantServerExport.ReactantServerExport — Module
ReactantServerExportOffline tooling that assembles ReactantServer model bundles (manifest.yaml, model[.b{N}].mlir, weights.safetensors) from StableHLO modules and weights, and the Reactant tracing frontend that turns a model + parameters into a bundle. This package is the bundle-format authority for producers; it is not depended on by the server runtime. The format is kept in sync with the server by the round-trip tests.
write_bundle/IOSpec are the low-level writer. export_bundle traces a Reactant model (the former LuxExport; it needs only Reactant, not Lux). PyTorch support (export_bundle(:pytorch, ...) and export_bundle(:torchscript, ...)) lives in a package extension that loads when PythonCall is present — using ReactantServerExport, PythonCall enables it.
ReactantServerExport.IOSpec — Type
IOSpec(name, dtype, shape; batch_axis=nothing, letters=nothing)A tensor's spec. shape is the Julia (column-major) shape, the reverse of the row-major shape a KServe client sends, with a concrete value at the batch axis; batch_axis is the 0-based Julia axis that carries the batch dimension, or nothing. By convention the batch axis is the last one: a client's [batch, max_bytes] wire tensor is IOSpec(name, UInt8, [-1, 1]; batch_axis = 1). The server reads the batch count of each request at this axis, so a spec written in row-major order makes it count the wrong axis and breaks coalescing and request splitting.
The manifest serialization encodes the shape as an einsum-style letter string ("chwn") plus a dims map of letter→size; the batch axis is emitted as n. Non-batch letters are auto-allocated from _AXIS_LETTERS and carry no semantic meaning across tensors.
letters optionally overrides that auto-allocation with explicit non-batch axis letters, one per non-batch axis in shape order (the batch axis is still emitted as n). Pass it to give axes meaningful names, e.g. ['w','h'] for an image input so its manifest reads whn. The reserved markers n/b are rejected.
ReactantServerExport.assert_bundle_arity — Method
assert_bundle_arity(dir) -> NamedTuplebundle_arity_report as a refusal: raise unless every compiled module in dir takes exactly declared inputs + serialized weights arguments. Returns the report when it passes.
ReactantServerExport.bundle_arity_report — Method
bundle_arity_report(dir) -> NamedTupleRead a written bundle's three numbers and say whether they agree: the inputs its manifest.yaml declares, the tensors its weights.safetensors holds, and the arity of each compiled module.
Returns (; name, n_inputs, n_weights, expected, modules, servable), where modules is one (; module_file, entry_args, servable) per *.mlir. Nothing is thrown for a mismatch, so this can be run across a directory of bundles to triage them; assert_bundle_arity is the same check as a refusal.
ReactantServerExport.bundle_entry_arity — Method
bundle_entry_arity(mlir_path) -> IntThe number of arguments the compiled program in mlir_path takes, read out of the StableHLO portable artifact.
model*.mlir is a serialized vhlo artifact, so parse(MLIR.IR.Module, ...) fails with "dialect 'vhlo' does not implement the bytecode interface": it has to be deserialized against a Reactant context first. That is the whole reason this is a function rather than a regex at a call site.
ReactantServerExport.collect_provenance — Function
collect_provenance(repo_dir=pwd(); extra=Dict()) -> Dict{String,Any}Best-effort reproducibility provenance for a bundle. Captures the state of the model repo at repo_dir (repo_remote, git_commit, git_tree_sha1, git_branch, git_dirty) plus the export environment (julia_version, reactantserverexport_version) and an exported_at UTC timestamp. Merge the result into export_bundle's provenance; the frontends separately stamp their framework versions (reactant_version, or torch_version/torchax_version).
When the work tree is dirty, the uncommitted changes are captured as git_diff (git diff --binary HEAD), so git_commit plus the patch reconstructs the exact exported code (untracked files are not captured). write_bundle extracts git_diff into a working_tree.patch file in the bundle dir rather than embedding it in the manifest.
Every git field is best-effort: when git is missing, repo_dir is not inside a work tree, or a repo has no origin remote, the affected fields are omitted (with a warning for the non-repo case) and the function never throws. extra entries are merged last and override collected fields.
ReactantServerExport.export_bundle — Method
export_bundle(frontend::Symbol, args...; kwargs...) -> dirExport a model to a bundle, naming the frontend explicitly so the call site is unambiguous:
export_bundle(:lux, model, ps, st, example_input; ...)— a Lux-style model with separate parameters/state (any Reactant-traceablemodel(x, ps, st); Lux itself is not required).export_bundle(:reactant, f, inputs::Tuple, weights; ...)— any Reactant-traceable functionf(inputs..., weights...)with explicitname => arrayweight pairs.export_bundle(:pytorch, model, example_inputs::Tuple; ...)— atorch.nn.Module; provided by the package extension, so it requiresusing PythonCall.
Every frontend takes layout (default :xla), passed to write_bundle: :legacy writes weights that servers older than ReactantServer 0.2 can load.
The Reactant frontends (:lux, :reactant, and export_two_stage_detector) also take compile_options: the keywords Reactant.@compile accepts, as a NamedTuple (for example compile_options = (; raise = true) to raise kernels such as KernelAbstractions or CUDA.jl code to StableHLO, or (; optimize = :all, no_nan = true)), or a Reactant.CompileOptions. They apply when the model is traced to StableHLO; the exporter sets client and drop_unsupported_attributes itself. Non-default options are recorded under compile_options in the manifest's provenance. The PyTorch frontends lower through JAX and do not take them.
ReactantServerExport.export_bundle — Method
export_bundle(:lux, model, ps, st, example_input; dir, name, input_name="input",
output_name="output", batch_sizes=[1], provenance=Dict(), layout=:xla,
compile_options=(;)) -> dirTrace model(x, ps, st) (taking the first return as the output) at each batch size and write a bundle. The batch dimension is the last Julia axis (Lux convention) and the leading network axis. Works for any Reactant-traceable model; Lux itself is not required.
ReactantServerExport.export_bundle — Method
export_bundle(:lux, model, ps, st, example_inputs::Tuple; dir, name,
input_names=nothing, output_names=nothing,
output_select = y -> (y isa Tuple ? y : (y,)),
input_batch_axes=nothing, output_batch_axes=nothing,
client_inputs=nothing, client_outputs=nothing,
batch_sizes=[1], provenance=Dict(), layout=:xla, compile_options=(;)) -> dirMulti-input / multi-output Lux export. client_inputs/client_outputs (each nothing or a Vector{IOSpec}) declare the wire-facing spec when a shipped model.jl postprocess transforms the executable tensors into different client tensors; they are passed through to write_bundle. Traces model(x_tuple, ps, st) where x_tuple is the tuple of array inputs the model's forward expects as its single positional argument. output_select(first(model(...))) maps the raw model output to the ordered tuple of arrays to export; non-array returns (e.g. an Int step count) are dropped by the selector and must not appear in its result. Weights are extracted from ps automatically (same as the single-array method). Per-tensor batch axes default to each array's last Julia axis (natural Julia batch-last, the Lux convention) so every exported input and output is batch-last without the caller hand-specifying it. Override with input_batch_axes/output_batch_axes (1-based Julia axes) when the batch axis is not last (e.g. a (D, N, K+1) output whose batch axis N is the middle one). An entry of nothing in either vector opts that one tensor out of batching (a genuinely unbatched tensor), so single-dispatch or partially-unbatched models can still be exported; note the server requires all executable inputs to be batched or none, so a per-tensor opt-out that leaves a mix of batched and unbatched inputs is rejected at load time.
ReactantServerExport.export_bundle — Method
export_bundle(:reactant, f, inputs::Tuple, weights; dir, name, input_names=nothing,
output_name="output", provenance=Dict(), layout=:xla, compile_options=(;)) -> dirGeneric single-size export for any Reactant-traceable f(inputs..., weights...). weights is an ordered collection of name => array pairs. Produces one unbatched model.mlir.
ReactantServerExport.export_two_stage_detector — Method
export_two_stage_detector(dir; name, input, stage1, stage1_weights, stage2, stage2_weights, cfg,
input_shapes=nothing, output_columns=6, output_layout=:cn,
glue_precision=Float64, provenance=Dict(), layout=:xla,
compile_options=(;)) -> dirExport a two-stage detector as one plain bundle.
Stage1, the traced ReactantServerExport.Detection glue, stage2, and the per-class NMS compile to a single StableHLO program per input shape, and a model.jl postprocess hook trims the program's fixed-size detection buffer.
input: the client imageIOSpec, for exampleIOSpec("INPUT__0", UInt8, [512, 512, 1]; letters = ['w', 'h', 'c']). Its first two Julia axes are the image width and height (the glue clips boxes to them); the reversed array is what stage1 takes. Withinput_shapes, the variable axes are-1. Batched inputs are not supported (one program handles one image).stage1:Dictfrom variant key to stage1 StableHLO text:Int[]for a single-shape detector, otherwise one entry perinput_shapeselement. Stage1 returns(feats..., objectness..., deltas...)in torch layout, seeReactantServerExport.Detection.stage2: stage2 StableHLO text,(K, C, pooled, pooled)ROI features to(cls_logits, bbox_deltas), withK == cfg.post_nms_topk.stage1_weights,stage2_weights:name => arraypairs in each program's argument order (read_stage_bundlereturns both halves from an existing stage bundle). They are written toweights.safetensorsprefixedstage1.andstage2.and stay program ARGUMENTS; the entry arity is checked against them before anything is written.output_columns: 6 for rows[x1, y1, x2, y2, score, class], 5 for[x1, y1, x2, y2, score].output_layout::cnreturnsOUTPUT__0as(columns, N);:cn1as(columns, N, 1).layout: howweights.safetensorsstores the weights, seewrite_bundle.compile_options: Reactant compile options for tracing the fused program, as forexport_bundle.
The executable returns DETECTIONS ((columns, detections_per_img) in the chosen layout, zero past the count) and NUM_DETECTIONS (Int64, length 1); client_outputs declares the trimmed OUTPUT__0 with a variable detection axis. dir should not already hold a bundle: files from an earlier export with more variants would be left behind.
ReactantServerExport.read_bundle_weights — Method
read_bundle_weights(dir) -> Vector{Pair{String, Array}}A bundle's weights as name => array in argument_order, each array in the axis order of the program's StableHLO signature (for a torch export, torch order), whichever layout the weights file uses. Reads the file the manifest's weights_file names (default weights.safetensors).
ReactantServerExport.read_stage_bundle — Method
read_stage_bundle(dir) -> (; texts, weights, input_shapes)Read a stage bundle's programs and weights back for fusing.
Returns the pieces export_two_stage_detector takes for one stage: texts maps each input-shape variant key (Int[] for a single-shape bundle, otherwise the manifest input_shapes entry in (input, axis) order of the variable axes) to the StableHLO text of its program, weights is the name => array list in argument_order with each array in the axis order of the program's StableHLO signature (for a torch export, torch order), which is what Ops.hlo_call takes, and input_shapes lists the variant keys in manifest order. Each variant must carry exactly one module (one batch size).
ReactantServerExport.stablehlo_text — Method
stablehlo_text(path_or_bytes) -> StringThe textual StableHLO of a serialized portable artifact (a bundle's model*.mlir, given as a file path or its bytes), for Ops.hlo_call.
ReactantServerExport.write_bundle — Method
write_bundle(dir; name, executable_inputs, executable_outputs, modules, weights,
client_inputs=nothing, client_outputs=nothing, input_shapes=nothing,
provenance=Dict(), layout=:xla) -> dirWrite a bundle. With input_shapes === nothing, modules is a Dict keyed by batch size of StableHLO modules (or text or bytes); a single entry under key 0 writes model.mlir, otherwise each writes model.b{N}.mlir. weights is an ordered collection of name => array pairs whose order becomes the safetensors argument_order. The per-input batch axis is recorded in each input's IOSpec.batch_axis; the manifest derives batching.batch_dim from there.
input_shapes (a Vector{Vector{Int}} of compiled input-shape variants) turns on the multi-shape layout: the variable executable-input axes are marked -1 in executable_inputs, and each variant gives the concrete sizes of those axes in (input, axis) order. modules is then keyed by variant (each key a Vector{Int} equal to one input_shapes entry) and maps to that variant's batch-size module dict; the files are written as model.v{i}.*.mlir (i indexing input_shapes), all sharing the single weights.safetensors. The variants must share one set of batch sizes.
client_inputs/client_outputs (each nothing or a Vector{IOSpec}) declare the wire-facing spec when it differs from the executable spec, for bundles that ship a model.jl whose preprocess/postprocess transform between the two. They are emitted only when given. A variable (non-batch) axis is encoded by passing -1 for that axis size (e.g. the variable detection count of a postprocessed detector). The server requires these only when a model.jl is present, so the caller is responsible for also shipping model.jl into the bundle dir (see the converter handlers).
layout chooses how weights.safetensors stores each array. :xla (the default) writes the array's memory unchanged under the program's shape (the reversed Julia shape) and marks the file axis_order: xla, so the server reads each tensor with no permute; the manifest's format_version is "2.1", which ReactantServer 0.2 and later load and older servers refuse. :legacy writes the Julia shape with the data permuted to row-major and format_version: "2.0", for servers older than 0.2 (ReactantServer 0.2 migrates such bundles with runtime.weights_migration).