Export API

Every exported name in ReactantServerExport that carries a docstring: the offline export tooling that turns a model into a bundle. See the API overview for the other packages and the unexported internals the guides refer to.

ReactantServerExport.ReactantServerExport — Module
ReactantServerExport

Offline tooling that assembles ReactantServer model bundles (manifest.yaml, model[.b{N}].mlir, weights.safetensors) from StableHLO modules and weights, and the Reactant tracing frontend that turns a model + parameters into a bundle. This package is the bundle-format authority for producers; it is not depended on by the server runtime. The format is kept in sync with the server by the round-trip tests.

write_bundle/IOSpec are the low-level writer. export_bundle traces a Reactant model (the former LuxExport; it needs only Reactant, not Lux). PyTorch support (export_bundle(:pytorch, ...) and export_bundle(:torchscript, ...)) lives in a package extension that loads when PythonCall is present — using ReactantServerExport, PythonCall enables it.

source
ReactantServerExport.IOSpec — Type
IOSpec(name, dtype, shape; batch_axis=nothing, letters=nothing)

A tensor's spec. shape is the Julia (column-major) shape, the reverse of the row-major shape a KServe client sends, with a concrete value at the batch axis; batch_axis is the 0-based Julia axis that carries the batch dimension, or nothing. By convention the batch axis is the last one: a client's [batch, max_bytes] wire tensor is IOSpec(name, UInt8, [-1, 1]; batch_axis = 1). The server reads the batch count of each request at this axis, so a spec written in row-major order makes it count the wrong axis and breaks coalescing and request splitting.

The manifest serialization encodes the shape as an einsum-style letter string ("chwn") plus a dims map of letter→size; the batch axis is emitted as n. Non-batch letters are auto-allocated from _AXIS_LETTERS and carry no semantic meaning across tensors.

letters optionally overrides that auto-allocation with explicit non-batch axis letters, one per non-batch axis in shape order (the batch axis is still emitted as n). Pass it to give axes meaningful names, e.g. ['w','h'] for an image input so its manifest reads whn. The reserved markers n/b are rejected.

source
ReactantServerExport.bundle_arity_report — Method
bundle_arity_report(dir) -> NamedTuple

Read a written bundle's three numbers and say whether they agree: the inputs its manifest.yaml declares, the tensors its weights.safetensors holds, and the arity of each compiled module.

Returns (; name, n_inputs, n_weights, expected, modules, servable), where modules is one (; module_file, entry_args, servable) per *.mlir. Nothing is thrown for a mismatch, so this can be run across a directory of bundles to triage them; assert_bundle_arity is the same check as a refusal.

source
ReactantServerExport.bundle_entry_arity — Method
bundle_entry_arity(mlir_path) -> Int

The number of arguments the compiled program in mlir_path takes, read out of the StableHLO portable artifact.

model*.mlir is a serialized vhlo artifact, so parse(MLIR.IR.Module, ...) fails with "dialect 'vhlo' does not implement the bytecode interface": it has to be deserialized against a Reactant context first. That is the whole reason this is a function rather than a regex at a call site.

source
ReactantServerExport.collect_provenance — Function
collect_provenance(repo_dir=pwd(); extra=Dict()) -> Dict{String,Any}

Best-effort reproducibility provenance for a bundle. Captures the state of the model repo at repo_dir (repo_remote, git_commit, git_tree_sha1, git_branch, git_dirty) plus the export environment (julia_version, reactantserverexport_version) and an exported_at UTC timestamp. Merge the result into export_bundle's provenance; the frontends separately stamp their framework versions (reactant_version, or torch_version/torchax_version).

When the work tree is dirty, the uncommitted changes are captured as git_diff (git diff --binary HEAD), so git_commit plus the patch reconstructs the exact exported code (untracked files are not captured). write_bundle extracts git_diff into a working_tree.patch file in the bundle dir rather than embedding it in the manifest.

Every git field is best-effort: when git is missing, repo_dir is not inside a work tree, or a repo has no origin remote, the affected fields are omitted (with a warning for the non-repo case) and the function never throws. extra entries are merged last and override collected fields.

source
ReactantServerExport.export_bundle — Method
export_bundle(frontend::Symbol, args...; kwargs...) -> dir

Export a model to a bundle, naming the frontend explicitly so the call site is unambiguous:

  • export_bundle(:lux, model, ps, st, example_input; ...) — a Lux-style model with separate parameters/state (any Reactant-traceable model(x, ps, st); Lux itself is not required).
  • export_bundle(:reactant, f, inputs::Tuple, weights; ...) — any Reactant-traceable function f(inputs..., weights...) with explicit name => array weight pairs.
  • export_bundle(:pytorch, model, example_inputs::Tuple; ...) — a torch.nn.Module; provided by the package extension, so it requires using PythonCall.

Every frontend takes layout (default :xla), passed to write_bundle: :legacy writes weights that servers older than ReactantServer 0.2 can load.

The Reactant frontends (:lux, :reactant, and export_two_stage_detector) also take compile_options: the keywords Reactant.@compile accepts, as a NamedTuple (for example compile_options = (; raise = true) to raise kernels such as KernelAbstractions or CUDA.jl code to StableHLO, or (; optimize = :all, no_nan = true)), or a Reactant.CompileOptions. They apply when the model is traced to StableHLO; the exporter sets client and drop_unsupported_attributes itself. Non-default options are recorded under compile_options in the manifest's provenance. The PyTorch frontends lower through JAX and do not take them.

source
ReactantServerExport.export_bundle — Method
export_bundle(:lux, model, ps, st, example_input; dir, name, input_name="input",
              output_name="output", batch_sizes=[1], provenance=Dict(), layout=:xla,
              compile_options=(;)) -> dir

Trace model(x, ps, st) (taking the first return as the output) at each batch size and write a bundle. The batch dimension is the last Julia axis (Lux convention) and the leading network axis. Works for any Reactant-traceable model; Lux itself is not required.

source
ReactantServerExport.export_bundle — Method
export_bundle(:lux, model, ps, st, example_inputs::Tuple; dir, name,
              input_names=nothing, output_names=nothing,
              output_select = y -> (y isa Tuple ? y : (y,)),
              input_batch_axes=nothing, output_batch_axes=nothing,
              client_inputs=nothing, client_outputs=nothing,
              batch_sizes=[1], provenance=Dict(), layout=:xla, compile_options=(;)) -> dir

Multi-input / multi-output Lux export. client_inputs/client_outputs (each nothing or a Vector{IOSpec}) declare the wire-facing spec when a shipped model.jl postprocess transforms the executable tensors into different client tensors; they are passed through to write_bundle. Traces model(x_tuple, ps, st) where x_tuple is the tuple of array inputs the model's forward expects as its single positional argument. output_select(first(model(...))) maps the raw model output to the ordered tuple of arrays to export; non-array returns (e.g. an Int step count) are dropped by the selector and must not appear in its result. Weights are extracted from ps automatically (same as the single-array method). Per-tensor batch axes default to each array's last Julia axis (natural Julia batch-last, the Lux convention) so every exported input and output is batch-last without the caller hand-specifying it. Override with input_batch_axes/output_batch_axes (1-based Julia axes) when the batch axis is not last (e.g. a (D, N, K+1) output whose batch axis N is the middle one). An entry of nothing in either vector opts that one tensor out of batching (a genuinely unbatched tensor), so single-dispatch or partially-unbatched models can still be exported; note the server requires all executable inputs to be batched or none, so a per-tensor opt-out that leaves a mix of batched and unbatched inputs is rejected at load time.

source
ReactantServerExport.export_bundle — Method
export_bundle(:reactant, f, inputs::Tuple, weights; dir, name, input_names=nothing,
              output_name="output", provenance=Dict(), layout=:xla, compile_options=(;)) -> dir

Generic single-size export for any Reactant-traceable f(inputs..., weights...). weights is an ordered collection of name => array pairs. Produces one unbatched model.mlir.

source
ReactantServerExport.export_two_stage_detector — Method
export_two_stage_detector(dir; name, input, stage1, stage1_weights, stage2, stage2_weights, cfg,
                          input_shapes=nothing, output_columns=6, output_layout=:cn,
                          glue_precision=Float64, provenance=Dict(), layout=:xla,
                          compile_options=(;)) -> dir

Export a two-stage detector as one plain bundle.

Stage1, the traced ReactantServerExport.Detection glue, stage2, and the per-class NMS compile to a single StableHLO program per input shape, and a model.jl postprocess hook trims the program's fixed-size detection buffer.

  • input: the client image IOSpec, for example IOSpec("INPUT__0", UInt8, [512, 512, 1]; letters = ['w', 'h', 'c']). Its first two Julia axes are the image width and height (the glue clips boxes to them); the reversed array is what stage1 takes. With input_shapes, the variable axes are -1. Batched inputs are not supported (one program handles one image).
  • stage1: Dict from variant key to stage1 StableHLO text: Int[] for a single-shape detector, otherwise one entry per input_shapes element. Stage1 returns (feats..., objectness..., deltas...) in torch layout, see ReactantServerExport.Detection.
  • stage2: stage2 StableHLO text, (K, C, pooled, pooled) ROI features to (cls_logits, bbox_deltas), with K == cfg.post_nms_topk.
  • stage1_weights, stage2_weights: name => array pairs in each program's argument order (read_stage_bundle returns both halves from an existing stage bundle). They are written to weights.safetensors prefixed stage1. and stage2. and stay program ARGUMENTS; the entry arity is checked against them before anything is written.
  • output_columns: 6 for rows [x1, y1, x2, y2, score, class], 5 for [x1, y1, x2, y2, score].
  • output_layout: :cn returns OUTPUT__0 as (columns, N); :cn1 as (columns, N, 1).
  • layout: how weights.safetensors stores the weights, see write_bundle.
  • compile_options: Reactant compile options for tracing the fused program, as for export_bundle.

The executable returns DETECTIONS ((columns, detections_per_img) in the chosen layout, zero past the count) and NUM_DETECTIONS (Int64, length 1); client_outputs declares the trimmed OUTPUT__0 with a variable detection axis. dir should not already hold a bundle: files from an earlier export with more variants would be left behind.

source
ReactantServerExport.read_bundle_weights — Method
read_bundle_weights(dir) -> Vector{Pair{String, Array}}

A bundle's weights as name => array in argument_order, each array in the axis order of the program's StableHLO signature (for a torch export, torch order), whichever layout the weights file uses. Reads the file the manifest's weights_file names (default weights.safetensors).

source
ReactantServerExport.read_stage_bundle — Method
read_stage_bundle(dir) -> (; texts, weights, input_shapes)

Read a stage bundle's programs and weights back for fusing.

Returns the pieces export_two_stage_detector takes for one stage: texts maps each input-shape variant key (Int[] for a single-shape bundle, otherwise the manifest input_shapes entry in (input, axis) order of the variable axes) to the StableHLO text of its program, weights is the name => array list in argument_order with each array in the axis order of the program's StableHLO signature (for a torch export, torch order), which is what Ops.hlo_call takes, and input_shapes lists the variant keys in manifest order. Each variant must carry exactly one module (one batch size).

source
ReactantServerExport.stablehlo_text — Method
stablehlo_text(path_or_bytes) -> String

The textual StableHLO of a serialized portable artifact (a bundle's model*.mlir, given as a file path or its bytes), for Ops.hlo_call.

source
ReactantServerExport.write_bundle — Method
write_bundle(dir; name, executable_inputs, executable_outputs, modules, weights,
             client_inputs=nothing, client_outputs=nothing, input_shapes=nothing,
             provenance=Dict(), layout=:xla) -> dir

Write a bundle. With input_shapes === nothing, modules is a Dict keyed by batch size of StableHLO modules (or text or bytes); a single entry under key 0 writes model.mlir, otherwise each writes model.b{N}.mlir. weights is an ordered collection of name => array pairs whose order becomes the safetensors argument_order. The per-input batch axis is recorded in each input's IOSpec.batch_axis; the manifest derives batching.batch_dim from there.

input_shapes (a Vector{Vector{Int}} of compiled input-shape variants) turns on the multi-shape layout: the variable executable-input axes are marked -1 in executable_inputs, and each variant gives the concrete sizes of those axes in (input, axis) order. modules is then keyed by variant (each key a Vector{Int} equal to one input_shapes entry) and maps to that variant's batch-size module dict; the files are written as model.v{i}.*.mlir (i indexing input_shapes), all sharing the single weights.safetensors. The variants must share one set of batch sizes.

client_inputs/client_outputs (each nothing or a Vector{IOSpec}) declare the wire-facing spec when it differs from the executable spec, for bundles that ship a model.jl whose preprocess/postprocess transform between the two. They are emitted only when given. A variable (non-batch) axis is encoded by passing -1 for that axis size (e.g. the variable detection count of a postprocessed detector). The server requires these only when a model.jl is present, so the caller is responsible for also shipping model.jl into the bundle dir (see the converter handlers).

layout chooses how weights.safetensors stores each array. :xla (the default) writes the array's memory unchanged under the program's shape (the reversed Julia shape) and marks the file axis_order: xla, so the server reads each tensor with no permute; the manifest's format_version is "2.1", which ReactantServer 0.2 and later load and older servers refuse. :legacy writes the Julia shape with the data permuted to row-major and format_version: "2.0", for servers older than 0.2 (ReactantServer 0.2 migrates such bundles with runtime.weights_migration).

source