Internal API

EnzymeError

Common supertype for Enzyme-specific errors.

This type is made public so that downstream packages can add custom error hints for the most common exceptions thrown by Enzyme.

Helper function for llvm-level rule generation. Will call callsamewithinvertedargifactive with corresponding extracted batches if width > 1, otherwise it will call it once.

Helper function for llvm-level rule generation. Will call the same function (and optional postprocessing), if the argument at index cmpidx isn't active. This takes into account runtime activity as a reason the value may not be active.

If postprocess_const is set, the original function will always be called, but the postprocessing will be conditionally gated as follows.

If the relevant input is active (and verified by runtime activity), postprocess(B, result, args) will run as normal Otherwise postprocess_const(B, result, args) will run

instantiate_annotation(A, rt, width)

Fill in the free parameters of a (possibly partially applied) activity annotation A with element type rt and batch width width.

The batch annotations take a second parameter carrying the batch width. Applying only A{rt} to them leaves that parameter free, and a subsequent A{rt} binds the element type to it, yielding an annotation whose batch_size is a type rather than the width. Filling both explicitly keeps batch_size(A) == width, which the shadow-return ABI in create_abi_wrapper and enzyme_call asserts.

link_split_existing!(mod::LLVM.Module, newmod::LLVM.Module)

Link newmod into mod like LLVM.link!(mod, newmod), but set LLVMInternalLinkage on any function defined in both modules before linking. This allows LLVM's linker to natively internalize and resolve duplicate definitions without string comparisons or linker collisions.

rewrite_abi_converter_calls!(mod::LLVM.Module)

Julia 1.12+ lowers @cfunction to a world-age-guarded dispatch site that calls jl_get_abi_converter to obtain a callable pointer for the target in the current world. That runtime resolver picks a code instance from the native JIT cache, which is the wrong one for GPUCompiler-emitted code: both its owner (ci->owner) and its ABI (compiled with gcstack_arg = true, i.e. pgcstack in the swiftself register) differ from this module (gcstack_arg = false), so the raw specsig pointer handed back is called with a mismatched ABI and reads a garbage GC stack out of the swiftself register (#3284).

Rewrite each such site to act like jl_apply_generic instead: Julia's codegen already emitted an in-module unspecialized dispatcher thunk for every site (stored in the cfuncdata global) that boxes the arguments and dispatches via jl_apply_generic. That thunk was compiled with this module's own ABI and resolves the callee from the correct cache in the current world, so replace every jl_get_abi_converter call with it and let the world-age guard fold away.

Enzyme.guess_activity(::Type{T}, mode::Enzyme.Mode)

Try to guess the most appropriate Annotation for arguments of type T passed to autodiff with a given mode.