Gateway

ReactantServerGateway is the KServe V2 gRPC reverse proxy that fronts the workers of a multi-GPU node. The node supervisor runs it as an embedded child when there is more than one worker. It depends only on ReactantServerCore and the gRPC/HTTP layer, never on Reactant. See Multi-GPU Gateway for the operational view.

ReactantServerGateway.serve_gateway — Function

serve_gateway(gateway_path=nothing; blocking=true) -> nothing | RunningGateway

Load gateway.yml (listen addresses and the worker endpoint list), build the worker client pool, start the admin HTTP server and the readiness/discovery prober (which probes each endpoint's ServerReady and RepositoryIndex and swaps in the discovered routing table), and serve the KServe gRPC proxy. When blocking is false the server runs in the background and a RunningGateway is returned (stop it with stop!).

gateway_path may be omitted (or nothing) to configure the gateway from defaults and REACTANT_GATEWAY_* environment variables alone; the endpoint list then comes from REACTANT_GATEWAY_WORKERS. The node supervisor uses this to run an embedded gateway without a gateway.yml.

ReactantServerGateway.probe_worker_ready — Function

probe_worker_ready(node_path, worker=nothing) -> Bool

Resolve a worker's port from the node file and call its KServe ServerReady on localhost, returning whether it reported ready. Used as the worker container's healthcheck (a Julia replacement for the former Go reactant-healthprobe). worker may be omitted when the node has a single worker.

ReactantServerGateway.RunningGateway — Type

RunningGateway

Handle to a gateway started with serve_gateway(...; blocking=false). Pass it to stop! to shut the gRPC server, the readiness prober, and the admin HTTP server down.

ReactantServerGateway.stop! — Function

stop!(g::RunningGateway)

Shut down a gateway started with serve_gateway(...; blocking=false): close the gRPC server, halt the readiness prober, and close the admin HTTP server.