Profiling
Quick profiling in your terminal
Note
This is only meant to be used for quick profiling or programmatically accessing the profiling results. For more detailed and GUI friendly profiling proceed to the next section.
Simply replace the use of Base.@time or Base.@timed with Reactant.Profiler.@time or Reactant.Profiler.@timed. We will automatically compile the function if it is not already a Reactant compiled function (with sync=true).
julia
using Reactant
x = Reactant.to_rarray(randn(Float32, 100, 2))
W = Reactant.to_rarray(randn(Float32, 10, 100))
b = Reactant.to_rarray(randn(Float32, 10))
linear(x, W, b) = (W * x) .+ b
Reactant.@time linear(x, W, b)┌ Debug: Profiling directory: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl
└ @ Reactant.Profiler ~/work/Reactant.jl/Reactant.jl/src/Profiler.jl:626
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1787707824.509099 4011 profiler_session.cc:171] Profiler session initializing.
I0000 00:00:1787707824.509146 4011 profiler_session.cc:186] Profiler session started.
I0000 00:00:1787707824.509598 4011 profiler_session.cc:134] Profiler session collecting data.
I0000 00:00:1787707824.574542 4011 save_profile.cc:205] Collecting XSpace to repository: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl/plugins/profile/2026_08_26_01_30_24/runnervm76f27.xplane.pb
I0000 00:00:1787707824.575598 4011 save_profile.cc:178] Creating directory: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl/plugins/profile/2026_08_26_01_30_24
I0000 00:00:1787707824.575696 4011 save_profile.cc:184] Dumped gzipped tool data for trace.json.gz to /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl/plugins/profile/2026_08_26_01_30_24/runnervm76f27.trace.json.gz
I0000 00:00:1787707824.575713 4011 profiler_session.cc:217] Profiler session tear down.
┌ Debug: Starting XProf gRPC server...
└ @ Reactant.Profiler ~/work/Reactant.jl/Reactant.jl/src/Profiler.jl:598
┌ Debug: Initializing XProf stubs for worker service at 0.0.0.0:46323
└ @ Reactant.Profiler ~/work/Reactant.jl/Reactant.jl/src/Profiler.jl:397
I0000 00:00:1787707824.587924 4011 stub_factory.cc:163] Created gRPC channel for address: 0.0.0.0:46323
┌ Debug: Starting XProf gRPC server on port 46323
└ @ Reactant.Profiler ~/work/Reactant.jl/Reactant.jl/src/Profiler.jl:413
I0000 00:00:1787707824.588353 4011 grpc_server.cc:94] Server listening on 0.0.0.0:46323 with max_concurrent_requests 1
I0000 00:00:1787707824.595461 4011 xplane_to_tools_data_with_profile_processor.cc:142] serving tool: memory_profile with options: {} using ProfileProcessor session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl/plugins/profile/2026_08_26_01_30_24
I0000 00:00:1787707824.595474 4011 xplane_to_tools_data_with_profile_processor.cc:165] Using local processing for tool: memory_profile
I0000 00:00:1787707824.595476 4011 memory_profile_processor.cc:47] Processing memory profile for host: runnervm76f27
I0000 00:00:1787707824.596016 4011 xplane_to_tools_data_with_profile_processor.cc:170] Total time for tool memory_profile: 548.097us session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl/plugins/profile/2026_08_26_01_30_24
I0000 00:00:1787707824.606070 4011 xplane_to_tools_data_with_profile_processor.cc:142] serving tool: op_profile with options: {} using ProfileProcessor session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl/plugins/profile/2026_08_26_01_30_24
I0000 00:00:1787707824.606083 4011 xplane_to_tools_data_with_profile_processor.cc:165] Using local processing for tool: op_profile
I0000 00:00:1787707824.606085 4011 multi_xplanes_to_op_stats.cc:118] ConvertMultiXSpaceToCombinedOpStatsWithCache: Started
I0000 00:00:1787707824.606117 4011 multi_xplanes_to_op_stats.cc:134] ConvertMultiXSpaceToCombinedOpStatsWithCache: Cache miss, calling ConvertMultiXSpacesToCombinedOpStats
I0000 00:00:1787707824.606119 4011 multi_xplanes_to_op_stats.cc:45] ConvertMultiXSpacesToCombinedOpStats: Started. Number of XSpaces: 1
I0000 00:00:1787707824.606122 4011 multi_xplanes_to_op_stats.cc:55] ConvertMultiXSpacesToCombinedOpStats: Starting to process XSpace 0/1
I0000 00:00:1787707824.606451 4011 derived_timeline.cc:693] GenerateDerivedTimeLines: creating derived_timeline_trace_events XprofThreadPoolExecutor
I0000 00:00:1787707824.606462 4011 xprof_thread_pool_executor.cc:22] Creating derived_timeline_trace_events XprofThreadPoolExecutor with 4 threads.
I0000 00:00:1787707824.606613 4011 derived_timeline.cc:705] GenerateDerivedTimeLines: waiting for derived_timeline_trace_events threads to join
I0000 00:00:1787707824.606894 4011 derived_timeline.cc:709] GenerateDerivedTimeLines: derived_timeline_trace_events threads joined successfully
I0000 00:00:1787707824.697601 4011 derived_timeline.cc:758] GenerateDerivedTimeLines: creating ProcessTensorCorePlanes XprofThreadPoolExecutor
I0000 00:00:1787707824.707092 4011 xprof_thread_pool_executor.cc:22] Creating ProcessTensorCorePlanes XprofThreadPoolExecutor with 4 threads.
I0000 00:00:1787707824.707278 4011 derived_timeline.cc:769] GenerateDerivedTimeLines: waiting for ProcessTensorCorePlanes threads to join
I0000 00:00:1787707824.707602 4011 derived_timeline.cc:772] GenerateDerivedTimeLines: ProcessTensorCorePlanes threads joined successfully
I0000 00:00:1787707824.858590 4011 xplane_to_op_stats.cc:405] ConvertXSpaceToOpStats: creating op_stats_threads XprofThreadPoolExecutor
I0000 00:00:1787707824.868238 4011 xprof_thread_pool_executor.cc:22] Creating op_stats_threads XprofThreadPoolExecutor with 4 threads.
I0000 00:00:1787707824.868720 4011 xplane_to_op_stats.cc:461] ConvertXSpaceToOpStats: Scheduled 0 OpMetricsDb generation tasks.
I0000 00:00:1787707824.869262 4011 xplane_to_op_stats.cc:417] ConvertXSpaceToOpStats: Combining 0 op_metrics_dbs.
I0000 00:00:1787707824.869269 4011 xplane_to_op_stats.cc:422] ConvertXSpaceToOpStats: Finished combining op_metrics_dbs.
I0000 00:00:1787707824.885949 4011 xplane_to_op_stats.cc:687] ConvertXSpaceToOpStats: Final OpStats size: 267 bytes (0.000254631 MiB).
I0000 00:00:1787707824.926703 4011 multi_xplanes_to_op_stats.cc:67] ConvertMultiXSpacesToCombinedOpStats: Finished processing XSpace 0.
I0000 00:00:1787707824.927249 4011 multi_xplanes_to_op_stats.cc:72] ConvertMultiXSpacesToCombinedOpStats: Finished extracting all 1 OpStats. Time: 321.133329ms
I0000 00:00:1787707824.927260 4011 multi_xplanes_to_op_stats.cc:85] ConvertMultiXSpacesToCombinedOpStats: Starting ComputeStepIntersectionToMergeOpStats.
I0000 00:00:1787707824.927264 4011 multi_xplanes_to_op_stats.cc:94] ConvertMultiXSpacesToCombinedOpStats: Finished ComputeStepIntersectionToMergeOpStats in 2.343us
I0000 00:00:1787707824.927265 4011 multi_xplanes_to_op_stats.cc:99] ConvertMultiXSpacesToCombinedOpStats: Starting CombineAllOpStats.
I0000 00:00:1787707824.927275 4011 multi_xplanes_to_op_stats.cc:106] ConvertMultiXSpacesToCombinedOpStats: Finished CombineAllOpStats in 8.215us
I0000 00:00:1787707824.927276 4011 multi_xplanes_to_op_stats.cc:109] ConvertMultiXSpacesToCombinedOpStats: Overall Finished in 321.157535ms
I0000 00:00:1787707824.927278 4011 multi_xplanes_to_op_stats.cc:138] ConvertMultiXSpaceToCombinedOpStatsWithCache: Starting to write cache file.
I0000 00:00:1787707824.927392 4011 multi_xplanes_to_op_stats.cc:145] ConvertMultiXSpaceToCombinedOpStatsWithCache: Finished writing cache file.
I0000 00:00:1787707824.927393 4011 multi_xplanes_to_op_stats.cc:149] ConvertMultiXSpaceToCombinedOpStatsWithCache: Overall Finished in 321.30881ms
I0000 00:00:1787707824.927534 4011 xplane_to_tools_data_with_profile_processor.cc:170] Total time for tool op_profile: 321.455237ms session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl/plugins/profile/2026_08_26_01_30_24
┌ Debug: `op_profile` data missing keys for metrics
│ data_available_keys =
│ KeySet for a JSON.Object{String, Any} with 4 entries. Keys:
│ "byProgram"
│ "deviceType"
│ "byProgramExcludeIdle"
│ "aggDvfsTimeScaleMultiplier"
│ by_program_available_keys =
│ KeySet for a JSON.Object{String, Any} with 3 entries. Keys:
│ "name"
│ "children"
│ "numChildren"
└ @ Reactant.Profiler ~/work/Reactant.jl/Reactant.jl/src/Profiler.jl:828
I0000 00:00:1787707825.141167 4011 xplane_to_tools_data_with_profile_processor.cc:142] serving tool: overview_page with options: {} using ProfileProcessor session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl/plugins/profile/2026_08_26_01_30_24
I0000 00:00:1787707825.141184 4011 xplane_to_tools_data_with_profile_processor.cc:165] Using local processing for tool: overview_page
I0000 00:00:1787707825.141186 4011 overview_page_processor.cc:84] OverviewPageProcessor::ProcessSession: Started
I0000 00:00:1787707825.141188 4011 overview_page_processor.cc:86] OverviewPageProcessor::ProcessSession: Starting ConvertMultiXSpaceToCombinedOpStatsWithCache
I0000 00:00:1787707825.141189 4011 multi_xplanes_to_op_stats.cc:118] ConvertMultiXSpaceToCombinedOpStatsWithCache: Started
I0000 00:00:1787707825.141240 4011 multi_xplanes_to_op_stats.cc:126] ConvertMultiXSpaceToCombinedOpStatsWithCache: Cache hit, reading binary proto
I0000 00:00:1787707825.141277 4011 multi_xplanes_to_op_stats.cc:131] ConvertMultiXSpaceToCombinedOpStatsWithCache: Finished reading cache file.
I0000 00:00:1787707825.141279 4011 multi_xplanes_to_op_stats.cc:149] ConvertMultiXSpaceToCombinedOpStatsWithCache: Overall Finished in 90.65us
I0000 00:00:1787707825.141282 4011 overview_page_processor.cc:90] OverviewPageProcessor::ProcessSession: Starting ConvertOpStatsToOverviewPage
I0000 00:00:1787707825.141290 4011 op_stats_to_overview_page.cc:388] ConvertOpStatsToOverviewPage: Starting ComputeRunEnvironment
I0000 00:00:1787707825.141295 4011 op_stats_to_overview_page.cc:393] ConvertOpStatsToOverviewPage: Starting ComputeAnalysisResult
I0000 00:00:1787707825.141300 4011 op_stats_to_overview_page.cc:396] ConvertOpStatsToOverviewPage: Starting ConvertOpStatsToInputPipelineAnalysis
I0000 00:00:1787707825.141443 4011 op_stats_to_overview_page.cc:401] ConvertOpStatsToOverviewPage: Starting ComputeBottleneckAnalysis
I0000 00:00:1787707825.141448 4011 op_stats_to_overview_page.cc:407] ConvertOpStatsToOverviewPage: Starting ComputeGenericRecommendation
I0000 00:00:1787707825.141532 4011 op_stats_to_overview_page.cc:412] ConvertOpStatsToOverviewPage: Starting SetCommonRecommendation
I0000 00:00:1787707825.141541 4011 op_stats_to_overview_page.cc:425] ConvertOpStatsToOverviewPage: Starting PopulateOverviewDiagnostics
I0000 00:00:1787707825.141543 4011 op_stats_to_overview_page.cc:429] ConvertOpStatsToOverviewPage: Starting setting utilizations
I0000 00:00:1787707825.141544 4011 op_stats_to_overview_page.cc:435] ConvertOpStatsToOverviewPage: Overall Finished in 254.515us
I0000 00:00:1787707825.141545 4011 overview_page_processor.cc:94] OverviewPageProcessor::ProcessSession: Not a training run, Starting to convert inference stats.
I0000 00:00:1787707825.141552 4011 xprof_thread_pool_executor.cc:22] Creating ConvertMultiXSpaceToInferenceStats XprofThreadPoolExecutor with 1 threads.
I0000 00:00:1787707825.143465 4011 overview_page_processor.cc:99] OverviewPageProcessor::ProcessSession: Starting to compute InferenceLatency
I0000 00:00:1787707825.143473 4011 overview_page_processor.cc:104] OverviewPageProcessor::ProcessSession: Starting to serialize OverviewPage toJson
I0000 00:00:1787707825.143669 4011 overview_page_processor.cc:107] OverviewPageProcessor::ProcessSession: Starting to set Output
I0000 00:00:1787707825.143674 4011 overview_page_processor.cc:109] OverviewPageProcessor::ProcessSession: Overall Finished in 2.488685ms
I0000 00:00:1787707825.143684 4011 xplane_to_tools_data_with_profile_processor.cc:170] Total time for tool overview_page: 2.504685ms session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_oyVnHl/plugins/profile/2026_08_26_01_30_24
runtime: 0.00039380s
compile time: 2.64405119sjulia
Reactant.@timed nrepeat=100 linear(x, W, b)AggregateProfilingResult(
runtime = 0.00002885s,
compile_time = 0.09680587s, )Note that the information returned depends on the backend. Specifically CUDA and TPU backends provide more detailed information regarding memory usage and allocation (something like the following will be displayed on GPUs):
julia
AggregateProfilingResult(
runtime = 0.00003829s,
compile_time = 2.18053260s, # time spent compiling by Reactant
GPU_0_bfc = MemoryProfileSummary(
peak_bytes_usage_lifetime = 64.010 MiB, # peak memory usage over the entire program (lifetime of memory allocator)
peak_stats = MemoryAggregationStats(
stack_reserved_bytes = 0 bytes, # memory usage by stack reservation
heap_allocated_bytes = 30.750 KiB, # memory usage by heap allocation
free_memory_bytes = 23.518 GiB, # free memory available for allocation or reservation
fragmentation = 0.514931, # fragmentation of memory within [0, 1]
peak_bytes_in_use = 30.750 KiB # The peak memory usage over the entire program
)
peak_stats_time = 0.04975365s,
memory_capacity = 23.518 GiB # memory capacity of the allocator
)
flops = FlopsSummary(
Flops = 2.8369974648038653e-9, # [flops / (peak flops * program time)], capped at 1.0
UncappedFlops = 2.8369974648038653e-9,
RawFlops = 4060.0, # Total FLOPs performed
BF16Flops = 4060.0, # Total FLOPs Normalized to the bf16 (default) devices peak bandwidth
RawTime = 0.00040298422s, # Raw time in seconds
RawFlopsRate = 1.0074836180930361e7, # Raw FLOPs rate in FLOPs/seconds
BF16FlopsRate = 1.0074836180930361e7, # BF16 FLOPs rate in FLOPs/seconds
)
)Additionally for GPUs and TPUs, we can use the Reactant.@profile macro to profile the function and get information regarding each of the kernels executed.
julia
Reactant.@profile linear(x, W, b)┌ Debug: Profiling directory: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM
└ @ Reactant.Profiler ~/work/Reactant.jl/Reactant.jl/src/Profiler.jl:626
I0000 00:00:1787707826.063662 4011 profiler_session.cc:171] Profiler session initializing.
I0000 00:00:1787707826.063873 4011 profiler_session.cc:186] Profiler session started.
I0000 00:00:1787707826.063987 4011 profiler_session.cc:134] Profiler session collecting data.
I0000 00:00:1787707826.126812 4011 save_profile.cc:205] Collecting XSpace to repository: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26/runnervm76f27.xplane.pb
I0000 00:00:1787707826.128111 4011 save_profile.cc:178] Creating directory: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
I0000 00:00:1787707826.128215 4011 save_profile.cc:184] Dumped gzipped tool data for trace.json.gz to /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26/runnervm76f27.trace.json.gz
I0000 00:00:1787707826.128235 4011 profiler_session.cc:217] Profiler session tear down.
I0000 00:00:1787707826.128293 4011 xplane_to_tools_data_with_profile_processor.cc:142] serving tool: memory_profile with options: {} using ProfileProcessor session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
I0000 00:00:1787707826.128298 4011 xplane_to_tools_data_with_profile_processor.cc:165] Using local processing for tool: memory_profile
I0000 00:00:1787707826.128299 4011 memory_profile_processor.cc:47] Processing memory profile for host: runnervm76f27
I0000 00:00:1787707826.128686 4011 xplane_to_tools_data_with_profile_processor.cc:170] Total time for tool memory_profile: 393.483us session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
I0000 00:00:1787707826.128717 4011 xplane_to_tools_data_with_profile_processor.cc:142] serving tool: op_profile with options: {} using ProfileProcessor session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
I0000 00:00:1787707826.128720 4011 xplane_to_tools_data_with_profile_processor.cc:165] Using local processing for tool: op_profile
I0000 00:00:1787707826.128722 4011 multi_xplanes_to_op_stats.cc:118] ConvertMultiXSpaceToCombinedOpStatsWithCache: Started
I0000 00:00:1787707826.128745 4011 multi_xplanes_to_op_stats.cc:134] ConvertMultiXSpaceToCombinedOpStatsWithCache: Cache miss, calling ConvertMultiXSpacesToCombinedOpStats
I0000 00:00:1787707826.128747 4011 multi_xplanes_to_op_stats.cc:45] ConvertMultiXSpacesToCombinedOpStats: Started. Number of XSpaces: 1
I0000 00:00:1787707826.128751 4011 multi_xplanes_to_op_stats.cc:55] ConvertMultiXSpacesToCombinedOpStats: Starting to process XSpace 0/1
I0000 00:00:1787707826.129075 4011 derived_timeline.cc:693] GenerateDerivedTimeLines: creating derived_timeline_trace_events XprofThreadPoolExecutor
I0000 00:00:1787707826.129085 4011 xprof_thread_pool_executor.cc:22] Creating derived_timeline_trace_events XprofThreadPoolExecutor with 4 threads.
I0000 00:00:1787707826.129247 4011 derived_timeline.cc:705] GenerateDerivedTimeLines: waiting for derived_timeline_trace_events threads to join
I0000 00:00:1787707826.129570 4011 derived_timeline.cc:709] GenerateDerivedTimeLines: derived_timeline_trace_events threads joined successfully
I0000 00:00:1787707826.215183 4011 derived_timeline.cc:758] GenerateDerivedTimeLines: creating ProcessTensorCorePlanes XprofThreadPoolExecutor
I0000 00:00:1787707826.224746 4011 xprof_thread_pool_executor.cc:22] Creating ProcessTensorCorePlanes XprofThreadPoolExecutor with 4 threads.
I0000 00:00:1787707826.224930 4011 derived_timeline.cc:769] GenerateDerivedTimeLines: waiting for ProcessTensorCorePlanes threads to join
I0000 00:00:1787707826.225324 4011 derived_timeline.cc:772] GenerateDerivedTimeLines: ProcessTensorCorePlanes threads joined successfully
I0000 00:00:1787707826.364580 4011 xplane_to_op_stats.cc:405] ConvertXSpaceToOpStats: creating op_stats_threads XprofThreadPoolExecutor
I0000 00:00:1787707826.374466 4011 xprof_thread_pool_executor.cc:22] Creating op_stats_threads XprofThreadPoolExecutor with 4 threads.
I0000 00:00:1787707826.374786 4011 xplane_to_op_stats.cc:461] ConvertXSpaceToOpStats: Scheduled 0 OpMetricsDb generation tasks.
I0000 00:00:1787707826.375086 4011 xplane_to_op_stats.cc:417] ConvertXSpaceToOpStats: Combining 0 op_metrics_dbs.
I0000 00:00:1787707826.375096 4011 xplane_to_op_stats.cc:422] ConvertXSpaceToOpStats: Finished combining op_metrics_dbs.
I0000 00:00:1787707826.391908 4011 xplane_to_op_stats.cc:687] ConvertXSpaceToOpStats: Final OpStats size: 307 bytes (0.000292778 MiB).
I0000 00:00:1787707826.428985 4011 multi_xplanes_to_op_stats.cc:67] ConvertMultiXSpacesToCombinedOpStats: Finished processing XSpace 0.
I0000 00:00:1787707826.429804 4011 multi_xplanes_to_op_stats.cc:72] ConvertMultiXSpacesToCombinedOpStats: Finished extracting all 1 OpStats. Time: 301.204153ms
I0000 00:00:1787707826.429958 4011 multi_xplanes_to_op_stats.cc:85] ConvertMultiXSpacesToCombinedOpStats: Starting ComputeStepIntersectionToMergeOpStats.
I0000 00:00:1787707826.429961 4011 multi_xplanes_to_op_stats.cc:94] ConvertMultiXSpacesToCombinedOpStats: Finished ComputeStepIntersectionToMergeOpStats in 2.296us
I0000 00:00:1787707826.429963 4011 multi_xplanes_to_op_stats.cc:99] ConvertMultiXSpacesToCombinedOpStats: Starting CombineAllOpStats.
I0000 00:00:1787707826.429973 4011 multi_xplanes_to_op_stats.cc:106] ConvertMultiXSpacesToCombinedOpStats: Finished CombineAllOpStats in 8.198us
I0000 00:00:1787707826.429974 4011 multi_xplanes_to_op_stats.cc:109] ConvertMultiXSpacesToCombinedOpStats: Overall Finished in 301.227498ms
I0000 00:00:1787707826.429977 4011 multi_xplanes_to_op_stats.cc:138] ConvertMultiXSpaceToCombinedOpStatsWithCache: Starting to write cache file.
I0000 00:00:1787707826.430099 4011 multi_xplanes_to_op_stats.cc:145] ConvertMultiXSpaceToCombinedOpStatsWithCache: Finished writing cache file.
I0000 00:00:1787707826.430101 4011 multi_xplanes_to_op_stats.cc:149] ConvertMultiXSpaceToCombinedOpStatsWithCache: Overall Finished in 301.379417ms
I0000 00:00:1787707826.430133 4011 xplane_to_tools_data_with_profile_processor.cc:170] Total time for tool op_profile: 301.41749ms session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
┌ Debug: `op_profile` data missing keys for metrics
│ data_available_keys =
│ KeySet for a JSON.Object{String, Any} with 4 entries. Keys:
│ "byProgram"
│ "deviceType"
│ "byProgramExcludeIdle"
│ "aggDvfsTimeScaleMultiplier"
│ by_program_available_keys =
│ KeySet for a JSON.Object{String, Any} with 3 entries. Keys:
│ "name"
│ "children"
│ "numChildren"
└ @ Reactant.Profiler ~/work/Reactant.jl/Reactant.jl/src/Profiler.jl:828
I0000 00:00:1787707826.430554 4011 xplane_to_tools_data_with_profile_processor.cc:142] serving tool: overview_page with options: {} using ProfileProcessor session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
I0000 00:00:1787707826.430559 4011 xplane_to_tools_data_with_profile_processor.cc:165] Using local processing for tool: overview_page
I0000 00:00:1787707826.430561 4011 overview_page_processor.cc:84] OverviewPageProcessor::ProcessSession: Started
I0000 00:00:1787707826.430562 4011 overview_page_processor.cc:86] OverviewPageProcessor::ProcessSession: Starting ConvertMultiXSpaceToCombinedOpStatsWithCache
I0000 00:00:1787707826.430563 4011 multi_xplanes_to_op_stats.cc:118] ConvertMultiXSpaceToCombinedOpStatsWithCache: Started
I0000 00:00:1787707826.430596 4011 multi_xplanes_to_op_stats.cc:126] ConvertMultiXSpaceToCombinedOpStatsWithCache: Cache hit, reading binary proto
I0000 00:00:1787707826.430624 4011 multi_xplanes_to_op_stats.cc:131] ConvertMultiXSpaceToCombinedOpStatsWithCache: Finished reading cache file.
I0000 00:00:1787707826.430626 4011 multi_xplanes_to_op_stats.cc:149] ConvertMultiXSpaceToCombinedOpStatsWithCache: Overall Finished in 62.808us
I0000 00:00:1787707826.430627 4011 overview_page_processor.cc:90] OverviewPageProcessor::ProcessSession: Starting ConvertOpStatsToOverviewPage
I0000 00:00:1787707826.430629 4011 op_stats_to_overview_page.cc:388] ConvertOpStatsToOverviewPage: Starting ComputeRunEnvironment
I0000 00:00:1787707826.430633 4011 op_stats_to_overview_page.cc:393] ConvertOpStatsToOverviewPage: Starting ComputeAnalysisResult
I0000 00:00:1787707826.430637 4011 op_stats_to_overview_page.cc:396] ConvertOpStatsToOverviewPage: Starting ConvertOpStatsToInputPipelineAnalysis
I0000 00:00:1787707826.430677 4011 op_stats_to_overview_page.cc:401] ConvertOpStatsToOverviewPage: Starting ComputeBottleneckAnalysis
I0000 00:00:1787707826.430679 4011 op_stats_to_overview_page.cc:407] ConvertOpStatsToOverviewPage: Starting ComputeGenericRecommendation
I0000 00:00:1787707826.430684 4011 op_stats_to_overview_page.cc:412] ConvertOpStatsToOverviewPage: Starting SetCommonRecommendation
I0000 00:00:1787707826.430690 4011 op_stats_to_overview_page.cc:425] ConvertOpStatsToOverviewPage: Starting PopulateOverviewDiagnostics
I0000 00:00:1787707826.430692 4011 op_stats_to_overview_page.cc:429] ConvertOpStatsToOverviewPage: Starting setting utilizations
I0000 00:00:1787707826.430693 4011 op_stats_to_overview_page.cc:435] ConvertOpStatsToOverviewPage: Overall Finished in 64.329us
I0000 00:00:1787707826.430694 4011 overview_page_processor.cc:94] OverviewPageProcessor::ProcessSession: Not a training run, Starting to convert inference stats.
I0000 00:00:1787707826.430698 4011 xprof_thread_pool_executor.cc:22] Creating ConvertMultiXSpaceToInferenceStats XprofThreadPoolExecutor with 1 threads.
I0000 00:00:1787707826.432419 4011 overview_page_processor.cc:99] OverviewPageProcessor::ProcessSession: Starting to compute InferenceLatency
I0000 00:00:1787707826.432425 4011 overview_page_processor.cc:104] OverviewPageProcessor::ProcessSession: Starting to serialize OverviewPage toJson
I0000 00:00:1787707826.432624 4011 overview_page_processor.cc:107] OverviewPageProcessor::ProcessSession: Starting to set Output
I0000 00:00:1787707826.432629 4011 overview_page_processor.cc:109] OverviewPageProcessor::ProcessSession: Overall Finished in 2.068955ms
I0000 00:00:1787707826.432639 4011 xplane_to_tools_data_with_profile_processor.cc:170] Total time for tool overview_page: 2.083425ms session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
I0000 00:00:1787707826.505590 4011 xplane_to_tools_data_with_profile_processor.cc:142] serving tool: kernel_stats with options: {} using ProfileProcessor session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
I0000 00:00:1787707826.505609 4011 xplane_to_tools_data_with_profile_processor.cc:165] Using local processing for tool: kernel_stats
I0000 00:00:1787707826.505611 4011 multi_xplanes_to_op_stats.cc:118] ConvertMultiXSpaceToCombinedOpStatsWithCache: Started
I0000 00:00:1787707826.505654 4011 multi_xplanes_to_op_stats.cc:126] ConvertMultiXSpaceToCombinedOpStatsWithCache: Cache hit, reading binary proto
I0000 00:00:1787707826.505690 4011 multi_xplanes_to_op_stats.cc:131] ConvertMultiXSpaceToCombinedOpStatsWithCache: Finished reading cache file.
I0000 00:00:1787707826.505691 4011 multi_xplanes_to_op_stats.cc:149] ConvertMultiXSpaceToCombinedOpStatsWithCache: Overall Finished in 80.486us
I0000 00:00:1787707826.505741 4011 xplane_to_tools_data_with_profile_processor.cc:170] Total time for tool kernel_stats: 136.155us session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
I0000 00:00:1787707826.603061 4011 xplane_to_tools_data_with_profile_processor.cc:142] serving tool: framework_op_stats with options: {} using ProfileProcessor session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
I0000 00:00:1787707826.603079 4011 xplane_to_tools_data_with_profile_processor.cc:165] Using local processing for tool: framework_op_stats
I0000 00:00:1787707826.603081 4011 multi_xplanes_to_op_stats.cc:118] ConvertMultiXSpaceToCombinedOpStatsWithCache: Started
I0000 00:00:1787707826.603114 4011 multi_xplanes_to_op_stats.cc:126] ConvertMultiXSpaceToCombinedOpStatsWithCache: Cache hit, reading binary proto
I0000 00:00:1787707826.603142 4011 multi_xplanes_to_op_stats.cc:131] ConvertMultiXSpaceToCombinedOpStatsWithCache: Finished reading cache file.
I0000 00:00:1787707826.603144 4011 multi_xplanes_to_op_stats.cc:149] ConvertMultiXSpaceToCombinedOpStatsWithCache: Overall Finished in 62.709us
I0000 00:00:1787707826.603261 4011 xplane_to_tools_data_with_profile_processor.cc:170] Total time for tool framework_op_stats: 184.018us session_id: /home/runner/.julia/scratchspaces/3c362404-f566-11ee-1572-e11a4b42c853/reactant_profiling/jl_TnviVM/plugins/profile/2026_08_26_01_30_26
╔================================================================================╗
║ SUMMARY ║
╚================================================================================╝
AggregateProfilingResult(
runtime = 0.00008750s,
compile_time = 0.09142876s, # time spent compiling by Reactant
)On GPUs this would look something like the following:
julia
╔================================================================================╗
║ KERNEL STATISTICS ║
╚================================================================================╝
┌───────────────────┬─────────────┬────────────────┬──────────────┬──────────────┬──────────────┬──────────────┬───────────┬──────────┬────────────┬─────────────┐
│ Kernel Name │ Occurrences │ Total Duration │ Avg Duration │ Min Duration │ Max Duration │ Static Shmem │ Block Dim │ Grid Dim │ TensorCore │ Occupancy % │
├───────────────────┼─────────────┼────────────────┼──────────────┼──────────────┼──────────────┼──────────────┼───────────┼──────────┼────────────┼─────────────┤
│ gemm_fusion_dot_1 │ 1 │ 0.00000250s │ 0.00000250s │ 0.00000250s │ 0.00000250s │ 2.000 KiB │ 64,1,1 │ 1,1,1 │ ✗ │ 100.0% │
│ loop_add_fusion │ 1 │ 0.00000131s │ 0.00000131s │ 0.00000131s │ 0.00000131s │ 0 bytes │ 20,1,1 │ 1,1,1 │ ✗ │ 31.2% │
└───────────────────┴─────────────┴────────────────┴──────────────┴──────────────┴──────────────┴──────────────┴───────────┴──────────┴────────────┴─────────────┘
╔================================================================================╗
║ FRAMEWORK OP STATISTICS ║
╚================================================================================╝
┌───────────────────┬─────────┬─────────────┬─────────────┬─────────────────┬───────────────┬──────────┬───────────┬──────────────┬──────────┐
│ Operation │ Type │ Host/Device │ Occurrences │ Total Self-Time │ Avg Self-Time │ Device % │ Memory BW │ FLOP Rate │ Bound By │
├───────────────────┼─────────┼─────────────┼─────────────┼─────────────────┼───────────────┼──────────┼───────────┼──────────────┼──────────┤
│ gemm_fusion_dot.1 │ Unknown │ Device │ 1 │ 0.00000250s │ 0.00000250s │ 65.55% │ 1.82 GB/s │ 1.6 GFLOP/s │ HBM │
│ +/add │ add │ Device │ 1 │ 0.00000131s │ 0.00000131s │ 34.45% │ 0.14 GB/s │ 0.05 GFLOP/s │ HBM │
└───────────────────┴─────────┴─────────────┴─────────────┴─────────────────┴───────────────┴──────────┴───────────┴──────────────┴──────────┘
╔================================================================================╗
║ SUMMARY ║
╚================================================================================╝
AggregateProfilingResult(
runtime = 0.00005622s,
compile_time = 2.32802137s, # time spent compiling by Reactant
GPU_0_bfc = MemoryProfileSummary(
peak_bytes_usage_lifetime = 64.010 MiB, # peak memory usage over the entire program (lifetime of memory allocator)
peak_stats = MemoryAggregationStats(
stack_reserved_bytes = 0 bytes, # memory usage by stack reservation
heap_allocated_bytes = 81.750 KiB, # memory usage by heap allocation
free_memory_bytes = 23.518 GiB, # free memory available for allocation or reservation
fragmentation = 0.514564, # fragmentation of memory within [0, 1]
peak_bytes_in_use = 81.750 KiB # The peak memory usage over the entire program
)
peak_stats_time = 0.00608052s,
memory_capacity = 23.518 GiB # memory capacity of the allocator
)
flops = FlopsSummary(
Flops = 2.033375207640664e-8, # [flops / (peak flops * program time)], capped at 1.0
UncappedFlops = 2.033375207640664e-8,
RawFlops = 4060.0, # Total FLOPs performed
BF16Flops = 4060.0, # Total FLOPs Normalized to the bf16 (default) devices peak bandwidth
RawTime = 0.00005622s, # Raw time in seconds
RawFlopsRate = 7.220987105380169e7, # Raw FLOPs rate in FLOPs/seconds
BF16FlopsRate = 7.220987105380169e7, # BF16 FLOPs rate in FLOPs/seconds
)
)Capturing traces
When running Reactant, it is possible to capture traces using the XLA profiler. These traces can provide information about where the XLA specific parts of program spend time during compilation or execution. Note that tracing and compilation happen on the CPU even though the final execution is aimed to run on another device such as GPU or TPU. Therefore, including tracing and compilation in a trace will create annotations on the CPU.
Let's setup a simple function which we can then profile
julia
using Reactant
x = Reactant.to_rarray(randn(Float32, 100, 2))
W = Reactant.to_rarray(randn(Float32, 10, 100))
b = Reactant.to_rarray(randn(Float32, 10))
linear(x, W, b) = (W * x) .+ blinear (generic function with 1 method)The profiler can be accessed using the Reactant.with_profiler function.
julia
Reactant.with_profiler("./") do
mylinear = Reactant.@compile linear(x, W, b)
mylinear(x, W, b)
end10×2 ConcretePJRTArray{Float32,2}:
5.37052 4.30388
3.58955 4.64127
-4.71294 0.0461382
-11.1881 -8.16435
4.63452 0.1246
-2.50214 2.56202
-5.20425 -4.00613
-9.56165 15.1854
1.91262 -0.906992
-20.4187 11.7293Running this function should create a folder called plugins in the folder provided to Reactant.with_profiler which will contain the trace files. The traces can then be visualized in different ways.
Note
For more insights about the current state of Reactant, it is possible to fetch device information about allocations using the Reactant.XLA.allocatorstats function.
Perfetto UI
The first and easiest way to visualize a captured trace is to use the online perfetto.dev tool. Reactant.with_profiler has a keyword parameter called create_perfetto_link which will create a usable perfetto URL for the generated trace. The function will block execution until the URL has been clicked and the trace is visualized. The URL only works once.
julia
Reactant.with_profiler("./"; create_perfetto_link=true) do
mylinear = Reactant.@compile linear(x, W, b)
mylinear(x, W, b)
endNote
It is recommended to use the Chrome browser to open the perfetto URL.
XProf
XProf is a complete web UI to analyze the log files captured by Reactant. It can be installed in the following manner:
bash
pip install xprof # or xprof-nightlyLaunching xprof is then as simple as:
bash
xprof --logdir=./which will then make the xprof interface available on port :8791 by default.
Tensorboard
Another option to visualize the generated trace files is to use the tensorboard profiler plugin. The tensorboard viewer can offer more details than the timeline view such as visualization for compute graphs.
First install tensorboard and its profiler plugin:
bash
pip install tensorboard tensorboard-plugin-profileAnd then run the following in the folder where the plugins folder was generated:
bash
tensorboard --logdir ./Adding Custom Annotations
By default, the traces contain only information captured from within XLA. The Reactant.Profiler.annotate function can be used to annotate traces for Julia code evaluated during tracing.
julia
Reactant.Profiler.annotate("my_annotation") do
# Do things...
endThe added annotations will be captured in the traces and can be seen in the different viewers along with the default XLA annotations. When the profiler is not activated, then the custom annotations have no effect and can therefore always be activated.