GPU Offload in Rust: Portable, Safe, and Fast

# GPU Offload in Rust: Portable, Safe, and Fast
Source: [https://arxiv.org/html/2608.13759](https://arxiv.org/html/2608.13759)
Manuel S\. Drehwaldemail:[manuel\.drehwald@utoronto\.ca](mailto:[email protected])Affiliation:University of Toronto,Toronto,CanadaAffiliation:Lawrence Livermore National Laboratory,Livermore,California,USAAlternate Affiliation:Vector Institute for Artificial Intelligence,Toronto,CanadaMarcelo Domínguezemail:[ma\.dominguez\.2022@alumnos\.urjc\.es](mailto:[email protected])Affiliation:Universidad Rey Juan Carlos,Madrid,Spain,Kevin Salaemail:[salapenades1@llnl\.gov](mailto:[email protected])Affiliation:Lawrence Livermore National Laboratory,Livermore,California,USA,Alán Aspuru\-Guzikemail:[alan@aspuru\.com](mailto:[email protected])Affiliation:Department of Computer Science,University of Toronto,Toronto,Ontario,CanadaAlternate Affiliation:Department of Chemistry,University of Toronto,Toronto,Ontario,CanadaAlternate Affiliation:Department of Materials Science & Engineering,University of Toronto,Toronto,Ontario,CanadaAlternate Affiliation:Department of Chemical Engineering & Applied Chemistry,University of Toronto,Toronto,Ontario,CanadaAlternate Affiliation:Institute of Medical Science,University of Toronto,Toronto,Ontario,CanadaAlternate Affiliation:Vector Institute for Artificial Intelligence,Toronto,Ontario,CanadaAlternate Affiliation:Acceleration Consortium,Toronto,Ontario,CanadaAlternate Affiliation:Canadian Institute for Advanced Research \(CIFAR\),Toronto,Ontario,CanadaandJohannes Doerfertemail:[jdoerfert@llnl\.gov](mailto:[email protected])Affiliation:Lawrence Livermore National Laboratory,Livermore,California,USA

###### Abstract\.

High\-performance GPU programming has traditionally forced a compromise between execution efficiency and memory safety\. While Rust guarantees compile\-time memory safety for host CPUs via its strict ownership model, applying these constraints to massively parallel GPU execution environments has previously mandated either vendor\-locked Domain\-Specific Languages \(DSLs\) or escaping to explicit unsafe raw pointers\. This paper presents a zero\-overhead, multi\-vendor GPU compilation framework built natively into the Rust compiler \(rustc\) and LLVM backends\.

We leverage Rust’s rich type system, ownership system, and strict aliasing guarantees \(n​o​a​l​i​a​snoalias\) to efficiently manage and optimize data transfers through LLVM’s Offload infrastructure\. We expose the technical challenges of cross\-vendor ABI lowering mismatches between Host and Device targets and introduce a two\-pass compilation pipeline capable of safely handling both manual and compiler\-generated memory movements\. Evaluating our framework on RAJAPerf demonstrates that our rustc\-based solution can generate competitive LLVM IR for GPU kernels, achieving a solid kernel performance against native, hand\-optimized CUDA and HIP C\+\+ baselines\.

###### Keywords:

GPU, Offload, LLVM, Rust, Performance, Benchmarks

## 1\.Introduction

High\-performance computing \(HPC\) and scientific applications remain heavily dominated by memory\-unsafe languages like C, C\+\+, or Fortran, with GPU support available through vendor\-specific APIs, e\.g\., CUDA, and HIP, and portable programming models such as OpenMP, and SYCL\([20](https://arxiv.org/html/2608.13759#bib.bib16)\)\. A defining characteristic of these paradigms is that they require developers to explicitly annotate data mappings, manually manage memory transfers, or embed their data into special buffers\. Since these traditional programming languages lack the concept of safety boundaries, the burden of preventing memory corruption and data races falls entirely on the programmer\.

Rust\([19](https://arxiv.org/html/2608.13759#bib.bib10)\)has emerged as a compelling alternative for modern systems programming, steadily gaining traction within the HPC\([21](https://arxiv.org/html/2608.13759#bib.bib7);[1](https://arxiv.org/html/2608.13759#bib.bib6);[33](https://arxiv.org/html/2608.13759#bib.bib5)\)and Scientific Computing\([23](https://arxiv.org/html/2608.13759#bib.bib4);[28](https://arxiv.org/html/2608.13759#bib.bib2);[9](https://arxiv.org/html/2608.13759#bib.bib1);[17](https://arxiv.org/html/2608.13759#bib.bib3)\)communities due to its ability to enforce\([15](https://arxiv.org/html/2608.13759#bib.bib11)\)memory safety and data\-race freedom at compile time\.

Furthermore, Rust’s strict ownership and aliasing guarantees provide benefits that extend beyond basic software correctness; they provide such a good information baseline for the compiler backend, that we can start thinking about interfaces and optimizations that would barely be usable in unsafe languages\. One of the examples is thenoaliasmetadata, which \(almost\) all safe Rust references receive by default, with no manualrestrictannotations required\.

Despite these compiler advantages, utilizing Rust for heterogeneous acceleration remains bottlenecked by the lack of a portable GPU programming interface and a highly fragmented GPU software ecosystem\. Existing GPU interfaces force developers into restrictive execution paradigms\. For instance,rust\-gpu, based on SPIR\-V, has recently been re\-oriented from graphics to compute workloads, but it lacks support for general pointers\. Other compute\-oriented solutions are either restricted to low\-level bindings that require unsafe code regions for all kernels \(rust\-cuda\), or are tightly coupled to specific hardware vendors \(cuda\-oxide\)\. A portable interface and infrastructure that supports safe and idiomatic Rust across multiple vendors has so far been missing\.

To close these gaps, we present a cross\-vendor interface for GPU programming in Rust, integrated directly within the upstream Rust compiler \(rustc\), and based on the LLVM Offload infrastructure\. This architecture generates native code for NVIDIA and AMD GPUs, and can extend to Intel and Apple targets as their upstream LLVM components mature\. Our design introduces a two\-pass compilation pipeline that strictly separates host and device intermediate representations \(IR\)\.

We designed three interfaces to accommodate a broad spectrum of use cases, ranging from managed, out\-of\-the\-box offloading to explicit user\-controlled offloading\. The first interface allows developers to write native GPU kernels in safe and unsafe Rust with automatic data transfers and transparent optimizations\. The second interface integrates vendor libraries like cuBLAS and rocBLAS, building upon these same automatic data\-movement mechanisms\. Lastly, the third interface permits manual execution of data transfers, granting developers absolute control over data movement, similar to CUDA or OpenMP\.

Our key contributions are:

- •Flexible GPU Programming:Three offloading interfaces spanning from convenient wrappers for vendor libraries to explicit, type\-enforced interfaces for custom kernel execution and manual memory management\.
- •Safe memory access strategies:A safe and extensible GPU programming frontend that separates parallel data\-indexing and memory partitioning from the actual kernel execution, eliminating the need for user\-writtenunsafeblocks in typical parallel workloads\.
- •An In\-Tree Cross\-Vendor Toolchain:Design and implement a portable two\-pass compiler pipeline linkingrustcto the LLVM Offload infrastructure, providing a unified target for NVIDIA and AMD hardware\.
- •Type\-Driven Offload Lowering:A Mid\-level Intermediate Representation \(MIR\) analysis enabling the automatic and efficient generation of memory transfers by leveraging Rust’s type, layout, and mutability information, and the prevention of cpu to gpu communication bugs through the Ownership system\.
- •Multi\-Vendor Benchmarking:A performance evaluation using the RAJAPerf\([22](https://arxiv.org/html/2608.13759#bib.bib15)\)benchmark suite on AMD and NVIDIA accelerators\. We demonstrate competitive GPU performance of the individual compute kernels\.

cargo host\-metadata pass\-Zoffload=HostMetadata=<manifest\>\(codegen omitted\)MonomorphizationCollection\(force kernel monomorphization\)Instance Filtering\(select only offload kernels\)Manifest Serialization<manifest\>cargo device pass\-Zoffload=Device=<manifest\>\-\-target <gpu\-device\-triple\>Device LLVM bitcodedevice\.bc\[Automated\] rustc Offload Packingclang\-offload\-packager\-equivalentPackaged device imagedevice\.bin\(embedsdevice\.bc\)cargo host pass\-Zoffload=Host\-\-target <cpu\-device\-triple\>/path/to/device\.binLowered offload intrinsicfrom Rust to LLVM\-IROffload Runtime API\(Kernel launch & memory mgmt\)clang\-linker\-wrapperLinker & Embedding Step\[Automation in progress\]a\.outHost CPU binary\(contains embedded GPU image\)lowers toPass 1: Host\-Side Metadata CollectionPass 2: Device\-Side CompilationPass 3: Host\-Side Compilation & Embedding

Figure 1\.Current three\-pass implementation pipeline for Rust offloading\.
## 2\.Background

### 2\.1\.Rust

While a complete introduction is out of scope, we will give a very brief summary of the aspects of Rust which we consider as most valuable to understand this paper\.

##### Unsafe

A strength of Rust is the explicit separation of safe and unsafe Rust\. It is a common goal to isolate unsafe operations, and then provide safe abstractions on top\. We “know” that GPU programming is not inherently unsafe \- Rust currently just often leaves us without an opportunity to express that our parallel writes into an object are safe\. In this work, we explore how such safe abstractions for GPU programming might look like\.

##### References and Raw Pointers

Rust offers raw, c\-style pointers, e\.g\.\*mutf32\. Their usage is very uncommon outside of Foreign\-Function\-Interfaces \(FFI\)\. References provide various benefits over raw pointers\. The compiler enforces that references can not be used after the object to which they point went out of scope or got dropped\. As such, there is no null\-pointer equivalent for references\. There are two types of references: const and unique \(aka\.mutable\)\. If a unique reference \(&mutT\) exists, no other reference \(neither const nor mut\) may exist to the same object\. However, if no mutable reference exists, arbitrary many const references \(&T\) may exist\. This rule is often referred to as thealiasing XOR mutability rule\. More detailed models are explained in\([31](https://arxiv.org/html/2608.13759#bib.bib23)\)and\([14](https://arxiv.org/html/2608.13759#bib.bib24)\)\. A user can only create a mutable reference if the underlying object was defined as mutable\.

1letmutx:Vec<i32\>=vec\!\[10;1024\];

2lety:&mut\[i32\]=&mutx;

The compiler enforces all of these requirements\. On the LLVM\-Intermediate Representation \(LLVM\-IR\) level, references are represented as pointers with thenoaliasattribute\. The only exception is described next\.

##### Interior Mutability

An advanced concept which allows users to create a const reference to a mutable object wrapped in anUnsafeCell\. For example, a reference of type &UnsafeCell<i32\>can be used to mutate an underlyingi32value, even if the reference is not unique\.

##### Generics

Generic functions and structs in Rust are similar to templates in C\+\+\. They are instantiated, aka “monomorphized”, at compile time, based on the concrete types with which they are invoked\.

##### Lifetimes

All references have a lifetime, which ties them to their underlying object\. Lifetimes can usually be inferred by the compiler, but at times have to be annotated explicitly with an apostrophe:&’af32\. An example from the Rust docs\([29](https://arxiv.org/html/2608.13759#bib.bib13)\)shows an application:

1fnlongest<’a\>\(x:&’astr,y:&’astr\)\-\>&’astr\{

2ifx\.len\(\)\>y\.len\(\)\{x\}else\{y\}

3\}

This function might return either of the two references\. As such, the returned reference is only alive while both of the input references are alive\. They therefore use the same lifetime\.

##### config \(“cfg”\)

Configurations in packages serve two main purposes\. Compile times in Rust are a major concern, so most crates offer a larger number of features\. A crate user can then select in theirCargo\.tomlwhich features of which dependency they want to enable or disable\. Inside their crate, a developer can use the\#\[cfg\(CONFIG==VALUE\)\]syntax to ask the Rust compiler if a certain config was set to the given value\. If acfgattribute evaluates to false, the guarded function, module, or struct will be discarded at the Abstract Syntax Tree \(AST\) level, to minimize compile\-time impact\. Compute heavy crates\([16](https://arxiv.org/html/2608.13759#bib.bib8)\)often also use differentcfgchecks to provide multiple algorithm implementations that are optimized for different compilation targets like aarch64 or x86\_64 and use intrinsics that are not generally available\.

### 2\.2\.LLVM/OpenMP and Libomptarget

We build our work on top of LLVM’s OpenMP/Offload library\. This has the benefit, that we are able to support both the “nvptx64\-nvidia\-cuda” and “amdgcn\-amd\-amdhsa” target, which are already available in today’s Rust compiler\. An Intel GPU target is under development, which will allow us to support safe abstractions for all major GPU vendors\.

##### OpenMP target vs\. Offload

OpenMP supports parallel CPU and GPU programming\. LLVM originally implemented the required OpenMP GPU support within the OpenMP target runtime\. Later, GPU Offload was split from OpenMP to make it easier for other language frontends to utilize the functionality\. This refactoring is still in progress\. Rustc currently generates OpenMP target runtime calls, but we consider this an implementation detail and will transfer to Offload calls in the near future\. For consistency, we will generally refer to Offload, unless talking about concrete elements like the libomptarget library\.

## 3\.A Frontend For Safe GPU Programming

### 3\.1\.Heterogeneous Execution Models

We define three programming models, from a fully automated host\-managed execution to explicit, type\-enforced device memory layout control\.

#### 3\.1\.1\.Interface A: Compiler\-Managed Rust Kernels

Interface A is the default interface for writing GPU kernels in Rust\. The programmer writes a Rust function over ordinary references and invokes it with an offload macro:

1offload\!\(matrix\_multiply,

2&matrix\_a,&matrix\_b,&mutmatrix\_c

3\);

The callee is compiled by rustc for the device target\. The host invocation is lowered into runtime calls that allocate device storage, transfer inputs, launch the generated kernel, and synchronize outputs according to the Rust types at the call boundary\. Immutable references \(&T\) are treated as read\-only inputs, while mutable references \(&mut T\) are values that may be written by the kernel and therefore must be made visible on the host after the call\.

This interface is intentionally plug\-and\-play: callers do not explicitly manage device memory\. That convenience also defines its performance limit\. Because each offload operation is specified in terms of host references, an intervening host use of a value forces the runtime to make the latest device value visible on the CPU\. Consequently, chains of kernels can suffer from implicit host\-device synchronization unless the compiler can prove that a transfer is redundant\.

1offload\!\(kernel\_1,&input,&mutoutput\);

2

3println\!\("\{:?\}",&output\);

4offload\!\(kernel\_2,&input,&mutoutput\);

Although the program does not mention memory movement, the host read ofoutputforces a device\-to\-host synchronization afterkernel\_1\. The second kernel then requires the updated value to be available on the device again, inducing a host\-to\-device transfer beforekernel\_2\. This behaviour is safe and convenient, but it makes data movement depend on seemingly unrelated host\-side uses such as logging or debugging\.

#### 3\.1\.2\.Interface B: Interoperability with Host\-launched GPU Vendor Libraries

Many GPU applications obtain their performance from highly optimized vendor libraries such as cuBLAS or rocBLAS\. These are called from host code and internally manage their own kernel launches, so the Rust compiler cannot optimize or inspect the device code itself\. However, the compiler can still use the Rust type system to reason about the data passed across the call boundary\.

Interface B treats such calls as host\-side offload operations\. Our intrinsic defaults to automatic data transfers to and from the device before and after invoking the underlying library, but other behaviour like forwarding existing GPU pointers can also be specified\. The compiler uses the same mapping infrastructure as for GPU kernels to materialize the required device pointers before invoking the library API\.

1core::intrinsics::offload\_args::<\_,\_,\(\)\>\(

2rocblas\_sgemv\_wrapper,

3\(&A,&x,&muty\)

4\);

The main benefit is that with this intrinsic normal Rust functions, vendor library calls and Rust GPU kernels can share one interface\. This lets programmers incrementally replace hot CPU operations with GPU vendor implementations\. It also allows the compiler to hoist, reuse, or eliminate data transfers around the call boundary\. We describe a set of optimizations to leverage this opportunity in[5](https://arxiv.org/html/2608.13759#S5)\. It also gives the runtime a uniform point at which to collect profiling data and diagnostics for offloaded operations, even when vendor\-library internals remain opaque\.

#### 3\.1\.3\.Interface C: Explicit Type\-Staged Memory Control

Interface C makes the Host or Device location of data explicit through Rust types\. In Interface B, host\-side uses can silently introduce transfers between kernels\. Interface C instead exposes this synchronization point in the program: values that may be modified on the device are managed through a type that keeps the corresponding host object borrowed until the GPU version of the value is dropped\.

1pubstructPreload<’a,T:?Sized\>\{

2cpu\_ptr:\*constT,

3\_marker:PhantomData<&’aT\>,

4\}

5pubstructPreloadMut<’a,T:?Sized\>\{

6cpu\_ptr:\*mutT,

7\_marker:PhantomData<&’amutT\>,

8\}

These types conservatively store raw host pointers rather than Rust references\. A Rust reference has significantly higher correctness requirements than a raw pointer\. For example the referenced memory must be valid for the duration of the borrow and the usual aliasing rules must be upheld whenever the reference is used\. After preloading, however, the most recent version of the data will live on the device, whose allocation has a Different address from the original host allocation\. The host pointer is therefore only a runtime key identifying the mapped allocation, not a reference to the current contents of the value\.

While we intentionally avoided references, the preloaded value must still participate in Rust’s lifetime and aliasing rules, to avoid trivial Undefined Behaviour \(UB\)\. ThePhantomDatafield provides this connection without requiring the struct to contain an actual reference\. APreload<’a, T\>carries the lifetime of an immutable borrow, while aPreloadMut<’a, T\>carries the lifetime of a mutable borrow\. The two variants express different access guarantees\. First,Preload<’a, T\>is read\-only: it behaves like an immutable borrow of the host value, so other host reads remain valid, but mutation is forbidden while the Preload handle exists\. Second,PreloadMut<’a, T\>may be modified by device code: it behaves like a mutable borrow, so Rust code cannot read or write the original host value until the GPU handle is dropped\. This follows Rust’s standard aliasing rule: either one mutable reference exists, or any number of immutable references exist, but not both\.

1fnmain\(\)\{

2letmutoutput=vec\!\[0\.0f32;1024\];

3

4

5

6letout\_gpu=

7core::intrinsics::preload\_mut\(

8&mutoutput

9\);

10

11offload\!\(kernel\_1,&input,&out\_gpu\);

12offload\!\(kernel\_2,&input,&out\_gpu\);

13

14

15drop\(out\_gpu\);

16

17println\!\("\{:?\}",output\);

18\}

The LLVM backend of rustc lowers the constructor of the Preload values into an offload begin\-data\-mapper operation and lowers drop \("destructor"\) calls into the corresponding end\-mapper operation\. Calling preload multiple times on a immutable reference simply creates multiple Preload handles and increases a reference counter in the runtime\. Dropping a Preload Type therefore decreases the reference counter and only results in an actual freeing of the gpu memory location if the counter reaches zero\. PreloadMut types can not alias by design, and as such the reference counter is irrelevant\. Each drop call does not only result in a free, but also generates a Device to Host transfer, since the underlying GPU allocation has likely been modified\. The memory transfer at the begin of a Kernel launch therefore becomes a no\-op, if all arguments have been preloaded\.

This design trades some convenience for predictable performance\. An intermediate host read ofoutputcan no longer silently introduce a transfer betweenkernel\_1andkernel\_2\. WithPreloadMut, such a read is rejected by the borrow checker until the mutably preloaded value is dropped; the drop marks the explicit point at which the device result is synchronized back to the host\. Interface C is therefore the appropriate interface for more complex gpu pipelines and library APIs that expect inputs or outputs to already be present on the GPU\.

### 3\.2\.Safe Rust Kernels

As described in Section[2\.1](https://arxiv.org/html/2608.13759#S2.SS1), references in Rust follow thealiasing XOR mutability rule\. A mutable reference therefore might be better described as a unique reference\. This is clearly at odds with even the most basicvec\_addexample:

1fnvec\_add\(a:&\[f32\],b:&\[f32\],c:&mut\[f32\]\)\{

2letidx=\_block\_dim\_x\(\);

3c\[idx\]=a\[idx\]\+b\[idx\];

4\}

In GPU programming, kernels are often executed with multiple threads, which can cause UB when sharing mutable references\. The most direct solution is turning all mutable inputs into raw pointers, as required byrust\-cuda\. However, building a GPU ecosystem on unsafe code blocks diminishes the purpose of the Rust language\. Drehwald et al\.\([8](https://arxiv.org/html/2608.13759#bib.bib12)\)andcuda\-oxideintroduce a path towards safe GPU programming by decoupling the memory access patterns from kernel implementation\. The underlying idea is that slices should fundamentally work safely, given that in most kernels, threads access disjoint elements of the slices they are modifying\. The challenge therefore lied in expressing this safely within the language, to the compiler\.

We want users to write safe Rust using standard slices, so our frontend handles it with raw pointers under the hood\. We introduce an abstraction called a`Region`, which lets users safely work with slices in a parallel environment\.

1letmutx=\[0\.0f64;256\];

2letmutreg=

3Region::<\_,Linear1D\>::new\(&mutx,\(\)\);

Every`Region`is tied to a`PartitioningStrategy`\. This strategy decides exactly which elements each thread is allowed to read and write to, ensuring that no pair of threads ever get overlapping memory regions\.

1pubunsafetraitPartitioningStrategy\{

2typeShape:Copy;

3typeView<’a,T:’a\>;

4typeViewMut<’a,T:’a\>;

5

6unsafefnget<’a,T\>\(

7ptr:\*constT,

8len:usize,

9shape:Self::Shape,

10\)\-\>Option<Self::View<’a,T\>\>;

11unsafefnget\_mut<’a,T\>\(

12ptr:\*mutT,

13len:usize,

14shape:Self::Shape,

15\)\-\>Option<Self::ViewMut<’a,T\>\>;

16\}

The trait here is unsafe, because any incorrect implementation of a PartitionStrategy will likely result in Undefined Behaviour\. Beyond that, both thegetandget\_mutfunctions are unsafe, since the passed pointer must point to an allocation with at leastlenelements\. Neither of these requirements can be verified by the compiler, therefore both trait and functions are unsafe\.

The safety is introduced by the individual implementations of the PartitionStrategy, which promise to uphold these invariants\. To support our benchmarks in[6](https://arxiv.org/html/2608.13759#S6), we implemented multiple PartitionStrategy variants\. We want to emphasize that all these strategies are implemented in pure Rust, without relying on any internal compiler features or any compiler internal abilities\. While it seems sensible to place at least the trait definition and some of the more popular strategies into the standard library, we have no technical need to do so\. By releasing our interface as a standalone crate, we hope to encourage users to explore additional partitioning schemes\.

##### Comparison withcuda\-oxide

While our implementation is highly inspired by the design of Drehwald et al\.\([8](https://arxiv.org/html/2608.13759#bib.bib12)\), we want to analyze the differences with thecuda\-oxidefrontend\. Althoughcuda\-oxidehas a similar design, there are three primary differences: Firstly, theirDisjointSliceis part of thecuda\-oxideproject and can not be extended by users\. Secondly, their design is currently scalar oriented, whereas our interface allows the return of disjoint chunks of data\. Finally,cuda\-oxideallows safe indexing into theirDisjointSlicevia a specialThreadIndexobject with an advanced set of type system checks, while our PartitionStrategies skip this complexity, compute the index internally, and directly return a mutable reference for each thread\.

While both frontends currently support a different set of access patterns, we see no fundamental reason why they can not be extended to support the same kernels\. We hope that through enough end\-user feedbackRust offloadandcuda\-oxidecan eventually converge into a single design for safe kernels\. Outside of the frontend, both projects diverge on other major axes \(vendor\-agnostic vs\. vendor specific, single\-pass vs\. multi\-pass compilation\)\. If both projects can use a similar or even identical frontend, we hope that the remaining Rust GPU projects will also adopt a compatible design to achieve safety\.

### 3\.3\.Exposing GPU Shared Memory

Advanced GPU algorithms frequently utilize shared memory as a high\-performance scratchpad\([18](https://arxiv.org/html/2608.13759#bib.bib9)\)to store temporary variables within kernels\. This memory space is private to each thread block, allowing threads within the same block to efficiently exchange data\. In Section[3\.2](https://arxiv.org/html/2608.13759#S3.SS2), we described how we can safely encode strictly disjoint thread\-access patterns to cover most use cases for input arguments by assigning disjoint subsections to individual threads\. However, shared memory usually has the opposite purpose, as threads actively cooperate over this block\-local storage to execute efficient block\-level primitives such as reductions and tiled operations\. To demonstrate its use in Rust, in[3\.3](https://arxiv.org/html/2608.13759#S3.SS3)we show how a kernel can requests two blocks ofBLOCK\_SIZE2\{\\lstinline\{\{\\lst@@@set@language\\lst@@@set@numbers\\lst@@@set@frame\\lst@@@set@rulecolor\\lst@@@set@language\{\\@listingGroup\{ltx\_lst\_identifier\}\{\{BLOCK\\textunderscore SIZE\}\}\}\}\}\}^\{2\}bytes from the shared memory space, in order to perform a tiled matrix multiplications within each block:

1core::intrinsics::offload::<\_,\_,\(\)\>\(

2gpu\_square\_matrix\_mult,

3\[grid\_cols,grid\_rows,1\],

4\[nthreads,nthreads,1\],

5\(2\*BLOCK\_SIZE\*BLOCK\_SIZE\)asu32,

6\.\.args

7\)

1pubunsafeextern"gpu\-kernel"

2fngpu\_square\_matrix\_mult\(\.\.args\)\{

3unsafe\{

4constLENGTH:usize=

5\(BLOCK\_SIZE\*BLOCK\_SIZE\)asusize;

6lettile\_a=

7gpu\_launch\_sized\_workgroup\_mem::<i32\>\(\)

8as\*mut\[i32;LENGTH\];

9lettile\_b=

10gpu\_launch\_sized\_workgroup\_mem::<i32\>\(\)

11\.add\(len\)as\*mut\[i32;LENGTH\];

12\.\.\.

13\}

As we can see in the example, exposing raw accelerator shared memory to high\-level code introduces four distinct safety hazards:

1. \(1\)Accessing an object stored on shared memory whose size exceeds the shared memory bytes requested at kernel launch can lead to out\-of\-bounds accesses\.
2. \(2\)If the shared memory space holds multiple objects, applying an incorrect offset when accessing a specific object can lead to incorrect outcomes and memory errors\.
3. \(3\)Casting a shared memory pointer to a standard Rust reference \(&Tor&mut T\), while multiple threads concurrently read and write from the memory at that address, causes UB\.
4. \(4\)The guaranteed alignment of the pointer returned bygpu\_launch\_sized\_workgroup\_mem::<T\>is the alignment of typeT\. This can lead to UB if the pointer is later casted to a type with a higher alignment requirement\.

Returning a raw pointer ingpu\_launch\_sized\_workgroup\_mem::<T\>is already a clear signal to developers\. Since the pointer is only usable within device code, additional safety guardrails seem unlikely to justify their complexity\.

## 4\.Toolchain & Lowering Implementation

The Rust compiler supports two modes of compilation\. Code can either be compiled for the host on which the compiler is running or cross\-compiled for a different target\. We need both abilities when compiling a single codebase for the CPU host and the GPU target\. Existing compiler infrastructures typically address this using either a single\-pass or a two\-pass compilation pipeline\. In a single\-pass pipeline, the compiler frontend \(e\.g\.,rustc\) is invoked once, duplicating an intermediate representation \(IR\) to target each architecture\. For our toolchain, we implement a two\-pass compilation pipeline, as shown in Figure[1](https://arxiv.org/html/2608.13759#S1.F1)\.

To give a realistic pipeline overview, we demonstrate our pipeline using Rust’s official build manager,cargo, even though our implementation modifications reside entirely within the compiler frontend \(rustc\) and the LLVM backend\. The experimental\-Z offload=Deviceflag is forwarded transparently fromcargotorustc, requiring no changes to the build manager itself and ensuring that alternative build systems like Bazel or Buck2 can support our offloading workflow with minimal changes\. This device pass uses existing upstream target definitions for AMD and NVIDIA architectures \(with ongoing development for Intel GPU support\)\. Our modified pipeline only alters the final stage of code generation: instead of just emitting device bitcode, our compiler wraps the bitcode into a device binary\. This part is similar to the Clang\-based C\+\+ OpenMP toolchain, although Clang delegates this packaging step to an externalclang\-offload\-packagerexecutable, while rustc invokes the underlying Offload APIs directly\. This allows us to minimize the number of binaries involved in our pipeline\.

The secondcargoinvocation manages the host\-side compilation\. We again trigger our modified pipeline via the modified pipeline flag\-Z Offload=Host=/path/to/device\.bin\. During this pass, the compiler lowers our new Rust offload intrinsics introduced in Section[3](https://arxiv.org/html/2608.13759#S3)into OpenMP target \(i\.e\.,libomptarget\) runtime calls\. Following the intrinsic lowering, our toolchain embeds the device binary generated in the previous pass directly into the host LLVM IR to produce the final host object\. Notably, because the underlying LLVM Offload infrastructure supports fat binaries, this embedded payload can bundle multiple device binaries, which allows support for both AMD and NVIDIA architectures from one executable\. To finalize the compilation, the pipeline invokes theclang\-linker\-wrapperexecutable, which links the required offload runtime libraries into the executable\.

While it may appear cumbersome to pass the device binary path explicitly between compiler invocations, it is intentional\. Even thoughrustcandcargoare co\-developed within the same GitHub organization, rustc is prohibited from making assumptions regarding the filesystem location of intermediate build artifacts\. This separation for example simplifies caching, and generally just ensures thatcargoandrustccan be developed and improved independently\. End\-users do not need to run this multi\-step sequence manually, since the entire workflow can be handled by a standardcargosubcommand wrapper\.

##### Alternative Design Considerations:

An alternative design which we considered was a single\-pass implementation, in which we would only invoke cargo once, similar tocuda\-oxide\. The Rust compiler fundamentally expects one compilation target per invocation\. If that target is also used to determine both host and device target semantics, we would need to choose between the CPU target \(e\.g\.x86\_64\-unknown\-linux\-gnu\), the GPU target \(e\.g\.amdgcn\-amd\-amdhsa\), or a new combined target\. None of these choices seemed optimal to us\. An alternative, as used by cuda\-oxide, is to retain the CPU target for frontend compilation and separate device code at a later stage\. This preserves host\-target semantics, but frontend target\-dependent decisions such ascfgevaluation are still made only once\. Rust is a systems language that has direct support for both inline assembly and target specific intrinsics like avx512 or neon instructions\. Compute\-heavy crates can therefore provide a default implementation, supported by faster but vendor\-specific implementations which are gated behind\#\[cfg\(target\_arch=<X\>\)\]checks\. Using the CPU target preserves these host\-side optimizations, but it means thatcfgevaluation may select CPU\-specific implementations before host and device code are separated\. Consequently, MIR reachable from a GPU kernel can contain CPU\-specific intrinsics or inline assembly\. For the general case, we consider translating such target\-specific code into GPU\-compatible IR infeasible\. We therefore favour separate frontend passes, which allow target\-dependent source constructs to be evaluated independently for the CPU and GPU\. This choice comes with a cost: single\-pass compilation keeps host and device information within one compilation, simplifying cross\-target communication and monomorphization, while our multi\-pass design must explicitly communicate this information between passes\.

It might seem tempting to compromise between the two approaches\. A granular compromise would require us not to evaluatecfgattributes and macros in the AST, and instead carry them throughout the compiler, till our final MIR layer, where we would split the IR between host and device IR\. This would require adjusting most IR and lowering code, and furthermore pessimize compile times and memory usage\. To avoid imposing this overhead on non\-GPU users, rustc would need to retain its existing behaviour for ordinary compilations, adding further implementation complexity\. A more coarse compromise would be to simply have two completely independent AST and IR copies in a single rustc invocation\. While the coarse compromise would simplify some of our communication, we believe the actual simplifications for us would not outweigh the increased complexity in other parts of the compiler\. For now, we therefore use a two\-pass compilation also performed by OpenMP offload, HIP, CUDA, and others\.

### 4\.1\.Lowering of our Offload intrinsics

Discrete accelerator memory spaces typically require explicit, developer\-directed data mapping, as alternative Unified Shared Memory \(USM\) abstractions or software\-managed runtimes often introduce notable performance overhead and microarchitectural constraints\([20](https://arxiv.org/html/2608.13759#bib.bib16)\)\. Rather than forcing programmers to manually annotate data\-sharing boundaries via verbose pragmas, as is common in OpenMP or SYCL, our toolchain derives data directionality semantics automatically from Rust’s type system and ownership model\.

Because safe Rust enforces immutability by default and triggers compiler warnings for redundantmutqualifiers, argument mutability provides a sound signal for data directionality\. During Rust MIR lowering, the framework walks the kernel parameter layouts to compute the number of bytes to be transferred and to synthesize corresponding LLVMLibomptargetdata\-mapping clauses:

- •Immutable references \(&T\) and constant raw pointers \(\*const T\) lower to theMapTodirective, copying data exclusively to the device\.
- •Mutable references \(&mut T\) and mutable raw pointers \(\*mut T\) lower to the bidirectionalMapToFromdirective and enable post\-kernel synchronization\.
- •Scalar arguments up to 64 bits are passed by value using OpenMP target’sIMPLICITandLITERALflags, bypassing pointer indirection and allocation overhead\.

##### Cross\-Boundary Monomorphization

Idiomatic Rust relies on compile\-time monomorphization to generate specialized, concrete function instances from generic definitions\. This mechanism is important for GPU targets, where static type resolution enables additional LLVM optimizations\. However, our two\-pass compilation pipeline breaks standard monomorphization tracking\. Because the host application’s entry point \(main\) is omitted or conditionally compiled out during the device pass, the concrete type substitutions requested by host\-sideoffload\!invocations are invisible during the device compilation pass\.

To resolve this mismatch, we considered two strategies:

1. \(1\)Dual\-Pass Main Compilation:Compiling the host entry point \(main\) during the device pass solely to force kernel instantiations\. This approach has a similar challenge to a single\-pass solution; target\-specific conditional compilation \(\#\[cfg\]\) can cause the device pass to miss code paths that are active during the host pass\.
2. \(2\)Cross\-Pass Metadata Exportation:Leveraging a dedicated compiler pass to serialize kernel definition identifiers \(DefId\) and their concrete type substitutions during the host phase\. The subsequent device pass imports this metadata to seed its root monomorphization collection\.

We implemented the second approach as a new query in the Rust compiler\. The compilation overhead of this metadata serialization is expected to be negligible, as it integrates with incremental compilation caching, and the first host pass can skip the expensive codegen part\.

### 4\.2\.Discussion on Future Support

While we implemented most of our desired features in this prototype, there are two important features that we plan to add or extend, namely the support to pass additional types between the host and device, and the ability to run most of the standard library on GPUs\.

##### Frontend ABI Validation

Since our two\-pass architecture decouples host and device compilations, type layouts and ABI lowerings can diverge between the two targets\. For instance, we identified a target discrepancy in how primitive slices are lowered: thex86\_64host andamdgcnbackends represent a slice as two scalar values \(ptr, int\), whereas the upstreamnvptx64target lowers it as a fixed\-size array \(\[i64; 2\]\)\. While we are engaging with upstream maintainers to unify these lowering targets where sensible, the divergence, even on the basic slice type, shows the necessity of automated cross\-boundary validation before adding support for more advanced types like structs\.

This static validation is only required for host\-device kernel entry points; GPU\-to\-GPU device calls are lowered by the same target, so arbitrary types can be passed\.

##### Standard Library Support on GPUs

Providing a full GPU implementation of Rust’sstdis outside the scope of this work\. Reimplementing broad standard\-library support inside the Rust frontend would also duplicate a difficult and largely orthogonal engineering effort\. We instead plan to adopt the idea established by LLVM’slibc\-for\-gpu\([26](https://arxiv.org/html/2608.13759#bib.bib30);[13](https://arxiv.org/html/2608.13759#bib.bib31)\)and later replicated for Rust \(outside of LLVM\) in\([30](https://arxiv.org/html/2608.13759#bib.bib17)\)\.

## 5\.Compiler Optimization Passes

Rust Offload provides two interfaces which support Rust\-written GPU kernels: an explicit interface, where users control data movement directly, and a more convenient interface, where data is transferred automatically for each kernel launch\. The convenient interface is easier to use, but it can be substantially slower when the same data is reused across multiple kernels\. Without relying on the evaluation results in detail, our experiments show that repeated automatic transfers can make this interface over 400×\\timesslower than the explicit version with dedicated data movement\.

We expect that most Rust offload users will start prototyping GPU applications with our convenient interface\. We therefore propose a set of optimizations that, for the common case, can make the convenient interface as performant as manual transfers\. These optimizations would be enough to match the overall runtime of our explicit interface on the evaluated RAJAPerf Benchmarks\. We prototyped most of these optimizations as an extension to the LLVM OpenMP\-opt pass\([7](https://arxiv.org/html/2608.13759#bib.bib32);[12](https://arxiv.org/html/2608.13759#bib.bib28)\)\. Based on initial experiments, we are confident that with further testing and upstreaming, we can eliminate the need for most Rust Offload users to reach for the dedicated data movement interface\.

##### Automatic data prefetching

LLVM’s offload API provide an interface which already matches our convenient offload intrinsic\. In a single call, it transfers all arguments to the GPU, launches a kernel, and transfers some arguments back to the host\. This is not easy to optimize, so we adjusted our code generation to be more explicit\. Each offload intrinsic now generates three calls: \(1\) a Host2Device transfer for all arguments, \(2\) a kernel launch, \(3\) a Device2Host transfer for selected arguments\. As part of this transfer, we prototyped an extension to turn these blocking transfers into async ones\. The Host2Device is started earlier, if we can prove that it is legal to do so\. We then await it at the kernel launch location\. Similarly, the Device2Host transfer is started directly after the kernel has terminated, but we only block to await it before its next use\([10](https://arxiv.org/html/2608.13759#bib.bib33)\)\.

##### LICM

The second optimization we prototyped is a variant of Loop\-Invariant\-Code\-Motion \(LICM\)\.

1foriin0\.\.100\{

2offload\!\(vec\_add,&a,&b,&mutc\);

3\}

Based on our work in the previous step, we have already generated separate memory transfer and kernel launch instructions\. We extended our previous optimization to try and hoist both transfers out of the loop\. If we succeed, we could then continue to turn them into async transfers for further improvements\.

In order to also handle unrolled loops, we prototyped handling of repeated kernel calls, which might only differ by scalar offsets:

1…

2offload\!\(vec\_add,&a,&b,&mutc,0\);

3offload\!\(vec\_add,&a,&b,&mutc,1\);

4offload\!\(vec\_add,&a,&b,&mutc,2\);

5offload\!\(vec\_add,&a,&b,&mutc,3\);

6…

Here we can clearly cancel out all intermediate data transfers that were generated in the lowering of our intrinsics\.

##### Further experiments

The benchmark that is the least amenable to our optimization experiments is the “Energy” benchmark in RAJAPerf\([22](https://arxiv.org/html/2608.13759#bib.bib15)\)\. It consists of six different kernels which share a large number of arguments\. However, each kernel has a few unique arguments, and such an optimization would need a heuristic for whether it is reasonable to preload all of the data\. If applied, the transformation would increase the peak memory consumption on the GPU, potentially significantly, so we have therefore not prototyped it\.

##### Rust\-based performance improvements

Most of our optimizations target the convenient offload interface and are therefore prototyped in LLVM, on which rustc already relies for many performance optimizations\. For explicit data movement, however, rustc could also help through Clippy, the official Rust linter\. In particular, a future lint could warn when a preload call is placed unnecessarily late, leaving a lot of wasted time between the last CPU usage and the start of our Host to Device transfer\.

## 6\.Evaluation

To show the portability and efficiency of our Rust GPU solution, we ported a subset of RAJAPerf\([22](https://arxiv.org/html/2608.13759#bib.bib15)\)to pure Rust and evaluated it against the original variants\. RAJAPerf is the benchmark suite of RAJA\([2](https://arxiv.org/html/2608.13759#bib.bib14)\), a portable C\+\+ framework for expressing loop\-level parallelism across accelerators using different backends such as CUDA, HIP, and OpenMP\. Its kernels are derived from HPC applications and are commonly used to evaluate backend performance\. RAJA and its benchmarks are useful for our study as they allow us to compare our Rust Offload against the same RAJA\-based code with different high\-performance backends\.

We ran the benchmarks on three different servers, using an AMD MI250X GPU, NVIDIA H100 GPU, and an NVIDIA RTX A2000 GPU\. Our extended Rust compiler is based on LLVM 23\.1\.0\-rc1\.

We evaluate five aspects: \(1\) kernel times, \(2\) memory transfer size, \(3\) total runtime, \(4\) fast\-math impact, and \(5\) kernel register usage\.

##### Kernel Times

Figure[2](https://arxiv.org/html/2608.13759#S6.F2)shows the kernel times for Rust and RAJA variants on MI250X and H100\. Rust kernels perform similarly to RAJA kernels except in the FIR and LTIMES benchmarks, where Rust shows slower kernel times\. Both are simple micro\-benchmarks where each thread repeatedly performs a small number of multiply/add operations, with FIR using a compile\-time loop count and LTIMES taking the iteration count as an argument\. Since these kernels are small and sensitive to unrolling decisions, larger differences are not surprising\.

##### Memory Transfer Sizes

While kernel times are similar, memory transfer behaviour differs between Rust and RAJA on both AMD and NVIDIA GPUs\. On H100 and across all benchmarks, RAJA performs slightly more and larger host\-to\-device transfers than Rust: 55 vs\. 53 transfers, totalling 468 MB vs\. 423 MB\. After the kernels, both perform 9 device\-to\-host transfers, but RAJA transfers 99 MB compared to Rust’s 69 MB\. Rust moves less data overall, but this does not translate into lower transfer time: its transfers take 46 ms compared to 16 ms for RAJA\. We suspect differences in memory kinds and asynchronous transfers to be the cause of the slowdown\.

##### Benchmark Runtimes

Figure[3](https://arxiv.org/html/2608.13759#S6.F3)and[4](https://arxiv.org/html/2608.13759#S6.F4)show the benchmark runtime for Rust and RAJA on MI250X and H100, respectively\. These mainly include the kernel launch time, the kernel time, and the synchronization time\. As per RAJAPerf default, each benchmark kernel is launched in a loop, between 50 and 700 times, and the total runtime of this loop is then reported\. Memory transfer times are excluded by RAJAPerf, as transfers are generally done at the begin and end of the program\. As a smoke test, we add a naive Rust Offload implementation in Figure[3](https://arxiv.org/html/2608.13759#S6.F3)based on interface A, which transfers data once per kernel launch instead of once per benchmark\. On our MI250X, this naive implementation can be over 400×\\timesslower than our optimized Rust implementation, which highlights the need for our optimizations described in[5](https://arxiv.org/html/2608.13759#S5)to close this gap\.

On MI250X \(Figure[3](https://arxiv.org/html/2608.13759#S6.F3)\), Rust is between 32% faster and 43% slower when measuring the whole runtime and not individual kernel timings\. On H100 \(Figure[4](https://arxiv.org/html/2608.13759#S6.F4)\), Rust is between 11% faster and 46% slower than the base CUDA implementation\. The biggest differences in favour of BaseCuda can be seen on the FIR and LTIMES benchmark, where Rust is 44% and 46% slower, respectively\. These two benchmarks also give Rust the biggest advantage over BaseHIP, with 15% and 32% respectively\. Both benchmarks consist only of a very small loop with few instructions, and as such different unroll decisions between the three compilers can have a major impact\.

##### Fast\-Math Impact

To assess whether Rust kernel times benefit from common GPU optimization settings, we evaluate relaxed floating\-point semantics\. C\+\+ GPU codes commonly use fast\-math, which combines seven flags that enable optimizations based on weaker floating\-point guarantees\. Rust does not allow users to enable fast\-math directly, since they include “no NaN” \(nnan\) and “no infinity” \(ninf\) assumptions that could trigger UB in safe Rust code\. Instead, Rust provides experimental algebraic\([27](https://arxiv.org/html/2608.13759#bib.bib19)\)floating\-point operations, which expose most of the relevant optimization opportunities while excluding the nnan and ninf assumptions\. As Figure[5](https://arxiv.org/html/2608.13759#S6.F5)shows, on an RTX A2000, algebraic floats provide a 2×\\timesspeedup on FIR and improveDEL\_DOT\_VEC\_2D,VOL3D, andMATVEC3Dby about 20%\. The FIR kernel is a trivial loop and already unrolled in the case of normal floats\. Algebraic floats allow LLVM to further vectorize it with a vector\-width of 4, explaining the large impact\. Algebraic floats do not have a relevant impact on other kernels\. Using algebraic floats on the MI250X has not shown significant performance improvements\.

##### Register Usage

We primarily care about the pure kernel times, but register usage is another indicator to confirm that we generate efficient LLVM\-IR\. On RTX 2070, the average register usage of Rust is 33, while the RAJA\-CUDA solution averages 28 across the 13 implemented RAJAPerf kernels\. Using fast\-math via algebraic floats in Rust only results in a minimal increase in two of the kernels\. The overall slightly higher register usage could be caused by additional bounds\-checking in the Rust code\. In Rust CPU code, bounds checking is usually avoided by iterating over all elements in an array instead of indexing into it\. GPU code uses explicit indexing based on the thread and block indices instead, so Rust must verify that the access is within bounds\. In RAJAPerf, most array, block, and thread dimensions are known at compile time, and we have not measured any runtime impact of bounds\-checking on Rust GPU kernels\.

002020404060608080100100120120140140160160180180200200220220pressurecalc2matvec3dpressurecalc1ltimesenergycalc5energycalc3deldotvec2denergycalc1vol3dfirenergycalc4energycalc2energycalc6Average runtime \[μ\\mus\]Rust \(H100\)RAJA \(H100\)Rust \(MI250X\)RAJA \(MI250X\)Figure 2\.Average kernel runtime comparison between Rust and RAJA on H100 and MI250X\.DEL\_DOT\_VEC\_2DENERGYFIRLTIMESMATVEC\_3D\_STENCILPRESSUREVOL3D100\.710^\{0\.7\}10110^\{1\}101\.310^\{1\.3\}101\.710^\{1\.7\}10210^\{2\}102\.310^\{2\.3\}102\.710^\{2\.7\}10310^\{3\}103\.310^\{3\.3\}Mean runtime \(ms, log scale\)Base\_SeqRust Interface ARust Interface CBase\_HIPRAJA\_HIPFigure 3\.Mean benchmark runtime of Rust and RAJAPerf implementations on an AMD MI250X GPU\.DEL\_DOT\_VEC\_2DENERGYFIRLTIMESMATVEC\_3D\_STENCILPRESSUREVOL3D10010^\{0\}100\.310^\{0\.3\}100\.710^\{0\.7\}10110^\{1\}101\.310^\{1\.3\}101\.710^\{1\.7\}10210^\{2\}102\.310^\{2\.3\}102\.710^\{2\.7\}10310^\{3\}103\.310^\{3\.3\}Mean runtime \(ms, log scale\)Base\_SeqRust Interface CBase\_CUDARAJA\_CUDAFigure 4\.Mean benchmark runtime of Rust and RAJAPerf implementations on an H100 NVIDIA GPU\.002002004004006006008008001,0001\{,\}0001,2001\{,\}200pressurecalc2matvec3dpressurecalc1ltimesenergycalc5energycalc3deldotvec2denergycalc1vol3dfirenergycalc4energycalc2energycalc6Average runtime \[μ\\mus\]RustRust \(Algebraic Floats\)Base\_CUDARAJA\_CUDAFigure 5\.Average kernel runtime comparison between Rust and RAJA on NVIDIA RTX A2000\.

## 7\.Related Work

The Rust GPU ecosystem is under active development, with multiple approaches currently in progress:

rust\-gpu: A SPIR\-V\-based approach\([25](https://arxiv.org/html/2608.13759#bib.bib22)\)which was originally targeting graphics programming in Rust, later extended to also target compute kernels\. Due to using the Vulkan flavour of SPIR\-V, the rust\-gpu project has to emulate pointers\([32](https://arxiv.org/html/2608.13759#bib.bib18)\), which we consider a blocking issue for most HPC benchmarks\.

rust\-cuda: An NVIDIA\-only approach\([24](https://arxiv.org/html/2608.13759#bib.bib21)\)that uses raw pointers for all mutable arguments, consequently giving up on the portability and safety aspects our work provides\.

A very early paper\([11](https://arxiv.org/html/2608.13759#bib.bib26)\)which precedes the first stable Rust release\. It demonstrates GPU programming in Rust, by targeting both OpenCL and PTX directly\. They also describe a two\-pass compilation pipeline\.

cuda\-oxide: A new NVIDIA\-led approach\([5](https://arxiv.org/html/2608.13759#bib.bib20)\)to allow safe programming for NVIDIA GPUs\. Although portability is not given, the safety ideas are comparable to those in our work and in\([8](https://arxiv.org/html/2608.13759#bib.bib12)\)\. Beyond existing implementation differences, we are therefore curious if a unified frontend in the style of\([4](https://arxiv.org/html/2608.13759#bib.bib25)\)is possible\. Their single\-pass compilation makes a different trade\-off from ours: it simplifies communication between host and device code, but frontend target\-dependent decisions are evaluated only once\. Our multi\-pass design instead preserves independent host and device target semantics at the cost of explicit cross\-pass communication\.

Beyond Rust, our work shares commonalities with many of the common offloading languages\. A subset of CUDA was implemented on top of the OpenMP target infrastructure\([6](https://arxiv.org/html/2608.13759#bib.bib29)\), and we follow a similar path to reuse the OpenMP target efforts in LLVM\([3](https://arxiv.org/html/2608.13759#bib.bib27)\)\. In LLVM, OpenMP target runtimes have been co\-developed with \(mostly\) language specific compiler optimizations\([12](https://arxiv.org/html/2608.13759#bib.bib28);[7](https://arxiv.org/html/2608.13759#bib.bib32)\)\. Similarly, we co\-developed compiler optimizations for redundant data transfers, stemming from automatic data movement at kernel boundaries\. It is worth to note that while our optimizations are API specific, they apply to any language, which for now includes OpenMP target; a language that can suffer from the same issues if code is ported naively to the GPU\. Ongoing efforts are currently porting SYCL on top of the LLVM Offload infrastructure, which is the generic part spun out of OpenMP target\. As these efforts mature, we will port our approach to the more generic Offload APIs as well and enable our transformations for more input languages\.

## 8\.Conclusions

In this work, we presented a suite of portable interfaces for offloading Rust code to GPU accelerators, proving that memory safety does not preclude high performance\. To cover a wide range of GPU programming paradigms, we provided three distinct interfaces: \(1\) offloading for standard Rust regions with automatic, compiler\-managed data movement; \(2\) offloading via third\-party libraries like cuBLAS and rocBLAS; and \(3\) a manual, user\-controlled data movement interface\. Regarding kernel programming, developers retain high flexibility: kernels can be implemented as unsafe code blocks, with CUDA and HIP\-style thread indexing, or written entirely as safe code by leveraging our novel partitioning abstractions for transparent data\-to\-thread mapping\. Built upon the LLVM Offload infrastructure, our interfaces achieve broad vendor portability, currently supporting NVIDIA and AMD accelerators, with planned support for Intel GPUs\.

Our experiments demonstrate that this Rust offloading approach achieves near\-parity with native kernel execution times on both NVIDIA and AMD GPUs\. While we observed overheads related to total benchmark execution time—primarily stemming from current host\-device synchronization inefficiencies—our approach stands as a highly competitive acceleration method for both safe and unsafe Rust code on major GPU vendors\.

Future work will focus on minimizing device\-host synchronization overhead by transparently integrating asynchronous data transfers and kernel launches\. Furthermore, we plan to extend the frontend to support multi\-device environments, enabling developers to scale their workloads across all available GPUs on a given node\.

## Acknowledgements

We thank the Rust Foundation for their support of this work\. Manuel Drehwald was supported in part by the Rust Foundation, and Marcelo Domínguez was supported in part through Google Summer of Code\. The views and opinions expressed in this paper are those of the authors and do not necessarily reflect those of the Rust Foundation or Google\.

We also especially thank Oli Scherer for discussion on supporting generics and for general code review, the Rust Infrastructure Team and Shota Sugano for their support in distributing our work through Rust’s infrastructure, and Travis Cross for championing this work within the Rust Language Team\.

Alán Aspuru\-Guzik thanks Anders G\. Frøseth for his generous support\. Alán Aspuru\-Guzik also acknowledges the generous support of Natural Resources Canada and the Canada 150 Research Chairs program\.

This work was performed under the auspices of the U\.S\. Department of Energy by Lawrence Livermore National Laboratory under Contract DE\-AC52\-07NA27344 \(LLNL\-CONF\-2023077\)\. This manuscript has been partially co\-authored by Lawrence Livermore National Security, LLC under Contract No\. DE\-AC52\-07NA27344 with the US\. Department of Energy\. The United States Government retains, and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non\-exclusive, paid\-up, irrevocable, world\-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes\.

## References

- Abdiet al\.\(2023\)J\. Abdi, G\. Zhang, and M\. C\. JeffreyBrief Announcement: Is the Problem\-Based Benchmark Suite Fearless with Rust?\.InSPAA ’23: Proceedings of the 35th ACM Symposium on Parallelism in Algorithms and Architectures,pp\. 303–305\.External Links:ISBN 978\-1\-45039545\-8,[Document](https://dx.doi.org/10.1145/3558481.3591313)Cited by:[§1](https://arxiv.org/html/2608.13759#S1.p2.1)\.
- Beckingsaleet al\.\(2019\)D\. A\. Beckingsale, J\. Burmark, R\. Hornung, H\. Jones, W\. Killian, A\. J\. Kunen, O\. Pearce, P\. Robinson, B\. S\. Ryujin, and T\. R\. ScoglandRAJA: portable performance for large\-scale scientific applications\.In2019 IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC \(P3HPC\),Vol\.,pp\. 71–81\.External Links:[Document](https://dx.doi.org/10.1109/P3HPC49587.2019.00012)Cited by:[§6](https://arxiv.org/html/2608.13759#S6.p1.1)\.
- Bertolliet al\.\(2015\)C\. Bertolli, S\. F\. Antao, G\. Bercea, A\. C\. Jacob, A\. E\. Eichenberger, T\. Chen, Z\. Sura, H\. Sung, G\. Rokos, D\. Appelhans, and K\. O’BrienIntegrating gpu support for openmp offloading directives into clang\.InProceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC,LLVM ’15,New York, NY, USA\.External Links:ISBN 9781450340052,[Link](https://doi.org/10.1145/2833157.2833161),[Document](https://dx.doi.org/10.1145/2833157.2833161)Cited by:[§7](https://arxiv.org/html/2608.13759#S7.p6.1)\.
- Churavy \(2026\)V\. ChuravyKernelAbstractions\.jl\.Zenodo\.External Links:[Document](https://dx.doi.org/10.5281/zenodo.19162524)Cited by:[§7](https://arxiv.org/html/2608.13759#S7.p5.1)\.
- \[5\]\(2026\)cuda\-oxide\.External Links:[Link](https://github.com/NVlabs/cuda-oxide)Cited by:[§7](https://arxiv.org/html/2608.13759#S7.p5.1)\.
- Doerfertet al\.\(2023\)J\. Doerfert, M\. Jasper, J\. Huber, K\. Abdelaal, G\. Georgakoudis, T\. Scogland, and K\. ParasyrisBreaking the vendor lock: performance portable programming through openmp as target independent runtime layer\.InProceedings of the International Conference on Parallel Architectures and Compilation Techniques,PACT ’22,New York, NY, USA,pp\. 494–504\.External Links:ISBN 9781450398688,[Link](https://doi.org/10.1145/3559009.3569687),[Document](https://dx.doi.org/10.1145/3559009.3569687)Cited by:[§7](https://arxiv.org/html/2608.13759#S7.p6.1)\.
- Doerfertet al\.\(2022\)J\. Doerfert, A\. Patel, J\. Huber, S\. Tian, J\. M\. M\. Diaz, B\. Chapman, and G\. GeorgakoudisCo\-designing an openmp gpu runtime and optimizations for near\-zero overhead execution\.In2022 IEEE International Parallel and Distributed Processing Symposium \(IPDPS\),Vol\.,pp\. 504–514\.External Links:[Document](https://dx.doi.org/10.1109/IPDPS53621.2022.00055)Cited by:[§5](https://arxiv.org/html/2608.13759#S5.p2.1),[§7](https://arxiv.org/html/2608.13759#S7.p6.1)\.
- Drehwaldet al\.\(2025\)M\. Drehwald, M\. Domínguez, K\. Sala, and J\. DoerfertTaming GPU programming with safe Rust\.Note:Presentation at the 2025 LLVM Developers’ Meeting[https://llvm\.org/devmtg/2025\-10/slides/technical\_talks/drehwald\.pdf](https://llvm.org/devmtg/2025-10/slides/technical_talks/drehwald.pdf)Cited by:[§3\.2](https://arxiv.org/html/2608.13759#S3.SS2.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2608.13759#S3.SS2.p1.3),[§7](https://arxiv.org/html/2608.13759#S7.p5.1)\.
- Goulart and Chen \(2024\)P\. J\. Goulart and Y\. ChenClarabel: an interior\-point solver for conic programs with quadratic objectives\.External Links:2405\.12762Cited by:[§1](https://arxiv.org/html/2608.13759#S1.p2.1)\.
- Guaiteroet al\.\(2022\)R\. A\. H\. Guaitero, J\. M\. M\. Diaz, T\. Applencourt, X\. Li, and J\. DoerfertAutomatic asynchronous execution of synchronously offloaded openmp target regions\.In2022 IEEE/ACM Eighth Workshop on the LLVM Compiler Infrastructure in HPC \(LLVM\-HPC\),Vol\.,pp\. 23–33\.External Links:[Document](https://dx.doi.org/10.1109/LLVM-HPC56686.2022.00008)Cited by:[§5](https://arxiv.org/html/2608.13759#S5.SS0.SSS0.Px1.p1.1)\.
- Holket al\.\(2013\)E\. Holk, M\. Pathirage, A\. Chauhan, A\. Lumsdaine, and N\. D\. MatsakisGPU programming in rust: implementing high\-level abstractions in a systems\-level language\.In2013 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum, Cambridge, MA, USA, May 20\-24, 2013,pp\. 315–324\.External Links:[Link](https://doi.org/10.1109/IPDPSW.2013.173),[Document](https://dx.doi.org/10.1109/IPDPSW.2013.173)Cited by:[§7](https://arxiv.org/html/2608.13759#S7.p4.1)\.
- Huberet al\.\(2022\)J\. Huber, M\. Cornelius, G\. Georgakoudis, S\. Tian, J\. M\. M\. Diaz, K\. Dinel, B\. Chapman, and J\. DoerfertEfficient execution of openmp on gpus\.InProceedings of the 20th IEEE/ACM International Symposium on Code Generation and Optimization,CGO ’22,pp\. 41–52\.External Links:ISBN 9781665405843,[Link](https://doi.org/10.1109/CGO53902.2022.9741290),[Document](https://dx.doi.org/10.1109/CGO53902.2022.9741290)Cited by:[§5](https://arxiv.org/html/2608.13759#S5.p2.1),[§7](https://arxiv.org/html/2608.13759#S7.p6.1)\.
- Huber \(2023\)J\. HuberLLVM C Library for GPUs\.Note:Presentation at the 2023 LLVM Developers’ Meeting[https://llvm\.org/devmtg/2023\-10/slides/techtalks/Huber\-LibCforGPUs\.pdf](https://llvm.org/devmtg/2023-10/slides/techtalks/Huber-LibCforGPUs.pdf)Cited by:[§4\.2](https://arxiv.org/html/2608.13759#S4.SS2.SSS0.Px2.p1.1)\.
- Junget al\.\(2019\)R\. Jung, H\. Dang, J\. Kang, and D\. DreyerStacked borrows: an aliasing model for Rust\.Proc\. ACM Program\. Lang\.4\(POPL\),pp\. 1–32\.External Links:[Document](https://dx.doi.org/10.1145/3371109)Cited by:[§2\.1](https://arxiv.org/html/2608.13759#S2.SS1.SSS0.Px2.p1.1)\.
- Junget al\.\(2017\)R\. Jung, J\. Jourdan, R\. Krebbers, and D\. DreyerRustBelt: securing the foundations of the Rust programming language\.Proc\. ACM Program\. Lang\.2\(POPL\),pp\. 1–34\.External Links:ISSN 2475\-1421,[Document](https://dx.doi.org/10.1145/3158154)Cited by:[§1](https://arxiv.org/html/2608.13759#S1.p2.1)\.
- Kazdadi \(2026\)S\. Q\. E\. KazdadiFaer: a linear algebra library for the rust programming language\.Journal of Open Source Software11\(123\),pp\. 6099\.External Links:[Document](https://dx.doi.org/10.21105/joss.06099),[Link](https://doi.org/10.21105/joss.06099)Cited by:[§2\.1](https://arxiv.org/html/2608.13759#S2.SS1.SSS0.Px6.p1.1)\.
- Köster \(2015\)J\. KösterRust\-Bio: a fast and safe bioinformatics library\.Bioinformatics32\(3\),pp\. 444–446\.External Links:ISSN 1367\-4803,[Document](https://dx.doi.org/10.1093/bioinformatics/btv573),[Link](https://doi.org/10.1093/bioinformatics/btv573),https://academic\.oup\.com/bioinformatics/article\-pdf/32/3/444/49016785/bioinformatics\_32\_3\_444\.pdfCited by:[§1](https://arxiv.org/html/2608.13759#S1.p2.1)\.
- \[18\]\(2024\)Make Inference Faster: Efficient GPU Memory Management for Butterfly Sparse Matrix Multiplication\.Note:\[Online; accessed 12\. Jun\. 2026\]External Links:[Link](https://arxiv.org/html/2405.15013v1)Cited by:[§3\.3](https://arxiv.org/html/2608.13759#S3.SS3.p1.1)\.
- Matsakis and Klock \(2014\)N\. D\. Matsakis and F\. S\. KlockThe rust language\.ACM SIGAda Ada Letters34\(3\),pp\. 103–104\.Cited by:[§1](https://arxiv.org/html/2608.13759#S1.p2.1)\.
- \[20\]\(2020\)Optimizing GPU Memory Allocation and Movement using SYCL\|\|Argonne Leadership Computing Facility\.External Links:[Link](https://www.alcf.anl.gov/support-center/training/optimizing-gpu-memory-allocation-and-movement-using-sycl-0)Cited by:[§1](https://arxiv.org/html/2608.13759#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.13759#S4.SS1.p1.1)\.
- Parrishet al\.\(2023\)J\. Parrish, N\. Wren, T\. H\. Kiang, A\. Hayashi, J\. Young, and V\. SarkarTowards Safe HPC: Productivity and Performance via Rust Interfaces for a Distributed C\+\+ Actors Library \(Work in Progress\)\.InMPLR 2023: Proceedings of the 20th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes,pp\. 165–172\.External Links:ISBN 979\-840070380\-5,[Document](https://dx.doi.org/10.1145/3617651.3622992)Cited by:[§1](https://arxiv.org/html/2608.13759#S1.p2.1)\.
- Pearceet al\.\(2024\)O\. Pearce, J\. Burmark, R\. Hornung, B\. Bogale, I\. Lumsden, M\. McKinsey, D\. Yokelson, D\. Boehme, S\. Brink, M\. Taufer, and T\. ScoglandRAJA Performance Suite: Performance Portability Analysis with Caliper and Thicket\.InACM Conferences,pp\. 1206–1218\.External Links:[Document](https://dx.doi.org/10.1109/SCW63240.2024.00162)Cited by:[5th item](https://arxiv.org/html/2608.13759#S1.I1.i5.p1.1),[§5](https://arxiv.org/html/2608.13759#S5.SS0.SSS0.Px3.p1.1),[§6](https://arxiv.org/html/2608.13759#S6.p1.1)\.
- Perkel \(2020\)J\. M\. PerkelWhy scientists are turning to Rust\.Nature588,pp\. 185–186\.External Links:[Document](https://dx.doi.org/10.1038/d41586-020-03382-2)Cited by:[§1](https://arxiv.org/html/2608.13759#S1.p2.1)\.
- \[24\]\(2026\)rust\-cuda\.External Links:[Link](https://github.com/Rust-GPU/Rust-CUDA)Cited by:[§7](https://arxiv.org/html/2608.13759#S7.p3.1)\.
- \[25\]\(2026\)rust\-gpu\.External Links:[Link](https://github.com/rust-gpu/rust-gpu)Cited by:[§7](https://arxiv.org/html/2608.13759#S7.p2.1)\.
- Tianet al\.\(2022\)S\. Tian, J\. Huber, K\. Parasyris, B\. Chapman, and J\. DoerfertDirect gpu compilation and execution for host applications with openmp parallelism\.In2022 IEEE/ACM Eighth Workshop on the LLVM Compiler Infrastructure in HPC \(LLVM\-HPC\),Vol\.,pp\. 43–51\.External Links:[Document](https://dx.doi.org/10.1109/LLVM-HPC56686.2022.00010)Cited by:[§4\.2](https://arxiv.org/html/2608.13759#S4.SS2.SSS0.Px2.p1.1)\.
- \[27\]\(2026\)Tracking Issue for algebraic floating point methods⋅\\cdotIssue \#136469⋅\\cdotrust\-lang/rust\.External Links:[Link](https://github.com/rust-lang/rust/issues/136469)Cited by:[§6](https://arxiv.org/html/2608.13759#S6.SS0.SSS0.Px4.p1.1)\.
- Treinishet al\.\(2022\)M\. Treinish, I\. Carvalho, G\. Tsilimigkounakis, and N\. SáRustworkx: a high\-performance graph library for python\.Journal of Open Source Software7\(79\),pp\. 3968\.External Links:[Document](https://dx.doi.org/10.21105/joss.03968),[Link](https://doi.org/10.21105/joss.03968)Cited by:[§1](https://arxiv.org/html/2608.13759#S1.p2.1)\.
- \[29\]\(2026\)Validating References with Lifetimes \- The Rust Programming Language\.External Links:[Link](https://doc.rust-lang.org/book/ch10-03-lifetime-syntax.html#generic-lifetimes-in-functions)Cited by:[§2\.1](https://arxiv.org/html/2608.13759#S2.SS1.SSS0.Px5.p1.1)\.
- VectorWare \(2026\)VectorWareRust’s standard library on the GPU\.External Links:[Link](https://www.vectorware.com/blog/rust-std-on-gpu)Cited by:[§4\.2](https://arxiv.org/html/2608.13759#S4.SS2.SSS0.Px2.p1.1)\.
- Villaniet al\.\(2025\)N\. Villani, J\. Hostert, D\. Dreyer, and R\. JungTree Borrows\.Proc\. ACM Program\. Lang\.9\(PLDI\),pp\. 1019–1042\.External Links:[Document](https://dx.doi.org/10.1145/3735592)Cited by:[§2\.1](https://arxiv.org/html/2608.13759#S2.SS1.SSS0.Px2.p1.1)\.
- \[32\]\(2026\)Writing gpu shaders in plain rust\.Note:Presentation at the 2026 RustWeek conferenceExternal Links:[Link](https://drive.google.com/file/d/1pX0kC0gN_8ueKGvnuAAxcH7B-EBAatKA/view)Cited by:[§7](https://arxiv.org/html/2608.13759#S7.p2.1)\.
- Zhanget al\.\(2022\)Y\. Zhang, Y\. Zhang, G\. Portokalidis, and J\. XuTowards Understanding the Runtime Performance of Rust\.InASE ’22: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering,pp\. 1–6\.External Links:ISBN 978\-1\-45039475\-8,[Document](https://dx.doi.org/10.1145/3551349.3559494)Cited by:[§1](https://arxiv.org/html/2608.13759#S1.p2.1)\.