Blog

Practical Lessons from Porting range-set-blaze to this Container-Like Environment

Do you want your Rust code to run everywhere — from large servers to web pages, robots, and even watches? In this first of three articles, I’ll detail the steps to make that happen.

Running Rust in constrained environments presents challenges. Your code may not have access to a complete operating system such as Linux, Windows, or macOS. You may have limited (or no) access to files, networks, time, random numbers, and even memory. We’ll explore workarounds and solutions.

This first article focuses on running code on “WASM WASI”, a container-like environment. We’ll see that WASM WASI may (or may not) be useful in its own right. However, it is valuable as a first step toward running Rust in browsers or embedded systems.

Porting code to run on WASM WASI requires many steps and choices. Navigating these choices can be time consuming. Missing a step can lead to failure. We’ll reduce this complication by offering nine rules, which we’ll explore in detail:

1. Prepare for disappointment: WASM WASI is easy, but — for now — mostly useless — except as a steppingstone.
2. Understand Rust targets.
3. Install the wasm32-wasip1 target and WASMTIME, then create “Hello, WebAssembly!”.
4. Understand conditional compilation.
5. Run regular tests but with the WASM WASI target.
6. Understand Cargo features.
7. Change the things you can: dependency issues by choosing Cargo features, 64-bit/32-bit issues.
8. Accept that you cannot change everything: Networking, Tokio, Rayon, etc.
9. Add WASM WASI to your CI (continuous integration) tests.

Aside: These articles are based on a three-hour workshop that I presented at RustConf24 in Montreal. Thanks to the participants of that workshop. A special thanks, also, to the volunteers from the Seattle Rust Meetup who helped test this material. These articles replace an article I wrote last year with updated information.

Before we look at the rules one by one, let’s define our terms.

• Native: Your home OS (Linux, Windows, macOS)
• Standard library (std): Provides Rust’s core functionality — Vec, String, file input/output, networking, time.
• WASM: WebAssembly (WASM) is a binary instruction format that runs in most browsers (and beyond).
• WASI: WebAssembly System Interface (WASI) allows outside-the-browser WASM to access file I/O, networking (not yet), and time handling.
• no_std: Instructs a Rust program not to use the full standard library, making it suitable for small, embedded devices or highly resource-constrained environments.
• alloc: Provides heap memory allocation capabilities (Vec, String, etc.) in no_std environments, essential for dynamically managing memory.

With these terms in mind, we can visualize the environments we want our code to run in as a Venn diagram of progressively tighter constraints. This article details how to move from native to WASM WASI. The second article tells how to then move to WASM in the Browser. The final article will cover running Rust in no_std environments, both with and without alloc, ideal for embedded systems.

Based on my experience with range-set-blaze, a data structure project, here are the decisions I recommend, described one at a time. To avoid wishy-washiness, I’ll express them as rules.

Rule 1: Prepare for disappointment: WASM WASI is easy, but — for now — mostly useless — except as a steppingstone.

In 2019, Docker co-creator Solomon Hykes tweeted:

If WASM+WASI existed in 2008, we wouldn’t have needed to created Docker. That’s how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let’s hope WASI is up to the task.

Today, if you follow technology news, you’ll see optimistic headlines like these:

If WASM WASI were truly ready and useful, everyone would already be using it. The fact that we keep seeing these headlines suggests it’s not yet ready. In other words, they wouldn’t need to keep insisting that WASM WASI is ready if it really were.

As of WASI Preview 1, here is how things stand: You can access some file operations, environment variables, and have access to time and random number generation. However, there is no support for networking.

WASM WASI might be useful for certain AWS Lambda-style web services, but even that’s uncertain. Because wouldn’t you prefer to compile your Rust code natively and run twice as fast at half the cost compared to WASM WASI?

Maybe WASM WASI is useful for plug ins and extensions. In genomics, I have a Rust extension for Python, which I compile for 25 different combinations (5 versions of Python across 5 OS targets). Even with that, I don’t cover every possible OS and chip family. Could I replace those OS targets with WASM WASI? No, it would be too slow. Could I add WASM WASI as a sixth “catch-all” target? Maybe, but if I really need portability, I’m already required to support Python and should just use Python.

So, what is WASM WASI good for? Right now, its main value lies in being a step toward running code in the browser or on embedded systems.

Rule 2: Understand Rust targets.

In Rule 1, I mentioned “OS targets” in passing. Let’s look deeper into Rust targets — essential information not just for WASM WASI, but also for general Rust development.

On my Windows machine, I can compile a Rust project to run on Linux or macOS. Similarly, from a Linux machine, I can compile a Rust project to target Windows or macOS. Here are the commands I use to add and check the Linux target to a Windows machine:

rustup target add x86_64-unknown-linux-gnu
cargo check --target x86_64-unknown-linux-gnu

Aside: While cargo check verifies that the code compiles, building a fully functional executable requires additional tools. To cross-compile from Windows to Linux (GNU), you’ll also need to install the Linux GNU C/C++ compiler and the corresponding toolchain. That can be tricky. Fortunately, for the WASM targets we care about, the required toolchain is easy to install.

To see all the targets that Rust supports, use the command:

rustc --print target-list

It will list over 200 targets including x86_64-unknown-linux-gnu, wasm32-wasip1, and wasm32-unknown-unknown.

Target names contain up to four parts: CPU family, vendor, OS, and environment (for example, GNU vs LVMM):

Now that we understand something of targets, let’s go ahead and install the one we need for WASM WASI.

Rule 3: Install the wasm32-wasip1 target and WASMTIME, then create “Hello, WebAssembly!”.

To run our Rust code on WASM outside of a browser, we need to target wasm32-wasip1 (32-bit WebAssembly with WASI Preview 1). We’ll also install WASMTIME, a runtime that allows us to run WebAssembly modules outside of the browser, using WASI.

rustup target add wasm32-wasip1
cargo install wasmtime-cli

To test our setup, let’s create a new “Hello, WebAssembly!” Rust project using cargo new. This initializes a new Rust package:

cargo new hello_wasi
cd hello_wasi

Edit src/main.rs to read:

fn main() {
    #[cfg(not(target_arch = "wasm32"))]
    println!("Hello, world!");
    #[cfg(target_arch = "wasm32")]
    println!("Hello, WebAssembly!");
}

Aside: We’ll look deeper into the #[cfg(…)] attribute, which enables conditional compilation, in Rule 4.

Now, run the project with cargo run, and you should see Hello, world! printed to the console.

Next, create a .cargo/config.toml file, which specifies how Rust should run and test the project when targeting WASM WASI.

[target.wasm32-wasip1]
runner = "wasmtime run --dir ."

Aside: This .cargo/config.toml file is different from the main Cargo.toml file, which defines your project’s dependencies and metadata.

Now, if you say:

cargo run --target wasm32-wasip1

You should see Hello, WebAssembly!. Congratulations! You’ve just successfully run some Rust code in the container-like WASM WASI environment.

Rule 4: Understand conditional compilation.

Now, let’s investigate #[cfg(…)]—an essential tool for conditionally compiling code in Rust. In Rule 3, we saw:

fn main() {
    #[cfg(not(target_arch = "wasm32"))]
    println!("Hello, world!");
    #[cfg(target_arch = "wasm32")]
    println!("Hello, WebAssembly!");
}

The #[cfg(…)] lines tell the Rust compiler to include or exclude certain code items based on specific conditions. A “code item” refers to a unit of code such as a function, statement, or expression.

With #[cfg(…)] lines, you can conditionally compile your code. In other words, you can create different versions of your code for different situations. For example, when compiling for the wasm32 target, the compiler ignores the #[cfg(not(target_arch = “wasm32”))] block and only includes the following:

fn main() {
    println!("Hello, WebAssembly!");
}

You specify conditions via expressions, for example, target_arch = “wasm32”. Supported keys include target_os and target_arch. See the Rust Reference for the full list of supported keys. You can also create expressions with Cargo features, which we will learn about in Rule 6.

You may combine expressions with the logical operators not, any, and all. Rust’s conditional compilation doesn’t use traditional if…then…else statements. Instead, you must use #[cfg(…)] and its negation to handle different cases:

#[cfg(not(target_arch = "wasm32"))]
...
#[cfg(target_arch = "wasm32")]
...

To conditionally compile an entire file, place #![cfg(…)] at the top of the file. (Notice the “!”). This is useful when a file is only relevant for a specific target or configuration.

You can also use cfg expressions in Cargo.toml to conditionally include dependencies. This allows you to tailor dependencies to different targets. For example, this says “depend on Criterion with Rayon when not targeting wasm32”.

[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
criterion = { version = "0.5.1", features = ["rayon"] }

Aside: For more information on using cfg expressions in Cargo.toml, see my article: Nine Rust Cargo.toml Wats and Wat Nots: Master Cargo.toml formatting rules and avoid frustration | Towards Data Science (medium.com).

Rule 5: Run regular tests but with the WASM WASI target.

It’s time to try to run your project on WASM WASI. As described in Rule 3, create a .cargo/config.toml file for your project. It tells Cargo how to run and test your project on WASM WASI.

[target.wasm32-wasip1]
runner = "wasmtime run --dir ."

Next, your project — like all good code — should already contain tests. My range-set-blaze project includes, for example, this test:

#[test]
fn insert_255u8() {
    let range_set_blaze = RangeSetBlaze::<u8>::from_iter([255]);
    assert!(range_set_blaze.to_string() == "255..=255");
}

Let’s now attempt to run your project’s tests on WASM WASI. Use the following command:

cargo test --target wasm32-wasip1

If this works, you may be done — but it probably won’t work. When I try this on range-set-blaze, I get this error message that complains about using Rayon on WASM.

 error: Rayon cannot be used when targeting wasi32. Try disabling default features.
  --> C:Userscarlk.cargoregistrysrcindex.crates.io-6f17d22bba15001fcriterion-0.5.1srclib.rs:31:1
   |
31 | compile_error!("Rayon cannot be used when targeting wasi32. Try disabling default features.");

To fix this error, we must first understand Cargo features.

Rule 6: Understand Cargo features.

To resolve issues like the Rayon error in Rule 5, it’s important to understand how Cargo features work.

In Cargo.toml, an optional [features] section allows you to define different configurations, or versions, of your project depending on which features are enabled or disabled. For example, here is a simplified part of the Cargo.toml file from the Criterion benchmarking project:

[features]
default = ["rayon", "plotters", "cargo_bench_support"]
rayon = ["dep:rayon"]
plotters = ["dep:plotters"]
html_reports = []
cargo_bench_support = []

[dependencies]
#...
# Optional dependencies
rayon = { version = "1.3", optional = true }
plotters = { version = "^0.3.1", optional = true, default-features = false, features = [
    "svg_backend",
    "area_series",
    "line_series",
] }

This defines four Cargo features: rayon, plotters, html_reports, and cargo_bench_support. Since each feature can be included or excluded, these four features create 16 possible configurations of the project. Note also the special default Cargo feature.

A Cargo feature can include other Cargo features. In the example, the special default Cargo feature includes three other Cargo features — rayon, plotters, and cargo_bench_support.

A Cargo feature can include a dependency. The rayon Cargo feature above includes the rayon crate as a dependent package.

Moreover, dependent packages may have their own Cargo features. For example, the plotters Cargo feature above includes the plotters dependent package with the following Cargo features enabled: svg_backend, area_series, and line_series.

You can specify which Cargo features to enable or disable when running cargo check, cargo build, cargo run, or cargo test. For instance, if you’re working on the Criterion project and want to check only the html_reports feature without any defaults, you can run:

cargo check --no-default-features --features html_reports

This command tells Cargo not to include any Cargo features by default but to specifically enable the html_reports Cargo feature.

Within your Rust code, you can include/exclude code items based on enabled Cargo features. The syntax uses #cfg(…), as per Rule 4:

#[cfg(feature = "html_reports")]
SOME_CODE_ITEM

With this understanding of Cargo features, we can now attempt to fix the Rayon error we encountered when running tests on WASM WASI.

Rule 7: Change the things you can: dependency issues by choosing Cargo features, 64-bit/32-bit issues.

When we tried running cargo test –target wasm32-wasip1, part of the error message stated: Criterion … Rayon cannot be used when targeting wasi32. Try disabling default features. This suggests we should disable Criterion’s rayon Cargo feature when targeting WASM WASI.

To do this, we need to make two changes in our Cargo.toml. First, we need to disable the rayon feature from Criterion in the [dev-dependencies] section. So, this starting configuration:

[dev-dependencies]
criterion = { version = "0.5.1", features = ["html_reports"] }

becomes this, where we explicitly turn off the default features for Criterion and then enable all the Cargo features except rayon.

[dev-dependencies]
criterion = { version = "0.5.1", features = [
        "html_reports",
        "plotters",
        "cargo_bench_support"],
      default-features = false }

Next, to ensure rayon is still used for non-WASM targets, we add it back in with a conditional dependency in Cargo.toml as follows:

[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
criterion = { version = "0.5.1", features = ["rayon"] }

In general, when targeting WASM WASI, you may need to modify your dependencies and their Cargo features to ensure compatibility. Sometimes this process is straightforward, but other times it can be challenging — or even impossible, as we’ll discuss in Rule 8.

Aside: In the next article in this series — about WASM in the Browser — we’ll go deeper into strategies for fixing dependencies.

After running the tests again, we move past the previous error, only to encounter a new one, which is progress!

#[test]
fn test_demo_i32_len() {
    assert_eq!(demo_i32_len(i32::MIN..=i32::MAX), u32::MAX as usize + 1);
                                                  ^^^^^^^^^^^^^^^^^^^^^ attempt to compute 
`usize::MAX + 1_usize`, which would overflow
}

The compiler complains that u32::MAX as usize + 1 overflows. On 64-bit Windows the expression doesn’t overflow because usize is the same as u64 and can hold u32::MAX as usize + 1. WASM, however, is a 32-bit environment so usize is the same as u32 and the expression is one too big.

The fix here is to replace usize with u64, ensuring that the expression doesn’t overflow. More generally, the compiler won’t always catch these issues, so it’s important to review your use of usize and isize. If you’re referring to the size or index of a Rust data structure, usize is correct. However, if you’re dealing with values that exceed 32-bit limits, you should use u64 or i64.

Aside: In a 32-bit environment, a Rust array, Vec, BTreeSet, etc., can only hold up to 2³²−1=4,294,967,295 elements.

So, we’ve fixed the dependency issue and addressed a usize overflow. But can we fix everything? Unfortunately, the answer is no.

Rule 8: Accept that you cannot change everything: Networking, Tokio, Rayon, etc.

WASM WASI Preview 1 (the current version) supports file access (within a specified directory), reading environment variables, and working with time and random numbers. However, its capabilities are limited compared to what you might expect from a full operating system.

If your project requires access to networking, asynchronous tasks with Tokio, or multithreading with Rayon, Unfortunately, these features aren’t supported in Preview 1.

Fortunately, WASM WASI Preview 2 is expected to improve upon these limitations, offering more features, including better support for networking and possibly asynchronous tasks.

Rule 9: Add WASM WASI to your CI (continuous integration) tests.

So, your tests pass on WASM WASI, and your project runs successfully. Are you done? Not quite. Because, as I like to say:

If it’s not in CI, it doesn’t exist.

Continuous integration (CI) is a system that can automatically run your tests every time you update your code, ensuring that your code continues to work as expected. By adding WASM WASI to your CI, you can guarantee that future changes won’t break your project’s compatibility with the WASM WASI target.

In my case, my project is hosted on GitHub, and I use GitHub Actions as my CI system. Here’s the configuration I added to .github/workflows/ci.yml to test my project on WASM WASI:

test_wasip1:
      name: Test WASI P1
      runs-on: ubuntu-latest
      steps:
        - name: Checkout
          uses: actions/checkout@v4
        - name: Set up Rust
          uses: dtolnay/rust-toolchain@master
          with:
            toolchain: stable
            targets: wasm32-wasip1
        - name: Install Wasmtime
          run: |
            curl https://wasmtime.dev/install.sh -sSf | bash
            echo "${HOME}/.wasmtime/bin" >> $GITHUB_PATH
        - name: Run WASI tests
          run: cargo test --verbose --target wasm32-wasip1

By integrating WASM WASI into CI, I can confidently add new code to my project. CI will automatically test that all my code continues to support WASM WASI in the future.

So, there you have it — nine rules for porting your Rust code to WASM WASI. Here is what surprised me about porting to WASM WASI:

The Bad:

• Running on WASM WASI offers little utility today. It, however, holds the potential to be useful tomorrow.
• In Rust, there’s a common saying: “If it compiles, it works.” Unfortunately, this doesn’t always hold true for WASM WASI. If you use an unsupported feature, like networking, the compiler won’t catch the error. Instead, it will fail at runtime. For example, this code compiles and runs on WASM WASI but always returns an error because networking isn’t supported.

use std::net::TcpStream;

fn main() {
    match TcpStream::connect("crates.io:80") {
        Ok(_) => println!("Successfully connected."),
        Err(e) => println!("Failed to connect: {e}"),
    }
}

The Good:

• Running on WASM WASI is a good first step toward running your code in the browser and on embedded systems.
• You can run Rust code on WASM WASI without needing to port to no_std. (Porting to no_std is the topic of the third article of this series.)
• You can run standard Rust tests on WASM WASI, making it easy to verify your code.
• The .cargo/config.toml file and Rust’s –target option make it incredibly straightforward to configure and run your code on different targets—including WASM WASI.

Stay tuned! In the next article, I’ll show you how to port your Rust code to run on WASM in the browser — an ability I find super useful. After that, the final article will explain porting code to embedded systems, which I find incredibly cool.

Aside: Interested in future articles? Please follow me on Medium. I write about Rust and Python, scientific programming, machine learning, and statistics. I tend to write about one article per month.

Nine Rules for Running Rust on WASM WASI was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.