Rust and WebAssembly - what I learnt from porting hrm-interpreter to wasm

Source

Some time ago, I wrote a compiler for a fictional assembly-like language. As this compiler (named hrm-compiler) performs some optimizations on the code, I wanted to know if the optimizations were breaking the compiled code. To test them, I wrote a small interpreter (hrm-interpreter) to run a JSON version of the compiled code, and then embedded the webassembly version in the bare-bone debugger I wrote as a web application.

If you want to try, the best way to get started is the Rust and Webassembly Book , a tutorial that lets you write a Rust crate that can interact with Javascript and publish it to npm. I followed the tutorial, and managed to create a wasm wrapper of hrm-interpreter, named (such fantasy! amaze!) hrm-interpreter-wasm. Let me explain what I learnt while creating it.

Update: the blog post you're reading right now was written in late 2018: some suggestions may be outdated. Let me suggest a newer article about Rust and WebAssembly, written by TopTal's Peter Suggate: WebAssembly/Rust Tutorial: Pitch-perfect Audio Processing.

Write a different crate to support wasm

If you need to convert Javascript types to Rust ones, I'd suggest to write a specific crate. It's better to think about it as if you're writing a wrapper. This new crate lets you write wasm-specific code, without modifying your library to support this new platform.

Not every library needs to create an ad-hoc crate, though: handlebars-rust offers users a flag to turn off filesystem-based I/O. This may work if your main library does not expose complex types, or you are already handling platform-specific stuff.

Don't force file I/O in the main library

What does an interpreter do?

  1. read a file

  2. convert its content to a list of operations

  3. execute each operation until the program ends or some error is raised

  4. writes the output of the program to a file (or a terminal)

Unfortunately, you cannot read or write files in a wasm environment - you cannot perform I/O! So we need to find another way: let's look at the list again.

  1. read a file and extracts its text as a huge string

  2. convert the content of the string to a list of operations

  3. execute each operation until the program ends or some error is raised

  4. captures the output of the program into a string

  5. writes the output string to a file (or a terminal)

So, we may notice that the focus is now on strings, not files.

As javascript can perform I/O (read files via FileReader, generate new files via FileWriter, or even do HTTP requests!), our wrapper crate may just take a string, pass it to the main crate and return the output string. In my case, I had to create some new methods to sidestep I/O from files.

There are other cases where you may need to modify your crate to port it to wasm: I suggest reading Which Crates Work with Wasm? to learn more.

try to expose as little state as possible - use functions instead

Javascript cannot manipulate complex stuff, such as options, so you cannot export whatever you want. Before wasting several hours trying to understand why you cannot "export" your beautiful struct with your awesome Option<Result<Beatiful<Type>>>, check out wasm-bindgen's Supported Types documentation chapter. Trust me :).

To work around it, in hrm-interpreter-wasm...

  • the main structure has no public fields

  • the internal state is serialized to a JSON-formatted string

  • data from Javascript are also JSON-formatted strings

This is not the best interface, but it works for now. "Now" means that I'll fix it once I integrate some work-in-progress stuff, such as the new Rust-based compiler and sourcemap support... If you can, try to avoid serialization, as it's pretty expensive... and we're using Rust in the browser to run code faster! ;)

Browser-side: use webpack 4!

Please note that this article was written in late 2018 - the TopTal article shows a different way to load wasm files by using webassembly.compile.

If you want to test your new crate in a real application, you may want to use webpack. Starting from version 4, Webpack ships an internal wasm loader: it makes integrating wasm-based libraries a breeze!

react-create-app offers a very simple way to get started. If you have an older project, I recommend you to switch to react-scripts ^2.1.1 and webpack ^4.26. I am not sure you need to do this in new projects, but I had to "eject" the webpack configuration (webpack.config.dev.js) and exclude ".wasm" files from the extensions managed by the "file" loader.

// "file" loader makes sure those assets get served by WebpackDevServer.
// When you `import` an asset, you get its (virtual) filename.
// In production, they would get copied to the `build` folder.
// This loader doesn't use a "test" so it will catch all modules
// that fall through the other loaders.
{
// Exclude `js` files to keep "css" loader working as it injects
// its runtime that would otherwise be processed through "file" loader.
// Also exclude `html` and `json` extensions so they get processed
// by webpacks internal loaders.
// PLEASE NOTE THE .wasm REGEX HERE!!!
exclude: [/\.(js|mjs|jsx|ts|tsx)$/, /\.html$/, /\.json$/, /\.wasm$/],
loader: require.resolve('file-loader'),
options: {
    name: 'static/media/[name].[hash:8].[ext]',
  },
}

If you found those notes useful, or you want to suggest something else, please tell me on Twitter , or consider offering me a Ko-fi !