Experimenting with Rust and CloudFoundry buildpacks

Before we go further... what is a buildpack?

A buildpack is a collection of scripts that prepares a virtual machine (dyno in Heroku slang) to build and run your application in the cloud. In this article we're going to see how to build one of these.

Ok, let's see...

The documentation about custom buildpack explains how a buildpack works, and which files are necessary to successfully build and run a project in the cloud (foundry).

Custom buildpacks need three (executable) files:

• bin/detect

  This script determines whethever or not to apply the buildpack to an application. Basically it checks if you are using the wrong buildpack for your project, and stops you before you try to use the Java buildpack for a Rust application. Buildpacks can also be "framework-specific", so it makes sense to check if you're trying to run a Gradle buildpack to build an application meant to be built by Maven.

  It is run with only one parameter: the path to the build directory, where the source code and the files loaded during cf push are loaded. In a Rust application, it will contain your source code.

  CloudFoundry expects it to return 0 if everything is ok and the buildpack can continue, 1 otherwise.

• bin/compile

  This script is in charge of configuring the environment and generating a runnable binary from sources. A Rust buildpack would download rustup, install it and use cargo build to build your application.

  This script is run with two parameters: the build directory and the cache directory. The last one will contain the assets created during the build process. A Rust buildpack we will speak about later used to store the compiled libraries there, in order to "cache" dependencies and make deploys faster.

• bin/release

  This script must generate a .yml file that describes how the application should be executed. An example of such a .yml follows:

  default_process_types:
      web: ./target/release/YOUR_APPLICATION_NAME # in Rust, this suffices.
  

  This script only takes one parameter, the build directory. Please note: some developers use this script as a simple template, in order to automatically generate the correct .yml file during the execution of bin/compile.

An example of a buildpack that has these three scripts is the Rust buildpack built by Aaron Santavicca. If we look at it, we can see that:

• bin/detect checks if our project has a Cargo.toml file. If it doesn't, it cannot be built with cargo.
• bin/compile installs rustup, builds your application with cargo and generates the real bin/release script.
• bin/release is a template to be modified when running bin/compile.

...and that's it. CloudFoundry runs these scripts, reads the .yml file and runs everything as requested. Yay!

Hey, I'm used to Heroku, do I need to change my trusted buildpack?

No, you don't need it.

CloudFoundry buildpacks are compatible with Heroku ones: the good guy HoverBear provides a working buildpack for Rust that relies on multirust and caches build artifacts (mostly dependencies), so you don't have to lose five minutes to build the world every time you launch cf push (at least, it used to do that before RustFest, to brutally fix an unexpected problem with removed dependencies and broken slug size limits).

However, making them run is a bit trickier. When I was evaluating Rust buildpacks, I looked at the HoverBear one, and... bin/release script wasn't there. What kind of magic was going here? How did it even work?

After some testing (and reading the docs), I found out that there are... two ways to make buildpacks missing bin/release work.

The first one is adding the command attribute to the manifest.yml file you wrote for your application. A command: ./target/release/YOUR_APPLICATION_NAME should suffice.

The second one is to add a Procfile to the project. A Procfile is a file that tells CloudFoundry (or Heroku) which command has to be executed to start your application. A simple Procfile for a Rust app may just contain the line

web: ./target/release/YOUR_APPLICATION_NAME

Conclusion

My team and I are developing web apps on Predix (a CloudFoundry instance managed by GE), and I use Rust to build some quick-and-dirty test backends to generate random responses for our frontend. To run Rust on CF, I needed to understand how to build a new application on it, and I spent some time to fully understand how things work. I hope these notes can be useful for you too!