Cargo Build of Crubit

This page explains how to build Crubit’s own tools with Cargo. To use Crubit in your project, start with Build System Integrations; the Bazel and CMake integrations build the tools for you.

cc_bindings_from_rs uses the Rust compiler’s internal APIs (rustc_private), so it requires a toolchain with the rustc-dev component and must be built by the same rustc that compiles the crates it generates bindings for. See Toolchain requirements for supported toolchains and setup instructions.

cc_bindings_from_rs

In a Crubit checkout, rustup automatically selects the nightly toolchain from rust-toolchain.toml, and you can build cc_bindings_from_rs with Cargo:

git clone https://github.com/google/crubit.git
cd crubit
cargo build --release -p cc_bindings_from_rs

Find more details on configuring the target directory and troubleshooting below.

If you are using a stable Rust toolchain, set the environment variable RUSTC_BOOTSTRAP=1:

$ RUSTC_BOOTSTRAP=1 cargo build # further args as above

Target Directory

A default Cargo build will generate a target/ directory adjacent to the Cargo.toml that will clutter your VCS client with intermediate build files. You can avoid this by setting your target directory somewhere outside your VCS. A config.toml can set your target directory to a non-standard location:

[build]
# You can also use something like `~/.cargo/target` if you want to keep the intermediaries around
target-dir = "/tmp/target/"

If you want your config.toml to apply to all your projects, place it at ~/.cargo/config.toml. See Cargo’s configuration documentation for the other locations you can place a config.toml and their scope.

Troubleshooting

`rustc_driver*.rmeta` build dependency

If cargo build ... --bin cc_bindings_from_rs fails to build, then check if the following problem has been reported as one of the first errors:

error[E0463]: can't find crate for `rustc_driver`
  --> cargo/cc_bindings_from_rs/run_compiler/../../../cc_bindings_from_rs/run_compiler.rs:10:1
   |
10 | extern crate rustc_driver;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

If you build Crubit using cargo and rustc that are managed by rustup, then you can just run the suggested command to add the rustc-dev component: rustup component add rustc-dev.

If you build Crubit using cargo and rustc that are built and installed using x.py install, then please ensure that x.py’s config.toml covers the rustc-dev component (h/t @Nadrieril for the PR to teach x.py install about this component). For example:

$ cat third_party/rust-src/config.toml
...
# With `extended = true`, `x.py install` will install rustc and the listed
# tools. This is the *only* way to install rust-analyzer-proc-macro-srv, which
# is necessary to use rust-analyzer.
extended = true
tools = [
    "cargo",
    "clippy",
    "rustc-dev",  # <= ASKS `./x.py install` TO COVER `rustc_driver*.rmeta`
    "rustfmt",
    "rust-analyzer",
    "rust-analyzer-proc-macro-srv",
    "src",
]
...

`librustc_driver*.so` runtime dependency

cc_bindings_from_rs depends at runtime on librustc_driver*.so. If the runtime linker cannot find this library, then launching cc_bindings_from_rs may fail with the following error:

$ third_party/rust-toolchain/bin/cc_bindings_from_rs --help
...
error while loading shared libraries: librustc_driver-871558eb5abca9d6.so:
cannot open shared object file: No such file or directory

rustc is able to find the librustc_driver*.so library, because (on certain platforms) x.py sets a RUNPATH in the rustc binary:

$ readelf -d third_party/rust-toolchain/bin/rustc | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../lib]
$ ls third_party/rust-toolchain/lib/librustc_driver-*.so
third_party/rust-toolchain/lib/librustc_driver-871558eb5abca9d6.so

If cc_bindings_from_rs is distributed/installed next to rustc, then you can fix the linking error by setting an identical RUNPATH when building cc_bindings_from_rs. This can be done by asking cargo to pass additional rustc command-line flags (e.g. using RUSTFLAGS or CARGO_ENCODED_RUSTFLAGS environment variables). The following command-line flags should work (they are based on x.py’s sources):

Linux:
- -Clink-args=-Wl,-z,origin
- -Clink-args=-Wl,-rpath,$ORIGIN/../lib
Mac:
- -Zosx-rpath-install-name
- -Clink-args=-Wl,-rpath,@loader_path/../lib
Windows:
- no extra command-line flags needed (TODO: verify this)

Unstable features on stable

Crubit uses unstable Rust features that are only available in nightly builds. We rely on setting the RUSTC_BOOTSTRAP environment variable to 1 to tell a stable Rust compiler to act like a nightly compiler.

If you attempt to build cc_bindings_from_rs with a stable Rust toolchain and do not set this variable, the build will fail with error messages like:

error[E0554]: `#![feature]` may not be used on the stable release channel

rs_bindings_from_cc

rs_bindings_from_cc is a Rust binary that depends on C++ libraries (Abseil, LLVM/Clang LibTooling, and Protobuf). To build it with Cargo in an open-source checkout, you first need to build and export these C++ dependencies.

Building with C++ dependencies from Bazel

This is how Crubit’s CI checks that the tool builds with Cargo. If you can use Bazel, prefer the full Bazel build instead.

Crubit provides a helper script cargo/build/setup_bazel_env.py that discovers prebuilt C++ dependencies in Bazel’s output tree, merges them into monolithic static archives, pre-generates Protobuf headers, and exports the necessary environment variables.

Prebuild C++ dependencies using bazelisk:

$ bazelisk build \
    --remote_download_outputs=all \
    rs_bindings_from_cc:rs_bindings_from_cc_main \
    @protobuf//:protoc

Discover and package Bazel outputs:

$ python3 cargo/build/setup_bazel_env.py

Configure your shell environment:

$ source target/bazel_outputs/bazel-env.sh

Build rs_bindings_from_cc using Cargo:

$ cargo build --locked -p rs_bindings_from_cc

Custom Build Systems (Chromium, Android, or Other Projects)

When integrating rs_bindings_from_cc into custom build systems (such as Chromium, Android, or other standalone environments), you must prepare prebuilt C++ dependencies and pre-generate C++ Protobuf headers before invoking cargo:

Pre-generate C++ Protobuf Headers: To avoid build ordering issues and race conditions during parallel Cargo builds, C++ Protobuf headers (rs_bindings_from_cc/ir.pb.h and rs_bindings_from_cc/generate_bindings/generate_bindings.pb.h) must be generated prior to invoking cargo build (see cargo_build_protobuf.md for architectural details). Crubit provides a helper script that invokes protoc to pre-generate these headers:
```
$ python3 cargo/build/generate_proto_headers.py \
    --protoc=<path/to/protoc> \
    --out_dir=<path/to/generated_headers>
```
Configure Environment Variables: Before invoking cargo, configure the following environment variables:
- Abseil:
  - ABSL_INCLUDE_PATH: Comma-separated list of Abseil header include roots (containing absl/...).
  - ABSL_LIB_STATIC_PATH: Comma-separated list of directories containing Abseil static archives (.a on Unix, .lib on Windows).
- LLVM / Clang:
  - CLANG_INCLUDE_PATH: Comma-separated list of include directories for LLVM and Clang headers (clang/..., llvm/..., and tablegen-generated headers).
  - CLANG_LIB_STATIC_PATH: Comma-separated list of directories containing Clang/LLVM static archives.
- Protobuf:
  - PROTOBUF_INCLUDE_PATH: Comma-separated list of include directories containing Google Protobuf headers (google/protobuf/..., utf8_range), as well as Crubit’s pre-generated Protobuf headers directory from step 1.
  - PROTOBUF_LIB_STATIC_PATH: Directory containing the Protobuf static archive (libprotobuf.a or protobuf.lib).
  - PROTOC: Absolute path to the protoc compiler executable. This is required by protobuf-codegen during the Cargo build to generate Rust Protobuf bindings for ir_rust_proto and generate_bindings_rust_proto.
- Toolchain & Linker:
  - CC / CXX: C and C++ compilers (e.g. Clang/Clang++).
  - CXXFLAGS: Extra C++ compiler flags. You must include -DCRUBIT_LLVM_DEV_DATE=<YYYYMMDD> in addition to standard flags like -stdlib=libc++. For example:
```
LLVM_DATE=$(python3 cargo/build/get_llvm_commit_date.py <llvm_commit_hash>)
export CXXFLAGS="-stdlib=libc++ -DCRUBIT_LLVM_DEV_DATE=${LLVM_DATE}"
```
  - RUSTFLAGS: Rust compiler flags for linking (e.g. -C linker=..., -C link-arg=-stdlib=libc++, -C link-arg=-lc++, -C link-arg=-lzstd).

Build rs_bindings_from_cc using Cargo:

$ cargo build --locked -p rs_bindings_from_cc

Keyboard shortcuts

Crubit Documentation