Crubit: C++/Rust Bidirectional Interop Tool
Crubit is a bidirectional bindings generator for C++ and Rust, allowing users to:
- Automatically reuse existing crates and headers: no IDL or glue code is necessary.
- Call back and forth between Rust and C++ with no performance cost: no copies or heap allocations required.
- Use idiomatic types from both Rust and C++, such as
Vec,Result,Option,std::vector,std::unique_ptr,std::shared_ptr, and more!
Example
Calling Rust from C++
Consider the following Rust library:
#![allow(unused)]
fn main() {
pub struct Account {
pub id: u64,
pub balance: f64,
}
impl std::fmt::Display for Account { ... }
// Takes shared references, mapped to const references in C++.
pub fn calculate_interest(account: &Account, rate: f64) -> f64 { ... }
// Takes a string slice, mapped to rs::StrRef in C++.
pub fn is_valid_username(username: &str) -> bool { ... }
}
You can call these Rust functions from C++:
#include "path/to/account.h"
#include <iostream>
void demo() {
account::Account my_account{.id = 123, .balance = 1000.0};
double interest = account::calculate_interest(my_account, 0.05);
// my_account is printable because Account implements Display in Rust!
std::cout << my_account << std::endl;
if (account::is_valid_username("bob")) {
std::cout << "Valid user" << std::endl;
}
}
Calling C++ from Rust
Consider the following C++ header:
#include <string_view>
#include <optional>
#include <memory>
struct User {
int id;
double balance;
};
std::optional<User> FindUser(int id);
std::unique_ptr<User> CreateUser(std::string_view name, int id);
You can call these C++ functions from Rust:
use cpp_std::unique_ptr;
use ffi_11::c_int;
use user_api::{CreateUser, FindUser, User};
let id: c_int = 123;
let user: Option<User> = FindUser(id);
if let Some(u) = user {
let balance: f64 = u.balance;
println!("User {} has balance {}", u.id, balance);
}
let new_user: unique_ptr<User> = CreateUser("Alice".into(), 456);
Why Crubit?
- No hand-written glue code. Crubit reads C++ headers and Rust crates directly and generates bindings in both directions. Your signatures can’t drift out of sync with the code they wrap, and you don’t need to maintain a separate interface file.
- More than the C ABI. Crubit can call functions that aren’t
extern "C", pass structs and enums by value, and match the memory layout of Rust structs and enums that aren’t#[repr(C)]. - Idiomatic types on both sides. References, slices,
&str,Option,Result, tuples, smart pointers,std::vector/Vec, callbacks,absl::Status, and more map to their natural counterparts. See the status page for the full table. - Traits and operators, too. C++ can call Rust trait implementations, and
Rust traits such as
Display,Clone, andDefaultbecome the matching C++ operators and special member functions. In the other direction, C++ operator overloads and special member functions become the matching Rust traits, such asPartialEq,Add,Clone, andDrop. - Safer than hand-written FFI. Crubit generates bindings from the real C++
headers and Rust crates at build time, with the same compiler and flags, so
mismatched ABIs, struct layouts, and signatures can’t creep in. C++
references get Rust lifetimes through lifetime elision and lifetime
annotations, so the Rust
borrow checker can check them. Slices, strings,
Option, and other types carry their bounds and invariants across the boundary instead of degrading to raw pointers. See Memory Safety for the details.
Getting Started
1. Set up your build
Crubit runs as part of your build, so that the bindings always match the code they wrap. Pick your build system:
| Build system | C++ calling Rust (cpp_api_from_rust) | Rust calling C++ (rust_api_from_cpp) |
|---|---|---|
| Bazel | Supported (example) | Supported |
| CMake | Supported (example) | Not yet |
| Other | Through the cargo cpp_api_from_rust subcommand | Not yet |
Crubit works with nightly Rust toolchains and with the two most recent stable
releases, as long as the rustc-dev component is installed (see the test
matrix below). There are no binary releases yet; pin a commit of
this repository. Build System
Integrations covers toolchain
requirements and how to set up each build system.
2. Use your Rust and C++ APIs together
Add a cpp_api_from_rust or rust_api_from_cpp target next to your library,
and start calling it from the other language. Each direction has a walkthrough
that explains what the generated API looks like and how to customize it, plus
copy-pastable examples that include snapshots of the generated bindings:
- C++ calling Rust: C++ Bindings for Rust
Libraries walkthrough and
examples/rust/. - Rust calling C++: Rust Bindings for C++
Libraries walkthrough and
examples/cpp/.
The links on the sidebar contain additional reference material.
Test Matrix
Supported Features
NOTE: The bug links below, of the form b/123456, are for Google-internal
tracking purposes.
This page is a high-level summary of what Crubit does and does not support today. See the per-feature pages linked from the sidebar for details.
Types
The following table summarizes the mapping of types between C++ and Rust. Rows
link to more detailed documentation where available. Most of these types are
layout-compatible: they have the same
in-memory representation in both languages. Primitives and raw pointers are
additionally ABI-compatible, and
std::string and protocol buffers are
bridged (converted by value). See
Types for what each of these means in practice.
| Type | C++ | Rust |
|---|---|---|
| Primitives | ||
| Integers | int32_t, int64_t, etc. | i32, i64, etc. |
| Floating point | float, double | f32, f64 |
| Pointers & References | ||
| Raw pointers | T*, const T* | *mut T, *const T |
| References | const T&, T& | &T, &mut T |
| String slices | rs::StrRef | &str |
| Slices | rs::SliceRef<T> | &[T], &mut [T] |
| Smart Pointers | ||
| Unique pointer | std::unique_ptr<T> | cpp_std::unique_ptr<T> |
| Shared pointer | std::shared_ptr<T> | cpp_std::shared_ptr<T> |
| Containers | ||
| C++ vector | std::vector<T> | cpp_std::vector<T> |
Rust Vec | rs::Vec<T> | Vec<T> |
| Fixed-size array | std::array<T, N> | [T; N] |
| Strings | ||
| String view | std::string_view | cpp_std::string_view |
| C++ string | std::string | cpp_std::string_wrapper |
Rust String | rs::alloc::string::String | String |
| Option & Result | ||
| C++ optional | std::optional<T> | cpp_std::optional<T> |
Rust Option | rs::Option<T> | Option<T> |
Rust Result | rs::Result<T, E> | Result<T, E> |
| Other | ||
| User-defined types | struct, class | struct, enum, union |
| Tuples | rs::Tuple<T1, T2, ..> | (T1, T2, ..) |
| Protocol Buffers | C++ proto | Rust proto |
| Status | absl::Status | status::Status |
| StatusOr | absl::StatusOr<T> | status::StatusOr<T> |
Not yet supported: Rust ! (b/254507801), () except as a return type or a
generic argument such as Result<(), E> (b/254099023), std::byte
(b/213960614), and Rust i128/u128 when calling Rust from C++ (b/254094650;
C++ __int128 is supported when calling C++ from Rust).
Rust bindings for C++ libraries
Crubit generates Rust bindings for everyday C++ APIs:
- Structs, unions, and enums, including non-Rust-movable (non-trivial) types.
- Functions and methods, including overloaded constructors and assignment
operators. Other overloads are supported when given distinct Rust names with
CRUBIT_RUST_NAME. constexpr/constinteger,bool, and character-array constants (at namespace scope or as static data members), which become Rustconsts; other global variables, which become Ruststatics.- References, with lifetimes inferred by Rust’s elision rules or given by explicit lifetime annotations.
- Type aliases, forward-declared types, and concrete class template
specializations such as
MyTemplate<int>.
Unreleased: non-virtual inheritance (upcasting, downcasting, inherited methods).
Not yet supported: automatic overload resolution (b/213280424), virtual dispatch
and overriding C++ methods from Rust (b/313733992), checking that C++ code
actually honors the elided or annotated lifetimes, C++ templates with generic
(non-concrete) type parameters (see Generics and
Templates), non-type using aliases, constants of other
types (e.g. floating-point or class types), and macros.
C++ bindings for Rust libraries
Crubit generates C++ bindings for everyday Rust APIs:
- Structs,
repr(C)unions, and enums, which become C++structs by default or C++enums with#[cpp_enum]. Other user-defined types get opaque bindings that can be passed around but not inspected from C++. - Functions, methods, and trait methods, along with associated types and constants.
- Type aliases,
usedeclarations, and simpleconstconstants.
Note
Well-known traits are mapped to their idiomatic C++ equivalents rather than exposed as plain methods:
Clone,Copy,Default, andDropbecome the special member functions;PartialEq,PartialOrd/Ord,Index/IndexMut, and the arithmetic and bitwise traits become operator overloads;HashbecomesAbslHashValue/std::hash;Displaybecomesoperator<<; andIntoIteratorbecomesbegin/end. See Traits.
Unreleased (behind feature flags): generic types (generics), and richer
bindings for enums with payloads and non-repr(C) unions.
Not yet supported: dyn Trait objects, defining C++ abstractions from Rust
(inheriting from or defining a C++ base class), statics and complex const
constants, and macros.
Usage outside of Google
Today:
- Bazel:
cpp_api_from_rustandrust_api_from_cppvia therules_crubitmodule. - CMake:
cpp_api_from_rustvia Corrosion. - Cargo:
cargo cpp_api_from_rustbuilds a Cargo package and emits a C++ header and static library.
See Build System Integrations for details.
Planned:
rust_api_from_cppin CMake and Cargo builds (b/539627008).- Prebuilt Crubit binaries that work with public Clang and Rust releases, so that you don’t need to build Crubit yourself against a matching toolchain.
Cardi, the Crubit Mascot
Meet Cardi, the official mascot of the Crubit project: a friendly blue squid.
Lore
As a squid, Cardi is naturally at home in the deep ocean. But in the world of software, Cardi lives under the C (well, C++).
Cardi’s mission is to help bridge the gap between C++ and Rust, ensuring they can communicate smoothly.
Types
Overview
crubit.rs/overview/status#types describes Crubit’s current and future for various types.
In brief, Crubit supports:
- Primitive types, such as
floatori32. - Pointer types, such as
float*or*const i32, including function pointers. - Rust standard library (
std) types, such asString,Option, andVec. - Callbacks, such as
absl::AnyInvocableand Rust callable traits (Fn,FnMut,FnOnce). - User-defined types, with some language-specific rules and restrictions. (See crubit.rs/cpp and crubit.rs/rust).
ABI-Compatibility
Certain references to C++ or Rust types will not receive Crubit bindings. Some types may only be usable in certain locations due to current Crubit limitations, inherent properties of the type, or both. Supported types fall into one of three categories ranging from “most widely supported” to “most restricted”:
- ABI-compatible: these types have a C-ABI-equivalent representation which can be used anywhere a value of this type is expected from both C++ and Rust.
- Layout-compatible: these types have equivalent in-memory representations
in C++ and Rust but cannot be represented using standard C ABI. These types
will only be usable as by-value function arguments if they are C++-movable.
For example,
Box<i32>is not C++-movable because it has nonullptr/ moved-from representation. - Bridged: these types may have different in-memory representations in C++
and Rust, and so can only be passed by-value between the two languages.
Examples include Rust tuples, which are bridged by-value into C++
std::tuple.
| Level of Support | Example | Pass by-reference | Pass by-value | Return by-value | Fields | In Function Pointer Types |
|---|---|---|---|---|---|---|
| ABI Compatible | i32 | Y | Y | Y | Y | Y |
| Layout-compatible C++ type | absl::string_view | Y | if Rust-movable1 | if Rust-movable2 | Y | N |
| Layout-compatible Rust type | UserDefinedStruct | Y | if C++ movable3 | Y | Y | N |
| Bridged | (i32, i32) | N | Y | Y | N | N |
NOTE: All primitive and pointer types are ABI-compatible. However, due to b/369895805, all non-bridged user-defined types are only layout-compatible.
In the following examples, foo receives bindings, but bad_foo will not
receive bindings, because while the types it uses in its function signature are
supported by Crubit, they are not supported in this particular context.
C++
void foo(int32_t);
void foo(void (*)(int32_t));
void foo(Status);
struct LayoutCompatibleType {
UnsupportedType field;
// or [[no_unique_address]] int field; or...
};
// bad_foo cannot receive bindings, because the function pointer type
// does not work with non-ABI-compatible types
void bad_foo(void (*)(LayoutCompatibleType));
Rust
#![allow(unused)]
fn main() {
pub fn foo1(_: i32) {}
pub fn foo2(_: fn(i32)) {}
pub fn foo3(_: Status) {}
}
#![allow(unused)]
fn main() {
struct LayoutCompatibleType {
field: UnsupportedType
}
// bad_foo cannot receive bindings, because the function pointer type
// does not work with non-ABI-compatible types
pub fn bad_foo(_: fn(LayoutCompatibleType)) {}
}
#![allow(unused)]
fn main() {
// bad_foo cannot receive bindings, because bridged types cannot be passed
// by reference
pub fn bad_foo(_: &(i32, i32)) {}
}
Incomplete types in generic parameters
In C++, forward-declared (incomplete) types can sometimes appear as template
arguments, such as in std::optional<Incomplete>,
std::unique_ptr<Incomplete>, std::shared_ptr<Incomplete>, or
absl::StatusOr<Incomplete>.
However, Crubit cannot generate bindings for layout-compatible generic types instantiated with an incomplete type:
- Unsized / extern types: In Rust, forward-declared C++ types are
represented as incomplete types (conceptually unsized or
extern types, such asforward_declare::Incomplete<...>) because their size and alignment are unknown. - Generic parameters require
Sized: Layout-compatible generic types (such asOption<T>,unique_ptr<T>,shared_ptr<T>, orStatusOr<T>) rely on the concrete in-memory layout, size, and alignment ofTto achieve layout compatibility between Rust and C++. In Rust, generic type parameters have an implicitSizedbound (T: Sized). An unsized orextern typecannot be used as a generic type argument in these containers.
To resolve this, include the C++ header providing the complete definition of the type so that Crubit has access to its size, alignment, and layout.
Bidirectionality
Usually, the mapping of types between languages is bidirectional. For example, a
C++ function which returns an int32_t will become a Rust function returning an
i32, and vice versa. In some sense, an i32 is an int32_t.
However, in other cases, the mapping is not reversible. C++ and Rust have types
or aliases that the other language does not. For example, isize becomes
intptr_t, but intptr_t is (on some platforms) the same type as int64_t,
and so intptr_t becomes i64.
-
See http://crubit.rs/cpp/classes_and_structs#trivially_relocatable ↩
-
See http://crubit.rs/cpp/classes_and_structs#trivially_relocatable ↩
Primitive types
Crubit maps primitive types1 to the direct equivalent in the
other language. For example, C++ int32_t is Rust i32, C++ int is Rust
ffi_11::c_int, C++ double is Rust f64, and so on.
Exceptions:
- C++: There is no mapping for the currently-unsupported types
nullptr_t,char8_t, andwchar_t. - Rust: There is no mapping for the never (
!) type except as a return type, andi128/u128when calling Rust from C++.
For more information, see Unsupported types
Bidirectional type mapping
The following map is bidirectional. If you call a C++ interface from Rust using
Crubit, then int32_t in C++ becomes i32 in Rust. Vice versa, if you call a
Rust interface from C++ using Crubit, i32 in Rust becomes int32_t in C++.
| C++ | Rust |
|---|---|
void | () as a return type, ::ffi_11::c_void otherwise. |
int8_t | i8 |
int16_t | i16 |
int32_t | i32 |
int64_t | i64 |
intptr_t | isize |
uint8_t | u8 |
uint16_t | u16 |
uint32_t | u32 |
uint64_t | u64 |
uintptr_t | usize |
bool | bool |
double | f64 |
float | f32 |
char | ::ffi_11::c_char 2 |
signed char | ::ffi_11::c_schar |
unsigned char | ::ffi_11::c_uchar |
short | ::ffi_11::c_short |
unsigned short | ::ffi_11::c_ushort |
int | ::ffi_11::c_int |
unsigned int | ::ffi_11::c_uint |
long | ::ffi_11::c_long |
unsigned long | ::ffi_11::c_ulong |
long long | ::ffi_11::c_longlong |
unsigned long long | ::ffi_11::c_ulonglong |
rs::char_ | char |
One-way type mapping
The types below are mapped in only one direction, but do not round trip back to
the original type. For example, size_t maps to usize, but usize maps to
uintptr_t.
C++ to Rust
The following C++ types become the following Rust types, but not vice versa:
One-way mapping of Rust to C++ types
The following Rust types become the following C++ types, but not vice versa:
| Rust | C++ |
|---|---|
! (return type) | void |
Unsupported types
Bindings for the following types are not supported at this point:
C++
nullptr_tandchar8_thave not yet been implemented.- b/283268558:
wchar_tis currently unsupported, for portability reasons.
Rust
- b/254507801:
!has not yet been implemented except for return types. - b/254094650:
i128andu128are not yet supported when calling Rust from C++.
-
Rust calls these types primitive types, while C++ calls them fundamental types. Since the Rust terminology is probably well understood by everybody, we use it here. ↩
-
Note that Rust
c_charand C++charhave different signedness in Google, or any other codebase with widespread use of unsignedcharin x86.TODO(jeanpierreda): document this in more detail. ↩
-
Unlike Rust
char,char16_tandchar32_tmay contain invalid Unicode characters. ↩ -
b/532400117. The reverse direction, Rust
i128/u128to C++, is not yet supported (b/254094650). ↩
Pointer types
C++ defines two categories of pointer types, while Rust adds a third. They are:
- Pointers to some (non-function) object, without lifetime information. C++ calls these object pointers, while Rust calls them raw pointers.
- Function pointers (C++, Rust).
- Finally, Rust references: non-aliasing pointers with lifetime information.
With the exception of Rust references, which are only
permitted in limited circumstances, pointer types are fully supported as long as
the type they point to is supported. For example, const int32_t* maps
bidirectionally to *const i32, and void (*)(int32_t) maps bidirectionally to
fn(i32).
Object pointers
An “object pointer” is the C++ terminology for any pointer that is not a function pointer. Rust would call these “raw pointers”. These are mapped to each other bidirectionally:
| C++ | Rust |
|---|---|
const T* | *const T |
T* | *mut T |
C++ pointers with lifetime
C++ allows attaching lifetime annotations to arbitrary types, including pointers. There are two competing annotations for this:
[[clang::lifetimebound]]: Crubit honors this on references (withassume_lifetimes, see Rust references).- Lifetime attributes: not supported.
Function pointers
C++ function pointers map to Rust extern "C" fn(...) -> ... function pointers,
and vice versa:
| C++ | Rust |
|---|---|
void(&)(int32_t) | extern "C" fn(i32) |
void(*)(int32_t) | Option<extern "C" fn(i32)> |
std::type_identity_t<void(int32_t)> | Not supported 1 |
If the corresponding C++ function definition would be unsafe in Rust (per the
rules for C++ function declarations), then so is
the function pointer – for example, a C++ reference to void(void*) becomes a
Rust unsafe extern "C" fn(_: *mut c_void).
Not all function pointers receive bindings. If the function cannot be called directly, due to a known or potential ABI mismatch between Rust and C++, then the function pointer receives no bindings. In particular, function pointers cannot take layout-compatible types by value. You can work around this by taking or returning such problematic types by pointer instead of by value.
Lifetime
All function pointers are 'static.
There is no way to specify the lifetime of a function pointer in Rust, nor in
C++: both assume a 'static lifetime. In scenarios where the lifetime may be
shorter than 'static (e.g., JIT compilation, or dynamic loading and unloading
of shared libraries at runtime), the developer is responsible for managing the
lifetime of the function pointer.
Rust references
Rust references, unlike C++ references, cannot mutably alias. This introduces a
new form of Undefined Behavior (UB) that many C++ programmers may not be
accustomed to. C++ pointers map to Rust raw pointers. C++ references map to Rust
references (in parameter position) or to cref::CRef/cref::CMut (in return
position) when assume_lifetimes is enabled, which it is by default under
//features:supported. Without it, C++ references map to raw
pointers. Vice versa, Rust references are an unsupported type which do not map
to any C++ type at all.
The one exception to this rule are function parameters. In some limited circumstances, Rust functions may accept references, and the corresponding C++ interface will accept C++ references. This is documented in crubit.rs/rust/functions.
-
C++ has plain function types: the type pointed to by function pointers. There is no Rust equivalent. However, since C++ functions implicitly coerce to function pointers, this only comes up in template classes like
std::functionorabsl::AnyInvocable. Or, in this case,type_identity_t. ↩
Callbacks: absl::AnyInvocable and Fn/FnMut/FnOnce
Crubit supports bidirectional function callbacks across C++ and Rust:
- Rust bindings for C++ callbacks
(
absl::AnyInvocable): Calling C++ APIs that take or returnabsl::AnyInvocablefrom Rust. - C++ bindings for Rust callbacks (
Fn,FnMut,FnOnce): Calling Rust APIs that accept closures from C++ viars::FnRefandrs::Fn.
Type mappings
C++ callbacks in Rust
| C++ type | Rust type | Semantics |
|---|---|---|
absl::AnyInvocable<R(...) const> | Box<dyn Fn(...) -> R + Send + Sync> | Multiple invocations |
absl::AnyInvocable<R(...) &&> | Box<dyn FnOnce(...) -> R + Send> | Single-use invocation |
absl::AnyInvocable<R(...)> (non-const) | Box<dyn Fn(...) -> R + Send + Sync> | Promoted to Fn |
Rust callbacks in C++
| C++ type | Rust type | Semantics |
|---|---|---|
rs::FnRef<R(...) const> | &dyn Fn(...) -> R / impl Fn(...) -> R | Borrowed, zero allocation |
rs::FnRef<R(...)> | &mut dyn FnMut(...) -> R / impl FnMut(...) -> R | Borrowed mutable, zero allocation |
rs::Fn<R(...) const> | Box<dyn Fn(...) -> R> | Owned |
rs::Fn<R(...)> | Box<dyn FnMut(...) -> R> | Owned mutable |
rs::Fn<R(...) &&> | Box<dyn FnOnce(...) -> R> / impl FnOnce(...) + 'static | Owned, single-use (std::move) |
rs::DynCallable<Sig> | (Rust callback in struct) | Implicitly converts to absl::AnyInvocable<Sig> |
Examples
Runnable, tested examples for both directions are in
examples/types/callbacks/.
absl::Status in Rust
In Google C++, the standard types for communicating an error are absl::Status
and absl::StatusOr<T>. In Rust, these are represented using Status and
StatusOr<T> from @abseil-cpp//absl/status, which are layout-compatible with
the corresponding C++ types. For example:
absl::Status Foo();
absl::StatusOr<int> Bar();
This becomes:
#![allow(unused)]
fn main() {
use status::{Status, StatusOr};
pub fn Foo() -> Status { ... }
pub fn Bar() -> StatusOr<i32> { ... }
}
Calling C++ APIs using Status
Rust code that uses the status types depends on
@abseil-cpp//absl/status:status_rust.
C++ functions returning Status/StatusOr return the corresponding Rust types:
absl::Status ReturnsStatus(bool ok);
absl::StatusOr<int> ReturnsStatusOrInt(bool ok);
absl::Status ReturnsStatusWithProto(
let status = cpp_api::ReturnsStatus(true);
let status = cpp_api::ReturnsStatus(false);
let status_or_int = cpp_api::ReturnsStatusOrInt(true);
let status_or_int = cpp_api::ReturnsStatusOrInt(false);
let status = cpp_api::ReturnsStatusWithProto(true, msg.as_mut());
let status = cpp_api::ReturnsStatusWithProto(false, msg.as_mut());
Calling Rust APIs using Status
Rust APIs can directly return Status or StatusOr<T> in public functions:
use status::{err, ok, Status, StatusOr};
pub fn returns_status(is_ok: bool) -> Status {
if is_ok {
ok(())
} else {
err(status::internal("Something went wrong, oh no!"))
}
}
pub fn returns_status_or_int(is_ok: bool) -> StatusOr<i32> {
if is_ok {
ok(42)
} else {
err(status::internal("Something went wrong, oh no!"))
}
}
cpp_api_from_rust will automatically generate C++ bindings returning
absl::Status and absl::StatusOr<T>:
EXPECT_OK(rust_api::returns_status(true));
EXPECT_FALSE(rust_api::returns_status(false).ok());
auto status_or_int = rust_api::returns_status_or_int(true);
EXPECT_FALSE(rust_api::returns_status_or_int(false).ok());
Working with Status in Rust
Construction and Conversion
To construct status instances in Rust, use status::{ok, err}:
#![allow(unused)]
fn main() {
use status::{err, ok, Status, StatusOr};
let success: Status = ok(());
let value: StatusOr<i32> = ok(42);
let failure: Status = err(status::internal("error message"));
}
Existing Rust Result types can be converted using
status::into_new_status(...).
Testing with Googletest
When using googletest alongside status::{ok, err}, import matchers with
aliases to avoid name collisions:
#![allow(unused)]
fn main() {
use googletest::matchers::{eq, err as is_err, ok as is_ok, status_is};
use status::StatusCodeError;
}
You can then assert on Status or StatusOr values:
#![allow(unused)]
fn main() {
expect_that!(result, is_ok(eq(&42)));
expect_that!(result, is_err(status_is(StatusCodeError::INTERNAL)));
}
Types inside StatusOr must be layout-compatible
StatusOr<T> is layout-compatible with C++ absl::StatusOr<T>, enabling
zero-cost passing across the FFI boundary. However, this layout compatibility
requires that the inner payload type T is also layout-compatible between
C++ and Rust.
Types that are bridged across the FFI boundary via runtime conversions (such as
Protocol Buffers, which are not layout-compatible between C++ and
Rust; see b/534900713, or std::string) do not share the same in-memory
representation between C++ and Rust. Because T is stored directly inside
StatusOr<T>, absl::StatusOr<T> cannot be used with these types.
As a result, functions that accept or return absl::StatusOr<MyProto> (or
c9::Co<absl::StatusOr<MyProto>>) cannot be bound directly by Crubit.
Workaround: Out-Parameters with Proto Views
Instead of returning absl::StatusOr<MyProto> directly by value across FFI, the
C++ API can return absl::Status and accept a mutable reference (MyProto&)
output parameter:
absl::Status ReturnsStatusWithProto(
bool ok, crubit::absl_status_example::ExampleProto& out);
In Rust, create an instance of the message and pass its mutable view
(.as_mut()):
#[gtest]
fn test_returns_status_with_proto() {
let mut msg = example_rust_proto::ExampleProto::new();
let status = cpp_api::ReturnsStatusWithProto(true, msg.as_mut());
expect_that!(status, is_ok(eq(&())));
expect_that!(msg.value(), eq(42));
let mut msg = example_rust_proto::ExampleProto::new();
let status = cpp_api::ReturnsStatusWithProto(false, msg.as_mut());
expect_that!(
status,
status_is(StatusCodeError::INTERNAL).with_message(eq("Something went wrong, oh no!"))
);
}
Migration and Future Evolution
Status and StatusOr<T> are layout-compatible with absl::Status and
absl::StatusOr<T>, enabling zero-cost passing across the FFI boundary as well
as use in struct fields, arrays, or behind pointers and references. Error
handling with ? is supported via the Try trait.
Deprecated API
Status and StatusOr were previously named NewStatus and NewStatusOr,
which remain as aliases; prefer Status and StatusOr in new code. The legacy
Result-based OldStatus and OldStatusOr are deprecated, as is
StatusWrapper; don’t use them in new code. CRUBIT_NEW_STATUS is no longer
needed and is ignored. Targets that have not migrated yet opt into the old
bindings with local_defines = ["CRUBIT_OLD_STATUS"].
Box types
Rust’s owning box types map to C++ class templates in rs:
| Rust | C++ | Header |
|---|---|---|
Box<T> | rs::Box<T> | crubit/support/rs/box.h |
Pin<Box<T>> | rs::PinBox<T> | crubit/support/rs/pin_box.h |
Box<[T]> | rs::SliceBox<T> | crubit/support/rs/slice_box.h |
Box<str> | rs::StrBox | crubit/support/rs/str_box.h |
These are ordinary C++ templates: T can be any complete C++ or Rust type.
Box<T> and PinBox<T> have the layout of a single non-null T*;
SliceBox<T> and StrBox have the layout of a Rust fat pointer (data pointer
and length), the same as rs::SliceRef<T> and
rs::StrRef.
The mapping is bidirectional: a Rust function taking or returning Box<T> gets
C++ bindings using rs::Box<T>, and a C++ function using rs::Box<T> gets Rust
bindings using Box<T>. The types can be used in fields, as parameters and
return values, and behind pointers and references.
Ownership and allocation
A box always owns memory allocated by the Rust global allocator, regardless
of which language created it. Use rs::make_box (or rs::make_pin_box) to
allocate and construct the pointee in place, analogous to std::make_unique:
#include "crubit/support/rs/box.h"
rs::Box<MyStruct> b = rs::make_box<MyStruct>(/*args...*/);
b->field = 1;
MyStruct& inner = *b;
// `~Box()` destroys `*b` and returns the memory to the Rust allocator.
rs::make_box and rs::make_pin_box also accept an rs::Movable<T> directly
(deducing T), so a non-C++-movable Rust value returned by value can be boxed
without spelling <T>:
auto b = rs::make_box(return_non_cpp_movable_rust_type());
b->method();
rs::Movable<rs::Box<T>> and rs::Movable<rs::PinBox<T>> are pointer-sized
partial specializations that forward operator->, operator*, and get()
directly to the pointee T (like std::unique_ptr<T>), so
auto b = rs::make_box<MyStruct>(...) and rs::make_box<MyStruct>(...)->field
do not require double-dereferencing. When an lvalue reference to the
rs::Box<T> or rs::PinBox<T> itself is needed (for example to call a Rust
fn(&Box<T>)), use b.as_box() or b.as_pin_box().
Convenience constructors Box(T&&), Box(const T&), Box(rs::Movable<T>), and
Box(std::in_place, /*args...*/) are also available. SliceBox<T> can be built
from a std::span<const T>, an initializer list, or a count and fill value.
StrBox follows the same UTF-8 conversion rules as rs::StrRef: its
constructor accepts rs::StrRef (including consteval-checked string literals
and constexpr std::string_views), while runtime strings use the explicit
fallible rs::StrBox::FromUtf8(sv) (returning
std::optional<rs::Movable<rs::StrBox>>) or
rs::StrBox::FromUtf8Unchecked(sv).
unsafe_from_raw adopts a pointer that was allocated with crubit_alloc with
the size and alignment of T; it is primarily for code that already interacts
with the allocator shims directly.
Not C++-movable
Rust boxes have no null or moved-from state, so the C++ types delete their copy and move constructors and assignment operators. They can still be:
-
returned by value, using C++17 guaranteed copy elision:
rs::Box<int> make() { return rs::make_box<int>(42); } -
destructively relocated via the
crubit::UnsafeRelocateTagconstructor, which is whatrs::Movable<T>and Crubit’s generated thunks use. -
passed by value to Rust through
rs::Movable<rs::Box<T>>, which is whatrs::make_boxandrs::make_pin_boxreturn and howcpp_api_from_rustspells by-valueBox<T>parameters:// Rust: pub fn consume(b: Box<i32>) consume(rs::make_box<int32_t>(42));
See Movable types for the general rules.
Inside Option
An Option<Box<T>> (as well as options of pinned or slice boxes) is represented
as rs::Option<rs::Box<T>>. Because these boxes use a non-null pointer
representation, the Option takes advantage of Rust’s niche layout
optimization: the rs::Option is pointer-sized and layout-compatible with
Option<Box<T>>, with the None state represented as a null pointer. The same
applies for other non-movable payloads that fall back to rs::Option instead of
std::optional.
Box<T> requires T to be Rust-movable
In Rust, the contents of a Box<T> can be moved out by value (for example with
*b), which relocates the T by copying its bytes. That is only sound if T
is Rust-movable, i.e. Unpin. A C++ type whose move constructor is non-trivial
or deleted is not Rust-movable: Rust bindings
for it are !Unpin, and relocating it by memcpy would be undefined behavior.
For that reason, rust_api_from_cpp refuses to generate bindings for a C++
API that uses rs::Box<T> or rs::SliceBox<T> with a T that is not
Rust-movable. Use rs::PinBox<T> instead: it maps to Pin<Box<T>>, which
guarantees that Rust never moves the pointee. In C++, PinBox<T> offers the
same get(), operator*, and operator-> accessors as Box<T>; pinning only
restricts what the Rust side may do.
// `NonTrivial` has a user-defined move constructor, so it is `!Unpin` in Rust.
rs::Box<NonTrivial> Rejected(); // error: use rs::PinBox<NonTrivial>
rs::PinBox<NonTrivial> Accepted(); // Rust: fn Accepted() -> Pin<Box<NonTrivial>>
cpp_api_from_rust does not need this restriction: a Rust Box<T> of a
!Unpin Rust type is still a valid Box<T>, and the C++ bindings for T are
not C++-movable, so C++ cannot move the pointee either.
For the background on how Crubit decides whether a type is Rust-movable, see Unpin and trivially relocatable types.
C++/Rust Protobuf interop
WARNING: This page documents functionality that is currently internal to the Google monorepo.
Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data. Once you define how you want your data to be structured once, you can generate source code in a variety of languages to manipulate and serialize/deserialize your structured data. Protobuf messages are among the most common types at Google, appearing in vast majority of APIs.
The usual way to passing data from one language to another using Protobufs is to serialize a message in one language, and deserialize it in another. This serialization/deserialization has costs which makes this approach unsuitable for hot code paths.
To avoid those costs, we’ve intentionally designed C++ and Rust Protobuf message types to have identical layouts. We avoid the need for serialization/deserialization and instead we directly use the same message object from both languages. Crubit automatically generates the zero-cost glue code for us. For example, take this piece of a C++ header:
MyProto Foo();
This becomes available to Rust as:
#![allow(unused)]
fn main() {
pub fn Foo() -> MyProto {...}
}
(Specifically, Crubit will detect that this is a Protobuf message, and it will convert from the C++ message type to the Rust message type.)
Calling Rust APIs using Protobuf message types
| Rust | C++ |
|---|---|
Message | Message |
MessageView | const Message* |
MessageMut | Message* |
&Message | const proto::Rust<Message>& |
*const Message | const proto::Rust<Message>* |
Message (as a struct field) | proto::Rust<Message> |
Vec<Message> | rs_std::Vec<proto::Rust<Message>> |
Protocol buffers are supported by value, and using the View and Mut view
types, where they are mapped to C++ pointers.
When an owned Rust Message appears in a layout-compatible position (such as a
public struct field, behind a reference or pointer, or inside a generic
container like Vec), it maps to proto::Rust<Message> in C++, a non-null
handle providing get(), operator->, and operator* access to the underlying
C++ Message.
Calling C++ APIs using Protobuf message types
Passing by value
| C++ | Rust |
|---|---|
Message | Message |
When a C++ proto message is passed or returned by value, it is mapped directly to the Rust message type, as you would expect.
Passing by reference
| C++ | Rust |
|---|---|
const Message& | MessageView |
Message& | MessageMut |
C++ protobuf references map directly to safe Rust MessageView and MessageMut
types. This relies on lifetime annotations (assume_lifetimes) and
//features:proto_references, both of which are enabled by
default, so no extra configuration is needed beyond the usual
rust_api_from_cpp setup:
load("//bazel:rust_api_from_cpp.bzl", "rust_api_from_cpp")
cc_library(
name = "my_cpp_lib",
hdrs = ["my_cpp_lib.h"],
aspect_hints = [":my_cpp_lib_rust.hint"],
deps = [":my_cc_proto"],
)
rust_api_from_cpp(
name = "my_cpp_lib_rust",
cpp_target = ":my_cpp_lib",
)
No cpp_coerce() (or deprecated cpp_cast()) or unsafe block is required
when calling C++ functions that take or return protobuf references:
#![allow(unused)]
fn main() {
// Pass an owned Rust proto using .as_view() / .as_mut(),
// or pass an existing MessageView / MessageMut directly:
my_cpp_lib::ProcessConstRef(my_proto.as_view());
my_cpp_lib::ProcessMutRef(my_proto.as_mut());
}
(If proto_references is not enabled on a target with assume_lifetimes, C++
const Message& maps to &Incomplete<symbol!("Message"), ...> and Message&
maps to Pin<&mut Incomplete<symbol!("Message"), ...>>. In that case, you
should enable proto_references on the target.)
Passing by pointer
| C++ | Rust |
|---|---|
const Message* | *const Incomplete<symbol!("Message"), ...> |
Message* | *mut Incomplete<symbol!("Message"), ...> |
When a C++ proto is passed or returned by raw pointer (or by reference on legacy
no_assume_lifetimes targets), the Rust type is a raw pointer to a forward
declaration (Incomplete<...>) of the C++ protocol buffer type. Raw pointers
cannot be mapped to MessageView or MessageMut because C++ pointers may be
null or lack lifetime guarantees, and the generated .proto.h header is not
processed directly by Crubit.
To convert between Rust protobuf view types and C++ Incomplete pointers, use
the CppCoerce (cpp_coerce()) and UnsafeCppCoerce (unsafe_cpp_coerce())
traits from forward_declare (which replace the deprecated cpp_cast() and
unsafe_cpp_cast() methods):
-
To convert a Rust
Proto(orProtoView) to a C++const Proto*: usemy_proto.as_view().cpp_coerce()(ormy_view.cpp_coerce()) -
To convert a Rust
Proto(orProtoMut) to a C++Proto*: usemy_proto.as_mut().cpp_coerce()(ormy_mut.cpp_coerce()) -
To convert a C++
(const) Proto*to a RustView/Mut: bind the pointer to a local variable and useunsafe { my_ptr.unsafe_cpp_coerce() }.
See support/forward_declare.rs for the definition of
Incomplete, CppCoerce, and UnsafeCppCoerce.
Quick guide: Non-Rust-Movable Types
This is a short introduction to Crubit’s handling of non-Rust-movable types, and how to use them in Rust. For a more thorough introduction, see crubit.rs/types/non_rust_movable/intro_advanced.
Many, if not most, types in C++ are not Rust-movable. That is, you cannot
initialize them using let x = y, you cannot assign them using x = y, and you
cannot otherwise relocate the bits in memory, such as by std::mem::swap.
In general, it’s better to make these types Rust-movable. See Rust-movable classes and Making types Rust-movable.
If the class cannot be made Rust-movable, then it will be given a less ergonomic
interface, using the Ctor trait.
What is the Ctor trait?
NOTE: The API documentation and source code is at support/ctor.rs
Ctor<Output=T, Error=E> is a trait for implementing lazily evaluated
values. These values are constructed in-place, in a C++-compatible way, and
pinned. It is typically used as an impl Ctor<Output=T, ...>, which is most
easily spelled as Ctor![T] or Ctor![T, Error=...].
Ctor is not a lazily-initialized value itself. It is a value initialization
procedure, which returns T upon success and E upon failure. So a Ctor
creates a value upon request, which is why we describe it as lazy.
Since exceptions are disabled at Google, we currently only work with
Error=Infallible, and for exposition will omit the error type.
Functions accepting and returning a non-Rust movable value in C++ will accept
and return an impl Ctor in Rust, as so:
#![allow(unused)]
fn main() {
pub fn accepts_value(x: Ctor![CppType]) {...}
// equivalent to x: impl Ctor<Output=CppType, Error=Infallible>
pub fn returns() -> Ctor![CppType] {...}
}
The easiest way to work with these types in Rust is to box them into a
Pin<Box<T>> using Box::emplace(). Then they can be passed by value using
mov!(x) (similar to std::move in C++), and by mutable reference using
.as_mut(). For example:
#![allow(unused)]
fn main() {
use ctor::{Emplace, mov}; // gives Box::emplace and mov!().
let mut x /* : Pin<Box<T>> */ = Box::emplace(returns());
takes_mutable_reference(x.as_mut());
takes_const_reference(&x);
accepts_value(mov!(x));
}
NOTE: The call to returns() does not construct a value of type CppType, it
merely returns a Ctor<Output=CppType>, which is a procedure to lazily create
such a value. A CppType is created only when the Ctor is executed as part of
the Box::emplace(...) expression in the caller.
If you happen to be directly passing a return value into a parameter, you can avoid the intermediate boxed value:
#![allow(unused)]
fn main() {
accepts_value(returns());
}
If you want to avoid heap allocations in general, you will need to use the more
advanced features of ctor.rs. These are described in more detail in the doc
comments, and in crubit.rs/types/non_rust_movable/intro_advanced.
Non-Rust-Movable Types
This is an in-depth exploration of Crubit’s support for non-Rust movable types. For a quick introduction, see crubit.rs/types/non_rust_movable/intro_short
Overview
ctor is an attempt at a satisfactory solution to representing
non-Rust-movable C++ objects in Rust. We cover how to store and manipulate
non-Rust-movable C++ objects as local variables (as well as temporaries), as
heap-allocated values, and as by-value struct data members.
Objective
If our C++ codebase were to be ported to Rust, it would be ideal if any C++ function or type had an equivalent Rust function or type which does the same thing: takes the same parameters, stores the same values, with the same performance characteristics and safety tradeoffs.
However, C++ has types which at first seem impossible to represent in Rust:
types which have nontrivial move/copy constructors, and can’t be relocated via
memcpy. For example, if a struct has a self-referential pointer, this is fine in
C++: the move constructor will re-point the pointer to its new target location,
and it will never be invalid. But in Rust, the built-in “move” operation (and
swap operation, and so on) does not run a user-provided move constructor, but
instead simply runs memcpy, performing a relocation. Since a self-referential
type is not Rust-movable, Rust code cannot be allowed to access it by value
like this.
The traditional approach to solving this is to box a non-Rust-movable value in a
heap-allocated pointer: the pointer is relocatable, even if its target is not.
If we forbid dereferencing the pointer in safe code (using Pin, described
below), then we’re good, and we’re back to
having basically normal Rust values.
ctor extends the approach to storing not just in a heap-allocated pointer, but
by value in all the usual places. This allows any C++ function or type to be
rewritten in equivalent Rust code, even if it deals with by-value
non-Rust-movable types.
Introduction: pinned pointers
In safe Rust code, we can expect to encounter a non-Rust-movable C++ type T
through the following three families of Rust types:
Pin<Box<T>>: equivalent to a non-null C++unique_ptr.- As well as
Pin<Arc<T>>(shared_ptr),Pin<Rc<T>>(non-atomicshared_ptr), etc.
- As well as
Pin<&mut T>: equivalent to a C++ non-const reference.&T: equivalent to a C++ const reference.
Safe Rust code cannot ever directly access a C++ value which is not
Rust-movable, because then it would be able to move it, causing UB! Instead, it
is safe only to deal indirectly with such values behind a reference. If the
reference is mutable, it must be wrapped behind a Pin, which makes it
impossible to access the underlying storage in safe code.
Unsafe code is allowed to access the underlying storage, and as part of its safety guarantee, it promises to do so without invalidating the object.
Accessing Pinned pointers
Unpin
If T is relocatable, Pin<&mut T> etc. is pointless: it isn’t protecting
against anything. Therefore, for such a type, Rust allows you to undo the
Pin in
safe code.
Immutable access
Pin<P> implements Deref<Target = P::Target>. In other words, for
non-mutating methods, a foo of type Pin<P> can be used just as if it were of
type P: you can just call foo.bar() either way. And to obtain a &T from a
Pin<&mut T> or Pin<Box<T>> and so on, you can use &*foo as normal.
Mutable access
Mutating methods are callable from a Pin<&mut T> if they have that type as
their receiver. For example, if std::vector were a non-Rust-movable type, its
clear method as seen from Rust would have the following method signature:
#![allow(unused)]
fn main() {
fn clear(self: Pin<&mut vector>) {
/* body omitted -- it uses the unsafe interface of Pin to access the underlying
storage. */
}
}
This is callable via my_ptr.as_mut().clear().
Why not my_ptr.clear() ? That works if my_ptr is of type Pin<&mut T>, but
since it takes self by value, my_ptr would no longer be usable after the
call. And that also doesn’t work for my_ptr : Pin<Box<T>>!
Most mutable accesses to pinned pointers like my_ptr are done via
my_ptr.as_mut() to reborrow the pointer. With &mut, the reborrowing is
implicit and part of the language, but Pin is a library type, and requires
doing it by hand. Callers will use .as_mut() frequently, as it’s required
whenever mutating and then continuing to use a pinned pointer.
Storage of pinned values
The Ctor trait
So as to make initialization composable, let’s start with a trait for
construction: Ctor. Ctor takes an uninitialized *mut T and initializes it,
promising as a postcondition that the memory has been fully initialized if no
error (such as a C++ exception) occurred.
#![allow(unused)]
fn main() {
pub trait Ctor {
type Output;
type Error;
unsafe fn ctor(self, dest: *mut Self::Output) -> Result<(), Self::Error>;
}
}
Ctor is a lazy value, akin to a prvalue in C++. Its main purpose is as the
building block for initialization so that we can write generic code, defer to
subroutines, etc. However, since it is lazy, it also allows for guaranteed copy
elision in much the same way as C++: as long as you keep forwarding a Ctor
around without calling ctor(), you are not yet performing any copies or moves.
Universal initialization
ctor has a blanket impl of Ctor<Output=Self, Error=Infallible> for every
Rust type except for those that specifically are designed to be a Ctor for
some other type. This makes Ctor nearly a perfect superset of safe
initialization. It is equivalent to a Rust move for Rust-movable values, but
also works for non-Rust-movable values.
#![allow(unused)]
fn main() {
pub auto trait SelfCtor {}
impl<T: SelfCtor> Ctor for T {
type Output = T;
type Error = Infallible;
unsafe fn ctor(self, mut dest: *mut T) -> Result<(), Self::Error> {
dest.write(self);
Ok(())
}
}
}
Code which intends to perform initialization of a value, and which permits the
use of non-Rust-movable values, can take a Ctor trait impl instead of a
concrete T, and work for both. For example, this is how the ctor! macro
below works.
Heap allocation
The easiest way to store a pinned value is to invoke a static method on Box
with the following signature:
#![allow(unused)]
fn main() {
fn emplace<C: Ctor<Error=Infallible>>(ctor: C) -> Pin<Box<C::Output>>;
}
The resulting pinned box is a completely ordinary Rust value, safe to relocate etc.
Just to give the reader a picture about how all of these are implemented, here
is what the source code for emplace would look like:
#![allow(unused)]
fn main() {
fn emplace<C: Ctor<Error=Infallible>>(ctor: C) -> Pin<Box<C::Output>> {
let mut uninit = Box::new(MaybeUninit::<C::Output>::uninit());
unsafe {
ctor.ctor(uninit.as_mut_ptr()).unwrap();
Pin::new_unchecked(uninit.assume_init())
}
}
}
The unsafe code here and inside of the Ctor::ctor definition is allowed to
unwrap the Pin, arbitrarily destructively mutate it, etc., but by the end of
it all, this eventually returns a pinned owned pointer to a fully initialized
value.
(This emplace function can be thought of as the Ctor-based dual of
Box::pin.)
Local variables
The second easiest way to store a pinned value is to store it as a local. A
common misconception is that this is impossible, but it’s actually nearly
identical to the Box case. Instead of creating a Box<T> and then wrapping
the Box in a Pin, we create a plain T and hide it, only exposing it via a
variable with type Pin<&mut T> that points back to the original value. This
makes that local variable impossible to move to a new location using a Rust
move, but of course, that was the whole point. If you want to move it to a new
location, you will need to do it some other way, such as via a move constructor,
which will be covered later.
This entire mechanism is hidden behind a macro (defined here), which can be used as so:
#![allow(unused)]
fn main() {
// Example use:
let x = emplace!(some_ctor);
// x is now a fully-initialized variable of type Pin<&mut T>:
let y: Pin<&mut T> = x;
}
(This emplace macro can be thought of as the Ctor-based dual of
std::pin.)
Struct fields
If you store the T as a Pin<Box<T>>, then that’s safe to store in any Rust
data structure, and the problem is solved. If, however, you want to store it
by value as a T, then this forces the parent struct to itself also be
non-Rust-movable. This propagates the problem to the parent struct, which itself
can be stored as a local, on the heap, or, recursively, in another struct.
Let’s start with the struct:
#![allow(unused)]
fn main() {
struct MyStruct {
normal_value: u32,
non_movable: T,
}
}
We would like to initialize non_movable in-place, and normal_value – well,
normal_value can be initialized however we like. This is just a handful of
steps:
- As a parameter, receive a pointer to an uninitialized
MyStruct– this allows us to initialize the struct anywhere we want to place it, whether in aBox, a local, or another struct. Indeed, this means we will be implementingCtor! - Obtain the pointers for
normal_valueandnon_movableso that we can initialize them in place. This can be done withaddr_of_mut!. - Initialize in place.
This is all wrapped up in a macro. The ctor! macro returns an impl Ctor
(step #1), the implementation of which looks up the different fields (step
#2), and initializes them using user-provided Ctors (#3). Since relocatable
types like u32 are their own Ctor, this looks much like normal
initialization, generalized to work in-place with custom Ctor implementations
for non-Rust-movable types:
#![allow(unused)]
fn main() {
let sub_ctor = …; // a Ctor<Output=T>
let my_struct = emplace!(ctor!(MyStruct {
normal_value: 0,
non_movable: sub_ctor,
}));
}
Due to the complexity of struct syntax, this macro is a bit involved to define, but its definition is here.
Return values and parameters
TL;DR: where in C++ it would be a prvalue, in Rust use a Ctor.
Functions often return by value, which in C++ will construct the object in a caller-determined location. Since non-Rust-movable values cannot be moved, and Rust returns are always moves, this means that non-Rust-movable values cannot be returned by value as normal. The tricks used for locals and temporaries won’t work either. Something else must be done.
To solve this, we use lazy function evaluation: every function which in C++
would accept or return a non-Rust-movable type by value, must in Rust, return an
impl Ctor<Output=T, Error=Infallible>. In effect, since return values and
arguments don’t have a known location yet, we defer initialization until the
location is known. The expectation is that the Ctor is immediately
invoked, to best match normal initialization behavior.
impl Ctor is so common that it’s given a convenience macro: Ctor![X] is
impl Ctor<Output=X, Error=Infallible>, and will be used for all examples.
So where in C++ we might write, for a non-Rust-movable type MyStruct, the
function MyStruct ReturnsByValue(…) { return MyStruct {normal_value: 0, … }; }, in Rust we would write:
#![allow(unused)]
fn main() {
fn returns_by_value(...) -> Ctor![MyStruct] {
ctor!(MyStruct { normal_value: 0, … })
}
}
And if wrapping such a function that was written in C++, we would return a
Ctor which does not invoke the C++ function until the Ctor itself is
invoked.
Similarly, for parameters:
#![allow(unused)]
fn main() {
fn accepts_by_ctor(x: Ctor![MyStruct]) { … }
// example use:
accepts_by_ctor(MyStruct::ctor_new(...));
}
The common thread here is that Ctor really looks like, and is used like, a C++
prvalue.
Object-compatibility
The cost of using Ctor![] for a parameter value is that a function doing so
can no longer be a method on a trait object. Since we do not use trait objects
anywhere in Crubit, that isn’t immediately a concern. In the case that user code
needs to pass non-Rust-movable objects into a dynamically dispatched trait
method, it can do so by reference, materializing the Ctor in the caller,
rather than passing by value.
Unions
Unsafe unions of nontrivial types were added in C++11, but there are currently
fewer than 1000 uses of them. All other unions are of
trivial types, which would get ported to Rust unions without using the Ctor
functionality described in this document.
Conclusion
We can initialize pinned, non-Rust-movable values in-place approximately
anywhere that we would expect to for Rust types, provided the storage can be
made non-Rust-movable-aware. By doing so, we propagate the constraints to where
we store it: if it is in a Box, we prevent direct access to the Box
contents. if it is in a local, we prevent that local from being assigned
elsewhere. And if it is in a struct, then that struct itself becomes
non-Rust-movable. If we are forced to store a value in something which requires
relocatability, we must heap-allocate the storage.
Moving and mutating
Given that we can safely initialize non-Rust-movable values in a pinned location, but cannot relocate them, it next falls on us to ask how to mutate them and move them around.
C++ invented all the solutions for this. We want to reimplement copy and move
constructors, and copy and move assignment. While we’re at it, we’ll do
overloaded constructors and operator= in general.
Though, before we can build up abstractions for mutating move, we first need to actually build support for mutation to begin with.
Mutating pinned values
The previous section constructed a T, and hid it behind a pinned pointer (e.g.
Pin<&mut T>). This prevents direct access to the underlying &mut T in safe
code. So how on earth does one mutate it, without access to a mutable reference?
The first answer is “unsafe code”. Indeed, if this is a C++ class, then the generated method implementations will look something like this:
#![allow(unused)]
fn main() {
impl MyType {
fn Mutate(self: Pin<&mut MyType>) {
unsafe {
// Pin::into_inner_unchecked() allows obtaining a real mutable reference in
// unsafe code. This could be mutated directly, in pure Rust, or passed to an
// FFI function like this, if wrapping C++.
_ZN6MyType6MutateEv(Pin::into_inner_unchecked(self))
}
}
}
}
When writing our own struct, we can use libraries that help safely mutate a
pinned struct. In particular, we introduce a macro which allows for
pin-projection, analogous to the
pin-project. For example, if we were to
wrap MyType into another struct by value, and wished to mutate the wrapped
type:
#![allow(unused)]
fn main() {
#[recursively_pinned]
struct MyWrapperStruct {
field_1: u32,
field_2: MyType,
}
impl MyWrapperStruct {
pub fn Mutate(self: Pin<&mut MyWrapperStruct>) {
let mut s = self.project_pin();
*s.field_1 += 1; // u32 is Unpin, can directly mutate it.
s.field_2.as_mut().Mutate(); // call into the Mutate function defined above.
}
}
}
Here, #[recursively_pinned] is allowing us to define a struct which can be
mutated while pinned. (We use our own macro, instead of pin-project, because
ctor!() requires all fields to be pinned. This macro guarantees that (it never
creates non-pinned fields, unlike pin-project) and implements the unsafe marker
trait RecursivelyPinned to advertise this fact to ctor!(). It also supports
negative impls of !Unpin, which pin-project does not currently support.)
So in summary: wrapped C++ types can have generated unsafe code which obtains
a raw pointer and passes it to C++ for mutation. Manually written
non-Rust-movable types can use a macro to access pinned fields without writing
unsafe code.
Rvalue References in Rust
The next thing we need is to represent a reference that is “intended to be moved from”, as opposed to ordinary mutable and immutable references.
#![allow(unused)]
fn main() {
// T&& equivalents:
pub struct RvalueReference<'a, T>(Pin<&'a mut T>); // has as_mut() -> Pin<&mut T>, get_ref() -> &T
// std::move()
macro_rules! mov {
($p:expr) => {
$crate::DerefRvalueReference::deref_rvalue_reference(&mut ($p))
};
}
// const T&& equivalents
pub struct ConstRvalueReference<'a, T>(pub &'a T); // has get_ref() -> &T
macro_rules! const_mov {
($p:expr) => {
$crate::ConstRvalueReference(&*{ $p })
};
}
}
(const T&& is not very important, as it is not really used except for overload
resolution tricks. However, since we would like to be able to represent any
C++ function in Rust, and some take const T&&, it is simpler not to omit it
entirely. If we do omit it, we need a different story for how to translate every
piece of existing code which uses the type.)
When moving a value behind a pinned pointer, we use the mov! macro: it
takes any pinned pointer type, e.g. Pin<Box<T>> or Pin<&mut T>, consumes it,
and evaluates to an RvalueReference with temporary lifetime which can be used
to trigger move construction and assignment.
By default, we should prefer to use pointers by value: mov!(x) makes x
inaccessible to Rust code thereafter, as it has been Rust-moved into a
temporary. Only if we specifically want to use a pointer after it has been moved
do we use mov!(x.as_mut()).
The const T&& APIs parallel the non-const APIs. They are rarely useful, but
are included for parity with C++.
Constructors and assignment operators
With these in place, we now have the tools in place to define the construction and assignment traits:
#![allow(unused)]
fn main() {
pub trait CtorNew<ConstructorArgs> {
type CtorType: Ctor<Output = Self, Error=Self::Error>;
type Error;
fn ctor_new(args: ConstructorArgs) -> Self::CtorType;
}
pub trait Assign<From> {
fn assign(self: Pin<&mut Self>, src: From);
}
}
Implicit conversion, moves, and copies
In general, this scheme does not support implicit conversion. All constructor calls must be explicit.
However, in the narrow case where a type U only exists in order to implicitly
convert to T, we can make that type itself implement Ctor<Output=T>., and
implement Assign<U> for T.
And so we have a rough solution for the two most common implicit conversions in C++, to try to make them a little more ergonomic:
moving:
#![allow(unused)]
fn main() {
let mut x = emplace!(mov!(y));
x.as_mut().assign(mov!(z));
}
RvalueReference<T> (returned by the mov! macro) implements Ctor<Output=T>,
for move-constructible T so that you can just let x = emplace!(mov!(y));
where in C++ you would auto x = std::move(y). Conversely, move-assignable T
implements Assign<RvalueReference<T>>, so that you can x.assign(mov!(y))
where in C++ you would x = std::move(y);.
copying:
#![allow(unused)]
fn main() {
let mut x = emplace!(copy(y));
x.as_mut().assign(copy(z)); // or assign(&*z)
}
Copy<P> (returned by the copy function) implements Ctor<Output=P::Target>,
for copy-constructible P::Target so that you can just let x = emplace!(copy(y)); where in C++ you would auto x = y. Conversely,
copy-assignable T implements Assign<Copy<P>> for all P: Deref<Target=T>,
so that you can x.assign(copy(y)) where in C++ you would x = y; – although
&*y works as well and is preferable.
(For copying, there are good reasons not to have let x = c; work. Also,
references are ordinary Rust values, and so they already implement Ctor for
themselves – remember, this is an extension of regular Rust initialization!)
Note that for both copying and moving, nothing happens until the actual call to
ctor or assign.
Example use
#![allow(unused)]
fn main() {
let original_value = emplace!(...);
let copied = emplace!(copy(&*original_value)); // or: MyType::ctor_new(&*original_value)
let mut moved = emplace!(mov!(original_value.as_mut())); // or: MyType::ctor_new(mov!(original_value.as_mut()))
// original_value is still available here, but moved-from
let moved_2 = emplace!(mov!(original_value));
// original_value is no longer available, since we used the form of mov that
// prohibits use-after-move.
// ERROR: let ... = something(&*original_value)
moved_2.assign(mov!(moved.as_mut()));
// as before, moved is still around, but moved-from
moved.assign(mov!(copied));
// this, otoh, used the preferred form of mov which prohibits use-after-move,
// and is no longer available.
// ERROR: something(&*copied)
}
Example definition of a copyable, movable type
The following definitions would be automatically generated for a C++ type, and
user-defined types should automatically derive them instead. Here is what the
automatic derivations might expand to, using ctor! and
#![feature(impl_trait_in_assoc_type)]
C++
Rust
MyType(const MyType& other) :
field_(other.field_) {}
#![allow(unused)]
fn main() {
impl CtorNew<&MyType> for MyType {
type CtorType = Ctor![MyType];
type Error = Infallible;
fn ctor_new(other: &Self) -> Ctor![MyType] {
ctor!(MyType {
field: FieldType::ctor_new(&other.field),
})
}
}
}
MyType(MyType&& other) :
field_(std::move(other.field_)) {}
#![allow(unused)]
fn main() {
impl CtorNew<RvalueReference<'_, Self>> for MyType {
type CtorType = Ctor![Self];
type Error = Infallible;
fn ctor_new(mut other: RvalueReference<'_, Self>) -> Ctor![Self] {
ctor!(MyType {
field: FieldType::ctor_new(mov!(other.as_mut().project_pin().field)),
})
}
}
}
MyType& operator=(const MyType& other) {
field_ = other.field_;
return *this;
}
#![allow(unused)]
fn main() {
impl Assign<&MyType> for MyType {
fn assign(self: Pin<&mut Self>, other: &Self) {
let this = self.project_pin();
this.field.assign(&other.field);
}
}
}
MyType& operator=(MyType&& other) {
field_ = std::move(other.field_);
return *this;
}
#![allow(unused)]
fn main() {
impl Assign<RvalueReference<'_, Self>> for MyType {
fn assign(self: Pin<&mut Self>,
mut other: RvalueReference<'_, Self>) {
let this = self.project_pin();
let other = other.as_mut().project_pin();
this.field.assign(mov!(other.field));
}
}
}
Reconstruction instead of operator=
Where in C++, temporary materialization is implicit, in Rust it is explicit. We can see every move, and wish it were elided. Well, why not elide it?
Instead of using operator=, we could use a reconstruct function, which
destroys the existing object, and constructs a new one in its place. This is
implemented
here.
#![allow(unused)]
fn main() {
let mut x = emplace!(MyType::ctor_new(0));
// constructs as a temporary, and then move-assigns:
x.assign(mov!(emplace!(MyType::ctor_new(42))));
// destroys, and then constructs in place:
unsafe {
reconstruct(x.as_mut(), MyType::ctor_new(100));
}
}
reconstruct is, in effect, a Ctor-based dual of
Pin::set.
However, this is unsafe, because the behavior is undefined for base class
subobjects and [[no_unique_address]] fields. This is not suitable for general
use, and should only be used when performance considerations demand eliding as
many moves as possible.
Conclusion
We’ve now explained and even implemented every operation in the table at the start of this doc.
Cheat Sheet: Non-Rust-Movable Types
Function parameter and return types
C++
Rust (Non-Rust-movable)
Rust (Rust-movable)
[const] T
#![allow(unused)]
fn main() {
Ctor![T]
}
#![allow(unused)]
fn main() {
T
}
T&&
#![allow(unused)]
fn main() {
RvalueReference<T>
}
#![allow(unused)]
fn main() {
&mut T
}
const T&&
#![allow(unused)]
fn main() {
ConstRvalueReference<T>
}
#![allow(unused)]
fn main() {
&T
}
T&
#![allow(unused)]
fn main() {
Pin<&mut T>
}
#![allow(unused)]
fn main() {
&mut T
}
const T&
#![allow(unused)]
fn main() {
&T
}
#![allow(unused)]
fn main() {
&T
}
Basic object creation and storage
C++
Rust (Non-Rust-movable)
Rust (Rust-movable)
auto x = T();
#![allow(unused)]
fn main() {
let x = emplace!(T::ctor_new(()));
}
#![allow(unused)]
fn main() {
let x = T::default();
}
auto x = y;
#![allow(unused)]
fn main() {
let x = emplace!(ctor::copy(&*y));
// sugar for:
let x = emplace!(
T::ctor_new(&*y));
}
#![allow(unused)]
fn main() {
let x = y.clone();
}
auto x = std::move(y);
#![allow(unused)]
fn main() {
let x = emplace!(ctor::mov!(y));
// sugar for:
let x = emplace!(
T::ctor_new(ctor::mov!(y)));
}
#![allow(unused)]
fn main() {
let x = y;
}
auto x = T(1, 2);
#![allow(unused)]
fn main() {
let x = emplace!(
T::ctor_new((1, 2)));
}
#![allow(unused)]
fn main() {
let x = T::new(1, 2);
}
// Takes T or const T&
TakesTemporary(T())
#![allow(unused)]
fn main() {
// if it takes a value:
TakesTemporary(T::ctor_new(()))
// if it takes a const reference:
TakesTemporary(
&*emplace!(T::ctor_new(())))
// (Materialization is explicit!)
}
#![allow(unused)]
fn main() {
// if it takes a value
TakesTemporary(T::default())
// if it takes a const reference:
TakesTemporary(&T::default())
}
// Takes T&&
TakesMutableTemporary(T())
#![allow(unused)]
fn main() {
TakesMutableTemporary(
ctor::mov!(emplace!(T::ctor_new(()))))
}
#![allow(unused)]
fn main() {
TakesMutableTemporary(
&mut T::default())
}
T Source() { return T(); }
#![allow(unused)]
fn main() {
fn Source() -> Ctor![T] {
// copy-elided and lazy.
// constructor won't run until
// the returned Ctor is "emplaced".
T::ctor_new(())
}
}
#![allow(unused)]
fn main() {
fn Source() -> T { T::default() }
}
T Source2() {
T x = T();
x.Mutate();
return x;
}
#![allow(unused)]
fn main() {
fn Source2() -> Ctor![T] {
T::ctor_new(()).ctor_then(|mut x| {
x.as_mut().Mutate();
Ok(())
})
}
}
#![allow(unused)]
fn main() {
fn Source2() -> T {
let mut x = T::default();
x.Mutate();
x
}
}
void Sink(T x) {
x.Something();
}
#![allow(unused)]
fn main() {
fn Sink(x: Ctor![T]) {
emplace!(x).Something()
}
}
#![allow(unused)]
fn main() {
fn Sink(x: T) {
x.Something()
}
}
Compound data types
C++
Rust (Non-Rust-movable)
Rust (Rust-movable)
struct T { U field; };
#![allow(unused)]
fn main() {
#[recursively_pinned]
struct T { field: U }
}
#![allow(unused)]
fn main() {
struct T { field: U }
}
std::make_unique<T>()
#![allow(unused)]
fn main() {
Box::emplace(T::ctor_new(()))
}
#![allow(unused)]
fn main() {
Box::new(T::default())
}
T x = {.field = std::move(y)}
#![allow(unused)]
fn main() {
let x = emplace!(ctor!(
T {field: ctor::mov!(y)}));
}
#![allow(unused)]
fn main() {
let x = T { field: y };
}
N/A: C++ doesn’t have tuple-structs.
#![allow(unused)]
fn main() {
#[recursively_pinned]
struct T2(U);
}
#![allow(unused)]
fn main() {
struct T2(U);
}
N/A: C++ doesn’t have tuple-structs.
#![allow(unused)]
fn main() {
let x = emplace!(ctor!(T2(ctor::mov!(y))));
}
#![allow(unused)]
fn main() {
let x = T2(y);
}
Object access and mutation
C++
Rust (Non-Rust-movable)
Rust (Rust-movable)
x.ConstMethod()
#![allow(unused)]
fn main() {
x.ConstMethod()
}
#![allow(unused)]
fn main() {
x.ConstMethod()
}
x.NonConstMethod()
#![allow(unused)]
fn main() {
x.as_mut().NonConstMethod()
}
#![allow(unused)]
fn main() {
x.NonConstMethod()
}
x = {};
#![allow(unused)]
fn main() {
x.as_mut().assign(T::ctor_new(()));
}
#![allow(unused)]
fn main() {
x = T::default();
}
x = y;
#![allow(unused)]
fn main() {
x.as_mut().assign(&*y);
}
#![allow(unused)]
fn main() {
x = y.clone();
}
x = std::move(y);
#![allow(unused)]
fn main() {
x.as_mut().assign(ctor::mov!(y));
}
#![allow(unused)]
fn main() {
x = y;
}
x = New(); // returns T by value
#![allow(unused)]
fn main() {
x.as_mut().assign(ctor::mov!(
emplace!(New())));
}
#![allow(unused)]
fn main() {
x = New();
}
x.~T();
new (&x) T();
#![allow(unused)]
fn main() {
unsafe { ctor::reconstruct(x.as_mut(),
T::ctor_new(())) };
}
#![allow(unused)]
fn main() {
x = T::default();
}
x.y = …; // x is a struct
#![allow(unused)]
fn main() {
x.as_mut().project_pin()
.y.assign(…);
}
#![allow(unused)]
fn main() {
x.y = …;
}
C++ bindings for Rust libraries
Rust libraries can be used directly from C++. This page documents roughly what that entails, and additional subpages (available in the left-hand navigation) document specific aspects of the generated bindings.
Tip: The code examples below are pulled straight from https://github.com/google/crubit/tree/main/examples/rust/function/. The other examples in https://github.com/google/crubit/tree/main/examples/rust/ are also useful. If you prefer just copy-pasting something, start there.
How to use Crubit
Crubit allows you to call some Rust interfaces from C++. It supports
functions (including methods), structs,
traits, and even enums as “opaque” objects. Crubit does
not support advanced features like generics or dynamic dispatch with dyn.
The rest of this document goes over how to create a Rust library that can be called from C++, and how to actually use it from C++. The quick summary is:
-
All
rust_librarytargets can receive C++ bindings. -
To use the bindings for a target
//path/to:example_crate, you must create a C++ rule exporting the bindings, usingcpp_api_from_rust(name="any_name_here", crate=":example_crate"). -
The header name is the Rust target’s label with a
.happended: to include the header for the Rust library//path/to:example_crate, you use#include "path/to/example_crate.h". -
The namespace name is the Rust target name, e.g.
example_crate. To change the namespace, usecc_bindings_from_rust_library_config, described below. -
To see the generated C++ API, right click the
"path/to/example_crate.h"include in Cider, and select “Go to Definition”.NOTE: In some cases the generated file in Cider may be out of date. If it isn’t refreshing, you can manually inspect the bindings using
bazel build //path/to:example_crate --config=crubit-genfiles. See Look at the generated bindings.
Write a rust_library target
The first part of creating a library that can be used by Crubit is to write a
rust_library target. For example:
pub fn add_two_integers(x: i32, y: i32) -> i32 {
x + y
}
In the BUILD file, in addition to defining the rust_library, you should also
define the cpp_api_from_rust target (loaded via
load("//bazel:cpp_api_from_rust.bzl", "cpp_api_from_rust"))
to make it easier to use from C++:
# This declares an "example_crate_cc_api" target that provides Crubit-generated
# C++ bindings for the Rust crate behind the `":example_crate"` target.
rust_library(
name = "example_crate",
srcs = ["example.rs"],
edition = "2024",
)
cpp_api_from_rust(
name = "example_crate_cc_api",
crate = ":example_crate",
)
Example: If your Rust library is named //path/to:example_crate, then the C++
header file is "path/to/example_crate.h", and the C++ namespace is
example_crate by default.
Use a Rust library from C++
C++ build rules do not have a rust_deps parameter, so to depend on the C++
bindings for a target, they must depend on the cpp_api_from_rust rule.
For example:
cc_binary(
name = "main",
srcs = ["main.cc"],
deps = [":example_crate_cc_api"],
)
#include <cstdint>
#include <iostream>
// The generated bindings are in a header at the same path as the
// `example_crate` rust_library, with a `.h` suffix.
#include "examples/rust/function/example_crate.h"
int main(int argc, char* argv[]) {
// The generated bindings are in a namespace with the same name as the
// target crate:
int32_t sum = example_crate::add_two_integers(2, 2);
std::cout << "sum = " << sum << std::endl;
return 0;
}
NOTE: Other than for declaring the dependency, all other information about the
generated bindings comes from the actual rust_library rule. For example, the
#include for the above is #include "examples/rust/function/example_crate.h", not
example_crate_cc_api.h.
(Optional) Customize the generated C++ API
Give it a better namespace
The crate name might make a poor namespace. In addition, typically, multiple C++
headers and build targets share the same namespace. To customize the namespace
name, use cc_bindings_from_rust_library_config:
# This configuration places all names inside the namespace `my::library`, instead of the crate name.
cc_bindings_from_rust_library_config(
name = "custom_namespace",
namespace = "my::library",
)
# This declares an "example_crate_cc_api" target that provides Crubit-generated
# C++ bindings for the Rust crate behind the `":example_crate"` target.
rust_library(
name = "example_crate",
srcs = ["example.rs"],
aspect_hints = [":custom_namespace"],
edition = "2024",
)
Now, instead of the crate name, the generated bindings will use the namespace name you provided:
#include <cstdint>
#include <iostream>
#include "examples/rust/library_config/example_crate.h"
int main(int argc, char* argv[]) {
// The generated bindings are in `my::library`, not `example_crate`.
int32_t sum = my::library::add_two_integers(2, 2);
std::cout << "sum = " << sum << std::endl;
return 0;
}
Look at the generated bindings
The generated C++ API for a rust_library is a header named after the
rust_library target, in the build output directory.
When something is missing
If Crubit can’t generate bindings for a Rust item, the build doesn’t fail by default. Crubit leaves the item out of the generated header and writes a comment in its place explaining why:
// Error generating bindings for function `tuples_golden::param_option_in_tuple`
// defined at
// path/to/tuples.rs;l=370:
// Error handling parameter #0 of type `(std::option::Option<i32>,)`:
// crubit.rs/errors/bridge_compound_type: Tuples containing bridged type
The first error you see is usually in the C++ code that calls the missing item, which doesn’t say why it’s missing. Search the generated header for the item’s name to find the reason. The reason often names a type rather than the function itself; fix or wrap that type, and the function gets bindings too.
To make a missing item fail the build instead, annotate it with
#[crubit_annotate::must_bind]. crubit_annotate is a procedural macro crate,
so add //support:crubit_annotate to your rust_library’s
proc_macro_deps (see Attributes to fine-tune generated
bindings).
Common Errors
Unsupported features
Some features are either unsupported, or else only supported with experimental feature flags . In order to get bindings for a Rust interface, that interface must only use the subset of features currently supported.
For a particularly notable example, references are supported as function parameters and return types, but only in a subset of cases that we can prove does not add aliasing UB to C++ callers.
The way to work around this kind of problem, in all cases, is to wrap or hide the problematic interface behind an interface Crubit can handle:
- Use raw pointers instead of references, if this use of references falls into a case Crubit does not support.
- Hide unsupported types behind a wrapper type. For example, a
HashMap<K, V>is not supported by Crubit, butpub struct MyStruct(HashMap<i32, i32>);is.
Building
This page is about calling Rust from C++ with cpp_api_from_rust. If you
want to call C++ from Rust, see Rust bindings for C++
libraries instead.
cpp_api_from_rust is available in these build systems:
| Build system | How | Example |
|---|---|---|
| Bazel | The cpp_api_from_rust rule from the rules_crubit module | examples/build_systems/bazel |
| CMake | corrosion_experimental_crubit() from Corrosion | examples/build_systems/cmake |
Each build system page above walks through the setup. Other build systems can
use the cargo cpp_api_from_rust subcommand, which writes a C++ header and a
static library for a Cargo package; see Build System
Integrations.
Quick start
The Crubit repository includes ready-to-run CMake and Bazel example projects
where a C++ main binary calls into a Rust library through bindings generated
by cpp_api_from_rust.
To run the examples on Linux x86_64, you need:
- CMake: CMake 3.22 or later, Ninja, Clang (which the example’s
CMakeLists.txtuses to compile the C++ binary), andrustup(which installs the Rust toolchain from Crubit’srust-toolchain.toml). - Bazel:
bazelisk(which downloads the Rust and Clang toolchains for you), plus arustcon yourPATH, whichcrubit_toolchainsuses to check LLVM version compatibility.
git clone https://github.com/google/crubit.git
cd crubit/examples/build_systems/cmake
cmake -B build -S . -G Ninja
cmake --build build
./build/main
Toolchains
cpp_api_from_rust works with nightly Rust toolchains and with the two most
recent stable releases, as long as the rustc-dev component is installed. It
generates portable C++20 that doesn’t depend on a particular C++ compiler. See
Toolchain requirements
for details.
Building cc_bindings_from_rs yourself
The Bazel and CMake integrations build Crubit for you. If you’re integrating
Crubit into another build system, see Cargo build of
Crubit for how to build the
cc_bindings_from_rs binary from source.
Attributes to fine-tune generated bindings
Crubit offers attributes to allow users to customize the functionality of the generated bindings, such as renaming a type or function or converting Rust values into a particular C++ type.
To use these attributes, add //support:crubit_annotate to
your rust_library’s proc_macro_deps (not deps, since crubit_annotate is
a procedural macro crate). Specific documentation for the attributes can be seen
in the docs for that library.
C++ bindings for Rust functions
C++ code can call functions defined in Rust, provided that the parameter and return types are supported by Crubit:
- If a parameter or return type is a fundamental type, then the bindings for the function use the corresponding C++ type.
- Similarly, if a parameter or return type is a pointer type, then the bindings for the function use the corresponding C++ pointer type.
- If the type is a user-defined type, such as a struct or enum, then the bindings for the function use the bindings for that type.
As a special case, functions also support reference parameters to supported types, with some restrictions to ensure safety. See References.
Example
Given the following Rust crate:
pub fn add_two_integers(x: i32, y: i32) -> i32 {
x + y
}
Crubit will generate the following function declaration, which calls into accompanying glue code:
::std::int32_t add_two_integers(::std::int32_t x, ::std::int32_t y);
inline ::std::int32_t add_two_integers(::std::int32_t x, ::std::int32_t y) {
return __crubit_internal::__crubit_thunk_add_utwo_uintegers(x, y);
}
unsafe functions
C++ does not have an unsafe marker at this time. In the future, Crubit may
introduce a way to mark unsafe functions to help increase the reliability of
C++ callers.
References
In general, Rust references are not exposed to C++. However, some Rust functions which accept reference parameters do get mapped to C++ functions accepting C++ references:
- All references must have an unbound parameter lifetime – not
'static, for example. - Only the parameter itself can be a reference type. References to references, vectors of references, etc. are still unsupported.
- If there is a
mutreference parameter, it is the only reference parameter.
This set of rules is intended to describe a safe subset of Rust functions, which do not introduce substantial aliasing risk to a mixed C++/Rust codebase.
For example, the following Rust functions will receive C++ bindings, and can be called from C++:
#![allow(unused)]
fn main() {
fn foo(&self) {}
fn foo(_: &mut i32) {}
fn foo(_: &i32, _: &i32) {}
}
However, none of the below will receive bindings:
#![allow(unused)]
fn main() {
fn foo(_: &'static i32) {} // 'static lifetime is bound
fn foo(_: &&i32) {} // Reference in non-parameter type
fn foo(_: &mut i32, _: &i32) {} // More than one reference, one of which is mut
fn foo(_: &'a i32) {} // 'a is not a lifetime parameter of `foo`
}
Returned references are mapped to C++ references (for self borrows) or C++
pointers, depending on the lifetime.
If you wish to accept more than one reference/pointer in C++, a raw pointer
(*const T, *mut T) can be used instead. However, all of the usual unsafe
caveats apply.
Slice parameters
Rust slice parameters (&[T], &mut [T], &str) map to rs::SliceRef<const T>, rs::SliceRef<T>, and rs::StrRef in C++.
C++ callers can pass absl::Span<const T>, std::span<const T>, or contiguous
containers like std::vector<T> and std::array<T, N> directly to satisfy
&[T] slice parameters with no conversion boilerplate:
#![allow(unused)]
fn main() {
fn process_text(_: &str) {} // C++ callers can pass "hello" or std::string_view
fn process_bytes(_: &[u8]) {} // C++ callers can pass absl::Span<const uint8_t>, std::span, "hello", or std::string_view
fn process_items(_: &[i32]) {} // C++ callers can pass absl::Span<const int32_t>, std::span, std::vector, etc.
}
However, mutable slices cannot be implicitly constructed from read-only views or string literals:
#![allow(unused)]
fn main() {
fn process_mut_bytes(_: &mut [u8]) {} // Cannot pass string literals or const spans to mutable byte slices
}
&[T]: Maps tors::SliceRef<const T>(crubit/support/rs_std/slice_ref.h). Converts implicitly fromabsl::Span<const T>,absl::Span<T>,std::span<const T>,std::span<T>,std::vector<T>, andstd::array<T, N>.&mut [T]: Maps tors::SliceRef<T>(crubit/support/rs_std/slice_ref.h). Converts implicitly from mutablestd::span<T>andabsl::Span<T>, or construct viars::SliceRef(ptr, size).&str: Maps tors::StrRef(crubit/support/rs_std/str_ref.h). Converts from string literals andstd::string_view.&[u8]: Maps tors::SliceRef<const uint8_t>. In addition toabsl::Spanandstd::span, also converts from string literals andstd::string_view.
Generic functions
Crubit doesn’t support generating bindings for arbitrary generic functions. Nevertheless, a small set of generic type parameters is supported by using canonical substitutions which are defined as follows:
| Generic constraints | Canonical substitution |
|---|---|
T: AsMut<U> | &mut U |
T: AsRef<U> | &U |
T: Into<U> | U |
F: Fn(Args...) -> R | rs::FnRef<R(Args...) const> |
F: FnMut(Args...) -> R | rs::FnRef<R(Args...)> |
F: FnOnce(Args...) -> R + 'static | rs::Fn<R(Args...) &&> |
Callbacks (Fn, FnMut, FnOnce)
Rust functions can accept callbacks as borrowed trait objects (&dyn Fn, &mut dyn FnMut), owned boxes (Box<dyn Fn>, Box<dyn FnMut>, Box<dyn FnOnce>),
or generic impl Trait parameters:
&dyn Fn(...)/impl Fn(...): Maps tors::FnRef<... const>(borrowed, no allocation).&mut dyn FnMut(...)/impl FnMut(...): Maps tors::FnRef<...>(borrowed, no allocation).impl FnOnce(...)(without'static): Maps tors::FnRef<...>(borrowed, no allocation).Box<dyn Fn(...)>: Maps tors::Fn<... const>(owned).Box<dyn FnMut(...)>: Maps tors::Fn<...>(owned mutable).Box<dyn FnOnce(...)>/impl FnOnce(...) + 'static: Maps tors::Fn<... &&>(owned, single-use).
C++ callers can satisfy these parameters by passing C++ lambdas, function
pointers, or absl::AnyInvocable. See Callbacks for full
details.
Callbacks: Fn, FnMut, FnOnce
Crubit supports exporting Rust functions that take closures (Fn, FnMut,
FnOnce) to C++ via cpp_api_from_rust.
In C++, callable parameters are represented using rs::FnRef (for borrowed
callbacks) and rs::Fn (for owned callbacks). C++ callers can satisfy these
parameters using C++ lambdas, function pointers, std::function, or
absl::AnyInvocable.
For the reverse direction (calling C++ APIs taking absl::AnyInvocable from
Rust), see Rust bindings for C++ callbacks.
BUILD configuration
When generating C++ bindings for a Rust library via cpp_api_from_rust:
rust_library(
name = "rust_api",
srcs = ["rust_api.rs"],
)
cpp_api_from_rust(
name = "rust_api_cc",
crate = ":rust_api",
)
cc_test(
name = "user_of_rust_api",
srcs = ["user_of_rust_api.cc"],
deps = [
":rust_api_cc",
"//testing/base/public:gunit_main",
],
)
The complete, tested version is in
examples/types/callbacks/BUILD.
Type mapping
Rust functions can take callable parameters as borrowed references, owned boxes,
or generic impl Trait parameters:
| Rust parameter type | Generated C++ parameter type | Ownership / Allocation |
|---|---|---|
&dyn Fn(Args...) -> R | rs::FnRef<R(Args...) const> | Borrowed (no allocation) |
&mut dyn FnMut(Args...) -> R | rs::FnRef<R(Args...)> | Borrowed (no allocation) |
impl Fn(Args...) -> R | rs::FnRef<R(Args...) const> | Borrowed (no allocation) |
impl FnMut(Args...) -> R | rs::FnRef<R(Args...)> | Borrowed (no allocation) |
impl FnOnce(Args...) -> R (without 'static) | rs::FnRef<R(Args...)> | Borrowed (no allocation) |
Box<dyn Fn(Args...) -> R> | rs::Fn<R(Args...) const> | Owning (heap-allocated) |
Box<dyn FnMut(Args...) -> R> | rs::Fn<R(Args...)> | Owning (heap-allocated) |
Box<dyn FnOnce(Args...) -> R> | rs::Fn<R(Args...) &&> | Owning (heap-allocated) |
impl FnOnce(Args...) -> R + 'static | rs::Fn<R(Args...) &&> | Owning (heap-allocated) |
The generated headers include the necessary support headers automatically:
rs::FnRef:#include "support/rs_std/fn_ref.h"rs::Fn:#include "support/rs_std/fn.h"
Passing callbacks from C++ to Rust
Given a Rust library defining functions that accept closures:
#![allow(unused)]
fn main() {
// In rust_api.rs:
// Borrowed reference: zero allocation
pub fn apply_fn(f: impl Fn(i32) -> i32, x: i32) -> i32 {
f(x)
}
// Mutable borrowed reference: zero allocation
pub fn apply_fn_mut(mut f: impl FnMut(i32) -> i32, x: i32) -> i32 {
f(x)
}
// Owned callback: invoked once
pub fn apply_fn_once(f: Box<dyn FnOnce(i32) -> i32>, x: i32) -> i32 {
f(x)
}
}
C++ callers can pass C++ lambdas, function pointers, std::function, or
absl::AnyInvocable:
#include "examples/types/callbacks/rust_api.h"
// 1. Pass a C++ lambda to an `impl Fn` parameter:
int32_t res1 = rust_api::apply_fn([](int32_t x) { return x * 2; }, 21);
// res1 == 42
// 2. Pass an absl::AnyInvocable:
absl::AnyInvocable<int32_t(int32_t) const> cb = [](int32_t x) { return x * 3; };
int32_t res2 = rust_api::apply_fn(cb, 14);
// res2 == 42
// 3. Pass a mutable lambda to `impl FnMut`:
int32_t total = 0;
auto mutable_lambda = [&total](int32_t x) mutable {
total += x;
return total;
};
rust_api::apply_fn_mut(mutable_lambda, 10);
rust_api::apply_fn_mut(mutable_lambda, 32);
// total == 42
// 4. Pass a move-only lambda to `Box<dyn FnOnce>`:
auto ptr = std::make_unique<int32_t>(10);
int32_t res3 = rust_api::apply_fn_once(
[p = std::move(ptr)](int32_t x) { return x + *p; }, 32);
// res3 == 42
rs::FnRef vs. rs::Fn
rs::FnRef(borrowed): Non-owning view of a callable, similar toabsl::FunctionRef. It stores a pointer to the callable and an invoker trampoline. It performs no heap allocation and must not outlive the function call.rs::Fn(owned): Owning callable wrapper. It heap-allocates the provided callable and manages its destruction, making it suitable for storing in Rust structs or transferring ownership across threads.rs::Fn<Sig &&>indicates a callable that can only be invoked once; call it usingstd::move(f)(...).
Converting Rust callbacks to absl::AnyInvocable (rs::DynCallable)
When C++ code receives an owned Rust callback (for example, stored in a Rust
struct field exposed to C++), the C++ type is rs::DynCallable<Sig>
(support/rs_std/dyn_callable.h).
rs::DynCallable is a move-only functional wrapper around Rust’s Box<dyn Trait>. It is implicitly convertible to absl::AnyInvocable:
rs::DynCallable<void() &&> rust_callback = ...;
absl::AnyInvocable<void() &&> any_invocable = std::move(rust_callback);
Because rs::DynCallable uses an internal representation compatible with
absl::AnyInvocable, this conversion is efficient and avoids re-wrapping or
additional heap allocations.
C++ bindings for Rust structs
A Rust struct is mapped to a C++ class/struct with the same fields. If any
field cannot be represented in C++, the struct itself will still have bindings,
but the relevant field will be private.
If a struct is not C++ movable, then it will be passed
by-value via the rs::Movable wrapper type.
See Movable Types.
Example
Given the following Rust module:
#[derive(Default, Clone)]
pub struct Struct {
pub a: i32,
}
Crubit will generate the following bindings:
struct CRUBIT_INTERNAL_RUST_TYPE(":: example_crate_golden :: Struct") alignas(4)
[[clang::trivial_abi]] Struct final {
public:
::std::int32_t a{};
private:
static void __crubit_field_offset_assertions();
};
Fields
The fields on the C++ class are the corresponding Rust types:
- If the Rust field has primitive type, then the C++ field uses the corresponding C++ type.
- Similarly, if the Rust field has pointer type, then the C++ field has the corresponding C++ pointer type.
- If the field has a user-defined type, such as a struct or enum, then the C++ field uses the bindings for that type.
Unsupported fields
Fields that do not receive bindings are made private, and replaced with an
opaque blob of maybe-uninitialized bytes, as well as a comment in the generated
source code explaining why the field could not receive bindings. For example,
since HashMap is not supported, the space of the object occupied by a
HashMap field will instead be this opaque blob of bytes:
#![allow(unused)]
fn main() {
// Rust: `my_field` is some unsupported type, such as `HashMap`
pub my_field: HashMap<i32, i32>,
}
// C++: `my_field` becomes `private`, and its type is replaced by bytes.
private: ::std::array<unsigned char, 24> my_field;
Specifically, the following subobjects are hidden and replaced with opaque blobs:
- Non-public fields (
privateorpub(...)fields). - Fields whose type does not have bindings.
- Fields that have an unrecognized or unsupported attribute.
Aggregate initialization
Qualifying Rust structs are generated as C++ aggregates (satisfying
std::is_aggregate_v<T>),
which enables C++ aggregate initialization such as designated initializers
(MyStruct{.a = 1, .b = 2}) and braced initialization (MyStruct{1, 2}).
A Rust struct is generated as a C++ aggregate if:
- All of its fields are public (
pub) and supported by Crubit. - It is not marked
#[non_exhaustive]. - It does not implement
Drop. - At most one field requires drop glue, or the struct is annotated with
#[crubit_annotate::field_drop_order_does_not_matter]. - Any field requiring drop glue is movable in C++.
Defaultis not implemented manually (i.e.Defaultis either derived via#[derive(Default)]or not implemented at all).
Field Drop Order and #[crubit_annotate::field_drop_order_does_not_matter]
In Rust, struct fields are dropped in definition order (first to last). In C++, aggregate members are destroyed in reverse definition order (last to first).
When at most one field requires drop glue, destruction order is trivial and matches between Rust and C++. However, when multiple fields require drop glue, the drop order in C++ will be the reverse of Rust’s drop order. To prevent unintended semantic differences, Crubit generates such structs as non-aggregates (with custom C++ destructors calling into Rust) by default.
If the drop order of the fields does not matter, you can annotate the struct
with #[crubit_annotate::field_drop_order_does_not_matter] from
//support:crubit_annotate to opt into C++ aggregate
generation:
#![allow(unused)]
fn main() {
use crubit_annotate::field_drop_order_does_not_matter;
#[field_drop_order_does_not_matter]
pub struct Config {
pub name: MyDropType1,
pub buffer: MyDropType2,
}
}
Manual Default Implementation
If a struct implements Default manually (impl Default for MyStruct { ... }),
it is generated as a non-aggregate. This allows Crubit to generate an explicit
C++ default constructor (MyStruct()) that calls Rust’s Default::default(),
preserving any custom initial values defined in Rust.
Structs using #[derive(Default)] remain C++ aggregates, as derived Default
initializes primitive fields to zero/false, matching C++ value initialization.
Differences between Aggregate and Non-Aggregate Structs
- Direct Member Variables: C++ aggregates expose fields as direct member
variables (
Type field_name;) without anonymous union wrappers or padding fields. - No User-Declared Constructors: C++ aggregates do not have user-declared constructors (such as converting constructors or tuple constructors).
- Value Initialization vs
Default::default(): In C++, value-initializing an aggregate (MyStruct s{};) zero-initializes primitive fields according to C++ rules. If a struct derivesDefaultor does not implementDefault, zero-initialization in C++ matches Rust derived default semantics.
C++ movable
If a struct is not C++ movable, its C++ move constructor
and move assignment operator will be = deleted, and it will be passed by-value
via the rs::Movable wrapper type. See
Movable Types.
Zero-sized types (ZSTs)
C++ does not allow 0-byte types, so Crubit cannot generate bindings for
zero-sized Rust structs (such as pub struct Empty;, pub struct Empty {}, or
structs containing only zero-sized fields) and emits a Zero-sized types (ZSTs) are not supported (b/258259459) comment instead. To expose an empty Rust struct
to C++, add a placeholder non-zero-sized field (for example, _unused: u8,
_unused: bool, or _unused: Option<()>).
CRUBIT_INTERNAL_RUST_TYPE annotation
You may notice that the generated C++ structs are annotated with the
CRUBIT_INTERNAL_RUST_TYPE macro. This annotation instructs Crubit
(specifically rs_bindings_from_cc) to disable automated bindings for this C++
type, and instead map all C++ uses of the type back to the existing Rust type.
This ensures that a Rust struct passed to C++ and then back to Rust resolves to
the original Rust type rather than a newly generated one.
While Crubit generates this annotation automatically for Rust-to-C++ bindings, you can also apply it manually on your own C++ types if you want them to map to an existing Rust type:
struct CRUBIT_INTERNAL_RUST_TYPE("char") char_ {
std::uint32_t c;
};
Template Arguments and Interpolation
For C++ templates, you can use {} interpolation syntax within the Rust type
string to substitute template arguments:
template <typename T>
struct CRUBIT_INTERNAL_RUST_TYPE("RustType<{}>", T) CppType {
T* value;
};
This ensures that a C++ instantiation like CppType<int> maps correctly to
RustType<i32> in Rust.
Importantly, this interpolation syntax allows you to express Rust generic parameters that have no direct C++ equivalent, such as lifetimes or default generic arguments. For example:
template <typename T>
struct CRUBIT_INTERNAL_RUST_TYPE("RustType<'static, {}>", T) CppType {
T* value;
};
Const generics arguments can also be provided with
crubit::rust_type::Const<N>:
template <typename T>
struct CRUBIT_INTERNAL_RUST_TYPE(
"RustType<'static, {}, {}>",
T,
crubit::rust_type::Const<123>,
) CppType {
T* value;
};
C++ bindings for Rust enums
A Rust enum is mapped to an opaque C++ type.
If an enum is not C++ movable, then it will be passed
by-value via the rs::Movable wrapper type.
See Movable Types.
Example
Given the following Rust crate:
#[derive(Copy, Clone, Default)]
pub enum Color {
#[default]
Red,
Blue,
Green,
}
Crubit will generate the following bindings:
struct CRUBIT_INTERNAL_RUST_TYPE(":: example_crate_golden :: Color") alignas(1)
[[clang::trivial_abi]] Color final {
public:
// Default::default
Color();
static constexpr Color MakeRed();
static constexpr Color MakeBlue();
static constexpr Color MakeGreen();
// No custom `Drop` impl and no custom "drop glue" required
~Color() = default;
Color(Color&&) = default;
Color& operator=(Color&&) = default;
// Rust types that are `Copy` get trivial, `default` C++ copy constructor and
// assignment operator.
Color(const Color&) = default;
Color& operator=(const Color&) = default;
Color(::crubit::UnsafeRelocateTag, Color&& value);
private:
// Field type has been replaced with a blob of bytes: No support for bindings
// of individual non-repr(C) `enum`s
::std::array<unsigned char, 1> __opaque_blob_of_bytes;
private:
struct PrivateBytesTag {};
constexpr Color(PrivateBytesTag, ::std::array<unsigned char, 1> bytes)
: __opaque_blob_of_bytes(bytes) {}
private:
static void __crubit_field_offset_assertions();
};
Why isn’t it a C++ enum?
A repr(i32) or fieldless repr(C) enum is very similar to a C++ enum.
However, Rust enums are exhaustive: any value not explicitly listed in the
enum declaration does not exist, and it is undefined
behavior
to attempt to create one.
C++ enums, in contrast, are “non-exhaustive”: a C++ enum can have any
value supported by the underlying type, even one not listed in the enumerators.
For example, if the above example were a C++ enum, static_cast<Color>(42)
would be a valid instance of Color, even though neither Red, Blue, nor
Green have that value.
In order to prevent invalid Rust values from being produced by C++, a C++ enum
cannot be used to represent a Rust enum. Instead, the C++ bindings are a
struct, even for fieldless enums.
C++ movable
If an enum is not C++ movable, its C++ move constructor
and move assignment operator will be = deleted, and it will be passed by-value
via the rs::Movable wrapper type. See
Movable Types.
Enums with payload
Each variant of a Rust enum may contain an additional payload (a tuple or a
struct). C++ bindings for Rust enums provide the following ways of working
with an enum payload:
-
Constructing an
enumvariant with the given payload by calling astaticMake<variant name>method (one such method is injected for each ofenumvariants). The following bugs track future work in this area:- b/487357254: Constructing variants with a struct payload
-
TODO(b/262737383): Matching
enumvariants and inspecting their payload.
Example
Given the following Rust crate:
#[derive(Copy, Clone)]
pub enum Color {
/// A completely transparent color (no payload)
Transparent,
/// A grayscale value from 0 to 255
Grayscale(u8),
/// Red, Green, and Blue values from 0 to 255
Rgb(u8, u8, u8),
}
Crubit will generate the following bindings:
struct CRUBIT_INTERNAL_RUST_TYPE(":: example_crate_golden :: Color") alignas(1)
[[clang::trivial_abi]] Color final {
public:
// `example_crate_golden::Color` doesn't implement the `Default` trait
Color() = delete;
// A completely transparent color (no payload)
static constexpr Color MakeTransparent();
// A grayscale value from 0 to 255
static ::example_crate::Color MakeGrayscale(::std::uint8_t __param_0);
// Red, Green, and Blue values from 0 to 255
static ::example_crate::Color MakeRgb(::std::uint8_t __param_0,
::std::uint8_t __param_1,
::std::uint8_t __param_2);
// No custom `Drop` impl and no custom "drop glue" required
~Color() = default;
Color(Color&&) = default;
Color& operator=(Color&&) = default;
// Rust types that are `Copy` get trivial, `default` C++ copy constructor and
// assignment operator.
Color(const Color&) = default;
Color& operator=(const Color&) = default;
Color(::crubit::UnsafeRelocateTag, Color&& value);
private:
// Field type has been replaced with a blob of bytes: No support for bindings
// of individual non-repr(C) `enum`s
::std::array<unsigned char, 4> __opaque_blob_of_bytes;
private:
struct PrivateBytesTag {};
constexpr Color(PrivateBytesTag, ::std::array<unsigned char, 4> bytes)
: __opaque_blob_of_bytes(bytes) {}
private:
static void __crubit_field_offset_assertions();
};
Generating C++ enums from Rust enums
By default, a Rust enum is mapped to an opaque C++ type (see C++ bindings for
Rust enums). However, Crubit can try to map Rust enums to C++
enums if requested using the #[crubit_annotate::cpp_enum(kind = "enum")] or
#[crubit_annotate::cpp_enum(kind = "enum class")] attribute. C++ code can use
such enums like any other C++ enum.
But #[cpp_enum] cannot be used with exhaustive Rust enums. It may only be
used on non-exhaustive enums, such as those created with #[open_enum] from the
open_enum crate. Therefore, to generate C++ enum bindings, you must annotate
your Rust enum with #[cpp_enum(kind = ...)], #[repr(...)] (where ... is an
integer type like i32), and #[open_enum].
C++ enums are non-exhaustive by default, meaning they can hold values other than
the explicitly named enumerators. #[open_enum] generates a Rust enum that is
similarly non-exhaustive. Additionally, C++ allows multiple enumerators to have
the same value, which can be enabled in Rust by using
#[open_enum(allow_alias)].
Example
Given the following Rust crate that uses #[cpp_enum] and
#[open_enum(allow_alias)]:
#[open_enum(allow_alias)]
#[cpp_enum(kind = "enum class")]
#[repr(i32)]
#[derive(Copy, Clone, Default)]
pub enum Color {
Red,
Blue,
Green = 5,
Gray = 5,
Magenta = 7,
}
Crubit will generate the following bindings:
// CRUBIT_ANNOTATE: cpp_enum=enum class
enum class CRUBIT_INTERNAL_RUST_TYPE(
":: example_crate_golden :: Color") Color : ::std::int32_t {
Red = INT32_C(0),
Blue = INT32_C(1),
Green = INT32_C(5),
Gray = INT32_C(5),
Magenta = INT32_C(7),
};
C++ bindings for Rust traits
Crubit allows C++ callers to invoke Rust trait implementations. For each trait SomeTrait, Crubit generates a matching C++ struct SomeTrait. To call a trait
method SomeTrait::some_fn implemented by SomeType, C++ callers can use
SomeTrait::impl<SomeType>::some_fn(args...).
Example
Given the following Rust crate:
pub trait MyTrait {
fn add_with(&self, y: i32) -> i32;
fn describe(&self) -> &'static str;
}
#[derive(Clone, Copy, Default)]
pub struct MyStruct {
x: i32,
}
impl MyStruct {
pub fn new(x: i32) -> Self {
Self { x }
}
}
impl MyTrait for MyStruct {
fn add_with(&self, y: i32) -> i32 {
self.x + y
}
fn describe(&self) -> &'static str {
"MyStruct"
}
}
You can call the trait from C++ using the following code:
int main(int argc, char* argv[]) {
// The generated bindings are in a namespace with the same name as the
// target crate:
example_crate::MyStruct s = example_crate::MyStruct::new_(2);
uint32_t result =
example_crate::MyTrait::impl<example_crate::MyStruct>::add_with(s, 3);
rs::StrRef description =
example_crate::MyTrait::impl<example_crate::MyStruct>::describe(s);
std::cout << "Result: " << result
<< "\ndescription: " << description.to_string_view() << std::endl;
return 0;
}
Each trait has an associated impl member which is generic upon the Self type
of the trait implementation, and which provides all of the associated types,
consts, and functions.
Checking if a trait is implemented
C++ users can check if a trait implementation is available using rs::where_v<T, SomeTrait>. This is useful when writing templated C++ functions, as it can be
used with requires (or enable_if) to specify that a particular trait is
implemented by a type parameter. For example:
template <typename T>
requires(rs::where_v<T, example_crate::MyTrait>)
uint32_t add_with_2(T const& self) {
return example_crate::MyTrait::impl<T>::add_with(self, 2);
}
Limitations
Some trait implementations will not receive bindings:
- Trait implementations with generic parameters (e.g.
impl<T> ...) will not receive bindings.- For example, an implementation
impl<U> TwoArgs<i32, U> for MyStructwill not receive bindings because theimplhas a type parameter (U). - This also means that blanket impls (e.g.
impl<T> Trait for T) are not supported.
- For example, an implementation
- Traits with
constparameters (e.g.trait T<const V: usize>) do not yet receive bindings. - Trait methods will not receive bindings if their parameter or return types are not yet supported by Crubit.
How does it work?
Under the hood, Crubit will generate three bindings from the example Rust crate.
A template specialization for impl MyTrait for MyStruct:
template <>
struct ::rs::impl<::example_crate::MyStruct, ::example_crate::MyTrait> {
static constexpr bool kIsImplemented = true;
static ::std::int32_t add_with(::example_crate::MyStruct const& self,
::std::int32_t y);
static ::rs::StrRef describe(::example_crate::MyStruct const& self);
};
Our generated rs::impl specialization is an implementation detail of binding
generation. It holds the actual thunks that call into rust to implement our
trait, but they should be accessed through our template struct MyTrait:
struct CRUBIT_INTERNAL_RUST_TYPE(":: example_crate_golden :: MyTrait") MyTrait {
template <typename T>
using impl = ::rs::impl<T, MyTrait>;
};
Each struct generated for a Rust trait has the impl member that provides
access to the generated rs::impl. This is the preferred way to reference the
generated rs::impl.
Finally, a struct for MyStruct:
struct CRUBIT_INTERNAL_RUST_TYPE(":: example_crate_golden :: MyStruct") alignas(
4) [[clang::trivial_abi]] MyStruct final {
public:
// Default::default
MyStruct();
// No custom `Drop` impl and no custom "drop glue" required
~MyStruct() = default;
MyStruct(MyStruct&&) = default;
MyStruct& operator=(MyStruct&&) = default;
// Rust types that are `Copy` get trivial, `default` C++ copy constructor and
// assignment operator.
MyStruct(const MyStruct&) = default;
MyStruct& operator=(const MyStruct&) = default;
MyStruct(::crubit::UnsafeRelocateTag, MyStruct&& value);
static ::example_crate::MyStruct new_(::std::int32_t x);
private:
// Field type has been replaced with a blob of bytes: Field is private
::std::array<unsigned char, 4> x;
private:
static void __crubit_field_offset_assertions();
};
Our struct is generated normally and exists purely so we have something to implement our trait for.
C++ bindings for Rust type aliases.
A rust Rust type aliases, such as pub type X = ...;, is mapped to the
equivalent C++ type alias, such as using X = ...;.
Limitations:
- The type must be a supported type.
- The alias must not be generic: aliases with generic parameters, such as
pub type X<T> = ..., are not supported.
Example
Given the following Rust crate:
pub type TypeAlias = i32;
Crubit will generate the following bindings:
using TypeAlias CRUBIT_INTERNAL_RUST_TYPE(
":: example_crate_golden :: TypeAlias") = ::std::int32_t;
C++ bindings for Rust use declarations
Crubit supports use declarations for functions and types, mapping them to
equivalent using declarations in C++.
Limitations:
- The
usedeclaration must refer to a function or type. - The
usedeclaration must import exactly one entity per name. For example,pub use m::x;is supported ifxrefers to a function, or to a type, but not if it refers to both a function and a type.
Example
Given the following Rust crate:
pub use module::function;
pub use module::Type;
Crubit will generate the following bindings:
using Type CRUBIT_INTERNAL_RUST_TYPE(":: example_crate_golden :: Type") =
::example_crate::Type;
using __crubit_assert_type = ::example_crate::Type;
using ::example_crate::function;
C++ bindings for Rust standard library (std) types
When generating C++ bindings for a Rust crate (via cpp_api_from_rust) bindings
to the Rust standard library types (std, alloc, and core) are
automatically generated for those types under the namespaces rs::std,
rs::alloc, and rs::core.
Working with String in C++
Rust’s String becomes rs::std::string::String in C++ (also spelled
rs::alloc::string::String). It can be constructed from a C++ string literal:
rs::std::string::String s("hello, world!");
An existing std::string_view (or absl::string_view) can be converted to a
Rust String using rs::StrRef::FromUtf8:
void AcceptsStringView(std::string_view view) {
std::optional<rs::StrRef> str_ref = rs::StrRef::FromUtf8(view);
if (!str_ref.has_value()) {
return;
}
rs::std::string::String s(*str_ref);
// Continue on with using your Rust String...
}
An instance of String can be converted back to a std::string by calling
.as_str() and using the resulting rs::StrRef to construct a std::string:
std::string s2(s.as_str());
EXPECT_EQ(s2, "hello, world!");
rs::StrRef supports implicit conversion to std::string_view to make this
work.
A Rust method that takes an &mut String such as:
#![allow(unused)]
fn main() {
pub fn append_to_rust_string(val: &mut String, s: &str) {
val.push_str(s);
}
}
becomes a C++ method taking a reference to a rs::std::string::String:
// You could just as easily call `push_str` directly. It receives Crubit
// bindings. We use a wrapper method here for expository purposes.
void append_to_rust_string(rs::std::string::String& val, rs::StrRef s);
Allowing for C++ to call it:
append_to_rust_string(s, " I'm a neat addition");
Result
The Rust Result<T, E> generic receives C++ bindings as rs::Result<T, E>, so
long as both T and E are non-ZST types supported by Crubit.
rs::Result<T, E> has a similar API to
std::expected.
(Alternatively, it has a similar API to
std::optional, but with an
additional error payload.) If has_value() returns false, then error() will
return a reference to the error payload of type E.
if (myresult.has_value()) {
Foo(*myresult);
} else {
Foo(myresult.error());
}
A more complete description of the API is in the common ResultBase public base
class: support/rs_std/result.h
Option
The Rust Option<T> generic receives C++ bindings as rs::Option<T>, so long
as T is a non-ZST type supported by Crubit. It has a similar API to
std::optional, and
implicitly converts to and from std::optional.
Unlike std::optional, rs::Option has no operator bool, so check for a
value with has_value() rather than if (myoption). Conversion to
std::optional is only available from an rvalue, so either convert the result
of a call directly or std::move an existing rs::Option:
if (myoption.has_value()) {
Foo(*myoption);
}
std::optional<int32_t> converted = std::move(myoption);
std::optional<int32_t> direct = rust_crate::find_index(name);
A more complete description of the API is in the common OptionBase public base
class: support/rs_std/option.h
Tuple
A Rust tuple (T1, T2, ...) receives C++ bindings as rs::Tuple<T1, T2, ...>.
It can be constructed from its elements (including with brace-initialization,
e.g. {a, b}) when all elements are move-constructible in C++, or converted
from a std::tuple<T1, T2, ...>. Elements are accessed with rs::get<I> or
structured bindings:
rs::Tuple<int32_t, int32_t> pair = {1, 2};
auto [first, second] = pair;
The support header is support/rs_std/tuple.h.
Vec
The Rust Vec<T> generic receives C++ bindings as rs::Vec<T>, so long as T
is a type supported by Crubit. Each rs::Vec<T> is a generated specialization
that inherits its public API from the common VecBase<T> base class in
support/rs_std/vec.h. The API is modeled on
std::vector, with a few
differences noted below.
rs::Vec<T> is a real Rust Vec under the hood: it has the same layout, and
its buffer is allocated and freed through Rust’s global allocator. A Vec
created in C++ can therefore be handed to Rust (and vice versa) without any
copying or conversion.
Element access and iteration
rs::Vec<T> provides the usual contiguous-container accessors:
size(),capacity(),empty()data()(returns a pointer to the contiguous element buffer)operator[],front(),back()– these are bounds-checked and abort the process on misuse (e.g. an out-of-range index orfront()on an empty vector), mirroring the panicking behavior of Rust’sVec.begin()/end(),cbegin()/cend(),rbegin()/rend(),crbegin()/crend()
Iterators are raw pointers, so rs::Vec<T> works with range-based for loops
and standard algorithms, and satisfies std::ranges::contiguous_range. This
also makes it implicitly convertible to std::span<const T> / std::span<T>
and, by extension, to rs::SliceRef<const T> / rs::SliceRef<T>:
rs::Vec<int32_t> v = my_crate::return_vec();
for (int32_t& x : v) {
x += 1;
}
int32_t sum = std::accumulate(v.begin(), v.end(), 0);
std::span<const int32_t> span = v;
rs::SliceRef<const int32_t> slice = v;
Modifying a Vec from C++
Elements can be added, removed, and constructed in place from C++:
rs::Vec<int32_t> v;
v.reserve(3); // Total capacity, like std::vector::reserve.
v.push_back(1);
v.push_back(2);
v.emplace_back(3); // Constructs in place; returns a T&.
v.insert(0, 0); // Inserts at *index* 0, shifting the rest right.
v.pop_back(); // Removes and destroys the last element.
v.clear(); // Destroys all elements but keeps the buffer.
Differences from std::vector worth knowing about:
insert(index, value)takes an index rather than an iterator (as in Rust’sVec::insert).reserve(new_cap)hasstd::vector::reservesemantics: it ensures the total capacity is at leastnew_cap. To request room fornadditional elements (Rust’sVec::reserve(additional)semantics), usereserve_additional_capacity(n)instead.- Only the operations listed above are currently supported. In particular,
there is no
erase,resize, or rangeinsert. For other operations, write a small Rust helper function and call it through its Crubit bindings.
Availability of generic rs_std specializations
Generic standard library types such as rs::Option<T>, rs::Result<T, E>,
rs::Tuple<Ts...>, and rs::Vec<T> are only defined for type arguments that
are actually used in a Crubit-bound Rust API. The support headers (option.h,
result.h, tuple.h, vec.h) provide only the generic base implementations
(OptionBase, ResultBase, VecBase, etc.) and leave the primary template
uninstantiable; the concrete specialization for a given set of type arguments is
emitted into the generated header of a crate whose public API mentions that
instantiation. Attempting to use one of these templates with type arguments that
do not appear in any of the Rust APIs you depend on is a compile-time error.
Callables (Fn, FnMut, FnOnce)
Rust function traits and callable parameters receive bindings through
rs::FnRef and rs::Fn:
rs::FnRef<Sig>(support/rs_std/fn_ref.h): Non-owning, borrowed view of a callable corresponding to&dyn Fnor&mut dyn FnMut(or genericimpl Fn/impl FnMut). Does not allocate on the heap.rs::Fn<Sig>(support/rs_std/fn.h): Owning, heap-allocated callable corresponding toBox<dyn Fn>,Box<dyn FnMut>, orBox<dyn FnOnce>(or genericimpl FnOnce + 'static).rs::Fn<Sig &&>corresponds toFnOnceand must be invoked viastd::move(f)(...).rs::DynCallable<Sig>(support/rs_std/dyn_callable.h): Move-only wrapper for RustBox<dyn Trait>objects received in C++, implicitly convertible toabsl::AnyInvocable<Sig>.
See Callbacks for the complete guide and examples.
Box
The Rust Box<T>, Pin<Box<T>>, Box<[T]>, and Box<str> types receive C++
bindings as rs::Box<T>, rs::PinBox<T>, rs::SliceBox<T>, and rs::StrBox.
See Box types for more details.
Rust’s MaybeUninit type
Rust provides a type called
MaybeUninit<T>,
which represents a T which may be incompletely initialized, or even totally
uninitialized. While a variable of type T, &T or &mut T must refer to a
fully initialized T, a variable of type MaybeUninit<T>, &MaybeUninit<T>,
or &mut MaybeUninit<T> may refer to an incompletely initialized and invalid
T. This allows working with uninitialized memory, even though Rust otherwise
requires initialization.
C++ is different: many types T can be uninitialized, and every pointer or
reference (const T&, T&, const T*, or T*) can point to uninitialized
memory.
Correspondingly, Rust references or pointers to MaybeUninit<T> are treated the
same as Rust references or pointer to T. In all other contexts, including when
passing or returning a MaybeUninit<T> by value, MaybeUninit<T> does not map
to any C++ type1.
| Rust | C++ |
|---|---|
&MaybeUninit<T> | const T& |
&mut MaybeUninit<T> | T& |
*const MaybeUninit<T> | const T* |
*mut MaybeUninit<T> | T* |
MaybeUninit<T> | no bindings |
Why only by pointer/reference?
Given C++’s take on uninitialized memory, one could draw the conclusion that it
would be fine to always represent a MaybeUninit<T> as a T on the C++ side
(as is done by
cbindgen).
However, this can quickly create UB. For instance, take the following Rust function:
#![allow(unused)]
fn main() {
fn foo() -> MaybeUninit<String> {
MaybeUninit::uninit()
}
}
If we make this callable via something like String foo() in C++, the behavior
is undefined. For example, when the returned String is destroyed, it would
access uninitialized bytes as part of its destructor.
-
We investigated creating a C++ type that shadows behaviour of
MaybeUninitin C++, but concluded that this would only be properly feasible with C++ 23 and only for a subset of types. (See b/362475441#comment3.) ↩
Movable types
Crubit requires types to be “movable” to be passed directly by value: if a Rust
type does not logically support a C++ move operation, then it can receive
bindings, but it cannot be passed directly by value (it is passed via the
rs::Movable wrapper type).
A Rust type can be made movable in C++ in one of three ways:
- Copyable: the Rust type implements
Clone. - Trivially move-constructible and destructible: the Rust type
does not have a
destructor. (It does
not implement
Drop, and nor do any of its fields.) - Non-trivially move-constructible: the Rust type has a
destructor, but implements
Default.
The easiest way to ensure your type is useful to end users, even if it is
changed in the future, is to implement Clone and Default. This makes the
type default-constructible and copyable1, as well as efficiently
movable.
Copyable
If the Rust type implements Clone, then the C++ type will be copyable:
- Copy construction has the same behavior as
Clone::clone. - Copy assignment has the same behavior as
Clone::clone_from.
Because the type is copyable, it is also movable, at worst by a copy operation.
Trivially move-constructible and destructible
If no logic occurs during destruction, because the type doesn’t implement
Drop, and none of its fields do, then the C++ type will be trivially-movable
and trivially-destructible:
- Move construction and assignment copy the bytes of the object, with the same behavior as a Rust move operation.
NOTE: All Copy types are guaranteed to be trivially move-constructible and
destructible.
If the Rust type is Copy, then the moved-from object is guaranteed to hold its
old value, and be valid for all operations.
Otherwise, the object is only valid for assignment and destruction, and the behavior of performing any other operation is undefined.
Non-trivially move-constructible
If the Rust type is not trivially movable and destructible, but implements
Default, then the resulting C++ type will be (non-trivially) move
constructible:
- Move construction has the same behavior as
std::mem::take(): it copies the bytes to the new object, as if by a Rust move, and replaces the moved-from object withDefault::default(). - Move assignment copies the bytes to the new object, as if by a Rust move, and replaces the moved-from object with an unspecified but valid object.
Why is this required?
In general, Crubit needs to be able to move objects as part of the implementation of pass-by-value, even in C++17, due to platform ABI restrictions. Even without this requirement, types are not very useful in C++ if they are not movable.
Unlike Rust, C++ has no “destructive move”. There is no way to change an object’s location in memory, only to create a new object with the same value, and leave behind something in the old (still valid) object. Sometimes, what’s left behind is an identical copy of the object state: this is a copy operation, implemented by the C++ copy constructor or copy assignment operator. But sometimes, copying is expensive, and instead what we might leave behind is some kind of junk value. It still must be a valid object (at least so that its destructor and assignment operator can be invoked), but it might represent some invalid or moved-from state.
For example, to “move” a unique_ptr (the C++ equivalent of Box) from one
variable to another, you copy the bytes, and then replace the old location with
a special null value representing an unoccupied / moved-from unique_ptr. This
is why unique_ptr must be nullable in the C++ type system: otherwise, it
could not be moved!
The rs::Movable wrapper
If a Rust type is not C++ movable (has a destructor or drop glue, but does not
implement Clone or Default), Crubit still generates C++ bindings for it, but
its C++ move constructor and move assignment operator will be = deleted:
// C++ move operations are unavailable for this type.
MyType(MyType&&) = delete;
MyType& operator=(MyType&&) = delete;
Because C++ requires moved-from objects to remain valid and destructible, Crubit cannot generate a native C++ move constructor for such types.
Instead, when a function accepts such a type by value as a parameter, Crubit
wraps the parameter in the
rs::Movable<T>
wrapper type.
rs::Movable<T> makes the type movable in C++ by tracking whether the
underlying value has been moved out, ensuring that the destructor is not run on
moved-from instances:
- Accessing members: Like a smart pointer or
std::optional, you can access the underlyingT’s fields and methods usingoperator*andoperator->(e.g.val->method(),(*val).field). Note thatrs::Movable<T>does not provide implicit conversions toTorT&. - Move semantics: You can move
rs::Movable<T>instances withstd::moveto pass them by value into functions. Moving relocates the underlying Rust object into the target without calling intermediate destructors. - Destruction:
~Movable()invokesT’s destructor only if the value has not been moved out.
Example
Given the following Rust code:
#![allow(unused)]
fn main() {
pub struct Unmovable(pub i32);
impl Drop for Unmovable {
fn drop(&mut self) {}
}
pub fn make_unmovable(x: i32) -> Unmovable {
Unmovable(x)
}
pub fn consume_unmovable(x: Unmovable) {
// ...
}
}
Crubit generates C++ bindings where make_unmovable returns Unmovable
directly and consume_unmovable accepts rs::Movable<Unmovable>:
Unmovable make_unmovable(::std::int32_t x);
void consume_unmovable(::rs::Movable<Unmovable> x);
In C++, you call and manipulate them like this:
// Receiving an unmovable return value directly into a local variable
// (uses C++17 guaranteed copy elision):
Unmovable val = make_unmovable(42);
std::cout << val.__field0 << "\n";
// Passing to a function accepting rs::Movable<T> by constructing in-place:
consume_unmovable(rs::Movable<Unmovable>(std::in_place, 100));
// Passing an rs::Movable<T> received from another function or parameter:
void forward_unmovable(rs::Movable<Unmovable> movable) {
// Accessing fields and methods through rs::Movable<T>:
std::cout << movable->__field0 << "\n"; // operator->
std::cout << (*movable).__field0 << "\n"; // operator*
// Passing an rs::Movable<T> by value:
consume_unmovable(std::move(movable));
}
-
The combination of default-constructible and copyable is so important for making types useful in C++ that it even has a name: “semiregular” ↩
Rust bindings for C++ libraries
When a C++ library enables Crubit, that library can be used directly from Rust. This page documents roughly what that entails, and additional subpages (available in the left-hand navigation) document specific aspects of the generated bindings.
Tip: The code examples below are pulled straight from https://github.com/google/crubit/tree/main/examples/cpp/function/. The other examples in https://github.com/google/crubit/tree/main/examples/cpp/ are also useful. If you prefer just copy-pasting something, start there.
How to use Crubit
Crubit allows you to call some C++ interfaces from Rust. It supports
functions, classes and structs, and
enums, as well as concrete class template specializations (such as
MyTemplate<int>) that appear in your API. Crubit does not support advanced
features like function templates, uninstantiated templates, or virtual
inheritance.
The rest of this document goes over how to create a C++ library that can be called from Rust, and how to actually call it from Rust. The quick summary is:
-
Define a
rust_api_from_cpptarget in the sameBUILDfile as yourcc_library. -
Add the generated
.hinttarget (e.g.:<name_of_rust_target>.hint) to theaspect_hintsof yourcc_library. -
Depend on the
rust_api_from_cpptarget in thedepsof your Rust target. In Rust, the crate is named after thecc_library, not therust_api_from_cpptarget. See Crate name.
The bindings can be previewed by building the cc_library with
--config=crubit-genfiles. See Look at the generated bindings.
$ bazel build --config=crubit-genfiles //path/to:my_cc_library
Write a cc_library target
The first part of creating a library that can be used by Crubit is to write a
cc_library target. For example:
#ifndef THIRD_PARTY_CRUBIT_EXAMPLES_CPP_FUNCTION_EXAMPLE_H_
#define THIRD_PARTY_CRUBIT_EXAMPLES_CPP_FUNCTION_EXAMPLE_H_
#include <stdint.h>
namespace gshoe {
inline int32_t add_two_integers(int32_t x, int32_t y) { return x + y; }
} // namespace gshoe
#endif // THIRD_PARTY_CRUBIT_EXAMPLES_CPP_FUNCTION_EXAMPLE_H_
If you write a BUILD target as normal, it will not actually get Crubit bindings, but we’ll start from there:
# This BUILD rule is here for exposition only. This is an example of what happens if
# you _don't_ set things up for Crubit.
cc_library(
name = "example_lib_broken",
hdrs = ["example.h"],
)
Enable Crubit on a target
To enable Crubit on a C++ target, you must define a rust_api_from_cpp target
associated with it, and link them using aspect_hints.
Behind the scenes, Rust APIs are generated via Bazel aspects1
which run on the cc_library target. When examining a cc_library, Rust API
generation looks for the aspect_hints so that it can find the corresponding
rust_api_from_cpp target.
Define a rust_api_from_cpp target in the same BUILD file as your
cc_library (loaded via
load("//bazel:rust_api_from_cpp.bzl", "rust_api_from_cpp")), and add its .hint target to the aspect_hints of the
cc_library:
cc_library(
name = "example_lib",
hdrs = ["example.h"],
# Opt into using Crubit.
aspect_hints = [":example_lib_rust.hint"],
)
rust_api_from_cpp(
name = "example_lib_rust",
cpp_target = ":example_lib",
)
The .hint target is automatically created by the rust_api_from_cpp macro
(named <name>.hint) and is used to avoid circular dependencies between the C++
library and the generated Rust API.
Note that having Rust callers does constrain library evolution. Certain changes cannot be made in C++ without breaking Rust callers, unless care is taken. crubit.rs/cpp/cookbook#compatibility
Look at the generated bindings
To examine the generated Rust bindings for a C++ library, build the cc_library
target with --config=crubit-genfiles:
$ bazel build --config=crubit-genfiles //examples/cpp/function:example_lib
Build the cc_library itself, not the rust_api_from_cpp target. The bindings
are generated by an aspect that runs on the cc_library, so building the
rust_api_from_cpp target with this flag doesn’t print any generated files.
This is the best way to preview the generated bindings for a given C++ target right now. You might end up using this a lot, so keep it in your shell history.
If you run the above command, you should see some output like the following:
Aspect //rs_bindings_from_cc/bazel_support:rust_bindings_from_cc_aspect.bzl%rust_bindings_from_cc_aspect of //examples/cpp/function:example_lib up-to-date:
bazel-bin/examples/cpp/function/example_lib_rust_api_impl.cc
bazel-bin/examples/cpp/function/example_lib_rust_api.rs
bazel-bin/examples/cpp/function/example_lib_namespaces.json
These files are the generated bindings which are used under the hood when depending on a C++ target from Rust. They consist of:
- The supporting C++ code to glue Rust and C++ together. (The
.ccfile.) - The public Rust interface. (The
.rsfile.) - Supporting internal implementation details. (The
.jsonfile.)
You don’t need to check them in, as they are regenerated automatically whenever you build a Rust build target which depends on C++.
The .rs file is the interesting one for end users. It should contain an
actually useful API for the target:
// Automatically @generated Rust bindings for the following C++ target:
// //examples/cpp/function:example_lib
#![rustfmt::skip]
#![feature(custom_inner_attributes)]
#![allow(stable_features)]
#![allow(improper_ctypes)]
#![allow(nonstandard_style)]
#![allow(unused)]
#![allow(deprecated)]
#![allow(unknown_lints, suspicious_runtime_symbol_definitions)]
#![deny(warnings)]
pub mod gshoe {
#[inline(always)]
pub fn add_two_integers(x: i32, y: i32) -> i32 {
unsafe { crate::detail::__rust_thunk___ZN5gshoe16add_two_integersEii(x, y) }
}
}
// namespace gshoe
mod detail {
#[allow(unused_imports)]
use super::*;
unsafe extern "C" {
pub(crate) unsafe fn __rust_thunk___ZN5gshoe16add_two_integersEii(x: i32, y: i32) -> i32;
}
}
When something is missing
If Crubit can’t generate bindings for a function or type, the build doesn’t fail
by default. Crubit leaves the item out of the generated .rs file and writes a
comment in its place explaining why:
#![allow(unused)]
fn main() {
// error: function `UseNotCrubitExposed` could not be bound
// Unsupported parameter type `NotCrubitExposed not_crubit_exposed`:
// Crubit is not enabled on defining target:
// //path/to:not_crubit_exposed
}
The first error you see is usually in the Rust code that calls the missing item,
which doesn’t say why it’s missing. Search the generated .rs file for the
item’s name to find the reason. The reason often names a type rather than the
function itself; fix or wrap that type, and the function gets bindings too.
crubit.rs/errors explains the most common reasons.
To make a missing item fail the build instead, apply the CRUBIT_MUST_BIND
attribute to it. See Customizing the generated bindings.
Use a C++ library from Rust
To depend on a C++ library from Rust, add the corresponding rust_api_from_cpp
target to your Rust target’s deps:
rust_binary(
name = "main",
srcs = ["main.rs"],
edition = "2024",
# Declare a dependency on Rust bindings for calling into the C++ `example_lib` library:
deps = [":example_lib_rust"],
)
At that point, the bindings are directly usable from Rust. The interface is
identical to the .rs file previewed earlier, but can be used directly:
fn main() {
let sum = example_lib::gshoe::add_two_integers(2, 2);
println!("sum = {sum}");
}
Crate name
The Rust crate is named after the cc_library, not the rust_api_from_cpp
target. In the example above, the crate is example_lib, not
example_lib_rust. C++ namespaces become Rust modules inside it, so
gshoe::add_two_integers in C++ is example_lib::gshoe::add_two_integers in
Rust.
The crate name is the cc_library’s target name with every character that isn’t
a letter or digit replaced by _. The package path isn’t part of it. Two
exceptions:
- If the name starts with a digit, it gets an
nprefix:3d_mathbecomesn3d_math. - A target named
corebecomescore_plus the last directory of its package, so that it doesn’t shadow Rust’scorecrate://foo/bar:corebecomescore_bar.
Common Errors
See crubit.rs/errors
Unsupported features
Some features are either unsupported, or else only supported with experimental feature flags . In order to get bindings for a C++ interface, that interface must only use the subset of features currently supported.
The way to work around this kind of problem, in all cases, is to wrap or hide the problematic interface behind an interface Crubit can handle:
- Hide unsupported types behind a wrapper. For example, a
std::set<T>is not supported, but a struct which wraps astd::set<T>is. crubit.rs/errors/unsupported_type describes the process in more detail. - Wrap unsupported functions, in general, behind wrappers.
- Embed C++ directly in Rust: Use
inline_cpp!to allow embedding arbitrary C++ code within a Rust target using therust_library_with_embedded_cppBazel rule.
-
Crubit is an aspect: an automatically generated entity that exists on every build target. It is disabled by default, so that Rust callers don’t accidentally impose on C++ libraries that weren’t expecting them.
Aspects allow Crubit to fully understand the dependency graph: the bindings for X are in the Crubit aspect of X. This allows Crubit to generate bindings which themselves rely on bindings: if a function in target
Areturns a struct from targetB, we know that the bindings forAwill depend on the bindings forB. Because Crubit is an aspect, it already knows the name of the bindings forB: it’s simply the Crubit aspect onB!Without aspects, or something like aspects, you would need to write down, for every library, the location of its Rust bindings. There is no need for that kind of boilerplate when aspects are involved, and that is why most things shaped like Crubit use aspects. For example, protocol buffers use aspects for their generated implementations in multiple languages. (They also use named rules, but the rules simply re-export the aspect, and the underlying aspect is what is used within the rule for referring to transitive dependencies.) Thanks to aspects, the
proto_librarydoesn’t need to re-specify “ah, and the Go proto is named'x'”.Be not afraid! Aspects are what make transitive dependencies work seamlessly, without boilerplate. So when you see aspect this, or aspect that, remember: this is a Good Thing. ↩
Building
This page is about calling C++ from Rust with rust_api_from_cpp. If you
want to call Rust from C++, see C++ bindings for Rust
libraries instead.
Using rust_api_from_cpp in your build
rust_api_from_cpp is available in Bazel, through the rules_crubit
module. See Bazel for setup
instructions. Bazel builds the rs_bindings_from_cc tool for you, including the
LLVM libraries it depends on, so there is nothing else to install.
CMake and Cargo integrations for rust_api_from_cpp are
planned but not available yet. See Build System
Integrations for the current status of every
build system.
Building rs_bindings_from_cc yourself
If you’re integrating Crubit into another build system, you can build the
rs_bindings_from_cc binary yourself and run it the way the Bazel
rules
do.
rs_bindings_from_cc is a Rust program that links against Clang’s C++
libraries, plus Abseil and Protobuf. It is built against a specific
llvm-project commit, pinned in MODULE.bazel and updated regularly by CI.
Open-source CI covers Linux x86_64.
There are two ways to build it: with Bazel (which builds LLVM from source for you) or with Cargo against your own LLVM, Abseil, and Protobuf installation.
Bazel
This is how Crubit’s CI builds the tool. Bazel downloads the Rust and Clang
toolchains, and it builds LLVM from source at the commit pinned in
MODULE.bazel.
Because it builds LLVM, the first build takes a long time.
Prerequisites: bazelisk, python3, and the development headers for zlib,
libxml2, and zstd in /usr/include. On Debian or Ubuntu, install them with apt install zlib1g-dev libxml2-dev libzstd-dev.
git clone https://github.com/google/crubit.git
cd crubit
bazelisk build //rs_bindings_from_cc:rs_bindings_from_cc_main
To build against a different llvm-project commit, set the CRUBIT_LLVM_COMMIT
environment variable to the full commit SHA.
Cargo, with your own LLVM, Abseil, and Protobuf
This option is for packagers and for projects that already build LLVM, such as Chromium. CI doesn’t test it.
Prerequisites
- LLVM and Clang libraries, built and installed from the
llvm-projectcommit pinned inMODULE.bazel(or one close to it).- They must be built with compression support (zlib), which is the default build config.
- Abseil libraries, built and installed.
- Protobuf libraries and the
protoccompiler (v27.1 or later), installed. Crubit’s CI uses Protobuf 36.1. - zlib (for example,
libz.so) in the system include and lib paths. - A recent Rust toolchain. In a Crubit checkout,
rustupautomatically selects the nightly toolchain fromrust-toolchain.toml.
Linux-specific setup:
# Choice of compiler is optional.
export CC=/path/to/clang
export CXX=/path/to/clang++
# We must use `lld` linker via clang. It must be in the PATH.
export PATH="$PATH:/dir/containing/lld"
export RUSTFLAGS="$RUSTFLAGS -Clinker=/path/to/clang"
export RUSTFLAGS="$RUSTFLAGS -Clink-arg=-fuse-ld=lld"
# If you want to use a sysroot.
# SYSROOT_FLAG=--sysroot=$SYSROOT
# export CXXFLAGS="$CXXFLAGS $SYSROOT_FLAG"
# export RUSTFLAGS="$RUSTFLAGS -Clink-arg=$SYSROOT_FLAG"
macOS-specific setup (untested):
export CC=clang
export CXX=clang++
export RUSTFLAGS="$RUSTFLAGS -Clinker=clang"
export RUSTFLAGS="$RUSTFLAGS -Clink-arg=-fuse-ld=lld"
# Point to the Xcode sysroot.
export CXXFLAGS="$CXXFLAGS -isysroot $(xcrun --show-sdk-path)"
export RUSTFLAGS="$RUSTFLAGS -Clink-arg=-isysroot -Clink-arg=$(xcrun --show-sdk-path)"
Windows-specific setup:
- Windows is currently unsupported, and the APIs generated by Crubit may not compile and will change over time.
- All commands must be run from a development shell, where MSVC environment variables are set up.
# We use clang compiler (clang-cl); MSVC may work too but is unsupported.
export CC=clang-cl
export CXX=clang-cl
# We must use lld to link, which is spelt lld-link. So user-specified linker
# flags must be in MSVC format.
export RUSTFLAGS="$RUSTFLAGS -Clinker=/path/to/lld-link"
# LLVM was built with Zlib support. Point Crubit to the same library.
export CXXFLAGS="$CXXFLAGS /I/path/to/zlib"
export RUSTFLAGS="$RUSTFLAGS -Clink-arg=/LIBPATH:/path/to/zlib"
# Avoid deprecation warnings.
export CXXFLAGS="$CXXFLAGS /D_CRT_SECURE_NO_DEPRECATE"
# If LLVM (-DCMAKE_MSVC_RUNTIME_LIBRARY) and Abseil (-DABSL_MSVC_STATIC_RUNTIME)
# are built against static CRT, then Rust needs to match, or vice-versa.
# export RUSTFLAGS="$RUSTFLAGS -Ctarget-feature=+crt-static"
Run the build step via cargo:
# Pre-generate the C++ headers for Crubit's own .proto files. This avoids
# races during the parallel Cargo build.
python3 cargo/build/generate_proto_headers.py \
--protoc=/path/to/protoc --out_dir=/path/to/generated_headers
export PROTOC=/path/to/protoc
# Paths for Crubit's cargo to use. Most accept a comma-separated list.
## This path contains clang/ and llvm/ dirs with their respective headers.
export CLANG_INCLUDE_PATH=/path/to/llvm/and/clang/headers
## This path contains libLLVM*.a and libclang*.a. On Windows it may also hold the
## import libraries of LLVM's DLLs (LLVM-C.lib and friends); those are recognised by
## their contents and skipped, so the whole `lib` dir can be given as-is.
export CLANG_LIB_STATIC_PATH=/path/to/llvm/and/clang/libs
## This path contains absl/ dir with all the includes.
export ABSL_INCLUDE_PATH=/path/to/absl/include/dir
## This path contains libabsl_*
export ABSL_LIB_STATIC_PATH=/path/to/absl/libs
## These paths contain google/protobuf/ dir with all the includes, and the
## headers generated above.
export PROTOBUF_INCLUDE_PATH=/path/to/protobuf/include/dir,/path/to/generated_headers
## This path contains libprotobuf.a and libutf8_validity.a. Other archives that a
## protobuf install tree holds (libutf8_range.a, libprotobuf-lite.a, libprotoc.a,
## libupb.a) are ignored; the first two would collide with the ones above.
export PROTOBUF_LIB_STATIC_PATH=/path/to/protobuf/libs
# Tell Crubit which LLVM it is compiled against, as the commit date (YYYYMMDD) of
# your llvm-project checkout.
LLVM_DATE=$(python3 cargo/build/get_llvm_commit_date.py <llvm_commit_sha>)
export CXXFLAGS="$CXXFLAGS -DCRUBIT_LLVM_DEV_DATE=${LLVM_DATE}"
cargo build --locked --release -p rs_bindings_from_cc
Cargo build of Crubit explains each of these variables in more detail.
If you need to build LLVM and Clang yourself, here is an example:
git clone https://github.com/llvm/llvm-project
cd llvm-project
CC=clang CXX=clang++ cmake -S llvm -B build -DLLVM_ENABLE_PROJECTS='clang' -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=install
cmake --build build -j
# wait...
cmake --install build
C++ Bindings Cookbook
This document presents a collection of techniques for creating Rust bindings for C++ libraries.
These techniques are often workarounds for gaps in what Crubit can do. Expect the recommended practices to evolve over time, as Crubit’s capabilities expand!
BEST PRACTICE: The tips below describe deviations from typical C++ style. (If typical C++ style worked, you wouldn’t need a cookbook.) When you deviate from typical C++ style, document why, and try to keep changes limited in scope, close to the interop boundary.
If possible, solve the same problem while staying within more typical C++ style. For example: you may be able to add
ABSL_ATTRIBUTE_TRIVIAL_ABIto a type you control, instead of boxing the type in a pointer.
Making types Rust-movable
As described in crubit.rs/cpp/classes_and_structs#rust_movable, types have a more pleasant API when they are Rust-movable.
This can happen for a couple of easily fixable reasons, described in subsections:
- The type defines a destructor or copy/move constructor / assignment
operator. If it is in-principle still Rust-movable, and these functions do
not care about the address of the object in memory, then the type can be
annotated with
ABSL_ATTRIBUTE_TRIVIAL_ABI - The type has a field which is not rust-movable. In that case, the field can be boxed in a pointer.
There are other reasons a type can become non-Rust-movable, which do not have these easy fixes described below. For example, virtual methods, or non-Rust-movable base classes. For those, your only option is the hard option of more radically restructuring your code to avoid those patterns.
ABSL_ATTRIBUTE_TRIVIAL_ABI
crubit.rs/cpp/cookbook#trivial_abi
One of the ways a type can become non-Rust-movable is if it has a copy/move
constructor / assignment operator, or a destructor. In that case, Clang will
assume that it cannot be trivially relocated, unless it is annotated with
ABSL_ATTRIBUTE_TRIVIAL_ABI.
struct LogWhenDestroyed {
~LogWhenDestroyed() {
std::cerr << "I was destroyed!\n";
}
};
struct ABSL_ATTRIBUTE_TRIVIAL_ABI LogWhenDestroyed {
~LogWhenDestroyed() {
std::cerr << "I was destroyed!\n";
}
};
WARNING: Only use
ABSL_ATTRIBUTE_TRIVIAL_ABIif changing the location of an object in memory is safe. In particular, if the object is self-referential, usingABSL_ATTRIBUTE_TRIVIAL_ABIwill result in Undefined Behavior (UB).class SelfReferential { public: SelfReferential(const SelfReferential& other) : x(other.x), x_ptr(&x) {} private: int x = 0; int* x_ptr = &x; };Types like this, if Rust-moved, will contain invalid pointers. Carefully review any code adding
ABSL_ATTRIBUTE_TRIVIAL_ABI.
Boxing in a pointer
crubit.rs/cpp/cookbook#boxing
One of the ways a type can become non-Rust-movable is if it has a field, where the type of that field is not Rust-movable. There is no way to override this: there is nothing a type can do to make itself Rust-movable if one subobject is not.
For example, consider a field like std::string name;. std::string defines a
custom destructor and copy / move constructor/assignment operator, in order to
correctly manage owned heap memory for the string. Because of this, it also is
not Rust-movable. And, at the time of writing, std::string currently cannot
use ABSL_ATTRIBUTE_TRIVIAL_ABI in any STL implementation. In the case of
libstdc++, for example, std::string contains a self-referential pointer: when
the string is small enough, the data() pointer refers to the inside of the
string. Rust-moving it would cause the pointer to refer back to the old
object, which would cause undefined behavior.
If a struct or class contains a std::string as a subobject by value, or any
other non-Rust-movable object, then that struct or class is itself also not
Rust-movable. (If you somehow were able to Rust-move the parent object, this
would also Rust-move the string, causing the very same issues.)
Instead, what you can do is change the type of the field, so that it doesn’t contain the problematic type by value. Instead, it can hold the non-Rust-movable type by pointer.
BEST PRACTICE: Except where necessary for better Rust interop, this is not
good C++ style. When you use this trick, document why, and try to limit it to
types close to the interop boundary. If possible, instead of boxing T, make
T itself rust-movable. (This is not easy for standard library types, but if
the type is under your control, it may be as easy as adding
ABSL_ATTRIBUTE_TRIVIAL_ABI.)
unique_ptr
NOTE: The following is non-portable, and only works in libc++ with the unstable ABI. If you aren’t sure about whether you are using the unstable ABI, it is likely that you are not, but you might want to check in with your local toolchain maintainer.
If you tightly control your dependencies, you might be using libc++’s unstable
ABI. The unstable ABI, among other things, makes unique_ptr<T> (and
shared_ptr<T>) Rust-movable. In fact, it is Rust-movable even if T itself is
not.
This means that if a particular field is making its parent type
non-Rust-movable, one fix is to wrap it in a unique_ptr:
struct Person {
std::string name;
int age;
};
struct Person {
// boxed to make Person rust-movable: crubit.rs/cpp/cookbook#boxing
std::unique_ptr<std::string> name;
int age;
};
Raw pointers
BEST PRACTICE: This should only be used in codebases that do not use a
Rust-movable unique_ptr or unique_ptr equivalent. Consider wrapping this in
an ABSL_ATTRIBUTE_TRIVIAL_ABI type which resembles unique_ptr, instead.
When not using libc++’s unstable ABI, the most straightforward way to make a
field Rust-movable is to instead use a raw pointer, and delete it in the
destructor (as if it were held by a unique_ptr).
struct Person {
std::string name;
int age;
};
struct ABSL_ATTRIBUTE_TRIVIAL_ABI Person {
// Owned, boxed to make Person rust-movable: crubit.rs/cpp/cookbook#boxing
std::string* name;
int age;
~Person() {
delete name;
}
};
(Note the use of ABSL_ATTRIBUTE_TRIVIAL_ABI: because we added a destructor, we
also need to add ABSL_ATTRIBUTE_TRIVIAL_ABI to indicate that the destructor
does not care about the address of Person.)
Renaming functions for Rust
crubit.rs/cpp/cookbook#renaming
Overloaded functions cannot be called from Rust (yet: b/213280424). To make them
available anyway, you can use the CRUBIT_RUST_NAME attribute macro to specify
a unique Rust name for each overload:
void Foo(int x);
void Foo(float x);
#include "support/annotations.h"
CRUBIT_RUST_NAME("FooInt")
void Foo(int x);
CRUBIT_RUST_NAME("FooFloat")
void Foo(float x);
Thread-safety
WARNING: Thread-safety in interop is subtle. Ensure you understand the implications before exposing mutating methods to Rust.
SUMMARY: The preferred way to make a thread-safe C++ type available to Rust is
to annotate it with CRUBIT_THREAD_SAFE. This automatically implements Send + Sync in Rust, wraps the type in UnsafeCell, and allows non-const methods to
be called via shared references (&self). However, this requires all mutated
fields to be marked mutable, or else for the object to never be stored in a
C++ const.
CRUBIT_THREAD_SAFE
WARNING: This is an unsafe operation. Only use this annotation on thread-safe
types which are never created as const, or else which mark all fields that can
be mutated as mutable, even if those fields can only be mutated from a
non-const method.
To make a C++ class that internally synchronizes access (e.g., using mutexes or
atomics) safe to use concurrently in Rust, annotate its definition with
CRUBIT_THREAD_SAFE:
#include "support/annotations.h"
class CRUBIT_THREAD_SAFE ThreadSafeCounter {
public:
void Increment(); // Can be called via `&self` in Rust.
int Get() const; // Can be called via `&self` in Rust.
private:
mutable std::atomic<int> counter_;
};
This annotation tells Crubit to:
- Implement
SendandSyncfor the type in Rust. - Wrap the Rust representation in
UnsafeCell, which permits shared mutation without undefined behavior in Rust. - Expose non-const C++ methods as taking
&self(shared reference) instead of&mut selfor raw pointers.
This allows Rust callers to use the type concurrently across threads using standard shared references.
Because CRUBIT_THREAD_SAFE makes the type internally mutable in Rust, the type
must either also be internally mutable in C++, by making all mutated fields
mutable, or else it must never be stored as a const object.
To help prevent accidentally passing in references to C++ objects with const
storage when calling back into Rust, Crubit automatically annotates generated
Rust bindings for CRUBIT_THREAD_SAFE types with a Rust-side attribute
(#[crubit_annotate::cpp_thread_safe]). When generating C++ bindings from Rust
(cc_bindings_from_rs) for functions that accept or return shared references
(&MyThreadSafeType) to these types, Crubit generates non-const references in
C++ (MyThreadSafeType& instead of const MyThreadSafeType&). This prevents
C++ callers from passing immutable const objects into Rust functions that
might internally mutate them through a shared reference.
Manual Approach (Legacy)
Before CRUBIT_THREAD_SAFE was available, or if you cannot annotate the class
directly, making a mutating C++ method safe to call concurrently in Rust
required a manual workaround:
- The class must be documented as thread-safe.
- The field being mutated must be marked as
mutable, even if it is only mutated in non-const methods, so that internal mutation in Rust does not cause UB. - The method must be manually wrapped in a C++
constmethod (or wrapped in Rust withunsafecasts).
Public mutable fields and opaque bytes are safely handled by wrapping them in
std::cell::Cell.
Example of manual wrapping:
class FakeClock {
public:
CRUBIT_RUST_NAME("advance_mut")
void Advance(int n) {
InternalDoNotUseForRustOnlyAdvance(n);
}
CRUBIT_RUST_NAME("advance")
void InternalDoNotUseForRustOnlyAdvance(int n) const {
mytime_ += n;
}
private:
// Mutable for Rust.
mutable int mytime_;
};
You can also use the CRUBIT_THREAD_SAFE annotation, but this is more
dangerous, as there is no enforcement that every method only modifies mutable
fields, and requires more extensive review:
class CRUBIT_THREAD_SAFE FakeClock {
...
};
Using const methods, while less idiomatic in C++, structurally forces
soundness, so that there is no UB.
Working around blocking bugs in Crubit
Crubit is still in development, and has bugs which can completely stop your work if Crubit was in the critical path. These can take the form of parsing errors or crashes when Crubit runs, or else generated bindings which do not compile.
The following workarounds can help get you moving again:
Disable Crubit on a declaration
If a declaration causes hard failures within Crubit, that declaration alone can
be disabled using the CRUBIT_DO_NOT_BIND attribute macro:
#include "support/annotations.h"
CRUBIT_DO_NOT_BIND // b/123: Crubit crashes on std::crash_rust_plz_t
void foo(std::crash_rust_plz_t*);
NOTE: By disabling Crubit on this declaration, items which depend on it may also, in turn, not receive bindings. For example, if it declares a type, then functions which accept or return that type will also not receive bindings.
For that reason, except for functions,
CRUBIT_DO_NOT_BINDrequires modifying an allowlist at rs_bindings_from_cc/bazel_support/generate_bindings.bzl. Changes to that file are most easily mailed using_manage: add AUTO_MANAGE=testing:TGPto the CL description.
Disable Crubit on a header
If an entire header is giving problems (e.g. is unparseable), then it can be
removed from consideration by Crubit. Once disabled, Crubit will avoid reading
the header directly, although it is still included via #include preprocessor
directives.
Add the target name and header name to public_headers_to_remove in
rs_bindings_from_cc/bazel_support/rust_bindings_from_cc_aspect.bzl.
See the example in
rs_bindings_from_cc/test/disable/disable_header/.
To mail the CL performing this change, use AUTO_MANAGE=testing:TGP to the CL description.
NOTE: By disabling Crubit on this header, items which are defined in that header will not receive bindings. For example, this means that functions which use types defined in that header will also not get bindings, even if the function was defined in a header that was not disabled.
When possible, it’s preferable to use a smaller fix. For example, if the same header is owned by two targets, it’s preferable to move the header into a third target, depended on by both. That way, functions which use types defined in that header will still get bindings, in both targets.
Making backwards-compatible changes to C++ code with Rust callers
Because Rust does not have all the features C++ does, some changes that do not usually break a C++ caller can break a Rust caller. For example, most C++ callers will not be broken by adding a new overload to an overload set, but that does break Rust callers, because Rust doesn’t have overloading.
Most of the time, you can think of Rust users (and Crubit) as the most pathological possible C++ caller, relying on all the details of your API.
The following sections attempt to be an exhaustive list of the things you might expect to be able to do if you only have C++ callers, but cannot do with Rust callers.
Making a type non-Rust-movable
If a type is made non-Rust-movable, when it was previously Rust-movable, this will break all but the most trivial of Rust callers.
Specifically, the following operations are compatibility-breaking if the type was Rust-movable:
- Adding a virtual method.
- Adding a user-defined copy constructor, move constructor, or destructor,
without specifying
ABSL_ATTRIBUTE_TRIVIAL_ABI. - Adding a non-Rust-movable field or base class (such as
std::string).
struct Before {
int x;
};
struct After {
~After() {}
int x;
std::string y;
};
Before ReturnBefore();
After ReturnAfter();
#![allow(unused)]
fn main() {
#[derive(Copy, Clone)]
struct Before {...}
struct After {...}
pub fn ReturnBefore() -> Before {...}
pub fn ReturnAfter() -> Ctor![After] {...}
}
Fix: Preserve Rust-movability, as described above in Making types Rust-movable
struct ABSL_ATTRIBUTE_TRIVIAL_ABI CompatibleAfter {
~CompatibleAfter() {}
int x;
std::unique_ptr<std::string> y;
};
Adding or removing virtual destructors
If a type’s destructor is made virtual, or was previously virtual and now
becomes nonvirtual, this changes how Rust callers invoke destruction. For
example, unique_ptr<T> becomes virtual_unique_ptr<T> if T has a virtual
destructor.
class Before {};
class After { public: virtual ~After() {} };
std::unique_ptr<Before> ReturnBefore();
std::unique_ptr<After> ReturnAfter();
#![allow(unused)]
fn main() {
pub fn ReturnBefore() -> cpp_std::unique_ptr<Before> {...}
pub fn ReturnAfter() -> cpp_std::virtual_unique_ptr<After> {...}
let _: cpp_std::unique_ptr<_> = ReturnAfter(); // error[E0308]: mismatched types
}
Fix: Migrate callers that use heap allocated types such as unique_ptr.
#![allow(unused)]
fn main() {
let _: cpp_std::virtual_unique_ptr<_> = ReturnAfter();
}
Adding an overload
Except for constructors and operators, adding an overload to a C++ function will
cause it to no longer receive bindings, unless the overloads are given distinct
names in Rust (via CRUBIT_RUST_NAME).
void Before();
void After();
void After(int);
#![allow(unused)]
fn main() {
pub fn Before() {...}
// No bindings for After.
}
Fix: Use CRUBIT_RUST_NAME("NewName") to specify a per-overload name.
void CompatibleAfter();
CRUBIT_RUST_NAME("CompatibleAfterInt")
void CompatibleAfter(int);
#![allow(unused)]
fn main() {
pub fn CompatibleAfter() {...}
pub fn CompatibleAfterInt(_: c_int) {...}
}
Changing types with implicit conversions
In C++, it is often compatible to change from, for example, accepting an
int32_t to a int64_t, or similar. This is not a compatible change in Rust,
because implicit conversions do not exist.
void Before(int32_t);
void After(int64_t);
#![allow(unused)]
fn main() {
pub fn Before(_: i32) {...}
pub fn After(_: i64) {...}
let x: i32 = ...;
After(x); // error[E0308]: mismatched types
}
Fix: Migrate callers.
#![allow(unused)]
fn main() {
After(x.into())
}
Unsupported features
Use of other unsupported features of C++, such as decltype or various
unsupported clang attributes, will also cause breakages for Rust callers.
This is an open-ended failure category, and includes, at minimum:
- Changing from a concrete function to a template function
- Changing from an ordinary class to a type alias to a template specialization
- Unrecognized attributes
- Types that do not have Crubit support yet
For instance:
int Before(int x) {...}
[[clang::very_interesting_advanced_attribute]]
auto After(int x) -> decltype(x + std::declval<int>()) {...}
#![allow(unused)]
fn main() {
pub fn Before(_: c_int) -> c_int {...}
// No bindings for After
}
Fix: Avoid the use of these features, or wrap with Rust-only functions that avoid the use of these features.
int CompatibleAfter(int x) {return After(x);}
#![allow(unused)]
fn main() {
pub fn CompatibleAfter(_: c_int) -> c_int {...}
}
Best Practices Writing Rust Bindings for Existing C++ Libraries
Introduction
This document is an attempt at guidance for how Rust changes can be made to existing C++ libraries, including core foundational libraries.
For an introduction, see Rust Bindings for C++ Libraries.
Code Organization
A C++ library and its Rust bindings are defined as separate targets in the same
BUILD file: the cc_library target itself, and a corresponding
rust_api_from_cpp target.
load(
"//bazel:rust_api_from_cpp.bzl",
"rust_api_from_cpp",
)
cc_library(
name = "my_lib",
hdrs = ["my_lib.h"],
aspect_hints = [":my_lib_rust.hint"],
)
rust_api_from_cpp(
name = "my_lib_rust",
cpp_target = ":my_lib",
)
The rust_api_from_cpp target acts like a rust_library and can be depended on
by other Rust targets via deps. The cc_library rule receives an
aspect_hint called :my_lib_rust.hint. This aspect hint is generated by
rust_api_from_cpp, and tells Crubit how to generate bindings for the given
cc_library.
The rust_api_from_cpp target should be defined in the same BUILD file as the
original cc_library target. This helps ensure that there is a single
easily-discoverable Rust API for a given C++ library.
Adding custom Rust source files (srcs)
Handwritten Rust code can be added to the generated bindings by specifying the
srcs attribute on rust_api_from_cpp:
rust_api_from_cpp(
name = "my_lib_rust",
srcs = ["my_lib_custom.rs"],
cpp_target = ":my_lib",
)
To allow Crubit bindings to be #![no_std] by default, the generated crate does
not include the regular std Rust prelude. Consequently, standard library types
like Box have to be imported manually:
#![allow(unused)]
fn main() {
extern crate std;
use std::boxed::Box;
}
Types in srcs do not receive C++ bindings
Types defined in the srcs of a rust_api_from_cpp target are intended for
customizing or extending the generated Rust API. However, Crubit does not
generate C++ bindings for types defined in srcs.
Because Crubit only generates C++ bindings for the underlying C++ target, any struct, enum, or union defined in these custom Rust source files has no C++ counterpart. If a downstream Rust crate attempts to expose a function, method, or type that uses one of these types in its public interface, Crubit cannot generate C++ bindings for that item and will report an error.
If you need a type to be accessible from both C++ and Rust, you can either:
- Define the type in C++: Define the type in the C++ library wrapped by
rust_api_from_cppso that Crubit generates Rust bindings for it, or - Define the type in a separate
rust_library: Define the type in a separaterust_librarytarget that is then used as a dependency in both Rust and (via acpp_api_from_rusttarget) in C++.
If the type is purely for internal use within Rust and is never intended to be
used from C++, you can silence the warning on the type definition using the
#[crubit_annotate::allow_unbindable_type] attribute (or the
CRUBIT_ALLOW_UNBINDABLE_TYPE macro in C++-accessible contexts).
Limiting C++ function visibility to srcs with CRUBIT_PUB_CRATE
When writing custom Rust code in srcs to wrap or customize a C++ API, you may
need C++ helper functions or internal methods to be callable from your Rust
srcs files, but without exposing them in the public Rust API of the generated
crate.
You can mark a C++ function or method with CRUBIT_PUB_CRATE from
"support/annotations.h":
#include "support/annotations.h"
class MyClass {
public:
// Publicly available to all Rust callers:
void SafeMethod();
// Internal helper: generated with `pub(crate)` visibility in Rust,
// making it accessible only to custom `srcs` of this `rust_api_from_cpp`:
CRUBIT_PUB_CRATE void InternalHelper();
};
// Free function limited to this crate:
CRUBIT_PUB_CRATE void HelperFunction();
In the generated Rust bindings, functions and methods marked with
CRUBIT_PUB_CRATE receive pub(crate) visibility instead of pub. This allows
custom Rust code in srcs to call them while hiding them from external crates
that depend on this rust_api_from_cpp target.
Note:
CRUBIT_PUB_CRATEcan only be used on functions and methods. Using it on types (structs, classes, enums, unions, or type aliases) results in a compile-time error.
Read on only if you’re curious about why Rust bindings targets are structured this way.
Technical Justification
Crubit generates bindings using Bazel aspects: given an arbitrary C++ Bazel target, Crubit generates, in an aspect, the Rust library which wraps it.
This is necessary for the same reason that it’s necessary for protocol buffers: to support transitive dependencies (if A depends on B, then bindings(A) must depend on bindings(B) so that bindings(A) can use types from B).
However, unlike protocol buffers which use a single target for both languages,
Crubit uses a separate rust_api_from_cpp target to represent the Rust
bindings. This target is where library owners configure the bindings (for
example, by adding custom Rust code which should be added to the generated
target).
To configure the Crubit code generation for a C++ target, we use aspect_hints.
The rust_api_from_cpp macro automatically generates a .hint target (named
<name>.hint) which contains the configuration. This hint target is then
attached to the C++ target’s aspect_hints.
Example
To enable Crubit on a C++ target, define a rust_api_from_cpp target and add
its .hint target to the C++ target’s aspect_hints:
load(
"//bazel:rust_api_from_cpp.bzl",
"rust_api_from_cpp",
)
cc_library(
name = "my_lib",
hdrs = ["my_lib.h"],
aspect_hints = [":my_lib_rust.hint"],
)
rust_api_from_cpp(
name = "my_lib_rust",
cpp_target = ":my_lib",
)
FAQ: How do separate rules work without cycles?
A library A, and its bindings bindings(A), must be linked together in the
build graph: if B uses a type from A, then bindings(B) uses a type from
bindings(A).
If we were to point my_lib directly to my_lib_rust via aspect_hints, and
my_lib_rust depended on my_lib (to trigger bindings generation), we would
create a dependency cycle: my_lib_rust -> my_lib -> my_lib_rust.
To avoid this, the rust_api_from_cpp macro splits the definition into two
parts:
- A configuration target (
my_lib_rust.hint) which contains the configuration but does not depend onmy_lib. - A bindings target (
my_lib_rust) which depends onmy_lib.
The cc_library target (my_lib) then references the configuration target
(my_lib_rust.hint) in its aspect_hints.
The dependency flow is: my_lib_rust -> my_lib -> my_lib_rust.hint
Since my_lib_rust.hint does not depend on my_lib_rust or my_lib, the cycle
is broken.
FAQ: Why are there extra dependencies in deps(target)?
Because the Rust bindings are created using an aspect on the C++ target,
everything that the Rust bindings need to depend on will appear in a Bazel query
/ depserver query for deps(target).
For example, if you wanted to add some extra source file to the Rust bindings,
you might specify them in aspect_hints. This file will show up in
deps(target).
These Rust-only deps are not used at all in pure-C++ builds (the Bazel actions registered by them won’t be executed), but they will show up in the dependency graph anyway, due to how Bazel query and depserver track dependencies.
NOTE: In particular, if your project has tests that count/limit the transitive dependencies of a C++ binary, they will overcount the dependencies, and the overcounting will get worse as Rust support is rolled out through the C++ build graph.
Wrapping and type bridging vs direct use of types
Crubit automatically generates layout-compatible Rust equivalents of C++ types.
When the C++ type is Rust-movable, the
Crubit-generated Rust type is Rust-movable, these can be used by value, by
pointer, in struct fields, arrays, and any other compound data type. A C++
pointer const T* can become a Rust *const T, and a C++ T field can become
a Rust T field, and so on, with few restrictions.
For example, the following C++ type:
struct Vec2d {
float x;
float y;
};
Becomes (roughly) the following Rust type:
#![allow(unused)]
fn main() {
#[repr(C)]
struct Vec2d {
pub x: f32,
pub y: f32,
}
}
These have an identical layout, and so a C++ pointer or field containing a C++
Vec2d is exactly equivalent to a Rust pointer or field containing a Rust
Vec2d.
(See Types for more information about layout-compatibility.)
Because of this, it is often not required to manually write any new types. The bindings generated by Crubit will produce a working type automatically.
When to wrap a type
There are, still, a handful of reasons to manually write “wrapper” types which encapsulate or replace the original C++ type (or its Crubit-generated Rust type).
-
If the type is not naturally Rust-movable, but it’s important for the Rust type to be Rust-movable. It may be possible to make changes to the C++ code to make the type Rust-movable using some of the strategies described in the cookbook. This allows the greatest flexibility, as the type becomes usable in almost every context. But if that is not possible, writing a new “wrapper” type can keep Rust programmers productive.
-
Some Rust types have very special semantics, which are impossible to implement in the bindings for a C++ type. For example, Rust has special support for
ResultandOptionin error handling via the?operator, which cannot yet be implemented byStatusorstd::optionalusing stable Rust features. These privileged Rust types can be used instead of the equivalent C++ types, as a wrapper type. -
The type is simply not supported/supportable in Rust, and needs a wrapper as a workaround. (See also: crubit.rs/errors/unsupported_type.)
In these cases, Crubit may bridge to a wrapper type as a workaround, while we hopefully fix the underlying issues that mean we cannot directly use the underlying type. This offers us a subset of the API we want, and allows continued progress.
Why not to wrap a type
Wrapper types work best when passed by value: if you return a T in C++, the
corresponding Rust function can automatically convert it to and return a
WrappedT.
However, no conversion is possible for references or fields, which really are
the original type, with its size and alignment and address in memory - to make
this work transparently requires an ever-expanding network of wrapper types, one
for every compound data type that might contain T:
Tmust becomeWrappedTconst T&, if it is supported at all, must become something likeTRef<'a>, or a dynamically sized&TView.std::vector<T>, if it is supported at all, must become something likeTVector.struct MyStruct {T x;}must become a wrappedWrappedMyStruct.- …
The problems introduced by wrapper types can easily outweigh the benefits that they bring. Crubit aims to reduce their necessity to zero over time.
Bad reasons to wrap a type
In most other circumstances where one might want to reach for wrapper types, alternatives exist:
-
If we want to use a wrapper type in order to give the type a nicer Rust API, then, as an alternative, one can customize the Rust API of the wrapped type using an aspect hint. You can define new methods and trait implementations to the side, without altering any C++ code.
-
If we want to use a wrapper type in order to change the type invariants – to make them stricter or looser – this is fine, as long as it doesn’t replace the not-as-nice type. For example, if a C++ API returns
std::string(bytes, “probably” UTF-8), the Rust equivalent should not return a RustString(Unicode, definitely UTF-8). Changing type invariants in-place causes some APIs to become impossible to call, and causes the Rust and C++ ecosystems to diverge and become incompatible. The bindings should be high fidelity. Wrapper types of this form should be optional, and available equally to both C++ and Rust to avoid fragmenting the ecosystem.
Fidelity
Anything possible in C++ should be possible in Rust. See
The Rust API for a given C++ API should not try to make the interface “better” at more than a superficial level, because it can compromise the ability of other teams to write new Rust code, or port existing C++ code to Rust.
Good changes:
- Changing method names, especially to names that Rust callers might expect.
For example, changing
Status::ok()(C++) toStatus::is_ok()(Rust) – Rust callers expect many of these boolean functions to be prefixed withis_. - Adding new APIs that Rust users expect. For example, trait implementations
that allow the type to better interoperate with the Rust ecosystem, or
functions which accept a
Pathor&strin addition to a raw C++string_view. - Reifying C++ comments around lifetime or safety as actual lifetime
annotations or
unsafedeclarations.
If the Rust type is outright unnatural to use, people won’t use it, and it’s worse for the ecosystem to have two APIs than one API.
Bad changes:
- Removing deprecated APIs which still have C++ callers.
- Placing new requirements on Rust callers that were not placed on C++ callers, such as requiring UTF-8 when C++ does not.
Customizing bindings using annotations
The Rust bindings for a C++ declaration can be customized using an attribute
macro from "support/annotations.h".
For instance:
- A function can be marked
unsafein Rust, even if Crubit would otherwise assume it was safe, usingCRUBIT_UNSAFE. - Missing bindings for an item can be treated as an error, instead of ignored,
using
CRUBIT_MUST_BIND. - An item can be given a different name in Rust using
CRUBIT_RUST_NAME("rust_name_here"). - A function or method can be given
pub(crate)visibility in Rust, limiting its visibility to the generated crate (including customsrcsonrust_api_from_cpp), usingCRUBIT_PUB_CRATE.
More information:
- Dependency:
//support:annotations - Include:
#include "support/annotations.h" - Full API documentation: support/annotations.h
Example
Given the following C++ header:
CRUBIT_UNSAFE inline void SafeSignatureButAnnotatedUnsafe() {}
Crubit will generate the following bindings:
/// # Safety
///
/// The C++ function is explicitly annotated as unsafe. Ensure that its safety requirements are upheld.
#[inline(always)]
pub unsafe fn SafeSignatureButAnnotatedUnsafe() {
unsafe { crate::detail::__rust_thunk___Z31SafeSignatureButAnnotatedUnsafev() }
}
Rust bindings for C++ functions
Rust code can call (non-member) functions defined in C++, provided that the parameter and return types are supported by Crubit:
- If a parameter or return type is a primitive type, then the bindings for the function use the corresponding Rust type.
- Similarly, if a parameter or return type is a pointer type, then the bindings for the function use the corresponding Rust pointer type.
- If a parameter or return type is a C++ reference type, then the bindings for the function use a Rust reference (if the lifetime was either explicitly specified or elided and inferred) or a Rust pointer (otherwise). See References and lifetimes.
- If the type is a user-defined type, such as a class type or enum, then the bindings for the function use the bindings for that type.
Additionally, code can call member functions defined in C++ if the parameter and return types are supported by Crubit (see above). Non-static member functions become Rust methods, and static member functions become associated functions.
Examples
Functions
Given the following C++ header:
inline int32_t add_two_integers(int32_t x, int32_t y) { return x + y; }
Crubit will generate the following bindings, with a safe public function that calls into the corresponding FFI glue:
#[inline(always)]
pub fn add_two_integers(x: i32, y: i32) -> i32 {
unsafe { crate::detail::__rust_thunk___ZN5gshoe16add_two_integersEii(x, y) }
}
Which can be used like this:
fn main() {
let sum = example_lib::gshoe::add_two_integers(2, 2);
println!("sum = {sum}");
}
Methods
Given the following C++ header:
struct Bar {
int x;
void MyMethod() {};
};
Crubit will generate the following bindings:
#[derive(Clone, Copy, ::ctor::MoveAndAssignViaCopy)]
#[cfi_encoding = "N3foo3BarE"]
#[repr(C)]
///CRUBIT_ANNOTATE: cpp_type=foo :: Bar
///CRUBIT_ANNOTATE: cpp_move_constructible=
pub struct Bar {
pub x: ::ffi_11::c_int,
}
impl Bar {
#[inline(always)]
pub fn MyMethod<'__this>(&'__this mut self) {
unsafe { self::bar::MyMethod(self) }
}
}
Which can be used like this:
fn main() {
let mut x = Bar { x: 1 };
x.MyMethod();
}
The this parameter of a method becomes &self or &mut self. See References
and lifetimes for more details.
unsafe functions
Which C++ functions are marked unsafe in Rust?
By default, the Rust binding to a C++ function is marked as safe or unsafe
based on the types of its parameters. If a C++ function accepts only simple
types like integers, the resulting Rust binding will be marked as safe.
Functions which accept a raw pointer are automatically marked as unsafe.
This behavior can be overridden using the CRUBIT_UNSAFE,
CRUBIT_UNSAFE_MARK_SAFE and CRUBIT_OVERRIDE_UNSAFE(is_unsafe) macros.
For example, given the following C++ header:
inline void SafeSignatureWithoutAnnotation() {}
CRUBIT_UNSAFE inline void SafeSignatureButAnnotatedUnsafe() {}
inline void UnsafeSignatureWithoutAnnotation(void*) {}
CRUBIT_UNSAFE_MARK_SAFE inline void UnsafeSignatureButAnnotatedSafe(void*) {}
CRUBIT_OVERRIDE_UNSAFE(/*is_unsafe=*/false) inline void SafeBasedOnBoolean() {}
CRUBIT_OVERRIDE_UNSAFE(/*is_unsafe=*/true) inline void UnsafeBasedOnBoolean() {}
Crubit will generate the following bindings:
#[inline(always)]
pub fn SafeSignatureWithoutAnnotation() {
unsafe { crate::detail::__rust_thunk___Z30SafeSignatureWithoutAnnotationv() }
}
#[inline(always)]
pub unsafe fn SafeSignatureButAnnotatedUnsafe() {
unsafe { crate::detail::__rust_thunk___Z31SafeSignatureButAnnotatedUnsafev() }
}
#[inline(always)]
pub unsafe fn UnsafeSignatureWithoutAnnotation(__param_0: *mut ::ffi_11::c_void) {
unsafe { crate::detail::__rust_thunk___Z32UnsafeSignatureWithoutAnnotationPv(__param_0) }
}
#[inline(always)]
pub fn UnsafeSignatureButAnnotatedSafe(__param_0: *mut ::ffi_11::c_void) {
unsafe { crate::detail::__rust_thunk___Z31UnsafeSignatureButAnnotatedSafePv(__param_0) }
}
#[inline(always)]
pub fn SafeBasedOnBoolean() {
unsafe { crate::detail::__rust_thunk___Z18SafeBasedOnBooleanv() }
}
#[inline(always)]
pub unsafe fn UnsafeBasedOnBoolean() {
unsafe { crate::detail::__rust_thunk___Z20UnsafeBasedOnBooleanv() }
}
mod detail {
#[allow(unused_imports)]
use super::*;
unsafe extern "C" {
pub(crate) unsafe fn __rust_thunk___Z30SafeSignatureWithoutAnnotationv();
pub(crate) unsafe fn __rust_thunk___Z31SafeSignatureButAnnotatedUnsafev();
pub(crate) unsafe fn __rust_thunk___Z32UnsafeSignatureWithoutAnnotationPv(
__param_0: *mut ::ffi_11::c_void,
);
pub(crate) unsafe fn __rust_thunk___Z31UnsafeSignatureButAnnotatedSafePv(
__param_0: *mut ::ffi_11::c_void,
);
pub(crate) unsafe fn __rust_thunk___Z18SafeBasedOnBooleanv();
pub(crate) unsafe fn __rust_thunk___Z20UnsafeBasedOnBooleanv();
}
}
Which can be used like this:
#[deny(unused_unsafe)]
fn main() {
// Can safely invoke functions that accept simple types or are marked safe.
example_lib::SafeSignatureWithoutAnnotation();
example_lib::UnsafeSignatureButAnnotatedSafe(core::ptr::null_mut());
example_lib::SafeBasedOnBoolean();
// `unsafe` is needed to invoke functions that accept raw pointers or are marked
// unsafe.
unsafe {
example_lib::UnsafeSignatureWithoutAnnotation(core::ptr::null_mut());
example_lib::SafeSignatureButAnnotatedUnsafe();
example_lib::UnsafeBasedOnBoolean();
}
}
Correct usage of unsafe
Functions marked unsafe cannot be called outside of an unsafe block. In
order to avoid undefined behavior when using unsafe, callers must:
-
Ensure that the pointer being passed to C++ is a valid C++ pointer. In particular, it must not be dangling (e.g.
Nonnull::dangling()). -
Ensure that the safety conditions documented in C++ are upheld. For example, if the C++ function accepts a reference or non-null pointer, then do not pass in
0 as *const _.
Soundness
Note that many “safe” C++ functions may still trigger undefined behavior if used
incorrectly. Regardless of whether a C++ function is marked as unsafe, calls
into C++ will only be memory-safe if the caller verifies that all function
preconditions are met.
Function Visibility: pub(crate)
By default, bindings for C++ functions and methods are generated with pub
visibility in Rust.
You can restrict the visibility of a function or method to the generated crate
using CRUBIT_PUB_CRATE from "support/annotations.h":
#include "support/annotations.h"
CRUBIT_PUB_CRATE void HelperFunction();
class MyClass {
public:
CRUBIT_PUB_CRATE void InternalMethod();
};
This is especially useful when providing handwritten Rust APIs via the srcs
attribute of rust_api_from_cpp (see Best
Practices). The
custom Rust code in srcs can call the pub(crate) functions or methods, while
keeping them hidden from downstream crates.
CRUBIT_PUB_CRATE is only valid on functions and methods; applying it to types
will produce an error.
Function Attributes
Most function attributes are not supported. For example, functions marked
[[noreturn]] don’t get bindings. [[nodiscard]] is supported and becomes
#[must_use].
Rust bindings for C++ operators and special member functions
Here we describe how Crubit bindings work with C++ special member functions and
operator overloading (e.g. the copy constructor, or operator==) and with
traits from the Rust standard library (e.g., the Clone or PartialEq traits).
Rust traits which return Self by value (e.g., Default or Clone) or use
mutable references (e.g., AddAssign) are only implemented in Rust bindings for
a given C++ type if that type is Unpin.
Bidirectional map
The following special member functions and traits are mapped bidirectionally:
| C++ | Rust |
|---|---|
| Default constructor | Default |
| Trivial copy constructor | Copy |
| Non-trivial copy constructor | Clone |
| Destructor | Drop |
Note
Rust bindings for a C++ type only implement
Copyif the C++ type is non-abstract and has a public, trivial copy constructor and destructor.
One-way map of C++ special member functions into Rust traits
If the C++ type is Unpin, then the C++ special member
functions below are mapped one-way into the corresponding Rust traits as
follows:
| C++ API | Rust bindings |
|---|---|
Constructor taking a single T parameter | From<T> |
Note
This applies regardless of whether the constructor is
explicitin the C++ API.
The C++ binary operators below are mapped one-way into the corresponding Rust traits as follows:
| C++ API | Rust bindings |
|---|---|
operator== | PartialEq |
operator<=> | PartialOrd (and opt-in Eq + Ord for std::strong_ordering) |
operator< | PartialOrd |
operator> | PartialOrd |
operator<= | PartialOrd |
operator>= | PartialOrd |
operator+ | Add |
operator- | Sub |
operator* | Mul |
operator/ | Div |
operator% | Rem |
operator& | BitAnd |
operator| | BitOr |
operator^ | BitXor |
operator<< | Shl |
operator>> | Shr |
operator+= | AddAssign |
operator-= | SubAssign |
operator*= | MulAssign |
operator/= | DivAssign |
operator%= | RemAssign |
operator&= | BitAndAssign |
operator|= | BitOrAssign |
operator^= | BitXorAssign |
operator<<= | ShlAssign |
operator>>= | ShrAssign |
The C++ unary operators below are mapped one-way into the corresponding Rust traits as follows:
| C++ API | Rust bindings |
|---|---|
operator- | Neg |
operator! | Not |
PartialOrd
Rust only permits a single PartialOrd impl for a given pair of operand types,
so Crubit picks one of the comparison operators (<=>, <, >, <=, >=) to
back the impl, in that order of preference. The remaining comparison operators
are not bound directly; they are served by the default PartialOrd methods,
which are derived from partial_cmp.
operator<=> (the “spaceship” operator) implements partial_cmp directly. It
must return std::strong_ordering, std::weak_ordering, or
std::partial_ordering (std::partial_ordering::unordered maps to None). For
the other operators, Crubit synthesizes partial_cmp from the operator together
with operator==.
A PartialOrd impl is only generated if operator== is also available for the
same operand types, and if the left- and right-hand operands have the same type.
Eq and Ord
If operator<=> returns std::strong_ordering (and Crubit implements
PartialOrd with it), then Crubit also implements Eq and Ord. This lets you
use the type in a BTreeMap or BTreeSet, and sort it with sort().
For now, this requires the strong_ordering_traits Crubit
feature ,
which is also part of experimental. This lets existing Crubit users migrate
gradually, if they have manual Eq or Ord impls that conflict with the
generated impls.
Crubit doesn’t implement Eq and Ord for std::weak_ordering and
std::partial_ordering.
Operators inherited from base classes
C++ operators are sometimes defined as “hidden friends” of a base class, for example by a CRTP mixin which implements comparison operators for the derived class:
template <typename T>
struct EqualityMixin {
friend bool operator==(const T& lhs, const T& rhs) { ... }
};
struct S final : EqualityMixin<S> { ... };
Argument-dependent lookup finds such operators for S, even though S doesn’t
declare them itself. To map them into Rust traits as well (e.g. PartialEq for
S), enable the inherited_friend_operators Crubit
feature
on the target that defines S. Only operators whose first parameter is (a
reference to) S are mapped.
This is opt-in, because the generated trait implementations conflict with any existing manual implementations of the same traits.
One-way map into Display
By default: for a C++ type T, Crubit maps one-way the following C++ signatures
into the Rust trait Display:
template <typename Sink> void AbslStringify(Sink&, const T&)template <typename Sink> void AbslStringify(Sink&, T)std::ostream& operator<<(std::ostream&, const T&)std::ostream& operator<<(std::ostream&, T)
Crubit prefers
AbslStringify but falls
back to operator<<.
The attribute macro CRUBIT_OVERRIDE_DISPLAY forces whether
to implement the Display binding. See the macro’s documentation:
// Overrides the `Display` binding detection for a type to true or false.
//
// If detected: binds to Rust's `Display` trait, preferring `AbslStringify` over
// `std::ostream&` `operator<<` but requiring either.
//
// By default, infers a type `T`'s formatability based on one of:
// * `template <typename Sink> void AbslStringify(Sink&, const T&)`
// * `template <typename Sink> void AbslStringify(Sink&, T)`
// * `std::ostream& operator<<(std::ostream&, const T&)`
// * `std::ostream& operator<<(std::ostream&, T)`
// * This annotation on `T`'s bases
//
// in any of the following:
// * `T`'s namespace
// * `T`'s friends
// * `T`'s bases' namespace and friends
//
// and recurses on the bases of `T`. e.g., if `T` inherits from `B`, then
// looks for both `AbslStringify(Sink&, T)` and `AbslStringify(Sink&, B)` in
// `B`.
//
// However, default inference does *not* handle all cases, including:
// * Other function or function template signatures e.g.,
// ```c++
// template <typename Sink, typename T>
// void AbslStringify(Sink&, const Foo<T>&);
//
// template <typename T>
// struct Bar {
// template <
// typename U = T,
// typename = std::enable_if_t<absl::HasOstreamOperator<U>::value>>
// friend std::ostream& operator<<(std::ostream&, const Bar&);
// };
//
// template <typename T>
// struct Baz {
// friend std::ostream& operator<<(std::ostream&, const Bar&)
// requires(absl::HasOstreamOperator<T>::value);
// };
//
// struct Qux {
// template <typename T, typename Traits>
// friend std::basic_ostream<T, Traits>& operator<<(
// std::basic_ostream<T, Traits>&, const Qux&);
// };
// ```
// * Formattable but non-public bases
// * Deleted functions
#define CRUBIT_OVERRIDE_DISPLAY(should_bind) \
CRUBIT_INTERNAL_ANNOTATE("crubit_override_display", should_bind)
Callbacks: absl::AnyInvocable
Crubit supports calling C++ functions that accept or return
absl::AnyInvocable
from Rust.
In Rust, absl::AnyInvocable parameters and return types map to standard Rust
trait objects (Box<dyn Fn...> and Box<dyn FnOnce...>).
For the reverse direction (calling Rust functions that take closures from C++), see C++ bindings for Rust callbacks.
BUILD configuration
To generate Rust bindings for a C++ library that uses absl::AnyInvocable, ensure
the cc_library depends on @abseil-cpp//absl/functional:any_invocable and is opted
into Crubit using rust_api_from_cpp:
cc_library(
name = "cpp_api",
srcs = ["cpp_api.cc"],
hdrs = ["cpp_api.h"],
aspect_hints = [":cpp_api_rust.hint"],
deps = [
"@abseil-cpp//absl/functional:any_invocable",
],
)
rust_api_from_cpp(
name = "cpp_api_rust",
cpp_target = ":cpp_api",
)
rust_test(
name = "user_of_cpp_api",
srcs = ["user_of_cpp_api.rs"],
deps = [
":cpp_api_rust",
"//third_party/gtest_rust/googletest",
],
)
The complete, tested version is in
examples/types/callbacks/BUILD.
Type mapping
Crubit maps absl::AnyInvocable to heap-allocated Rust trait objects:
C++ absl::AnyInvocable signature | Rust binding type |
|---|---|
absl::AnyInvocable<R(Args...) const> | Box<dyn Fn(Args...) -> R + Send + Sync> |
absl::AnyInvocable<R(Args...) &&> | Box<dyn FnOnce(Args...) -> R + Send> |
absl::AnyInvocable<R(Args...)> (non-const) | Box<dyn Fn(Args...) -> R + Send + Sync> |
Non-const absl::AnyInvocable is promoted to Fn
Non-const absl::AnyInvocable<R(Args...)> is mapped to Box<dyn Fn...> (rather than
FnMut) in Rust.
In C++, invoking a non-const absl::AnyInvocable does not require exclusive
mutable access to the function object. Rust’s FnMut, by contrast, requires an
exclusive &mut borrow that would be unsound if called concurrently or re-entrantly
from C++. Mapping to Fn guarantees that the callback is safe to invoke in any
context.
If you need to mutate state across invocations in a Rust closure passed to a non-const
absl::AnyInvocable, use interior mutability types such as Atomic* (e.g. AtomicI32),
Cell, RefCell, or Mutex instead of &mut capture.
Passing callbacks from Rust to C++
C++ functions can accept absl::AnyInvocable by value:
// In cpp_api.h:
#include <cstdint>
#include "third_party/absl/functional/any_invocable.h"
// Accepts a callback that may be called multiple times:
int32_t ApplyMultiplier(absl::AnyInvocable<int32_t(int32_t) const> callback,
int32_t value);
// Accepts a single-use callback:
void InvokeOnce(absl::AnyInvocable<void() &&> callback);
In Rust, pass a boxed closure (Box::new(...)):
#![allow(unused)]
fn main() {
// In user_of_cpp_api.rs:
// 1. Calling a const callback:
let result = cpp_api::ApplyMultiplier(Box::new(|x: i32| x * 2), 21);
assert_eq!(result, 42);
// 2. Calling a single-use (FnOnce) callback:
let data = String::from("resource");
cpp_api::InvokeOnce(Box::new(move || {
println!("Consumed: {data}");
}));
}
Returning callbacks from C++ to Rust
C++ functions can return absl::AnyInvocable:
// In cpp_api.h:
absl::AnyInvocable<int32_t(int32_t) const> MakeAdder();
In Rust, the returned value is a callable trait object that can be called directly:
#![allow(unused)]
fn main() {
let adder = cpp_api::MakeAdder();
assert_eq!(adder(41), 42);
}
Type aliases
Type aliases to absl::AnyInvocable in C++ receive corresponding type alias
bindings in Rust:
using IntMapper = absl::AnyInvocable<int32_t(int32_t) const>;
int32_t ApplyAlias(IntMapper callback, int32_t value);
In Rust, cpp_api::IntMapper is an alias for
Box<dyn Fn(i32) -> i32 + Send + Sync>.
Limitations
std::functionis not bound: Crubit does not generate bindings for functions taking or returningstd::function. Replacestd::functionwithabsl::AnyInvocablein your C++ API, or write a thin C++ wrapper takingabsl::AnyInvocable.
Rust bindings for C++ classes and structs
A C++ class or struct is mapped to a Rust struct with the same fields. If
any subobject of the class cannot be represented in Rust, the class itself will
still have bindings, but the relevant subobject will be
private.
Example
Given the following C++ header:
struct Position {
int x;
int y;
};
Crubit will generate a struct with the same layout:
#[derive(Clone, Copy, ::ctor::MoveAndAssignViaCopy)]
#[cfi_encoding = "8Position"]
#[repr(C)]
///CRUBIT_ANNOTATE: cpp_type=Position
///CRUBIT_ANNOTATE: cpp_move_constructible=
pub struct Position {
pub x: ::ffi_11::c_int,
pub y: ::ffi_11::c_int,
}
Which can be used like this:
fn main() {
// Can pass this to functions accepting a `Position`, and so on.
let _ = example_lib::Position { x: 1, y: 2 };
}
For an example of a Rust-movable class with a destructor, see examples/cpp/trivial_abi_struct/.
Fields
The fields on the Rust struct type are the corresponding Rust types:
- If the C++ field has primitive type, then the Rust field uses the corresponding Rust type.
- Similarly, if the C++ field has pointer type, then the Rust field has the corresponding Rust pointer type.
- If the field has a user-defined type, such as a class type or enum, then the Rust field uses the bindings for that type.
Unsupported fields
Subobjects that do not receive bindings are made private, and replaced with an
opaque blob of [MaybeUninit<u8>; N], as well as a comment in the generated
source code explaining why the subobject could not receive bindings. For
example, base class subobjects are not exposed as fields, so the space of the
object occupied by a base class will instead be this opaque blob of bytes, with
no comment. (See Base classes for what you can do with base
classes instead.)
Specifically, the following subobjects are hidden and replaced with opaque blobs:
- Base class subobjects
- Non-
publicfields (privateorprotectedfields) - Fields that have nontrivial destructors
- Fields whose type does not have bindings
- Fields that have any unrecognized attribute, including
no_unique_address - Fields of a packed struct whose Rust type has
#[repr(align(N))]
A Rust struct with opaque blobs is ABI-incompatible with the C++ struct or class that it corresponds to. As a consequence, if the struct is used for FFI outside of Crubit, it should not be passed by value. Within Crubit, it can’t be passed by value in function pointers, but can otherwise be used as normal.
Static data members
Public static data members get bindings: compile-time integer, bool, integer
array, and string_view constants become associated consts of the Rust struct,
and other static data members become Rust statics. See Static data
members for details.
Rust-movable classes
The easiest C++ classes to work with are “Rust-movable”, meaning they support
being relocated in memory using memcpy without running the move constructor.
When passed and returned by value, Rust-movable classes will use the direct
type. For example, T Identity(T) becomes pub fn Identity(_: T) -> T.
If it is non-Rust-movable, then Crubit will instead use in-place initialization
types, Ctor, and requires a bit of syntactic overhead when moving objects
around. The same function might instead become pub fn Foo(_: Ctor![T]) -> Ctor![T].
(For an introduction to Ctor![T], see
crubit.rs/types/non_rust_movable/intro_short.)
Types are considered Rust-movable by default, meaning they can be relocated
using memcpy. If the type defines a destructor or copy/move constructor, then
it requires a special annotation to be considered Rust-movable:
ABSL_ATTRIBUTE_TRIVIAL_ABI.
If it has a non-Rust-movable field or base class, then it is not Rust-movable.
It can be worth putting some effort into designing the type to be Rust-movable. crubit.rs/cpp/cookbook#rust_movable describes, in more detail, how to go about this.
Some examples of Rust-movable types:
- any primitive type (integers, character types, floats, etc.)
- raw pointers
string_viewstruct tm, or any other type in the C standard libraryunique_ptrandshared_ptr, in the Clang unstable ABI.absl::Status
Some examples of types that are not Rust-movable:
- (For now)
std::string,std::vector, and other nontrivial standard library types. - (For now)
absl::flat_hash_map,absl::AnyInvocable, and other nontrivial types used throughout the C++ ecosystem, even outside the standard library. absl::Mutex,absl::Notification, and other non-movable types.
Base classes
Rust has no inheritance, so a derived class and its base classes become
unrelated Rust structs. The base class subobject is an opaque
blob, so base class fields aren’t accessible through the
derived struct. With oo_casting, Rust code can upcast to
the base class and read the fields there.
Inherited methods
By default, methods that a class inherits from its base classes are not part of the derived class’s bindings, and no comment is emitted about them. They still exist on the base class’s bindings.
struct Base {
bool IsValid() const;
};
struct Derived : Base {};
Here, Base::IsValid() has bindings, but Derived has no IsValid() method in
Rust.
To get inherited methods, add //features:oo_casting to the
aspect_hints of the cc_library that defines the derived class. This is an
experimental feature. With it:
-
The derived struct gets the public methods, including static methods, of its public base classes, so
derived.IsValid()works in Rust. This only covers base classes defined in the samecc_libraryas the derived class, and leaves out methods whose name comes from more than one base class. -
The derived struct implements
oops::Inherits<Base>for each public base class, including base classes from othercc_librarytargets. Rust code can then use theoops::Upcasttrait to convert a&Derived,*const Derived, or*mut Derivedto the base class type. For example, to call a method of a base class in anothercc_library, upcast first:#![allow(unused)] fn main() { use oops::Upcast as _; // The type annotation is required. let base: &Base = unsafe { derived.upcast() }; }For a
virtualbase class, useoops::VirtualUpcaston a raw pointer instead. Rust code that uses these traits depends on//support:oops.
The tests in
rs_bindings_from_cc/test/struct/inheritance/
show both.
Without oo_casting, add a method to the derived class in C++ that forwards to
the base class method, or a function that returns the object as a pointer to its
base class.
using Base::Method; declarations in the derived class don’t get bindings, and
produce a “Function aliases are not yet supported” error comment. With
oo_casting, the method is usually inherited anyway. It’s missing only if the
using declaration makes a method of a non-public base class accessible, or
brings back a base class method that a derived class method of the same name
hides.
Virtual methods
Calling a C++ virtual method from Rust works like calling any other method, and
dispatches virtually. This includes pure virtual methods. Because a class with
virtual methods is not Rust-movable, non-const methods take
self: Pin<&mut Self>.
An abstract class (one with pure virtual methods that aren’t overridden) gets bindings for its methods but no constructor, so Rust code works with it through references and pointers to objects created elsewhere. A subclass that overrides the methods gets bindings for its overrides and can be constructed. The methods it doesn’t override are inherited methods.
Overriding a C++ virtual method in Rust, or otherwise implementing a C++
interface in Rust, is not supported. The workaround is a C++ subclass that
forwards to a Rust type; see
examples/cpp/virtual/.
Packed structs
A packed C++ struct becomes a packed Rust struct with the same layout:
__attribute__((packed))(orABSL_ATTRIBUTE_PACKED) on the struct becomes#[repr(C, packed)].- A struct defined under
#pragma pack(N)becomes#[repr(C, packed(N))]. If the pragma doesn’t lower the alignment of any field or base class, it doesn’t change the layout, and the struct is treated like any other struct.
For example:
struct __attribute__((packed)) PackedStruct {
uint8_t a;
uint32_t b; // At offset 1.
uint16_t c; // At offset 5.
};
becomes a 7-byte Rust struct with alignment 1:
#![allow(unused)]
fn main() {
#[repr(C, packed)]
pub struct PackedStruct {
pub a: u8,
pub b: u32,
pub c: u16,
}
}
Some packed structs don’t get bindings:
- Structs with
__attribute__((packed))on individual fields. - Packed structs with an explicit alignment (such as
alignasor__attribute__((aligned(N)))) greater than the packing alignment.
A field of a packed struct whose Rust type has #[repr(align(N))], directly or
in one of its own fields, becomes a private opaque blob,
because Rust doesn’t allow such types in packed structs. A struct type gets
#[repr(align(N))] if, for example, it has an explicit alignment, or it has an
alignment greater than 1 and has bitfields or fields whose type doesn’t have
bindings.
Note: Clang doesn’t pack fields whose type isn’t POD for the purpose of layout
(for example, a type with a user-defined copy constructor) and warns about this
with -Wpacked-non-pod. If such a field has an alignment greater than 1, the
Rust bindings won’t compile, because the Rust struct packs every field.
Using fields of packed structs
Fields of a packed struct may be unaligned, so Rust doesn’t allow references to
them (E0793). Reading a
field by value (let b = s.b;) works, but anything that implicitly takes a
reference, such as println!("{}", s.b) or s.b.to_string(), doesn’t compile.
Copy the field out first, for example with braces:
#![allow(unused)]
fn main() {
println!("{}", { s.b });
let text = { s.b }.to_string();
}
Assigning to a field (s.b = 5;) works as usual. For fields that aren’t Copy,
use raw pointers (&raw const s.field or &raw mut s.field) with
ptr::read_unaligned and ptr::write_unaligned.
Debug for packed structs
Crubit ignores CRUBIT_TRAIT_DERIVE(Debug) on packed structs, because
#[derive(Debug)] takes references to the fields.
Instead, with the experimental //features:record_impl_debug
feature, packed structs implement Debug like other structs, with these
differences:
- Each field is copied out before it’s printed. If a field that would be
printed isn’t
Copy(or ismutable, and thus wrapped in aCell), no fields are printed, and the output isPackedStruct { .. }, like for a union. - Base classes aren’t printed, since they may be unaligned too.
For example, with the struct above, format!("{s:?}") is
PackedStruct { a: 10, b: 200, c: 3000 }. The tests in
rs_bindings_from_cc/test/struct/packed/
show more examples.
Attributes
Crubit does not support most attributes on structs and their fields. If a struct
is marked using any attribute other than alignment, packing, or
ABSL_ATTRIBUTE_TRIVIAL_ABI, it will not receive bindings. If a field is marked
using any other attribute, it will be replaced with a private opaque blob.
Rust bindings for C++ static data members
Public static data members of a class receive bindings. Which kind depends on the member’s type and whether its value is a compile-time constant:
- A member of integer,
bool, integer array, orstd::string_view/absl::string_viewtype whose value is a compile-time constant (because it isconstexpr, orconstorinlinewith a constant initializer) becomes an associatedconstof the Rust struct. Its value is copied into the bindings, so Rust doesn’t need to access the C++ object. - A member whose value is not a compile-time constant (for example, one that
is only declared in the class and is defined in a
.ccfile) becomes a Ruststaticthat refers to the C++ object, because its value is only known at run time. Rust has no associated statics, so it is placed in the snake-cased module that holds the class’s other nested items. If it is mutable, it is astatic mut, and, as with other C++ globals, accessing it isunsafe.
For example, given:
struct Counter {
static constexpr int kMax = 10;
static const bool kEnabled = true;
static int count; // Defined in counter.cc.
};
Crubit generates (simplified):
#![allow(unused)]
fn main() {
impl Counter {
pub const kMax: ffi_11::c_int = ffi_11::new_c_int(10);
pub const kEnabled: bool = true;
}
pub mod counter {
unsafe extern "C" {
pub static mut count: ffi_11::c_int;
}
}
}
Compile-time constants of other types, such as floating-point or class types, and static data members of class template specializations don’t receive bindings yet.
Rust bindings for C++ references and lifetimes
Rust references have lifetimes, which let the Rust compiler check that a
reference doesn’t outlive the value it refers to. C++ references don’t have
lifetimes. To generate Rust bindings for C++ functions and methods that take or
return C++ references (T& or const T&), Crubit needs to know their
lifetimes. Crubit gets them from:
- Rust’s lifetime elision rules, or
- Explicit lifetime annotations in the C++ code.
This page also explains what happens if lifetime elision doesn’t
apply or gives wrong lifetimes, and describes
the CRef and CMut types that Crubit uses for references
returned from C++.
NOTE: C++ pointers don’t have lifetimes in the generated bindings. They map to Rust raw pointers. See Pointer types.
Lifetime elision
Crubit applies Rust’s lifetime elision rules to C++ references. For example, given the following C++ header:
struct Counter final {
int Get() const { return value; }
void Increment() { value++; }
const int& GetRef() const { return value; }
int& GetMutRef() { return value; }
int value;
};
Crubit will generate the following bindings (notice the inferred '__this
lifetime):
impl Counter {
#[inline(always)]
pub fn Get<'__this>(&'__this self) -> ::ffi_11::c_int {
unsafe { self::counter::Get(self) }
}
#[inline(always)]
pub fn Increment<'__this>(&'__this mut self) {
unsafe { self::counter::Increment(self) }
}
#[inline(always)]
pub fn GetRef<'__this>(&'__this self) -> ::cref::CRef<'__this, ::ffi_11::c_int> {
unsafe { self::counter::GetRef(self) }
}
#[inline(always)]
pub fn GetMutRef<'__this>(&'__this mut self) -> ::cref::CMut<'__this, ::ffi_11::c_int> {
unsafe { self::counter::GetMutRef(self) }
}
}
Reference parameters become Rust references, and each of them gets its own
lifetime (e.g. see 'x below):
#[inline(always)]
pub fn AddOne<'x>(x: &'x mut ::ffi_11::c_int) {
unsafe { crate::detail::__rust_thunk___ZN7example6AddOneERi(x) }
}
When lifetime elision doesn’t apply
Lifetime elision doesn’t apply to some functions. For example, in Rust, fn smaller(x: &i32, y: &i32) -> &i32 doesn’t compile (error E0106), because the
result can refer to x or to y. See also “Generic Lifetimes in
Functions”
in the Rust Book.
For such C++ functions, Crubit generates bindings that return a raw pointer instead of a reference. For example, given the following C++ function:
// Lifetime elision doesn't apply: there are two input references and no `this`.
inline const int& Smaller(const int& x, const int& y) { return x < y ? x : y; }
Crubit will generate the following bindings:
/// Lifetime elision doesn't apply: there are two input references and no `this`.
#[inline(always)]
pub fn Smaller<'x, 'y>(
x: &'x ::ffi_11::c_int,
y: &'y ::ffi_11::c_int,
) -> *const ::ffi_11::c_int {
unsafe { crate::detail::__rust_thunk___ZN7example7SmallerERKiS1_(x, y) }
}
Rust code needs unsafe to dereference the returned pointer. To bind the result
as a reference, add explicit lifetime annotations to the C++
function.
Explicit lifetime annotations
C++ code can use explicit lifetime annotations (e.g. $a) to specify lifetimes
that lifetime elision can’t infer. For example, given the following C++
function:
inline const int& $a SmallerWithLifetimes(const int& $a x, const int& $a y) {
return x < y ? x : y;
}
Crubit will generate the following bindings:
/// Explicit lifetime annotations: The result may refer to `x` or to `y`.
#[inline(always)]
pub fn SmallerWithLifetimes<'a>(
x: &'a ::ffi_11::c_int,
y: &'a ::ffi_11::c_int,
) -> ::cref::CRef<'a, ::ffi_11::c_int> {
unsafe { crate::detail::__rust_thunk___ZN7example20SmallerWithLifetimesERKiS1_(x, y) }
}
Notes:
- The
$a,$b, etc. macros come from"support/lifetime_annotations.h". $aannotates the lifetime of the reference that precedes it.- In a method,
$aafter the parameter list (and afterconst, if present) annotates the lifetime ofthis. See the example below. $staticannotates a'staticlifetime.
See Lifetime Annotations for C++ for more details and design discussions.
When lifetime elision gives wrong lifetimes
Lifetime elision is a good default for most C++ APIs, but neither the C++ compiler nor Crubit checks that the lifetimes match what the C++ code does. For example, given the following C++ struct:
struct Number final {
// The result may refer to `this` or to `other`, but lifetime elision ties the
// result only to `this`.
const int& GetLarger(const int& other) const {
return value > other ? value : other;
}
// The result may refer to `this` or to `other`.
const int& $a GetLargerWithLifetimes(const int& $a other) const $a {
return value > other ? value : other;
}
int value;
};
Crubit will generate the following bindings:
impl Number {
/// The result may refer to `this` or to `other`, but lifetime elision ties the
/// result only to `this`.
#[inline(always)]
pub fn GetLarger<'__this, 'other>(
&'__this self,
other: &'other ::ffi_11::c_int,
) -> ::cref::CRef<'__this, ::ffi_11::c_int> {
unsafe { self::number::GetLarger(self, other) }
}
/// The result may refer to `this` or to `other`.
#[inline(always)]
pub fn GetLargerWithLifetimes<'a>(
&'a self,
other: &'a ::ffi_11::c_int,
) -> ::cref::CRef<'a, ::ffi_11::c_int> {
unsafe { self::number::GetLargerWithLifetimes(self, other) }
}
}
Lifetime elision ties the result of GetLarger only to self. This is wrong,
because the result can refer to other. Rust code can keep the result after
other is dropped, and then the result is a dangling reference. This can cause
undefined behavior.
GetLargerWithLifetimes uses explicit lifetime annotations to tie the result to
both self and other. If lifetime elision gives wrong lifetimes, then please
add explicit lifetime annotations to the C++ API.
CRef and CMut
C++ references returned from C++ functions and methods become CRef<'a, T> (for
const T&) or CMut<'a, T> (for T&). CRef and CMut are defined in the
cref crate (//support:cref).
Crubit uses CRef and CMut (rather than Rust references) to accurately
represent that C++ references don’t offer the same guarantees as Rust
references. For example:
- A Rust shared reference (
&T) promises that nothing mutates the value while the reference is in use. On the other hand,const T&in C++ allows mutating the referenced value (e.g. through amutablefield, or through aconst_castif the referenced object itself is notconst). - A Rust mutable reference (
&mut T) promises that the reference is the only way to access the value while it is in use. On the other hand, C++ allows multipleT&(and/orconst T&) to co-exist.
See the documentation of the cref crate for more details.
Methods
Today Crubit binds this as &self or &mut self (rather than as self: CRef<Self> or self: CMut<Self>).
Future Rust language features may make CRef and CMut more ergonomic to use
as method receivers (see arbitrary self
types and
autoref in the “Beyond
Rust References” initiative). Once
that happens, Crubit may start binding this as CRef<Self> or CMut<Self>
(at least for inherent methods - trait methods may still require &self or
&mut self).
Rust bindings for C++ enums
A C++ enum is mapped to a Rust struct with a similar API to a Rust enum.
- The enumerated constants are present as associated constants:
MyEnum::kFooin C++ isMyEnum::kFooin Rust. - The enum can be converted to and from its underlying type using
FromandInto. For example,static_cast<int32_t>(x)isi32::from(x)in Rust, and vice versastatic_cast<MyEnum>(x)isMyEnum::from(x).
However, a C++ enum is not a Rust enum. Some features of Rust enums are not supported:
- C++ enums must be converted using
FromandInto, notas. - C++ enums do not have exhaustive pattern matching.
Example
Given the following C++ header:
enum Color {
kRed,
kBlue,
kGreen,
};
Crubit will generate the following bindings:
#[repr(transparent)]
#[derive(Debug, PartialEq, Eq, Copy, Clone, Hash, PartialOrd, Ord)]
#[cfi_encoding = "5Color"]
///CRUBIT_ANNOTATE: cpp_type=Color
pub struct Color(::ffi_11::c_uint);
impl Color {
pub const kRed: Color = Color(::ffi_11::new_c_uint(0));
pub const kBlue: Color = Color(::ffi_11::new_c_uint(1));
pub const kGreen: Color = Color(::ffi_11::new_c_uint(2));
}
impl From<::ffi_11::c_uint> for Color {
fn from(value: ::ffi_11::c_uint) -> Color {
Color(value)
}
}
impl From<Color> for ::ffi_11::c_uint {
fn from(value: Color) -> ::ffi_11::c_uint {
value.0
}
}
Which can be used like this:
fn main() {
let green = example_lib::Color::kGreen;
match green {
example_lib::Color::kGreen => println!("Green is green!"),
_ => todo!(),
}
}
Why isn’t it an enum?
A C++ enum cannot be translated directly to a Rust enum, because C++ enums
are “representationally non-exhaustive”: a C++ enum can have any value
supported by the underlying type, even one not listed in the enumerators. For
example, in the enum above, static_cast<Color>(42) is a valid instance of
Color, even though none of kRed, kBlue, or kGreen have that value.
Rust enums, in contrast, are representationally exhaustive. An enum declares a
closed set of valid discriminants, and it is undefined behavior to
attempt to create an enum with a value outside of that set, whether it’s via
transmute, a raw pointer cast, or Crubit. The behavior is undefined the moment
the invalid value is created, even if it is never used.
Since a value like static_cast<Color>(42) is not in the list of enumerators, a
Rust enum cannot be used to represent an arbitrary C++ enum. Instead, the
Rust bindings are a struct. This struct is given the most natural and
enum-like API possible, though there are still gaps. (Casts using as, for
example, will not work with a C++ enum.)
What about #[non_exhaustive]?
The #[non_exhaustive] attribute on an enum communicates to external
crates that more variants may be added in the future, and so a match requires
a wildcard branch. Within the defining crate, non_exhaustive has no effect. It
remains undefined behavior to transmute from integers not declared by the
enum.
Inline C++
Crubit allows Rust code to call C++ libraries directly through generated bindings. However, some C++ interfaces cannot yet be automatically bound by Crubit, such as function templates and preprocessor macros.
Crubit provides two macros, inline_cpp! and global_cpp!, that allow you to
embed C++ declarations and expressions directly inside your Rust source files.
This lets you call unsupported C++ code without having to write separate C++
wrapper libraries.
How to use Embedded C++
Embedded C++ uses two macros depending on scope:
global_cpp!: Used at module scope (outside functions) to declare C++#includeheaders, structs, namespaces, and helper functions.inline_cpp!: Used inside Rust functions to define and call an inline C++ expression.
You can use embedded C++ in two ways:
- In a
rust_api_from_cpptarget (Recommended): When extending the generated Rust bindings of an existing C++ library using custom Rust source files (thesrcsattribute). - In a
rust_library_with_embedded_cpporrust_test_with_embedded_cpptarget: When authoring a standalone Rust library or test with embedded C++ without an underlyingcc_library.
1. Using embedded C++ in a rust_api_from_cpp target
If you maintain a cc_library and want to provide hand-written Rust helper
methods or trait implementations alongside your generated Crubit API, you can
pass custom Rust source files directly to rust_api_from_cpp via srcs.
Given a C++ header containing a template method that cannot be natively bound by Crubit:
// widget.h
namespace widget {
class Widget {
public:
template <typename T = void>
static std::unique_ptr<Widget> Create(absl::string_view name);
private:
Widget(absl::string_view name);
};
} // namespace widget
You can define the BUILD targets as follows:
load(
"//bazel:rust_api_from_cpp.bzl",
"rust_api_from_cpp",
)
cc_library(
name = "widget",
hdrs = ["widget.h"],
srcs = ["widget.cc"],
aspect_hints = [":widget_rust.hint"],
deps = [
"@abseil-cpp//absl/strings",
],
)
rust_api_from_cpp(
name = "widget_rust",
cpp_target = ":widget",
root_namespaces = ["widget"],
srcs = ["widget_custom.rs"],
)
Inside widget_custom.rs, you can use inline_cpp! to bridge the C++ template
factory method directly into the crate’s public API:
#![allow(unused)]
fn main() {
// widget_custom.rs (compiled directly into the widget_rust crate)
use inline_cpp_macro::inline_cpp;
impl Widget {
/// Ergonomic Rust wrapper over a C++ factory function template.
pub fn new_with_name(name: &str) -> cpp_std::unique_ptr<Widget> {
let make_widget = inline_cpp! {
(rs::StrRef name) -> std::unique_ptr<widget::Widget> {
return widget::Widget::Create(name.to_string_view());
}
};
make_widget(name)
}
}
}
2. Using embedded C++ in a standalone rust_library or rust_test target
To write a standalone Rust library or test that uses embedded C++ without an
underlying cc_library, define your target with
rust_library_with_embedded_cpp or rust_test_with_embedded_cpp:
rust_library_with_embedded_cpp(
name = "greeting_lib",
srcs = ["greeting.rs"],
deps_of_cc_library = [
"@abseil-cpp//absl/strings",
"//support/rs_std:str_ref",
],
)
The deps_of_cc_library attribute lists the C++ libraries that provide the
headers and symbols used in your embedded C++ snippets. For a complete runnable
example, see
examples/cpp/inline_cpp/.
Writing Embedded C++
Declaring Headers with global_cpp!
Use global_cpp! at module scope (outside functions, including within nested
submodules) to specify #include headers and define supporting C++ types:
global_cpp! {
#include <algorithm>
#include <cmath>
#include "support/rs_std/str_ref.h"
#include "third_party/absl/strings/str_cat.h"
#include "third_party/absl/strings/string_view.h"
struct Point {
double x;
double y;
};
template <typename T>
struct MathHelper {
static T Multiply(T a, T b) { return a * b; }
};
}
NOTE: All
global_cpp!declarations across a crate (including those inside nested submodules) are combined into a single C++ companion header. Anyinline_cpp!block in the crate can access types declared inglobal_cpp!, regardless of which Rust module defined them. Standard Rust visibility (pub,pub(crate)) governs access to the wrapping Rust functions.
Calling C++ with inline_cpp!
Inside a Rust function, inline_cpp! defines an inline C++ expression. The
syntax requires a C++ parameter list, a return type arrow ->, and the C++
function body.
For example, calling a C++ standard library math function:
/// Computes the hypotenuse using C++ `std::hypot`.
pub fn compute_hypotenuse(a: f64, b: f64) -> f64 {
let hypot_fn = inline_cpp! {
(double a, double b) -> double {
return std::hypot(a, b);
}
};
hypot_fn(a, b)
}
Or calling an Abseil string utility:
/// Formats a greeting using C++ `absl::StrCat`.
pub fn format_greeting(name: &str) -> String {
let greet = inline_cpp! {
(rs::StrRef name) -> std::string {
return absl::StrCat("Hello, ", name);
}
};
let cpp_str = greet(name);
cpp_str.to_string().expect("Valid UTF-8")
}
inline_cpp! produces a callable Rust closure. The parameter and return types
of the generated closure match the corresponding Rust types generated by Crubit
(for example, i32 for int, and &Point for const Point&).
Passing types between Rust and C++
Primitive types
Primitive numeric types and booleans pass directly by
value between Rust and C++ (for example, int in C++ corresponds to i32 in
Rust, and double corresponds to f64).
References and pointers
You can pass Rust references to C++ as const T& or raw pointers:
/// Calculates Euclidean distance for a `Point`.
pub fn get_distance(p: &Point) -> f64 {
let calc = inline_cpp! {
(const Point& p) -> double {
return std::sqrt(p.x * p.x + p.y * p.y);
}
};
calc(p)
}
WARNING: Standard Rust lifetime rules apply. C++ code must not store or keep references or pointers after the
inline_cpp!call returns.
Common Use Cases
Calling and Defining C++ Templates
Crubit does not automatically generate Rust bindings for uninstantiated C++ templates. With embedded C++, you can:
- Instantiate library templates: Call templated functions or classes from
included C++ headers inside
inline_cpp!:
/// Calls an instantiated C++ standard library template.
pub fn clamp_value(val: f32, min: f32, max: f32) -> f32 {
let clamp = inline_cpp! {
(float val, float min, float max) -> float {
return std::clamp<float>(val, min, max);
}
};
clamp(val, min, max)
}
- Define helper templates in
global_cpp!: Define C++ template helpers at module scope, then instantiate them with concrete types ininline_cpp!:
/// Calls a C++ helper template defined in `global_cpp!`.
pub fn multiply_ints(a: i32, b: i32) -> i32 {
let mult = inline_cpp! {
(int a, int b) -> int {
return MathHelper<int>::Multiply(a, b);
}
};
mult(a, b)
}
Common Errors
inline_cpp! uses Crubit under the hood, so it can produce any error Crubit
could produce when bridging types.
Missing parameter list or return type
inline_cpp! requires an explicit parameter list and return type arrow. If your
C++ snippet takes no arguments or returns nothing, specify () -> void:
#![allow(unused)]
fn main() {
let no_args_fn = inline_cpp! {
() -> void {
DoWork();
}
};
no_args_fn();
}
Omitting the parameter list or return type results in an extraction syntax error:
error: inline_cpp! block must start with a parameter list `(args)`
--> src/lib.rs:10:5
|
10 | inline_cpp! { DoWork(); }
| ^^^^^^^^^^^^^^^^^^^^^^^^^
Unsupported features and syntax limitations
-
Unmatched braces: All
{}braces inside embedded C++ blocks must be properly balanced. If a brace is unmatched (such as in complex preprocessor macros or unclosed string literals), the extraction preprocessor reports an unmatched delimiter error:error: Unmatched delimiter starting at src/lib.rs:10: Context around open brace: inline_cpp! { (int x) -> int { if (x > 0) { return x; }
Rust bindings for C++ standard library (std) types
When generating Rust bindings for a C++ target, bindings to the C++ standard
library types are provided by the cpp_std support library (which re-exports
the legacy cc_std::std::*). This contains a combination of manually written
types and automatically generated bindings.
You can then use standard library types directly via cpp_std::{sometype},
e.g., cpp_std::unique_ptr<T>.
TODO(b/454337872): The following is stub documentation in the absence of rustdoc support. Instead of linking to API docs, we link to Rust source files.
std::string
Bindings for the layout-compatible std::string type are available behind the
//features:layout_compat_string feature flag, but are not
yet enabled by default (b/408961701).
std::string is only supported as a “bridge type”, with runtime conversion.
When a std::string is passed or returned by value, it is transformed into a
cpp_std::string_wrapper.
API: support/cc_std_impl/string.rs
std::string_view and absl::string_view
C++’s std::string_view type has two analogues in Rust. If no lifetime can be
specified or inferred, or the data may be mutably aliased, then one should use
cpp_std::raw_string_view. This is analogous to a *const [u8] in Rust.
Otherwise, cpp_std::string_view<'a>.
API: support/cc_std_impl/string_view.rs
std::unique_ptr<T>
C++’s std::unique_ptr<T> type has two analogues in Rust. If the type has a
virtual destructor, or an overloaded operator delete, then it becomes a
cpp_std::virtual_unique_ptr<T>. In particular, polymorphic types will be
stored in a cpp_std::virtual_unique_ptr<T>.
Otherwise, when the destructor is directly callable and not virtual, it becomes
a cpp_std::unique_ptr<T>.
For example, a C++ std::unique_ptr<int> is a Rust cpp_std::unique_ptr<int>,
but a C++ std::unique_ptr<ios_base> would become a Rust
cpp_std::virtual_unique_ptr<ios_base>.
Both types implement Deref<Target = T> and DerefMut<Target = T> (when T: Unpin). While C++ std::unique_ptr is only shallow-const (i.e. const std::unique_ptr<T>& allows mutating the underlying T), it is conventionally
treated as deep-const. In order for &unique_ptr<T> to be usable at all from
Rust, we treat it as deep-const. C++ code which mutates a value of type T
while Rust has obtained a &T via &unique_ptr<T> -> &T deref will result
in undefined behavior.
API: support/cc_std_impl/unique_ptr.rs
See also: crubit.rs/errors/delete
std::shared_ptr<T>
A C++ std::shared_ptr<T> becomes a Rust cpp_std::shared_ptr<T>. It
implements Deref<Target = T>, allowing &shared_ptr<T> to be dereferenced to
&T.
API: support/cc_std_impl/shared_ptr.rs
std::vector<T>
A C++ std::vector<T> becomes a Rust cpp_std::vector<T>.
API: support/cc_std_impl/vector.rs
std::optional<T>
std::optional<T> in C++ is bound as cpp_std::optional<T> or
cpp_std::trivial_optional<T> in Rust. trivial_optional is used when T: Copy, which allows optional of trivially copyable types to also be trivially
copyable (see the trivial_optional docs in the API below). Convert to and from
a Rust Option<T> with From/Into, e.g. Option::from(opt) or
Some(x).into().
API: support/cc_std_impl/optional.rs
Errors
When Crubit can’t generate bindings for an item, it usually leaves a comment in the generated bindings explaining why, often with a link to one of the pages below. To find the comment, look at the generated bindings for C++ libraries or Rust libraries.
- Unsupported types: a type that Crubit can’t bind, which also prevents bindings for every function that uses it.
- Unknown targets: a type defined in a header that none of the library’s direct dependencies provide, so Crubit can’t tell which target owns it.
- Nested types: C++ types nested inside other types, when the snake_case module Crubit generates for them collides with another name.
- Unknown attributes: C++ or Rust attributes that Crubit doesn’t understand, which cause it to skip the item.
- Bridge types as struct fields: struct or union fields whose type is a bridge type.
- Compound types containing bridge types: bridge types behind a pointer or inside another type that can’t bridge its contents.
- The
Deletetrait: usingvirtual_unique_ptr<T>with a type that has neither a virtual destructor nor a customoperator delete. - Visibility errors: using a type whose bindings are
restricted to
pub(crate)visibility in:wrappermode outside its defining library.
Unsupported type errors
Sometimes, a type is not supported in Crubit. This presents a problem for all functions that use that type, and so Crubit aims to actually cover all C++ types, even if in only minimal form. However, sometimes that aspiration is still not met today, and functions like the following are not callable at all:
Unsupported DoSomething(Unsupported in_val, Unsupported& in_ref);
This example will be used for reference below, though most of the advice also applies to functions and types defined in Rust.
Fix
There are roughly four approaches to solving this:
- (Most preferred) if possible, convince Crubit to generate bindings.
- Use wrapper types
- (For C++ libraries only) Use forward declarations
- (Dispreferred) Use void pointers
(Preferred) Convince Crubit to generate bindings
If Crubit is not enabled on the library defining Unsupported, first, enable
Crubit. (See crubit.rs/cpp/ and crubit.rs/rust.)
If Crubit is enabled on the target, the generated bindings will contain a
comment explaining the error. To
see the error message as a compilation error, apply the CRUBIT_MUST_BIND
Crubit attribute to the Unsupported type definition.
The error may describe some way to resolve the type. For example, if it has an unrecognized attribute, see crubit.rs/errors/unknown_attribute.
However, ultimately, it may boil down to a bug or missing feature in Crubit, and
you may need to work around Unsupported not having a type. This can be done by
writing a wrapper around the DoSomething function which doesn’t use that type.
Below are a handful of common approaches.
Replace it with a wrapper type
Even if the type is not supported, a type containing that type will likely work fine. Crubit relies on the abstract properties of the overall compound data type, like its size and alignment, and does not need each field to individually work with Crubit.
(In C++, you may also want to make the wrapper type Rust-movable for convenience. See crubit.rs/cpp/cookbook#rust_movable)
To make the example C++ function DoSomething callable from Rust:
// mylib_ffi.h
#include "third_party/absl/base/attributes.h"
#include "support/annotations.h"
struct ABSL_ATTRIBUTE_TRIVIAL_ABI UnsupportedWrapper {
Unsupported value;
};
CRUBIT_RUST_NAME("DoSomething")
UnsupportedWrapper DoSomethingRust(
UnsupportedWrapper in_val,
UnsupportedWrapper* in_ref);
// mylib_ffi.cc
#include "mylib_ffi.h"
#include "mylib.h"
UnsupportedWrapper DoSomethingRust(
UnsupportedWrapper in_val,
UnsupportedWrapper* in_ref) {
return UnsupportedWrapper{
DoSomething(std::move(in_val.value), in_ref->value)
};
}
Bridged types passed by reference or pointer
Most bridged types cannot be passed by reference (&T) or pointer (*const T).
Because Crubit performs value conversions across the FFI boundary, bridged types
must usually be passed by value.
(Note: Protocol Buffers are an exception; they can be passed by reference. This
uses the proto_references feature, which
//features:supported enables by default.)
Otherwise, if the function semantics strictly requires passing by reference, you
can wrap the data in a layout-compatible struct or write an FFI wrapper
function.
Generic types and parameters
Uninstantiated generic types (such as struct UnsupportedContainer<T>) and
generic type parameters (T) cannot receive bindings directly. Define a
monomorphized type alias or non-generic wrapper function to expose concrete
types:
#![allow(unused)]
fn main() {
// Uninstantiated generic type, which does not receive bindings.
pub struct UnsupportedContainer<T> {
pub value: T,
}
// These two workarounds receive callable bindings.
pub type IntContainer = UnsupportedContainer<i32>;
pub fn DoSomethingI32(val: i32) {
DoSomethingGeneric(val);
}
}
(C++) Replace it with a forward declaration in wrapper functions
TODO(b/482061078): Support this workaround in Rust, as well.
NOTE: Crubit does this automatically in wrapper mode, and this may become the
default behavior for unsupported types in supported targets in a future release,
so that no additional work is necessary.
In an FFI-oriented header, you can define a replacement DoSomething function
that uses pointers to a forward-declared type, and which, at worst,
heap-allocates when passing or returning by value. However, you must also free
the memory when complete. Using a wrapper is likely to be more user-friendly.
To make the example C++ function DoSomething callable from Rust:
// mylib_ffi.h
#include "support/annotations.h"
class Unsupported;
CRUBIT_RUST_NAME("DoSomething")
Unsupported* DoSomethingRust(Unsupported* in_val, Unsupported* in_ref);
// mylib_ffi.cc
#include "mylib_ffi.h"
#include "third_party/absl/memory/memory.h"
#include "mylib.h"
Unsupported* DoSomethingRust(Unsupported* in_val, Unsupported* in_ref) {
std::unique_ptr<Unsupported> in_val_owned = absl::WrapUnique(in_val);
return new auto(DoSomething(std::move(*in_val_owned), *in_ref));
}
Here, the wrapper uses raw pointers (in Rust) where C++ would use values, in order to allow passing an unsupported type by value. However, by reference and by pointer, there is no heap allocation.
(Dispreferred) Replace it with a void pointer
BEST PRACTICE: This is identical to the forward declaration solution when
applicable, but is more dangerous due to erasing the type and using void.
In an FFI-oriented header, you can define a replacement DoSomething function
that uses void pointers, and which, at worst, heap-allocates when passing or
returning by value. However, you must also free the memory when complete, this
is not type safe. Using a wrapper or a forward declaration is both safer and
more user-friendly.
To make the example C++ function DoSomething callable from Rust:
// mylib_ffi.h
#include "support/annotations.h"
CRUBIT_RUST_NAME("DoSomething")
void* DoSomethingRust(void* in_val, void* in_ref);
// mylib_ffi.cc
#include "mylib_ffi.h"
#include "third_party/absl/memory/memory.h"
#include "mylib.h"
void* DoSomethingRust(void* in_val, void* in_ref) {
std::unique_ptr<Unsupported> in_val_owned = absl::WrapUnique(reinterpret_cast<Unsupported*>(in_val));
return new auto(DoSomething(std::move(*in_val_owned), *reinterpret_cast<Unsupported*>(in_ref)));
}
Unknown target errors
When generating Rust bindings for a C++ library, Crubit needs to know which
Bazel target owns every header that defines a type used in the library’s API.
That is how it decides, for example, that absl::Span<T> maps to the Span
type from the Rust bindings for @abseil-cpp//absl/types:span, or which
generated crate a struct should be imported from.
Crubit only knows about the public headers of Crubit-enabled targets that your library depends on directly (plus those reachable through header-only or Crubit-enabled intermediate libraries). If a type’s header isn’t one of those, Crubit reports an error like:
crubit.rs/errors/unknown_target: Failed to complete template specialization type absl::Span<const std::string>:
the template is defined in
third_party/absl/types/span.h
which is not a public header of any Crubit-enabled target that
//my:lib
depends on directly. ...
or, for non-template types:
crubit.rs/errors/unknown_target: the type is defined in
foo/bar.h
which is not a public header of any Crubit-enabled target that
//my:lib
depends on directly. ...
Crubit can’t tell which of the following causes applies, so check each.
Cause 1: Crubit is not enabled on the defining library
The library that provides the header exists in your deps, but doesn’t have
Crubit enabled. Enable Crubit on it (see Enabling
Crubit). If you can’t, see Unsupported
types for workarounds.
Cause 2: the defining library is not a direct dependency
To keep builds fast, Crubit does not analyze every transitive dependency of a
C++ library. In particular, it does not look behind a cc_library that has
compiled sources (.cc files) and no Crubit aspect_hints. If your header uses
a type whose header is only reachable through such a library, Crubit can’t
attribute the header to a target, even if the defining library has Crubit
enabled.
This is the same rule as
include-what-you-use
and layering_check: a header that names a type should #include the header
that defines it, and the target should depend directly on the library that
provides that header.
For example, the following fails even though @abseil-cpp//absl/types:span has
Crubit enabled:
cc_library(
name = "middle",
srcs = ["middle.cc"],
hdrs = ["middle.h"], # #includes "third_party/absl/types/span.h"
deps = ["@abseil-cpp//absl/types:span"],
)
cc_library(
name = "my_lib",
hdrs = ["my_lib.h"], # uses absl::Span, but only #includes "middle.h"
aspect_hints = [":my_lib_rust.hint"],
deps = [":middle"], # :span is not a direct dependency
)
Fix it by adding a direct dependency on the library that provides the header,
and #include-ing the header directly:
cc_library(
name = "my_lib",
hdrs = ["my_lib.h"], # #includes "third_party/absl/types/span.h"
aspect_hints = [":my_lib_rust.hint"],
deps = [
":middle",
"@abseil-cpp//absl/types:span",
],
)
Cause 3: the header is private
Headers listed in srcs (rather than hdrs) of a cc_library are private to
that library, and Crubit does not attribute them to any target. Types defined in
private headers can’t receive bindings. If you need bindings for such a type,
move its definition to a public header (in hdrs), or wrap it in a public type.
See Unsupported types.
Nested type errors
Rust does not offer a way to model nested C++ types. To compensate for this, similar to Rust protocol buffers and their nested message support, Crubit creates a snake_case module to hold nested types inside. For example:
struct OuterType {
struct Inner {};
};
#![allow(unused)]
fn main() {
// generated Rust bindings
struct OuterType {...}
mod outer_type {
struct Inner {...}
}
}
If mod outer_type is already in use (for example, because OuterType is
already snake_case, because namespace outer_type exists, or because another
type in the same scope snake_cases to outer_type), Crubit disambiguates the
module name by appending _items (and additional trailing _ characters if
needed, such as mod outer_type_items).
Disambiguate the names
To avoid the disambiguated _items suffix (or if an older version of Crubit
reports a module name collision), you can disambiguate the C++ names directly.
For example, if there is an existing namespace outer_type, one can rename
either OuterType or outer_type. Or, if there are two types which
snake_case to the same name, one of them can be renamed.
Aliasing can also be used here:
struct OuterImpl {
struct Inner {...};
};
using OuterType = OuterImpl;
Unknown attribute errors
C++ and Rust allow the use of attributes to customize the behavior and semantics
of a piece of syntax. For example, the C++ [[nodiscard]] and Rust
#[must_use] attributes cause a compilation error if a return value is
discarded.
If Crubit encounters an attribute it is not aware of, it will fail to generate bindings for the item that the attribute affects. This is both for reasons of correctness, as well as forward-compatibility between Crubit releases.
Correctness
While many attributes, like [[nodiscard]], can be safely ignored, others
cannot be. For example, Crubit must correctly understand the
[[no_unique_address]] attribute to produce bindings, as it can alter the
layout of the type containing it. In Rust, Crubit must understand the
#[repr(...)] attribute, which changes type layout.
Simply ignoring these attributes would result in code which failed to compile, or worse, which had undefined behavior at runtime.
Because Crubit does not know whether an attribute is like this in advance, it conservatively must assume that an unknown attribute might impact correctness, and will not generate bindings.
Compatibility
Other attributes present a compatibility hazard. Take [[nodiscard]] and
#[must_use]. The bindings of a C++ [[nodiscard]] type or function should be
a Rust #[must_use] type or function, and
vice versa.
If Crubit simply ignored attributes that are relevant for API surface but not correctness, it would be a backwards-incompatible change to later stop ignoring them.
Fix
Crubit conservatively rejects items with unknown attributes. This can be resolved in one of three ways:
-
Remove the attribute from the declaration.
-
File a crubit.rs-bug to add the attribute to Crubit.
-
If the attribute does not impact the bindings (either their safety or the resulting API shape), add a
CRUBIT_UNSAFE_IGNORE_ATTR(...)attribute containing the name of the unknown attribute.See support/annotations.h
WARNING:
CRUBIT_UNSAFE_IGNORE_ATTRcan cause undefined behavior, or compilation failure, and can make code incompatible with future versions of Crubit. Only use this when the attribute has no impact on interface or semantics.
Bridge types as struct fields
Overview
Crubit does not support exposing struct or union fields with bridge types.
Bridge types are types that are converted at runtime between C++ and Rust, but
have different underlying representations. Examples include absl::AnyInvocable
(mapped to Box<dyn Fn...>) and types annotated with CRUBIT_BRIDGE. (See
crubit.rs/types)
If a field is a bridge type, this conversion is impossible: field accesses do not run any conversion logic, and the field is required to have identical representation across both languages. As a result, the field will not have the bridge type conversion applied, but instead will be replaced by an opaque blob of bytes.
Example
The examples below use either::Either<L, R>, a C++ struct that is bridged to a
Rust enum Either<L, R>. It is defined in
examples/types/bridging.
Consider the following C++ struct:
#include "examples/types/bridging/either.h"
struct Config {
either::Either<int, bool> value;
};
Crubit will treat the value field of Config as an opaque blob of bytes that
cannot be accessed directly from Rust.
Workaround: Add getter and setter methods
To access or modify the field from Rust, you can add getter and setter methods. Crubit can generate bindings for functions that accept or return bridge types by value.
For example:
#include "examples/types/bridging/either.h"
struct Config {
either::Either<int, bool> value;
// Getter returning by value (mapped to `Either<i32, bool>` in Rust).
either::Either<int, bool> get_value() const { return value; }
// Setter accepting by value.
void set_value(either::Either<int, bool> v) { value = v; }
};
Then in Rust, you can use these methods:
#![allow(unused)]
fn main() {
use either::either::Either;
let mut config = Config::default();
config.set_value(Either::Left(42));
assert_eq!(config.get_value(), Either::Left(42));
}
See crubit.rs/cpp/best_practices#bridging for more information.
Compound types containing bridge types
Overview
Crubit does not support compound data types containing bridge types, unless the compound data type itself can be bridged.
Bridge types are types that are converted at runtime between C++ and Rust, but
have different underlying representations. Examples include absl::AnyInvocable
(mapped to Box<dyn Fn...>) and types annotated with CRUBIT_BRIDGE. (See
crubit.rs/types.)
This conversion is only possible when the bridge type is used by value. It is
not possible in fields (crubit.rs/errors/bridge_field), inside of a pointer, or
more generally inside of a compound data type that doesn’t support recursively
bridging its interior (such as std::vector).
Example
The examples below use either::Either<L, R>, a C++ struct that is bridged to a
Rust enum Either<L, R>. It is defined in
examples/types/bridging.
Consider the following C++ function:
#include "examples/types/bridging/either.h"
const either::Either<int, bool>* Foo();
While Crubit can bridge an either::Either<int, bool> return value, it cannot
bridge it when it is behind a pointer. It cannot reuse the existing storage for
bridging, as it doesn’t know if the storage is valid, its size, or anything
else. And since Crubit does not have bindings for either::Either except via
bridge types and runtime conversion, this means that Foo() does not receive
bindings.
Workaround: Wrap the function
Whenever a type does not receive bindings, you can wrap the function with something that does. For example:
#include "examples/types/bridging/either.h"
const either::Either<int, bool>* Foo();
// A version of Foo() that returns a copy of the value instead of a pointer.
// Bridge types can be returned by value. This assumes Foo() never returns
// null.
inline either::Either<int, bool> FooForRust() { return *Foo(); }
For more workarounds for unsupported types, see crubit.rs/errors/unsupported_type.
For more information on bridging, see crubit.rs/cpp/best_practices#bridging.
The Delete trait
Quick fix
If you’ve encountered this error when using Crubit’s virtual_unique_ptr<T>,
note that virtual_unique_ptr<T> is only available when T is a C++ type with
a virtual destructor or which overrides operator delete.
When T does not have a virtual destructor and does not override operator
delete, use cpp_std::unique_ptr<T> instead.
Conversely, if T implements Delete (it has a virtual destructor or overrides
operator delete), Crubit won’t bind unique_ptr<T>. Use
cpp_std::virtual_unique_ptr<T> instead.
Why
For types with virtual destructors or custom operator delete implementations,
C++’s delete ptr; operation does more than simply destruct the value and free
the memory associated with the target type. Instead, C++ will use the custom
operator delete defined by the target type.
This makes it possible to convert a std::unique_ptr<DerivedClass> into a
std::unique_ptr<BaseClass> so long as BaseClass has a virtual destructor.
When the std::unique_ptr<BaseClass> is dropped, the operator delete on the
BaseClass will invoke the virtual destructor, resulting in a call to the
destructor of DerivedClass. After that, C++ knows to free memory the size of
DerivedClass rather than memory the size of BaseClass.
In order to replicate this behavior in Rust, C++’s unique_ptr<T> has been
split into two types: unique_ptr and virtual_unique_ptr. unique_ptr
assumes that the target type is concrete, while virtual_unique_ptr supports
C++ types with a virtual destructor or custom operator delete. Such types will
automatically receive a Crubit-generated implementation of the Delete trait.
Overview of Crubit’s C++/Rust bindings
Motivation for automating generation of FFI bindings
To call a C++ function from Rust, one can manually declare the foreign function interface (FFI) as follows:
#![allow(unused)]
fn main() {
extern "C" {
fn cpp_multiplication_function(x: i32, y: i32) -> i64;
}
}
This works, but has some disadvantages:
- The function signature in Rust’s
externdeclaration needs to match the signature of the actual C++ function (e.g. we can’t sayx: i16when the C++ parameter isint32_t x- doing this would lead to Undefined Behavior). Keeping those function signatures in sync can be error prone. - Rust does not support the C++ calling convention or any C++ features not present in C - this means that bindings for some functions may require an extra “thunk” (an extra function that exposes C ABI and forwards calls to the target function).
- It is possible to manually replicate in C++ the memory layout and ABI of
#[repr(C)]and#[repr(transparent)]structs, but not the implementation-defined layout of#[repr(Rust)]structs.
Similar disadvantages exist when manually setting up FFI bindings for calling a Rust function from C++. Automating generation of FFI bindings with Crubit can help alleviate the disadvantages above.
C++ bindings for Rust APIs
Crubit can take a Rust crate and create a C++ library which exposes Rust functions and types to C++ callers.
Let’s walk through the
examples/rust/function
example. In the example, Bazel is instructed to provide C++ bindings for a Rust
library:
load(
"//bazel:cpp_api_from_rust.bzl",
"cpp_api_from_rust",
)
rust_library(
name = "example_crate",
srcs = ["example.rs"],
)
# This declares an "example_crate_cc_api" target that provides Crubit-generated
# C++ bindings for the Rust crate behind the `":example_crate"` target.
cpp_api_from_rust(
name = "example_crate_cc_api",
crate = ":example_crate",
)
With the above, C++ code (e.g. cc_binary, cc_library, cc_test, etc.) can
just list the bindings in its deps:
cc_binary(
name = "main",
srcs = ["main.cc"],
# Declaring a dependency on C++ bindings for calling into the Rust
# `example` library:
deps = [ ":example_crate_cc_api" ],
)
The scaffolding above will let C++ call into Rust as follows:
#![allow(unused)]
fn main() {
// example.rs:
pub fn add_two_integers(x: i32, y: i32) -> i32 { x + y }
}
// main.cc:
// The generated bindings are in a header at the same path as the
// `example_crate` rust_library, with a `.h` suffix.
#include "examples/rust/function/example_crate.h"
int main(int argc, char* argv[]) {
// The generated bindings are in a namespace with the same name as the
// target crate:
int32_t sum = example_crate::add_two_integers(2, 2);
// ...
}
If needed, one can find and inspect the generated bindings in the “Generated Files and Directories” section for that directory in Code Search or manually building the bindings using:
$ bazel build --config=crubit-genfiles \
//examples/rust/function:example_crate
$ cat "$(bazel info bazel-bin)/examples/rust/function/example_crate.h"
Inspecting the generated file may be useful to look at comments that Crubit leaves behind when it is unable to generate bindings for a given Rust API - e.g.:
// Error generating bindings for `reinterpret_cast` defined at path/lib.rs;l=123:
// Error formatting function name:
// `reinterpret_cast` is a C++ reserved keyword and can't be used as a C++ identifier.
TODO: Provide integration with Chromium/GN and provide GN-oriented examples.
Highlights
For a comprehensive description of the generated FFI bindings, please see the “FFI Bindings Reference”. The list below highlights a few aspects of the FFI bindings that Crubit generates:
- Crubit can generate FFI bindings for C++ and Rust functions even if they
don’t follow the
extern "C"calling convention. - Crubit can replicate the memory layout of most C++ and Rust structs. This
includes non-
#[repr(C)]Rust structs and inheritance hierarchy of C++ classes. - Crubit can generate bindings for APIs that use types from other libraries.
For example, if a function provided by
foolibrary returns a struct defined by a separatebarlibrary, thenfoobindings will automatically reuse bindings generated forbar. - Crubit supports all the usual ways to pass function parameters or return values. In particular, structs can be passed by reference or by value. Values from either language can be moved across stack and heap memory on either side of the FFI boundary. This includes C++ structs with non-trivial, user-defined move constructors.
- Crubit supports generating bindings for specific instantiations of arbitrary
C++ class templates. In particular, bindings for
std::string_viewfrom C++ standard library can be generated even thoughrs_bindings_from_cchas minimal knowledge about this specific type (except for instructingrs_bindings_from_ccto includeFromtrait implementations fromcrubit/support/cc_std_impl/string_view.rs).
TODO: Help users make an informed decision when to use Crubit rather than other FFI tools. (This probably should become a separate sub-chapter of the overview. Some raw information can be currently found in the design.md doc.)
Build System Integrations
Crubit works in two directions. Each direction has a build rule that you add next to the library you want to call:
cpp_api_from_rustlets C++ call Rust.rust_api_from_cpplets Rust call C++.
Here is where you can use each of them today:
| Build system | cpp_api_from_rust (C++ calling Rust) | rust_api_from_cpp (Rust calling C++) |
|---|---|---|
| Bazel | Supported (example) | Supported |
| CMake | Supported (example) | Not yet |
| Other | Through the cargo cpp_api_from_rust subcommand | Not yet |
Crubit is used in production inside Google, and we are working towards the same level of support for the open-source integrations; see Usage outside of Google for the plan. Open-source CI builds the Bazel and CMake examples on Linux x86_64 on every change to Crubit.
Toolchain requirements
Crubit is closely tied to particular compiler releases. This is what lets it automatically generate bindings using idiomatic language features instead of being limited to the C ABI.
Rust
cpp_api_from_rust is built on the Rust compiler’s internal APIs, so it must be
built by the same rustc that compiles the crates it generates bindings for,
and that toolchain must include the rustc-dev component. Crubit supports
recent nightly and stable Rust versions.
A daily test matrix checks that Crubit builds against the last 7 nightly
toolchains and the 2 most recent stable releases. The badges in the
README show the current results, and
nightly.yaml
on the test-matrix branch is the workflow that produces them.
How to set up a compatible Rust toolchain depends on your build system and channel:
-
Bazel (nightly):
crubit_toolchains.configureinMODULE.bazeldownloads a pinned nightly toolchain withrustc-devfor you; see Bazel. -
Nightly with
rustup(CMake or Cargo): Install a recent nightly with therustc-devcomponent:rustup toolchain install nightly --component rustc-devIn a Crubit checkout,
rust-toolchain.tomlselects a nightly toolchain withrustc-devautomatically. -
Stable with
rustup(CMake or Cargo): Add therustc-devcomponent to one of the two most recent stable releases, and setRUSTC_BOOTSTRAP=1when building Crubit and the crates that use the generated bindings (because they rely on a few unstable features):rustup component add rustc-dev --toolchain stable export RUSTC_BOOTSTRAP=1
C++
cpp_api_from_rust generates portable C++20. The examples and CI use Clang, but
the generated code doesn’t depend on a particular compiler or Clang version.
rust_api_from_cpp parses C++ with Clang’s libraries, so the Crubit binary is
built against a specific LLVM commit. The Bazel integration takes care of this
for you, and MODULE.bazel pins a commit that CI updates regularly. The LLVM
release that Bazel downloads for compiling C++ needs to be at least as new as
the LLVM that your Rust toolchain was built with; crubit_toolchains checks
this against the rustc found on your PATH, so a rustc must be on PATH
when Bazel fetches dependencies.
Versions and releases
Crubit doesn’t publish releases or semantic versions yet. Development happens on
main, and every day a GitHub Action moves the
latest tag to the newest
commit on main that passed CI. We recommend that you pin a specific
commit, such as the one latest points to, and upgrade on purpose. Binary
releases are planned; see Usage outside of
Google.
Other build systems
Crubit’s tools are ordinary command-line programs, so you can run them from any build system.
For C++ calling Rust, the cargo cpp_api_from_rust subcommand builds a Cargo
library package, runs Crubit on it, and writes a C++ header and a static library
that any C++ build system can consume. This is what the CMake integration uses.
It is built from cargo/cc_bindings_from_rs/cargo-cpp_api_from_rust in the
Crubit repository.
See Cargo build of Crubit for how to build the tools
themselves, and the Bazel rules for
cpp_api_from_rust
and
rust_api_from_cpp
for how they are invoked.
Bazel
Crubit is a Bazel module named rules_crubit. It provides both directions of
bindings:
cpp_api_from_rustgenerates a C++ library from arust_library. Load it from@rules_crubit//bazel:cpp_api_from_rust.bzl.rust_api_from_cppgenerates a Rust library from acc_library. Load it from@rules_crubit//bazel:rust_api_from_cpp.bzl.
The
examples/build_systems/bazel
project is the reference setup. You can see Crubit in action by running:
git clone https://github.com/google/crubit.git
cd crubit/examples/build_systems/bazel
bazelisk run //:main
Set up your module
Add rules_crubit to your MODULE.bazel, together with the Rust and LLVM
toolchains. The example project’s MODULE.bazel shows the exact versions that
Crubit is tested with:
module(
name = "demo_project",
version = "0.0.1",
)
bazel_dep(name = "rules_crubit", version = "0.0.1")
local_path_override(
module_name = "rules_crubit",
path = "../../..",
)
bazel_dep(name = "rules_rust", version = "0.71.3")
bazel_dep(name = "rules_cc", version = "0.2.25")
bazel_dep(name = "toolchains_llvm", version = "1.9.0")
crate = use_extension("@rules_rust//crate_universe:extensions.bzl", "crate")
crate.from_cargo(
name = "crates",
manifests = [
"//:rust_lib/Cargo.toml",
],
)
use_repo(crate, "crates")
crubit_toolchains = use_extension("@rules_crubit//bazel:extensions.bzl", "crubit_toolchains")
crubit_toolchains.configure(
llvm_version = "23.1.0",
rust_version = "nightly/2026-09-29",
)
use_repo(crubit_toolchains, "llvm_toolchain", "rust_toolchains")
register_toolchains(
"@rust_toolchains//:all",
"@llvm_toolchain//:all",
)
The example uses local_path_override because it lives inside the Crubit
repository. In your own project, use git_override instead, and pin a commit
(for example, the one the
latest tag points to):
git_override(
module_name = "rules_crubit",
remote = "https://github.com/google/crubit.git",
commit = "<commit SHA>",
)
crubit_toolchains.configure configures the Rust and LLVM toolchains for the
build. See Toolchain requirements for
the versions that work.
Also copy the example’s
.bazelrc.
It sets the Rust channel and the C++20 and Clang flags that Crubit expects.
C++ bindings for a Rust library
Add a cpp_api_from_rust target for your rust_library, and depend on that
target from C++:
load("@rules_cc//cc:cc_binary.bzl", "cc_binary")
load(
"@rules_crubit//bazel:cpp_api_from_rust.bzl",
"cpp_api_from_rust",
)
load("@rules_rust//rust:rust_library.bzl", "rust_library")
cc_binary(
name = "main",
srcs = ["main.cpp"],
deps = [
":lib_cc_api",
],
)
rust_library(
name = "lib",
srcs = ["rust_lib/src/lib.rs"],
edition = "2024",
deps = [
"@crates//base64",
],
)
cpp_api_from_rust(
name = "lib_cc_api",
crate = ":lib",
)
The header name is the Rust target’s name with .h appended, and the
namespace is the target name:
#include <iostream>
#include <string_view>
#include "lib.h"
#include "support/rs_std/rs_std.h"
int main() {
// Test we can successfully use a type from a Rust dependency.
base64::engine::GeneralPurpose engine = lib::make_engine();
std::cout << "Successfully linked crubit generated C++ api: "
<< lib::add(1, 2) << std::endl;
// Test we can use bindings generated from the Rust standard library.
rs::std::path::PathBuf temp_dir = rs::std::env::temp_dir();
lib::Gymnastics g = lib::Gymnastics::new_(temp_dir);
std::string_view temp_dir_str = g.as_str();
std::cout << "Successfully used Rust std::env::temp_dir(): " << temp_dir_str
<< std::endl;
return 0;
}
See C++ bindings for Rust libraries for what the generated API looks like, and how to customize it.
Rust bindings for a C++ library
Define a rust_api_from_cpp target next to the cc_library, and add its
automatically generated <name>.hint to the cc_library’s aspect_hints:
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_crubit//bazel:rust_api_from_cpp.bzl", "rust_api_from_cpp")
load("@rules_rust//rust:rust_binary.bzl", "rust_binary")
cc_library(
name = "foo",
hdrs = ["foo.h"],
srcs = ["foo.cc"],
aspect_hints = [":foo_rust.hint"],
)
rust_api_from_cpp(
name = "foo_rust",
cpp_target = ":foo",
)
rust_binary(
name = "main",
srcs = ["main.rs"], # use foo_rust::...;
deps = [":foo_rust"],
)
See Rust bindings for C++ libraries for what the generated API looks like, and how to customize it.
CMake
CMake projects can call Rust from C++ with
cpp_api_from_rust, through
Corrosion, the standard way to
combine CMake and Cargo. Corrosion builds your crate with Cargo, runs Crubit on
it, and gives you a CMake target that contains the generated header and a static
library with the Rust code.
Crubit’s Corrosion support is in a fork of Corrosion while the upstream pull request is reviewed.
Rust bindings for C++ libraries (rust_api_from_cpp) aren’t available in CMake
yet.
There are two example projects, and CI builds both on every change to Crubit:
examples/build_systems/cmake: the minimal setup walked through below.examples/build_systems/cmake_demo: a slightly larger demo that uses several crates.io dependencies.
To run the minimal example, you need CMake 3.22 or later, Ninja, Clang, and
rustup (it installs the Rust toolchain from Crubit’s rust-toolchain.toml):
git clone https://github.com/google/crubit.git
cd crubit/examples/build_systems/cmake
cmake -B build -S . -G Ninja
cmake --build build
./build/main
Build configuration
Crubit uses Cargo to build your Rust library. Changes to your Cargo.toml are
respected when the Crubit CMake targets are built. Your project needs to
provide a library package, because Crubit needs it to produce an .rlib or
.rmeta file.
On the CMake side, Corrosion has many options for tuning the Cargo build. See the Corrosion usage docs for details. Most of these options are passed through to the underlying Cargo build, with a few exceptions:
CRATESdoesn’t work, becausecargo cpp_api_from_rustgenerates bindings for the single library package in the root manifest. Cargo workspaces aren’t supported yet.CRATE_TYPEShas no effect, because Crubit always imports a lib/rlib.OVERRIDE_CRATE_TYPEhas no effect, because Crubit must use--crate-type=lib.
The CMake target that Corrosion creates is an interface library. It contains the generated header and a static library with the Rust code and the generated FFI bindings.
Example setup
First, create a CMake project. See the CMake
documentation
for help. At a minimum, you need a directory (ours is named cmake) with a
CMakeLists.txt at its root that contains:
cmake_minimum_required(VERSION 3.22)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_C_COMPILER clang)
set(CMAKE_CXX_COMPILER clang++)
project(example_project
VERSION 0.1.0
LANGUAGES CXX)
Next, create a Cargo project with this shell command:
cmake/$ cargo new --lib rust_lib
C++ calls this code through Crubit, so the Cargo project must produce a normal Rust library. A binary, static library, or cdylib won’t work.
Corrosion connects the two projects. In CMakeLists.txt, fetch Crubit and
Corrosion, then turn on Crubit for the rust_lib package:
include(FetchContent)
FetchContent_Declare(
crubit
GIT_REPOSITORY https://github.com/google/crubit
GIT_TAG <commit SHA> # If omitted, Corrosion fetches Crubit's latest version.
)
FetchContent_Declare(
Corrosion
# NB: We temporarily require a fork of Corrosion while we upstream our changes.
GIT_REPOSITORY https://github.com/thunderseethe/corrosion
GIT_TAG main
)
FetchContent_MakeAvailable(Corrosion)
corrosion_experimental_crubit(rust_lib)
corrosion_import_crate(MANIFEST_PATH rust_lib/Cargo.toml)
rust_lib is the name of the library package in the Cargo project. By default,
it’s the same as the crate name. If you rename the package, update this line to
use the package name, not the crate or directory name. Corrosion reads the
project’s Cargo.toml to get the metadata it needs.
Corrosion creates a CMake library target named after the Rust package. C++ code
uses that target to depend on the Rust code. For example, to make a CMake
executable main depend on it, add the following to CMakeLists.txt:
add_executable(main main.cpp)
target_link_libraries(main PRIVATE rust_lib)
Crubit names the generated C++ header after the library package. In
main.cpp, include crubit/rust_lib.h:
#include <iostream>
#include <string_view>
#include "crubit/rust_lib.h"
#include "crubit/std.h"
int main() {
// Test we can successfully use a type from a Rust dependency.
base64::engine::GeneralPurpose engine = rust_lib::make_engine();
std::cout << "Successfully linked crubit generated C++ api: "
<< rust_lib::add(1, 2) << std::endl;
// Test we can use bindings generated from the Rust standard library.
rs::std::path::PathBuf temp_dir = rs::std::env::temp_dir();
rust_lib::Gymnastics g = rust_lib::Gymnastics::new_(temp_dir);
std::string_view temp_dir_str = g.as_str();
std::cout << "Successfully used Rust std::env::temp_dir(): " << temp_dir_str
<< std::endl;
// Test calling a function with mutable and shared references
// (aliasing checks).
std::uint64_t dest = 10;
std::uint64_t src = 32;
rust_lib::add_assign(dest, src);
std::cout << "Successfully called rust_lib::add_assign: " << dest
<< std::endl;
return 0;
}
Now C++ can call the Rust code. See C++ bindings for Rust libraries for what the generated API looks like. The CMake integration has the same limitations as Crubit’s other build integrations.
Cargo Build of Crubit
This page explains how to build Crubit’s own tools with Cargo. To use Crubit in your project, start with Build System Integrations; the Bazel and CMake integrations build the tools for you.
cc_bindings_from_rs uses the Rust compiler’s internal APIs (rustc_private),
so it requires a toolchain with the rustc-dev component and must be built by
the same rustc that compiles the crates it generates bindings for. See
Toolchain requirements for supported
toolchains and setup instructions.
cc_bindings_from_rs
In a Crubit checkout, rustup automatically selects the nightly toolchain from
rust-toolchain.toml, and you can build cc_bindings_from_rs with Cargo:
git clone https://github.com/google/crubit.git
cd crubit
cargo build --release -p cc_bindings_from_rs
Find more details on configuring the target directory and troubleshooting below.
If you are using a stable Rust toolchain, set the environment variable
RUSTC_BOOTSTRAP=1:
$ RUSTC_BOOTSTRAP=1 cargo build # further args as above
Target Directory
A default Cargo build will generate a target/ directory adjacent to the
Cargo.toml that will clutter your VCS client with intermediate build files. You
can avoid this by setting your target directory somewhere outside your VCS. A
config.toml can set your target directory to a non-standard location:
[build]
# You can also use something like `~/.cargo/target` if you want to keep the intermediaries around
target-dir = "/tmp/target/"
If you want your config.toml to apply to all your projects, place it at
~/.cargo/config.toml. See Cargo’s configuration
documentation for the
other locations you can place a config.toml and their scope.
Troubleshooting
rustc_driver*.rmeta build dependency
If cargo build ... --bin cc_bindings_from_rs fails to build, then check if the
following problem has been reported as one of the first errors:
error[E0463]: can't find crate for `rustc_driver`
--> cargo/cc_bindings_from_rs/run_compiler/../../../cc_bindings_from_rs/run_compiler.rs:10:1
|
10 | extern crate rustc_driver;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
|
= help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`
If you build Crubit using cargo and rustc that are managed by rustup, then
you can just run the suggested command to add the rustc-dev component: rustup component add rustc-dev.
If you build Crubit using cargo and rustc that are built and installed using
x.py install, then please ensure that x.py’s config.toml covers the
rustc-dev component (h/t @Nadrieril for the
PR to teach x.py install about
this component). For example:
$ cat third_party/rust-src/config.toml
...
# With `extended = true`, `x.py install` will install rustc and the listed
# tools. This is the *only* way to install rust-analyzer-proc-macro-srv, which
# is necessary to use rust-analyzer.
extended = true
tools = [
"cargo",
"clippy",
"rustc-dev", # <= ASKS `./x.py install` TO COVER `rustc_driver*.rmeta`
"rustfmt",
"rust-analyzer",
"rust-analyzer-proc-macro-srv",
"src",
]
...
librustc_driver*.so runtime dependency
cc_bindings_from_rs depends at runtime on librustc_driver*.so. If the
runtime linker cannot find this library, then launching cc_bindings_from_rs
may fail with the following error:
$ third_party/rust-toolchain/bin/cc_bindings_from_rs --help
...
error while loading shared libraries: librustc_driver-871558eb5abca9d6.so:
cannot open shared object file: No such file or directory
rustc is able to find the librustc_driver*.so library, because (on certain
platforms) x.py sets a RUNPATH in the rustc binary:
$ readelf -d third_party/rust-toolchain/bin/rustc | grep RUNPATH
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib]
$ ls third_party/rust-toolchain/lib/librustc_driver-*.so
third_party/rust-toolchain/lib/librustc_driver-871558eb5abca9d6.so
If cc_bindings_from_rs is distributed/installed next to rustc, then you can
fix the linking error by setting an identical RUNPATH when building
cc_bindings_from_rs. This can be done by asking cargo to pass additional
rustc command-line flags (e.g. using RUSTFLAGS or CARGO_ENCODED_RUSTFLAGS
environment
variables).
The following command-line flags should work (they are based on x.py’s
sources):
- Linux:
-Clink-args=-Wl,-z,origin-Clink-args=-Wl,-rpath,$ORIGIN/../lib
- Mac:
-Zosx-rpath-install-name-Clink-args=-Wl,-rpath,@loader_path/../lib
- Windows:
- no extra command-line flags needed (TODO: verify this)
Unstable features on stable
Crubit uses unstable Rust features that are only available in nightly builds. We
rely on setting the RUSTC_BOOTSTRAP environment
variable
to 1 to tell a stable Rust compiler to act like a nightly compiler.
If you attempt to build cc_bindings_from_rs with a stable Rust toolchain and
do not set this variable, the build will fail with error messages like:
error[E0554]: `#![feature]` may not be used on the stable release channel
rs_bindings_from_cc
rs_bindings_from_cc is a Rust binary that depends on C++ libraries (Abseil,
LLVM/Clang LibTooling, and Protobuf). To build it with Cargo in an open-source
checkout, you first need to build and export these C++ dependencies.
Building with C++ dependencies from Bazel
This is how Crubit’s CI checks that the tool builds with Cargo. If you can use Bazel, prefer the full Bazel build instead.
Crubit provides a helper script cargo/build/setup_bazel_env.py that discovers
prebuilt C++ dependencies in Bazel’s output tree, merges them into monolithic
static archives, pre-generates Protobuf headers, and exports the necessary
environment variables.
-
Prebuild C++ dependencies using
bazelisk:$ bazelisk build \ --remote_download_outputs=all \ rs_bindings_from_cc:rs_bindings_from_cc_main \ @protobuf//:protoc -
Discover and package Bazel outputs:
$ python3 cargo/build/setup_bazel_env.py -
Configure your shell environment:
$ source target/bazel_outputs/bazel-env.sh -
Build
rs_bindings_from_ccusing Cargo:$ cargo build --locked -p rs_bindings_from_cc
Custom Build Systems (Chromium, Android, or Other Projects)
When integrating rs_bindings_from_cc into custom build systems (such as
Chromium, Android, or other standalone environments), you must prepare prebuilt
C++ dependencies and pre-generate C++ Protobuf headers before invoking cargo:
-
Pre-generate C++ Protobuf Headers: To avoid build ordering issues and race conditions during parallel Cargo builds, C++ Protobuf headers (
rs_bindings_from_cc/ir.pb.handrs_bindings_from_cc/generate_bindings/generate_bindings.pb.h) must be generated prior to invokingcargo build(see cargo_build_protobuf.md for architectural details). Crubit provides a helper script that invokesprotocto pre-generate these headers:$ python3 cargo/build/generate_proto_headers.py \ --protoc=<path/to/protoc> \ --out_dir=<path/to/generated_headers> -
Configure Environment Variables: Before invoking
cargo, configure the following environment variables:- Abseil:
ABSL_INCLUDE_PATH: Comma-separated list of Abseil header include roots (containingabsl/...).ABSL_LIB_STATIC_PATH: Comma-separated list of directories containing Abseil static archives (.aon Unix,.libon Windows).
- LLVM / Clang:
CLANG_INCLUDE_PATH: Comma-separated list of include directories for LLVM and Clang headers (clang/...,llvm/..., and tablegen-generated headers).CLANG_LIB_STATIC_PATH: Comma-separated list of directories containing Clang/LLVM static archives.
- Protobuf:
PROTOBUF_INCLUDE_PATH: Comma-separated list of include directories containing Google Protobuf headers (google/protobuf/...,utf8_range), as well as Crubit’s pre-generated Protobuf headers directory from step 1.PROTOBUF_LIB_STATIC_PATH: Directory containing the Protobuf static archive (libprotobuf.aorprotobuf.lib).PROTOC: Absolute path to theprotoccompiler executable. This is required byprotobuf-codegenduring the Cargo build to generate Rust Protobuf bindings forir_rust_protoandgenerate_bindings_rust_proto.
- Toolchain & Linker:
-
CC/CXX: C and C++ compilers (e.g. Clang/Clang++). -
CXXFLAGS: Extra C++ compiler flags. You must include-DCRUBIT_LLVM_DEV_DATE=<YYYYMMDD>in addition to standard flags like-stdlib=libc++. For example:LLVM_DATE=$(python3 cargo/build/get_llvm_commit_date.py <llvm_commit_hash>) export CXXFLAGS="-stdlib=libc++ -DCRUBIT_LLVM_DEV_DATE=${LLVM_DATE}" -
RUSTFLAGS: Rust compiler flags for linking (e.g.-C linker=...,-C link-arg=-stdlib=libc++,-C link-arg=-lc++,-C link-arg=-lzstd).
-
- Abseil:
-
Build
rs_bindings_from_ccusing Cargo:$ cargo build --locked -p rs_bindings_from_cc
Protobuf Code Generation in Crubit’s Cargo Build
crubit.rs/overview/cargo_build_protobuf
In Crubit’s Cargo build for rs_bindings_from_cc, protoc is invoked in two
distinct places:
- Externally, before Cargo runs:
cargo/build/generate_proto_headers.py(invoked bycargo/build/setup_bazel_env.pyor external orchestrators like Chromium’sbuild_crubit.py) generates C++ Protobuf headers and source files (.pb.hand.pb.cc). - Internally, during
cargo build: Thebuild.rsscripts of Rust Protobuf crates (cargo/rs_bindings_from_cc/ir_rust_proto/build.rsandcargo/rs_bindings_from_cc/generate_bindings/generate_bindings_rust_proto/build.rs) invokeprotocviaprotobuf-codegento generate Rust bindings.
At first glance, invoking protoc in two separate mechanisms might seem
redundant or fragmented. This document explains why this separation exists, the
technical constraints that necessitate it, and the alternative designs that were
evaluated.
Core Reasons for the Separation
1. C++ Protobufs: Preventing Race Conditions in Parallel Cargo Builds
rs_bindings_from_cc is composed of many modular C++ support crates
(ast_consumer_sys, cc_ir_sys, decl_importer_sys, importer_sys,
ir_from_cc_sys, src_code_gen_sys, etc.). Many of these crates compile C++
code that directly or transitively #include "rs_bindings_from_cc/ir.pb.h".
When Cargo builds these crates in parallel:
- If each crate’s
build.rsattempted to generate.pb.hheaders into a shared directory, parallelprotocinvocations would race against each other, resulting in file locking contention, truncated files, and flaky build failures. - If each crate instead generated
.pb.hheaders into its own isolated$OUT_DIR,protocwould execute redundantly dozens of times. Furthermore, crates cannot easily reference headers generated in sibling crates’ private$OUT_DIRpaths.
Generating all C++ Protobuf headers into a shared directory upfront before Cargo starts ensures that all C++ compilation units find stable, fully-generated headers without any parallel file-system races.
2. Rust Protobufs: Relying on protobuf-codegen and Native Cargo Serialization
Unlike C++ headers, Rust Protobuf code generation does not suffer from parallel build races:
- Rust Protobuf code is generated inside dedicated, isolated crates
(
cargo/rs_bindings_from_cc/ir_rust_protoandcargo/rs_bindings_from_cc/generate_bindings/generate_bindings_rust_proto). - Cargo’s dependency graph guarantees that
ir_rust_protobuilds sequentially before any dependent Rust crates. - The generation is driven by
protobuf-codegen(the official Protobuf crate code generator), which handles:- Emitting the required
crate_mappingconfiguration so imported types resolve across crate boundaries. - Passing toolchain-specific flags (e.g.
--rust_opt=experimental-codegen=enabled,kernel=upb). - Post-processing generated code (e.g. inner/outer attribute adjustments).
- Emitting the required
Keeping Rust code generation in build.rs allows Crubit to rely on standard
Cargo idioms and upstream tooling rather than reverse-engineering
protobuf-codegen in custom scripts.
3. External Preparation is Already Required
rs_bindings_from_cc depends on C++ libraries (LLVM/Clang LibTooling, Abseil,
and Protobuf runtime libraries) that cannot be built from crates.io. An
external build system (Bazel in Crubit’s GitHub CI/CQ, CMake in Chromium
Toolchain Builds) must build these C++ dependencies first, and a Python script
must discover them and configure environment variables (CLANG_..., ABSL_...,
PROTOBUF_..., PROTOC) before cargo build can succeed.
Because an external preparation phase is already mandatory, running
generate_proto_headers.py as part of that step adds no additional phases or
overhead to the build.
Alternative Approaches Evaluated
Alternative A: Move Everything to Python (Pre-generate Rust Code Upfront)
- Concept: Have
generate_proto_headers.py(or a unified script) generate both C++ headers and Rust source files before invoking Cargo. - Why it was rejected:
- Fragility & upstream coupling: Replicating
protobuf-codegenin Python requires manually constructing crate mappings, passing experimentalprotocflags, and applying post-processing text replacements. Any upstream update to theprotobufcrate or its code generator would risk silently breaking our custom Python generator. - Cargo
$OUT_DIRisolation:ir_rust_proto/lib.rsincludes generated code viainclude!(concat!(env!("OUT_DIR"), "/ir.rs"));. Because Cargo assigns unpredictable, hash-suffixed$OUT_DIRpaths per build, Python cannot write directly to$OUT_DIR. Abuild.rsscript would still be needed to copy pre-generated files into$OUT_DIR, offering little benefit over generating them in place withprotobuf-codegen. - Generator consistency: The automated Cargo package generator
constructs Cargo packages for
rust_proto_librarytargets using standardprotobuf-codegeninbuild.rs. Diverging from this model would require special-casing proto crates in the generator.
- Fragility & upstream coupling: Replicating
Alternative B: Move Everything to Cargo via links in Cargo.toml
- Concept: Declare
links = "ir_cc_proto"inrs_bindings_from_cc_ir_cc_proto_sys/Cargo.tomland have itsbuild.rsexportcargo:includemetadata to dependent crates. - Why it was rejected:
- Cargo’s non-transitive metadata limitation: Cargo only exposes
DEP_<NAME>_<KEY>metadata to immediate, direct dependencies. In Crubit, dozens of crates#include "rs_bindings_from_cc/ir.pb.h"transitively. Every intermediate crate’sbuild.rswould have to manually forward these environment variables along the entire dependency tree. This is fragile, verbose, and difficult for the Cargo package generator to maintain.
- Cargo’s non-transitive metadata limitation: Cargo only exposes
Alternative C: Move Everything to Cargo via Ambient File Locking
- Concept: Have C++
_syscrates share an output directory, using an ambient inter-process file lock (e.g.flockon Unix,LockFileExon Windows) incargo/buildso the first crate generates the C++ headers and subsequent crates wait and reuse them. - Why it was rejected:
- Violation of Cargo build invariants: Cargo’s documentation
explicitly advises that
build.rsshould only write inside its own$OUT_DIR. Writing to shared directories outside$OUT_DIRcan interfere with Cargo’s caching, fingerprinting, rebuild detection,cargo clean, etc. - Cross-platform locking complexity: Robust cross-process file locking
across Linux, macOS, and Windows (accounting for stale locks after
aborted builds, permission nuances, and Windows sharing modes) adds
non-trivial complexity to
cargo/build. PROTOCis still needed: Becauseprotocis typically hermetic and not installed on system$PATH, Cargo still requires thePROTOCenvironment variable. Ambient locking would not eliminate the need forsetup_bazel_env.pyto discover and exportPROTOC.
- Violation of Cargo build invariants: Cargo’s documentation
explicitly advises that
Unstable Features
Unstable features used by Crubit
For each feature we use, we document the following:
- Crubit feature: Under which Crubit feature flags do we use this? If it
is under
supported, then the Rust feature breaking can break Crubit users in general. If it is underexperimental, then it only breaks tests and close partner teams. - Use case: What do we use the feature for?
- Exit strategy: What would happen if the feature went away?
auto_traits
-
Crubit feature:
supported -
Use case: To allow defining a deferred pininit trait,
Ctor, which accepts all existing Rust-movable types by value, without any overlapping trait impls. -
Exit strategy: Change to just
impl<T: Unpin> RustMove for T {}. We can go back to the old strategy of abusing theUnpintrait to try to capture “most types”. Any callers which were using Rust-movable non-Unpintypes would need to either add an explicitimpl Ctor for MyType {type Output=Self; ...}, or manually useRustMoveCtor(x)(which implementsCtor) when aCtoris requested forxbutxdoesn’t itself implementCtor<Output=Self, ...>.
custom_inner_attributes
- Crubit feature:
supported - Use case: Used to suppress automatic formatting, to make our golden
tests stable even in the face of e.g. non-idempotency bugs in
rustfmt. - Exit strategy: Disable
rustfmton the golden tests some other way.
negative_impls
- Crubit feature:
supported - Use case: Used to implement
ctor/ nontrivial initialization, so that we can dispatch on the existence of theSelfCtortrait. Also used so that we can pin/unpin a type without adding a field (which is not possible withPhantomPinned). - Exit strategy: For the fields, this is a compatibility break, and we’d
need to add a
PhantomDatafield to all C++ types to mark them as!Send,!Unpin, etc.
extern_types
- Crubit feature:
experimental - Use case: Used for forward declarations, which have an unknown size.
- Exit strategy: Hard to avoid: without this, we get UB for forward
declarations if you use references, due to provenance rules. Don’t need the
forward_declarecrate except for migration from existing C++ code to Rust, where it relies on forward declarations for build performance or cycle-breaking reasons.
sized_hierarchy
-
Crubit feature:
experimental -
Use case: Used for forward declarations, which have an unknown size.
-
Exit strategy: Same as
extern_types.
arbitrary_self_types
- Crubit feature:
experimental - Use case: We need this or an equivalent feature in order to make methods callable on rvalue references but not regular const references, so as to support C++ move semantics. Also likely need it for aliasing-safe references.
- Exit strategy: Likely can’t live without this feature. Lowish risk, in that Rust needs user-defined types to be usable as self types (e.g. it already has this for Rc), although the exact mechanism is as-yet undecided.
abi_vectorcall
- Crubit feature:
experimental - Use case: Used in a test to test non-C calling conventions.
- Exit strategy: Delete test at worst, replace with different non-C calling convention at best.
impl_trait_in_assoc_type
- Crubit feature:
experimental - Use case: Used for returning
impl Ctorfor nontrivial construction. - Exit strategy: Replace with RPITIT, which is stable now.
allocator_api
- Crubit feature: unreleased
- Use case: Used for an ABI-compatible implementation of
std::vectorandstd::unique_ptrthat can reuse the C++ allocator. - Exit strategy: Do not release, or release but require that the Rust global allocator is the C++ allocator.
cfg_sanitize
- Crubit feature: unreleased
- Use case: Used for an ABI-compatible implementation of
std::vectorthat poisons memory in the same way as libc++. - Exit strategy: Do not release, or release but with degraded AddressSanitizer results that hide bugs.
cfg_accessible, stmt_expr_attributes, proc_macro_hygiene
-
Crubit feature: N/A, used internally by Crubit
-
Use case: Crubit uses rustc as a library, which is subject to backwards-incompatible changes.
cfg_accessiblemakes it much easier to write code which is compatible with two versions of rustc at the same time.stmt_expr_attributes,proc_macro_hygienelets us handle backwards-incompatible changes affecting individual expressions. -
Exit strategy: If
cfg_accessiblechanges during the very same update we are trying to be compatible with, we can’t use it in the first place. If it changes in a future version, then the update has already completed successfully, so we can delete the usage (keeping thecfg_accessibleusage that is currently used, and deleting the other one). Same for the rest.For versions we must continue to support, we can use the
rustversioncrate.Finally, in some cases there may be workarounds we can deploy that are only “worse”, not infeasible. For example, to import a trait if it exists, we can
use path::to::*instead ofuse path::to::Trait.
unsized_const_params, adt_const_params
- Crubit feature:
supported - Use case: Better error messages when you get a type error on forward
declarations.
Symbol<(C<'F'>, C<'o'>, C<'o'>)>is so substantially worse thanSymbol<"Foo">that Rust will even hide the type entirely, forcing you to read the name from a separate file (which may not be available to you if you were doing a remote build). - Exit strategy: The
Symbol<(...)>approach works and is tested, and we can switch back with a single flag flip. The only impact is a regression on error messages.
super_let
- Crubit feature:
supported - Use case: Less painfully awkward syntax for in-place initialization.
- Exit strategy: Migrate all calls like
let x = emplace!(y)toemplace!{let x = y;}.
Without this feature, Crubit’s ctor.rs crate must use a syntax like
emplace!{let x = ...} to in-place initialize a pinned value using a C++
constructor. It cannot use the same trick that pin!() used to use, because it
needs to actually call a function to initialize the memory, and so lifetime
extension would not apply: the memory, being passed to a function, would have
its lifetime end at the end of the statement.
This doesn’t work great with rustfmt, which gives up inside the macro call. It
is also confusing, and even alarming, and makes the ctor.rs crate look less
like a set of ecosystem and vocabulary types/traits, and more like a language
dialect.
super_let allows us to instead write let x = emplace!(...), with the macro
expanding to a use of super_let to lifetime-extend the storage subexpression.
There’s another possible exit strategy: define the emplace! macro in an old
Rust edition so that the old temporary lifetime rules apply. This trick was
previously used to implement pin! in the standard library.
allow_internal_unstable
- Crubit feature:
supported - Use case: Allow callers of the super_let version of
emplacenot to require the feature. - Exit strategy: Add
feature(super_let)to all callers.
Unstable features not used by Crubit
The following features are ones we’d hypothetically like to use, but do not.
try_trait_v2
Crubit does aim to support the Abseil error type, absl::Status, which ideally
would be usable everywhere a Result is for ergonomics. For example, we’d
expect you to be able to write code like the following:
#![allow(unused)]
fn main() {
pub fn read() -> Status {
foo()?;
}
}
Where foo returns a Status, or even an io::Result.
For this to work with Status being exactly the same type as it is in C++,
with the same layout, we need to use try_trait_v2.
Until it is stabilized, or the cost/benefit becomes worthwhile, we can work
around it by using Result, and converting when a Result is passed or
returned by value.
fn_traits
Much like try_traits_v2, we’d like to support C++ “function objects” which
implement the call operator. In Rust, without fn_traits, the API is awkward:
given a foo of type std::function, absl::AnyInvocable, or
absl::FunctionRef, you cannot just write foo() to call it, because it cannot
implement the Fn* family of traits. This makes these types less convenient to
use than they are in C++, requiring something like foo.as_closure()() to
create a closure out of a FunctionRef or similar.
(Another use case we’ve examined in the past: in addition to implementing
function objects, fn_traits could also be used to implement overloading,
including overloaded function objects. For example, an overloaded function
could, instead, be a constant with many different implementations of Fn, for
different parameter types. Though, this ends up looking a bit odd:
(mystruct.overloaded)())
min_specialization
We have many use cases for min_specialization, including bindings for C++
templates that have explicit template specialization or partial template
specialization. However, they are not yet pressing, and specialization related
Rust features are especially risky.
inherent_associated_types
Inherent associated types give a natural Rust spelling of the following C++ type definition:
struct Foo {
using MyAlias = int;
};
disjoint_associated_types
disjoint_associated_types was
closed, but not necessarily permanently. See also
https://github.com/rust-lang/rust/issues/20400.
Not every C++ type supports the Rust move operation. For more on this, see Classes and Structs.
Crubit supports passing and returning these non-Rust-movable C++ objects by
value. But since they are not Rust-movable, they cannot literally be returned in
Rust by value: pub fn foo() -> X performs a Rust move of its return type by
definition.
Instead, these objects support lazy construction, in the same style as
moveit. See
ctor.rs. This
results in, for example, the following API differences:
X is rust-movable | X is not rust-movable |
|---|---|
pub fn foo() -> X | pub fn foo() -> impl Ctor<Output=X, ...> |
impl Add<X> for &C | impl<T: Ctor<Output=X>> Add<T> for &C |
The problem comes in with operator overloading: the following is valid:
#![allow(unused)]
fn main() {
impl Add<X> for &C {...}
impl Add<Y> for &C {...}
}
But the equivalent using this lazy-construction trait is not:
#![allow(unused)]
fn main() {
impl<T: Ctor<Output=X>> Add<T> for &C {...}
impl<T: Ctor<Output=Y>> Add<T> for &C {...}
}
Rust doesn’t know that these two are disjoint, meaning that we cannot use the
Ctor trait approach for traits.
An alternative fix would be language support for in-place pinned construction.
That would render the Ctor trait obsolete, and reduce Crubit’s needs around
trait coherence (as well as negative_impls, above).
register_tool
We previously used this attribute to allow Crubit to read annotations on
types/functions. However, the need for end-users to enable a feature gate and
re-declare the tool caused us to migrate away to using #[doc] attributes.
We use custom attributes so that Crubit can round-trip a type correctly, or to
implement automated bridging so that a C++ Status becomes a Rust Result<(), StatusError>, or what have you.
Memory Safety
Crubit is designed so that mixing C++ and Rust does not give up the memory safety that Rust provides. Compared to hand-written FFI, Crubit bindings are safer in several ways:
- Bindings come straight from your compilers. Crubit generates bindings
from the same C++ and Rust compilers, flags, and sources that build your
code, so the bindings always match the real declarations, layout, and ABI.
You no longer have to wrangle hand-written
extern "C"declarations or#[repr(C)]mirror structs that silently drift out of sync as an API changes; Crubit regenerates them on every build. unsafeonly where it matters. Only functions with real safety preconditions, such as those taking raw pointers, areunsafein Rust, rather than every call that crosses the language boundary. Theunsafeblocks in your code are the ones worth reviewing.- Lifetime annotations for C++. Crubit understands lifetimes on C++ references, inferred by Rust’s elision rules or written explicitly in the C++ declaration, and turns them into Rust references that the borrow checker enforces. See References and lifetimes.
What Crubit assumes about C++ APIs
Crubit cannot prove that arbitrary C++ code is sound. Instead, the safe Rust signatures it generates rest on two assumptions about the C++ API:
- A function has no safety preconditions beyond the validity of its parameters.
- A type that Crubit treats as Rust-movable can be moved by Rust without breaking any C++ invariants.
Where these do not hold, mark the function or type CRUBIT_UNSAFE (see
Customizing), and the Rust bindings will require
unsafe.
Build environment
Crubit’s guarantees depend on the bindings seeing exactly the C++ and Rust interfaces that are actually compiled:
- Wrapped libraries must be built with the same compiler version and
compilation settings (flags, environment variables, etc.) that Crubit uses.
An extra
-DSOME_DEFINEor a different compiler version can change a library’s ABI or API. For this reason, Crubit’s outputs are generated at build time and must not be checked into source control, where they would go out of date. - The ABI must be deterministic and reproducible, depending only on the
compilation settings, the compiler version, and the wrapped libraries. For
example,
-Zrandomize-layoutcannot be used without a fixed seed.
Crubit’s build integrations (Bazel and CMake) handle this for you: they run Crubit as part of the build, with the same toolchain and flags as the code it binds.
Reference Safety
The Rust reference documents Undefined Behavior (UB) and says that “it is the programmer’s responsibility when writing unsafe code to ensure that any safe code interacting with the unsafe code cannot trigger these [undefined] behaviors”. A programmer using Crubit bindings has the same responsibility: Crubit is implicitly “unsafe”, and incorrect usage can cause UB. The sections below document requirements for safely using Crubit when working with Rust and C++ references and pointers. The requirements are the same as the ones documented in the Rust reference, but are rephrased below from Crubit perspective.
The safety requirements below focus on avoiding UB related to Rust references and therefore matter in scenarios where Crubit-generated code may create Rust references out of C++ references or C++ pointers:
- Today: Using Crubit-generated Rust bindings for C++ APIs if the bindings
accept, store, or return Rust references. TODO: In the future Crubit should
use
CppRef<T>in the generated bindings (and this should mitigate some of the memory safety concerns). - Using Crubit-generated C++ bindings for Rust APIs if the bindings wrap Rust APIs that accept, store, or return Rust references.
Incorrect C++ lifetime annotations
Incorrect lifetime annotations may lead to UB. Rust’s borrow checker prevents
incorrect lifetime annotations, but lifetime annotations of C++ APIs are not
verified by the C++ compiler and Crubit’s optional lifetime analysis can’t
detect all incorrect annotations. Note that Crubit assumes that lifetime
annotations are correct both for explicit annotations (e.g. int& $a f2(int& $a);) as well as for annotations provided by #pragma clang lifetime_elision.
C++ mutating values referenced by Rust
Mutating a value in C++ is UB if the mutation happens while Rust holds a
references to that value. This applies to Rust shared references (e.g. &T) and
to exclusive references (e.g. &mut T).
Examples of C++ features that may mutate a value that Rust holds a reference to:
- Using copy or move assignment operator of C++ value that Rust has a reference to.
- Mutating public fields of a C++ struct that Rust has a reference to.
TODO: Try to succinctly mention the idea that short-lived / non-retained references are safe from the mutation risk.
Dangling or null references
All references and NonNull
pointers must not be
null, and if they point to a nonzero span of memory, must not be dangling. (The
behavior of a program which violates these rules is undefined.)
C++ doesn’t share these rules, and care must be taken when converting Rust references to and from C++ pointers.
Spans/slices are particularly error-prone: a Rust empty slice (&[]) is
represented using a dangling pointer with length zero, while a C++ empty span
typically uses nullptr. Using a nullptr with a Rust empty slice would result in
UB, as would using a dangling pointer in C++. In order to address this, Crubit
provides non-native types with conversion operators:
- Rust has
absl::{string_view, span<T>}types which provideFrom-based conversions to&strand&[T]. - C++ has
rs::{StrRef, SliceRef<T>}types which provide implicit conversions to and fromstring_viewandspan<T>.
References to uninitialized memory or invalid values
Creating a Rust reference that points to uninitialized memory is UB.
Care should be taken when passing C++ references across the FFI boundary to avoid creating Rust references that point to uninitialized memory. This is especially important for references to types that don’t enforce proper initialization through their constructors or other factory APIs - primitive types like integers are one example.
Note that Crubit-generated C++ bindings for Rust code won’t create C++ references to uninitialized memory for:
- Rust
structs,enums, andunions - all the constructors generated by Crubit guarantee proper initialization rs::char_when constructed in C++ viachar_::from_u32or when passed from Rust. (char_::from_u32_uncheckedis unsafe in Rust sense)
TODO: b/296287315: Support MaybeUninit<T>.
Breaking Rust aliasing rules
Constructing a Rust references that aliases the same address as an already
existing exclusive reference &mut T is UB.
TODO: Provide FFI-related examples.
TODO: Document what runtime checks are provided by Crubit (and link to a separate md document that explains why general checks are infeasible).
Panics and Exceptions in C++/Rust FFI bindings
SUMMARY: Crubit currently requires -fno-exceptions, and converts unwinding
panics into aborts.
Unwinding and Aborting
“Alright so I’m panicking, what else is there to do?”
– “The Hitchhiker’s Guide to the Galaxy” by Douglas Adams
Rust and C++ both support a form of unwinding exception/panic, where ordinary control flow is terminated, and instead control proceeds up the stack, calling destructors along the way, until it is caught, converted to a termination, or completely unwinds the stack (which also leads to termination).
This has a performance cost (in that data must be managed soundly, destroyed if appropriate, in case of unwinding), and a code complexity cost (in that unsafe code must handle unexpected control flow edges soundly), so implementations of both languages allow for unwinding to be disabled, in favor of immediate process termination:
- Rust:
-Cpanic=abort - C++
-fno-exceptions
Crubit, and any FFI, will work if unwinding is disabled for both sides of the FFI boundary. But if it is enabled for one or both sides, and an exception or panic unwinds past an FFI boundary, we need special support to ensure that the behavior is defined.
Supported Configurations
“Who said anything about panicking?” snapped Arthur. “This is still just the culture shock. You wait till I’ve settled down into the situation and found my bearings. Then I’ll start panicking.”
– “The Hitchhiker’s Guide to the Galaxy” by Douglas Adams
Rust: Crubit can create bindings for libraries built with either
-Cpanic=abort or -Cpanic=unwind.
If a panic unwinds past a Crubit FFI boundary, the process will terminate, with
the sole exception of extern "C-unwind" functions. If you define an
extern "C-unwind" function, you must ensure that it is only called by C++ code
which enables exceptions. This responsibility is left to the caller.
C++: Crubit can create bindings for libraries built with -fno-exceptions.
We do not generate extern "C-unwind" interfaces which could propagate a C++
exception, and the behavior is undefined if an exception propagates past a
Crubit FFI boundary. (See b/200067087 for catching this at compile time.)
Crubit assumptions about extern "C" ABI of built-in Rust types
cc_bindings_from_rs makes certain assumptions about internal implementation
details of the Rust compiler. In particular, cc_bindings_from_rs assumes a
specific ABI of extern “C” functions that pass/return values of types that the
improper_ctypes_definitions warning complains about. Because of these
assumptions, the generated ..._cc_api_impl.rs code disables the warning via
#![allow(improper_ctypes_definitions)]. These ABI assumptions are documented
below.
Rust built-in char type
extern “C” thunks generated in ..._cc_api_impl.rs can take char arguments
(and can return char values). (Note that this section talks about the Rust
char type which is different from the C++ char type.)
Rust documentation
says
that “Rust char is 32-bit wide” and
that size_of::<char>() == 4 . Additionally, Rust documentation
describes
some invalid bit patterns that may result in undefined behavior: “a value in a
char which is a surrogate or above char::MAX”.
Rust does not directly document the alignment of char, although it does
say that “most
primitives are generally aligned to their size”. Furthermore Rust does not
guarantee a particular ABI (e.g. whether char value can be passed in a
general-purpose register VS in a vector register VS has to be passed by
pointer). cc_bindings_from_rs assumes that Rust char has the same alignment
and ABI as uint32_t (and therefore the same ABI as rs::char_ from
crubit/support/rs_std/char.h).
The assumptions are verified by assertions that verify the properties of the
target architecture when cc_bindings_from_rs runs (layout.align(),
layout.size(), and layout.abi() assertions in format_type.rs). Similar
assertions are verified on C++ side in support/rs_std/char_test.cc. These
assertions seem unlikely to fail, but if they do, then hopefully char_ can
just be tweaked to wrap another of the C++ integer types.
Rust built-in [T] slice type
extern “C” thunks generated in ..._cc_api_impl.rs can take &[T], &mut [T], *const [T] and similar arguments (or return them).
Rust documentation describes the layout of references and pointers to arrays and slices and also documents that slice references are “represented as a pointer and a length”.
Rust does not document the ABI of slice references (i.e. if the pointer comes
before or after the length in memory). cc_bindings_from_rs assumes that &[T]
has the same layout as rs::SliceRef<T> - a C++ struct with 2 fields: a T*
pointer, and the size_t number of slice elements. check_slice_layout inside
crubit/cc_bindings_from_rs/generate_bindings/format_type.rs verifies the
assumptions on the Rust slice each time a reference or a pointer to a slice is
processed by format_type.rs. Similar assertions are verified on C++ side in
support/rs_std/slice_ref_test.cc. The response to the unlikely failure of
these assertions can be changing the types from which rs::SliceRef is built to
match the Rust side again.
cc_bindings_from_rs does not assume that &[T] and rs::SliceRef<T> have
the same ABI as
std::span<T> from C++ 20.
In particular, empty slices have a different representation in C++ and in Rust -
conversions implemented by rs::SliceRef<T> will take care of using a null or
non-null pointer as appropriate.
Rust built-in &str string reference
extern “C” thunks generated in ..._cc_api_impl.rs can take &str and
similar arguments (or return them). Rust documentation
says that “a &str is made up
of two components: a pointer to some bytes, and a length”, but no additional ABI
guarantees are specified.
cc_bindings_from_rs assumes that &str has the same ABI as &[u8] (see the
previous section) with the additional
requirement that the contents
of [u8] “are always valid UTF-8”. The rs::StrRef type enforces the UTF-8
guarantees. Runtime assertions (check_slice_layout) further verify these ABI
layout assumptions.
cc_bindings_from_rs does not assume that &str and rs::StrRef have the
same ABI as
std::string_view
from C++ 17. In particular, references to empty string slices have a different
representation in C++ and in Rust - conversions implemented by rs::StrRef will
take care of using a null or non-null pointer as appropriate.
Debugging Crubit
crubit.rs/overview/debugging
Crubit is trying to solve a hard problem and is under rapid development. Things go wrong, and it’s nice to have tools and methods to diagnose those problems.
[TOC]
Verbose Logging
NOTE: This feature is under development. There might not be much in the logs yet. See b/449749452 for status, and the design doc crubit.rs-logging.
Crubit tool invocations only log at the error level or higher to keep spam out of build logs, but the stderr logging verbosity level is configurable.
We have a shell script checked in that automates the process of invalidating your local Bazel cache and rerunning Crubit on a specific target while emitting info logs:
common/bazel_support/run_with_verbose_logging.sh
This script accepts zero or more Bazel flags (each starting with --) followed
by one Bazel target label to be built.
Example usage with cc_bindings_from_rs
$ common/bazel_support/run_with_verbose_logging.sh \
--config=llvm-unstable \
//third_party/rust/bytes/v1:bytes
Example usage with rs_bindings_from_cc:
$ common/bazel_support/run_with_verbose_logging.sh \
--config=llvm-unstable \
//file/base
Implementation details
The stderr logging level is configured through the Bazel custom build setting
//common/bazel_support:verbose_log_targets, which accepts a
list of strings that are each interpreted as a target label. It’s probably
easier to just run the script.
Limits of Crubit
Crubit aims to make Rust a viable option for engineering teams with large, pre-existing C++ codebases. There are limits to how well C++ and Rust can interoperate. Tradeoffs still exist, and there can be tension between competing goals. And the tools we have for interop, including Crubit, are many years away from feature-completeness.
Crubit tries to push the envelope with what is possible with C++/Rust interop, automatically supporting cross-language calls with the minimum of developer toil. Where this is not possible, Crubit makes it possible (and easy, in most cases) to work around the mismatch, with documented patterns for the most common deficiencies.
High Level Features
Below is an incomplete overview of features we expect Crubit to support eventually. (This attempts to cover major axes in which interfaces can present challenges for interop.) Included in the overview is the short term estimation of overall difficulty, and long-term confidence levels of feasibility for fully automatic interop. Note that some of these depend on features outside the direct control of the Crubit team, such as in the Rust language and compiler.
We also describe, in abbreviated form, some of the patterns one can employ to work around limitations in Crubit. More detailed information is available in crubit.rs/cpp/cookbook and crubit.rs/errors.
Functions
| Feature | Expected Interop Effort (2027) | Will Crubit automate this (eventually)? |
|---|---|---|
| Call into a non-templated C++ function from Rust, if annotated correctly. | Light | Planned: 2027 |
| Call into a templated C++ function | Depends, see Generics and Templates. | |
| Call into a non-generic Rust function from C++. | Fully automatic | Confidence: Very High |
| Call into a generic Rust function | Depends, see Generics and Templates. |
Most functions will be callable if Crubit is enabled, but it may require more or different code in the other language. A notable example is functions which return non-Rust-movable types, which must be invoked differently from Rust than functions which do not. Or, for Rust calling C++, while you can call any function that is designed to be looked up by ADL, you cannot use ADL to do so.
In the immediate future, there is a long tail of reasons that functions may not get bindings. For example, any unrecognized function attribute will cause the bindings to be disabled. For the foreseeable future, barring changes to Rust, we will also require annotations if a function is overloaded, to give it a unique Rust name. However, we expect unsupported parameter or return types to be the most common reason that a function does not receive bindings. These reasons are expected to go away over time, and we aim for something like 95%+ of non-templated functions receiving bindings by the end of 2027.
Fortunately, missing support for a function is easy to work around. Sometimes, this can be solved with a simple Crubit annotation (such as crubit.rs/errors/unknown_attribute), or some other minor change to the function declaration. In the case of the type being unsupported, enabling Crubit on the target that defines that type will be a common fix. But, at worst, one can wrap it behind a new function which Crubit does support. The new function can use fewer attributes, fewer language features, hide unsupported types behind wrapper types (or void pointers), and so on.
Using C++ as an example:
// This function will always get bindings
inline void DoStuff(void* untyped_out, const void* untyped_in) {
auto* out = reinterpret_cast<UnsupportedType*>(untyped_out);
const auto* in = reinterpret_cast<const OtherUnsupportedType*>(untyped_in);
// arbitrary C++ logic here
out->UnsupportedMethod(*in);
}
Any C++ feature or type that is not supported in the automatically generated Rust bindings can be hidden in this way. (crubit.rs/errors/unsupported_type describes other solutions for the unsupported type case.)
A similar strategy can be employed for Rust.
Non-templated C++ Types
| Feature | Expected Interop Effort (2027) | Will Crubit automate this (eventually)? |
|---|---|---|
| Pass any Rust-movable C++ type by pointer or by value, including inside Rust structs. | Fully automatic | Planned: 2027 |
| Pass any non-Rust-movable C++ type by pointer, including inside Rust structs. | Fully automatic | Planned: 2027 |
| Pass any non-Rust-movable C++ type by value, including inside Rust structs. | Fully automatic | Planned: 2027 |
| A lightweight syntax can be used to perform C++ move construction or assignment | Fully automatic | Planned: 2027 |
C++ types are more complicated than Rust types, and can maintain invariants Rust does not. For example, they can enforce non-destructibility (via a deleted destructor), or automatically maintain pinnedness (via implicitly invoked copy and move constructors). These types will not be as easy to use from Rust, but we are aiming for near 100% coverage, so that functions can typically receive bindings.
Bindings can be generated for a type, even if the fields or other subobjects inside of it do not have bindings. What matters are the high level properties: the type’s layout/ABI, what operations it supports, and so on. So when a type is not supported by Crubit, it almost always works to wrap it in a new type. However, more than for functions, we want to make this necessary as little as possible. Types have a wide blast radius: if a type does not get bindings, then no function using that type gets bindings. It is more important to support all types than to support all functions, and we plan to entirely cover the space of types and fully characterize any cases where they cannot receive bindings.
Non-Generic Rust Types
| Feature | Expected Interop Effort (2027) | Will Crubit automate this (eventually)? |
|---|---|---|
| Pass any non-generic Rust type by pointer or by value in C++. | Fully automatic | Confidence: Very High |
There are currently no known cases where we cannot in principle support using a (non-generic) Rust type from C++.
Containers and Smart Pointers
| Feature | Expected Interop Effort (2027) | Will Crubit automate this (eventually)? |
|---|---|---|
| Pass all standard containers, smart pointers, and idiomatic vocabulary types between languages, with zero or constant-time overhead bridging between corresponding types. | Medium | Confidence: High |
Every type gets some baseline level of support (and the wording around C++ types above applies), but these vocabulary types are valuable to special-case to receive a better Rust API. We do this by rewriting the types in pure Rust. For example, the C++ template class:
template <typename T> struct Span {T* ptr; size_t len;};
Has a Rust equivalent:
#![allow(unused)]
fn main() {
#[repr(C)] pub struct Span<T>(*mut T, usize);
}
If the layouts match, then these can be used interchangeably in most places, even with otherwise no template/generic support.
However, there is a long tail of vocabulary types, and types in the standard
library often have differing implementations, some of which have incompatible
Rust semantics. (GCC’s std::string is pinned, Clang’s is Rust-movable.)
We expect users of Crubit to gravitate towards a set of types that works well in both languages, especially on multiplatform codebases.
Generics and Templates
| Feature | Expected Interop Effort (2027) | Will Crubit automate this (eventually)? |
|---|---|---|
| Instantiate C++ class templates from Rust with concrete types and constants as template parameters. | Fully automatic | Planned: 2027 |
| Instantiate C++ function templates from Rust with concrete types and constants as template parameters. | Medium | Confidence: Speculative |
| Instantiate C++ function and class templates using types defined in the current crate. | Heavy | Confidence: Uncertain |
| Instantiate C++ function and class templates with generic type parameters. Checking of function calls will be deferred to monomorphization time. | Heavy | Confidence: Speculative |
| Instantiate generic Rust types and functions from C++. | Medium | Confidence: Uncertain |
Templates (and generics) are instantiated lazily on use, while Crubit is designed as an ahead-of-time interface/type transpiler. Ultimately, this approach hits its limit here, and a new design involving more directly hooking the compilers together will be necessary to fully support templates and generics in every circumstance.
Crubit plans to (by the end of 2027) support fully instantiated templates or
generic types: functions which return or accept a Foo<Bar>, where Bar is not
a template type parameter. Other uses of templates will need to be decomposed
into building blocks that can receive bindings, or else reimplemented in the
other language (as we do with types like Span).
Abstraction and interfaces
| Feature | Expected Interop Effort (2027) | Will Crubit automate this (eventually)? |
|---|---|---|
| Implement an interface defined by a C++ base class from Rust code. | Heavy | Confidence: High |
| Implement a trait defined in Rust from C++ statically. | Heavy | Confidence: High |
| Implement a dyn-compatible trait defined in Rust using an abstract base class and virtual dispatch in C++. | Heavy | Confidence: Uncertain |
Rust and C++ have many features that are not just “a function can be called”, or “a type can be used”. The way in which you define or dispatch to functions can be a load bearing part of the interface, and doesn’t always have a natural equivalent on the other side, and Crubit will not support all of these for some time yet.
The recommended workaround, in most places, is to define a part of your logic in one language, and then call into the other. For example, write a derived class in C++, but implement all of its virtual methods by calling into Rust functions.
There is an example of this in crubit/examples/cpp/virtual/.
Safety
| Feature | Expected Interop Effort (2027) | Will Crubit automate this (eventually)? |
|---|---|---|
| Safely call C++ functions from Rust, with lifetime preconditions checked by the Rust borrow checker. | Light (where possible) | Crubit gives x% of functions safe bindings |
| Check lifetime constraints on Rust APIs used from C++. | Fully automatic (where possible) | Confidence: Speculative |
Rust and C++ have very different stances on memory safety, but both languages do share a goal to, when reasonable in that language, make it safe to call given functions. In Rust, this is embedded in the type system. In C++, this often takes the form of implementation-specific attributes which the compiler can read to inform what warnings to give.
It is possible to lose safety unless FFI correctly propagates the relevant information to the other side. And in Rust, incorrectly marking functions as unsafe can lead to safety fatigue, where most calls are trivially correct, and the few that aren’t have insufficient review.
Confidence Levels
Confidence levels:
Planned: 2027 We plan on finishing this before the end of 2027, as part of our next major milestone.
Confidence: Very High We know we can achieve this. There may still be caveats for edge cases that require language or compiler work to fix.
Confidence: High We think we can achieve this, but some of the details are not fully designed, and there is always a risk of “unknown unknowns”. May depend on getting features upstream, but we are reasonably confident we can do so.
Confidence: Uncertain There are substantial unknowns in the “how” that will take some dedicated engineering effort to resolve. We think there is a path that looks feasible.
Confidence: Speculative At a high level it seems like this should be possible, but there are major technical, social, or resourcing risks that could prevent it from happening.
Expected Interop Effort (2027)
Fully Automatic Interop Crubit must be enabled on targets.
Light Some declarations need to be annotated. For example, overloaded functions must have a Rust name specified.
Medium A layer of FFI glue code is required, such as new function or type definitions. The patterns involved are straightforward, and documented. Expertise is helpful, but not required.
Heavy Multiple layers of ad-hoc FFI glue are required, even with Crubit, or else the task may require reimplementing significant chunks of code in the other language. Some expertise may be required.
Irreconcilable differences
Expressiveness
C++ is fundamentally a more expressive language, in terms of accepted code patterns, than Rust. While Rust will continue to evolve and gain some forms of expressiveness, many of its strengths draw from its relatively limited expressiveness in certain areas. Therefore, differences between C++ and Rust will always exist, and show up in the source code.
Examples of these include:
- Safety annotations
- Implicit invocation of user-defined constructors, conversion operators, and assignment operators
- Overloading operators in C++ that are not overloadable in Rust
- Text-based macro substitution
Typically, this means that Rust code ends up more verbose than C++ code, though it can also require adding additional annotations to the C++ to better inform how to match C++’s very general semantics with Rust’s more narrow interfaces.
Diverging patterns
Idiomatic patterns are not the same between the two languages, and will continue to diverge further over time. For example, many more objects are pinned in C++ than are pinned in Rust, and this will present an ongoing impedance mismatch.
Ecosystem considerations
The Rust and C++ languages are limited in how far they can evolve to accommodate interoperability. Within each community there are priorities besides interop at play.
High-level design of C++/Rust interop
This document describes the high-level design choices of Crubit, a C++/Rust Bidirectional Interop Tool.
[TOC]
C++/Rust interop goal
The primary goal of Crubit is to enable Rust to be used side-by-side with C++ in large existing codebases.
In the short term we would like to focus on codebases that roughly follow the Google C++ style guide to improve the interop fidelity. Other, more diverse codebases are possible prospective users in the long term, and their needs will be addressed by customization and extension points.
C++/Rust interop requirements
In support of the interop goal, we identify the following requirements:
- Enable using existing C++ libraries from Rust with high fidelity
- High fidelity means that interop will make C++ APIs available in Rust, even when those API projections would not be idiomatic, ergonomic, or safe in Rust, to facilitate cheap, small step incremental migration workflow. Based on the experience of other cross-language interoperability systems and language migrations (for example, Objective-C/Swift, Java/Kotlin, JavaScript/TypeScript), we believe that working in a mixed C++/Rust codebase would be significantly harder if some C++ APIs were not available in Rust.
- Interop will bridge C++ constructs to Rust constructs only when the semantics match closely. Bridging large semantic gaps creates a risk of making C++ APIs unusable in Rust, as well as a risk of creating performance problems. For example, interop will not bridge destructive Rust moves and non-destructive C++ moves; instead it will make C++ move constructors and move assignment operators available to use in Rust code. As another example, interop will not bridge C++ templates and Rust generics by default.
- Interop should be performant, as close to having no runtime cost as possible. The performance costs of the interop should be documented, and where possible, intuitive to the user.
- Interop should be ergonomic and safe, as long as ergonomic and safety accommodations do not hurt performance or fidelity. Where a tradeoff is possible, the interop will choose performance and fidelity over ergonomics; the user will be allowed to override this choice.
- Enable owners of the C++ API to control their Rust API projection, for example, with attributes in C++ headers and by extending generated bindings with a manually implemented overlay. Such an overlay will wrap or extend generated bindings to improve ergonomics and safety.
- Enable using Rust libraries from C++
- However, using C++ libraries from Rust has a higher priority than using Rust libraries from C++.
- Put little to no barriers to entry
- Ideally, no boilerplate code needs to be written in order to start using a C++ library from Rust. Adding some extra information can make the generated bindings more ergonomic to use.
- The amount of duplicated API information is minimized.
- Future evolution of C++ APIs should be minimally hindered by the presence of Rust users.
Proposal and high-level design
We propose to develop our own C++/Rust interop tooling. There are no existing tools that satisfy all of our requirements. Modifying an existing tool to fulfill these requirements would take more effort than building a new tool from scratch or might require forking its codebase given that some existing tools have goals that conflict with our goals.
See the “alternatives considered” section for a discussion of existing tools.
Source of information about C++ API
Interop tooling will read C++ headers, as they contain the information needed to generate Rust API projections and the necessary glue code. Interop tooling that is used during builds will not read C++ source files, to maintain the principle that C++ API information is only located in headers, and that a C++ library can’t break the build of its dependencies by changing source files.
Some interop-adjacent tools (e.g., large-scale refactoring tools that seed the initial set of lifetime annotations) will also read C++ sources. These tools will not be used during builds.
Pros
- Minimal barrier to entry: minimal amount of manual work is required to
start using a C++ library from Rust.
- Encourages leaf projects to start incrementally adopting Rust in new code, or incrementally rewriting C++ targets in Rust.
- C++ API information is located only in headers, regardless of the language that the API consumer is written in (C++ or Rust).
- Interop tooling that generates Rust API projections from a C++ header can
get exactly the same information that the C++ compiler has when processing
a translation unit that uses one of the APIs declared within that header.
- Interop tooling can generate the most performant calls to C++ APIs, without C++-side thunks that translate the C++ ABI into a C ABI.
- Interop tooling can autodetect implementation details that are critical for interop but are not a part of the API surface (for example, the size and alignment of C++ classes that have private data members).
- In alternative solutions, users need to repeat these implementation details in sidecar files. Interop can verify that the specified information is correct through static assertions in generated C++ code, but the overall user experience is inferior.
Cons
- Having to read C++ headers makes interop tooling more complex.
- The Rust projection of the C++ API is only visible in machine-generated
files.
- These are not trivially accessible.
- There is a limit on how readable these files can be made.
- We can mitigate these issues by building tooling that shows the Rust view of a C++ header (for example in Code Search, or in editors as an alternative go-to-definition target).
Customizability
Interop tooling will be sufficiently customizable to accommodate the unique needs of different C++ libraries in the codebase. Interop should be customizable enough to accommodate existing codebases. C++ API owners can:
- Guide how interop tooling generates Rust API projections from C++
headers. For example, headers can provide:
- Custom Rust names for C++ function overloads (instead of applying the general interop strategy for function overloads),
- Custom Rust names for overloaded C++ operators,
- Custom Rust lifetimes for pointers and references mentioned in the C++ API,
- Nullability information for pointers in the C++ API,
- Assertions (verified at compile time) and promises (not verified by tooling) that certain C++ types are Rust-movable.
- Provide custom logic to bridge types, for example, mapping C++
absl::StatusOrto RustResult. - Provide API overlays that improve the automatically generated Rust API.
- For example, the overlays could inject additional methods into automatically generated Rust types or hide some of the generated methods.
More intrusive customization techniques will be useful for template and macro-heavy libraries where the baseline import rules just won’t work. We believe customizability will be an essential enabler for providing high-fidelity interop.
Source of additional information that customizes C++ API projection into Rust
Where C++ headers don’t already provide all information necessary for interop tooling to generate a Rust API projection, we will add such information to C++ headers whenever possible. If it is not desirable to edit a certain C++ header, extra information can be stored in a sidecar file.
Examples of additional information that interop tooling will need:
- Nullability annotations. C++ APIs often expose pointers that are
documented or assumed by convention to be never null, but can’t be
refactored to references due to language limitations (for example,
std::vector<MyProtobuf *>). If C++ headers don’t provide nullability information for pointers in a machine-readable form, interop tooling has to conservatively mark all C++ pointers as nullable in the Rust API projection. The Rust compiler will then force users to write unnecessary (and untestable) null checks. - Lifetimes of references and pointers in C++ headers are not described in a machine-readable way (and sometimes are not even documented in prose). Lifetime information is essential to generate safe and idiomatic Rust APIs from C++ headers.
Additional information is stored in C++ headers
Pros
- Additional information needed for C++/Rust interop will be expressed as
annotations on existing syntactic elements in C++.
- The annotations are located in the most logical place.
- The annotations are more likely to be noticed and updated by C++ API owners.
- API owners retain full control over how the API looks in Rust.
- C++ users may find lifetime and nullability annotations useful. For example, information about lifetimes is highly important to C++ and Rust users alike.
- C++ API definitions are only written once, minimizing duplication and maintenance burden.
Cons
- Annotations that benefit Rust users can bother C++ API owners who don’t
care about Rust. Especially at the beginning of integrating Rust into an
existing codebase, C++ API owners can push back on adding annotations.
- To encourage adoption of annotations, we can develop tooling for C++ that uses lifetime and nullability annotations to find bugs in C++ code.
- The pushback is likely to be short-term: if Rust takes off in a C++ codebase, C++ library owners in that codebase will need to care about Rust users and how their API looks in Rust.
- There may be headers that we cannot (or would not want to) change, for
example, headers in third-party code, headers that are open-sourced, or when
first-party owners are not cooperating.
- We can apply the sidecar strategy to these headers.
Additional information is stored in sidecar files
Additional information needed for C++/Rust interop can be stored in sidecar files, similarly to Swift APINotes, CLIF etc. If sidecar files get sufficiently broad adoption (for example, if annotating third-party code turns out to be sufficiently important that optimizing C++/Rust interop ergonomics there would be worth it), it would make sense to write sidecar files in a Rust-like language, as that provides the most natural way to define Rust APIs.
Pros
- Sidecar files enable more broad adoption of annotations by providing additional interop information without modifying C++ headers. Sidecar files will allow us to annotate headers in third-party code, headers that can’t adopt annotations for technical reasons, or headers owned by first-party owners who are not cooperating.
Cons
- Like in the Use Rust code to customize API projection into Rust alternative, some part of C++ API information is duplicated, which is a burden for the C++ API owners.
- The projection of C++ APIs to Rust is defined in a new language.
- C++ API owners and Rust users will have to learn this language.
- If we expect wide adoption of sidecar files, we will need to create tooling to parse, edit, and run LSCs against this language.
- Annotations in sidecar files are more prone to become out of sync with the
C++ code. When making changes to C++ code, engineers are less likely to
notice and update the annotations in sidecar files.
- Presubmits can catch some cases of desynchronization between C++ headers and sidecar files. However, presubmit errors that remind engineers to edit more files create an inferior user experience.
- Sidecar files create extra friction to modify the code. Where previously
one had to edit only a C++ header and a C++ source file, now one also likely
needs to update a sidecar file.
- When engineers realize that they need to update a sidecar file, opening another file and finding the right place to update creates extra friction to modify code.
- Once engineers understand the extra maintenance burden associated with sidecar files that tend to go out of sync with headers, they will be less likely to adopt annotations in the first place.
Glue code generation
C++/Rust interop tooling will generate executable glue code and type definitions
in Rust and in C++ (not just merely extern "C" function declarations) in order
to achieve the following goals:
- Enable instantiating C++ templates from Rust, and monomorphizing Rust
generics from C++. Enable Rust types to participate in C++ inheritance
hierarchies.
- For example, imagine Rust code using an object of type
std::vector<MyProtobuf>, while C++ code in the same program is never instantiating this type. The Bazelrust_librarytarget that mentions this type must therefore be responsible for instantiating this template and linking the resulting executable code into the final program. We propose that this instantiation happens in an automatically generated “glue” C++ translation unit that is a part of thatrust_library.
- For example, imagine Rust code using an object of type
- Enable automatically wrapping C++ code to be more ergonomic in Rust. For
example:
extern "C"functions in Rust are necessarily unsafe (it is a language rule). We would like the vast majority of C++ API projections into Rust to be safe. In the current Rust language, we can achieve that only by wrapping the unsafeextern "C"function in a safe function marked with#[inline(always)].- C++ API owners can provide rules for automatic type bridging, for
example, mapping C++
absl::StatusOrto RustResult. This conversion necessitates generation of a Rust wrapper function around a C++ entry point that takes advantage of such type bridging.
- Provide stable locations (C++ modules, Rust crates) that “own” the types
from the language point of view.
- For example, when we project a C++ type into Rust, its Rust definition must be located in a Rust crate. Furthermore, all Rust users of this type must observe it as being defined in the same crate in order for every users to consider that they use the same type. Indeed, this is a rule in Rust, that types defined in different crates are unrelated types.
- When we project a Rust type into C++ we could repeat its C++ definition in C++ code any number of times (for example, in every C++ user of a Rust type). This is technically fine because C++ allows the same type to be defined multiple types within a program. Nevertheless, such duplication is error-prone.
Glue code is generated as C++ and Rust source code
Interop tooling will generate glue code as C++ and Rust source files, which are then compiled with an unmodified compiler for that language. The alternative is to generate LLVM IR or object files with machine code directly from interop tooling.
Pros
- It is easy to inject customizations provided by API owners into generated
source code.
- The customizations will be written in the target language, making it (hopefully) intuitive to write them.
- Generated source code can be easily inspected by compiler engineers while debugging interop problems and compiler bugs.
- Generated source code can be inspected and understood by interop users,
who are not compiler experts.
- LLVM IR wouldn’t be meaningful to them.
- Generated source code is processed by the regular toolchain like any other
code in the project.
- It automatically benefits from all performance optimizations and sanitizers that are newly implemented in Clang and Rust compilers.
- We avoid adding a new tool that generates unique LLVM IR patterns.
- We avoid making the job of the C++ toolchain maintainers harder.
Cons
- Interop tooling will be limited to generating LLVM IR and machine code that Clang and Rust compilers can generate.
Glue code and API projections will assume implementation details of the target execution environment
To provide the most ergonomic and performant interop, C++/Rust interop tooling will allow the target codebase to opt into assuming various implementation details of the target execution environment. For example:
- When calling C++ from Rust, interop tooling can either wrap C++ functions in thunks with a C calling convention, or call C++ entry points directly. Thunks cause code bloat and can collectively add up to become a performance problem, so it is desirable to call C++ entry points from Rust directly. Interop tooling can do that only if it may assume a specific target platform and C++ ABI.
Implementation details of the target execution environment that are considered stable enough will be reflected in API projections, for example:
- The C++ standard does not specify sizes of integer types (
short,int,longetc.) To map them to Rust, interop tooling will need to assume a size that they have on the platform that targets in practice. The alternative would be to create target-agnostic integer types (for example,Intin Swift is a strong typedef forInt32on 32-bit targets, andInt64on 64-bit targets), but this makes it harder to provide idiomatic, transparent, high-performance interop. - The C++ standard does not specify whether standard library types like
std::vectorare in any sense Rust-movable; it is an implementation detail. Universal interop tooling would have to conservatively assume non-Rust-movable types. Interop tooling specific to certain environments can rely on libc++ providing a Rust-movablestd::vectorand project it into Rust in a much more ergonomic way.
Pros
- Interop tooling will generate the most performant code sequences to call
foreign language functions.
- If interop tooling generates portable code, it would have some overhead. The overhead can be eliminated by C++ and Rust optimizers at least in some cases, but at the cost of increased build times. For example, eliminating thunks would require turning on LTO, which is not fast, and usually only used for release builds. It is much preferable to not generate thunks in the first place, if the target platform does not need them.
- Ergonomics of API projections will be improved.
- For example, whether a C++ type is Rust-movable or not is an implementation detail in C++, transparent to C++ users of that type, but it makes a huge ergonomic difference in the Rust API projection.
Cons
- C++ code will have additional evolution constraints.
- For example, changing a type from Rust-movable to non-Rust-movable is a non-API-breaking change for C++ users, but it would break Rust users.
- It would be more difficult to switch internal environments to a different C++ standard library.
- Code that is deployed in environments that have incompatible
implementation details won’t be able to use this C++/Rust interop system.
- Alternatively, these executables would have to bring a suitable execution environment with them (e.g., a copy of libc++).
Interop tooling should be maintainable and evolvable for a long time
We should design and implement C++/Rust interop tooling in such a way that we can maintain and evolve it for more than a decade. If Rust becomes tightly integrated into an existing C++ project, specific requirements for interop and API projection rules will keep changing. The more Rust adoption we will have, the more library and team-specific interop customizations we will have to support, and the more it will make sense for the performance team to tweak generated code to implement sweeping optimizations. These kinds of changes should be readily possible, and they should not create conflicts of interest between different users of the interop tooling.
Interop tooling should facilitate C++ to Rust migration
C++/Rust interop tooling should try to create a favorable environment for migrating C++ code to Rust. Specifically, projections of C++ APIs into Rust should be implementable in Rust. This way, a C++ library can be converted from C++ into Rust transparently for its users, as its public API won’t change.
Alternatives Considered: Design decisions
Repeat C++ API completely in a separate IDL
Instead of reading C++ headers in the interop tooling, we would require the user to repeat the C++ API in some other form, for example, in a Rust-based IDL like in the cxx crate, or in sidecar files in a completely new format.
Pros
- Interop tooling can be simpler if it does not have to read C++ headers. But even under this alternative approach, tooling might want to read C++ headers, nullifying this advantage. For example, tooling might want to automatically generate an initial Rust snippet or to suggest in presubmits to adjust the Rust code that mirrors a C++ API when that C++ API changes.
- The most natural way to define Rust APIs is by using Rust code or Rust-like syntax in sidecar files.
- Available Rust APIs are defined in easily accessible checked-in files.
- API definitions written by a human might have higher quality, on average.
Cons
- A big part of the C++ API needs to be duplicated to reliably match the
Rust code with the C++ declarations. The initial code can be generated by
tooling, but it has to be kept in sync. This is a burden for the C++ API
owners, potentially a bigger one than allowing annotations in C++ headers.
- There is a risk that C++ API owners might refuse to own IDL files.
- The need to create a sidecar file creates a barrier to start using C++
libraries from Rust.
- While the duplication overhead is justifiable for widely-used libraries, it is relatively high for libraries with few users and binaries, making it less likely that leaf teams will start adopting Rust.
- When the C++ API is changed, the Rust definitions become out-of-sync with it. Tooling needs to detect this, and the Rust definitions need to be changed (either manually or tool-assisted).
- There is no effective way to verify Rust binding code at the presubmit time of a C++ library other than building downstream projects.
- Mapping Rust API definitions to the original C++ API definitions is more complicated and error-prone. For example, how would we target a specific overload of a function or constructor?
- There is a risk that individual teams will build team-specific tooling that generates IDL files from C++ headers or generates both IDL files and C++ headers from a single source. These solutions are unlikely to scale to existing large codebases and will likely only work for that specific team.
Use Rust code to customize API projection into Rust
An alternative to storing additional information in C++ headers is to put it into Rust code. For example, the cxx crate requires users to re-state the C++ API in Rust syntax, adding information about lifetimes and nullability. The pros and cons of this choice are the same as when defining a special IDL that repeats the C++ API completely (see above).
Generate glue code in binary formats
Instead of generating glue code as textual sources, interop tooling could use Clang and LLVM APIs to emit object files with C++ glue code and use Rust compiler APIs to generate rmeta and rlib files with Rust glue code.
Pros
- More flexibility in the code that can be generated. Controlling LLVM IR
generation allows interop tooling to generate code that an unmodified
compiler can’t generate from textual source code. For example, the Rust
language does not have any constructs that map to
linkonce_odrfunctions in LLVM IR; if the interop tooling embedded the Rust compiler as a library and had more control over how it generates the IR, we could make that happen.
Cons
- Injecting customizations provided by API owners is harder.
- LLVM, Clang, and Rust compiler APIs are not stable. The format of Rust metadata files is not stable either. The larger the API subset we consume from Clang and Rust, the more difficult it becomes to maintain the tooling.
- To generate object files the interop tooling has to ensure that its
Clang/LLVM version and configuration is identical with the Clang compiler
used to build other C++ code.
- We can solve this problem, but it makes the system more fragile, compared to using existing C++ and Rust compilers to compile generated sources.
- From time to time LLVM introduces bugs that cause miscompilations. If interop tooling embeds LLVM, we would be adding another tool that toolchain engineers will need to look into when debugging a miscompilation. We would be making the job of C++ toolchain maintainers harder.
Alternatives Considered: Existing tools
bindgen
bindgen automatically generates
Rust bindings from C and C++ headers, which it consumes using libclang. The
generated bindings are pure Rust code that interfaces with C and C++ using
Rust’s built-in FFI for C
(#[repr(C)] to indicate that a struct should use C memory layout and extern "C" to indicate that a function should use a C calling convention). C++
functions are handled by generating a Rust extern "C" function that has the
same ABI as the C++ function and attaching a link_name attribute with the
mangled name.
See here for an in-depth description of the use of bindgen in Stylo, a Rust component in Firefox.
Pros
- The oldest and the most mature of the existing C++ interop tools (developed since Feb 2012).
Cons
- Deficiencies in safety and ergonomics, for example:
- References are imported as pointers. No lifetimes, no null-safety.
- Constructors and destructors are not called automatically.
- Overloads are distinguished by a numbered suffix in Rust. These numbers clutter the source code and are hard to remember, as they have no meaning. Adding overloads can change the numbering and hence break Rust callers.
- It is impossible to use C++ inline functions and templates from Rust because of bindgen’s architecture1. The architecture is unlikely to change, and therefore, this is a dealbreaker.
Evaluation
bindgen could be used in a project that has very limited C++ interop needs. However, creating safe and ergonomic wrappers for the generated bindings would require additional effort. Our vision and goals for C++ interop are very different from what bindgen provides.
cbindgen
cbindgen automatically generates C or C++ headers for Rust libraries which expose a public C API.
Pros
- An old and mature tool (developed since March 2017).
Cons
-
Shallow understanding of Rust’s modules and types.
cbindgen’s docs point out that “A major limitation of cbindgen is that it does not understand Rust’s module system or namespacing. This means that if cbindgen sees that it needs the definition for MyType and there exists two things in your project with the type name MyType, it won’t know what to do. Currently, cbindgen’s behaviour is unspecified if this happens.”- This limitation seems mostly caused by building
cbindgenon top of thesyncrate.synis able to parse Rust source code into an AST, but there is no facility at thesynlevel for type deduction or module traversal. Building such functionality would require replicating parts of therustccompiler intocbindgen, or alternatively rewritingcbindgenon top of therustc_drivercrate).
-
Support of only
extern "C"functions.- Supporting Rust functions that use the default calling convention would
require generating not only C/C++ headers, but also generating Rust
source with
extern "C"thunks that trampoline into the original function (requiring thatcbindgenstarts generating Rust sources).
- Supporting Rust functions that use the default calling convention would
require generating not only C/C++ headers, but also generating Rust
source with
-
Support of only
#[repr(C)]structs.- Default memory layout of Rust structs is
unspecified
and therefore cannot be determined by code examination at the
synlevel. - Even if the memory layout could be determined, the layout can change in
a future compiler version, or change depending on compilation command
line flags. To prevent using stale layout information, the
auto-generated FFI code should therefore include compile-time assertions
that the layout didn’t change from the FFI generation time. The
assertions should be present both in the generated C/C++ headers and
on the Rust side (requiring that
cbindgenstarts generating Rust sources). The assertions would effectively verify that the FFI generation is driven by the build system (i.e. by Bazel, or Cargo, or GN/ninja, rather than manually) and that the integration between the FFI tools and the build system doesn’t have any bugs (e.g. that it faithfully replicates all relevant compilation flags).
- Default memory layout of Rust structs is
unspecified
and therefore cannot be determined by code examination at the
Evaluation
cbindgen could be used in a project that can create a narrow extern "C" /
#[repr(C)] API and that is ready to manage the risk of incorrect name/module
resolution. Wrapping additional Rust APIs would require extra effort.
Take-aways for Crubit design
Notes and observations about cbindgen can guide some design aspects of
Crubit’s cc_bindings_from_rs tool (that
similarly to cbindgen generates C++ bindings for Rust crates). Using internal
compiler knowledge (e.g. memory layout of structs, name and type resolution)
requires that cc_bindings_from_rs depends on rustc_driver and other internal
crates of rustc. The API of these crates is unstable which might increase the
risk and maintenance cost of Crubit. Nevertheless, our experience with
maintaining tools based on (also unstable) Clang APIs suggests that this extra
risk and cost is likely going to be acceptable.
Build determinism requires that the Rust compiler produces the same output for
the same set of inputs (the same compiler version, the same command-line flags,
the same sources, etc.). This means that (despite conservative reservations
about layout
determinism)
it should be okay to assume that cc_bindings_from_rs and rustc invocations
will observe the same memory layout of structs, but this requires that
cc_bindings_from_rs is built against exactly the same version of
rustc_driver libraries as rustc. (This should also be reinforced by
compile-time assertions in the generated FFI layer.)
cxx
cxx generates Rust bindings for C++ APIs and vice versa from an interface definition language (IDL) included inline in Rust source code. cxx generates Rust and C++ source code from IDL definitions. To check that the IDL definitions match the actual C++ API, cxx inserts static assertions2 into the generated C++ code; it does not, however, read the C++ headers itself. cxx contains built-in bindings for various Rust and C++ standard library types that are not customizable.
As far as we understand, cxx has the following design constraints and goals:
- Ship a stable product for its intended audience.
- As a consequence, improvements such as integrating move semantics are not going to be accepted soon. We understand that cxx is not a vehicle for experimentation. cxx maintainers would prefer us to first show that our ideas work in a fork of cxx or in a different system, such as autocxx, and that our improvements pull their weight given the added complexity.
- Remain simple and transparent. There is a limit on the amount of
complexity that will be tolerated.
- There is a chance that improvements such as modeling C++ move semantics or various attempts at eliminating thunks will not be ever accepted in upstream cxx.
- Non-goal: Automatically provide high fidelity interop.
- cxx is designed for the use case of an executable where C++ and Rust parts communicate through a narrow interface.
- Non-goal: Automatically provide the most performant interop in as many
cases as possible. For example:
- cxx does not attempt to eliminate C++-side thunks. Instead, using LTO is recommended.
- cxx considers it acceptable to allocate all objects of “opaque” types on the heap. Users who find these heap allocations unacceptable for performance reasons are expected to implement a different C++ entry point that does not hit this limitation and bind it to Rust instead of the original C++ API. Heap allocation is acceptable for many C++ classes in most environments, but the exceptions are important enough for us that this is a major restriction.
Pros
- Mature and ergonomic enough today for mixing C++ and Rust in existing codebases with limited C++ interop needs.
- We avoid being on a tech island.
Cons
- cxx’s stability goal makes it hard to experiment with how the Rust API looks.
- Our goals are unlikely to align well with the goals of the intended user audience of cxx. We would be pulling cxx in directions that make it a worse product for its current users.
- Almost no customizability. Users who are not satisfied with what cxx does are expected to wrap the target C++ API in a different C++ API that is more friendly to cxx.
- cxx tries to be compatible with most standard C++ implementations found in the real world, so it cannot take advantage of unique guarantees provided by the target execution environment.
Evaluation
cxx could be used in projects with limited C++/Rust interop requirements. However, we would not be able to implement many interop features that we consider essential (for example, move semantics, templates).
autocxx
autocxx automatically generates Rust bindings from C++ headers. As the name implies, it automatically generates IDL definitions for cxx, which then produces the actual bindings. In addition, autocxx generates its own Rust and C++ code to extend the Rust API beyond what cxx itself would provide, for example to support passing POD types by value. autocxx consumes C++ headers indirectly by first running bindgen on them and then parsing the Rust code output by bindgen.
autocxx’s design goals are similar to our own in this document.
We did a case study on using an existing project’s C++ API from Rust using autocxx.
Pros
- Low barrier to entry: Bindings are generated from C++ headers, no need to write duplicate API definitions.
- Ergonomic mappings for many C++ constructs.
- Open to contributions that change the generated Rust APIs or make architectural changes.
Cons
- Relatively new and immature.
- Cannot (yet) consume complex headers without errors. We’ve managed to import some actual Spanner headers, but there are still enough outstanding issues that we can’t yet do anything useful with Spanner.
- Architecture can make modifications difficult. autocxx is built on top
of two other tools, bindgen and cxx, and the interfaces between these
components can make it harder to make a modification than it would be in a
monolithic tool. Specifically:
- autocxx uses bindgen to generate a description of the C++ API that it can parse easily (as opposed to trying to parse C++ headers either directly or using Clang APIs). Since bindgen was not intended for this purpose, its output lacks some information that autocxx needs, so autocxx has forked bindgen to adapt it to its needs. The forked version emits additional information about the C++ API in the form of attributes attached to various API elements.
- bindgen in turn is built on the libclang API, which doesn’t surface all of the functionality available through Clang’s C++ API. Adding features to libclang requires additional effort and has a 6 month lead time to appear in a stable release (to become eligible to be used from bindgen).
- When errors occur, it can be hard to figure out which of the components is responsible.
- Adding features can require touching multiple components, which requires commits to multiple repositories.
Evaluation
We initially intended to use autocxx to prototype various interop ideas and potentially as a basis for a field trial. We still believe this would be feasible, but after trying to modify autocxx and its bindgen fork during an internal C++/Rust interop study, we feel that autocxx’s complex architecture is enough of an impediment that we could achieve our goals with less total effort by creating an interop tool from scratch that consists of a single codebase and uses the Clang C++ API to directly interface with Clang.
-
Doing so would require either generating C++ source code or interfacing deeply enough with Clang to generate object code for inline functions and template instantiation. ↩
-
And tricks such as suitable type conversions that force the C++ compiler to perform appropriate checks at compile time. ↩
Lifetime Annotations for C++
Summary: We propose a scheme for annotating lifetimes for references and pointers in C++.
Note: This is a living document that is intended to always reflect the most current semantics and syntax of the lifetime annotations.
Introduction
This document proposes an attribute-based annotation scheme for C++ that describes object lifetime contracts. Lifetime annotations serve the following goals:
- They allow relatively cheap, scalable, local static analysis to find many common cases of heap-use-after-free and stack-use-after-return bugs.
- They allow other static analysis algorithms to be less conservative in their modeling of the C++ object graph and potential mutations done to it.
- They serve as documentation of an API’s lifetime contract, which is often not described in the prose documentation of the API.
- They enable better C++/Rust and C++/Swift interoperability.
The annotation scheme is inspired by Rust lifetimes, but it is adapted to C++ so that it can be incrementally rolled out to existing C++ codebases. Furthermore, the annotations can be automatically added to an existing codebase by a tool that infers the annotations based on the current behavior of each function’s implementation.
While the annotation scheme can express a large subset of Rust’s lifetime
semantics, we have omitted some constructs that we do not expect to be necessary
for our purposes. For example, lifetime
bounds
(e.g. 'a: 'b or T: 'a) may be needed rarely enough that we can do without
them, and higher-ranked trait
bounds
(e.g. where for<'a> F: Fn(&'a i32)) are possible only for function types,
which is what they are usually needed for.
We are aware of two existing schemes for annotating lifetimes and flagging
lifetime violations in C++; we describe them in the sections “Alternative
considered:
[[clang::lifetimebound]]” and
“Alternative considered: P1179 /
-Wdangling-gsl” below. Both of
these schemes have limitations that make them unsuitable for our purposes. We
plan to enable our lifetime analysis to understand the existing annotations by
translating them into our annotation syntax internally (where possible).
Proposal
Examples
To give a feel for how the annotations work in practice, we will first show some examples.
Here is a simple example:
const std::string& [[lifetime(a)]] smaller(
const std::string& [[lifetime(a)]] s1,
const std::string& [[lifetime(a)]] s2) {
if (s1 < s2) {
return s1;
} else {
return s2;
}
}
The annotation states that both s1 and s2 may be referred to by the return
value of the function. This implies that the lifetime of the return value is the
shorter of the lifetimes of s1 and s2. In Rust, this example would be
expressed as follows:
#![allow(unused)]
fn main() {
pub fn smaller<'a>(s1: &'a String, s2: &'a String) -> &'a String;
}
Note how the syntax is broadly similar. The main difference is that, unlike in Rust, our proposal does not require lifetimes to be declared.
A lifetime annotation placed after a member function refers to the lifetime of the object the member function is called on:
struct string {
// The returned pointer should not outlive ``*this``.
const char *[[lifetime(a)]] data() const [[lifetime(a)]];
};
Similar to Rust, [[lifetime(static)]] is used to denote a static lifetime. A
common pattern is for a class to have a static function returning a reference to
some default value:
class Options final {
public:
// ...
static const Options &[[lifetime(static)]] DefaultOptions();
// ...
};
The attribute can be applied to references that appear inside a more complex type expression. For example:
const std::vector<const A *[[lifetime(static)]]> &[[lifetime(static)]]
get_static_as();
This expresses that both the reference to the vector and the pointers to the
As contained inside it have static lifetimes.
This roughly corresponds to the following in Rust (with the difference that, unlike C++ pointers, Rust references cannot be null):
#![allow(unused)]
fn main() {
fn get_static_as() -> &'static CxxVector<&'static A>;
}
Lifetimes
Lifetimes are associated with certain types that we call reference-like types. A reference-like type is one of the following:
- A pointer (except pointers to functions and pointers to members)
- A reference (except references to functions)
- A user-defined type that has been annotated as having lifetime parameters. (We will explain user-defined reference-like types in detail in a later section.)
The reason that pointers to functions and references to functions do not have
lifetimes is to be consistent with Rust, where fn types do not have lifetimes
either. In C++, the function that a pointer or reference refers to almost always
exists for the duration of the program execution. There are some exceptions,
such as functions created by a JIT compiler or functions in plugins loaded and
unloaded at runtime. Such functions may be destroyed before the program exits,
but we consider them to be unusual enough that we don’t support annotating their
lifetimes.
Pointers to members don’t have lifetimes because they aren’t pointers in the narrower sense. A pointer to member doesn’t refer to a specific object in memory; rather, it can be used to refer to a specific member of any object of a given type. In implementation terms, a pointer to member is not an address but an offset.
Lifetimes are annotated using the new attribute lifetime1. The
attribute takes one or several lifetime names as arguments. Appendix
A contains a formal description
of the attribute syntax.
For brevity, lifetimes may be implicitly inferred in some situations; this is referred to as lifetime elision, and we describe the specific rules for this later.
There are two lifetime names with special meaning:
static: A lifetime that lasts for the duration of the program.unsafe: A lifetime that cannot otherwise be represented correctly using lifetime annotations. We will discuss the semantics of an unsafe lifetime in more detail below.
In addition, there are two types of lifetimes that cannot be named in a lifetime attribute but that are implicitly associated with reference-like types in certain situations:
- Local lifetime: The lifetime of a pointer to a variable with automatic storage duration.
- Unknown lifetime: A lifetime that has not been annotated and cannot be
implicitly inferred.
The concept of unknown lifetimes is important because it allows us to migrate a codebase to lifetime annotations incrementally. Tools that verify lifetime correctness should assume that operations involving unknown lifetimes are lifetime-correct; this avoids generating large numbers of nuisance errors for code that has not been annotated yet. Note that this makes unknown lifetimes fundamentally different from unsafe lifetimes.
We call static, unsafe, local, and unknown lifetimes constant lifetimes. We call all other lifetimes variable lifetimes; this reflects the fact that they may be substituted by other lifetimes.
The lifetime attribute can be applied to reference-like types in function
signatures, variable declarations (including member variable declarations),
alias declarations, and to user-defined reference-like types when referring to
static members of such types. The sections below give details on how the
attribute can be applied to these constructs and what the semantics are in each
case.
Note that, unlike in Rust, lifetimes are not part of the type. For the purposes of C++ semantics (e.g. function overloading), two types that differ only in their lifetime annotations are considered the same type. This is by design: We don’t want to change the semantics of existing code by adding lifetimes, and this is one of the reasons we have chosen to use C++ attributes; the C++ standard allows compilers to ignore attributes they don’t know, which implies that they have no effect on the C++ semantics.
Lifetime-correctness
The implementation of a function must be lifetime-correct. This section explains what that means.
Most expressions propagate lifetimes in ways that are straightforward. We will therefore explain lifetime-correctness rules only for those cases that are non-trivial.
Dereferencing a pointer or accessing the value referred to by a reference is lifetime-correct in exactly the following cases:
- If its lifetime is static or a variable lifetime
- If its lifetime is local and the access happens during the lifetime of the corresponding local variable.
Dereferencing a pointer with unknown lifetime or accessing the value referred to by a reference with unknown lifetime is not lifetime-correct, but tools should not emit lifetime verification errors in these cases.
operator new returns a pointer with unsafe lifetime. operator delete takes a
pointer parameter that has unsafe lifetime.
Initializing or assigning an object of reference-like type with another object is always correct if the lifetimes of the two objects are the same.
In addition, there are a number of cases where it is permissible to initialize or assign an object of reference-like type with another object that has different lifetimes. We call such an operation a lifetime conversion.
To define lifetime correctness of conversions, we first need to define what it means for one lifetime to outlive another:
- Any lifetime outlives itself.
- The
staticlifetime outlives any variable or local lifetime. - Any variable lifetime outlives any local lifetime.
- A local lifetime
local1outlives another local lifetimelocal2if the object associated withlocal1outlives the object associated withlocal2according to C++’s lifetime rules. - The unsafe lifetime does not outlive any lifetime except itself, and no other lifetime outlives the unsafe lifetime.
- The unknown lifetime does not outlive any lifetime except itself, and no other lifetime outlives the unknown lifetime. However, tools should not emit lifetime verification errors for lifetime conversions involving unknown lifetimes.
Note that no variable lifetime a outlives any other variable lifetime b; our
annotation scheme does not permit specifying lifetime bounds between lifetimes
in the way that Rust
does.
Here are the rules for the correctness of lifetime conversions:
- Lifetime-converting a non-const pointer of type
T_from *[[lifetime(l_from)]]to typeT_to *[[lifetime(l_to)]]is lifetime-correct if and only ifl_fromoutlivesl_to, and- any lifetimes in
T_fromandT_toare identical.
- Lifetime-converting a const pointer of type
T_from * const [[lifetime(l_from)]]to typeT_to * const [[lifetime(l_to)]]is lifetime-correct if and only ifl_fromoutlivesl_to, and- converting
T_fromtoT_tois lifetime-correct.
- The rules for converting references are analogous to those for converting pointers.
- An object of a class
Twith lifetime parameters may not be converted to an object of the same classTbut with different lifetime parameters; see also the sections on variance and special member functions.
We will describe the lifetime-correctness rules for certain other constructs in the specific sections that deal with those constructs below.
lifetime_cast
To permit building safe abstractions on top of APIs that use unsafe lifetimes,
we provide a way to cast unsafe lifetimes to safe lifetimes and vice versa using
a function template called lifetime_cast2. A lifetime_cast is
similar to C++ cast operations such as const_cast and static_cast but may
only be used to change lifetimes.
Obviously, code that uses lifetime_cast must guarantee that the operation is
actually lifetime-correct, i.e. that there is no risk of a use-after-free. Like
unsafe code in Rust, uses of lifetime_cast should therefore be carefully
reviewed and constrained to small parts of the codebase.
lifetime_cast is a function template defined suitably such that the call
lifetime_cast<T>(e) evaluates to e and does not perform any copy or move
operations. Tools will assume that the lifetimes of the result are those
specified in the template argument for T. Apart from lifetime attributes, T
must be the same as decltype(e).
A typical use case for lifetime_cast would be building a container such as
std::vector on top of raw memory allocation primitives such as operator new.
For example, one of the constructors for a vector might look like this:
template <class T>
void vector<T>::vector(size_t size) [[lifetime(a)]]
: size_(size), capacity_(size), data_(lifetime_cast<T *[[lifetime(a)]]>(new
T[size])) {}
Concise syntax using macros
Even with lifetime elision, there is a potential concern that the annotations
will introduce excessive clutter. A lifetime in Rust typically requires only two
characters, e.g. 'a. In contrast, the attribute proposed above,
[[lifetime(a)]], requires at least 15 characters, or more if the attribute is
scoped inside a namespace.
To reduce verbosity, we suggest providing a macro with a short name that expands
to the actual lifetime attribute. The single-character macro name “$” is not
in widespread use in many codebases3; a codebase maintainer would
obviously want to consider carefully what to use it for, but we think lifetimes
could be a worthwhile use. In addition to a general
$(lifetime) macro, we could also define lifetime macros
$a through $z to allow an even more concise
annotation. As an example, this is what the smaller() example from
the beginning would look like with this concise syntax:
const std::string &$a smaller(
const std::string &$a s1,
const std::string &$a s2);
For a more extensive example, see appendix
B, which shows what
std::string_view would look like with these annotations.
Every codebase can of course define its own macro shortcuts that work within the
context of the codebase. A more traditional and still concise macro name would
be LT, with additional macros LT_A through LT_Z for concise single-letter
lifetimes.
For brevity, in the examples that follow, we will use the $ convention.
Pointers and References
As already noted, pointers and references can be annotated with a lifetime, which specifies the lifetime of the object the pointer or reference refers to (the pointee). The lifetime of the pointee must outlive the lifetime of the pointer or reference itself.
For example, let’s look at the example of a double pointer int * $a * $b. The
annotation $b on the outer pointer specifies the lifetime of the inner pointer
of type int *; the annotation $a on the inner pointer specifies the lifetime
of the int. When these lifetime variables are substituted with constant
lifetimes, the lifetime substituted for $a must outlive the lifetime
substituted for $b. This ensures that the int lives for at least as long as
the int * pointer that refers to it.
Functions
Lifetime attributes may be placed in the parameter types and return type of a
function or function type. In addition, for non-static member functions, a
lifetime attribute may be placed after the function declaration to describe the
lifetime of the object the member function is called on, i.e. the lifetime of
the implicit this parameter.
If a translation unit contains multiple declarations of the same function (including its definition), the lifetime attributes in all declarations must be the same.
As in Rust, a function is considered to be parameterized by the lifetimes that
appear in its signature. To express this, a lifetime_param attribute
containing the variable lifetime parameters may be placed in front of the
function definition, like this:
[[lifetime_param(a)]]
int *$a ReturnPtr(int *$a p) {
return p;
}
However, for brevity, this lifetime_param attribute may and should be left out
in most cases. The exception to this is when the signature of the function
contains a function type that itself contains lifetimes; in this case, a
lifetime_param attribute must be added to disambiguate whether the lifetime
should be considered a parameter of the function type or the function. For
example:
// Lifetime $a is a parameter of the function type int*(int*).
void AddCallback(std::function<int *$a(int *$a) [[lifetime_param(a)]]> f);
// Lifetime $a is a parameter of the function AddCallback().
[[lifetime_param(a)]]
void AddCallback(std::function<int *$a(int *$a)> f, int *$a p);
Lifetime parameters on function types are analogous to higher-ranked trait bounds in Rust; unlike Rust, however, we only allow this concept in the context of function types, which is where it is typically required.
TODO: Show an example where we’re passing a pointer to a local variable into the callback and discuss how this is allowed in the HRTB case but not the other case.
Lifetime-converting a function pointer from to a function pointer to of the
same type but with different lifetimes is lifetime-correct if from has either
the same lifetimes as to or lifetimes that are more permissive. This means
that we must be able to substitute the lifetime parameters of from with
lifetime parameters of to such that:
-
Every parameter of
tois lifetime-convertible to the corresponding parameter offrom. (Note the direction of the conversion, which is reversed from what one might initially expect. The idea is thatfromneeds to be able to stand in forto, so we need to be able to convert the parameters oftoto the parameters offrom.4) -
The return type of
fromis lifetime-convertible toto.
Similarly, a virtual member function Derived::f that overrides a base class
function Base::f must have either the same lifetimes or lifetimes that are
more permissive. This means that there we must be able to substitute the
lifetime parameters of Derived::f with lifetime parameters of Base::f such
that:
- Every parameter of
Base::fis lifetime-convertible to the corresponding parameter ofDerived::f. - The return type of
Derived::fis lifetime-convertible toBase::f.
A function call is lifetime-correct if the lifetime parameters of the callee can be substituted by lifetimes from the caller in such a way that converting all arguments to the respective parameter lifetimes is lifetime-correct. If no such substitution can be found, the function call is not lifetime-correct.
Here is an example that illustrates how this works:
void copy_ptr(int *$x from, int *$x *$y to) {
*to = from;
}
int *$a return_ptr(int *$a p) {
int* copy;
copy_ptr(p, ©);
return copy;
}
First of all, the copy pointer is inferred to have lifetime $a because it is
used in the return statement. Let’s use the name $local1 for the lifetime of
the copy variable itself.
Now let’s look at the call to copy_ptr. If we make the substitutions $x =
$a and $y = $local1, we see that the lifetimes of the arguments are
identical to those of the parameters, so it is trivially correct to
lifetime-convert them.
Assume now that return_ptr had been declared with different lifetimes for its
parameter and return type:
int *$a return_ptr(int *$b p) {
int* copy;
copy_ptr(p, ©); // Error, not lifetime-correct.
return copy;
}
Again, the copy pointer has lifetime $a. If we choose the substitution $x
= $a, we can lifetime-convert the second argument but not the first argument
(we need an int *$a but we have an int *$b). If we choose $x = $b, we
can lifetime-convert the first argument but not the second argument (we need an
int *$b * but we have an int *$a *).
Because there is no substitution we can make for $x that allows a
lifetime-correct conversion of the arguments of copy_ptr to the respective
parameter lifetimes, the call is not lifetime-correct.
Lifetime elision
As in Rust, to avoid unnecessary annotation clutter, we allow lifetime annotations to be elided (omitted) from a function signature when they conform to certain regular patterns. Lifetime elision is merely a shorthand for these regular lifetime patterns. Elided lifetimes are treated exactly as if they had been spelled out explicitly; in particular, they are subject to lifetime verification, so they are just as safe as explicitly annotated lifetimes.
We adopt the same lifetime elision rules as Rust. We will expand on the rationale for this below, but first let us present the rules.
We call lifetimes on parameters input lifetimes and lifetimes on return values output lifetimes. There are three rules:
- Each input lifetime that is elided (i.e. not stated explicitly) becomes a distinct lifetime.
- If there is exactly one input lifetime (whether stated explicitly or elided), that lifetime is assigned to all elided output lifetimes.
- If there are multiple input lifetimes but one of them applies to the
implicit
thisparameter, that lifetime is assigned to all elided output lifetimes.
If a function signature contains a function type (in a parameter or the return value), lifetime elision is performed separately for any lifetimes that occur in this function type, independent of the lifetimes in the surrounding function signature. Any elided lifetimes within the function type become lifetime parameters of the function type. See also the discussion of lifetime parameters on function types in this section.
Lifetime elision rules have two requirements:
- They need to be easy for a programmer to remember and apply.
- They should be applicable to as many functions as possible, i.e. they should maximize the percentage of functions whose lifetime semantics correspond to the elided lifetimes. Put differently, they should minimize the percentage of functions which need explicit, non-elided lifetimes.
There is some alignment between these requirements, but some tension too. Working out what the best set of rules is likely requires quite a bit of testing. Instead of doing this, we have for the time being adopted the same set of rules that Rust uses, which presumably have a lot of collective experience embedded in them. The underlying assumption is that Rust and C++ functions do similar things with lifetimes in their interfaces; this assumption seems passable, though surely not perfect. An added benefit of using the Rust rules is that programmers using both languages don’t need to keep two sets of rules in their head.
Once we have static analysis tooling that can run on real-world codebases, we may do some tweaking of the lifetime elision rules, but there would need to be clear benefits to justify giving up commonality with Rust.
Introducing lifetimes to a codebase will have to happen incrementally, and this requires some additional considerations. During the transition, there will be some files that have not yet been annotated, and we may indeed decide to exclude some parts of the code base from annotation permanently. Lifetime elision should not be applied to files that have not been annotated or verified for lifetime correctness; instead, the lifetimes should be assumed to be unknown, as described above.
We propose using a pragma or suitable comment string to mark source files where lifetime elision is allowed, e.g.:
#pragma clang lifetime_elision
Static member variables and non-member variables
Static member variable declarations and non-member variable declarations need not contain lifetime attributes but may do so for clarity.
In general, it may not even be possible to annotate a local variable correctly with the current lifetime annotation syntax. This happens when a local variable may refer to objects of different, unrelated lifetimes. Such a situation is entirely permissible; lifetime inference and verification tools need to deal with this by using a richer internal representation for the lifetimes of local variables.
If a variable has static storage duration, all lifetimes in its type are
implicitly assumed to be static. Any manual annotations that are present may
only specify the lifetimes static or unsafe.
Taking the address of a static member variable or non-member variable yields a
pointer with a lifetime that depends on the variable’s storage duration. If the
variable has static storage duration, the pointer has static lifetime. If the
variable has automatic storage duration, the pointer has a local lifetime.
Classes and non-static member variables
A class may be annotated with one or several lifetime parameters by placing the
new attribute lifetime_param in the class declaration, and a class annotated
in this way is considered to be a reference-like type. All declarations of a
class must be annotated with the same lifetime parameters. (See appendix
A for a formal description of the
attribute syntax.)
When lifetime parameters are substituted with constant lifetime arguments, all of these lifetime arguments must outlive the lifetime of the object they are applied to. This is analogous to the corresponding rule for pointers and references.
Lifetime parameters are necessary when an object of the class contains
references to data that has a different lifetime than the object itself; the
standard C++ types
std::string_view
and std::span
are examples of this.
The lifetime parameters may be used in the declarations of non-static member functions and non-static member variables of the class.
As an example, here is how parts of std::string_view might be
annotated5:
class [[lifetime_param(a)]] string_view {
string_view(const char *$a data, size_type len)
: ptr_(data), len_(len) {}
const char *$a data() const { return ptr_; }
string_view $a substr(size_t pos, size_t count) const;
private:
const char *$a ptr_;
size_t len_;
};
All reference-like types in the declaration of a non-static member variable must
be annotated with the lifetimes static, unsafe, or one of the lifetime
parameters of the class.
If a class contains owning pointers to manually allocated memory, these pointers
will typically be annotated with an unsafe lifetime. Collection types such as
std::vector are examples of this. Member functions that provide access to the
owned memory will typically perform a lifetime_cast to the lifetime of the
owning object. For example, std::vector::at() has the lifetime signature T& $a std::vector<T>::at(size_type) $a.
A class is not required to use any of its lifetime parameters; it may declare lifetime parameters solely for the purpose of associating a lifetime with objects of the class.
Derived classes
Derived classes inherit the lifetime parameters of their base classes. It is not permissible to add lifetime parameters to a derived class; in other words, all lifetime parameters need to be declared on the base class. If a derived class has multiple base classes, only one of these base classes may declare lifetime parameters.
TODO: Having a derived class “silently” inherit the lifetime parameters of its base classes isn’t great because it doesn’t make the lifetime parameters of the derived class visible at the place where it is defined. We should instead consider requiring the lifetime parameters to be re-declared.
The motivation for this rule is to cover the case where a call to a virtual member function in the base class may access member variables of reference-like type in a derived class. A similar situation exists when casting a pointer from the base class to the derived class. In both cases, we want all lifetimes that are relevant to the derived class to be known on the base class.
Special member functions
Special member functions can be annotated with lifetimes just like other member functions, but they deserve special attention because they can be implicitly declared and because they are central to the semantics of C++ value types.
The default constructor and destructor are trivial as they only take a single
reference-like parameter, the implicit this parameter, so we will not discuss
them further.
The lifetimes in the copy and move operations for a type A without lifetime
parameters are as follows (using $s and $o as mnemonics for “self” and
“other”):
A(const A& $o) $s;
A(A&& $o) $s;
A& $s operator=(const A& $o) $s;
A& $s operator=(A&& $o) $s;
Conveniently, these are the lifetimes that are implied by lifetime elision, so they would be omitted in practice.
The implication of these lifetimes is that it is possible to move or assign an
object of type A to another object with a different lifetime.
The situation is slightly more complicated for a type with lifetime parameters. As an example, consider the following class:
struct [[lifetime_param(p)]] B {
int* $p p;
};
(The special member functions are implicitly defaulted.)
The lifetimes of the special member functions on B are as follows:
B(const B $p & $o) $s;
B(B $p && $o) $s;
B& $s operator=(const B $p & $o) $s;
B& $s operator=(B $p && $o) $s;
Note that while the lifetimes of the “self” and “other” objects themselves are
different, their lifetime parameters are the same. This implies that the copy
and move operations cannot extend the lifetime of B::p.
The lifetimes above are not the same as those implied by lifetime elision. Classes with lifetime parameters that use the defaulted copy and move operations need to add explicitly defaulted definitions for these operations.
Alias declarations
Alias declarations can declare lifetime parameters in a similar way to classes.
These lifetime parameters can then be used on the right-hand side of the alias
declaration. In addition, any alias declaration, whether it has lifetime
parameters or not, can use the lifetimes static and unsafe on its right-hand
side.
If an alias declaration is contained inside a class, its right-hand side may not use any lifetime parameters of that class. This is because, in general, an instance of the alias type has no connection to an instance of the class.
Here is an example for an alias declaration with lifetime parameters, again
using std::string_view:
class [[lifetime_param(a)]] string_view {
public:
// ...
using const_iterator [[lifetime_param(i)]] = const char *$i;
const_iterator $a begin() const;
const_iterator $a end() const;
// ...
};
Note that the lifetime_param attribute comes after the type alias name,
whereas in a class declaration it comes before the class name. This may seem
inconsistent, but the placement is dictated by the C++ grammar.
So far, we have pretended that string_view is a class, but it is in fact
itself an alias declaration for basic_string_view<char>, and this alias
declaration therefore has a lifetime parameter:
template <class T> class [[lifetime_param(a)]] basic_string_view {
// ...
};
using string_view [[lifetime_param(a)]] = basic_string_view<char> $a;
The interpretation of this is that string_view is a type with a lifetime
parameter a, and that this lifetime parameter should be forwarded to the
lifetime parameter of basic_string_view<char>.
Templates
A function template or class template may be annotated with lifetime
attributes and, in the case of class templates, a lifetime_param attribute,
just like a non-template function or class.
Explicit template instantiations may not contain lifetime attributes.
Lifetime-correctness of a template may, in general, depend on the template arguments. A template is lifetime-correct if there exists at least one set of arguments for which no specialization exists, that do not result in substitution failure, and for which the specialized template is lifetime-correct.
In general, therefore, lifetimes can only be inferred and verified on a template instantiation. This implies that inference and verification may need to be done multiple times if the same template instantiation is used in multiple translation units. This is slightly unfortunate, but there does not seem to be a good way around it, and it mirrors the fact that such a template instantiation is also compiled multiple times.
To the extent that it is possible to infer and verify lifetimes on the template itself, independent of the template arguments, tooling should do this. In other words, a lifetime-correctness error should be flagged if there is no set of template arguments for which the specialized template is lifetime-correct. Lifetimes should be inferred if they are correct for any template arguments for which no specialization exists and which do not result in a substitution failure.
Partial template specializations should be treated the same way as primary template definitions: Tooling should infer and verify lifetimes on the partial specialization to the extent that this can be done independent of the template arguments.
Full template specializations should be treated the same way as non-template functions and classes: Lifetimes should be inferred and verified on the full template specialization.
When analyzing code that uses a template for which partial or full specializations exist, tooling must of course make sure to refer to the correct specialization.
Function templates
A function template’s type arguments as well as other dependent types may, in general, be reference-like types. Therefore, when a function template instantiation is used (either by calling it or by taking its address), tooling should do the following:
- Verify the lifetime-correctness of the function template instantiation.
- Infer lifetimes for all reference-like types in the signature of the function template instantiation, except for reference-like types that occur in the function template itself and are already annotated with lifetimes there.
The lifetimes inferred for the specialized function template should be used when inferring and verifying lifetimes of functions that use the specialized function template.
Class templates
The type arguments to a class template may be reference-like types. A class
template that is specialized with reference-like types in this way is itself
considered to be a reference-like type. The specialized class template has a
lifetime parameter for each reference-like type that occurs in the template
arguments; these lifetime parameters are in addition to any lifetime parameters
that are annotated on the class template itself using the lifetime_param
attribute. The lifetime parameters associated with a template argument are
implicitly propagated to all uses of that argument in the class template.
TODO: Add a discussion of template template arguments
Lifetimes are assigned to a specialized class template’s lifetime arguments as for any other reference-like type, i.e. depending on the context in which the specialized class template is used they may be explicitly annotated, implied by lifetime elision, or inferred. However, there is a syntactical difference: When lifetimes are explicitly annotated, they are placed in the template arguments instead of after the type, as they would be for other lifetime parameters. For example, here a function that takes a vector of pointers and returns an element of the vector:
int* $a get_ith(const std::vector<int* $a>& $b v, size_t i) {
return v[i];
}
TODO: Discuss dependent types.
A member function of a specialized class template need not be lifetime-correct for all possible assignments of the lifetime parameters associated with the template arguments.
Instead, we only require that every ODR-use of a member function of a specialized class template is lifetime-correct for the lifetimes assigned to the lifetime parameters for that particular use.
A (slightly contrived) example will help to illustrate why these rules are written the way they are.
template <class From, class To>
struct Convert {
To convert(From from) { return from; }
};
void constify(int* [[lifetime(a)]] p,
const int *[[lifetime(a)]] *[[lifetime(b)]] pp) {
Convert<int*, const int*> c;
*pp = c.convert(p);
}
The specialized class template Convert<int*, const int*> has two lifetime
parameters: one lifetime parameter (which we will call x) for the int*
template argument, and one lifetime parameter (which we will call y) for the
const int* template argument.
The Convert::convert() member function is not lifetime-correct if we consider
x and y to be arbitrary variable lifetimes, as it is not lifetime-correct to
lifetime-convert an int *with lifetime x to a const int * with lifetime
y.
However, for the use of Convert in the declaration of c, we infer that both
x and y should be substituted by the lifetime a. Convert::convert() is
lifetime-correct when x and y are substituted in this way.
TODO: Do we need to make this distinction between lifetime parameters and the lifetimes they are substituted with, or can we make the substitution directly?
Variance
As in Rust, we need to establish some variance rules for type and lifetime parameters, but the specific rules differ slightly from Rust.
- Const references and pointers
const T &andconst T *are covariant with respect toT. - Non-const references and pointers
T &andT *are invariant with respect toT. - Class templates are invariant with respect to their type parameters (including lifetimes contained in them).
- All lifetime-parameterized types (classes and alias declarations) are invariant with respect to their lifetime parameters.
The last two rules differ from Rust, which infers the variance of type and lifetime parameters on user-defined types. Unlike Rust generics, C++ class templates are invariant with respect to their type parameters6, and we want to be consistent with this.
Regarding lifetime parameters on types, we restrict ourselves to invariance for simplicity. Rust infers the variance of lifetime parameters from the way they are used in the definition of the type, but in C++, this is impossible to do, at least on a single-translation-unit basis, as a lifetime-parameterized class may only be forward-declared in the current translation unit. For simplicity, and consistency with template parameters, we have therefore decided that lifetime parameters will always be invariant, as we expect this to be sufficient in practice. If this turns out to be too limiting, we may need to provide a way of annotating the variance of lifetime parameters.
Alternative considered: [[clang::lifetimebound]]
Clang already provides a limited ability to annotate lifetimes with the
[[clang::lifetimebound]]
attribute7.
Quoting from the documentation:
The
lifetimeboundattribute on a function parameter or implicit object parameter indicates that objects that are referred to by that parameter may also be referred to by the return value of the annotated function (or, for a parameter of a constructor, by the value of the constructed object).
If the lifetime annotation is applied to aggregates (arrays and simple structs), those aggregates are considered to refer to any pointers or references transitively contained within them.
Here, again, is the smaller() example, but annotated with
[[clang::lifetimebound]]:
const std::string& smaller(
const std::string& s1 [[clang::lifetimebound]],
const std::string& s2 [[clang::lifetimebound]]);
The attribute may also be applied to a member function to indicate that the
lifetime of the return value corresponds to the lifetime of the object. Here is
an example from the [[clang::lifetimebound]] documentation:
struct string {
// The returned pointer should not outlive ``*this``.
const char *data() const [[clang::lifetimebound]];
};
This is an example of the very common case where a member function returns a pointer or reference to part of the object, or to another object owned by it.
The [[clang::lifetimebound]] attribute provides a way to express lifetimes in
many common scenarios, but it does have its limitations:
-
There is no way to differentiate between different lifetimes.
-
There is no way to annotate a static lifetime.
-
The attribute attaches to function parameters and always implicitly refers to the outermost reference-like type8; it is not possible to attach it to part of a type (e.g. to the
T *in aconst std::vector<T *> &). -
The single lifetime is implicitly applied to the outermost reference-like type in the function’s return type (or the value of the constructed object, in the case of a constructor). Again, it is not possible to associate the lifetime with inner reference types in the return value (e.g. the
T *inconst std::vector<T *> &). -
The lifetime of a constructor parameter can be associated with the lifetime of the object being constructed, i.e. with the lifetime of the
thispointer, but this isn’t possible in other member functions. In other words, a member function cannot associate the lifetime of a parameter with the lifetime of the object the member function is called on. -
There is no way to add a lifetime parameter to a struct.
Alternative considered: P1179 / -Wdangling-gsl
The WG21 proposal P1179 describes a static analysis that aims to prevent many common types of use-after-free. It uses an attribute-based annotation scheme to describe the lifetime contracts of functions and to annotate user-defined types containing indirections.
Preliminary implementations of this scheme exist in MSVC and a
fork of Clang. In addition, Clang
trunk implements statement-local warnings inspired by the scheme, which are
enabled by the on-by-default flag -Wdangling-gsl.
The scheme has both advantages and disadvantages compared to the scheme proposed here:
- Advantages
- Can express independent pre- and postconditions for lifetimes, e.g. to
annotate
std::swap(ptr1, ptr2), where the lifetimes of the pointers after the call are swapped compared to before the call. - Can diagnose some cases of iterator invalidation.
- Can express independent pre- and postconditions for lifetimes, e.g. to
annotate
- Disadvantages
- User-defined types can only be annotated as having one of a class of fairly specific lifetime semantics (“SharedOwner”, “Owner”, “Pointer”); arbitrary annotation of classes with lifetime parameters is not possible.
- Cannot refer to lifetimes of pointers in template arguments, e.g. no way
to express
int *$a return_first(const vector<int *$a> &$b v); - Annotations can be verbose and syntactically removed from the objects they refer to.
We believe the limitations of this scheme will restrict its usefulness in the use cases we are interested in. A more in-depth comparison of P1179 with our proposed scheme can be found here.
Appendix A: Lifetime attribute specification
This appendix describes where lifetime attributes may appear and what arguments they can take.
Temporary syntax
We are currently still experimenting with the exact syntax and semantics for the
lifetime annotations. While we are doing so, we will use the general-purpose
annotate and
annotate_type
attributes as stand-ins for the new attributes proposed below.
Attribute definitions
We introduce two new attributes, lifetime and lifetime_param. In practice,
these would be scoped to a namespace (probably clang), but for ease of
exposition, we assume they are in the global namespace.
Attribute lifetime_param
This attribute may be applied to the following:
- A class definition (more formally, it may appear in the attribute-specifier-seq of a class-head)
- An alias-declaration (specifically, the attribute-specifier-seq following the identifier)
The attribute takes one or more arguments. Each of these arguments must be an identifier9; each argument defines a lifetime parameter for the corresponding class.
If the class definition or alias declaration is nested within a class that
itself has a lifetime_param attribute, none of the lifetime parameter names of
the outer class may be used as lifetime parameter names on the nested class
definition or alias declaration.
Attribute lifetime
This attribute may be applied to the following:
-
Types and pointer operators in a function declaration, member function declaration, non-static member variable declaration, or alias declaration
More formally, within the return type, the trailing-return-type or the parameter-declaration-clause of a function declaration or member function declaration, within a non-static member variable declaration, or within the defining-type-id of an alias declaration the attribute may appear:
- In the attribute-specifier-seq of a decl-specifier-seq
- In the attribute-specifier-seq of a type-specifier-seq
- In the attribute-specifier-seq of a ptr-operator (both within a declarator and an abstract-declarator)
-
A non-static member function declaration
More formally, within a member-declarator for a non-static member function, the attribute may appear in the attribute-specifier-seq of the parameters-and-qualifiers.
The attribute takes one or more arguments, each of which must be an identifier
or the keyword static. We call these identifiers lifetime names.
In addition, the following constraints apply:
-
When the
lifetimeattribute is applied to a type, the type must be a class type or alias declaration whose definition contains alifetime_paramattribute.The
lifetimeattribute must have the same number of arguments as thelifetime_paramattribute on the corresponding class or alias declaration. (These arguments define lifetime parameters for the object instance.) -
When the lifetime attribute is applied to a pointer operator, it must take exactly one argument. (This defines a lifetime for the object referenced by the pointer operator.).
-
When the
lifetimeattribute is applied to a non-static member function declaration, it must take exactly one argument. (This defines a lifetime for the implicit object parameter). -
Every lifetime name that appears in a function’s return value must either be
staticor also appear either in- the function’s parameter list, or
- the
lifetimeattribute for the implicit object parameter (in the case of a non-static member function), or - the
lifetime_paramattribute of the class (in the case of a non-static member function).
-
For every constructor of a class that has a
lifetime_paramattribute, every lifetime name that appears in thelifetime_paramattribute must appear in the constructor’s parameter list. -
Every lifetime name that appears in a non-static member variable declaration must either be
staticor one of the lifetime parameters declared in alifetime_paramattribute on the class containing the member variable declaration. -
Every lifetime name that appears in the defining-type-id of an alias declaration must either be
staticor one of the lifetime parameters declared in alifetime_paramattribute on the alias declaration. Note that if the alias declaration is nested within a class that also has lifetime parameters, those lifetime parameters may not appear in the defining-type-id of the alias declaration.
Appendix B: std::string_view with lifetime annotations
To illustrate how lifetime annotations work on a larger code sample, here is an
annotated version of interesting parts of std::string_view. To keep the code
clear, we have omitted basic_string_view and simply stamped out string_view
for the template arguments used in its definition.
// Lifetime "s" is mnemonic for "lifetime parameter of string_view"
class LIFETIME_PARAM(s) string_view {
public:
using const_pointer LIFETIME_PARAM(iter_lifetime) = const char *$(iter_lifetime);
using const_reference LIFETIME_PARAM(iter_lifetime) = const char &$(iter_lifetime);
using const_iterator LIFETIME_PARAM(iter_lifetime) = const char *$(iter_lifetime);
using iterator LIFETIME_PARAM(iter_lifetime) = const_iterator $(iter_lifetime);
using const_reverse_iterator LIFETIME_PARAM(iter_lifetime) =
std::reverse_iterator<const_iterator $(iter_lifetime)>;
using reverse_iterator LIFETIME_PARAM(iter_lifetime) =
const_reverse_iterator $(iter_lifetime);
using size_type = size_t;
static constexpr size_type npos = static_cast<size_type>(-1);
constexpr string_view() noexcept;
constexpr string_view(const string_view $s & other) noexcept = default;
constexpr string_view(const char* $s data, size_type len);
constexpr const_iterator $s begin() const noexcept;
constexpr const_iterator $s end() const noexcept;
constexpr const_reverse_iterator $s rbegin() const noexcept;
constexpr const_reverse_iterator $s rend() const noexcept;
constexpr const_reference $s front() const;
constexpr const_reference $s back() const;
constexpr const_pointer $s data() const noexcept;
constexpr const_reference $s operator[](size_type i) const;
constexpr const_reference $s at(size_type i) const;
// The annotation cannot express that the lifetime parameter of `this` and
// `other` are swapped after the call, so we have to be overly restrictive and
// require `this` and `other` to have the same lifetime parameter.
constexpr void swap(string_view $s & other) noexcept;
// Output buffer may have a different lifetime than this string view's data.
size_type copy(char* buf, size_type n, size_type pos = 0) const;
// Returned substring has the same lifetime parameter as this `string_view`.
constexpr string_view $s substr(size_type pos = 0, size_type n = npos) const;
// `string_view` to compare against does not need to share the same lifetime.
constexpr int compare(string_view x) const noexcept;
private:
const char* $s ptr_;
size_type length_;
};
Notes
-
The attribute will be scoped to some suitable namespace, but for ease of exposition we assume here that it is placed in the global namespace. ↩
-
lifetime_castwill be placed in a suitable namespace, but for ease of exposition, we assume here that it is in the global namespace. ↩ -
“
$” is not part of the standard set of characters allowed in C++ identifiers (including macro names), but the C++ standard permits implementations to allow additional implementation-defined characters, and gcc, Clang, and MSVC allow$as an implementation-defined character. ↩ -
More formally, this is because function types are contravariant in their parameter types. ↩
-
For simplicity, we are showing
std::string_viewas if it was a non-template type. ↩ -
Unless converting constructors and conversion constructors are used to simulate variance. ↩
-
This attribute is inspired by the C++ Standards Committee paper P0936R0. ↩
-
Quoting Richard Smith: “The Clang attribute behaves as if each type has exactly one associated lifetime, and the attribute says in which cases the outermost lifetime of a parameter matches the outermost lifetime of the return value.” ↩
-
Note that this automatically disallows the special lifetime name
static, which is allowed inlifetimeattributes. We make no other constraints on identifiers, but codebases that want to use the lifetime annotations for C++ / Rust interop may want to enforce a rule that prohibits invalid Rust identifiers (e.g. Rust keywords) in thelifetime_paramandlifetimeattributes.. ↩
Static Analysis for C++ Lifetimes
Summary: We describe a static analysis that infers lifetimes in C++ function signatures.
NOTE: This document describes the approach we are currently pursuing but it is a) incomplete, and b) out of date. It has become clear that we are still making changes to the static analysis frequently enough that it does not seem worth updating a document in parallel with those changes. Once the static analysis appears reasonably stable, we plan to update this document to describe it.
Introduction
Lifetime analysis has two goals:
- Infer lifetime annotations to put in C++ function signatures, using the attributes described in this doc.
- Verify lifetime-correctness of function bodies.
To infer and verify lifetimes, we perform a pointer analysis1. For each pointer or other reference-like type, a pointer analysis determines a points-to set consisting of the storage locations it may point to.
There are different approaches to pointer analysis that can be classified according to various properties. The pointer analysis we perform here has the following properties:
- Intraprocedural, context-insensitive. We analyze each function individually and do not take into account how it is called from different callsites.
- Array-insensitive. We treat all elements in an array containing a reference-like type as having the same lifetime.
- Field-insensitive. We treat member variables of reference-like type as having the same lifetime as the object they are contained in (unless they carry a lifetime annotation).
- Flow-sensitive. When analyzing a function, we take statement ordering and control flow into account. We believe flow sensitivity is important to avoid inferring overly restrictive lifetimes and emitting false positive errors.
The pointer analysis we perform is relatively coarse-grained in that we do not distinguish between different storage locations with the same lifetime; equivalently, we can say that we identify a storage location merely by its lifetime.
A points-to set is therefore just a set of lifetimes; a reference-like object is also simply identified by its lifetime. The state that is tracked during the analysis is therefore just a mapping from a lifetime (identifying the reference-like object) to a set of lifetimes (identifying the storage locations it may point to).
This coarse-grained approach simplifies the analysis and is sufficient for our purposes because we are only attempting to infer and verify statements about lifetimes.
Analysis of a translation unit
We analyze all functions in a translation unit for which we have a definition.
We attempt to analyze all of these functions in topological order so that callees are analyzed before callers. Where recursion makes this impossible, we analyze the functions that take part in the recursive cycle in arbitrary order. We accept that this may make it impossible to infer lifetimes for functions in a recursive cycle.
Analysis of a function
As explained in the introduction, we identify an object (often called a storage location in pointer analysis) merely by its lifetime.
A points-to set is therefore simply a set of lifetimes. It represents the set of objects that a reference-like type or glvalue can be referencing at some point of execution of the program. We will sometimes refer to the objects in a points-to set as pointees.
We associate each local variable in the function with a different local lifetime. This serves two purposes: a) It reflects the fact that local variables do, in general, have different lifetimes, and this is important for lifetime verification. b) It allows us to associate a different points-to set with different local variables of reference-like type, and this is required to make the analysis precise enough.
We perform a data-flow analysis using the Clang dataflow framework (documentation) to propagate points-to sets through the function. After the analysis is complete, we produce lifetime annotations from the points-to sets; if these lifetime annotations are different from existing annotations (ignoring pure renamings), we output the new annotations as suggested edits.
The data-flow analysis tracks the following state:
- For each reference-like object (identified by its lifetime), the points-to set of that reference-like object
- For each expression of reference-like type, the points-to set of the expression
- For each glvalue expression, the points-to set representing the glvalue’s referent
- If the function’s return type is a reference or pointer type, a points-to set for the return value
The join operation on points-to sets means taking the union of the two sets.
The initial state for the data flow analysis is produced as follows:
- Associate each parameter of reference-like type with a points-to set containing a new unique regular lifetime representing the pointee.
- If the pointee is itself of reference-like type, recursively associate that pointee with a points-to set containing a new regular lifetime, and so on.
During the analysis, we propagate points-to sets through expressions and update the points-to sets of reference-like objects.
After the analysis is complete, we obtain lifetime annotations by examining the points-to sets of all parameters of reference-like type and the return value (if applicable), descending into pointees that are themselves of reference-like type.
For every points-to set, we look at the set of lifetimes of its pointees. If there are multiple lifetimes, they are substituted by a single lifetime. This lifetime then becomes the lifetime of the corresponding reference or pointer type in the signature.
Here are some examples:
void foo(int* from, int** to) {
// from_pointee (int): '1
// to_pointee (int *): '2
// to_pointee_pointee (int): '3
// from: { from_pointee }
// to: { to_pointee }
// to_pointee: { to_pointee_pointee }
*to = from;
// to_pointee: { from, to_pointee_pointee }
}
TODO: Explain. Also talk about why, after the assignment *to = from, we keep
to_pointee_pointee in the points-to set and how we can, in some cases,
eliminate it. (Distinguish between scalar and aggregate pointees – the latter
are arrays, for example. We can only delete existing pointees if *to has a
single pointee and it’s scalar.)
int* target(int* p1, int* p2) {
// p1_pointee (int): '1
// p1: { p1_pointee }
// p2_pointee (int): '2
// p2: { p2_pointee }
int** pp;
if (foo()) {
pp = &p1; // pp: { p1 }
} else {
pp = &p2; // pp: { p2 }
}
// pp: { p1, p2 }
int local = 42;
*pp = &local; // glvalue on left side is { p1, p2 }, so:
// p1: { p1_pointee, local }
// p2: { p2_pointee, local }
return p1; // rval: { p1_pointee, local }
}
TODO: Explain. Also mention how this is an example where we have two pointees on
the left hand side, so we can’t eliminate existing pointees from p1 and p2.
Function calls
Here is how we handle function calls:
- Create a mapping from callee lifetimes to points-to sets. For each variable lifetime that occurs in the callee’s parameter list, find the union of all points-to sets in those argument positions to yield a mapping from lifetimes to points-to sets.
- Propagate points-to sets to output parameters. For each lifetime
'lin an invariant argument position, replace the argument’s existing points-to set with the points-to set established for'lin Step 1. - Step 3: Determine points-to set for the return value. If the return
value is of reference-like type with lifetime
'l, find the points-to set established for'lin Step 1; this becomes the points-to set for the call expression’s value.
If the 'static lifetime occurs in output parameters (i.e. in invariant
position) or in the return value, the callee may be returning references to
pointees that do not occur as inputs to the callee. Therefore, when we encounter
the ’static lifetime in these positions, we create new pointees for the
corresponding outputs.
Here is an example of how this works:
void copy_ptr(int *'x from, int *'x *'y to) {
*to = from;
}
int * get_lesser_of(int * arg1, int * arg2) {
// arg1_pointee (int): '1
// arg2_pointee (int): '2
// arg1: { arg1_pointee }
// arg2: { arg2_pointee }
int* result = arg2;
// result: { arg2_pointee }
if (*arg1 < *arg2) {
copy_ptr(arg1, &result);
// &result: { result }
// 'x pointees: { arg1_pointee, arg2_pointee }
// result: { arg1_pointee, arg2_pointee }
}
return result;
}
TODO: Continue exposition
Virtual member functions
Inferring lifetimes for virtual member functions is complicated by two factors:
- The lifetimes of the base class member function are constrained by the lifetimes of all of its overrides.
- The definitions of the overrides and the base class function (if it is not pure virtual) are typically contained in different translation units, and we plan to analyze each translation unit individually.
For more details, see this section in the lifetime annotation specification.
We will describe an approach that can infer and update lifetimes for virtual member functions progressively, as each translation unit is processed.
If a translation unit contains definitions for multiple overrides, or if it contains the definition of the the base class function and at least one override, we analyze these definitions in topological order from base class to more derived class.
If the definitions are contained in different translation units, we effectively process them in the same order because we analyze dependencies of a library before analyzing the library itself, and libraries containing derived classes generally depend on the library containing the base class.
TODO: The description above implicitly assumes we’re talking about the initial change where we add lifetimes across the codebase. Discuss also how this applies when people are editing code.
When we encounter the definition of a virtual member function (whether it is the base class implementation or an override), we first perform lifetime inference on its implementation, as for any other function, and update the declaration of the member function in its containing class.
If the function is an override, call it Derived::f, we then update the
lifetimes of every base class function Base::f that it overrides. (There may
be several if there is a chain of overrides.) We do so as follows:
- If the declaration of
Base::fdoes not yet contain any lifetime annotations, annotate it with the lifetimes ofDerived::f. Because we process base class functions before derived class functions, this case can only occur ifBase::fis pure virtual. - If the existing lifetimes of
Base::fare more permissive than the lifetimes inferred forDerived::f, perform lifetime substitutions on the lifetimes ofBase::funtil they are at most as permissive as those ofDerived::f. - If the existing lifetimes of
Base::fat most as permissive as the lifetimes inferred forDerived::f, do nothing.
TODO: Can we ever get caught in a situation where neither the second nor the
third point above applies? I think we’ll always be able to restrict the
lifetimes of Base::f until they’re compatible with Derived::f, but this
needs a formal argument.
TODO: Discuss how the lifetime changes affect callers – may need to process them again.
TODO: Show an example
Templates
Templates pose a specific challenge to lifetime analysis:
- Reference-like types may occur in the template itself as well as in template arguments and dependent types.
- For reference-like types that occur in the template, we wish to infer and
check lifetimes on the template itself to the greatest extent possible. This
reflects the fact that, even though C++ templates are not really generics,
they are often used as if they were. However, the semantics of C++ templates
pose two difficulties here:
- Templates may be specialized, and we must be careful not to apply the lifetimes inferred on the primary template to the specialization.
- The inferred lifetimes and the lifetime correctness of a template may, in general, depend on the template arguments, even if the template arguments and dependent types do not contain any reference-like types. We show an example of this below.
The lifetime annotation specification defines what the semantics of lifetimes on templates should be but does not say how they should be implemented. That is the purpose of this section.
Example scenarios
Before we discuss generally how we will analyze templates, let us look at some scenarios that may occur.
As an example of why we want to be able to analyze templates themselves, let’s
take a look at part of a simplified implementation of std::vector:
template <class T>
class vector {
public:
vector(const vector& other);
T* $a begin() $a { return data_; }
T* $a end() $a { return data_ + size_; }
private:
T* data_;
size_t size_;
};
We should be able to infer the lifetimes of begin() and end() from the
template itself. These member functions operate only on pointers to T, and the
lifetime behavior of a pointer to T is independent of the type T
itself2.
On the other hand, we cannot infer the lifetimes of the copy constructor. It
calls the copy constructor of T, and as
explained in the
lifetime annotation specification, copy and move operations can have two
different lifetime signatures.
Here is another example of how lifetimes can depend on a template argument:
template <int i>
int* return_ith(int* i0, int* i1) {
if (i == 0) {
return i0;
} else {
return i1;
}
}
This example is contrived, but it is certainly not implausible that a trait argument could affect behavior in a similar way.
While these examples do show the limitations of lifetime analysis on templates,
we likely won’t need to do anything subtle to detect them within the analysis.
In the case of the copy constructor of vector, we will notice when calling the
copy constructor of T that we’re doing member lookup on a dependent type and
that we can’t continue the analysis. In the case of return_ith(), we will be
able to analyze the function, but we will conclude that the lifetimes of all
pointers involved are the same. This is more restrictive than the result we
would obtain if we analyzed a template instantiation, but this limitation may be
acceptable.
General approach
The constraints described above imply that lifetime analysis of templates need to proceed in two phases:
-
Analysis of the template itself. We first attempt to infer lifetimes on the template itself, as well as any partial or full specializations, to the extent that the lifetimes do not depend on template arguments. If the inferred lifetimes are different from the function’s current (possibly elided) lifetimes, we generate a corresponding annotation. If we cannot infer lifetimes for the function, we annotate all lifetimes on the function as unsafe. This is required to distinguish this case from the situation where we were able to infer lifetimes and those lifetimes are elided.
TODO: Is there any alternative to marking the lifetimes unsafe? This isn’t what we usually use unsafe lifetimes for, but I also don’t really want to invent yet another syntax.
Performing lifetime analysis on the template itself, rather than only on instantiations, serves two purposes: a) It documents the lifetimes in the code, and b) it saves us from having to analyze every instantiation in cases where the lifetimes don’t depend on template arguments.
-
Analysis of template instantiations. In the following situations, we infer lifetimes on a function template instantiation or member function of a class template instantiation that is called in the translation unit we are analyzing:
- If the template itself contains reference-like types but does not provide lifetimes for these.
- If the template arguments contain reference-like types.
We use the inferred lifetimes when performing lifetime analysis on the callers of these functions, but we obviously cannot produce annotations for these inferred lifetimes.
As discussed in the lifetime annotation specification, any lifetimes in a template argument should be propagated to all uses of the argument. Clang does not provide a built-in mechanism for this, so this needs to be done in the lifetime analysis code.
Verifying lifetime correctness
TODO
Generating error messages
If we detect that there is a lifetime error – either because a function is returning a reference to a local or because there is a lifetime error inside the function – we want to produce an easily comprehensible error message that explains the error.
TODO: Explain how
Alternative considered
We previously considered an alternative approach that built a set of constraints between lifetimes involved in the function. Unfortunately, this approach produced wrong results on some fairly simple examples involving variable overwrites. A coworker identified a way to extend the approach in a way that overcame many of these limitations, but this extension introduced additional complexity. In the end, we decided that the approach based on points-to-sets was the simpler alternative.
Differences from Rust
The exclusivity rule
The borrow checker in Rust, in addition to checking lifetimes, also enforces the exclusivity rule: at any given time the program may have either one mutable reference or any number of immutable references to the same storage location.
The exclusivity rule protects against certain kinds of memory safety errors. For example, if it was applied to C++, it would catch the use after free here:
int test() {
std::vector<int> xs;
xs.push_back(10);
const int &x0 = xs[0]; // `x0` borrows `xs` here.
xs.push_back(20); // exclusivity error: `xs` is mutably borrowed here,
// overlapping with the `x0` borrow.
return x0; // `xs` is borrowed by `x0` at least until here
// because `x0` is used here.
}
Most C++ iterator invalidation bugs could be prevented by enforcing exclusivity: while there are outstanding iterators that borrow the container, the container can’t be mutated.
From our experience porting woff2 from C++ to Rust, adjusting existing code to follow the exclusivity rule is one of the most difficult steps in porting. Therefore, it makes sense to separate rolling out lifetime checking from exclusivity checking. Lifetime checks without exclusivity checks don’t guarantee memory safety, but they catch memory safety issues on their own, and should not require many adjustments to C++ code.
Exclusivity checking could be rolled out in an optional second step. This would not only provide additional memory safety to the C++ code but would facilitate a manual or automatic conversion of C++ code to Rust.
Spatial memory safety
Lifetime verification does not establish spatial memory safety, that is, it does not prove that all accesses are in bounds. Rust collections perform these checks at runtime.
Notes
-
The following documents provide more background material: doc 1, doc 2. ↩
-
Note, however, that Clang is currently very conservative in assigning types to type-dependent expressions. ↩
Struct Layout
C++ (in the Itanium ABI) extends the C layout rules, and so repr(C) isn’t
enough. This pages documents the tweaks to Rust structs to give them the same
layout as C++ structs.
In particular:
- C++ classes and Rust structs must have the same alignment, so that
references can be exchanged without violating the alignment rules. This is
usually ensured by the regular
#[repr(C)]layout algorithm, but sometimes the interop tool needs to generate explicit#[repr(align(n))]annotations. - C++ classes and Rust structs must have the same size, so that arrays of objects can be exchanged.
- Public subobjects must have the same offsets in C++ and Rust versions of the structs.
Non-field data
Rust bindings introduce a __non_field_data: [MaybeUninit<u8>; N] field to
cover data within the object that is not part of individual fields. This
includes:
- Base classes.
- VTable pointers.
- Empty struct padding.
Empty Structs
One notable special case of this is the empty struct padding. An empty struct or
class (e.g. struct Empty{};) has size 1, while in Rust, it has size 0. To
make the layout match up, bindings for empty structs will always enforce that
the struct has size of at least 1, via __non_field_data.
(In C++, different array elements are guaranteed to have different addresses,
and also, arrays are guaranteed to be contiguous. Therefore, no object in C++
can have size 0. Rust, like C++, has only contiguous arrays, but unlike C++
Rust does not guarantee that distinct elements have distinct addresses.)
Potentially-overlapping objects
In C++, in some circumstances, the requirement that objects do not overlap is
relaxed: base classes and [[no_unique_address]] member variables can have
subsequent objects live inside of their tail padding. The most famous instance
of this is the empty base class optimization
(EBCO): a base class with no
data members is permitted to take up zero space inside of derived classes.
NOTE: This has other, non-layout consequences for Rust: for example, it is not
safe to obtain two &mut references to overlapping objects, unless they are of
size 0. (To prevent this, classes that might be base classes are always
!Unpin.)
This is impossible to represent in a C-like struct. (Indeed, it’s impossible to
represent even in a C++-like struct, before the introduction of
[[no_unique_address]]). Therefore, in Rust, we don’t even try:
potentially-overlapping subobjects are replaced in the Rust layout by a
[MaybeUninit<u8>; N] field, where N is large enough to ensure that the next
subobject starts at the correct offset. The alignment of the struct is still
changed so that it matches the C++ alignment, but via #[repr(align(n))]
instead of by aligning the field.
Example
For example, consider these two C++ classes:
// This is a class, instead of a struct, to ensure that it is not POD for the
// purpose of layout. (The Itanium ABI disables the overlapping subobject
// optimization for POD types.)
class A {
int16_t x_;
int8_t y_;
};
struct B final : A {
int8_t z;
};
In memory, this may be laid out as so:
| x_ | x_ | y_ | z |
<------------> <->
A subobject | B
<------------------>
sizeof(A)
(also sizeof(B))
The correct representation for B, in Rust, is something like this:
#[repr(C)]
#[repr(align(2))] // match the alignment of the int16_t variable.
struct B {
// We don't use a field of type `A`, because it would have a size of 4,
// and Rust wouldn't permit `z` to live inside of it.
// Nor do we align the array, for the same reason -- correct alignment must be
// achieved via the repr(align(2)) at the top.
__non_field_data: [MaybeUninit<u8>; 3],
pub z: i8,
}
Thunks for class template member functions
Problem definition
Given the C++ header below…
#pragma clang lifetime_elision
template <typename T>
class MyTemplate {
public:
MyTemplate(T value) : value_(value) {}
const T& GetValue() const;
private:
T value_;
};
using MyIntTemplate = MyTemplate<int>;
… Crubit will generate Rust bindings that can call into the
MyTemplate<int>::GetValue() member function. To support such calls, Crubit has
to generate a C++ thunk (to instantiate the class template and to provide a
symbol for a C-ABI-compatible function that Rust can call into):
extern "C" // <- C ABI
int const& __rust_thunk___ZNK10MyTemplateIiE8GetValueEv(
const class MyTemplate<int>* __this) {
return __this->GetValue();
}
There are other (non-template-related) scenarios that require generating
thunks (e.g. inline functions, or functions that use a custom calling
convention), but templates bring one extra requirement: a class template can be
defined in one header (say my_template.h) and used in multiple other headers
(e.g. library_foo/template_user1.h and library_bar/template_user2.h).
Because of this, the same thunk might need to be present in multiple generated
..._rs_api_impl.cc files (e.g. in library_foo_rs_api_impl.cc and
library_bar_rs_api_impl.cc). This may lead to duplicate symbol errors from the
linker:
ld: error: duplicate symbol: __rust_thunk___ZNK10MyTemplateIiE8GetValueEv
Implemented solution: Encoding target name in the thunk name
One solution is to give each of the generated thunks a unique,
target/library-specific name, e.g.:
__rust_thunk___ZNK10MyTemplateIiE8GetValueEv__library_foo (note the
library_foo suffix).
Pros:
- Minimal extra code complexity (e.g. no need for templates-specific code
in thunk-related code in
src_code_gen.rs). - Obviously correct behavior-wise (e.g. since it is just like other thunks which we assume are implemented correctly).
Cons:
-
Performance guarantees are unclear. Binary size depends on link time optimization (LTO) recognizing that all the thunks are identical and deduplicating them.
- This seems to work in practice (at least for production binaries).
- Future work: add tests + consider asking LLVM to provide LTO guarantees
-
Requires escaping Bazel target names into valid C identifiers. See
ConvertToCcIdentifier(const BazelLabel&)inbazel_types.cc.
Alternative solutions
Function template
An alternative solution would be to use a function template that we immediately explicitly instantiate. These still generate the code we need, but their duplicated symbol definitions (across multiple binding crates) won’t cause an ODR violation. It is expected that a single function template is instantiated multiple times in multiple translation units, therefore the linker silently merges these equivalent definitions.
Example:
// Thunk is expressed as a function template:
template <typename = void>
__attribute__((__always_inline__)) int const&
__rust_thunk___ZNK10MyTemplateIiE8GetValueEv(
const class MyTemplate<int>* __this) {
return __this->GetValue();
}
// Explicit instantiation of the function template:
// (to generate a symbol that `..._rs_api.rs` can call into)
template int const& __rust_thunk___ZNK10MyTemplateIiE8GetValueEv(
const class MyTemplate<int>* __this);
Pros:
- Naturally deduplicated (just depending on what C++ already does for function templates).
Cons:
- Assumes a particular ABI - a function template specialization uses the
calling convention prescribed by the platform C++ ABI. We know that the
Itanium ABI maps C++ signatures to the C
ABI and
therefore will be compatible with the calling convention expected by the
generated
..._rs_api.rs. Further research is needed to investigate the guarantees offered by other platforms (e.g., the MSVC ABI). - Requires extra complexity to calculate the mangled name of the function
template specialization.
- Crubit doesn’t have a
clang::FunctionDeclcorresponding to the function-template-based thunk, and therefore Crubit can’t useclang::MangleContext::mangleNameto calculate the linkable/mangled name of the thunk. - Reimplementing
clang::MangleContext::mangleNamein Crubit seems fragile. One risk is bugs in Crubit’s code that would make it behave differently from Clang (e.g. code review of the initial prototype identified that mangling compression was missing). Another risk is having to implement not justItaniumMangleContext, but alsoMicrosoftMangleContext. - One idea to avoid reimpliementing mangling is to explicitly specify the
name for the function template instantiation using
__asm__("abc")(sadly this doesn’t seem to work - it may be a Clang bug).
- Crubit doesn’t have a
An abandoned prototype of this approach can be found in a (Google-internal) cl/450495903.
Explicit linkonce_odr attribute
Example:
extern "C"
int const& __rust_thunk___ZNK10MyTemplateIiE8GetValueEv(
const class MyTemplate<int>* __this)
__attribute__((linkonce_odr)) // <- THIS IS THE PROPOSAL
{
return __this->GetValue();
}
Pros:
- All the “pros” of the “Encoding target name in the thunk name” approach (simplicity + correctness of behavior)
- All the “pros” of the “Function template” template approach (deduplication)
Cons:
- Requires changing Clang to support the new attribute (e.g. requires convincing the Clang community that this is a language extension that is worth supporting). TODO(b/234889162): Send out a short RFC to gauge interest?
Rejected solutions
-
selectanydoesn’t work with functions, only data members. Furthermore, we need something that maps tolinkonce_odr, and selectany maps only tolinkonce. -
__attribute__((weak))has the disadvantage that a weak definition can be overridden by a strong one. This rule makes weak definitions non-inlineable except in full-program LTO. C++ function template instead follows the ODR rule that says that all definitions must be equivalent, making them inlineable.
Unpin for C++ Types
SUMMARY: A C++ type is Unpin if it is Rust-movable (e.g., a trivial type, or a
nontrivial type which is [[clang::trivial_abi]]). Any such type can be used by
value or plain reference/pointer in interop, all non-Unpin types must instead
be used behind pinned pointers and references.
digraph {
rankdir=LR;
node [shape=box, style="rounded,filled", fillcolor="#f8f9fa", fontname="Arial", fontsize=11];
edge [fontname="Arial", fontsize=9, fontcolor="#555555"];
subgraph cluster_rust {
label="Rust Representation"; style=dashed; color="#aaaaaa"; fontname="Arial";
r_triv [label="Unpin,\n!needs_drop()"];
r_move [label="Unpin,\nneeds_drop()"];
r_pin [label="!Unpin"];
}
subgraph cluster_cpp {
label="C++ Representation"; style=dashed; color="#aaaaaa"; fontname="Arial";
c_triv [label="trivial"];
c_move [label="[[clang::trivial_abi]],\nnontrivial"];
c_pin [label="not [[clang::trivial_abi]],\nnontrivial"];
}
r_triv -> c_triv;
r_triv -> c_triv [dir=back];
r_move -> c_move;
r_move -> c_move [dir=back];
r_pin -> c_pin;
r_pin -> c_pin [dir=back, label="pinned type-state"];
}
A C++ type T is Unpin if it is known to be a Rust-movable type
(move+destroy is logically equivalent to memcpy+release).
Unpin C++ types can be used like any other normal Rust type: they are always
safe to access by reference or by value. Non-Unpin C++ types, in contrast, can
only be accessed behind pins such as Pin<&mut T>, or Pin<Box<T>>, because it
may not be safe to directly mutate. These types are never used directly by value
in Rust, because value-like assignment has incorrect semantics: it fails to run
the overloaded C++ assignment operator or constructor.
Note that not every object with an Unpin type is actually safe to hold in a
mutable reference. Objects with live aliases still must not be used with &mut,
and “potentially overlapping objects” can produce unexpected behavior in Rust.
(See Reference Safety.)
Rust-movable types
In C++, moving a value between locations in memory involves executing code to either initialize (move-construct) or overwrite (move-assign) the new location. The old location still exists, but is in a moved-from state, and must still be destroyed to release resources.
(For example, std::string x = std::move(y); will run the move constructor, so
that x contains the same value that y used to have before the move. The
variable y will still be a valid string, but might be empty, or might contain
some garbage value. The destructors for both x and y will run when they go
out of scope.)
Rust does not have move constructors or move assignment. In fact, there is no
way to customize what happens during moving or assignment: in Rust, moving or
swapping an object means changing its location in memory, as if by memcpy
without running the destructor logic in the old location. Another way of looking
at it is that it’s as if an object moved around in memory over time: it is
constructed in one place, and then further operations and eventual destruction
might happen in other places. This is a Rust move.
Despite C++ moves using explicit construction and destruction calls, many C++ types could also have used the Rust movement model. We call such types Rust-movable types.
For example, a C++ std::unique_ptr, implemented in the obvious way, is
Rust-movable: its actual location in memory does not matter. In contrast, a
self-referential type is not Rust-movable, because to move it, you must also
update the pointer it has to itself. This is done inside the move constructor in
C++, but cannot be done in the Rust model, where the move operation is not
customizable.
Which types are Rust-movable?
For the purpose of Rust/C++ interop, we define a type to be Rust-movable if, and only if, it is “trivial for calls” in Clang. That is, either:
- It is actually trivial, or
- It uses
[[clang::trivial_abi]]to make itself trivial for calls
This definition is conservative: some types that could be considered
Rust-movable are not trivial for calls. (For example, std::unique_ptr uses
[[clang::trivial_abi]] only in the unstable libc++ ABI; the stable libc++ ABI
predates this attribute, and adding it now is ABI-breaking.)
This definition is, however, sound: all types which are trivial for calls are Rust-movable, because a type which is trivial for calls is Rust-moved when passed by value as a function argument.
Reference Safety
Not every object with an Unpin type can actually safely be pointed to by a
Rust reference.
Conventional aliasing
If a C++ reference mutably aliases, it is unsafe to pass to Rust as a Rust reference. Do not under any circumstance create mutably aliasing Rust references, the behavior of doing so is undefined.
For example:
#![allow(unused)]
fn main() {
pub fn foo(_: &mut i32, _: &mut i32) {}
}
It is Undefined Behavior to, in C++, call foo(x, x).
Tail padding
In C++, tail padding is not part of the object, and the space in the tail
padding can be taken up by other unrelated objects. Avoid creating a mutable
Rust reference to a base class, or to a [[no_unique_address]] field, as these
are “potentially overlapping”. This can cause surprising behavior, or unintended
aliasing and undefined behavior.
Consider the following struct:
struct A {};
struct B {
[[no_unique_address]] A field_1_;
char field_2_;
A& field_1() { return field_1_; }
char& field_2() { return field_2_; }
};
Here, while sizeof(A) is 1, it has no data, only tail padding. A C++
assignment to field_1_ will not write anything. And so C++ can store an
unrelated object inside of the tail padding. [[no_unique_address]] marks the
tail padding as available for use. field_2_ may actually be stored inside the
tail padding of field_1_, and the sizeof(B) may also be 1.
(Base classes also allow their tail padding to be reused, and the same example
works with struct B : A.)
static_assert(sizeof(A) == sizeof(B));
static_assert(offsetof(B, field_1) == offsetof(B, field_2));
Rust does not work this way. In Rust, tail padding is part of the object. Rust
references refer to the full span of the pointed-to object, including that tail
padding. And so a Rust reference to field_1_ would encompass field_2_ by
accident.
This means that the following code has undefined behavior via conventional aliasing, despite looking fairly innocent:
B b = ...;
// Rust: pub fn foo(_: &mut A, _: &mut u8)
foo(b.field_1, b.field_2); // C++
And the following Rust code would perform unintended mutations to field_2:
#![allow(unused)]
fn main() {
let mut b1: B = ...;
let mut b2: B = ...;
// This also swaps field_2!
std::mem::swap(&mut *b1.field_1(), &mut *b2.field_1());
}
C++20
In C++17 and earlier, there was only one way to create a potentially-overlapping
object: inheritance (“EBO”).
Making inheritable types non-Unpin could have removed or mitigated the risk of
overlapping objects in C++17 and below.
However, as of C++20, any object can alias another in the tail padding.
C++20 introduced [[no_unique_address]], which makes tail padding available for
reuse for any type. Since [[no_unique_address]] may be used fairly extensively
in library code (it has no negative effects in C++), we can’t assume that it
does not exist.
In modern C++, final types are not much safer than other types. One must be
careful when creating Rust references, to ensure that those Rust references
do not alias, including via the tail padding. There is no way to guarantee this
at the type level.
Problem Description: forward-declared types
C++ allows for the same type to be defined in multiple places, where each of these declarations refers to the same type. Types in C++ are not “owned” by any particular target, and anyone can redefine it as a forward declaration / opaque enum, so long as their definition is compatible with the complete definition.
// h1.h
class Foo; // a forward declaration
enum Bar: int; // an opaque enum declaration
// h2.h
class Foo; // the same type as Foo above
enum Bar: int; // the same type as Bar above
// h3.h
class Foo {}; // the same type, but if this is included, it now has known size/methods
enum Bar: int {}; // the same type, but if this is included, enumerators are known
The same is also true, implicitly, of template specializations: the same class template can be instantiated from multiple places, and each instantiation refers to the same type, even if they are totally independent.
This is basically incompatible with Rust, where types have one canonical owner:
two different targets cannot define the same type, and have it be the same in
type, except for the special cases of tuple types (which are structurally
typed), generic instantiations, and dyn Trait. Thankfully, this does mean we
have tools at our disposal to emulate what C++ can do.
Design axioms for a solution
Example headers:
// incomplete1.h
class Foo;
Foo* CreateIncomplete1();
void ConsumeIncomplete1(Foo*);
// incomplete2.h
class Foo;
Foo* CreateIncomplete2();
void ConsumeIncomplete2(Foo*);
// complete.h
class Foo {};
Foo* CreateComplete();
void ConsumeComplete(Foo*);
1. Dependency Graph
The libraries must be usable standalone. For each of these headers, it must be
possible to use the header from Rust without modifying any of them to become
aware of the others. In C++, it would be valid for complete to depend on
incomplete1, and for incomplete2 to depend on complete, so we can’t
require modifications of the dependency graph without grappling with dependency
cycles and a global analysis of the codebase.
2. Fidelity
The functions must be usable. Users must be able to call every combination of functions from these three libraries, though perhaps not with the same syntax as C++. For example:
| Operation | Example code | |
|---|---|---|
| A | Incomplete → Incomplete (same library) | ConsumeIncomplete1(CreateIncomplete1()) |
| B | Incomplete → Incomplete | ConsumeIncomplete1(CreateIncomplete2()) |
| C | Incomplete → Complete | ConsumeComplete(CreateIncomplete1()) |
| D | Complete → Incomplete | ConsumeIncomplete1(CreateComplete()) |
| E | Complete → Complete | ConsumeComplete(CreateComplete()) |
3. Code Maintenance
To the greatest extent possible, backwards compatible changes in C++ should also be backwards-compatible in Rust. This means that changing from a forward declaration to a complete type (or dependency on the complete type) should not be a breaking change to Rust callers. It is fine for the reverse to break callers: it is expected that switching to forward declarations might break callers.
In particular, that means that the exact same source code should continue to compile and behave the same way if we perform any of the following changes:
- A→E (Incomplete→Incomplete, but change it to Complete)
- B→C (Incomplete→Incomplete, change recipient to Complete)
- B→D (Incomplete→Incomplete, change source to Complete)
- B→E (Incomplete→Incomplete, change source and destination to Complete)
- C→E (Incomplete→Complete, change source to Complete)
- D→E (Complete→Incomplete, change destination to Complete)
4. Type Aliases
This is a bit of an aside, but it is tricky to get right. In general, when analyzing the source code for a C++ header, you cannot actually know if a type was complete or incomplete from the perspective of a foreign library. In particular, given the following header:
// alias.h
#include "..."
using FooAlias = Foo;
// complete.h
#include "alias.h"
struct Foo {};
void Foo(FooAlias);
If you are analyzing alias.h from a transitive include graph that includes a
complete definition for Foo, it is difficult to work out whether FooAlias
will always be complete in any include graph or not. Is FooAlias complete
because alias.h transitively included complete.h, or is Foo complete
because something else happened to both include alias.h and also completed the
type?
The naive approach, analyzing alias.h purely on the basis of what we know in
the caller, can produce a compilation error: from the POV of complete.h,
looking at the C++ AST, FooAlias is the same type as Foo, and so reusing the
type alias in the generated Rust function looks fine: pub fn Foo(_: alias::FooAlias). Unfortunately, if alias.h does not have a complete type
definition when viewed alone, this results in compilation errors in the
generated code.
This matters for type aliases – and only type aliases – because types are the
only thing that can affect public APIs in Crubit (if we “mispredict” a type, we
get compilation errors), and because type aliases are the only place where this
confusion surfaces as a top-level type. Whether a type is forward-declared or
not never changes the type properties of a struct, because if it could be
either, then it will only ever be held behind e.g. a pointer or similar compound
data type which erases any difference from the top level POV. A struct will be
trivial or not independent of whether the T in its T* field is complete or
incomplete. We can still be confused and “mispredict” the types of function
parameters or fields in other libraries, but because Crubit does not generate
calls or field accesses to entities defined in foreign libraries, this is never
an issue.
Any mapping of forward declared types (and other redeclarable types) to Rust should evade this problem somehow! Some options:
- Track the include chain so that there is absolute certainty about whether a type is complete because it includes the completed definition, instead of simply asking Clang whether it’s complete or not.
- (What we do today) do not use type aliases in other libraries, at least if the type alias refers to a complete type that could hypothetically be incomplete.
- (Impossible?) make it not a compilation error to guess wrong, somehow.
- Don’t use the C++ AST to determine if aliases in a dependency are to a
complete type.
- Use an IDL approach instead of Crubit’s “automatic” approach.
- Store the answer to the question and make it available to people that depend on you. (Linearize the build.)
Non-solution: extern types
Rust has an (unstable) “extern types” feature, usable as so:
#![allow(unused)]
fn main() {
unsafe extern "C" {
type Foo;
}
}
This defines a PointeeSized type Foo – analogous to C++’s “incomplete
type”, a PointeeSized type has no known size/alignment and cannot be held by
value, only behind a pointer.
This does not, by itself, present a solution to the design problem above. Two
different crates which define an extern type Foo are not, from Rust’s point of
view, defining the same type Foo, and there is no way to define a non-extern
type Foo which is the same (“completed”) type. This means that, absent
additional work, design axiom 2 (and/or 1) are not satisfied. However, we can
build on it to define an auxiliary wrapper type or set of conversions.
(Extern types also do not help with opaque enums, or the related problem of “redeclared” template instantiations.)
Existing Solution: forward_declare.rs
Very briefly, forward_declare currently only supports forward-declared struct
types (not opaque enum definitions), which is present as a simple wrapper around
an extern type with conversions and reasonably safe type properties (e.g.
!Send, !Sync, !Unpin).
The core idea is to define equivalence classes of “types which are different in
Rust, but the same type in C++”, via the CppCast trait. If two types are the
same type in C++, but forced to be different types in Rust, then you should be
able to call .cpp_cast() to perform the conversion. For example, the
equivalent of C++ ConsumeIncomplete1(CreateComplete()) is
ConsumeIncomplete1(CreateComplete().cpp_cast()).
For the sake of code maintenance concerns, above, always use a unique per-crate
forward declared type. If you hold an x: &Foo (incomplete), and you call into
a library which accepts a &Foo (incomplete), then even though these are “the
same” – they’re both forward declarations to the same class – we force the use
of .cpp_cast(). This ensures that if either you or the library you are calling
ever switch to including the complete type definition, code will continue to
compile unchanged. In other words, we force a call to .cpp_cast() just in case
it ever becomes necessary later.
This means that the table of calls above, in 2. Fidelity, is as follows:
| Operation | Example code | |
|---|---|---|
| A | Incomplete → Incomplete (same library) | ConsumeIncomplete1(CreateIncomplete1()) |
| B | Incomplete → Incomplete | ConsumeIncomplete1(CreateIncomplete2().cpp_cast()) |
| C | Incomplete → Complete | ConsumeComplete(CreateIncomplete1().cpp_cast()) |
| D | Complete → Incomplete | ConsumeIncomplete1(CreateComplete().cpp_cast()) |
| E | Complete → Complete | ConsumeComplete(CreateComplete()) |
In theory we could have dropped the requirement for .cpp_cast() from case B,
but this would make it a backwards-incompatible change to do any of B→C or
B→D, which was called out as a requirement above in 3. Code
Maintenance
Mutable Details
The following details to how forward_declare works are not core to how it
satisfies the design axioms.
How are types determined to be the same?
At its core, forward_declare has a blanket impl that goes something like this:
#![allow(unused)]
fn main() {
impl<T, U> CppCast<U> for T where same_type!(T, U) {...}
}
How do we create this “same type” relationship when neither T nor U can directly mention each other? The answer is to lean on something I mentioned way at the start: Rust does have a couple of ways of defining the same type from multiple targets.
In particular, today, forward_declare does the following:
#![allow(unused)]
fn main() {
/// # Safety
/// if `X : CppType<Name=Y>`, then `X` has the same layout and set of valid
/// representations as everything with name `Y`
pub unsafe trait CppType {
type Name;
}
impl<T, U> CppCast<U> for T where T: CppCast, U: CppCast<Name=T::Name> {...}
}
With this, we can lean on any way of declaring two types to be the same. In stable Rust, you can do it like so:
#![allow(unused)]
fn main() {
pub struct C<C: const char>;
unsafe impl CppType for Foo {
type Name = (C<'F'>, C<'o'>, C<'o'>);
}
}
But this produces truly unreadable error messages, and we can do the same trick a bit better in unstable Rust:
#![allow(unused)]
#![feature(adt_const_params, unsized_const_params)]
fn main() {
pub struct Symbol<S: const &'static str>;
unsafe impl CppType for Foo {
type Name = Symbol<"Foo">;
}
}
(I mentioned above that traits also are structural – but please do not do
trait C<I: usize, C: const char> {} type Name = dyn C<0, 'F'> + C<1, 'o'> + C<2, 'o'>;)
Alternative: constant unification
Instead of using type unification, and relying on unique constants to distinguish types, unstable Rust does allow actually directly using constant unification in trait bounds:
#![allow(unused)]
#![feature(generic_const_exprs, min_generic_const_args, unsized_const_params, adt_const_params)]
fn main() {
pub trait CppType {
type const Name: &'static str;
}
impl<T, U> CppCast<U> for T where T: CppCast, U: CppCast<Name={T::Name}> {...}
}
The main downside to this is that it does not have a clean “fallback” if the
unstable features are revoked or changed substantially. With the type based
approach, users can directly write something type Name = symbol!("abc") and we
can hide whether symbol!() expands to a tuple or a string container type. The
direct use of constants has the risk of leaking into more places and being
harder to unwind.
How are forward declarations made to be unique?
This is the part of forward_declare I think is the most likely to be outright
wrong!
Currently, forward declare defines a type as follows:
#![allow(unused)]
fn main() {
pub struct Incomplete<Name, Crate>(...);
unsafe impl<Name, Crate> CppType for Incomplete<Name, Crate> {
type Name = Name;
}
}
Users that wish to use Incomplete create a local alias, which defines a
crate-local Crate type. The equivalent of C++ struct Foo; is Rust
forward_declare!(pub Foo = symbol!("Foo"));, which expands to:
#![allow(unused)]
fn main() {
pub struct _forward_declare_Foo;
pub type Foo = Incomplete<symbol!("Foo"), _forward_declare_Foo>;
}
This means that every user of forward declarations is using the same
Incomplete type, but with different type parameters to make them distinct
types. This does have the following advantages compared to the obvious
alternative of explicitly defining a new type:
- Users do not need to unsafely implement a trait to consume incomplete types
in pure Rust code. They can simply use
Incomplete<...>, even via a helper macro to declare it, and there is no unsafe code anywhere here. (TheCppTypetrait itself must necessarily be unsafe.) - Theoretically, only requires defining one extra type per crate, instead of N (where N is the number of forward declarations).
- Extends easily to “comprehensive fallbacks”: if a function is private, it
can use
Incomplete<Name, ()>. For understandable architectural reasons, it is simpler to use a type from deep within our bindings generation logic, than to compel Crubit to define a new type, and so “comprehensive fallbacks” – where we substitute in a forward declaration any time we see an unsupported type – is much easier to support if we can reuse an existing type.
As well as the following downsides:
- Users can be generic over
Incomplete<Name, _>when they should instead be generic overT: CppType<Name=Name>(which also includes the complete type). - Because it’s a foreign type, forward declarations can’t add extra trait impls or inherent impls.
- It is kind of strange. If you want a new type, shouldn’t you just… define that type?
Alternative: explicit definition
Instead, forward declarations could expand to:
#![allow(unused)]
fn main() {
pub struct Foo;
unsafe impl CppType for Foo {type Name = symbol!("Foo");}
}
Missing Features
Enums
Unlike class types, opaque “forward-declared” enums have a known size. If
forward_declare kept its current direction, one would likely imagine that they
are implemented as something like:
#![allow(unused)]
fn main() {
struct OpaqueEnum<Name, Underlying>(pub Underlying);
// Add Underlying to the name so that these are actually safe to use --
// if we just used `Name`, then totally safe code would get UB when it did
// something stupid like mix OpaqueEnum<symbol!("Bar", i32) with
// OpaqueEnum<symbol!("Bar"), i8>
unsafe impl CppType<Name, Underlying> for Foo {type Name = (Name, Underlying);}
}
And the equivalent of enum Bar: int32_t; would be:
#![allow(unused)]
fn main() {
pub type Bar = forward_declare::OpaqueEnum<symbol!("Bar"), i32>;
}
Conversions
There are a lot of conversions that need to exist for the CppType
equivalence class setup, and only a small fraction are implemented. For example,
we currently implement conversions for Vec, but not std::vector.
Compatibility for Pin
The compatibility story above is imperfect when the complete type is non-Rust movable, and the type is behind a mutable reference.
When the complete type is Rust-movable, then moving referents from incomplete to
complete means moving from (something like) Pin<&mut T> to &mut T. This is
not in general backwards-compatible.
In particular, this breaks calls of the form <...>.as_mut().cpp_cast(): if
<...> was incomplete (and therefore pinned), but becomes complete (and
therefore Rust-movable and behind only a &mut T), then the calls stop working,
as &mut T doesn’t have a .as_mut() method.
Changing the destination to be complete can be made to work: if .cpp_cast()
also can wrap in a Pin, then cpp_cast-conversion of &mut T to Pin<&mut T> is the same syntax as cpp_cast-conversion of &mut T to itself.
Full matrix, in the presence of a reborrow:
| Operation | Example code | |
|---|---|---|
| A | Incomplete → Incomplete (same library) | ConsumeIncomplete1(CreateIncomplete1().as_mut()) |
| B | Incomplete → Incomplete | ConsumeIncomplete1(CreateIncomplete2().as_mut().cpp_cast()) |
| C | Incomplete → Complete | ConsumeComplete(CreateIncomplete1().as_mut().cpp_cast()) |
| D | Complete → Incomplete | ConsumeIncomplete1(CreateComplete().cpp_cast()) |
| E | Complete → Complete | ConsumeComplete(CreateComplete()) |
Moving from A/B/C to D/E is compatibility breaking if a call to .as_mut() was
present.
Solutions
-
Pin ergonomics might make reborrowing a pinned reference implicit. We will still suffer from all the other reasons that
Pin<&mut T>is not 1:1 compatible with&mut T, but these can’t be made perfectly compatible. -
The actual cause is not forward declarations per se, but the fact that Crubit conditionally uses
Pin, which is a compatibility hazard. If we always usedPin<&mut T>, or always used a type likeCMut<'_, T>, which implicitly pins when possible, then the compatibility concerns for forward declarations vanish.
[Pre-RFC] Allow stride != size
Summary
Rust should allow for values to be placed at the next aligned position after the previous value, ignoring the tail padding of that previous field. This requires changing the meaning of “size”, so that a value’s size in memory (for the purpose of reference semantics and layout) is not definitionally the same as the distance between consecutive values of that type (its “stride”).
Motivation
Some other languages (C++ and Swift, in particular) can lay out values more compactly than Rust conventionally can, leading to better performance at greater convenience, and less than ideal Rust interoperability.
Optimization opportunity
Consider the difference between (u16, u8, u8) and ((u16, u8), u8). The first
can fit in 4 bytes, while the second requires 6 bytes. A (u16, u8) is a 4 byte
value with 1 byte of tail padding. And a (T, u8) can’t just stuff the u8
inside the tail padding for T! If, instead, we declared that (u16, u8) were
a 3 byte value with alignment 2, then ((u16, u8), u8) could be 4 bytes
instead of 6 bytes. This is not possible today.
(For backwards compatibility reasons described later, we can’t literally do this for tuples, but only for user-defined types. But this gives the gist of the optimization opportunity this proposal supports.)
By inventing the concept of a “data size”, which doesn’t need to be a multiple
of the alignment, we can allow fields in specially-designed types to be packed
closer together than they would be today, saving space. This is similar to the
performance benefits of #[repr(packed)], but safer: all values would still be
correctly aligned, just placed more closely together.
This optimization has already been implemented in other programming languages.
Swift applies this to every type and every field: a type’s size excludes tail
padding, and a neighboring value can be laid out immediately next to it when
stored in the same type, with no padding between the two. In C++, the
optimization automatically applies to base classes
(“EBO”, the Empty Base
Optimization), and is opt-in on fields via the
[[no_unique_address]]
attribute.
For example, here’s an example in Swift and in C++. These types are compact! Rust does not work like this today.
Interoperability with C++ and Swift
(Note that the author works on C++ interop, Swift is mentioned for completeness.)
In fact, exactly because this optimization is already implemented in other
languages, those languages are theoretically not as compatible with Rust as they
are with each other. In C++ and Swift, writing to a pointer or reference does
not write to neighboring fields. But if that pointer or reference were passed to
Rust, and you used any Rust facility to write to it – whether it were vanilla
assignment or ptr::write – Rust could overwrite that neighboring field.
Because the use of this optimization is pervasive in both Swift and C++,
interoperating with these languages is difficult to do safely.
Concretely, consider the following C++ struct:
struct MyStruct {
[[no_unique_address]] T1 x;
[[no_unique_address]] T2 y;
...
};
Which is equivalent to this Swift struct:
struct MyStruct {
let x: T1
let y: T2
...
}
If you are working with cross-language interop, and obtain in Rust a &mut T1
which refers to x, and a &mut T2 which refers to y, it may be immediately
UB, because these references can overlap in Rust: y may be located inside what
Rust would consider the tail padding of the T1 reference.
For the same reason, even if you avoid aliasing, if you obtain a &mut T1 for
x, and then write to it, it may partially overwrite y with garbage data,
causing unexpected or undefined behavior down the line.
This also cannot be avoided by forbidding the use of MyStruct: even if you do
not directly use it from Rust, from the point of view of Swift and C++, it is
just a normal struct, and Swift and C++ codebases can freely pass around
references and pointers to its interior. Someone passing a reference to a T1
may have no idea whether it came from MyStruct (unsafe to pass to Rust) or an
array (safe). You would need to ban (or correctly handle) any C++ and Swift type
which can have tail padding, in case that padding contains another object.
(To add insult to injury, the struct MyStruct itself – not just references to
fields inside it – cannot be represented directly as so in Rust, either.)
And anyway, such structs are unavoidable. In Swift, this is the default
behavior, and pervasive. In C++, [[no_unique_address]] is permitted to be used
pervasively in the standard library, and it is impractical to only interoperate
with C++ codebases that avoid the standard library.
In order for C++ and Swift pointers/references to be safely representable in
Rust as mut references, a &mut T1 would need to exclude the tail padding,
which means that Rust would need to separate out the concept of a type’s
interior size from its array stride. And in order to represent MyStruct in
Rust, we would need a way to use the same layout rules that are available in
these other languages.
Explanation
(I haven’t separated this out to guide-level vs reference-level – this is a pre-RFC! Also, all names TBD.)
As a quick summary, the proposal is to introduce the following new traits, functions, and attributes, and behaviors:
std::mem::data_size_of<T>(), returning the size but not necessarily rounded to alignment / not necessarily the same as stride.- In the memory model, pointers and references only refer to
data_size_of::<T>()bytes. AlignSized, a trait for types where the data size and stride are the same.#[repr(compact)], to mark a type as not implementingAlignSized, and thus having a potentially smaller data size.#[compact], to mark a field as laid out using the data size instead of the stride.
Data size vs stride
Semantically, Rust types would gain a new kind of size: “data size”. This is the size of the type, minus the tail padding. In fact, it’s in some sense the “true” size of the type: array stride is the data size rounded up to alignment.
Data size would be exposed via a new function std::mem::data_size_of::<T>();
array stride continues to be returned by std::mem::size_of::<T>().
The semantics of a write (e.g. via ptr::write, mem::swap, or assignment) are
to only write “data size” number of bytes, and a &T or &mut T would only
refer to “data size” number of bytes for the purpose of provenance and aliasing
semantics. (&[T; 1], in contrast, continues to refer to size_of::<T>()
bytes.)
The AlignSized trait and std::array::from_ref
It is fundamentally a backwards-incompatible change to make stride and size not
the same thing, because of functions like
std::array::from_ref
and
std::slice::from_ref.
The existence of these functions means that Rust guarantees that for an
arbitrary generic type today, that type has identical size and stride.
This means that if we want to allow for data size and stride to be different,
they must not be different for any generic type as written today. Existing code
without trait bounds can call from_ref! So we must add an implicit trait
bound on AlignSized : Sized, which, like Sized, guarantees that the data
size and the stride are the same. This trait would be automatically implemented
for all pre-existing types, which retain their current layout rules.
In other words, the following two generics are equivalent:
fn foo<T>() {}
fn foo<T: Sized + AlignSized>() {}
… and to opt out of requiring AlignSized, one must explicitly remove a trait
bound:
fn foo2<T: ?AlignSized>() {}
// AlignSized requires Sized, and so this will also do it:
fn foo3<T: ?Sized>() {}
To opt out of implementing this trait, and to opt in to being placed closer to
neighboring types inside a compound data structure, types can mark themselves as
#[repr(compact)]. This causes the data size not to be rounded up to alignment:
#[repr(C, compact)]
struct MyCompactType(u16, u8);
// data_size_of::<MyCompactType>() == 3
// size_of::<MyCompactType>() == 4
Taking advantage of non-AlignSized types with #[compact] storage
If a field is marked #[compact], then the next field is placed after the data
size of that field, not after the stride. (These can only differ for a
non-AlignSized type.) This provides easy control, and provides compatibility
with C++, where this behavior can be configured per-field.
It is an error to apply this attribute on non-#[repr(C)] types.
#[repr(C, compact)]
struct MyCompactType(u16, u8);
#[repr(C)]
struct S {
#[compact]
a: MyCompactType, // occupies the first 3 bytes
b: u8, // occupies the 4th byte
}
// data_size_of::<S>() == size_of::<S>() == 4
Example
Putting everything together:
#[repr(C, compact)]
struct MyCompactType(u16, u8);
// data_size_of::<MyCompactType>() == 3
// size_of::<MyCompactType>() == 4
#[repr(C)]
struct S {
#[compact]
a: MyCompactType, // occupies the first 3 bytes
b: u8, // occupies the 4th byte
}
// data_size_of::<S>() == size_of::<S>() == 4
We can take mut references to both fields a and b, and writes to those
references will not overlap:
let mut x : S = ...;
let S {a, b} = &mut x;
*a = MyCompactType(4, 2); // writes 3 bytes
*b = 0; // writes 1 byte
If we had not applied the repr(compact) attribute, or had not applied the
#[compact] attribute, then data_size_of<S>() would have been 6, and so would
size_of<S>(). The assignment *a = ... would have (potentially) written 4
bytes.
Drawbacks
Backwards compatibility and the AlignSized trait
In order to be backwards compatible, this change requires a new implicit trait
bound, applied everywhere. However, that makes this change substantially less
useful. If that became the way things worked forever, then #[repr(compact)]
types would be very difficult to use, as almost no generic functions would
accept them. Very few functions actually need AlignSized, but every generic
function would get it implicitly.
We could change this at an edition boundary: a later edition could drop the
implicit AlignSized bound on all generics, and automated migration tooling
could remove the implicit bound from any generic function which doesn’t use the
bound, and add an explicit bound for everything that does. After enough
iterations, the only code with a bound on AlignSized would be code which
transmutes between T and [T]/[T; 1]. Though this would be a disruptive and
long migration.
Alternatively, we could simply live with repr(compact) types being difficult
and usually not usable in generic code. They would still be useful in
non-generic code, and in cross-language interop.
alloc::Layout
std::alloc::Layout might not work as is. Consider the following function:
fn make_c_struct() -> Layout {
Layout::from_size_align(0, 1)?
.extend(Layout::new::<T1>())?.0
.extend(Layout::new::<T2>())?.0
.pad_to_align()
}
This function was intended to return a Layout that is interchangeable with
this Rust struct:
#[repr(C)]
struct S {
x: T1,
y: T2,
}
In order for this to continue returning the same Layout, it must work the same
even if T1 is changed to be repr(compact). In other words, if Layout::new
is to accept ?AlignSized types, it must use the stride as the size. The same
applies to for_value*.
(Alternatively, it may be okay to reject non-AlignSized types.)
One assumes, then, that we need *_compact versions of all the layout
functions, which use data size instead of stride. And then:
fn make_c_struct() -> Layout {
Layout::from_size_align(0, 1)?
.extend(Layout::new_compact::<T1>())?.0
.extend(Layout::new::<T2>())?.0
.pad_to_align()
}
Would generate the same Layout as for the following struct:
#[repr(C)]
struct S {
#[compact] x: T1,
y: T2,
}
Alternatively, perhaps we could introduce separated data_size and stride
fields into the Layout, and have extend and extend_compact, supplementing
from_size_align(stride, align) with from_data_size_stride_align(data_size, stride, align).
… but this author is very interested to hear opinions about how this should all work out.
It’s yet another (implicit) size/alignment trait
There is also some desire for an Aligned
trait or a DynSized
trait.
This would be yet another one, which may require changes throughout the Rust
standard library and ecosystem to support everywhere one would ideally hope.
Rationale and alternatives
Alternative: manual layout
One could in theory do it all by hand.
User-defined padding-less references
Instead of references, one could use Pin-like smart pointer types which
forbids direct writes and reads. To avoid aliasing UB, this cannot actually be
Pin<&mut T> etc. – it must be a (wrapper around a) raw pointer, as one must
never actually hold a &mut T or even a &T. This must be done for all Swift
or C++ types which contain (what Rust would consider) tail padding, unless it is
specifically known that they are held in an array, where it’s safe to use Rust
references.
Something like this:
struct PadlessRefMut<'a, T>(*mut T, PhantomData<&'a mut T>);
Unfortunately, today, a generic type like PadlessRefMut is difficult to use:
you cannot use it as a self type for methods, for instance, though there are
workarounds.
Even there, various bits of the Rust ecosystem expect references: for instance,
you can’t return a PadlessRef or PadlessRefMut from an Index or IndexMut
implementation. This, too, could be fixed by replacing the indexing traits (and
everything else with similar APIs) with a more general trait that uses GATs…
but we can see already that, at least right now, this type would be quite
unpleasant.
Layout
For emulating the layout rules of Swift and C++, you could manually lay out
structs (e.g. via a proc macro) and use the same Pin-like pointer type:
// instead of C++:
// `struct Foo {[[no_unique_address]] T1 x; [[no_unique_address]] T2 y; }`
#[repr(C, align( /* max(align_of<T1>(), align_of<T2>()) */ ... ))]
struct Foo {
// These arrays are not of size size_of<T1>() etc., but rather the same as the proposed data_size_of<T1>().
x: [u8; SIZE_OF_T1_DATA],
y: [u8; SIZE_OF_T2_DATA],
}
impl Foo {
fn x_mut(&mut self) -> PadlessRefMut<'_, T1> {
PadlessRefMut::new((&mut self.x).as_mut_ptr() as *mut _)
}
// etc.
}
This is especially easy to do when writing a bindings generator, since you can automatically query the other language’s to find the struct layout, and automatically generate the corresponding Rust.) But otherwise, it’s quite a pain – one would hope, perhaps, for a proc macro to automate this, similar to how Rust automatically infers layout for paddingful structs and types.
Conclusion: manual layout is unpleasant
Almost nothing is impossible in Rust, including this. But it does mean virtually
abandoning Rust in a practical sense: Rust’s references cannot exclude tail
padding, so we use raw pointers instead. Rust’s layout rules cannot omit
padding, and so we replace the layout algorithm with a pile of manually placed
u8 arrays and manually specified alignment. And the result integrates poorly
with the rest of the Rust ecosystem, where most things expect conventional
references, and things that don’t or can’t use references are difficult to work
with.
Alternative: repr(packed), but with aligned fields
We could replicate the layout of C++ and Swift structs, but make them very
unsafe to use, similar to repr(packed). One would still, like repr(packed),
avoid taking or using references to fields inside such structs, and these are
still going to be difficult to work with as a result.
Prior art
Languages with this feature
Swift: Swift implicitly employs this layout strategy for all types and all fields. A type has three size-related properties: its “size”, meaning the literal size taken up by its field, not including padding; its “stride”, meaning the difference between addresses of consecutive elements in an array; and its alignment.
C++: Unlike Swift, C++ does not separate out size and stride into separate
concepts. Instead, it claims that array stride and size are the same thing, as
they are in Rust and C, but that objects can live inside the tail padding of
other objects and that you are simply mutably aliasing into the tail padding in
a way which the language defines the behavior for. C++ nominally allows this for
the tail padding of all types, but only when they are stored in certain places:
objects may be placed inside the tail padding of the previous object when that
previous object is a subobject in the same struct (not, for instance, a separate
local variable), and it is either a base class subobject (so-called “EBO”), or a
[[no_unique_address]] data member (“field”). In practice, however, the
compiler is free to not reuse the tail padding for some types. In the Itanium
ABI, C-like structs (“POD”
types, with an Itanium-ABI-specific definition of
“POD”) do not allow
their tail padding to be reused.
Papers and blog posts
- I worked around this in Crubit, a C++/Rust bindings generator. The design is
here: https://github.com/google/crubit/blob/main/docs/design/unpin.md .
tl;dr: if we assume that the only source of this layout phenomenon is base
classes, then only non-
finalclasses needed to get the uncomfortablePin-like API. Unfortunately, this does not work if[[no_unique_address]]becomes pervasive.
Unresolved questions
- What do we do about
std::alloc::Layout? - What’s the long term future of the
AlignSizedbound? - Clearly, for compatibility reasons if nothing else, Rust types must not have
reusable tail padding unless specially marked. But what about fields: should
it be opt-in per field (like C++), or automatic (like Swift)? In this doc,
it’s assumed to be opt-in per field for
repr(C)(for C++-compatibility), and automatic forrepr(Rust). - How free should Rust be to represent fields compactly in
repr(Rust)types? - Is
repr(C)allowed to use this new layout strategy with specially marked fields using a new attribute, or do we need a newrepr? The documentation is very prescriptive. - This is part of a family of issues with interop, where Rust reference
semantics do not match other languages’ reference semantics. (The other
prominent member of the family is “aliasing”.) Part of the reason for
wanting to use Rust references is simply the raw ergonomics: generic APIs
take and return
&T, self types requiresDeref(which requires reference-compatible semantics), etc. It is worth asking: rather than modifying references, does this cross the line to where we should instead make it more pleasant to use pointers that cannot safely deref? - “Language lawyering”: how does this interact with existing features? For
example, is a
repr(transparent)type alsorepr(compact)? (I believe the answer should be yes.) - TODO: better names for everything. For example,
repr(compact), “data size” anddata_size_of.AlignSizedespecially. - How much of the standard library should be updated to
?AlignSized?