Memory Address Pinning

RecipeCrates
Give Data a Stable Memory Address with Pinstd
Related Data Structuresstd

Give Data a Stable Memory Address with Pin

std

The following is an advanced topic. It is highly recommended to read the documentation↗ in addition to the following summary.

In Rust, the address at which a value is located is not necessarily stable in-between borrows. The compiler is allowed to move a value to a new memory address in many places, for example during assignment or when passing a value into a function.

It is sometimes useful to be able to rely upon memory addresses not changing, especially when there are pointers pointing at that value.

The std::pin↗ module enables pinning data - preventing it from being moved in memory, and, more generally, guaranteeing it remains valid at that same memory location.

Pinning is typically used for self-referential types, compiler-generated generators for async fn↗, and intrusive data structure, where moving would break safety invariants:

  • In asynchronous programming, you may have futures that need to maintain a stable memory address, as they may need to reference themselves or other data that should not be moved.
  • Pinning a value is an useful building block for unsafe code to be able to reason about whether a raw pointer to the pinned value is still valid.

The main type in std::pin↗ is Pin<Ptr>↗, a smart pointer wrapper that flags that the data behind the pointer Ptr should not be moved in memory, even if it is mutable, unless it implements Unpin(see below).

Note that the pointer wrapped by Pin is not the value which we want to pin itself, but rather a pointer to that value. A Pin<Ptr> does not pin the Ptr; instead, it pins the pointer's pointee value.

It is important to note that pinning does not make use of any compiler "magic". It does not change the way the compiler behaves towards the inner value (it still considers the inner value fundamentally movable). Instead, Pin<Ptr> is a wrapper that prohibits calling code that would perform a move on the pinned value and enforces the use of unsafe code for dangerous operations. BEWARE: It is the responsibility of the programmer to implement that unsafe code correctly to satisfy the Pin invariants.

Because Pin has a restrictive API, Rust provide an "escape hatch" for the vast majority of Rust types, which have no address-sensitive states.

The Unpin↗ trait is a built-in, auto-implemented marker trait that signifies a type can be safely moved in memory, even after it has been "pinned." It cancels the restrictive effects of Pin.

  • Most types implement Unpin by default. Builtin types that are Unpin include all of the primitive types, like bool, i32, and f32, references (&T and &mut T), etc., as well as many core and standard library types like Box<T>, String.
  • The compiler automatically implements Unpin for any struct or enum if all of its fields are also Unpin.

Therefore, to define custom types that must not move in memory, you must "opt out" of the Unpin trait by having a field that is !Unpin. Rust provides for this purpose the marker type std::marker::PhantomPinned↗, which is !Unpin.

The following example demonstrates the basic usage of Pin, Unpin, PhantomPinned and the pin! macro:

fn main() {
    use std::pin::Pin;

    // In order to pin a value, we wrap a pointer to that value (of some type
    // `Ptr`) in a `Pin<Ptr>`. `Pin<Ptr>` can wrap any pointer type,
    // forming a promise that the _pointee_ will not be moved or otherwise
    // invalidated.

    // 1. **Trivial case:** If the pointee value's type implements `Unpin`,
    //    which
    // cancels the effect of `Pin`, we can wrap any pointer to that value in
    // `Pin` directly via `Pin::new`.
    // That allows `x` to participate in `Pin`-bound APIs.
    let mut x = 42;
    let pinned_x: Pin<&mut i32> = Pin::new(&mut x);

    // Conversely, we can unwrap the pin to get the underlying mutable reference
    // to the value.
    let r = Pin::into_inner(pinned_x);
    assert_eq!(*r, 42);

    // 2. **General case:**
    // If the pointee value's type does not implement `Unpin`, then Rust will
    // not let us use the `Pin::new` function and we'll need to
    // use specialized ways, here unsafe code, to manipulate the inner value.
    //
    // The programmer MUST guarantee the underlying data truly
    // does not move in the `unsafe` code. Here, we have exclusive control of
    // the `String`.
    {
        use std::marker::PhantomPinned;
        pub struct Unpinned {
            // The `PhantomPinned` field makes the struct as `!Unpin`.
            _pin: PhantomPinned,
        }

        let mut y = Unpinned {
            _pin: PhantomPinned,
        };
        let _pinned_y = unsafe { Pin::new_unchecked(&mut y) };
    }

    // 3. **Pinned Box**: The simplest way to pin a value that
    // does not implement `Unpin` is to put that value inside a `Box` and
    // then wrapping it in a `Pin`.
    // Example fom <https://doc.rust-lang.org/std/pin/struct.Pin.html#pinning-a-value-inside-a-box>.
    async fn add_one(x: u32) -> u32 {
        x + 1
    }

    // Call the async function to get a future back.
    let fut = add_one(42); // `impl Future<Output = u32>`.

    // Pin the future inside a box.
    let pinned_box_fut: Pin<Box<_>> = Box::pin(fut);

    // Gets a shared reference to the pinned value this `Pin` points to.
    // This is a generic method to go from `&Pin<Pointer<T>>` to `Pin<&T>`.
    let _pinned_fut = pinned_box_fut.as_ref(); // `Pin<&(impl Future<Output = u32>)>`.

    // 4. **`pin!` Macro**:
    // There are some situations where it is desirable or even required (e.g.,
    // in a `#[no_std]` context) to pin a value which does not implement `Unpin`
    // to its location _on the stack_. Doing so is possible using the `pin!`
    // macro. Unlike `Box::pin`, this does not systematically create a new
    // heap allocation. <https://doc.rust-lang.org/std/pin/macro.pin.html>
    use core::pin::pin;

    struct Foo;

    fn stuff(_foo: Pin<&mut Foo>) {
        // ...
    }

    let pinned_foo = pin!(Foo { /* ... */ });
    stuff(pinned_foo);
}
  • Futures.
  • Traits.