Although I am now mostly comfortable with Rust, some concepts still elude me. One of them is the exact meaning of Unpin
. The documentation says:
The documentation of Unpin says:
Types that do not require any pinning guarantees.
Where pinning is described as:
The ability to rely on this guarantee that the value a pointer is pointing at (its pointee) will
- Not be moved out of its memory location
- More generally, remain valid at that same memory location
From this, you could naturally deduce that Unpin
is the trait that allows Rust values to be moved.
Except it is not. You can move !Unpin
values:
fn is_unpin<T: Unpin>(_: &T) {}
fn f<T>(_: T) {}
fn main() {
let val = async { 1 }; // this is a Future, which is !Unpin
//is_unpin(a); // error: the trait `Unpin` is not implemented
let other_val = val; // move, ok
f(other_val); // move, ok
}
I am definitely not the first one to get confused by Unpin
. The thing is that, if you’re here, you are probably thinking of Unpin
wrong. Unpin
is actually a trait for very niche use cases, which you probably don’t need. And, to understand why, you first need to understand Pin
and pinning.
Self-Referential Objects
Some objects contain references to part of themselves. Common examples are pointing to a slice of a buffer, or linked lists. However, in practice, the most common case in Rust is futures, from asynchronous code.
For example, consider the asynchronous function below.
async fn f() {
let a = 3;
let b = &a;
tokio::time::sleep(Duration::from_secs(1)).await;
println!("{b}");
}
When you write f()
, it calls the asynchronous function. More specifically, it creates an impl Future
object, its role is to store the data that needs to be kept while the function is not running. That is, across await
points. The lifetime of the future will look like this:
- Create the
impl Future
when callingf()
; let’s call itfuture
. - The user passes that future to a runtime, for instance with
tokio::spawn(future)
. - At some point, the runtime calls
future.poll()
. - The code from
f
is executed until the firstawait
. future
stores the current state of the execution, which includes the position in the code (theawait
from line 4), as well as the local variables,a
andb
.- That first call to
future.poll()
returns Poll::Pending, so the runtimes knows it is not done yet. - At some point, the runtime calls
future.poll()
again. - The rest of the code from
f
is executed, resuming from theawait
from line 4. - Since we reached the end of the function,
future.poll()
returnsPoll::Ready(())
, where()
is the value returned by the code above. - The runtime drops
future
.
Step 5 is where future
becomes self-referential. Indeed, it must store both a
and &a
.
(Not) Moving
Self-referential objects must not be moved in memory. If you were to do so, it would change their address, and invalidate the references that point to them1.
For example, that means that you cannot put such objects in a Vec
. The Vec
would need to move them in memory when reallocating. You would think that this means that Vec<T>
requires T: Unpin
. But that’s not how it works. Unpin
does not mean objects can be moved.
In fact, Rust values can always be moved. As shown in the example in the introduction, you can even move objects which do not implement Unpin
. You won’t be able to get Rust to complain that you cannot put some value into a vector.
Technically, safe Rust does let you build self-referential objects:
struct SelfReferential {
self_ptr: *const Self,
}
fn main() {
let mut s = SelfReferential {
self_ptr: std::ptr::null(),
};
s.self_ptr = &s as *const SelfReferential
}
However, this only works with raw pointers, whose validity is not tracked by the compiler. If you move the object, that pointer becomes invalid, and using it would be undefined behavior (it would break the aliasing rules). This is why dereferencing pointers is unsafe
: it is up to the user to make sure that the pointer remains valid.
So, safe Rust won’t let you use these self-references. And borrow rules will also prevent you from constructing such an object with references instead of raw pointers:
struct SelfReferential<'a> {
self_ref: Option<&'a Self>,
}
fn main() {
let mut s = SelfReferential {
self_ref: None,
};
// error: cannot assign to `s.self_ref` because it is borrowed
s.self_ref = Some(&s);
}
In short, you can move any value in Rust, and that’s fine, because values cannot be self-referential.
Pinning
As discussed above, safe Rust won’t let you work directly with self-referential objects. So, all you can do is holding a pointer to that value.
Also, note that, to remain safe, such a pointer should not let you “take” that value to put it elsewhere (e.g. let x = *p
, or std::mem::replace(&mut p, …)
). That’s what Pin
does. It is a safe interface for pointers to objects that should not be moved.
In other words, you can have a Pin<&T>
, a Pin<&mut T>
, a Pin<Rc<T>>
, a Pin<Box<T>>
. And these will let you interact with the T
object, without ever letting you move it. The pin!
macro lets you create a pinned pointer to a value on the stack2, and the Box::pin
method puts the value on the heap and returns a pinned pointer to it.
Note that, since Pin<…>
is a pointer, it itself is totally fine to move. In fact, it is itself Unpin
:
fn is_unpin<T: Unpin>(_: &T) {}
fn main() {
let val = async { 1 };
//is_unpin(a); // error: the trait `Unpin` is not implemented
let ptr = std::pin::pin!(val);
is_unpin(&ptr); // ok
*ptr; // error: cannot move out of dereference of Pin<…>
}
Pinning Futures
As an aside, I would like to discuss something that sometimes surprised me when working with asynchronous code. Specifically, when you call an asynchronous function, you get an impl Future
which is not pinned, but Future::poll
takes Pin<&mut Self>
.
The reason for that is actually simple: this allows the user to pin the future on the stack. If calling an asynchronous function returned a pinned future, that would require systematically using the heap, like you would do with Box::pin
.
This is perfectly fine, because futures are not self-referential until you call Future::poll
. When the user does, they have to do so on a pinned reference to the future, ensuring that they won’t be able to move it anymore. This is guaranteed by the safe interface of Pin
.
Unpin
The title of this article is about Unpin
, so I do need to talk about it.
Unpin
is actually something very niche. To simplify a bit, the official documentation explains that it is automatically implemented for types that cannot be self-referential, and allows these types to be safely moved out of Pin
. However, it does not explain why it would be useful. The best I could find is an article that uses it when implementing complex future wrapper; however it still requires unsafe.
Conclusion
There are three main takeaways from this article:
- All Rust values can be moved
Pin
is a pointer wrapper for objects that should not be movedUnpin
is a very niche trait that is only indirectly related to moving
Writing this article forced me to actually understand the relationship between Pin
and Unpin
, which is not what I expected. This will probably help my blazingly fast code in the future!