Implement cursor reallocation handling in DynamicBuffer pinning example

src/unsafe-deep-dive/pinning/README.md

@@ -0,0 +1,19 @@
+# pinning
+
+> **Important Note**
+>
+> To not add this section to the project's SUMMARY.md yet. Once CLs/PRs to
+> accept all the new segments for the Unsafe Deep Dive have been included in the
+> repository, an update to SUMMARY.md will be made.
+
+## About
+
+This segment explains pinning, Rust's `Pin<Ptr>` type and concepts that relate
+to FFI rather than its async use case. Treatment of the `Unpin` trait and the
+`PhantomPinned` type is provided.
+
+## Status
+
+Provisional/beta.
+
+## Outline

src/unsafe-deep-dive/pinning/phantompinned.md

@@ -0,0 +1,37 @@
+# PhantomPinned
+
+The idiomatic way to opt-out of Rust's aliasing
+
+Usage
+
+```rust,editable
+pub struct DynamicBuffer {
+    data: Vec<u8>,
+    cursor: NonNull<u8>,
+    _pin: std::marker::PhantomPinned,
+}
+
+impl DynamicBuffer {
+    pub fn push(&mut self, byte: u8) {
+        // Calculate the cursor offset before the push (which may reallocate)
+        let offset = unsafe {
+            self.cursor.as_ptr().offset_from(self.data.as_ptr())
+        };
+
+        self.data.push(byte);
+
+        // Update cursor to point to the same offset in the (potentially new) buffer
+        self.cursor = unsafe {
+            NonNull::new_unchecked(self.data.as_mut_ptr().offset(offset))
+        };
+    }
+}
+```
+
+<details>
+
+If a type contains a `PhantomPinned`, it will not implement `Unpin` by default.
+
+<!-- TODO: Monitor issue https://github.com/rust-lang/rust/issues/125735 as this guidance will change at some point and future code will move to UnsafePinned -->
+
+</details>

src/unsafe-deep-dive/pinning/self-referential-buffer/cpp.md

@@ -0,0 +1,53 @@
+# Motivating Example: C++
+
+```cpp,editable,ignore
+class SelfReferentialBuffer {
+    char data[1024];
+    char* cursor;
+
+public:
+    SelfReferentialBuffer() = default;
+
+    SelfReferentialBuffer(SelfReferentialBuffer&& other) 
+        : cursor(data + (other.cursor - other.data))
+    {
+        std::memcpy(data, other.data, 1024);
+    }
+};
+```
+
+Investigate on [Compiler Explorer](https://godbolt.org/z/ascME6aje)
+
+<details>
+
+The `SelfReferentialBuffer` contains two members, `data` is a kilobyte of memory
+and `cursor` is a pointer into the former.
+
+Its move constructor ensures that cursor is updated to the new memory address.
+
+This type can't be expressed easily in Rust.
+
+> Note: `char*` is dated, but exists in legacy codebases and is used here for
+> simplicity.
+>
+> If your class includes experienced C++ developers, consider replacing `char*`
+> with `std::byte*`.
+>
+> ```cpp
+> #include <cstddef>
+> #include <cstring>
+>
+> class SelfReferentialBuffer {
+>     std::byte data[1024];
+>     std::byte* cursor = data;
+>     
+> public:
+>     SelfReferentialBuffer(SelfReferentialBuffer&& other) 
+>         : cursor{data + (other.cursor - other.data)}
+>     {
+>         std::memcpy(data, other.data, 1024);
+>     }
+> };
+> ```
+
+</details>

src/unsafe-deep-dive/pinning/self-referential-buffer/rust.md

@@ -0,0 +1,37 @@
+# Rust
+
+```rust,editable
+// class SelfReferentialBuffer {
+//     char data[1024];
+//     char* cursor;
+//  
+//     ...
+//
+// };
+
+// Close to the original, but requires unsafe
+struct SelfReferentialBuffer {
+    data: [u8; 1024],
+    cursor: *const u8,
+}
+
+
+// More idiomatic, with different semantics
+struct SelfReferentialBufferSafe {
+    data: [i8; 1024],
+    position: usize,
+}
+```
+
+<details>
+
+While Rust would allow us to create a similar struct to the C++ class, it has a
+significant cost.
+
+We would give up references, falling back to raw pointers. This imposes unsafe
+code later on.
+
+A more idiomatic version would be to maintain an offset using a `usize`, then
+creating a reference to `self` on demand.
+
+</details>

src/unsafe-deep-dive/pinning/unpin-trait.md

@@ -0,0 +1,19 @@
+# Unpin trait
+
+- `T: Unpin` implies that `T` is not pinned
+- Automatically implemented by the compiler for nearly every type
+- To opt out of this for your type, add a [`PhantomPinned`] field to your type
+  (required for FFI)
+
+<details>
+
+Most types implement `Unpin` automatically `Unpin` types can be moved even when
+pinned
+
+`!Unpin` types cannot be moved once pinned
+
+Unpin is a promise: "moving me is always safe"
+
+</details>
+
+[`Pantom`]: https://doc.rust-lang.org/std/marker/struct.PhantomPinned.html

src/unsafe-deep-dive/pinning/welcome.md

@@ -0,0 +1,30 @@
+# Welcome
+
+This segment of the course covers:
+
+- What "pinning" is
+- Why it is necessary
+- How Rust implements it
+- How it interacts with unsafe and FFI
+
+<details>
+
+"Pinning, or holding a value's memory address in a fixed location,is one of the
+more challenging concepts in Rust."
+
+"Normally only seen within async code, i.e. [`poll(self: Pin<&mut Self>)`],
+pinning has wider applicability."
+
+Some some data structures that are difficult or impossible to write without the
+unsafe keyword, including self-referential structs and intrusive data
+structures.
+
+FFI with C++ is a prominent use case that's related to this. Rust must assume
+that any C++ with a reference might be a self-referential data structure.
+
+"To understand this conflict in more detail, we'll first need to make sure that
+we have a strong understanding of Rust's move semantics."
+
+<details>
+
+[poll]: https://doc.rust-lang.org/std/future/trait.Future.html#tymethod.poll

src/unsafe-deep-dive/pinning/what-a-move-is.md

@@ -0,0 +1,66 @@
+# What a move is in Rust
+
+Always a bitwise copy, even for types that do not implement `Copy`:
+
+```rust
+#[derive(Debug, Default)]
+pub struct DynamicBuffer {
+    data: Vec<u8>,
+    position: usize,
+};
+
+pub fn move_and_inspect(x: DynamicBuffer) { println!("{x:?}"); }
+
+pub fn main() {
+   let a = DynamicBuffer::default();
+   let mut b = a;
+   b.data.push(b'R');
+   b.data.push(b'U');
+   b.data.push(b'S');
+   b.data.push(b'T');
+   move_and_inspect(b);
+}
+```
+
+Generated [LLVM IR] for calling `move_and_expect()`:
+
+```llvm
+call void @llvm.memcpy.p0.p0.i64(ptr align 8 %_12, ptr align 8 %b, i64 32, i1 false)
+invoke void @move_and_inspect(ptr align 8 %_12)
+```
+
+- `memcpy` from variable `%b` to `%_12`
+- Call to `move_and_inspect` with `%_12` (the copy)
+
+<details>
+
+Note that `DynamicBuffer` does not implement `Copy`.
+
+Implication: a value's memory address is not stable.
+
+To show movement as a bitwise copy, either [open the code in the playground]()
+and look at the or [the Compiler Explorer].
+
+Optional for those who prefer assembly output:
+
+The Compiler Explorer is useful for discussing the generated assembly and focus
+the cursor assembly output in the `main` function on lines 128-136 (should be
+highlighted in pink).
+
+Relevant code generated output `move_and_inspect`:
+
+```assembly
+mov     rax, qword ptr [rsp + 16]
+mov     qword ptr [rsp + 48], rax    
+mov     rax, qword ptr [rsp + 24]
+mov     qword ptr [rsp + 56], rax
+movups  xmm0, xmmword ptr [rsp]
+movaps  xmmword ptr [rsp + 32], xmm0
+lea     rdi, [rsp + 32]
+call    qword ptr [rip + move_and_inspect@GOTPCREL]
+```
+
+</details>
+
+[LLVM IR]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=6f587283e8e0ec02f1ea8e871fc9ac72
+[The Compiler Explorer]: https://rust.godbolt.org/z/6o6nP7do4

src/unsafe-deep-dive/pinning/what-pinning-is.md

@@ -0,0 +1,46 @@
+# What pinning is
+
+Abridged `Pin` from the Rust standard library:
+
+```rust,ignore
+#[repr(transparent)]
+pub struct Pin<Ptr> {
+    pointer: Ptr,
+}
+
+impl<Ptr: Deref<Target: Unpin>> Pin<Ptr> {
+    pub fn new(pointer: Ptr) -> Pin<Ptr> { ... }
+
+    pub fn into_inner(pin: Pin<Ptr>) -> Ptr { ... }
+
+    pub unsafe fn new_unchecked(pointer: P) -> Pin<Ptr> { ... }
+}
+```
+
+<details>
+
+Conceptually, pinning prevents the default movement behavior.
+
+This appears to be a change in the language itself.
+
+However, the `Pin` wrapper doesn't actually change anything fundamental about
+the language.
+
+`Pin` doesn't expose safe APIs that would allow a move. Thus, it can prevent
+bitwise copy.
+
+Unsafe APIs allow library authors to wrap types that do not implement `Unpin`,
+but they must uphold the same guarantees.
+
+The documentation of `Pin` uses the term "pointer types".
+
+The term "pointer type" is much more broad than the pointer primitive type in
+the language.
+
+A "pointer type" wraps every type that implements `Deref` with a target that
+implements `Unpin`.
+
+Rust style note: This trait bound is enforced through trait bounds on the
+`::new()` constructor, rather than on the type itself.
+
+</details>