Fix variable deallocation order in panic unwinding paths

[sladyn98]

View all comments

This PR fixes a soundness bug where local variables are deallocated out of order during panic unwinding, allowing destructors to access freed memory. This violates Rust's safety guarantees and has caused real-world unsoundness in crates like generatively.

This PR removes the is_generator check and unconditionally emits StorageDead statements during unwinding for ALL functions, bringing non-generator behavior in line with generators. It ensures that during unwinding, when a local variable goes out of scope, its storage is properly marked as dead via StorageDead, allowing the borrow checker to enforce the
invariant that values must outlive their references even in panic paths.

Fixes #147875

[rustbot]

r? @wesleywiser

rustbot has assigned @wesleywiser.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[sladyn98]

r? @dianne

[dianne]

It looks like there's a bug causing an assertion failure when building the standard library. I've given it a look and offered a guess at what's causing it below. There's still work to do here beyond fixing that, though.

First, could you add a ui test to demonstrate that this fixes #147875? It looks like it might not yet, since the code for scheduling unwind drops on calls panicking looks unchanged.

Second, after verifying that this results in the correct borrow-checking behavior, we need to make sure that this change doesn't negatively affect codegen. Per the old comment on needs_cleanup, at least at the time it was written, LLVM didn't handle the unnecessary cleanup blocks and StorageDeads particularly well. If you can demonstrate with codegen tests that that's not an issue anymore, and perf isn't too bad, that might be all that's needed. But my expectation is that we'll have to get rid of or ignore the StorageDeads later in compilation (sometime after they serve their purpose in borrowck). Unless there's a reason to keep the StorageDeads around longer, my gut feeling is that this cleanup would be best as a post-borrowck MIR pass (maybe as part of CleanupPostBorrowck?), since then optimization passes can be done on cleaner MIR and we can test it works with MIR tests rather than codegen tests. Could you also add a test for this not affecting later stages of compilation? If you accomplish that by removing the unwind-path StorageDeads as part of a MIR pass, that'd be a mir-opt test.

Before you push again, you'll probably want to run the codegen and mir-opt tests to make sure the former is clean and to bless the latter. Regardless of what approach we take here, if we're changing how the MIR is built, there should be differences in the MIR building test output (part of the mir-opt suite).

View changes since this review

[rustbot]

Reminder, once the PR becomes ready for a review, use @rustbot ready.

[dianne]

Also, could you change the PR description? #147875 on its own doesn't allow destructors to access freed memory, it doesn't allow for the creation of dangling references, and I'm at least not aware of a safety guarantee that it violates. You should only get unsoundness out of it if you write unsafe code on the assumption that the borrow checker will enforce the relative drop order of locals that may have destructors and those that definitely don't. Of course, per language team decision, consistent drop order is a promise Rust would like to make. But it's not quite the same as the borrow-checker failing to ensure places outlive their references.

[sladyn98]

So what i did was write this simple rust program panic drop.rs

 //@ compile-flags: -C no-prepopulate-passes

#![crate_type = "lib"]

#[no_mangle]
pub fn function_with_drops() {

    let _a = String::from("first");
    let _b = String::from("second");
    let _c = String::from("third");
    might_panic();
}

#[inline(never)]
pub fn might_panic() {
    // This might panic
    panic!()
}

I ran the llvm to get the intermediate representaion and on looking at the IR I cannot find any llvm.lifetime.end statements suggesting to us that on master the StorageDead statements are missing, which according to my understanding means that the borrowchecker does not know when the storage becomes invalid. Let me now write the UI test to see what is up

[dianne]

StorageDead is indeed not present there on the main branch, but that doesn't create a soundness hole in the borrow-checker; those locals are simply treated as being maybe live until the end of the stack frame, so it's technically sound for destructors on the unwind path to reference them. When unwinding, the stack frame will be popped anyway, so we don't need to hint to llvm that the memory's no longer in use. The issue is that we'd like to be more strict in the borrow-checker, both for consistency and so that unsafe code can rely on drop order being enforced.

edit: adjusted wording

[dianne]

This still needs CI to pass before I can review it properly. I've left a few comments on obvious things, but I don't think reviewing the code changes would be helpful at this point. Please test your changes locally. You don't have to run the whole test suite yourself, but for this change, you'll at least want make sure that the mir-opt and codegen tests all pass, that any relevant ui tests pass, and that tidy passes as well.

Could you rebase onto a more recent commit, also? I don't expect there will be conflicts in the MIR building part of this, but I'm not sure about the rest.

I don't mean to be harsh, but this is a relatively complex and nuanced change. If you're not familiar with what's being changed, why it's being changed, the consequences/needs of that, and general contribution procedure, I'd recommend gaining familiarity with easier issues instead.

View changes since this review

[rustbot]

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

[Dylan-DPC]

@sladyn98 any updates on this? thanks

[rustbot]

This PR modifies tests/ui/issues/. If this PR is adding new tests to tests/ui/issues/,
please refrain from doing so, and instead add it to more descriptive subdirectories.

[dianne]

Since this seems to be a significant breaking change, we'll need to issue a future-compatibility warning on code it'll break before we can land it. To implement the warning, you might want to look into adding BackwardIncompatibleDropHints to the MIR at points where we want to add StorageDeads. That's how the tail_expr_drop_order edition migration lint warns when shorter lifetimes in the 2024 edition will result in a borrowck error. Hopefully the same machinery should be usable for this?

I also discussed this with @Amanieu at the All Hands. If I recall correctly, StorageDeads are needed on the unwind path for MIR move elimination (rust-lang/rfcs#3943). Probably they actually shouldn't be cleaned up immediately post-borrowck in that case (oops)? Regardless, I'll be handing the review over. Sorry for all the sudden changes ^^

r? @Amanieu

[Amanieu]

To clarify, the plan would be to emit BackwardIncompatibleDropHint in the unwind path in places where we should emit StorageDead, and then over an edition we would switch this to actually emit StorageDead.

As for keeping these after borrowck for MIR optimizations, this isn't something that needs to be done in this PR, it can be deferred to a future PR.

[rustbot]

This PR changes MIR

cc @oli-obk, @RalfJung, @JakobDegen, @vakaras

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[rustbot]

The Miri subtree was changed

cc @rust-lang/miri