Fix MaybeUninit codegen using GVN by saethlin · Pull Request #147827 · rust-lang/rust

saethlin · 2025-10-17T21:13:52Z

This is an alternative to #142837, based on #146355 (comment).

The general approach I took here is to aggressively propagate anything that is entirely uninitialized. GVN generally takes the approach of only synthesizing small types, but we need to generate large consts to fix the codegen issue.

I also added a special case to MIR dumps for this where now an entirely uninit const is printed as const <uninit>, because otherwise we end up with extremely verbose dumps of the new consts.

After GVN though, we still end up with a lot of MIR that looks like this:

StorageLive(_1);
_1 = const <uninit>;
_2 = &raw mut _1;

Which will break tests/codegen-llvm/maybeuninit-rvo.rs with the naive lowering. I think the ideal fix here is to somehow omit these _1 = const <uninit> assignments that come directly after a StorageLive, but I'm not sure how to do that. For now at least, ignoring such assignments (even if they don't come right after a StorageLive) in codegen seems to work.

Note that since GVN is based on synthesizing a ConstValue which has a defined layout, this scenario still gets deoptimized by LLVM.

#![feature(rustc_attrs)]
#![crate_type = "lib"]
use std::mem::MaybeUninit;

#[unsafe(no_mangle)]
pub fn oof() -> [[MaybeUninit<u8>; 8]; 8] {
    #[rustc_no_mir_inline]
    pub fn inner<T: Copy>() -> [[MaybeUninit<T>; 8]; 8] {
        [[MaybeUninit::uninit(); 8]; 8]
    }

    inner()
}

This case can be handled correctly if enough inlining has happened, or it could be handled by post-mono GVN. Synthesizing UnevaluatedConst or some other special kind of const seems dubious.

saethlin · 2025-10-17T21:14:03Z

@bors try @rust-timer queue

Fix MaybeUninit codegen using GVN

rust-bors · 2025-10-17T23:23:51Z

☀️ Try build successful (CI)
Build commit: e6fd12c (e6fd12c9ca883127b096afd871b40a5714f3d107, parent: f46475914de626785090a05ae037578aaa119fc8)

rust-timer · 2025-10-18T00:40:46Z

Finished benchmarking commit (e6fd12c): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.7%, 0.7%]	1
Regressions ❌ (secondary)	0.5%	[0.5%, 0.5%]	3
Improvements ✅ (primary)	-7.4%	[-7.4%, -7.4%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-3.3%	[-7.4%, 0.7%]	2

Max RSS (memory usage)

Results (primary 0.9%, secondary 0.8%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.9%	[0.9%, 0.9%]	1
Regressions ❌ (secondary)	3.0%	[1.8%, 4.2%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.7%	[-3.7%, -3.7%]	1
All ❌✅ (primary)	0.9%	[0.9%, 0.9%]	1

Cycles

Results (primary -3.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.6%	[-3.6%, -3.6%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-3.6%	[-3.6%, -3.6%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 475.372s -> 474.527s (-0.18%)
Artifact size: 390.39 MiB -> 390.41 MiB (0.00%)

rustbot · 2025-10-19T22:12:51Z

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri

Some changes occurred to the CTFE machinery

cc @RalfJung, @oli-obk, @lcnr

Some changes occurred in compiler/rustc_codegen_ssa

cc @WaffleLapkin

rustbot · 2025-10-19T22:12:53Z

r? @petrochenkov

rustbot has assigned @petrochenkov.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

saethlin · 2025-10-19T22:13:02Z

r? scottmcm

tmiasko · 2025-10-20T06:00:28Z

I think the ideal fix here is to somehow omit these _1 = const assignments that come directly after a StorageLive, but I'm not sure how to do that.

If you go in that direction, please make sure #137936 is fixed first. Otherwise such a transformation would expose the issue in Rust, as opposed to being merely limited to a custom MIR (probably?).

scottmcm · 2025-11-14T20:11:21Z

Note that since GVN is based on synthesizing a ConstValue which has a defined layout, this scenario still gets deoptimized by LLVM.

I think once we move transmute to being union semantics we'll have more options here, can we can rewrite those repeats to transmute(()) even in generic MIR (hopefully).

scottmcm · 2025-11-14T20:14:35Z

+                if let mir::Operand::Constant(const_op) = operand {
+                    let val = self.eval_mir_constant(&const_op);
+                    if val.all_bytes_uninit(self.cx.tcx()) {
+                        return;


pondering: I wonder if there's any places that are intentionally writing uninit somewhere to tell LLVM that the value doesn't matter. It might be interesting to (outside of debug) write undef to the place for the situations where LLVM doesn't do the bad behaviour of copying a constant for that (such as *p = undef; for *p being Scalar or ScalarPair).

But I don't think that would need to be in this PR.

scottmcm · 2025-11-14T20:16:03Z

    }

+    // Printing [MaybeUninit<u8>::uninit(); N] or any other aggregate where all fields are uninit
+    // becomes very verbose. This special case makes the dump terse and clear.


scottmcm · 2025-11-14T20:20:17Z

 pub const fn fully_uninit() -> MaybeUninit<[u8; 10]> {
    const M: MaybeUninit<[u8; 10]> = MaybeUninit::uninit();
-    // CHECK: call void @llvm.memcpy.{{.+}}(ptr align 1 %_0, ptr align 1 {{.*}}[[FULLY_UNINIT]]{{.*}}, i{{(32|64)}} 10, i1 false)
+    // CHECK: ret void


fix: this updated test would still pass without the fix from the PR (since there was already a ret void in it), so you need to change the test somehow.

Perhaps something like

Suggested change

// CHECK: ret void

// returning uninit doesn't need to do anything to the return place at all

// CHECK: start:

// CHECK-NEXT: ret void

bors · 2025-11-22T23:03:26Z

⌛ Testing commit 2d836b9 with merge 1a6ec91...

bors · 2025-11-23T00:20:45Z

💔 Test failed - checks-actions

…cottmcm Fix MaybeUninit codegen using GVN This is an alternative to rust-lang#142837, based on rust-lang#146355 (comment). The general approach I took here is to aggressively propagate anything that is entirely uninitialized. GVN generally takes the approach of only synthesizing small types, but we need to generate large consts to fix the codegen issue. I also added a special case to MIR dumps for this where now an entirely uninit const is printed as `const <uninit>`, because otherwise we end up with extremely verbose dumps of the new consts. After GVN though, we still end up with a lot of MIR that looks like this: ``` StorageLive(_1); _1 = const <uninit>; _2 = &raw mut _1; ``` Which will break tests/codegen-llvm/maybeuninit-rvo.rs with the naive lowering. I think the ideal fix here is to somehow omit these `_1 = const <uninit>` assignments that come directly after a StorageLive, but I'm not sure how to do that. For now at least, ignoring such assignments (even if they don't come right after a StorageLive) in codegen seems to work. Note that since GVN is based on synthesizing a `ConstValue` which has a defined layout, this scenario still gets deoptimized by LLVM. ```rust #![feature(rustc_attrs)] #![crate_type = "lib"] use std::mem::MaybeUninit; #[unsafe(no_mangle)] pub fn oof() -> [[MaybeUninit<u8>; 8]; 8] { #[rustc_no_mir_inline] pub fn inner<T: Copy>() -> [[MaybeUninit<T>; 8]; 8] { [[MaybeUninit::uninit(); 8]; 8] } inner() } ``` This case can be handled correctly if enough inlining has happened, or it could be handled by post-mono GVN. Synthesizing `UnevaluatedConst` or some other special kind of const seems dubious.

saethlin · 2025-11-23T13:24:59Z

@bors r=scottmcm

bors · 2025-11-23T13:25:01Z

📌 Commit 1a4852c has been approved by scottmcm

It is now in the queue for this repository.

bors · 2025-11-23T17:09:11Z

⌛ Testing commit 1a4852c with merge e9acbd9...

bors · 2025-11-23T20:33:01Z

☀️ Test successful - checks-actions
Approved by: scottmcm
Pushing e9acbd9 to main...

github-actions · 2025-11-23T20:36:23Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 122cbd0 (parent) -> e9acbd9 (this PR)

Test differences

Show 8 test diffs

Stage 1

[mir-opt] tests/mir-opt/const_prop/maybe_uninit.rs: [missing] -> pass (J2)
[codegen] tests/codegen-llvm/maybeuninit-array.rs: [missing] -> pass (J3)

Stage 2

[mir-opt] tests/mir-opt/const_prop/maybe_uninit.rs: [missing] -> pass (J0)
[codegen] tests/codegen-llvm/maybeuninit-array.rs: [missing] -> pass (J1)

Additionally, 4 doctest diffs were found. These are ignored, as they are noisy.

Job group index

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard e9acbd99d384280874129fb7fa0da9faeae0d051 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

dist-x86_64-apple: 8070.0s -> 6581.0s (-18.5%)
x86_64-gnu-gcc: 3075.7s -> 3488.2s (+13.4%)
pr-check-1: 1664.4s -> 1887.2s (+13.4%)
x86_64-rust-for-linux: 2633.1s -> 2965.1s (+12.6%)
dist-aarch64-apple: 7581.8s -> 6718.1s (-11.4%)
dist-apple-various: 3564.0s -> 3967.1s (+11.3%)
x86_64-gnu-llvm-20: 2438.3s -> 2712.4s (+11.2%)
x86_64-gnu-llvm-21-2: 5668.1s -> 6297.2s (+11.1%)
i686-gnu-1: 7435.8s -> 8187.7s (+10.1%)
x86_64-gnu-llvm-21-1: 3132.9s -> 3442.2s (+9.9%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-11-23T21:53:36Z

Finished benchmarking commit (e9acbd9): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

If the regression was expected or you think it can be justified,
please write a comment with sufficient written justification, and add
@rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
If you think that you know of a way to resolve the regression, try to create
a new PR with a fix for the regression.
If you do not understand the regression or you think that it is just noise,
you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.1%	[0.1%, 0.1%]	1
Regressions ❌ (secondary)	0.3%	[0.1%, 0.5%]	6
Improvements ✅ (primary)	-7.3%	[-7.3%, -7.3%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-3.6%	[-7.3%, 0.1%]	2

Max RSS (memory usage)

Results (primary -2.1%, secondary 2.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.0%	[1.5%, 6.5%]	9
Improvements ✅ (primary)	-2.1%	[-2.1%, -2.1%]	1
Improvements ✅ (secondary)	-1.6%	[-2.5%, -0.7%]	3
All ❌✅ (primary)	-2.1%	[-2.1%, -2.1%]	1

Cycles

Results (primary -4.1%, secondary -1.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.9%	[1.9%, 7.9%]	2
Improvements ✅ (primary)	-4.1%	[-4.1%, -4.1%]	1
Improvements ✅ (secondary)	-4.1%	[-6.6%, -1.7%]	6
All ❌✅ (primary)	-4.1%	[-4.1%, -4.1%]	1

Binary size

Results (primary 0.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.2%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.2%	[0.2%, 0.2%]	1

Bootstrap: 471.242s -> 472.15s (0.19%)
Artifact size: 386.25 MiB -> 388.25 MiB (0.52%)

Kobzol · 2025-11-25T07:29:55Z

Tiny regressions on secondary benchmarks, but a large (and real) win on the cranelift-codegen opt benchmark. This should ideally produce better codegen.

@rustbot label: +perf-regression-triaged

…ttmcm Fix MaybeUninit codegen using GVN This is an alternative to rust-lang#142837, based on rust-lang#146355 (comment). The general approach I took here is to aggressively propagate anything that is entirely uninitialized. GVN generally takes the approach of only synthesizing small types, but we need to generate large consts to fix the codegen issue. I also added a special case to MIR dumps for this where now an entirely uninit const is printed as `const <uninit>`, because otherwise we end up with extremely verbose dumps of the new consts. After GVN though, we still end up with a lot of MIR that looks like this: ``` StorageLive(_1); _1 = const <uninit>; _2 = &raw mut _1; ``` Which will break tests/codegen-llvm/maybeuninit-rvo.rs with the naive lowering. I think the ideal fix here is to somehow omit these `_1 = const <uninit>` assignments that come directly after a StorageLive, but I'm not sure how to do that. For now at least, ignoring such assignments (even if they don't come right after a StorageLive) in codegen seems to work. Note that since GVN is based on synthesizing a `ConstValue` which has a defined layout, this scenario still gets deoptimized by LLVM. ```rust #![feature(rustc_attrs)] #![crate_type = "lib"] use std::mem::MaybeUninit; #[unsafe(no_mangle)] pub fn oof() -> [[MaybeUninit<u8>; 8]; 8] { #[rustc_no_mir_inline] pub fn inner<T: Copy>() -> [[MaybeUninit<T>; 8]; 8] { [[MaybeUninit::uninit(); 8]; 8] } inner() } ``` This case can be handled correctly if enough inlining has happened, or it could be handled by post-mono GVN. Synthesizing `UnevaluatedConst` or some other special kind of const seems dubious.

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Oct 17, 2025

This comment has been minimized.

Sign in to view

rust-bors Bot added a commit that referenced this pull request Oct 17, 2025

Auto merge of #147827 - saethlin:maybeuninit-codegen2, r=<try>

e6fd12c

Fix MaybeUninit codegen using GVN

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Oct 17, 2025

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Oct 18, 2025

saethlin force-pushed the maybeuninit-codegen2 branch from 6d353c3 to 1a410f7 Compare October 19, 2025 21:47

saethlin marked this pull request as ready for review October 19, 2025 22:12

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 19, 2025

rustbot assigned petrochenkov Oct 19, 2025

rustbot assigned scottmcm and unassigned petrochenkov Oct 19, 2025

This comment has been minimized.

Sign in to view

saethlin force-pushed the maybeuninit-codegen2 branch from 1a410f7 to 177e9fc Compare October 19, 2025 23:18

RalfJung mentioned this pull request Oct 20, 2025

Creating a nested MaybeUninit array unnecessarily calls memset #139355

Closed

scottmcm reviewed Nov 14, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 23, 2025

Fix MaybeUninit codegen using GVN

1a4852c

saethlin force-pushed the maybeuninit-codegen2 branch from 2d836b9 to 1a4852c Compare November 23, 2025 13:23

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 23, 2025

saethlin linked an issue Nov 23, 2025 that may be closed by this pull request

Creating a nested MaybeUninit array unnecessarily calls memset #139355

Closed

bors added the merged-by-bors This PR was explicitly merged by bors. label Nov 23, 2025

bors merged commit e9acbd9 into rust-lang:main Nov 23, 2025
12 checks passed

rustbot added this to the 1.93.0 milestone Nov 23, 2025

saethlin mentioned this pull request Nov 23, 2025

Add OperandValue::Uninit to improve lowering of MaybeUninit::uninit #142837

Closed

This was referenced Nov 23, 2025

Replace Rvalue::NullaryOp by a variant in mir::Operand. #148766

Merged

Use let...else instead of match foo { ... _ => return }; and if let ... else return #148837

Merged

rustbot added the perf-regression-triaged The performance regression has been triaged. label Nov 25, 2025

saethlin deleted the maybeuninit-codegen2 branch November 25, 2025 22:12

matthiaskrgr mentioned this pull request Dec 4, 2025

ICE interpreter ran out of memory #149643

Closed

theemathas mentioned this pull request Dec 7, 2025

Optimization of deinitialization became worse in nightly #149727

Open

steffahn mentioned this pull request Feb 12, 2026

MaybeUninit::uninit zero initializes large uninit array besides small other field #152541

Open

-    // CHECK: ret void
+    // returning uninit doesn't need to do anything to the return place at all
+    // CHECK: start:
+    // CHECK-NEXT: ret void

Uh oh!

Conversation

saethlin commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

saethlin commented Oct 17, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors Bot commented Oct 17, 2025

Uh oh!

This comment has been minimized.

rust-timer commented Oct 18, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

rustbot commented Oct 19, 2025

Uh oh!

rustbot commented Oct 19, 2025

Uh oh!

saethlin commented Oct 19, 2025

Uh oh!

This comment has been minimized.

tmiasko commented Oct 20, 2025

Uh oh!

scottmcm commented Nov 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bors commented Nov 22, 2025

Uh oh!

This comment has been minimized.

bors commented Nov 23, 2025

Uh oh!

saethlin commented Nov 23, 2025

Uh oh!

bors commented Nov 23, 2025

Uh oh!

bors commented Nov 23, 2025

Uh oh!

bors commented Nov 23, 2025

Uh oh!

Uh oh!

github-actions Bot commented Nov 23, 2025

Test differences

Stage 1

Stage 2

Job duration changes

Uh oh!

rust-timer commented Nov 23, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

Kobzol commented Nov 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

saethlin commented Oct 17, 2025 •

edited

Loading