From: Michael Anthony Knyszek Date: Wed, 25 Jun 2025 15:47:05 +0000 (+0000) Subject: runtime: make explicit nil check in (*spanInlineMarkBits).init X-Git-Tag: go1.25rc2~2^2~28 X-Git-Url: http://www.git.cypherpunks.su/?a=commitdiff_plain;h=f8ccda2e05;p=gostls13.git runtime: make explicit nil check in (*spanInlineMarkBits).init The hugo binary gets slower, potentially dramatically so, with GOEXPERIMENT=greenteagc. The root cause is page mapping churn. The Green Tea code introduced a new implicit nil check on value in a freshly-allocated span to clear some new heap metadata. This nil check would read the fresh memory, causing Linux to back that virtual address space with an RO page. This would then be almost immediately written to, causing Linux to possibly flush the TLB and find memory to replace that read-only page (likely deduplicated as just the zero page). This CL fixes the issue by replacing the implicit nil check, which is a memory read expected to fault if it's truly nil, with an explicit one. The explicit nil check is a branch, and thus makes no reads to memory. The result is that the hugo binary no longer gets slower. No regression test because it doesn't seem possible without access to OS internals, like Linux tracepoints. We briefly experimented with RSS metrics, but they're inconsistent. Some system RSS metrics count the deduplicated zero page, while others (like those produced by /proc/self/smaps) do not. Instead, we'll add a new benchmark to our benchmark suite, separately. For #73581. Fixes #74375. Change-Id: I708321c14749a94ccff55072663012eba18b3b91 Reviewed-on: https://go-review.googlesource.com/c/go/+/684015 Reviewed-by: Keith Randall Reviewed-by: Michael Pratt Auto-Submit: Michael Knyszek LUCI-TryBot-Result: Go LUCI Reviewed-by: Keith Randall --- diff --git a/src/runtime/mgcmark_greenteagc.go b/src/runtime/mgcmark_greenteagc.go index 75c347b9e9..ac2b1732f9 100644 --- a/src/runtime/mgcmark_greenteagc.go +++ b/src/runtime/mgcmark_greenteagc.go @@ -111,6 +111,26 @@ func (o *spanScanOwnership) or(v spanScanOwnership) spanScanOwnership { } func (imb *spanInlineMarkBits) init(class spanClass) { + if imb == nil { + // This nil check and throw is almost pointless. Normally we would + // expect imb to never be nil. However, this is called on potentially + // freshly-allocated virtual memory. As of 2025, the compiler-inserted + // nil check is not a branch but a memory read that we expect to fault + // if the pointer really is nil. + // + // However, this causes a read of the page, and operating systems may + // take it as a hint to back the accessed memory with a read-only zero + // page. However, we immediately write to this memory, which can then + // force operating systems to have to update the page table and flush + // the TLB, causing a lot of churn for programs that are short-lived + // and monotonically grow in size. + // + // This nil check is thus an explicit branch instead of what the compiler + // would insert circa 2025, which is a memory read instruction. + // + // See go.dev/issue/74375 for details. + throw("runtime: span inline mark bits nil?") + } *imb = spanInlineMarkBits{} imb.class = class }