From: Michael Anthony Knyszek Date: Thu, 2 Oct 2025 17:16:49 +0000 (+0000) Subject: runtime: optimistically CAS atomicstatus directly in enter/exitsyscall X-Git-Tag: go1.26rc1~412 X-Git-Url: http://www.git.cypherpunks.su/?a=commitdiff_plain;h=8683bb846dfc1460c476cfed1696aad8e681926f;p=gostls13.git runtime: optimistically CAS atomicstatus directly in enter/exitsyscall This change steals the performance trick from the coro implementation to try to do the CAS directly first before calling into casgstatus, a much more heavyweight function. We have to be careful about synctest bubbling, but overall it's a good bit faster, and easy low-hanging fruit. goos: linux goarch: amd64 pkg: internal/runtime/cgobench cpu: AMD EPYC 7B13 │ after-2-2.out │ after-3.out │ │ sec/op │ sec/op vs base │ CgoCall-64 34.62n ± 1% 30.55n ± 1% -11.76% (p=0.002 n=6) Change-Id: Ic38620233b55f58b8a07510666aa18648373e2e7 Reviewed-on: https://go-review.googlesource.com/c/go/+/708596 Auto-Submit: Michael Knyszek LUCI-TryBot-Result: Go LUCI Reviewed-by: Michael Pratt --- diff --git a/src/runtime/proc.go b/src/runtime/proc.go index 53f64e8032..89080b48de 100644 --- a/src/runtime/proc.go +++ b/src/runtime/proc.go @@ -4622,7 +4622,12 @@ func reentersyscall(pc, sp, bp uintptr) { } // As soon as we switch to _Gsyscall, we are in danger of losing our P. // We must not touch it after this point. - casgstatus(gp, _Grunning, _Gsyscall) + // + // Try to do a quick CAS to avoid calling into casgstatus in the common case. + // If we have a bubble, we need to fall into casgstatus. + if gp.bubble != nil || !gp.atomicstatus.CompareAndSwap(_Grunning, _Gsyscall) { + casgstatus(gp, _Grunning, _Gsyscall) + } if staticLockRanking { // casgstatus clobbers gp.sched via systemstack under staticLockRanking. Restore it. save(pc, sp, bp) @@ -4825,7 +4830,12 @@ func exitsyscall() { // need to be held ahead of time. We're effectively atomic with respect to // the tracer because we're non-preemptible and in the runtime. It can't stop // us to read a bad status. - casgstatus(gp, _Gsyscall, _Grunning) + // + // Try to do a quick CAS to avoid calling into casgstatus in the common case. + // If we have a bubble, we need to fall into casgstatus. + if gp.bubble != nil || !gp.atomicstatus.CompareAndSwap(_Gsyscall, _Grunning) { + casgstatus(gp, _Gsyscall, _Grunning) + } // Caution: we're in a window where we may be in _Grunning without a P. // Either we will grab a P or call exitsyscall0, where we'll switch to