David Chase [Wed, 2 Jul 2025 22:00:12 +0000 (18:00 -0400)]
[dev.simd] cmd/compile: minor tweak for race detector
This makes the front-end a little bit less temp-happy
when instrumenting, which repairs the "is it a constant?"
test in the simd intrinsic conversion which is otherwise
broken by race detection.
Also, this will perhaps be better code.
Change-Id: I84b7a45b7bff62bb2c9f9662466b50858d288645
Reviewed-on: https://go-review.googlesource.com/c/go/+/685637
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Austin Clements [Mon, 30 Jun 2025 20:42:19 +0000 (16:42 -0400)]
[dev.simd] runtime: remove write barrier in xRegRestore
Currently, there's a write barrier in xRegRestore when it assigns
pp.xRegs.cache = gp.xRegs.state. This is bad because that gets called
on the asyncPreempt return path, where we have really limited stack
space, and we don't currently account for this write barrier.
We can't simply mark xRegState as sys.NotInHeap because it's also
embedded in runtime.p as register scratch space, and runtime.p is heap
allocated.
Hence, to fix this, we rename xRegState to just "xRegs" and introduce
a wrapper "xRegState" type that embeds xRegs and is itself marked
sys.NotInHeap. Then, anywhere we need a manually-managed pointer to
register state, we use the new type.
To ensure this doesn't happen again in the future, we also mark
asyncPreempt2 as go:nowritebarrierrec.
Change-Id: I5ff4841e55ff20047ff7d253ab659ab77aeb3391
Reviewed-on: https://go-review.googlesource.com/c/go/+/684836
Auto-Submit: Austin Clements <austin@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
[dev.simd] runtime: save scalar registers off stack in amd64 async preemption
Asynchronous preemption must save all registers that could be in use
by Go code. Currently, it saves all of these to the goroutine stack.
As a result, the stack frame requirements of asynchronous preemption
can be rather high. On amd64, this requires 368 bytes of stack space,
most of which is the XMM registers. Several RISC architectures are
around 0.5 KiB.
As we add support for SIMD instructions, this is going to become a
problem. The AVX-512 register state is 2.5 KiB. This well exceeds the
nosplit limit, and even if it didn't, could constrain when we can
asynchronously preempt goroutines on small stacks.
This CL fixes this by moving pure scalar state stored in non-GP
registers off the stack and into an allocated "extended register
state" object. To reduce space overhead, we only allocate these
objects as needed. While in the theoretical limit, every G could need
this register state, in practice very few do at a time.
However, we can't allocate when we're in the middle of saving the
register state during an asynchronous preemption, so we reserve
scratch space on every P to temporarily store the register state,
which can then be copied out to an allocated state object later by Go
code.
This commit only implements this for amd64, since that's where we're
about to add much more vector state, but it lays the groundwork for
doing this on any architecture that could benefit.
Change-Id: I123a95e21c11d5c10942d70e27f84d2d99bbf735
Reviewed-on: https://go-review.googlesource.com/c/go/+/680898 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Austin Clements <austin@google.com>
Michael Pratt [Fri, 27 Jun 2025 21:21:20 +0000 (17:21 -0400)]
runtime: stash allpSnapshot on the M
findRunnable takes a snapshot of allp prior to dropping the P because
afterwards procresize may mutate allp without synchronization.
procresize is careful to never mutate the contents up to cap(allp), so
findRunnable can still safely access the Ps in the slice.
Unfortunately, growing allp is problematic. If procresize grows the allp
backing array, it drops the reference to the old array. allpSnapshot
still refers to the old array, but allpSnapshot is on the system stack
in findRunnable, which also likely no longer has a P at all.
This means that a future GC will not find the reference and can free the
array and use it for another allocation. This would corrupt later reads
that findRunnable does from the array.
The fix is simple: the M struct itself is reachable by the GC, so we can
stash the snapshot in the M to ensure it is visible to the GC.
The ugliest part of the CL is the cleanup when we are done with the
snapshot because there are so many return/goto top sites. I am tempted
to put mp.clearAllpSnapshot() in the caller and at top to make this less
error prone, at the expensive of extra unnecessary writes.
Fixes #74414.
Change-Id: I6a6a636c484e4f4b34794fd07910b3fffeca830b
Reviewed-on: https://go-review.googlesource.com/c/go/+/684460 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Damien Neil [Fri, 27 Jun 2025 15:46:28 +0000 (08:46 -0700)]
sync: disassociate WaitGroups from bubbles on Wait
Fix a race condition in disassociating a WaitGroup in a synctest
bubble from its bubble. We previously disassociated the WaitGroup
when count becomes 0, but this causes problems when an Add call
setting count to 0 races with one incrementing the count.
Instead, disassociate a WaitGroup from its bubble when Wait returns.
Wait must not be called concurrently with an Add call with a
positive delta and a 0 count, so we know that the disassociation
will not race with an Add call trying to create a new association.
Fixes #74386
Change-Id: I9b519519921f7691869a64a245a5ee65d071d054
Reviewed-on: https://go-review.googlesource.com/c/go/+/684635 Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
David Chase [Sat, 28 Jun 2025 15:05:44 +0000 (11:05 -0400)]
[dev.simd] cmd/compile: reorder operands for some simd operations
This adds support for one ad hoc reordering, which
requires a new intrinsic-to-ssa helper matching the
name that is used in the generator (and this in the
generated code). In this case, it is opLen{2,3}Imm8_2I
which expects the immediate after the self (0) and
first (1) parameters to the method, and before the
mask if there is one. I.e., the immediate is arg 2
in the call.
The changes to simdintrinsics and stubs are generated
by simdgen CL 684019.
Change-Id: Ia54aab9825d469a2f3efa6d1fb079242181c0ca6
Reviewed-on: https://go-review.googlesource.com/c/go/+/684776 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
limeidan [Fri, 13 Jun 2025 03:48:44 +0000 (11:48 +0800)]
runtime: update skips for TestGdbBacktrace
We encountered a new type of "no such process" error on loong64, it's like this
"Couldn't get NT_PRSTATUS registers: No such process.", I checked the source code
of gdb, NT_PRSTATUS is not fixed, it may be another name, so I use regular
expression here to match possible cases.
Updates #50838
Fixes #74389
Change-Id: I3e3f7455b2dc6b8aa10c084f24f6a2a114790855
Reviewed-on: https://go-review.googlesource.com/c/go/+/684195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: abner chenc <chenguoqi@loongson.cn> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
qmuntal [Fri, 27 Jun 2025 10:45:22 +0000 (12:45 +0200)]
cmd/doc: fix -http on Windows
On Windows, GOMODCACHE almost never starts with a slash, and
"go doc -http" constructs a GOPROXY URL by doing "file://" + GOMODCACHE,
resulting in an invalid file URI.
For example, if GOMODCACHE is "C:\foo", then the file URI should be
"file:///C:/foo", but it becomes "file://C:/foo" instead, where "C:" is
understood as a host name, not a drive letter.
Fixes #74137.
Change-Id: I23e776e0f649a0062e01d1a4a6ea8268ba467331
Reviewed-on: https://go-review.googlesource.com/c/go/+/684575
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Michael Matloob <matloob@google.com>
Cherry Mui [Fri, 27 Jun 2025 23:45:22 +0000 (19:45 -0400)]
runtime: remove arbitrary 5-second timeout in TestNeedmDeadlock
The NeedmDeadlock test program currently has a 5-second timeout,
which is sort of arbitrary. It is long enough in regular mode
(which usually takes 0.0X seconds), but not quite so for
configurations like ASAN. Instead of using an arbitrary timeout,
just use the test's deadline. The test program is invoked with
testenv.Command, which will send it a SIGQUIT before the deadline
expires.
Fixes #56420 (at least for the asan builder).
Change-Id: I0b13651cb07241401837ca2e60eaa1b83275b093
Reviewed-on: https://go-review.googlesource.com/c/go/+/684697
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Joe Tsai [Fri, 27 Jun 2025 17:59:44 +0000 (10:59 -0700)]
reflect: fix TypeAssert on nil interface values
In the Go language a type assertion of a nil interface value
will always report false:
var err error
v, ok := err.(error) // always reports (nil, false)
Consequently, assertion on a reflect.Value.Interface()
will also report false:
var err error
rv := ValueOf(&err).Elem()
v, ok := rv.Interface().(error) // reports (nil, false)
However, prior to this change, a TypeAssert would report true:
var err error
rv := ValueOf(&err).Elem()
v, ok := TypeAssert[error](rv) // reports (nil, true)
when it should report false.
This fixes TypeAssert to match the Go language by
pushing the typ != v.typ check to the very end after
we have validated that neither v nor T are interface kinds.
Fixes #74404
Change-Id: Ie14d5cf18c8370c3e27ce4bdf4570c89519d8a16
Reviewed-on: https://go-review.googlesource.com/c/go/+/684675 Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Mateusz Poliwczak <mpoliwczak34@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
qmuntal [Fri, 27 Jun 2025 09:16:54 +0000 (11:16 +0200)]
os: use minimal file permissions when opening parent directory in RemoveAll
On Windows, the process might not have read permission on the parent
directory, but still can delete files in it. This change allows
RemoveAll to open the parent directory with minimal permissions, which
is sufficient for deleting child files.
Fixes #74134.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: I5d5c5977caaebf6e0f93fb2313b0ceb346f70e05
Reviewed-on: https://go-review.googlesource.com/c/go/+/684515 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
Roland Shoemaker [Thu, 26 Jun 2025 19:19:23 +0000 (12:19 -0700)]
encoding/json: add security section to doc
Add a section to the package doc which details the security
considerations of using encoding/json, in particular with respect to
parser misalignment issues.
Additionally, clarify previously ambiguous statement in the Unmarshal
doc about how case is used when matching keys in objects, and add a note
about how duplicate keys are handled.
Fixes #14750
Change-Id: I66f9b845efd98c86a684d7333b3aa8a456564922
Reviewed-on: https://go-review.googlesource.com/c/go/+/684315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Roland Shoemaker <roland@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
Michael Anthony Knyszek [Fri, 27 Jun 2025 00:59:49 +0000 (00:59 +0000)]
runtime: account for missing frame pointer in preamble
If a goroutine is synchronously preempted, then taking a
frame-pointer-based stack trace at that preemption will skip PC of the
caller of the function which called into morestack. This happens because
the frame pointer is pushed to the stack after the preamble, leaving the
stack in an odd state for frame pointer unwinding.
Deal with this by marking a goroutine as synchronously preempted and
using that signal to load the missing PC from the stack. On LR platforms
this is available in gp.sched.lr. On non-LR platforms like x86, it's at
gp.sched.sp, because there are no args, no locals, and no frame pointer
pushed to the SP yet.
For #68090.
Change-Id: I73a1206d8b84eecb8a96dbe727195da30088f288
Reviewed-on: https://go-review.googlesource.com/c/go/+/684435 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
Andy Nitschke [Tue, 10 Jun 2025 15:09:35 +0000 (11:09 -0400)]
net/http: fix RoundTrip context cancellation for js/wasm
The existing js/wasm implementation of RoundTrip calls abort() on the
fetch() call when the context is canceled but does not wait for for the
resulting promise to be rejected. The result is the failure callback for the
promise will be called at some later point in time when the promise
rejection is handled. In some case this callback may be called after the Go
program has exited resulting in "Go program has already exited" errors.
Fixes #57098
Change-Id: Ia37fd22cb9f667dbb0805ff5db0ceb8fdba7246b
Reviewed-on: https://go-review.googlesource.com/c/go/+/680937 Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Joe Tsai [Fri, 27 Jun 2025 01:18:32 +0000 (18:18 -0700)]
encoding/json: fix typo in hotlink for jsontext.PreserveRawStrings
Updates #71845
Change-Id: Ie099e7ac77293696fd9e69559487e27f4b70ab3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/684416
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Cherry Mui [Thu, 26 Jun 2025 19:46:31 +0000 (15:46 -0400)]
cmd/link: permit a larger size BSS reference to a smaller DATA symbol
Currently, if there is a BSS reference and a DATA symbol
definition with the same name, we pick the DATA symbol, as it
contains the right content. In this case, if the BSS reference
has a larger size, we error out, because it is not safe to access
a smaller symbol as if it has a larger size.
Sometimes code declares a global variable in Go and defines it
in assembly with content. They are expected to be of the same
size. However, in ASAN mode, we insert a red zone for the variable
on the Go side, making it have a larger size, whereas the assembly
symbol is unchanged. This causes the Go reference (BSS) has a
larger size than the assembly definition (DATA). It results in an
error currently. This code is valid and safe, so we should permit
that.
We support this case by increasing the symbol size to match the
larger size (of the BSS side). The symbol content (from the DATA
side) is kept. In some sense, we merge the two symbols. When
loading symbols, it is not easy to change its size (as the object
file may be mapped read-only), so we add it to a fixup list, and
fix it up later after all Go symbols are loaded. This is a very
rare case, so the list should not be long.
We could limit this to just ASAN mode. But it seems okay to allow
this in general. As long as the symbol has the larger size, it is
safe to access it with the larger size.
Fixes #74314.
Change-Id: I3ee6e46161d8f59500e2b81befea11e563355a57
Reviewed-on: https://go-review.googlesource.com/c/go/+/684236
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
thepudds [Wed, 25 Jun 2025 23:01:11 +0000 (19:01 -0400)]
cmd/compile/internal/escape: evaluate any side effects when rewriting with literals
CL 649035 and CL 649079 updated escape analysis to rewrite
certain operands in OMAKE and OCONVIFACE nodes from non-constant
expressions to basic literals that evaluate to the same value.
However, when doing that rewriting, we need to evaluate any
side effects prior to replacing the expression, which is what
this CL now does.
Issue #74379 reported a problem with OCONVIFACE nodes due to CL 649079.
CL 649035 has essentially the same issue with OMAKE nodes. To illustrate
that, we add a test for the OMAKE case in fixedbugs/issue74379b.go,
which fails without this change. To avoid introducing an unnecessary
temporary for OMAKE nodes, we also conditionalize the main work of
CL 649035 on whether the OMAKE operand is already an OLITERAL.
CL 649555 and CL 649078 were related changes that created read-only
global storage for composite literals used in an interface conversion.
This CL adds a test in fixedbugs/issue74379c.go to illustrate
that they do not have the same problem.
Updates #71359
Fixes #74379
Change-Id: I6645575ef34f1fe2b0241a22dc205875d66b7ada
Reviewed-on: https://go-review.googlesource.com/c/go/+/684116
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Keith Randall <khr@google.com>
David Chase [Tue, 24 Jun 2025 20:26:47 +0000 (16:26 -0400)]
[dev.simd] cmd/compile: support simd(imm,fp) returns gp
These changes are required to make gp-returning simd
ops work. amd64/ssa.go includes a new code generator
helper, gc/main.go initializes intrinsics AFTER
the types, ssa/_gen/*AMD64.go add another register
shape to the simd ops function.
This CL should be submitted after simdgen CL 683858
which generated some of the changes.
WARNING: This commit contains a breaking change.
This is permissible since jsontext is experimental and
not subject to the Go 1 compatibility agreement.
Existing callers of UnusedBuffer should use AvailableBuffer instead.
Updates #71497
Change-Id: Ib080caf306d545a8fb038e57f0817b18dd0f91cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/683897
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
both with methods that return a zero-length buffer that
is intended to only be used with a following Write call.
This keeps the older UnusedBuffer method around so that
at least one commit that has both methods for migration purposes.
Updates #71497
Change-Id: I3815f593e09f645280ae5ad9cbdd63a6c147123b
Reviewed-on: https://go-review.googlesource.com/c/go/+/683896 Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Cherry Mui [Wed, 25 Jun 2025 19:00:04 +0000 (15:00 -0400)]
[dev.simd] all: merge master (f8ccda2) into dev.simd
Merge List:
+ 2025-06-25 f8ccda2e05 runtime: make explicit nil check in (*spanInlineMarkBits).init
+ 2025-06-25 f069a82998 runtime: note custom GOMAXPROCS even if value doesn't change
+ 2025-06-24 e515ef8bc2 context: fix typo in context_test.go
+ 2025-06-24 47b941f445 cmd/link: add one more linkname to the blocklist
+ 2025-06-24 34cf5f6205 go/types: add test for interface method field type
+ 2025-06-24 6e618cd42a encoding/json: use zstd compressed testdata
+ 2025-06-24 fcb9850859 net/http: reduce allocs in CrossOriginProtection.Check
+ 2025-06-24 11f11f2a00 encoding/json/v2: support ISO 8601 durations
+ 2025-06-24 62deaf4fb8 doc: fix links to runtime Environment Variables
+ 2025-06-24 2e9bb62bfe encoding/json/v2: reject unquoted dash as a JSON field name
+ 2025-06-23 ed7815726d encoding/json/v2: report error on time.Duration without explicit format
+ 2025-06-23 f866958246 cmd/dist: test encoding/json/... with GOEXPERIMENT=jsonv2
+ 2025-06-23 f77a0aa6b6 internal/trace: improve gc-stress test
+ 2025-06-23 4506796a6e encoding/json/jsontext: consistently use JSON terminology
+ 2025-06-23 456a90aa16 runtime: add missing unlock in sysReserveAlignedSbrk
+ 2025-06-23 1cf6386b5e Revert "go/types, types2: don't register interface methods in Info.Types map"
+ 2025-06-20 49cdf0c42e testing, testing/synctest: handle T.Helper in synctest bubbles
+ 2025-06-20 3bf1eecbd3 runtime: fix struct comment
+ 2025-06-20 8ed23a2936 crypto/cipher: fix link to crypto/aes
+ 2025-06-20 ef60769b46 go/doc: add a golden test that reproduces #62640
+ 2025-06-18 8552bcf7c2 cmd/go/internal/fips140: ignore GOEXPERIMENT on error
+ 2025-06-18 4c7567290c runtime: set mspan limit field early and eagerly
+ 2025-06-18 c6ac736288 runtime: prevent mutual deadlock between GC stopTheWorld and suspendG
+ 2025-06-17 53af292aed encoding/json/jsontext: fix spelling error
+ 2025-06-16 d058254689 cmd/dist: always include variant in package names
+ 2025-06-16 3254c2bb83 internal/reflectlite: fix comment about meaning of flag field
+ 2025-06-16 816199e421 runtime: don't let readTrace spin on trace.shutdown
+ 2025-06-16 ea00461b17 internal/trace: make Value follow reflect conventions
+ 2025-06-13 96a6e147b2 runtime: comment that some linknames are used by runtime/trace
+ 2025-06-13 644905891f runtime: remove unused unique.runtime_blockUntilEmptyFinalizerQueue
+ 2025-06-13 683810a368 cmd/link: block new standard library linknames
+ 2025-06-12 9149876112 all: replace a few user-visible mentions of golang.org and godoc.org
+ 2025-06-12 934d5f2cf7 internal/trace: end test programs with SIGQUIT
+ 2025-06-12 5a08865de3 net: remove some BUG entries
+ 2025-06-11 d166a0b03e encoding/json/jsontext, encoding/json/v2: document experimental nature
+ 2025-06-11 d4c6effaa7 cmd/compile: add up-to-date test for generated files
+ 2025-06-10 7fa2c736b3 os: disallow Root.Remove(".") on Plan 9, js, and Windows
+ 2025-06-10 281cfcfc1b runtime: handle system goroutines later in goroutine profiling
+ 2025-06-10 4f86f22671 testing/synctest, runtime: avoid panic when using linker-alloc WG from bubble
Junyang Shao [Tue, 24 Jun 2025 03:59:30 +0000 (03:59 +0000)]
[dev.simd] cmd/compile: glue codes for Shift and Rotate
This CL adds two more intrinsic lowering functions. They can issue an
OpCopy to move a scalar value to vector value. This is needed by Shift
and Rotate APIs.
Change-Id: I8a83197d33207072c4a9221a931e67dddd5cd0bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/683476
Auto-Submit: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Michael Anthony Knyszek [Wed, 25 Jun 2025 15:47:05 +0000 (15:47 +0000)]
runtime: make explicit nil check in (*spanInlineMarkBits).init
The hugo binary gets slower, potentially dramatically so, with
GOEXPERIMENT=greenteagc. The root cause is page mapping churn. The Green
Tea code introduced a new implicit nil check on value in a
freshly-allocated span to clear some new heap metadata. This nil check
would read the fresh memory, causing Linux to back that virtual address
space with an RO page. This would then be almost immediately written to,
causing Linux to possibly flush the TLB and find memory to replace that
read-only page (likely deduplicated as just the zero page).
This CL fixes the issue by replacing the implicit nil check, which is a
memory read expected to fault if it's truly nil, with an explicit one.
The explicit nil check is a branch, and thus makes no reads to memory.
The result is that the hugo binary no longer gets slower.
No regression test because it doesn't seem possible without access to OS
internals, like Linux tracepoints. We briefly experimented with RSS
metrics, but they're inconsistent. Some system RSS metrics count the
deduplicated zero page, while others (like those produced by
/proc/self/smaps) do not.
Instead, we'll add a new benchmark to our benchmark suite, separately.
For #73581.
Fixes #74375.
Change-Id: I708321c14749a94ccff55072663012eba18b3b91
Reviewed-on: https://go-review.googlesource.com/c/go/+/684015 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com>
Michael Pratt [Tue, 24 Jun 2025 20:33:10 +0000 (16:33 -0400)]
runtime: note custom GOMAXPROCS even if value doesn't change
When an application calls runtime.GOMAXPROCS(runtime.GOMAXPROCS(0)), the
runtime does not need to change the actual GOMAXPROCS value (via STW).
However, this call must still transition from "automatic" to "custom"
GOMAXPROCS state, thus disabling background updates.
Thus this case shouldn't return quite as early as it currently does.
Change-Id: I6a6a636c42f73996532bd9f7beb95e933256c9e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/683815 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Joe Tsai [Mon, 4 Sep 2023 19:14:13 +0000 (12:14 -0700)]
encoding/json: use zstd compressed testdata
There is a non-public zstd decoder in the stdlib (CL 473356) and
also zstd compressed testdata already present.
Delete testdata/code.json.gz and
instead use internal/jsontest/testdata/golang_source.json.zst,
which has exactly the same content:
$ cat internal/jsontest/testdata/golang_source.json.zst | zstd -d | sha1sum 3f70b6fd429f4aba3e8e1c3e5a294c8f2e219a6e -
$ cat testdata/code.json.gz | zstd -d | sha1sum 3f70b6fd429f4aba3e8e1c3e5a294c8f2e219a6e -
This will reduce the size of the final Go release by 118KB.
Updates #71845
Change-Id: I6da2df27bd260befc0a44c6bc0255365be0a5b0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/525516
Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Bypass: Damien Neil <dneil@google.com>
Julien Cretel [Mon, 23 Jun 2025 16:19:19 +0000 (16:19 +0000)]
net/http: reduce allocs in CrossOriginProtection.Check
Rather than repeatedly creating error values on
CrossOriginProtection.Check's unhappy paths, return non-exported and
effectively constant error variables.
For #73626.
Change-Id: Ibaa036c29417071b3601b8d200ab0902359d1bb9
GitHub-Last-Rev: e704d63cd63665845d544796e802134ea608e217
GitHub-Pull-Request: golang/go#74251
Reviewed-on: https://go-review.googlesource.com/c/go/+/681178 Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: qiu laidongfeng2 <2645477756@qq.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Joe Tsai [Thu, 19 Jun 2025 01:35:36 +0000 (18:35 -0700)]
encoding/json/v2: support ISO 8601 durations
Based on the discussion in #71631, it is hotly contested
whether the default JSON representation for a Go time.Duration
should be the time.Duration.String format or
a particular profile of ISO 8601.
Regardless of the default, it seems clear that we should
at least support ISO 8601 if specified via a format flag.
Note that this CL does not alter the default representation.
Unfortunately, ISO 8601 is a large and evolving standard
with many optional extensions and optional restrictions.
Thus, the term "ISO 8601 duration" unfortunately does not
resolve to a particular grammar, nor one that is stable.
However, there is precedence that we can follow in this matter.
JSON finds its heritage in JavaScript and
JavaScript is adding a Temporal.Duration type whose default
JSON representation is ISO 8601.
There is a well-specified grammar for their particular
profile of ISO 8601, which is documented at:
https://tc39.es/proposal-temporal/#prod-Duration
This particular CL adds support for ISO 8601 according to
the exact same grammar that JavaScript uses.
While Temporal.Duration is technically still a proposal,
it is already in stage 3 of the TC39 proposal process
(i.e., "no changes to the proposal are expected"
and "has been recommended for implementation")
and therefore close to final adoption.
One major concern with ISO 8601 is that it supports
nominal date units like years, months, weeks, and days
that do not have an accurate meaning without being
anchored to a particular point in time and place on Earth.
Fortunately, JavaScript (by default) avoids producing
Temporal.Duration values with nominal units unless
arithmetic in JavaScript explicitly sets a largestUnits
value that is larger than "hours". In the Go implementation,
we support syntactically parsing the full ISO 8601 grammar
(according to JavaScript), but semantically report an error if
nominal units are present. This ensures that ISO 8601 durations
remain accurate so long as they only use the accurate units
of hours, minutes, or seconds.
Updates #71631
Change-Id: I983593662f2150461ebc486a5acfeb72f0286939
Reviewed-on: https://go-review.googlesource.com/c/go/+/682403 Reviewed-by: Daniel Martà <mvdan@mvdan.cc> Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
the concern was raised that whenever "-" is combined with other options,
the "-" is intepreted as as a name, rather than an ignored field,
which may go contrary to user expectation.
Static analysis demonstrates that there are ~2k instances of `json:"-,omitempty"
in the wild, where almost all of them intended for the field to be ignored.
To prevent this footgun, reject any tags that has "-," as a prefix
and warn the user to choose one of the reasonable alternatives.
The documentation of json/v2 already suggests `json:"'-'"`
as the recommended way to explicitly specify dash as the name.
See Example_fieldNames for example usages of the single-quoted literal.
Update the v1 json documentation to suggest the same thing.
Updates #71497
Change-Id: I7687b6eecdf82a5d894d057c78a4a90af4f5a6e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/683175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Daniel Martà <mvdan@mvdan.cc> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Joe Tsai [Tue, 17 Jun 2025 19:34:22 +0000 (12:34 -0700)]
encoding/json/v2: report error on time.Duration without explicit format
The default representation of a time.Duration is still undecided.
In order to keep the future open, report an error on a time.Duration
without an explicit format flag provided.
Updates #71631
Change-Id: I08248404ff6551723851417c8188a13f53c61937
Reviewed-on: https://go-review.googlesource.com/c/go/+/682455
Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
Joe Tsai [Mon, 23 Jun 2025 23:36:25 +0000 (16:36 -0700)]
cmd/dist: test encoding/json/... with GOEXPERIMENT=jsonv2
This also updates wasip1_wasm to use a 8MiB stack, which is
the same stack size as what is used by go_js_wasm_exec.
The increase of stack size is necessary because the jsonv2
tests exercise that the jsonv2 and jsontext packages support
a hard limit of a maximum JSON nesting depth of 10000.
However, even with a depth limit of 10000, this still exceeds
the previously specified maximum stack size of 1 MiB.
For use of JSON with untrusted inputs in WASM,
we really need to support #56733 as there is no right answer
for the default max depth limit to use since the max wasm
stack size is determined on a per-system basis.
Updates #71845
Change-Id: I3b32c58cc9f594a5c59bb3e4b20f5e86d85d8209
Reviewed-on: https://go-review.googlesource.com/c/go/+/683575 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Mon, 23 Jun 2025 20:14:18 +0000 (20:14 +0000)]
internal/trace: improve gc-stress test
The gc-stress test is useful for trying to exercise GC-related trace
events by producing a lot of them in many different situations.
Unfortunately this test is flaky, because allocating in a loop can
easily out-run the GC when it's trying to preempt the allocating
goroutine.
It's been a long standing problem that a program that allocates in a
loop can outrun a GC. The problem isn't the GC persay, it's consistently
correlated with a high STW time (likely a high 'stopping' time, not a
'stopped' time), suggesting that in the window of time when the garbage
collector is trying to stop all goroutines, they continue to allocate.
This should probably be fixed in general, but for now, let's focus on
this flaky test.
This CL changes the gc-stress test to (1) set a memory limit and (2) do
more work in between allocations. (2) is really what makes things less
flaky, but (2) unfortunately also means the GC is less exercised. That's
where (1) comes in. By setting a low memory limit, we increase GC
activity (in particular, assist activity). The memory limit also helps
prevent the heap from totally blowing up due to the heap goal inflating
from floating garbage, but it's not perfect.
After this change, under stress2, this test exceeds a heap size of 500
MiB only 1 in 5000 runs on my 64-vCPU VM. Before this change, it got
that big about 1/4th of the time.
Fixes #74052.
Change-Id: I49233c914c8b65b1d593d3953891fddda6685aec
Reviewed-on: https://go-review.googlesource.com/c/go/+/683515 Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Joe Tsai [Sat, 21 Jun 2025 17:51:08 +0000 (10:51 -0700)]
encoding/json/jsontext: consistently use JSON terminology
RFC 8259, section 2 uses the term "begin-array" amd "begin-object"
rather than "start array" or "start object".
Be consistent in our documentation.
Change-Id: I172eb354c5e64b84c74bd662b1e581424e719a32
Reviewed-on: https://go-review.googlesource.com/c/go/+/683155
Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Cherry Mui [Fri, 20 Jun 2025 20:28:14 +0000 (16:28 -0400)]
[dev.simd] cmd/compile: generate function body for bodyless intrinsics
For a compiler intrinsic, if it is used in a non-call context, e.g.
as a function pointer, currently it requires fallback
implementation (e.g. assembly code for atomic operations),
otherwise it will result in a build failure. The fallback
implementation needs to be maintained and tested, albeit rarely
used in practice.
Also, for SIMD, we're currently adding a large number of compiler
intrinsics without providing fallback implementations (we might in
the future). As methods, it is not unlikely that they are used in
a non-call context, e.g. referenced from the type descriptor.
This CL lets the compiler generate the function body for
bodyless intrinsics. The compiler already recognizes a call to
the function as an intrinsic and can directly generate code for it.
So we just fill in the body with a call to the same function.
Change-Id: I2636e3128f28301c9abaf2b48bc962ab56e7d1a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/683096 Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Cherry Mui [Fri, 20 Jun 2025 16:02:18 +0000 (12:02 -0400)]
[dev.simd] cmd/compile: don't treat devel compiler as a released compiler
The compiler has a logic to print different messages on internal
compiler error depending on whether this is a released version of
Go. It hides the panic stack trace if it is a released version. It
does this by checking the version and see if it has a "go" prefix.
This includes all the released versions. However, for a non-
released build, if there is no explicit version set, cmd/dist now
sets the toolchain version as go1.X-devel_XXX, which makes it be
treated as a released compiler, and causes the stack trace to be
hidden. Change the logic to not match a devel compiler as a
released compiler.
Change-Id: I5d3b2101527212f825b6e4000b36030c4f83870b
Reviewed-on: https://go-review.googlesource.com/c/go/+/682975 Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Pratt [Wed, 11 Jun 2025 20:46:21 +0000 (16:46 -0400)]
cmd/go/internal/fips140: ignore GOEXPERIMENT on error
During toolchain selection, the GOEXPERIMENT value may not be valid for
the current version (but it is valid for the selected version). In this
case, cfg.ExperimentErr is set and cfg.Experiment is nil.
Normally cmd/go main exits when ExperimentErr is set, so Experiment is
~never nil. But that is skipped during toolchain selection, and
fips140.Init is used during toolchain selection.
Fixes #74111.
Change-Id: I6a6a636c65ee5831feaf3d29993a60613bbec6f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/680976
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Wed, 18 Jun 2025 17:42:16 +0000 (17:42 +0000)]
runtime: set mspan limit field early and eagerly
Currently the mspan limit field is set after allocSpan returns, *after*
the span has already been published to the GC (including the
conservative scanner). But the limit field is load-bearing, because it's
checked to filter out invalid pointers. A stale limit value could cause
a crash by having the conservative scanner access allocBits out of
bounds.
Fix this by setting the mspan limit field before publishing the span.
For large objects and arena chunks, we adjust the limit down after
allocSpan because we don't have access to the true object's size from
allocSpan. However this is safe, since we first initialize the limit to
something definitely safe (the actual span bounds) and only adjust it
down after. Adjusting it down has the benefit of more precise debug
output, but the window in which it's imprecise is also fine because a
single object (logically, with arena chunks) occupies the whole span, so
the 'invalid' part of the memory will just safely point back to that
object. We can't do this for smaller objects because the limit will
include space that does *not* contain any valid objects.
Fixes #74288.
Change-Id: I0a22e5b9bccc1bfdf51d2b73ea7130f1b99c0c7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/682655 Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
Michael Anthony Knyszek [Sat, 14 Jun 2025 02:45:08 +0000 (02:45 +0000)]
runtime: prevent mutual deadlock between GC stopTheWorld and suspendG
Almost everywhere we stop the world we casGToWaitingForGC to prevent
mutual deadlock with the GC trying to scan our stack. This historically
was only necessary if we weren't stopping the world to change the GC
phase, because what we were worried about was mutual deadlock with mark
workers' use of suspendG. And, they were the only users of suspendG.
In Go 1.22 this changed. The execution tracer began using suspendG, too.
This leads to the possibility of mutual deadlock between the execution
tracer and a goroutine trying to start or end the GC mark phase. The fix
is simple: make the stop-the-world calls for the GC also call
casGToWaitingForGC. This way, suspendG is guaranteed to make progress in
this circumstance, and once it completes, the stop-the-world can
complete as well.
We can take this a step further, though, and move casGToWaitingForGC
into stopTheWorldWithSema, since there's no longer really a place we can
afford to skip this detail.
While we're here, rename casGToWaitingForGC to casGToWaitingForSuspendG,
since the GC is now not the only potential source of mutual deadlock.
Fixes #72740.
Change-Id: I5e3739a463ef3e8173ad33c531e696e46260692f
Reviewed-on: https://go-review.googlesource.com/c/go/+/681501 Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Michael Anthony Knyszek [Mon, 16 Jun 2025 16:31:10 +0000 (16:31 +0000)]
cmd/dist: always include variant in package names
Our attempt to evenly distribute tests across shards struggles a bit
because certain long-running targets are very difficult to distinguish
in ResultDB, namely racebench and the test directory tests. These are
the only tests where the JSON output from dist omits the variant from
the package, making it impossible to distinguish them in the test result
data. My current suspicion is that this is preventing the load balancing
from being effective for the race builders in particular, though I worry
the longtest builders have a similar situation with the test directory
tests.
For #65814.
Change-Id: I5804c2af092ff9aa4a3f0f6897b4a57c4628f837
Reviewed-on: https://go-review.googlesource.com/c/go/+/681955 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Michael Anthony Knyszek [Tue, 10 Jun 2025 21:44:56 +0000 (21:44 +0000)]
runtime: don't let readTrace spin on trace.shutdown
Issue #74045 describes a scenario in which gopark is inlined into
readTrace, such that there are no preemption points. This is only a
problem because readTrace spins if trace.shutdown is set, through
traceReaderAvailable. However, trace.shutdown is almost certainly
overkill for traceReaderAvailable. The first condition, checking whether
the reader gen and the flushed gen match, should be sufficient to ensure
the reader wakes up and finishes flushing all buffers. The first
condition is also safe because it guarantees progress. In the case of
shutdown, all the trace work that will be flushed has been flushed, and
so the trace reader will exit into a regular goroutine context when
it's finished. If not shutting down, then the trace reader will release
doneSema, increase readerGen, and then the gopark unlockf will let it
block until new work actually comes in.
Fixes #74045.
Change-Id: Id9b15c277cb731618488771bd484577341b68675
Reviewed-on: https://go-review.googlesource.com/c/go/+/680738
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Wed, 11 Jun 2025 21:35:29 +0000 (21:35 +0000)]
internal/trace: make Value follow reflect conventions
A previous change renamed Value.Uint64 to Value.ToUint64 to accomodate
string values. The method for a string value is then Value.ToString,
while the method for a debug string (for example, for fmt) is just
called String, as per fmt.Stringer.
This change follows a request from Dominik Honnef, maintainer of
gotraceui, to make Value follow the conventions of the reflect package.
The Value type there has a method String which fulfills both purposes:
getting the string for a String Value, and as fmt.Stringer. It's
not exactly pretty, but it does make sense to just stick to convention.
Change-Id: I55b364be88088d2121527bffc833ef03dbdb9764
Reviewed-on: https://go-review.googlesource.com/c/go/+/680978 Reviewed-by: Florian Lehner <lehner.florian86@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
David Chase [Tue, 10 Jun 2025 18:15:46 +0000 (14:15 -0400)]
cmd/compile: add up-to-date test for generated files
This runs the ssa/_gen generator writing files into
a temporary directory, and then checks that there are
no differences with what is currently in the ssa directory,
and also checks that any file with the "generated from
_gen/..." header was actually generated, and checks that
the headers on the generated file match the expected
header prefix.
Change-Id: Ic8eeb0b06cf6f2e576a013e865b331a12d3a77aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/680615
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
(cherry picked from commit d4c6effaa7b95a2ea149ece4a400c0ace2773839)
Reviewed-on: https://go-review.googlesource.com/c/go/+/680975
TryBot-Bypass: David Chase <drchase@google.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>