Keith Randall [Thu, 24 Apr 2025 15:38:56 +0000 (08:38 -0700)]
cmd/compile: add cast in range loop final value computation
When replacing a loop where the iteration variable has a named type,
we need to compute the last iteration value as i = T(len(a)-1), not
just i = len(a)-1.
Fixes #73491
Change-Id: Ic1cc3bdf8571a40c10060f929a9db8a888de2b70
Reviewed-on: https://go-review.googlesource.com/c/go/+/667815 Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@google.com> Reviewed-by: Junyang Shao <shaojunyang@google.com> Reviewed-by: Keith Randall <khr@google.com>
os,internal/poll: disassociate handle from IOCP in File.Fd
Go 1.25 will gain support for overlapped IO on handles passed to
os.NewFile thanks to CL 662236. It was previously not possible to add
an overlapped handle to the Go runtime's IO completion port (IOCP),
and now happens on the first call the an IO method.
This means that there is code that relies on the fact that File.Fd
returns a handle that can always be associated with a custom IOCP.
That wouldn't be the case anymore, as a handle can only be associated
with one IOCP at a time and it must be explicitly disassociated.
To fix this breaking change, File.Fd will disassociate the handle
from the Go runtime IOCP before returning it. It is then not necessary
to defer the association until the first IO method is called, which
was recently added in CL 661955 to support this same use case, but
in a more complex and unreliable way.
Updates #19098.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race,gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: Id8a7e04d35057047c61d1733bad5bf45494b2c28
Reviewed-on: https://go-review.googlesource.com/c/go/+/664455 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Keith Randall [Wed, 23 Apr 2025 21:15:51 +0000 (14:15 -0700)]
runtime: use precise bounds of Go data/bss for race detector
We only want to call into the race detector for Go global variables.
By rounding up the region bounds, we can include some C globals.
Even worse, we can include only *part* of a C global, leading to
race{read,write}range calls which straddle the end of shadow memory.
That causes the race detector to barf.
Fix some off-by-one errors in the assembly comparisons. We want to
skip calling the race detector when addr == racedataend.
Fixes #73483
Change-Id: I436b0f588d6165b61f30cb7653016ba9b7cbf585
Reviewed-on: https://go-review.googlesource.com/c/go/+/667655 Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Keith Randall [Tue, 15 Apr 2025 21:55:06 +0000 (14:55 -0700)]
runtime: align taggable pointers more so we can use low bits for tag
Currently we assume alignment to 8 bytes, so we can steal the low 3 bits.
This CL assumes alignment to 512 bytes, so we can steal the low 9 bits.
That's 6 extra bits!
Aligning to 512 bytes wastes a bit of space but it is not egregious.
Most of the objects that we make tagged pointers to are pretty big.
Update #49405
Change-Id: I66fc7784ac1be5f12f285de1d7851d5a6871fb75
Reviewed-on: https://go-review.googlesource.com/c/go/+/665815 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Alan Donovan [Wed, 23 Apr 2025 18:23:45 +0000 (14:23 -0400)]
cmd/vet: add hostport analyzer
+ test, release note
Fixes #28308
Change-Id: I190e2fe513eeb6b90b0398841f67bf52510b5f59
Reviewed-on: https://go-review.googlesource.com/c/go/+/667596
Auto-Submit: Alan Donovan <adonovan@google.com>
Commit-Queue: Alan Donovan <adonovan@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Tue, 4 Mar 2025 19:02:48 +0000 (19:02 +0000)]
runtime: move sizeclass defs to new package internal/runtime/gc
We will want to reference these definitions from new generator programs,
and this is a good opportunity to cleanup all these old C-style names.
Change-Id: Ifb06f0afc381e2697e7877f038eca786610c96de
Reviewed-on: https://go-review.googlesource.com/c/go/+/655275
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Reason for revert: cockroachdb benchmark failing. Likely due to CL 564197.
For #73474
Change-Id: Id5d83cd8bb8fe9ee7fddb8dc01f1a01f2d40154e
Reviewed-on: https://go-review.googlesource.com/c/go/+/667336
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Rhys Hiltner [Fri, 14 Mar 2025 19:38:34 +0000 (12:38 -0700)]
runtime: commit to spinbitmutex GOEXPERIMENT
Use the "spinbit" mutex implementation always (including on platforms
that need to emulate atomic.Xchg8), and delete the prior "tristate"
implementations.
The exception is GOARCH=wasm, where the Go runtime does not use multiple
threads.
For #68578
Change-Id: Ifc29bbfa05071d776c23a19ae185891a03a82417
Reviewed-on: https://go-review.googlesource.com/c/go/+/658456
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com> Reviewed-by: Junyang Shao <shaojunyang@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Ian Lance Taylor [Thu, 15 Feb 2024 01:54:00 +0000 (17:54 -0800)]
runtime: only poll network from one P at a time in findRunnable
For #65064
Change-Id: Ifecd7e332d2cf251750752743befeda4ed396f33
Reviewed-on: https://go-review.googlesource.com/c/go/+/564197
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Artur M. Wolff <artur.m.wolff@gmail.com> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Keith Randall [Wed, 19 Mar 2025 17:17:22 +0000 (10:17 -0700)]
cmd/compile: don't evaluate side effects of range over array
If the thing we're ranging over is an array or ptr to array, and
it doesn't have a function call or channel receive in it, then we
shouldn't evaluate it.
Typecheck the ranged-over value as a constant in that case.
That makes the unified exporter replace the range expression
with a constant int.
Change-Id: I0d4ea081de70d20cf6d1fa8d25ef6cb021975554
Reviewed-on: https://go-review.googlesource.com/c/go/+/659317 Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Robert Griesemer <gri@google.com>
thepudds [Wed, 12 Mar 2025 18:45:17 +0000 (14:45 -0400)]
cmd/compile/internal/escape: avoid reading ir.Node during inner loop of walkOne
Broadly speaking, escape analysis has two main phases. First, it
traverses the AST while building a data-flow graph of locations and
edges. Second, during "solve", it repeatedly walks the data-flow graph
while carefully propagating information about each location, including
whether a location's address reaches the heap.
Once escape analysis is in the solve phase and repeatedly walking the
data-flow graph, almost all the information it needs is within the
location graph, with a notable exception being the ir.Class of an
ir.Name, which currently must be checked by following a pointer from
the location to its ir.Node.
For typical graphs, that does not matter much, but if the graph becomes
large enough, cache misses in the inner solve loop start to matter more,
and the class is checked many times in the inner loop.
We therefore store the class information on the location in the graph
to reduce how much memory we need to load in the inner loop.
The package github.com/microsoft/typescript-go/internal/checker
has many locations, and compilation currently spends most of its time
in escape analysis.
This CL gives roughly a 30% speedup for wall clock compilation time
for the checker package:
go1.24.0: 91.79s
this CL: 64.98s
Linux perf shows a healthy reduction for example in l2_request.miss and
dTLB-load-misses on an amd64 test VM.
We could tweak things a bit more, though initial review feedback
has suggested it would be good to get this in as it stands.
Subsequent CLs in this stack give larger improvements.
Updates #72815
Change-Id: I3117430dff684c99e6da1e0d7763869873379238
Reviewed-on: https://go-review.googlesource.com/c/go/+/657295
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Jake Bailey <jacob.b.bailey@gmail.com> Reviewed-by: David Chase <drchase@google.com>
Change-Id: Ib68b934276ec08246d4ae60ef9870c233f0eac69
Reviewed-on: https://go-review.googlesource.com/c/go/+/665595
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Joel Sing <joel@sing.id.au> Reviewed-by: Roland Shoemaker <roland@golang.org>
Joel Sing [Sat, 19 Apr 2025 05:57:37 +0000 (15:57 +1000)]
crypto/internal/fips140/aes: actually use the VTBL instruction on arm64
Support for the VTBL instruction was added in CL 110015 - use it
directly, rather than using WORD encodings. Note that one of the
WORD encodings does not actually match the instruction in the
comment - use the instruction that matches the existing encoding
instead.
Change-Id: I1933162f8144a6b86b38e8b550d36907131b1dd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/666795 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
math/big: use clearer loop bounds check elimination
Checking that the lengths are equal and panicking teaches the compiler
that it can assume “i in range for z” implies “i in range for x”, letting us
simplify the actual loops a bit.
It also turns up a few places in math/big that were playing maybe a little
too fast and loose with slice lengths. Update those to explicitly set all the
input slices to the same length.
These speedups are basically irrelevant, since they only happen
in real code if people are compiling with -tags math_big_pure_go.
But at least the code is clearer.
The arith assembly is big enough, and the details that you have to keep
in mind are complex enough and varied enough, that it is worth using
a Go program to generate the assembly. That way, all the architectures
can use the same algorithms, and porting to new architectures will be
easier.
This is the first of a sequence of CLs to introduce a new mini-compiler
for generating the arith assembly, in math/big/internal/asmgen.
This CL has the basics of the compiler as well as a couple simple
architectures and the generator for addVV/subVV. It does not check
in the generated assembly yet. That will happen in a followup CL after
the other architectures and generators have been added.
Change-Id: Ib704c60fd972fc5690ac04d8fae3712ee2c1a80a
Reviewed-on: https://go-review.googlesource.com/c/go/+/664935
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
math/big: replace addVW/subVW assembly with fast pure Go
The vast majority of the time, carry propagation is limited and
addVW/subVW only need to consider a single word for carry propagation.
As Josh Bleecher-Snyder pointed out in 2019 (CL 164968), once carrying
is done, the remaining words can be handled faster with copy (memmove).
In the benchmarks below, this is the data=random case.
Even more important, if the source and destination are the same,
the copy can be optimized away entirely, making a small in-place
addition to a big.Int O(1) instead of O(N). To date, only a few
systems (amd64, arm64, and pure Go, meaning wasm) make use of this
asymptotic improvement. This is the data=shortcut case.
This CL deletes the addVW/subVW assembly and replaces it with
an optimized pure Go version. Using Go makes it easy to call
the real copy builtin, which will use optimized memmove code,
instead of recreating a worse memmove in assembly (as arm64 does)
or omitting the copy optimization entirely (as most others do).
The worst case for the Go version versus assembly is the case
of incrementing 2^N-1 by 1, which has to propagate a carry
the entire length of the array. This is the data=carry case.
On balance, we believe this case is rare enough to be worth
taking a hit in that case, in exchange for significant wins
in the other cases and the deletion of significant amounts of
assembly of varying quality. (Remember that half the assembly has
the copy optimization and shortcut, while half does not.)
In the benchmarks, the systems are:
c2s16 GOARCH=amd64 c2s16 perf gomote (Intel, Google Cloud)
c3h88 GOARCH=amd64 c3h88 perf gomote (newer Intel, Google Cloud)
s7 GOARCH=amd64 rsc basement server (AMD Ryzen 9 7950X)
c4as16 GOARCH=arm64 c4as16 perf gomote (Google Cloud)
mac GOARCH=arm64 Apple M3 Pro in MacBook Pro
386 GOARCH=386 gotip-linux-386 gomote
arm GOARCH=arm gotip-linux-arm gomote
loong64 GOARCH=loong64 gotip-linux-loong64 gomote
ppc64le GOARCH=ppc64le gotip-linux-ppc64le gomote
riscv64 GOARCH=riscv64 gotip-linux-riscv64 gomote
benchmark \ system c2s16 c3h88 s7 c4as16 mac 386 arm loong64 ppc64le riscv64
Russ Cox [Wed, 8 Jan 2025 16:27:07 +0000 (11:27 -0500)]
regexp/syntax: recognize category aliases like \p{Letter}
The Unicode specification defines aliases for some of the general
category names. For example the category "L" has alias "Letter".
The regexp package supports \p{L} but not \p{Letter}, because there
was nothing in the Unicode tables that lets regexp know about Letter.
Now that package unicode provides CategoryAliases (see #70780),
we can use it to provide \p{Letter} as well.
This is the only feature missing from making package regexp suitable
for use in a JSON-API Schema implementation. (The official test suite
includes usage of aliases like \p{Letter} instead of \p{L}.)
For better conformity with Unicode TR18, also accept case-insensitive
matches for names and ignore underscores, hyphens, and spaces;
and add Any, ASCII, and Assigned.
Fixes #70781.
Change-Id: I50ff024d99255338fa8d92663881acb47f1e92a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/641377
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Russ Cox [Wed, 8 Jan 2025 16:21:30 +0000 (11:21 -0500)]
unicode: add CategoryAliases, Cn, LC
CategoryAliases is for regexp to use, for things like \p{Letter} as an alias for \p{L}.
Cn and LC are special-case categories that were never implemented
but should have been.
These changes were generated by the updated generator in CL 641395.
Fixes #70780.
Change-Id: Ibba20ff76191c8ae9631ac5ba19965790fe0cc81
Reviewed-on: https://go-review.googlesource.com/c/go/+/641376
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Michael Pratt [Fri, 18 Apr 2025 10:23:33 +0000 (06:23 -0400)]
internal/runtime/maps: move tombstone test to swiss file
This test fails on GOEXPERIMENT=noswissmap as it is testing behavior
specific to swissmaps. Move it to map_swiss_test.go to skip it on
noswissmap.
We could also switch the test to use NewTestMap, which provides a
swissmap even in GOEXPERIMENT=noswissmap, but that is tedious to use and
noswissmap is going away soon anyway.
For #70886.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-noswissmap
Change-Id: I6a6a636c5ec72217d936cd01e9da36ae127ea2c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/666437 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
When GOEXPERIMENT=jsonv2 is set, the encoding/json/v2 and
encoding/jsontext packages are visible, the encoding/json
package is implemented in terms of encoding/json/v2, and
the encoding/json package include various additional APIs.
(See #71497 for details.)
When GOEXPERIMENT=jsonv2 is not set, the new API is not
present and the encoding/json package is unchanged.
The experimental API is not bound by the Go compatibility
promise and is expected to evolve as updates are made to
the json/v2 proposal.
The contents of encoding/json/internal/jsontest/testdata
are compressed with zstd v1.5.7 with the -19 option.
Fixes #71845
For #71497
Change-Id: Ib8c94e5f0586b6aaa22833190b41cf6ef59f4f01
Reviewed-on: https://go-review.googlesource.com/c/go/+/665796
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TestFileFdsAreInitialised and TestSerialFdsAreInitialised were added
to ensure handles passed to os.NewFile were not added to the runtime
poller. This used to be problematic because the poller could crash
if an external I/O event was received (see #21172).
This is not an issue anymore since CL 482495 and #19098.
Change-Id: I292ceae27724fefe6f438a398ebfe351dd5231d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/665315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Damien Neil <dneil@google.com>
Keith Randall [Tue, 7 Jan 2025 05:53:22 +0000 (21:53 -0800)]
internal/runtime/maps: prune tombstones in maps before growing
Before growing, if there are lots of tombstones try to remove them.
If we can remove enough, we can continue at the given size for a
while longer.
Fixes #70886
Change-Id: I71e0d873ae118bb35798314ec25e78eaa5340d73
Reviewed-on: https://go-review.googlesource.com/c/go/+/640955 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Philip Roberts [Thu, 9 Mar 2023 11:16:56 +0000 (11:16 +0000)]
database/sql: wake cleaner if maxIdleTime set to less than maxLifetime
The existing implementation wouldn't wake the connection cleaner if
maxIdleTime was set to a value less than maxLifetime while an existing
connection was open - resulting in idle connections not being discarded
until after the first maxLifetime had passed.
Fixes #45993
Change-Id: I074ed7ba9803354c8b3a41f2625ae0d8a7d5059b
GitHub-Last-Rev: 0d149d8d38bc9c2ad42a2a20dcfc73994d54fe23
GitHub-Pull-Request: golang/go#58490
Reviewed-on: https://go-review.googlesource.com/c/go/+/467655
Auto-Submit: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Daniel Theophanes <kardianos@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Sean Liao [Sun, 13 Apr 2025 16:48:48 +0000 (17:48 +0100)]
crypto/rand: add and update examples
Change-Id: I77406c22b82c9f8bc57323c783f63c4897486e7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/665096 Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Roland Shoemaker <roland@golang.org>
Eric Young [Fri, 3 Jun 2022 02:22:50 +0000 (02:22 +0000)]
crypto/tls: fix a testing deadlock that occurs on a TLS protocol error
A Go routine was, on an error, returning without sending a message on its
signaling channel, so the main program was blocking forever waiting for
a message that was never sent. Found while breaking crypto/tls.
Change-Id: Id0b3c070a27cabd852f74e86bb9eff5c66b86d28
GitHub-Last-Rev: 4d84fb8b556589ec98eba6142a553fbd45683b96
GitHub-Pull-Request: golang/go#53216
Reviewed-on: https://go-review.googlesource.com/c/go/+/410274
Auto-Submit: Sean Liao <sean@liao.dev> Reviewed-by: Roland Shoemaker <roland@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
cmd/compile/internal/escape: add hash for bisecting stack allocation of variable-sized makeslice
CL 653856 enabled stack allocation of variable-sized makeslice results.
This CL adds debug hashing of that change, plus a debug flag
to control the byte threshold used.
The debug hashing machinery means we also now have a way to disable just
the CL 653856 optimization by doing -gcflags='all=-d=variablemakehash=n'
or similar, though the stderr output will then typically have many
lines of debug hash output.
Using this CL plus the bisect command, I was able to retroactively
find one of the lines of code responsible for #73199:
$ bisect -compile=variablemake go test -skip TestListWireGuardDrivers
[...]
bisect: FOUND failing change set
--- change set #1 (enabling changes causes failure)
./security_windows.go:1321:38 (variablemake)
./security_windows.go:1321:38 (variablemake)
---
Previously, I had tracked down those lines by diffing '-gcflags=-m=1'
output and brief code inspection, but seeing the bisect was very nice.
This CL also adds a compiler debug flag to control the threshold for
stack allocation of variably sized make results. This can help
us identify more code that is relying on certain stack allocations.
This might be a temporary flag that we delete prior to Go 1.25
(given we would not want people to rely on it), or maybe it
might make sense to keep it for some period of time beyond the release
of Go 1.25 to help the ecosystem shake out other bugs.
Using these two flags together (and picking a threshold of 64 rather
than the default of 32), it looks for example like this
x/sys/windows code might be relying on stack allocation of
a byte slice:
$ bisect -compile=variablemake go test -gcflags=-d=variablemakethreshold=64 -skip TestListWireGuardDrivers
[...]
bisect: FOUND failing change set
--- change set #1 (enabling changes causes failure)
./syscall_windows_test.go:1178:16 (variablemake)
Updates #73199
Fixes #73253
Change-Id: I160179a0e3c148c3ea86be5c9b6cea8a52c3e5b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/663795
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Constantin Konstantinidis [Sun, 13 Apr 2025 14:21:52 +0000 (16:21 +0200)]
strings: duplicate alignment test from bytes package
Fixes #26129
Change-Id: If98f85b458990dbff7ecfeaea6c81699dafa66ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/665275 Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Keith Randall [Tue, 15 Apr 2025 16:36:06 +0000 (09:36 -0700)]
math/big: fix loong64 assembly for vet
Vet is failing on this code because some arguments of mulAddVWW
got renamed in the go decl (CL 664895) but not the assembly accessors.
Looks like the assembly got written before that CL but checked in
after that CL.
Change-Id: I270e8db5f8327aa2029c21a126fab1231a3506a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/665717 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Damien Neil [Fri, 7 Mar 2025 01:33:27 +0000 (17:33 -0800)]
net/http: test intended behavior in TestClientInsecureTransport
This test wasn't testing the HTTP/2 case, because it didn't
set NextProtos in the tls.Config.
Set "Connection: close" on requests to make sure each request
gets a new connection.
Change-Id: I1ef470e7433a602ce88da7bd7eeec502687ea857
Reviewed-on: https://go-review.googlesource.com/c/go/+/655676
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Damien Neil [Tue, 15 Apr 2025 18:17:03 +0000 (11:17 -0700)]
cmd/go/internal/imports: remove test dependency on json internals
TestScan loads encoding/json and verifies that various imports
match expectations. The new v2 encoding/json violates these
expectations. Since this test is testing the ScanDir function,
not encoding/json, change it to use a test package with defined
imports instead.
Change-Id: I68a0813ccf37daadbd6ea52872a8ac132141e82a
Reviewed-on: https://go-review.googlesource.com/c/go/+/665795 Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Bypass: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Damien Neil [Thu, 10 Apr 2025 20:58:17 +0000 (13:58 -0700)]
os: handle trailing slashes in os.RemoveDir on Windows
CL 661575 inadvertently caused os.RemoveDir on Windows to
fail when given a path with a trailing / or \, due to the
splitPath function not correctly stripping trailing
separators.
Fixes #73317
Change-Id: I21977b94bb08ff1e563de6f5f16a4bdf5024a15e
Reviewed-on: https://go-review.googlesource.com/c/go/+/664715
Auto-Submit: Damien Neil <dneil@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alan Donovan <adonovan@google.com>
Before CL, all instances of gQueue and gList stored the size of
structures in a separate variable. The size changed manually and passed
as a separate argument to different functions. This CL added an
additional field to gQueue and gList structures to store the size. Also,
the calculation of size was moved into the implementation of API for
these structures. This allows to reduce possible errors by eliminating
manual calculation of the size and simplifying functions' signatures.
Change-Id: I087da2dfaec4925e4254ad40fce5ccb4c175ec41
Reviewed-on: https://go-review.googlesource.com/c/go/+/664777
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Achille Roussel [Fri, 22 Dec 2023 00:54:24 +0000 (16:54 -0800)]
iter: reduce memory footprint of iter.Pull functions
The implementation of iter.Pull and iter.Pull2 functions is based on
closures and sharing local state, which results in one heap allocation
for each captured variable.
The number of heap allocations can be reduced by grouping the state
shared between closures in a struct, allowing the compiler to allocate
all local variables in a single heap region instead of creating
individual heap objects for each variable.
This approach can sometimes have downsides when it couples unrelated
objects in a single memory region, preventing the garbage collector from
reclaiming unused memory. While technically only a subset of the local
state is shared between the next and stop functions, it seems unlikely
that retaining the rest of the state until stop is reclaimed would be
problematic in practice, since the two closures would often have very
similar lifetimes.
The change also reduces the total memory footprint due to alignment
rules, the two booleans can be packed in memory and sometimes can even
exist within the padding space of the v value. There is also less
metadata needed for the garbage collector to track each individual heap
allocation.
goos: darwin
goarch: arm64
pkg: iter
cpu: Apple M2 Pro
│ /tmp/bench.old │ /tmp/bench.new │
│ sec/op │ sec/op vs base │
Pull-12 218.6n ± 7% 146.1n ± 0% -33.19% (p=0.000 n=10)
Pull2-12 239.8n ± 5% 155.0n ± 5% -35.36% (p=0.000 n=10)
geomean 229.0n 150.5n -34.28%
internal/cpu: add a detection for Neoverse(N3, V3, V3ae) cores
The memmove implementation relies on the variable
runtime.arm64UseAlignedLoads to select fastest code
path. Considering Neoverse N3, V3 and V3ae cores
prefer aligned loads, this patch adds code to detect
them for memmove performance.
Change-Id: I7266fc35d8b2c15ff516c592b987bafacb82b620
Reviewed-on: https://go-review.googlesource.com/c/go/+/664038 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com>
James Tucker [Fri, 11 Apr 2025 21:41:00 +0000 (21:41 +0000)]
internal/poll: disable SIO_UDP_NETRESET on Windows
Disable the reception of NET_UNREACHABLE (TTL expired) message reporting
on UDP sockets to match the default behavior of sockets on other
plaforms.
See https://learn.microsoft.com/en-us/windows/win32/winsock/winsock-ioctls#sio_udp_netreset
This is similar to, but a different case from the prior change 3114bd6 /
https://golang.org/issue/5834 that disabled one of the two flags
influencing behavior in response to the reception of related ICMP.
Updates #5834
Updates #68614
Change-Id: I39bc77ab68f5edfc14514d78870ff4a24c0f645e
GitHub-Last-Rev: 78f073bac226aeca438b64acc2c66f76c25f29f8
GitHub-Pull-Request: golang/go#68615
Reviewed-on: https://go-review.googlesource.com/c/go/+/601397
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
os,internal/poll: support I/O on overlapped files not added to the poller
This fixes the support for I/O on overlapped files that are not added to
the poller. Note that CL 661795 already added support for that, but it
really only worked for pipes, not for plain files.
Additionally, this CL also makes this kind of I/O operations to not
notify the external poller to avoid confusing it.
Updates #15388.
Change-Id: I15c6ea74f3a87960aef0986598077b6eab9b9c99
Reviewed-on: https://go-review.googlesource.com/c/go/+/664415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
Change-Id: Id1c3fbfed049d9a665f05f7c1af84e9fbd45fddf
Reviewed-on: https://go-review.googlesource.com/c/go/+/663395 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Meidan Li <limeidan@loongson.cn>
Than McIntosh [Sat, 5 Apr 2025 22:59:59 +0000 (18:59 -0400)]
cmd: fix DWARF gen bug with packages that use assembly
When the compiler builds a Go package with DWARF 5 generation enabled,
it emits relocations into various generated DWARF symbols (ex:
SDWARFFCN) that use the R_DWTXTADDR_* flavor of relocations. The
specific size of this relocation is selected based on the total number
of functions in the package -- if the package is tiny (just a couple
funcs) we can use R_DWTXTADDR_U1 relocs (which target just a byte); if
the package is larger we might need to use the 2-byte or 3-byte flavor
of this reloc.
Prior to this patch, the strategy used to pick the right relocation
size was flawed in that it didn't take into account packages with
assembly code. For example, if you have a package P with 200 funcs
written in Go source and 200 funcs written in assembly, you can't use
the R_DWTXTADDR_U1 reloc flavor for indirect text references since the
real function count for the package (asm + go) exceeds 255.
The new strategy (with this patch) is to have the compiler look at the
"symabis" file to determine the count of assembly functions. For the
assembler, rather than create additional plumbing to pass in the Go
source func count we just use an dummy (artificially high) function
count so as to select a relocation that will be large enough.
Fixes #72810.
Updates #26379.
Change-Id: I98d04f3c6aacca1dafe1f1610c99c77db290d1d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/663235 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
TestRootChtimes currently fails on illumos [1] because the times
returned by os.Stat have only microsecond precision on that builder.
Truncate them to make the test pass again.
Russ Cox [Tue, 4 Mar 2025 15:31:02 +0000 (10:31 -0500)]
runtime: handle m0 padding better
The SpinbitMutex experiment requires m structs other than m0
to be allocated in 2048-byte size class, by adding padding.
Do the calculation more explicitly, to avoid future CLs like CL 653335.
Change-Id: I83ae1e86ef3711ab65441f4e487f94b9e1429029
Reviewed-on: https://go-review.googlesource.com/c/go/+/654595 Reviewed-by: Rhys Hiltner <rhys.hiltner@gmail.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
It is annoying that non-x86 implementations of shlVU and shrVU
have to go out of their way to handle the trivial case shift==0
with their own copy loops. Instead, arrange to never call them
with shift==0, so that the code can be removed.
Unfortunately, there are linknames of shlVU, so we cannot
change that function. But we can rename the functions and
then leave behind a shlVU wrapper, so do that.
Since the big.Int API calls the operations Lsh and Rsh, rename
shlVU/shrVU to lshVU/rshVU. Also rename various other shl/shr
methods and functions to lsh/rsh.
Change-Id: Ieaf54e0110a298730aa3e4566ce5be57ba7fc121
Reviewed-on: https://go-review.googlesource.com/c/go/+/664896 Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
addMulVVW is an unnecessarily special case.
All other assembly routines taking []Word (V as in vector) arguments
take separate source and destination. For example:
addVV: z = x+y
mulAddVWW: z = x*m+a
addMulVVW uses the z parameter as both destination and source:
addMulVVW: z = z+x*m
Even looking at the signatures is confusing: all the VV routines take
two input vectors x and y, but addMulVVW takes only x: where is y?
(The answer is that the two inputs are z and x.)
It would be nice to fix this, both for understandability and regularity,
and to simplify a future assembly generator.
We cannot remove or redefine addMulVVW, because it has been used
in linknames. Instead, the CL adds a new final addend argument ‘a’
like in mulAddVWW, making the natural name addMulVVWW
(two input vectors, two input words):
addMulVVWW: z = x+y*m+a
This CL updates all the assembly implementations to rename the
inputs z, x, y -> x, y, m, and then introduces a separate destination z.
Change-Id: Ib76c80b53f6d1f4a901f663566e9c4764bb20488
Reviewed-on: https://go-review.googlesource.com/c/go/+/664895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Sam Thanawalla [Tue, 29 Oct 2024 14:22:11 +0000 (14:22 +0000)]
cmd/go: add subdirectory support to go-import meta tag
This CL adds ability to specify a subdirectory in the go-import meta tag.
A go-import meta tag now will support:
<meta name="go-import" content="root-path vcs repo-url subdir">
Fixes: #34055
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: Iedac520f97e0646254cc1bd2f97d5a9a5236829b
Reviewed-on: https://go-review.googlesource.com/c/go/+/625577 Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
The sendfile implementation for platforms supporting it is now in
net/sendfile.go, rather than being duplicated in separate files for
each platform.
The only difference between the implementations was the poll.SendFile
parameters, which have been harmonized, and also linux strictly
asserting for os.File, which now have been relaxed to allow any
type implementing syscall.Conn.
Change-Id: Ia1a2d5ee7380710a36fc555dbf681f7e996ea2ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/664075 Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
Windows sendfile optimization is skipped since CL 472475, which started
passing an os.fileWithoutWriteTo instead of an os.File to sendfile,
and that function was only implemented for os.File.
This CL fixes the issue by asserting against an interface rather than
a concrete type.
Some tests have been reenabled, triggering bugs in poll.SendFile which
have been fixed in this CL.
Fixes #67042.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: Id6f7a0e1e0f34a72216fa9d00c5bf36f5a994219
Reviewed-on: https://go-review.googlesource.com/c/go/+/664055 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
runtime: explicitly exclude a potential deadlock in the scheduler
The following sequence in the scheduler may potentially lead to
deadlock:
- globrunqget() -> runqput() -> runqputslow() -> globrunqputbatch()
However, according to the current logic of the scheduler it is not
possible to face the deadlock.
The patch explicitly excludes the deadlock, even though it is impossible
situation at the moment.
Additionally, the "runq" and "globrunq" APIs were partially refactored,
which allowed to minimize the usage of these APIs by each other.
This will prevent situations described in the CL.
Change-Id: I7318f935d285b95522998e0903eaa6193af2ba48
Reviewed-on: https://go-review.googlesource.com/c/go/+/662216 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
qiulaidongfeng [Wed, 23 Oct 2024 14:45:58 +0000 (22:45 +0800)]
cmd/go: only print GOCACHE value in env -changed if it's not the default
When other environment variables are set to default values,
we will not print it in go env -changed,
GOCACHE should do the same.
For #69994
Change-Id: I16661803cf1f56dd132b4db1c2d5cb4823fc0e58
Reviewed-on: https://go-review.googlesource.com/c/go/+/621997 Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
cmd/go/internal/modcmd: remove documentation for auto-converting legacy formats
CL 518776 dropped the ability of 'go mod init' to convert
legacy pre-module dependency configuration files, such as automatically
transforming a Gopkg.lock to a go.mod file with similar requirements,
but some of the documentation remained.
In this CL, we remove it from the cmd/go documentation.
(CL 662675 is a companion change that removes it from the Modules
Reference page).
Updates #71537
Change-Id: Ieccc64c811c4c25a657c00e42f7362a32b5fd661
Reviewed-on: https://go-review.googlesource.com/c/go/+/662695 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev>
Jakob Ackermann [Mon, 3 Mar 2025 22:16:38 +0000 (22:16 +0000)]
net/http: reduce memory usage when hijacking
Previously, Hijack allocated a new write buffer and the existing
connection write buffer used an extra 4KiB of memory until the handler
finished and the "conn" was garbage collected. Now, hijack re-uses the
existing write buffer and re-attaches it to the raw connection to avoid
referencing the net/http "conn" after returning.
After a handler that hijacked exited, the "conn" reference in
"connReader" will now be unset. This allows all of the "conn",
"response" and "Request" to get garbage collected.
Overall, this is reducing the memory usage by 43% or 6.7KiB per hijacked
connection (see BenchmarkServerHijackMemoryUsage in an earlier revision
of the CL).
CloseNotify will continue to work _before_ the handler has exited
(i.e. while the "conn" is still referenced in "connReader"). This aligns
with the documentation of CloseNotifier:
> After the Handler has returned, there is no guarantee that the channel
> receives a value.
goos: linux
goarch: amd64
pkg: net/http
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
│ before │ after │
│ sec/op │ sec/op vs base │
ServerHijack-8 42.59µ ± 8% 39.47µ ± 16% ~ (p=0.481 n=10)
│ before │ after │
│ B/op │ B/op vs base │
ServerHijack-8 16.12Ki ± 0% 12.06Ki ± 0% -25.16% (p=0.000 n=10)
│ before │ after │
│ allocs/op │ allocs/op vs base │
ServerHijack-8 51.00 ± 0% 49.00 ± 0% -3.92% (p=0.000 n=10)
Change-Id: I20a37ee314ed0d47463a4657d712154e78e48138
GitHub-Last-Rev: 80f09dfa273035f53cdd72845e5c5fb129c3e230
GitHub-Pull-Request: golang/go#70756
Reviewed-on: https://go-review.googlesource.com/c/go/+/634855 Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
There is a potential race between a concurrent call to FD.initIO, which
calls FD.pd.init, and a call to FD.Close, which calls FD.pd.evict.
This is solved by calling FD.initIO in FD.Close, as that will block
until the concurrent FD.initIO has completed. Note that FD.initIO is
no-op if first called from here.
The race window is so small that it is not possible to write a test
that triggers it.
Change-Id: Ie2f2818e746b9d626fe3b9eb6b8ff967c81ef863
Reviewed-on: https://go-review.googlesource.com/c/go/+/663815 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>