Change-Id: Ib68b934276ec08246d4ae60ef9870c233f0eac69
Reviewed-on: https://go-review.googlesource.com/c/go/+/665595
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Joel Sing <joel@sing.id.au> Reviewed-by: Roland Shoemaker <roland@golang.org>
Joel Sing [Sat, 19 Apr 2025 05:57:37 +0000 (15:57 +1000)]
crypto/internal/fips140/aes: actually use the VTBL instruction on arm64
Support for the VTBL instruction was added in CL 110015 - use it
directly, rather than using WORD encodings. Note that one of the
WORD encodings does not actually match the instruction in the
comment - use the instruction that matches the existing encoding
instead.
Change-Id: I1933162f8144a6b86b38e8b550d36907131b1dd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/666795 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
math/big: use clearer loop bounds check elimination
Checking that the lengths are equal and panicking teaches the compiler
that it can assume “i in range for z” implies “i in range for x”, letting us
simplify the actual loops a bit.
It also turns up a few places in math/big that were playing maybe a little
too fast and loose with slice lengths. Update those to explicitly set all the
input slices to the same length.
These speedups are basically irrelevant, since they only happen
in real code if people are compiling with -tags math_big_pure_go.
But at least the code is clearer.
The arith assembly is big enough, and the details that you have to keep
in mind are complex enough and varied enough, that it is worth using
a Go program to generate the assembly. That way, all the architectures
can use the same algorithms, and porting to new architectures will be
easier.
This is the first of a sequence of CLs to introduce a new mini-compiler
for generating the arith assembly, in math/big/internal/asmgen.
This CL has the basics of the compiler as well as a couple simple
architectures and the generator for addVV/subVV. It does not check
in the generated assembly yet. That will happen in a followup CL after
the other architectures and generators have been added.
Change-Id: Ib704c60fd972fc5690ac04d8fae3712ee2c1a80a
Reviewed-on: https://go-review.googlesource.com/c/go/+/664935
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
math/big: replace addVW/subVW assembly with fast pure Go
The vast majority of the time, carry propagation is limited and
addVW/subVW only need to consider a single word for carry propagation.
As Josh Bleecher-Snyder pointed out in 2019 (CL 164968), once carrying
is done, the remaining words can be handled faster with copy (memmove).
In the benchmarks below, this is the data=random case.
Even more important, if the source and destination are the same,
the copy can be optimized away entirely, making a small in-place
addition to a big.Int O(1) instead of O(N). To date, only a few
systems (amd64, arm64, and pure Go, meaning wasm) make use of this
asymptotic improvement. This is the data=shortcut case.
This CL deletes the addVW/subVW assembly and replaces it with
an optimized pure Go version. Using Go makes it easy to call
the real copy builtin, which will use optimized memmove code,
instead of recreating a worse memmove in assembly (as arm64 does)
or omitting the copy optimization entirely (as most others do).
The worst case for the Go version versus assembly is the case
of incrementing 2^N-1 by 1, which has to propagate a carry
the entire length of the array. This is the data=carry case.
On balance, we believe this case is rare enough to be worth
taking a hit in that case, in exchange for significant wins
in the other cases and the deletion of significant amounts of
assembly of varying quality. (Remember that half the assembly has
the copy optimization and shortcut, while half does not.)
In the benchmarks, the systems are:
c2s16 GOARCH=amd64 c2s16 perf gomote (Intel, Google Cloud)
c3h88 GOARCH=amd64 c3h88 perf gomote (newer Intel, Google Cloud)
s7 GOARCH=amd64 rsc basement server (AMD Ryzen 9 7950X)
c4as16 GOARCH=arm64 c4as16 perf gomote (Google Cloud)
mac GOARCH=arm64 Apple M3 Pro in MacBook Pro
386 GOARCH=386 gotip-linux-386 gomote
arm GOARCH=arm gotip-linux-arm gomote
loong64 GOARCH=loong64 gotip-linux-loong64 gomote
ppc64le GOARCH=ppc64le gotip-linux-ppc64le gomote
riscv64 GOARCH=riscv64 gotip-linux-riscv64 gomote
benchmark \ system c2s16 c3h88 s7 c4as16 mac 386 arm loong64 ppc64le riscv64
Russ Cox [Wed, 8 Jan 2025 16:27:07 +0000 (11:27 -0500)]
regexp/syntax: recognize category aliases like \p{Letter}
The Unicode specification defines aliases for some of the general
category names. For example the category "L" has alias "Letter".
The regexp package supports \p{L} but not \p{Letter}, because there
was nothing in the Unicode tables that lets regexp know about Letter.
Now that package unicode provides CategoryAliases (see #70780),
we can use it to provide \p{Letter} as well.
This is the only feature missing from making package regexp suitable
for use in a JSON-API Schema implementation. (The official test suite
includes usage of aliases like \p{Letter} instead of \p{L}.)
For better conformity with Unicode TR18, also accept case-insensitive
matches for names and ignore underscores, hyphens, and spaces;
and add Any, ASCII, and Assigned.
Fixes #70781.
Change-Id: I50ff024d99255338fa8d92663881acb47f1e92a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/641377
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Russ Cox [Wed, 8 Jan 2025 16:21:30 +0000 (11:21 -0500)]
unicode: add CategoryAliases, Cn, LC
CategoryAliases is for regexp to use, for things like \p{Letter} as an alias for \p{L}.
Cn and LC are special-case categories that were never implemented
but should have been.
These changes were generated by the updated generator in CL 641395.
Fixes #70780.
Change-Id: Ibba20ff76191c8ae9631ac5ba19965790fe0cc81
Reviewed-on: https://go-review.googlesource.com/c/go/+/641376
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Michael Pratt [Fri, 18 Apr 2025 10:23:33 +0000 (06:23 -0400)]
internal/runtime/maps: move tombstone test to swiss file
This test fails on GOEXPERIMENT=noswissmap as it is testing behavior
specific to swissmaps. Move it to map_swiss_test.go to skip it on
noswissmap.
We could also switch the test to use NewTestMap, which provides a
swissmap even in GOEXPERIMENT=noswissmap, but that is tedious to use and
noswissmap is going away soon anyway.
For #70886.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-noswissmap
Change-Id: I6a6a636c5ec72217d936cd01e9da36ae127ea2c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/666437 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
When GOEXPERIMENT=jsonv2 is set, the encoding/json/v2 and
encoding/jsontext packages are visible, the encoding/json
package is implemented in terms of encoding/json/v2, and
the encoding/json package include various additional APIs.
(See #71497 for details.)
When GOEXPERIMENT=jsonv2 is not set, the new API is not
present and the encoding/json package is unchanged.
The experimental API is not bound by the Go compatibility
promise and is expected to evolve as updates are made to
the json/v2 proposal.
The contents of encoding/json/internal/jsontest/testdata
are compressed with zstd v1.5.7 with the -19 option.
Fixes #71845
For #71497
Change-Id: Ib8c94e5f0586b6aaa22833190b41cf6ef59f4f01
Reviewed-on: https://go-review.googlesource.com/c/go/+/665796
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TestFileFdsAreInitialised and TestSerialFdsAreInitialised were added
to ensure handles passed to os.NewFile were not added to the runtime
poller. This used to be problematic because the poller could crash
if an external I/O event was received (see #21172).
This is not an issue anymore since CL 482495 and #19098.
Change-Id: I292ceae27724fefe6f438a398ebfe351dd5231d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/665315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Damien Neil <dneil@google.com>
Keith Randall [Tue, 7 Jan 2025 05:53:22 +0000 (21:53 -0800)]
internal/runtime/maps: prune tombstones in maps before growing
Before growing, if there are lots of tombstones try to remove them.
If we can remove enough, we can continue at the given size for a
while longer.
Fixes #70886
Change-Id: I71e0d873ae118bb35798314ec25e78eaa5340d73
Reviewed-on: https://go-review.googlesource.com/c/go/+/640955 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Philip Roberts [Thu, 9 Mar 2023 11:16:56 +0000 (11:16 +0000)]
database/sql: wake cleaner if maxIdleTime set to less than maxLifetime
The existing implementation wouldn't wake the connection cleaner if
maxIdleTime was set to a value less than maxLifetime while an existing
connection was open - resulting in idle connections not being discarded
until after the first maxLifetime had passed.
Fixes #45993
Change-Id: I074ed7ba9803354c8b3a41f2625ae0d8a7d5059b
GitHub-Last-Rev: 0d149d8d38bc9c2ad42a2a20dcfc73994d54fe23
GitHub-Pull-Request: golang/go#58490
Reviewed-on: https://go-review.googlesource.com/c/go/+/467655
Auto-Submit: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Daniel Theophanes <kardianos@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Sean Liao [Sun, 13 Apr 2025 16:48:48 +0000 (17:48 +0100)]
crypto/rand: add and update examples
Change-Id: I77406c22b82c9f8bc57323c783f63c4897486e7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/665096 Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Roland Shoemaker <roland@golang.org>
Eric Young [Fri, 3 Jun 2022 02:22:50 +0000 (02:22 +0000)]
crypto/tls: fix a testing deadlock that occurs on a TLS protocol error
A Go routine was, on an error, returning without sending a message on its
signaling channel, so the main program was blocking forever waiting for
a message that was never sent. Found while breaking crypto/tls.
Change-Id: Id0b3c070a27cabd852f74e86bb9eff5c66b86d28
GitHub-Last-Rev: 4d84fb8b556589ec98eba6142a553fbd45683b96
GitHub-Pull-Request: golang/go#53216
Reviewed-on: https://go-review.googlesource.com/c/go/+/410274
Auto-Submit: Sean Liao <sean@liao.dev> Reviewed-by: Roland Shoemaker <roland@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
cmd/compile/internal/escape: add hash for bisecting stack allocation of variable-sized makeslice
CL 653856 enabled stack allocation of variable-sized makeslice results.
This CL adds debug hashing of that change, plus a debug flag
to control the byte threshold used.
The debug hashing machinery means we also now have a way to disable just
the CL 653856 optimization by doing -gcflags='all=-d=variablemakehash=n'
or similar, though the stderr output will then typically have many
lines of debug hash output.
Using this CL plus the bisect command, I was able to retroactively
find one of the lines of code responsible for #73199:
$ bisect -compile=variablemake go test -skip TestListWireGuardDrivers
[...]
bisect: FOUND failing change set
--- change set #1 (enabling changes causes failure)
./security_windows.go:1321:38 (variablemake)
./security_windows.go:1321:38 (variablemake)
---
Previously, I had tracked down those lines by diffing '-gcflags=-m=1'
output and brief code inspection, but seeing the bisect was very nice.
This CL also adds a compiler debug flag to control the threshold for
stack allocation of variably sized make results. This can help
us identify more code that is relying on certain stack allocations.
This might be a temporary flag that we delete prior to Go 1.25
(given we would not want people to rely on it), or maybe it
might make sense to keep it for some period of time beyond the release
of Go 1.25 to help the ecosystem shake out other bugs.
Using these two flags together (and picking a threshold of 64 rather
than the default of 32), it looks for example like this
x/sys/windows code might be relying on stack allocation of
a byte slice:
$ bisect -compile=variablemake go test -gcflags=-d=variablemakethreshold=64 -skip TestListWireGuardDrivers
[...]
bisect: FOUND failing change set
--- change set #1 (enabling changes causes failure)
./syscall_windows_test.go:1178:16 (variablemake)
Updates #73199
Fixes #73253
Change-Id: I160179a0e3c148c3ea86be5c9b6cea8a52c3e5b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/663795
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Constantin Konstantinidis [Sun, 13 Apr 2025 14:21:52 +0000 (16:21 +0200)]
strings: duplicate alignment test from bytes package
Fixes #26129
Change-Id: If98f85b458990dbff7ecfeaea6c81699dafa66ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/665275 Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Keith Randall [Tue, 15 Apr 2025 16:36:06 +0000 (09:36 -0700)]
math/big: fix loong64 assembly for vet
Vet is failing on this code because some arguments of mulAddVWW
got renamed in the go decl (CL 664895) but not the assembly accessors.
Looks like the assembly got written before that CL but checked in
after that CL.
Change-Id: I270e8db5f8327aa2029c21a126fab1231a3506a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/665717 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Damien Neil [Fri, 7 Mar 2025 01:33:27 +0000 (17:33 -0800)]
net/http: test intended behavior in TestClientInsecureTransport
This test wasn't testing the HTTP/2 case, because it didn't
set NextProtos in the tls.Config.
Set "Connection: close" on requests to make sure each request
gets a new connection.
Change-Id: I1ef470e7433a602ce88da7bd7eeec502687ea857
Reviewed-on: https://go-review.googlesource.com/c/go/+/655676
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Damien Neil [Tue, 15 Apr 2025 18:17:03 +0000 (11:17 -0700)]
cmd/go/internal/imports: remove test dependency on json internals
TestScan loads encoding/json and verifies that various imports
match expectations. The new v2 encoding/json violates these
expectations. Since this test is testing the ScanDir function,
not encoding/json, change it to use a test package with defined
imports instead.
Change-Id: I68a0813ccf37daadbd6ea52872a8ac132141e82a
Reviewed-on: https://go-review.googlesource.com/c/go/+/665795 Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Bypass: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Damien Neil [Thu, 10 Apr 2025 20:58:17 +0000 (13:58 -0700)]
os: handle trailing slashes in os.RemoveDir on Windows
CL 661575 inadvertently caused os.RemoveDir on Windows to
fail when given a path with a trailing / or \, due to the
splitPath function not correctly stripping trailing
separators.
Fixes #73317
Change-Id: I21977b94bb08ff1e563de6f5f16a4bdf5024a15e
Reviewed-on: https://go-review.googlesource.com/c/go/+/664715
Auto-Submit: Damien Neil <dneil@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alan Donovan <adonovan@google.com>
Before CL, all instances of gQueue and gList stored the size of
structures in a separate variable. The size changed manually and passed
as a separate argument to different functions. This CL added an
additional field to gQueue and gList structures to store the size. Also,
the calculation of size was moved into the implementation of API for
these structures. This allows to reduce possible errors by eliminating
manual calculation of the size and simplifying functions' signatures.
Change-Id: I087da2dfaec4925e4254ad40fce5ccb4c175ec41
Reviewed-on: https://go-review.googlesource.com/c/go/+/664777
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Achille Roussel [Fri, 22 Dec 2023 00:54:24 +0000 (16:54 -0800)]
iter: reduce memory footprint of iter.Pull functions
The implementation of iter.Pull and iter.Pull2 functions is based on
closures and sharing local state, which results in one heap allocation
for each captured variable.
The number of heap allocations can be reduced by grouping the state
shared between closures in a struct, allowing the compiler to allocate
all local variables in a single heap region instead of creating
individual heap objects for each variable.
This approach can sometimes have downsides when it couples unrelated
objects in a single memory region, preventing the garbage collector from
reclaiming unused memory. While technically only a subset of the local
state is shared between the next and stop functions, it seems unlikely
that retaining the rest of the state until stop is reclaimed would be
problematic in practice, since the two closures would often have very
similar lifetimes.
The change also reduces the total memory footprint due to alignment
rules, the two booleans can be packed in memory and sometimes can even
exist within the padding space of the v value. There is also less
metadata needed for the garbage collector to track each individual heap
allocation.
goos: darwin
goarch: arm64
pkg: iter
cpu: Apple M2 Pro
│ /tmp/bench.old │ /tmp/bench.new │
│ sec/op │ sec/op vs base │
Pull-12 218.6n ± 7% 146.1n ± 0% -33.19% (p=0.000 n=10)
Pull2-12 239.8n ± 5% 155.0n ± 5% -35.36% (p=0.000 n=10)
geomean 229.0n 150.5n -34.28%
internal/cpu: add a detection for Neoverse(N3, V3, V3ae) cores
The memmove implementation relies on the variable
runtime.arm64UseAlignedLoads to select fastest code
path. Considering Neoverse N3, V3 and V3ae cores
prefer aligned loads, this patch adds code to detect
them for memmove performance.
Change-Id: I7266fc35d8b2c15ff516c592b987bafacb82b620
Reviewed-on: https://go-review.googlesource.com/c/go/+/664038 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com>
James Tucker [Fri, 11 Apr 2025 21:41:00 +0000 (21:41 +0000)]
internal/poll: disable SIO_UDP_NETRESET on Windows
Disable the reception of NET_UNREACHABLE (TTL expired) message reporting
on UDP sockets to match the default behavior of sockets on other
plaforms.
See https://learn.microsoft.com/en-us/windows/win32/winsock/winsock-ioctls#sio_udp_netreset
This is similar to, but a different case from the prior change 3114bd6 /
https://golang.org/issue/5834 that disabled one of the two flags
influencing behavior in response to the reception of related ICMP.
Updates #5834
Updates #68614
Change-Id: I39bc77ab68f5edfc14514d78870ff4a24c0f645e
GitHub-Last-Rev: 78f073bac226aeca438b64acc2c66f76c25f29f8
GitHub-Pull-Request: golang/go#68615
Reviewed-on: https://go-review.googlesource.com/c/go/+/601397
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
os,internal/poll: support I/O on overlapped files not added to the poller
This fixes the support for I/O on overlapped files that are not added to
the poller. Note that CL 661795 already added support for that, but it
really only worked for pipes, not for plain files.
Additionally, this CL also makes this kind of I/O operations to not
notify the external poller to avoid confusing it.
Updates #15388.
Change-Id: I15c6ea74f3a87960aef0986598077b6eab9b9c99
Reviewed-on: https://go-review.googlesource.com/c/go/+/664415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
Change-Id: Id1c3fbfed049d9a665f05f7c1af84e9fbd45fddf
Reviewed-on: https://go-review.googlesource.com/c/go/+/663395 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Meidan Li <limeidan@loongson.cn>
Than McIntosh [Sat, 5 Apr 2025 22:59:59 +0000 (18:59 -0400)]
cmd: fix DWARF gen bug with packages that use assembly
When the compiler builds a Go package with DWARF 5 generation enabled,
it emits relocations into various generated DWARF symbols (ex:
SDWARFFCN) that use the R_DWTXTADDR_* flavor of relocations. The
specific size of this relocation is selected based on the total number
of functions in the package -- if the package is tiny (just a couple
funcs) we can use R_DWTXTADDR_U1 relocs (which target just a byte); if
the package is larger we might need to use the 2-byte or 3-byte flavor
of this reloc.
Prior to this patch, the strategy used to pick the right relocation
size was flawed in that it didn't take into account packages with
assembly code. For example, if you have a package P with 200 funcs
written in Go source and 200 funcs written in assembly, you can't use
the R_DWTXTADDR_U1 reloc flavor for indirect text references since the
real function count for the package (asm + go) exceeds 255.
The new strategy (with this patch) is to have the compiler look at the
"symabis" file to determine the count of assembly functions. For the
assembler, rather than create additional plumbing to pass in the Go
source func count we just use an dummy (artificially high) function
count so as to select a relocation that will be large enough.
Fixes #72810.
Updates #26379.
Change-Id: I98d04f3c6aacca1dafe1f1610c99c77db290d1d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/663235 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
TestRootChtimes currently fails on illumos [1] because the times
returned by os.Stat have only microsecond precision on that builder.
Truncate them to make the test pass again.
Russ Cox [Tue, 4 Mar 2025 15:31:02 +0000 (10:31 -0500)]
runtime: handle m0 padding better
The SpinbitMutex experiment requires m structs other than m0
to be allocated in 2048-byte size class, by adding padding.
Do the calculation more explicitly, to avoid future CLs like CL 653335.
Change-Id: I83ae1e86ef3711ab65441f4e487f94b9e1429029
Reviewed-on: https://go-review.googlesource.com/c/go/+/654595 Reviewed-by: Rhys Hiltner <rhys.hiltner@gmail.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
It is annoying that non-x86 implementations of shlVU and shrVU
have to go out of their way to handle the trivial case shift==0
with their own copy loops. Instead, arrange to never call them
with shift==0, so that the code can be removed.
Unfortunately, there are linknames of shlVU, so we cannot
change that function. But we can rename the functions and
then leave behind a shlVU wrapper, so do that.
Since the big.Int API calls the operations Lsh and Rsh, rename
shlVU/shrVU to lshVU/rshVU. Also rename various other shl/shr
methods and functions to lsh/rsh.
Change-Id: Ieaf54e0110a298730aa3e4566ce5be57ba7fc121
Reviewed-on: https://go-review.googlesource.com/c/go/+/664896 Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
addMulVVW is an unnecessarily special case.
All other assembly routines taking []Word (V as in vector) arguments
take separate source and destination. For example:
addVV: z = x+y
mulAddVWW: z = x*m+a
addMulVVW uses the z parameter as both destination and source:
addMulVVW: z = z+x*m
Even looking at the signatures is confusing: all the VV routines take
two input vectors x and y, but addMulVVW takes only x: where is y?
(The answer is that the two inputs are z and x.)
It would be nice to fix this, both for understandability and regularity,
and to simplify a future assembly generator.
We cannot remove or redefine addMulVVW, because it has been used
in linknames. Instead, the CL adds a new final addend argument ‘a’
like in mulAddVWW, making the natural name addMulVVWW
(two input vectors, two input words):
addMulVVWW: z = x+y*m+a
This CL updates all the assembly implementations to rename the
inputs z, x, y -> x, y, m, and then introduces a separate destination z.
Change-Id: Ib76c80b53f6d1f4a901f663566e9c4764bb20488
Reviewed-on: https://go-review.googlesource.com/c/go/+/664895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Sam Thanawalla [Tue, 29 Oct 2024 14:22:11 +0000 (14:22 +0000)]
cmd/go: add subdirectory support to go-import meta tag
This CL adds ability to specify a subdirectory in the go-import meta tag.
A go-import meta tag now will support:
<meta name="go-import" content="root-path vcs repo-url subdir">
Fixes: #34055
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: Iedac520f97e0646254cc1bd2f97d5a9a5236829b
Reviewed-on: https://go-review.googlesource.com/c/go/+/625577 Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
The sendfile implementation for platforms supporting it is now in
net/sendfile.go, rather than being duplicated in separate files for
each platform.
The only difference between the implementations was the poll.SendFile
parameters, which have been harmonized, and also linux strictly
asserting for os.File, which now have been relaxed to allow any
type implementing syscall.Conn.
Change-Id: Ia1a2d5ee7380710a36fc555dbf681f7e996ea2ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/664075 Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
Windows sendfile optimization is skipped since CL 472475, which started
passing an os.fileWithoutWriteTo instead of an os.File to sendfile,
and that function was only implemented for os.File.
This CL fixes the issue by asserting against an interface rather than
a concrete type.
Some tests have been reenabled, triggering bugs in poll.SendFile which
have been fixed in this CL.
Fixes #67042.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: Id6f7a0e1e0f34a72216fa9d00c5bf36f5a994219
Reviewed-on: https://go-review.googlesource.com/c/go/+/664055 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
runtime: explicitly exclude a potential deadlock in the scheduler
The following sequence in the scheduler may potentially lead to
deadlock:
- globrunqget() -> runqput() -> runqputslow() -> globrunqputbatch()
However, according to the current logic of the scheduler it is not
possible to face the deadlock.
The patch explicitly excludes the deadlock, even though it is impossible
situation at the moment.
Additionally, the "runq" and "globrunq" APIs were partially refactored,
which allowed to minimize the usage of these APIs by each other.
This will prevent situations described in the CL.
Change-Id: I7318f935d285b95522998e0903eaa6193af2ba48
Reviewed-on: https://go-review.googlesource.com/c/go/+/662216 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
qiulaidongfeng [Wed, 23 Oct 2024 14:45:58 +0000 (22:45 +0800)]
cmd/go: only print GOCACHE value in env -changed if it's not the default
When other environment variables are set to default values,
we will not print it in go env -changed,
GOCACHE should do the same.
For #69994
Change-Id: I16661803cf1f56dd132b4db1c2d5cb4823fc0e58
Reviewed-on: https://go-review.googlesource.com/c/go/+/621997 Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
cmd/go/internal/modcmd: remove documentation for auto-converting legacy formats
CL 518776 dropped the ability of 'go mod init' to convert
legacy pre-module dependency configuration files, such as automatically
transforming a Gopkg.lock to a go.mod file with similar requirements,
but some of the documentation remained.
In this CL, we remove it from the cmd/go documentation.
(CL 662675 is a companion change that removes it from the Modules
Reference page).
Updates #71537
Change-Id: Ieccc64c811c4c25a657c00e42f7362a32b5fd661
Reviewed-on: https://go-review.googlesource.com/c/go/+/662695 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev>
Jakob Ackermann [Mon, 3 Mar 2025 22:16:38 +0000 (22:16 +0000)]
net/http: reduce memory usage when hijacking
Previously, Hijack allocated a new write buffer and the existing
connection write buffer used an extra 4KiB of memory until the handler
finished and the "conn" was garbage collected. Now, hijack re-uses the
existing write buffer and re-attaches it to the raw connection to avoid
referencing the net/http "conn" after returning.
After a handler that hijacked exited, the "conn" reference in
"connReader" will now be unset. This allows all of the "conn",
"response" and "Request" to get garbage collected.
Overall, this is reducing the memory usage by 43% or 6.7KiB per hijacked
connection (see BenchmarkServerHijackMemoryUsage in an earlier revision
of the CL).
CloseNotify will continue to work _before_ the handler has exited
(i.e. while the "conn" is still referenced in "connReader"). This aligns
with the documentation of CloseNotifier:
> After the Handler has returned, there is no guarantee that the channel
> receives a value.
goos: linux
goarch: amd64
pkg: net/http
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
│ before │ after │
│ sec/op │ sec/op vs base │
ServerHijack-8 42.59µ ± 8% 39.47µ ± 16% ~ (p=0.481 n=10)
│ before │ after │
│ B/op │ B/op vs base │
ServerHijack-8 16.12Ki ± 0% 12.06Ki ± 0% -25.16% (p=0.000 n=10)
│ before │ after │
│ allocs/op │ allocs/op vs base │
ServerHijack-8 51.00 ± 0% 49.00 ± 0% -3.92% (p=0.000 n=10)
Change-Id: I20a37ee314ed0d47463a4657d712154e78e48138
GitHub-Last-Rev: 80f09dfa273035f53cdd72845e5c5fb129c3e230
GitHub-Pull-Request: golang/go#70756
Reviewed-on: https://go-review.googlesource.com/c/go/+/634855 Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
There is a potential race between a concurrent call to FD.initIO, which
calls FD.pd.init, and a call to FD.Close, which calls FD.pd.evict.
This is solved by calling FD.initIO in FD.Close, as that will block
until the concurrent FD.initIO has completed. Note that FD.initIO is
no-op if first called from here.
The race window is so small that it is not possible to write a test
that triggers it.
Change-Id: Ie2f2818e746b9d626fe3b9eb6b8ff967c81ef863
Reviewed-on: https://go-review.googlesource.com/c/go/+/663815 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
This CL removes some unnecessary code and duplicated NewFile tests
cases.
It also simplifies TestPipeCanceled by removing the need for using
SetReadDeadline. Using CancelIoEx instead of CancelIo makes the cancel
operations to finish almost instantly. The latter could take more than
20s to finish if called from a thread different from the one that
called ReadFile.
Change-Id: I9033cbcad277666bc2aec89b3e5a3ef529da2cd8
Reviewed-on: https://go-review.googlesource.com/c/go/+/663755 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Damien Neil [Tue, 8 Apr 2025 20:39:08 +0000 (13:39 -0700)]
context: don't return a nil Cause for a canceled custom context
Avoid a case where Cause(ctx) could return nil for a canceled context,
when ctx is a custom context implementation and descends from a
cancellable-but-not-canceled first-party Context.
Fixes #73258
Change-Id: Idbd81ccddea82ecabece4373d718baae6ca4b58e
Reviewed-on: https://go-review.googlesource.com/c/go/+/663936 Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
time: add examples for AppendBinary and AppendText
Change-Id: I61529b5162f8a77d3bbffcbbac98c834a7626e3a
Reviewed-on: https://go-review.googlesource.com/c/go/+/661935 Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sean Liao <sean@liao.dev> Reviewed-by: Carlos Amedee <carlos@golang.org>
Alan Donovan [Tue, 8 Apr 2025 18:12:00 +0000 (14:12 -0400)]
go/types: document that Defs[id] may be missing in ill-typed code
Updates #70968
Change-Id: Id0a4acd6bad917ba8a5c439625bca14469b6eb7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/663895
Auto-Submit: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Alan Donovan <adonovan@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
syscall: fix dangling pointers in Windows' process attributes
Windows' _PROC_THREAD_ATTRIBUTE_LIST can contain pointers to memory
owned by Go, but the GC is not aware of this. This can lead to the
memory being freed while the _PROC_THREAD_ATTRIBUTE_LIST is still in
use.
This CL uses the same approach as in x/sys/windows to ensure that the
attributes are not collected by the GC.
Fixes #73170.
Updates #73199.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: I7dca8d386aed4c02fdcd4a631d0fa4dc5747a96f
Reviewed-on: https://go-review.googlesource.com/c/go/+/663715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
internal/runtime/maps: pass proper func PC to race.WritePC/race.ReadPC
Fixes #73191
Change-Id: I0f8a5a19faa745943a98476c7caf4c97ccdce184
Reviewed-on: https://go-review.googlesource.com/c/go/+/663175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Follow the approach used in strconv's readFloat, decimal.set, and Atoi,
where leading '+' and '-' are handled using a switch for clarity and
consistency.
Change-Id: I41eff34ce90b5ac43fcdbc0bb88910d6d5fb4d39
GitHub-Last-Rev: 0c9d2efb5a828515fa00afdba8c436aa31fb0e53
GitHub-Pull-Request: golang/go#73185
Reviewed-on: https://go-review.googlesource.com/c/go/+/663257 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Some types are not linked.
This change adds a link to each type.
Change-Id: Id46fb64a74efb851ed76e4136f15e8fd9e445bb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/663075 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For some reason i have created a separate package instead
of using _test package. Let's move this test where it belongs.
Change-Id: Ib569ca433de1ef4e161b9d334125648e00b7d3c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/663555
Auto-Submit: Alan Donovan <adonovan@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
cuishuang [Mon, 31 Mar 2025 10:10:43 +0000 (18:10 +0800)]
all: use built-in max/min to simplify the code
Change-Id: I309d93d6ebf0feb462217a344d5f02c190220752
Reviewed-on: https://go-review.googlesource.com/c/go/+/661737 Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Sam Thanawalla [Mon, 31 Mar 2025 19:06:43 +0000 (19:06 +0000)]
cmd/go: fix version stamping for v2 modules and subdirectories
We were not passing the module path to newCodeRepo which caused it to
incorrectly parse the major version. This allowed v0 and v1 modules to
work because an empty major version is allowed in that case.
Additionally we need to pass the root module path to derive the correct tag
for subdirectories.
Fixes: #72877 Fixes: #71738
Change-Id: Id792923f426858513972e713623270edbc76c545
Reviewed-on: https://go-review.googlesource.com/c/go/+/661875
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Richard Miller [Fri, 4 Apr 2025 19:48:18 +0000 (20:48 +0100)]
runtime: protect plan9 time_now function with !faketime build tag
The introduction of monotonic time support for Plan 9 in CL 656755
causes a build error with multiple declaration of time_now when
built with tag faketime. Correct this by moving function time_now
into its own source file with !faketime build tag.
Fixes #73169
Change-Id: Id7a9a1c77e286511e25546089681f2f88a9538bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/662856 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: David du Colombier <0intro@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
cmd/go/internal/load: join incompatible and dirty build specifiers with .
Change "+incompatible+dirty" version to be "+incompatible.dirty" such
that it is SemVer spec compatible.
Fixes #71971
Change-Id: I714ffb3f1ad88c793656c3652367db34739a2144
Reviewed-on: https://go-review.googlesource.com/c/go/+/652955
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Sam Thanawalla <samthanawalla@google.com> Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev> Reviewed-by: Michael Matloob <matloob@golang.org>