]> Cypherpunks repositories - gostls13.git/log
gostls13.git
5 months agoos/user: fix race detector failure
qmuntal [Mon, 18 Nov 2024 08:31:59 +0000 (09:31 +0100)]
os/user: fix race detector failure

The race detector doesn't like that windows.GetSid* functions return
pointers to the SID structure. This change makes these functions return
values instead and mark them with nocheckptr.

Fixes #70378

Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race
Change-Id: Iba39d75bb31679d25a5ee43b51e4abb0c435dbac
Reviewed-on: https://go-review.googlesource.com/c/go/+/628995
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocrypto/internal/hpke: add Recipient role
Roland Shoemaker [Sun, 11 Aug 2024 01:19:09 +0000 (18:19 -0700)]
crypto/internal/hpke: add Recipient role

Adds the Recipient role, alongside the existing Sender role. Also factor
out all of the shared underlying bits and pieces into a shared type that
is embedded in the Sender/Recipient roles.

Change-Id: I7640d8732aa0dd5cc9e38b8c26f0cfa7856170f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/623575
Auto-Submit: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
5 months agocrypto/subtle: add DIT closure
Roland Shoemaker [Mon, 15 Jul 2024 17:05:37 +0000 (10:05 -0700)]
crypto/subtle: add DIT closure

Add a new function, WithDataIndependentTiming, which takes a function as
an argument, and encloses it with calls to set/unset the DIT PSTATE bit
on Arm64.

Since DIT is OS thread-local, for the duration of the execution of
WithDataIndependentTiming, we lock the goroutine to the OS thread, using
LockOSThread. For long running operations, this is likely to not be
performant, but we expect this to be tightly scoped around cryptographic
operations that have bounded execution times.

If locking to the OS thread turns out to be too slow, another option is
to add a bit to the g state indicating if a goroutine has DIT enabled,
and then have the scheduler enable/disable DIT when scheduling a g.

Additionally, we add a new GODEBUG, dataindependenttiming, which allows
setting DIT for an entire program. Running a program with
dataindependenttiming=1 enables DIT for the program during
initialization. In an ideal world PSTATE.DIT would be inherited from
the parent thread, so we'd only need to set it in the main thread and
then all subsequent threads would inherit the value. While this does
happen in the Linux kernel [0], it is not the case for darwin [1].
Rather than add complex logic to only set it on darwin for each new
thread, we just unconditionally set it in mstart1 and cgocallbackg1
regardless of the OS. DIT will already impose some overhead, and the
cost of setting the bit is only ~two instructions (CALL, MSR), so it
should be cheap enough.

Fixes #66450
Updates #49702

[0] https://github.com/torvalds/linux/blob/e8bdb3c8be08c9a3edc0a373c0aa8729355a0705/arch/arm64/kernel/process.c#L373
[1] https://github.com/apple-oss-distributions/xnu/blob/8d741a5de7ff4191bf97d57b9f54c2f6d4a15585/osfmk/arm64/status.c#L1666

Change-Id: I78eda691ff9254b0415f2b54770e5850a0179749
Reviewed-on: https://go-review.googlesource.com/c/go/+/598336
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocmd/go/internal/fsys: replace file tree with sorted list
Russ Cox [Tue, 19 Nov 2024 13:34:07 +0000 (08:34 -0500)]
cmd/go/internal/fsys: replace file tree with sorted list

Replace the tree of nodes with a sorted list of file replacements.
The most important property of this representation is that it
allows replacing directories: a replacement x -> y where y is
a directory could not be implemented before, because it would
require making a node for every file in the tree rooted at y,
or else it would require unsuccessful lookups for files like
x/a/b/c/d/e/f/g/h/i/j/k to try every possible parent in order
to discover the x -> y mapping.

The sorted list makes it easy to find the x -> y mapping:
when you do the binary search for x/a/b/c/d/e/f/g/h/i/j/k,
you end up immediately after the x -> y mapping, so stepping
backward one entry provides the mapping we need, if it exists.

This CL does not allow overlay files to include directories,
but now it is possible. This is at least useful for other kinds
of experiments (like FIPS).

Change-Id: Ief0afaee82e644dab8ae4eafeec20440afee2e36
Reviewed-on: https://go-review.googlesource.com/c/go/+/628701
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
5 months agocrypto/internal/fips/hkdf: new package
Filippo Valsorda [Sun, 20 Oct 2024 15:29:58 +0000 (17:29 +0200)]
crypto/internal/fips/hkdf: new package

Tests imported from x/crypto, but the actual implementation was simpler
to implement ex-novo with a #61477-like API.

Updates #61477
For #69536

Change-Id: I5a9e8a71d8abd5b2aa6b74e73bf7f631ed0115cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/621275
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Russ Cox <rsc@golang.org>
5 months agocrypto/cipher: add NewGCMWithRandomNonce
Filippo Valsorda [Mon, 18 Nov 2024 15:19:12 +0000 (16:19 +0100)]
crypto/cipher: add NewGCMWithRandomNonce

Fixes #69981

Change-Id: I0cad11f5d7673304c5a6d85fc598ddc27ab93738
Reviewed-on: https://go-review.googlesource.com/c/go/+/629175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
5 months agocrypto/internal/fips/aes/gcm: add GCMForSSH
Filippo Valsorda [Mon, 18 Nov 2024 14:07:37 +0000 (15:07 +0100)]
crypto/internal/fips/aes/gcm: add GCMForSSH

For #69536

Change-Id: Ia368f515893a95e176149e23239a8e253fc5272f
Reviewed-on: https://go-review.googlesource.com/c/go/+/629095
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
5 months agocrypto: check all cpu.X86 flags for features used in assembly
Filippo Valsorda [Sat, 9 Nov 2024 18:16:13 +0000 (19:16 +0100)]
crypto: check all cpu.X86 flags for features used in assembly

These are most likely redundant, but cmd/compile/internal/amd64's
TestGoAMD64v1 turns them off when clobbering those instructions, so we
need to know to skip the assembly in those cases.

Thankfully we have Avo now that adds a helpful comment with the list of
features used by each generated function!

Also improve the error output of TestGoAMD64v1. It had broken before in
#49402 and had required the exact same patch.

Change-Id: I7fab8f36042cdff630f806723aa1d8124c294f60
Reviewed-on: https://go-review.googlesource.com/c/go/+/626876
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocrypto/internal/fips/aes: add service indicators
Filippo Valsorda [Sat, 9 Nov 2024 10:09:19 +0000 (11:09 +0100)]
crypto/internal/fips/aes: add service indicators

For #69536

Change-Id: I485c165b1d9fcd7b09ff151bbeebc57d8908bcb8
Reviewed-on: https://go-review.googlesource.com/c/go/+/626835
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
5 months agocmd/go/internal/fsys: refactor overlay consistency checks
Russ Cox [Sat, 16 Nov 2024 21:49:00 +0000 (16:49 -0500)]
cmd/go/internal/fsys: refactor overlay consistency checks

Do the overlay consistency checks separate from constructing
the overlay data structure. This makes sure that the data structure
can be changed without worrying about losing the checks.

Change-Id: I9ff50cc366b5362adc5570f94e6caf646ddf5046
Reviewed-on: https://go-review.googlesource.com/c/go/+/628700
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
5 months agocmd/go/internal/fsys: refactor vfs lookup
Russ Cox [Fri, 15 Nov 2024 19:42:27 +0000 (14:42 -0500)]
cmd/go/internal/fsys: refactor vfs lookup

Refactor vfs lookup into 'func stat', which knows the internal
data structures for the vfs and returns information about a
given path. The callers can then all use stat and avoid direct
knowledge of the internal data structures.

This is setting up for a different internal data structure.

Change-Id: I496b7b3fb686cdde81b14687f65eb0bf51ec62be
Reviewed-on: https://go-review.googlesource.com/c/go/+/628699
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
5 months agocmd/go/internal/fsys: convert to proper ReadDir
Russ Cox [Fri, 15 Nov 2024 18:49:28 +0000 (13:49 -0500)]
cmd/go/internal/fsys: convert to proper ReadDir

Many releases ago we migrated
from ioutil.ReadDir, which returned []os.FileInfo,
to os.ReadDir, which returns []fs.DirEntry.
The latter is faster, but the former is expected by go/build.Context.

Convert fsys to use the new ReadDir signature.
This should make the go command faster when scanning
source trees, and it brings cmd/go up to date with the rest
of the tree.

Similarly, convert Walk to WalkDir.

Change-Id: I767a8548d7ca7cc3c05f2ff073d18070a4e8a0da
Reviewed-on: https://go-review.googlesource.com/c/go/+/628698
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
5 months agocmd/go/internal/fsys: minor cleanup
Russ Cox [Fri, 15 Nov 2024 17:54:37 +0000 (12:54 -0500)]
cmd/go/internal/fsys: minor cleanup

Rename canonicalize to abs.
Rename IsDirWithGoFiles to IsGoDir.
Remove Init argument.
Split OverlayPath into Actual and Renamed.
Clean up doc comments.
Other minor cleanups.

Preparation for larger changes.

Change-Id: Ida022588149a1618a63acc91e3800b09df873b6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/628697
TryBot-Bypass: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
5 months agocmd/go/internal/fsys: move glob, walk code into new files
Russ Cox [Fri, 15 Nov 2024 17:52:10 +0000 (12:52 -0500)]
cmd/go/internal/fsys: move glob, walk code into new files

The Glob and Walk code does not depend on any of the fsys internals;
it simply uses ReadDir as an opaque abstraction.
Move it to separate files so that when working on the
actual overlay abstraction, it is out of sight, out of mind.

Change-Id: Ifa98feaaaafe5c1d8d8edce82de4fd0c78f599c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/628696
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
5 months agodoc/godebug: document that unrecognized GODEBUG entries are ignored
Ian Lance Taylor [Tue, 19 Nov 2024 03:27:22 +0000 (19:27 -0800)]
doc/godebug: document that unrecognized GODEBUG entries are ignored

Change-Id: I6debc82913ae9fa02bee55dc87d35574f5265008
Reviewed-on: https://go-review.googlesource.com/c/go/+/629576
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>

5 months agodoc/godebug: fix tipo
Ian Lance Taylor [Tue, 19 Nov 2024 03:26:38 +0000 (19:26 -0800)]
doc/godebug: fix tipo

Change-Id: Ib67d81d8e3aa33c6a506813194cc5710b3d5cdd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/629575
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>

5 months agomime: disable TestLookupMallocs with ASAN
Michael Anthony Knyszek [Mon, 18 Nov 2024 22:16:57 +0000 (22:16 +0000)]
mime: disable TestLookupMallocs with ASAN

With the synchashtriemap GOEXPERIMENT, this allocates now, but just
under ASAN. That's OK, just skip the test under ASAN.

Fixes #70428.

Change-Id: I1ab4f7c06505c84309946358643715085f0db506
Reviewed-on: https://go-review.googlesource.com/c/go/+/629456
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agocmd/go: re-enable build JSON from go test -json, now with GODEBUG
Austin Clements [Mon, 18 Nov 2024 19:56:43 +0000 (14:56 -0500)]
cmd/go: re-enable build JSON from go test -json, now with GODEBUG

This re-enables the behavior of CL 536399 (by effectively reverting CL
628955), so now go test -json again includes build output and failures
as JSON rather than text.

However, since this behavior is clearly enough to trip up some build
systems, this CL includes a GODEBUG=gotestjsonbuildtext that can be
set to 1 to revert to the old behavior.

Fixes #70402.
Updates #62067.

Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64_13,gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: I84e778cd844783dacfc83433e391b5ccb5925127
Reviewed-on: https://go-review.googlesource.com/c/go/+/629335
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Austin Clements <austin@google.com>

5 months agocmd/compiler,internal/runtime/atomic: optimize Cas{64,32} on loong64
Guoqi Chen [Fri, 20 Sep 2024 03:06:18 +0000 (11:06 +0800)]
cmd/compiler,internal/runtime/atomic: optimize Cas{64,32} on loong64

In Loongson's new microstructure LA664 (Loongson-3A6000) and later, the atomic
compare-and-exchange instruction AMCAS[DB]{B,W,H,V} [1] is supported. Therefore,
the implementation of the atomic operation compare-and-swap can be selected according
to the CPUCFG flag LAMCAS: AMCASDB(full barrier) instruction is used on new
microstructures, and traditional LL-SC is used on LA464 (Loongson-3A5000) and older
microstructures. This can significantly improve the performance of Go programs on
new microstructures.

goos: linux
goarch: loong64
pkg: internal/runtime/atomic
cpu: Loongson-3A6000 @ 2500.00MHz
         |  bench.old   |  bench.new                           |
         |   sec/op     |   sec/op       vs base               |
Cas        46.84n ±  0%   22.82n ±  0%  -51.28% (p=0.000 n=20)
Cas-2      47.58n ±  0%   29.57n ±  0%  -37.85% (p=0.000 n=20)
Cas-4      43.27n ± 20%   25.31n ± 13%  -41.50% (p=0.000 n=20)
Cas64      46.85n ±  0%   22.82n ±  0%  -51.29% (p=0.000 n=20)
Cas64-2    47.43n ±  0%   29.53n ±  0%  -37.74% (p=0.002 n=20)
Cas64-4    43.18n ±  0%   25.28n ±  2%  -41.46% (p=0.000 n=20)
geomean    45.82n         25.74n        -43.82%

goos: linux
goarch: loong64
pkg: internal/runtime/atomic
cpu: Loongson-3A5000 @ 2500.00MHz
         |  bench.old  |  bench.new                         |
         |   sec/op    |   sec/op      vs base              |
Cas        50.05n ± 0%   51.26n ± 0%  +2.42% (p=0.000 n=20)
Cas-2      52.80n ± 0%   53.11n ± 0%  +0.59% (p=0.000 n=20)
Cas-4      55.97n ± 0%   57.31n ± 0%  +2.39% (p=0.000 n=20)
Cas64      50.05n ± 0%   51.26n ± 0%  +2.42% (p=0.000 n=20)
Cas64-2    52.68n ± 0%   53.11n ± 0%  +0.82% (p=0.000 n=20)
Cas64-4    55.96n ± 0%   57.26n ± 0%  +2.33% (p=0.000 n=20)
geomean    52.86n        53.83n       +1.82%

[1]: https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html

Change-Id: I9b777c63c124fb492f61c903f77061fa2b4e5322
Reviewed-on: https://go-review.googlesource.com/c/go/+/613396
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Qiqi Huang <huangqiqi@loongson.cn>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocmd/go: disable fuzz instrumentation for internal/godebug
Michael Anthony Knyszek [Mon, 18 Nov 2024 22:23:43 +0000 (22:23 +0000)]
cmd/go: disable fuzz instrumentation for internal/godebug

This is thought to be the cause of certain recent longtest failures.
Let's try it out.

This appears to fix the longtests fuzz failures. I suspect that the
sync.Map in internal/godebug is at fault with the implementation
changing. I'm not sure yet exactly why this is a problem, maybe inlining
that didn't happen before? I don't know exactly when coverage
instrumentation happens in the compiler, but this is definitely the
problem.

For good measure, let's add internal/sync. If sync is on the list,
internal/sync should be, too.

Fixes #70429.
Fixes #70430.
Fixes #70431.

Change-Id: Ic9f49daa0956e3a50192bcc7778983682b5d12b8
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/629475
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocrypto/internal/fips/aes/gcm: add GCMForTLS12 and GCMForTLS13
Filippo Valsorda [Fri, 8 Nov 2024 18:56:50 +0000 (19:56 +0100)]
crypto/internal/fips/aes/gcm: add GCMForTLS12 and GCMForTLS13

For #69536

Change-Id: I2d7b6e7b9932d0f0f582a5ab0bb871395dc2a1e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/626675
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>

5 months agocrypto/internal/fips/aes/gcm: add SealWithRandomNonce
Filippo Valsorda [Sun, 3 Nov 2024 21:30:43 +0000 (22:30 +0100)]
crypto/internal/fips/aes/gcm: add SealWithRandomNonce

We don't expose it as an AEAD yet because the logic for that is complex
due to overlap issues. For #69981 we will make a cipher.AEAD wrapper
outside the FIPS module, but maybe a v2 interface will make it easier,
and then we'll be able to use this method more directly.

Updates #69981
For #69536

Change-Id: Id88191c01443b0dec89ff0d6c4a6289f519369d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/624916
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
5 months agocrypto/internal/fips/aes/gcm: add CounterKDF
Filippo Valsorda [Thu, 7 Nov 2024 12:32:13 +0000 (13:32 +0100)]
crypto/internal/fips/aes/gcm: add CounterKDF

Change-Id: I236f9d8c4637793c1bda39f69573481356f8e362
Reviewed-on: https://go-review.googlesource.com/c/go/+/626436
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
5 months agocrypto/internal/fips/aes/gcm: add CMAC
Filippo Valsorda [Wed, 6 Nov 2024 13:03:58 +0000 (14:03 +0100)]
crypto/internal/fips/aes/gcm: add CMAC

Change-Id: I5602dbf485c5c8a221e71c79961588e33f90452d
Reviewed-on: https://go-review.googlesource.com/c/go/+/626435
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
5 months agocrypto/aes,crypto/cipher: test all available implementations
Filippo Valsorda [Sun, 3 Nov 2024 12:10:33 +0000 (13:10 +0100)]
crypto/aes,crypto/cipher: test all available implementations

TestEmptyPlaintext and TestCryptBlocks were folded into cryptotest.

Change-Id: I6131ab8582eb0e6d3a1b24bab1147a145d9766ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/624738
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocrypto/internal/fips/aes: move GCM to its own package
Filippo Valsorda [Sun, 3 Nov 2024 09:03:21 +0000 (10:03 +0100)]
crypto/internal/fips/aes: move GCM to its own package

For #69536

Change-Id: Idcef2411c1fd4da302412609f28e438839e8aa4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/624736
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>

5 months agocrypto/internal/cryptotest: add SkipTestAllocations
Filippo Valsorda [Fri, 8 Nov 2024 13:41:06 +0000 (14:41 +0100)]
crypto/internal/cryptotest: add SkipTestAllocations

[                                                                     ]
[    It has been [ 0 ] days since Filippo broke a TestAllocations.    ]
[                                                                     ]

Concentrate all the skips in one place, so we don't have to re-discover
always the same ones via trial and error.

This might over-skip fixable allocations, but all these targets are not
fast anyway, so they are not worth going back for.

Removed the sysrand TestAllocations because it causes an import loop
with cryptotest and it's covered by TestAllocations in crypto/rand.

Change-Id: Icd40e97f9128e037f567147f8c9dafa758a47fac
Reviewed-on: https://go-review.googlesource.com/c/go/+/626438
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>

5 months agocrypto/internal/fips/drbg: add CAST and service indicator
Filippo Valsorda [Tue, 5 Nov 2024 13:07:21 +0000 (14:07 +0100)]
crypto/internal/fips/drbg: add CAST and service indicator

For #69536

Change-Id: If216a7537013bd9bbe90026ffe983a6bf50029c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/625435
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocrypto/internal/fips/drbg: implement FIPS-mode randomness generation
Filippo Valsorda [Mon, 4 Nov 2024 14:33:00 +0000 (15:33 +0100)]
crypto/internal/fips/drbg: implement FIPS-mode randomness generation

For #69536

Change-Id: I5bc83360fcd9114cc76314c3570977e9811fd1c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/624978
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
5 months agocrypto/internal/fips/drbg: implement CTR_DRBG
Filippo Valsorda [Mon, 4 Nov 2024 13:02:58 +0000 (14:02 +0100)]
crypto/internal/fips/drbg: implement CTR_DRBG

For #69536

Change-Id: I016bb723841acbda50f013db46f9d2dda200e1fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/624977
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>

5 months agocrypto/rand: move OS interaction to crypto/internal/sysrand
Filippo Valsorda [Mon, 4 Nov 2024 11:56:04 +0000 (12:56 +0100)]
crypto/rand: move OS interaction to crypto/internal/sysrand

We're going to use that package as the passive entropy source for the
FIPS module, and we need to import it from a package that will be
imported by crypto/rand.

Since there is no overridable Reader now, introduced a mechanism to test
the otherwise impossible failure of the OS entropy source.

For #69536

Change-Id: I558687ed1ec896dba05b99b937970bb809de3fe7
Reviewed-on: https://go-review.googlesource.com/c/go/+/624976
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
5 months agocrypto/rand: replace crypto/aes with internal/chacha8rand for plan9
Filippo Valsorda [Mon, 4 Nov 2024 12:05:46 +0000 (13:05 +0100)]
crypto/rand: replace crypto/aes with internal/chacha8rand for plan9

We will need to import this functionality from the FIPS module, and we
can't import AES from there. Plan 9 is not going to be FIPS validated
anyway, so we can use non-approved cryptography.

For #69536

Change-Id: I7921ec0829b576de2e80f3a7d0a9a776ff387684
Reviewed-on: https://go-review.googlesource.com/c/go/+/624975
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
5 months agocrypto/aes: move to crypto/internal/fips/aes
Filippo Valsorda [Fri, 1 Nov 2024 10:46:34 +0000 (11:46 +0100)]
crypto/aes: move to crypto/internal/fips/aes

The crypto/aes <-> crypto/cipher interfaces and the hardware support
upgrades were layered over the years, and had grown unwieldily.

Before: conditionally wrap the private crypto/aes type in private types
that implement an interface that's interface-upgraded by crypto/cipher
to replace the generic implementation in crypto/cipher.

crypto/aes depended on crypto/cipher, which is backwards.

After: provide concrete exported implementations of modes in
crypto/internal/fips/aes that crypto/cipher returns if the input Block
is the crypto/internal/fips/aes concrete implementation.

crypto/aes and crypto/cipher both depend on crypto/internal/fips/aes.

Also, made everything follow go.dev/wiki/TargetSpecific by only putting
the minimal code necessary and no exported functions in build-tagged
files.

The GCM integration still uses an interface upgrade, because the generic
implementation is complex enough that it was not trivial to duplicate.
This will be fixed in a future CL to make review easier.

For #69536

Change-Id: I21c2b93a498edb31c562b1aca824e21e8457fdff
Reviewed-on: https://go-review.googlesource.com/c/go/+/624395
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>

5 months agocrypto/aes: speedup CTR mode on AMD64 and ARM64
Boris Nagaev [Thu, 8 Feb 2024 01:27:16 +0000 (01:27 +0000)]
crypto/aes: speedup CTR mode on AMD64 and ARM64

The implementation runs up to 8 AES instructions in different registers
one after another in ASM code. Because CPU has instruction pipelining
and the instructions do not depend on each other, they can run in
parallel with this layout of code. This results in significant speedup
compared to the regular implementation in which blocks are processed in
the same registers so AES instructions do not run in parallel.

GCM mode already utilizes the approach.

The ASM implementation of ctrAble has most of its code in XORKeyStreamAt
method which has an additional argument, offset. It allows to use it
in a stateless way and to jump to any location in the stream. The method
does not exist in pure Go and boringcrypto implementations.

[ Mailed as CL 413594, then edited by filippo@ to manage the counter
with bits.Add64, remove bounds checks, make the assembly interface more
explicit, and to port the amd64 to Avo. Squeezed another -6.38% out. ]

goos: linux
goarch: amd64
pkg: crypto/cipher
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
            │  19df80d792  │             c8b0409d40              │
            │    sec/op    │   sec/op     vs base                │
AESCTR/50-8    64.68n ± 0%   26.89n ± 0%  -58.42% (p=0.000 n=10)
AESCTR/1K-8   1145.0n ± 0%   135.8n ± 0%  -88.14% (p=0.000 n=10)
AESCTR/8K-8   9145.0n ± 0%   917.5n ± 0%  -89.97% (p=0.000 n=10)
geomean        878.2n        149.6n       -82.96%

            │  19df80d792  │               c8b0409d40               │
            │     B/s      │      B/s       vs base                 │
AESCTR/50-8   737.2Mi ± 0%   1773.3Mi ± 0%  +140.54% (p=0.000 n=10)
AESCTR/1K-8   848.5Mi ± 0%   7156.6Mi ± 0%  +743.40% (p=0.000 n=10)
AESCTR/8K-8   853.8Mi ± 0%   8509.9Mi ± 0%  +896.70% (p=0.000 n=10)
geomean       811.4Mi         4.651Gi       +486.94%

Fixes #20967
Updates #39365
Updates #26673

Co-authored-by: Filippo Valsorda <filippo@golang.org>
Change-Id: Iaeea29fb93a56456f2e54507bc25196edb31b84b
Reviewed-on: https://go-review.googlesource.com/c/go/+/621958
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
5 months agocmd/compile: strongly favor closure inlining
David Chase [Fri, 15 Nov 2024 22:08:34 +0000 (17:08 -0500)]
cmd/compile: strongly favor closure inlining

This tweaks the inlining cost knob for closures
specifically, they receive a doubled budget.  The
rationale for this is that closures have a lot of
"crud" in their IR that will disappear after inlining,
so the standard budget penalizes them unnecessarily.

This is also the cause of these bugs -- looking at the
code involved, these closures "should" be inlineable,
therefore tweak the parameters until behavior matches
expectations.  It's not costly in binary size, because
the only-called-from-one-site case is common (especially
for rangefunc iterators).

I can imagine better fixes and I am going to try to
get that done, but this one is small and makes things
better.

Fixes #69411, #69539.

Change-Id: I8a892c40323173a723799e0ddad69dcc2724a8f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/629195
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agoos: correctly handle errno==0 in (*Process).blockUntilWaitable
Damien Neil [Mon, 18 Nov 2024 22:39:20 +0000 (14:39 -0800)]
os: correctly handle errno==0 in (*Process).blockUntilWaitable

CL 627478 inadvertently returns a non-nil error
containing a syscall.Errno(0).

Change-Id: I1d6a9d0575d3ed651ddc02f30505437d0d266bb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/629515
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>

5 months agoweak: move internal/weak to weak, and update according to proposal
Michael Anthony Knyszek [Fri, 15 Nov 2024 20:42:32 +0000 (20:42 +0000)]
weak: move internal/weak to weak, and update according to proposal

The updates are:
- API documentation changes.
- Removal of the old package documentation discouraging linkname.
- Addition of new package documentation with some advice.
- Renaming of weak.Pointer.Strong -> weak.Pointer.Value.

Fixes #67552.

Change-Id: Ifad7e629b6d339dacaf2ca37b459d7f903e31bf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/628455
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agoos/user: skip tests that create users when running on dev machines
qmuntal [Mon, 18 Nov 2024 10:07:07 +0000 (11:07 +0100)]
os/user: skip tests that create users when running on dev machines

Creating and deleting users is tricky to get right, and it's not
something we want to do on a developer machine. This change skips the
tests that create users when not running on a Go builder.

This will fix #70396, although I still don't understand why the test
user couldn't be recreated.

Fixes #70396

Change-Id: Ie7004dc209f94e72152c7d6bd8ec95cc12c79757
Reviewed-on: https://go-review.googlesource.com/c/go/+/627877
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
5 months agoruntime: get rid of gc programs for types
Keith Randall [Thu, 26 Sep 2024 05:34:43 +0000 (22:34 -0700)]
runtime: get rid of gc programs for types

Instead, have the runtime build the gc bitmaps on demand
at runtime.

Change-Id: If7a245bc62e4bce3ce80972410b0ed307d921abe
Reviewed-on: https://go-review.googlesource.com/c/go/+/616255
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@google.com>
5 months agocmd/link: use types (and not GC programs) to build data/bss ptrmasks
khr@golang.org [Thu, 30 Nov 2023 06:03:22 +0000 (22:03 -0800)]
cmd/link: use types (and not GC programs) to build data/bss ptrmasks

The linker knows the types of the global variables. We can use those
types to build the GC programs that describe the data and bss pointer masks.

That way we don't use the GC programs of the constituent types.
This is part of an effort to remove GC programs from the runtime.

There's a major complication in that when we're linking against a
shared library (typically, libstd.so), the relocations we need to
break apart arrays and structs into constituent types are difficult to
find. Load that additional data when linking against shared libraries.

Change-Id: I8516b24a0604479895c7b8a8a358d3cd8d421530
Reviewed-on: https://go-review.googlesource.com/c/go/+/546216
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
5 months agosync: make HashTrieMap[any, any] the default implementation of Map
Michael Anthony Knyszek [Mon, 26 Aug 2024 14:18:26 +0000 (14:18 +0000)]
sync: make HashTrieMap[any, any] the default implementation of Map

This change adds a GOEXPERIMENT, synchashtriemap, which replaces the
internals of a sync.Map with internal/sync.HashTrieMap[any, any]. The
main purpose behind this change is improved performance. Across almost
every benchmark, HashTrieMap[any, any] performs better than Map.

Also, relax TestMapClearNoAllocations to allow for one allocation.
Currently, the HashTrieMap allocates a new empty root node and stores
it: that's the whole clear operation. At the cost of some complexity, we
could allow Clear to have zero allocations by clearing the root node.
The complexity comes down to allowing threads to race to install a new
root node *or* creating a top-level mutex for installing a root node.

But I'm not sure this is worth it. Whether Clear or some other operation
takes the hit for allocating a single node almost certainly doesn't
matter. And Clear is still much, much faster in the new implementation
than the old, so I don't consider this a regression.

Change-Id: I939aa70a0edf2e850cedbea239aaf29a11a77b79
Reviewed-on: https://go-review.googlesource.com/c/go/+/608335
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
5 months agointernal/sync: optimize CompareAndSwap and Swap
Michael Anthony Knyszek [Fri, 16 Aug 2024 15:13:52 +0000 (15:13 +0000)]
internal/sync: optimize CompareAndSwap and Swap

We observe the CompareAndSwap and Swap can both be substantially faster
if the value in each entry node is mutable. This change modifies the
map entry node to store the value indirectly, allowing us to perform
swaps for existing nodes and compare-and-swaps without taking the
parent node's lock.

Change-Id: I371343aa81a843d3a7e6bc5ac87b8a96c12ca3a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/606462
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agosync: add HashTrieMap to Map tests and benchmarks
Michael Anthony Knyszek [Tue, 13 Aug 2024 20:51:43 +0000 (20:51 +0000)]
sync: add HashTrieMap to Map tests and benchmarks

Also, rename Map benchmarks to make them easier to single out via
regexp.

Change-Id: I4dcb066745aba1c340f56050d08539ae2976274d
Reviewed-on: https://go-review.googlesource.com/c/go/+/606461
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agointernal/sync: add Range to HashTrieMap
Michael Anthony Knyszek [Thu, 15 Aug 2024 19:22:43 +0000 (19:22 +0000)]
internal/sync: add Range to HashTrieMap

This implementation is trivial given that All already exists. The
purpose of Range is to conform to sync.Map's API.

Change-Id: Icfe755b9986c46b88c7201644e562b1631a02b66
Reviewed-on: https://go-review.googlesource.com/c/go/+/606460
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agointernal/sync: add Clear to HashTrieMap
Michael Anthony Knyszek [Tue, 13 Aug 2024 16:44:29 +0000 (16:44 +0000)]
internal/sync: add Clear to HashTrieMap

This change adds the Clear operation to HashTrieMap to align it with
sync.Map.

Change-Id: I46069b018725d7fe3a07d667876235534bca86f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/606459
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agointernal/sync: add Delete to HashTrieMap
Michael Anthony Knyszek [Tue, 13 Aug 2024 16:33:30 +0000 (16:33 +0000)]
internal/sync: add Delete to HashTrieMap

This change adds the Delete operation (with the same semantics as
sync.Map's Delete) to HashTrieMap.

Change-Id: If90376ff735256444538719d670ed07e9e42870b
Reviewed-on: https://go-review.googlesource.com/c/go/+/606458
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agointernal/sync: refactor HashTrieMap tests into per-op suites
Michael Anthony Knyszek [Thu, 15 Aug 2024 18:53:33 +0000 (18:53 +0000)]
internal/sync: refactor HashTrieMap tests into per-op suites

This change does a minor refactor of the HashTrieMap tests to be better
organized into suites for each operation. It might be worthwhile to
deduplicate some code in here, but it's also helpful to see exactly
what's happening when something goes wrong.

Change-Id: I138515ee6de5aec5f3d38afe8a3bf3972afb2800
Reviewed-on: https://go-review.googlesource.com/c/go/+/606457
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agointernal/sync: add LoadAndDelete to HashTrieMap
Michael Anthony Knyszek [Tue, 13 Aug 2024 16:31:27 +0000 (16:31 +0000)]
internal/sync: add LoadAndDelete to HashTrieMap

This change adds the LoadAndDelete operation (with the same semantics as
sync.Map's LoadAndDelete) to HashTrieMap.

Change-Id: Id6777dffcd3ebc98490aa51f0e85e59a56f63074
Reviewed-on: https://go-review.googlesource.com/c/go/+/606456
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
5 months agointernal/sync: add Store to HashTrieMap
Michael Anthony Knyszek [Tue, 13 Aug 2024 15:12:21 +0000 (15:12 +0000)]
internal/sync: add Store to HashTrieMap

This change adds the Store operation (with the same semantics as
sync.Map's Store) to HashTrieMap.

Change-Id: Ida55421f28d8cc6527f983f7f1c2fd55bb280925
Reviewed-on: https://go-review.googlesource.com/c/go/+/606455
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agointernal/sync: add Swap to HashTrieMap
Michael Anthony Knyszek [Fri, 21 Jun 2024 21:00:15 +0000 (21:00 +0000)]
internal/sync: add Swap to HashTrieMap

This change adds the Swap operation (with the same semantics as
sync.Map's Swap) to HashTrieMap.

Change-Id: I8697a0c8c2eb761e2452a41b868b590ccbfa5c03
Reviewed-on: https://go-review.googlesource.com/c/go/+/594064
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agointernal/sync: add CompareAndSwap to HashTrieMap
Michael Anthony Knyszek [Fri, 21 Jun 2024 20:42:45 +0000 (20:42 +0000)]
internal/sync: add CompareAndSwap to HashTrieMap

This change adds the CompareAndSwap operation (with the same semantics
as sync.Map's CompareAndSwap) to HashTrieMap.

Change-Id: I86153799fc47304784333d17f0c6a7ad7682f04a
Reviewed-on: https://go-review.googlesource.com/c/go/+/594063
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agointernal/sync: make the HashTrieMap zero value ready to use
Michael Anthony Knyszek [Fri, 21 Jun 2024 20:23:13 +0000 (20:23 +0000)]
internal/sync: make the HashTrieMap zero value ready to use

This improves ergonomics a little and aligns the HashTrieMap with
sync.Map.

Change-Id: Idb2b981a4f59a35f8670c6b5038e2bd207484483
Reviewed-on: https://go-review.googlesource.com/c/go/+/594062
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agointernal/sync: relax value type constraint for HashTrieMap
Michael Anthony Knyszek [Fri, 21 Jun 2024 20:09:09 +0000 (20:09 +0000)]
internal/sync: relax value type constraint for HashTrieMap

Currently the HashTrieMap requires both keys and values to be
comparable, but it's actually OK if the value is not comparable. Some
operations may fail, but others will not, and we can check comparability
dynamically on map initialization. This makes the implementation
substantially more flexible.

Change-Id: Idc9c30dfa273d80ae4d46a9eefb5c155294408aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/594061
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
5 months agointernal/sync: use normal comparison for keys in HashTrieMap
Michael Anthony Knyszek [Fri, 21 Jun 2024 20:03:16 +0000 (20:03 +0000)]
internal/sync: use normal comparison for keys in HashTrieMap

There's are unnecessary calls to the key's equal function -- we can just
leverage the language here. Leave the values alone for now, we want to
relax that constraint.

Change-Id: Iccfaef030a2a29b6a24a7da41e5e816b70091c7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/594060
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
5 months agointernal/sync: factor out lookup for CompareAndDelete in HashTrieMap
Michael Anthony Knyszek [Fri, 21 Jun 2024 19:59:11 +0000 (19:59 +0000)]
internal/sync: factor out lookup for CompareAndDelete in HashTrieMap

This lookup will be reused for other operations, like CompareAndSwap.

Change-Id: I6698e3c99f7ef6d2b82b9ef489ba8a0be2a71d61
Reviewed-on: https://go-review.googlesource.com/c/go/+/594059
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
5 months agointernal/sync: move HashTrieMap from internal/concurrent
Michael Anthony Knyszek [Fri, 21 Jun 2024 17:20:00 +0000 (17:20 +0000)]
internal/sync: move HashTrieMap from internal/concurrent

This change moves internal/concurrent.HashTrieMap from
internal/concurrent into internal/sync just to clean up the packages a
bit. This is all in anticipation of using HashTrieMap from the sync
package.

Change-Id: I18c007a301f83979d72f5d6bea600c42eaf2421e
Reviewed-on: https://go-review.googlesource.com/c/go/+/594058
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agointernal/concurrent: remove dependency on math/rand/v2
Michael Anthony Knyszek [Fri, 21 Jun 2024 17:07:28 +0000 (17:07 +0000)]
internal/concurrent: remove dependency on math/rand/v2

This change uses linkname for the one random function
internal/concurrent needs to avoid taking a dependency on math/rand/v2.
This lowers the bar to using this package.

Change-Id: I9dba1121b66ba35f56521643937f220936ea5321
Reviewed-on: https://go-review.googlesource.com/c/go/+/594057
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agointernal/sync: move sync.Mutex implementation into new package
Michael Anthony Knyszek [Fri, 21 Jun 2024 17:01:23 +0000 (17:01 +0000)]
internal/sync: move sync.Mutex implementation into new package

This CL refactors sync.Mutex such that its implementation lives in the
new internal/sync package. The purpose of this change is to eventually
reverse the dependency edge between internal/concurrent and sync, such
that sync can depend on internal/concurrent (or really, its contents,
which will likely end up in internal/sync).

The only change made to the sync.Mutex code is the frame skip count for
mutex profiling, so that the internal/sync frames are omitted in the
profile.

Change-Id: Ib3603d30e8e71508c4ea883a584ae2e51ce40c3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/594056
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

5 months agoos/user: s/Acount/Account/ in function name
Ian Lance Taylor [Mon, 18 Nov 2024 19:21:59 +0000 (11:21 -0800)]
os/user: s/Acount/Account/ in function name

Change-Id: I9aa34951f2005c204aafd9da74e78c033c9b64ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/629315
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
5 months agocmd: change from sort functions to slices functions
Ian Lance Taylor [Wed, 22 May 2024 04:07:32 +0000 (21:07 -0700)]
cmd: change from sort functions to slices functions

Doing this because the slices functions are slightly faster and
slightly easier to use. It also removes one dependency layer.

We did this outside of bootstrap tools in CL 587655.
Now that the bootstrap compiler is 1.22, we can do this in more code.

Change-Id: I9ed2dd473758cacd14f76a0639368523ccdff72f
Reviewed-on: https://go-review.googlesource.com/c/go/+/626038
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
5 months agoos: sysctl-based Executable implementation for NetBSD
Benny Siegert [Mon, 18 Nov 2024 12:43:58 +0000 (13:43 +0100)]
os: sysctl-based Executable implementation for NetBSD

FreeBSD and Dragonfly have used the sysctl method for years, while
NetBSD has read the name of the executable from /proc. Unfortunately,
some folks are hitting errors when building Go software in a sandbox
that lacks a mounted /proc filesystem.

Switch NetBSD to use the same implementation as FreeBSD and Dragonfly.
Unfortunately, the order of the arguments in the MIB is also
OS-dependent.

Change-Id: I6fd774904af417ccd127e3779af45a20dc8696ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/629035
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>

5 months agosync: add explicit noCopy fields to Map, Mutex, and Once
Michael Anthony Knyszek [Fri, 15 Nov 2024 19:22:16 +0000 (19:22 +0000)]
sync: add explicit noCopy fields to Map, Mutex, and Once

Following CLs will refactor Mutex and change the internals of Map. This
ends up breaking tests in x/tools for the copylock vet check, because
the error message changes. Let's insulate ourselves from such things
permanently by adding an explicit noCopy field. We'll update the vet
check to accept that as the problem, rather than depend on less explicit
internals.

We capture Once here too to clean up the error message as well.

Change-Id: Iead985fc8ec9ef3ea5ff615f26dde17bb03aeadb
Reviewed-on: https://go-review.googlesource.com/c/go/+/627777
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Tim King <taking@google.com>
5 months agocmd/compile: remove gc programs from stack frame objects
Keith Randall [Wed, 15 Nov 2023 21:38:06 +0000 (13:38 -0800)]
cmd/compile: remove gc programs from stack frame objects

This is a two-pronged approach. First, try to keep large objects
off the stack frame. Second, if they do manage to appear anyway,
use straight bitmasks instead of gc programs.

Generally probably a good idea to keep large objects out of stack frames.
But particularly keeping gc programs off the stack simplifies
runtime code a bit.

This CL sets the limit of most stack objects to 131072 bytes (on 64-bit archs).
There can still be large objects if allocated by a late pass, like order, or
they are required to be on the stack, like function arguments.
But the size for the bitmasks for these objects isn't a huge deal,
as we have already have (probably several) bitmasks for the frame
liveness map itself.

Change-Id: I6d2bed0e9aa9ac7499955562c6154f9264061359
Reviewed-on: https://go-review.googlesource.com/c/go/+/542815
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
5 months agointernal/runtime/maps: don't copy indirect key/elem when growing maps
Keith Randall [Sat, 9 Nov 2024 01:17:15 +0000 (17:17 -0800)]
internal/runtime/maps: don't copy indirect key/elem when growing maps

We can reuse the same indirect storage when growing, so we don't
need an additional allocation.

Change-Id: I57adb406becfbec648188ec66f4bb2e94d4b9cab
Reviewed-on: https://go-review.googlesource.com/c/go/+/625902
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>
5 months agoruntime: fix MapCycle test
Keith Randall [Fri, 8 Nov 2024 23:48:33 +0000 (15:48 -0800)]
runtime: fix MapCycle test

It wasn't actually testing what it says it was testing.
A random permutation isn't cyclic. It only probably hits a few
elements before entering a cycle.
Use an algorithm that generates a random cyclic permutation instead.

Fixing the test makes the previous CL look less good. But it still helps.

(Theory: Fixing the test makes it less cache friendly, so there are
more misses all around. That makes the benchmark slower, suppressing
the differences seen. Also fixing the benchmark makes the loop
iteration count less predictable, which hurts the raw loop
implementation somewhat.)

(baseline = tip, experiment = tip+previous CL, noswiss = GOEXPERIMENT=noswissmap)

goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M2 Ultra
            │  baseline   │             experiment             │
            │   sec/op    │   sec/op     vs base               │
MapCycle-24   20.59n ± 4%   18.99n ± 3%  -7.77% (p=0.000 n=10)
khr@Mac-Studio src % benchstat noswiss experiment
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M2 Ultra
            │   noswiss   │             experiment              │
            │   sec/op    │   sec/op     vs base                │
MapCycle-24   16.12n ± 1%   18.99n ± 3%  +17.83% (p=0.000 n=10)

Change-Id: I3a4edb814ba97fec020a6698c535ce3a87a9fc67
Reviewed-on: https://go-review.googlesource.com/c/go/+/625900
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
5 months agocrypto/subtle: document and test XORBytes overlap rules
Filippo Valsorda [Thu, 24 Oct 2024 14:27:44 +0000 (16:27 +0200)]
crypto/subtle: document and test XORBytes overlap rules

XORBytes doesn't say anything about how it deals with destination and
source overlaps. Current implementations as written do work if the
destination overlaps perfectly with a source, but will unavoidably
return nonsensical results if the destination is ahead of the source.
Lock in the current behavior with tests, docs, and panics.

Note that this introduces a new panic, but if any applications run into
it we are potentially catching a security issue.

Also, expand the tests and move them outside the FIPS module per #69536
convention. (We want to minimize changes within the module boundary.)

Updates #53021

Change-Id: Ibb0875fd38da3818079e31b83b1a227b53755930
Reviewed-on: https://go-review.googlesource.com/c/go/+/622276
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocmd/compile: avoid static init of strings in FIPS mode
Russ Cox [Mon, 18 Nov 2024 00:53:18 +0000 (19:53 -0500)]
cmd/compile: avoid static init of strings in FIPS mode

Strings have relocations, and data relocations are bad.
Other literals are fine.

Fixes build failure in pending CL 628776.

Change-Id: I7a38bbff9776a365c5823d54c4a00e068dda5d9a
Reviewed-on: https://go-review.googlesource.com/c/go/+/628915
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>

5 months agocmd/internal/obj: exclude external test packages from FIPS scope
Russ Cox [Thu, 14 Nov 2024 18:37:38 +0000 (13:37 -0500)]
cmd/internal/obj: exclude external test packages from FIPS scope

Excluding external test packages allows them to use
//go:embed, which requires data relocations in data.
(Obviously the external test code is testing the FIPS module,
not part of it, so this is reasonable.)

Change-Id: I4bae71320ccb5faf718c045540a9ba6dd93e378f
Reviewed-on: https://go-review.googlesource.com/c/go/+/628735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
5 months agonet/http/httputil: return after handling error
Sean Liao [Wed, 13 Nov 2024 18:54:54 +0000 (18:54 +0000)]
net/http/httputil: return after handling error

Fixes #70237

Change-Id: Ieb22b6e7284cb4a40d4987b0e4b9cfb6e5158161
Reviewed-on: https://go-review.googlesource.com/c/go/+/627635
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
5 months agocrypto/cipher: add small CTR benchmark, remove CFB/OFB benchmarks
Filippo Valsorda [Sat, 26 Oct 2024 17:45:41 +0000 (19:45 +0200)]
crypto/cipher: add small CTR benchmark, remove CFB/OFB benchmarks

CFB and OFB are mostly unused, and not a performance target.

Updates #39365
Updates #69445

Change-Id: Ice6441e4fee2112a9e72607c63e49dbc50441ba6
Reviewed-on: https://go-review.googlesource.com/c/go/+/621957
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
5 months agocrypto/internal/impl: use base package name
Filippo Valsorda [Sun, 3 Nov 2024 12:05:57 +0000 (13:05 +0100)]
crypto/internal/impl: use base package name

Otherwise we risk using crypto/aes in one place and
crypto/internal/fips/aes in another.

Change-Id: I8f498c9457875a9a11c4576281432b5b1c0278c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/624737
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocrypto/internal/fips/subtle: move constant time functions from crypto/subtle
Filippo Valsorda [Sun, 3 Nov 2024 12:44:20 +0000 (13:44 +0100)]
crypto/internal/fips/subtle: move constant time functions from crypto/subtle

Change-Id: I267a3cac168fc0366fafac4c26e6a80ca545436a
Reviewed-on: https://go-review.googlesource.com/c/go/+/624755
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
5 months agocrypto/internal/alias: move to crypto/internal/fips/alias
Filippo Valsorda [Thu, 24 Oct 2024 14:10:53 +0000 (16:10 +0200)]
crypto/internal/alias: move to crypto/internal/fips/alias

For #69536

Change-Id: Id0bb46fbb39c205ebc903e72e706bbbaaeec6dbd
Reviewed-on: https://go-review.googlesource.com/c/go/+/622275
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
5 months agocmd/go: fix TestScript/build_json on Windows
Austin Clements [Mon, 18 Nov 2024 03:18:02 +0000 (22:18 -0500)]
cmd/go: fix TestScript/build_json on Windows

The file system directory separator affects this test's output.

Fixes #70397.

Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-linux-amd64-longtest
Change-Id: Id8a3f3e1c26c94ea8f635df4e62e7b0cc2cd5515
Reviewed-on: https://go-review.googlesource.com/c/go/+/628916
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agotext/template: don't crash piping to call with no arguments
Ian Lance Taylor [Thu, 14 Nov 2024 17:17:15 +0000 (09:17 -0800)]
text/template: don't crash piping to call with no arguments

Fixes #70341

Change-Id: I792b15d5e8d08c3762659fbcdfb3d620b59071ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/628096
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Zxilly Chou <zhouxinyu1001@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
5 months agointernal/poll: use ignoringEINTR2 in copyFileRange
Tobias Klauser [Fri, 15 Nov 2024 10:17:37 +0000 (11:17 +0100)]
internal/poll: use ignoringEINTR2 in copyFileRange

Change-Id: Id1b2cc4087885f01807f364ce107e4c9421a2ab8
Reviewed-on: https://go-review.googlesource.com/c/go/+/628295
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
5 months agoos: use ignoringEINTR in (*Process).blockUntilWaitable
Tobias Klauser [Fri, 15 Nov 2024 15:34:13 +0000 (16:34 +0100)]
os: use ignoringEINTR in (*Process).blockUntilWaitable

Instead of open-coding it.

Change-Id: I7430066550a82e5d69846a1ec08b74474207c006
Reviewed-on: https://go-review.googlesource.com/c/go/+/627478
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>

5 months agocmd/go: temporarily disable build JSON in test output
Austin Clements [Mon, 18 Nov 2024 03:41:03 +0000 (22:41 -0500)]
cmd/go: temporarily disable build JSON in test output

Unfortunately, this is tripping up the LUCI test output processor, so
we need to disable it until we can figure that out.

For #70402.
Updates #62067.

Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64_13,gotip-linux-amd64-longtest
Change-Id: I9ae722218e98b8060b8b4c46358f23381ac8537a
Reviewed-on: https://go-review.googlesource.com/c/go/+/628955
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
5 months agointernal/runtime/maps: fix noswiss builder
khr@golang.org [Sun, 17 Nov 2024 21:24:43 +0000 (13:24 -0800)]
internal/runtime/maps: fix noswiss builder

Missed initializing a field in the stub that lets the noswiss
builder test the swiss implementation.

Change-Id: Ie093478ad3e4301e4fe88ba65c132a9dbccd89a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/628895
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agoruntime: relax TestWindowsStackMemory from 100kB to 128kB
Russ Cox [Sun, 17 Nov 2024 15:20:18 +0000 (10:20 -0500)]
runtime: relax TestWindowsStackMemory from 100kB to 128kB

We've been getting intermittent flakes in this test since 2023,
all reporting values just barely over 100kB on windows-386.

If we were happy with 100kB, we should be happy with 128kB,
and it should fix the flakes.

Fixes #58570.

Change-Id: Iabe734cfbba6fe28a83f62e7811ee03fed424f0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/628795
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
5 months agoruntime/internal/maps: remove entryMask
Keith Randall [Mon, 4 Nov 2024 23:52:02 +0000 (15:52 -0800)]
runtime/internal/maps: remove entryMask

It is easily recomputed as capacity-1.

This reduces a table from 40 to 32 bytes (on 64-bit archs).
That gets us down one sizeclass.

Change-Id: Icb74fb2de50baa18ca62052c7b2fe8e6af4c8837
Reviewed-on: https://go-review.googlesource.com/c/go/+/625198
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>
5 months agointernal/runtime/maps: simplify small group lookup
Keith Randall [Fri, 15 Nov 2024 01:42:26 +0000 (17:42 -0800)]
internal/runtime/maps: simplify small group lookup

We don't really need the index of the slot we're looking at.
Just keep looking until there are no more filled slots.

This particularly helps when there are only a few filled entries
(packed at the bottom), and we're looking for something that isn't
there. We exit earlier than we would otherwise.

goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M2 Ultra
                                                 │  baseline   │              experiment               │
                                                 │   sec/op    │   sec/op     vs base                  │
MapSmallAccessHit/Key=int64/Elem=int64/len=1-24    2.759n ± 0%   2.779n ± 2%        ~ (p=0.055 n=10)
MapSmallAccessHit/Key=int64/Elem=int64/len=2-24    2.862n ± 1%   2.922n ± 1%   +2.08% (p=0.000 n=10)
MapSmallAccessHit/Key=int64/Elem=int64/len=3-24    3.003n ± 0%   3.061n ± 1%   +1.91% (p=0.000 n=10)
MapSmallAccessHit/Key=int64/Elem=int64/len=4-24    3.170n ± 1%   3.188n ± 1%   +0.57% (p=0.030 n=10)
MapSmallAccessHit/Key=int64/Elem=int64/len=5-24    3.387n ± 1%   3.391n ± 1%        ~ (p=0.362 n=10)
MapSmallAccessHit/Key=int64/Elem=int64/len=6-24    3.601n ± 1%   3.584n ± 0%   -0.49% (p=0.009 n=10)
MapSmallAccessHit/Key=int64/Elem=int64/len=7-24    3.785n ± 1%   3.778n ± 3%        ~ (p=0.987 n=10)
MapSmallAccessHit/Key=int64/Elem=int64/len=8-24    3.960n ± 1%   3.946n ± 1%        ~ (p=0.256 n=10)
MapSmallAccessMiss/Key=int64/Elem=int64/len=0-24   2.004n ± 1%
MapSmallAccessMiss/Key=int64/Elem=int64/len=1-24   5.145n ± 1%   2.411n ± 1%  -53.14% (p=0.000 n=10)
MapSmallAccessMiss/Key=int64/Elem=int64/len=2-24   5.128n ± 0%   3.313n ± 1%  -35.40% (p=0.000 n=10)
MapSmallAccessMiss/Key=int64/Elem=int64/len=3-24   5.159n ± 1%   3.690n ± 1%  -28.48% (p=0.000 n=10)
MapSmallAccessMiss/Key=int64/Elem=int64/len=4-24   5.117n ± 1%   4.466n ± 6%  -12.73% (p=0.000 n=10)
MapSmallAccessMiss/Key=int64/Elem=int64/len=5-24   5.115n ± 1%   4.308n ± 1%  -15.79% (p=0.000 n=10)
MapSmallAccessMiss/Key=int64/Elem=int64/len=6-24   5.111n ± 1%   4.538n ± 2%  -11.19% (p=0.000 n=10)
MapSmallAccessMiss/Key=int64/Elem=int64/len=7-24   4.896n ± 4%   4.831n ± 1%   -1.33% (p=0.001 n=10)
MapSmallAccessMiss/Key=int64/Elem=int64/len=8-24   4.905n ± 1%   5.121n ± 1%   +4.40% (p=0.000 n=10)
geomean                                            3.917n        3.631n       -11.11%

Change-Id: Ife26ac457a513af24fa0921b839ee6cd5fed6fba
Reviewed-on: https://go-review.googlesource.com/c/go/+/627717
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
5 months agointernal/runtime/maps: eliminate a load from the hot path
Keith Randall [Fri, 15 Nov 2024 00:58:07 +0000 (16:58 -0800)]
internal/runtime/maps: eliminate a load from the hot path

typ.Group.Size involves two loads.

Instead cache GroupSize as a separate fields of the map type
so we can get to it in just one load.

Change-Id: I10ffdce1c7f75dcf448da14040fda78f0d75fd1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/627716
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agoruntime/internal/maps: optimize long string keys for small maps
Keith Randall [Sat, 9 Nov 2024 17:49:40 +0000 (09:49 -0800)]
runtime/internal/maps: optimize long string keys for small maps

For large strings, do a quick equality check on all the slots.
Only if more than one passes the quick equality check do we
resort to hashing.

                               │    baseline    │             experiment              │
                               │     sec/op     │   sec/op     vs base                │
MegMap-24                        16609.50n ± 1%   13.91n ± 3%  -99.92% (p=0.000 n=10)
MegOneMap-24                     16655.00n ± 0%   12.27n ± 1%  -99.93% (p=0.000 n=10)
MegEqMap-24                         41.31µ ± 1%   25.03µ ± 1%  -39.40% (p=0.000 n=10)
MegEmptyMap-24                      2.034n ± 0%   2.027n ± 2%        ~ (p=0.541 n=10)
MegEmptyMapWithInterfaceKey-24      5.931n ± 2%   5.599n ± 1%   -5.60% (p=0.000 n=10)
MapStringKeysEight_16-24            8.473n ± 7%   8.224n ± 5%        ~ (p=0.315 n=10)
MapStringKeysEight_32-24            8.441n ± 2%   8.147n ± 1%   -3.48% (p=0.002 n=10)
MapStringKeysEight_64-24            8.769n ± 1%   8.517n ± 1%   -2.87% (p=0.000 n=10)
MapStringKeysEight_128-24           10.73n ± 4%   13.57n ± 8%  +26.57% (p=0.000 n=10)
MapStringKeysEight_256-24           12.97n ± 2%   14.35n ± 4%  +10.64% (p=0.001 n=10)
MapStringKeysEight_1M-24         17359.50n ± 3%   13.92n ± 4%  -99.92% (p=0.000 n=10)

Change-Id: I4cc2ea4edab12a4b03236de626c7bcf0f96b6cc0
Reviewed-on: https://go-review.googlesource.com/c/go/+/625905
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
5 months agocmd/go: change Printer.Output -> Printer.Printf for consistency
Austin Clements [Thu, 14 Nov 2024 00:50:44 +0000 (19:50 -0500)]
cmd/go: change Printer.Output -> Printer.Printf for consistency

Currently, the Printer interface has `Output`, which acts like Print
and `Errorf`, which acts like Printf. It's confusing that the
formatting style is tied to whether it's regular output or an error.

Fix this by replacing Output with Printf, so both use Printf-style
formatting.

Change-Id: I4c76f941e956f2599c5620b455bf41e21636b44e
Reviewed-on: https://go-review.googlesource.com/c/go/+/627795
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocmd/go: print build errors during go test -json in JSON
Austin Clements [Tue, 17 Oct 2023 21:27:00 +0000 (17:27 -0400)]
cmd/go: print build errors during go test -json in JSON

Currently, if a test or imported package fails to build during "go
test -json", the build error text will be interleaved with the JSON
output of tests. Furthermore, there’s currently no way to reliably
associate a build error with the test package or packages it affected.
This creates unnecessary friction and complexity in tools that consume
the "go test -json" output.

This CL makes "go test -json" enable JSON reporting of build errors.
It also adds a "FailedBuild" field to the "fail" TestEvent, which
gives the package ID of the package that failed to build and caused
the test to fail.

Using this, CI systems should be able to consume the entire output
stream from "go test -json" in a structured way and easily associate
build failures with test failures during reporting.

Fixes #62067.
Updates #35169.
Updates #37486.

Change-Id: I49091dcc7aa52db01fc9fa6042771633e97b8407
Reviewed-on: https://go-review.googlesource.com/c/go/+/536399
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocmd/go: track root failing Action
Austin Clements [Tue, 17 Oct 2023 21:21:47 +0000 (17:21 -0400)]
cmd/go: track root failing Action

Currently, each Action tracks whether it failed, which is propagated
up from dependencies. Shortly, we'll need to know the root cause if a
test fails because of a build failure. To support this, replace the
Failed boolean with a Failed *Action that tracks the root Action that
failed and caused other Actions to fail.

For #62067.

Change-Id: I8f84a51067354043ae9531a4368c6f8b11d688d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/536398
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocmd/go: report all loading errors in tests as "setup failed"
Austin Clements [Fri, 26 Jan 2024 19:45:30 +0000 (14:45 -0500)]
cmd/go: report all loading errors in tests as "setup failed"

Currently, under *most* circumstances, if there's a package loading
error during "go test", that will get reported as a "FAIL p [setup
failed]" or "FAIL p [build failed] message and won't prevent running
unaffected test packages.

However, if there's a loading error from a non-test file in a package
listed directly on the "go test" command line, that gets reported as
an immediate fatal error, without any "FAIL" line, and without
attempting to run other tests listed on the command line. Likewise,
certain early build errors (like a package containing no Go files) are
currently immediately fatal rather than reporting a test failure.

Fix this by eliminating the check that causes that immediate failure.
This causes one minor follow-up problem: since
load.TestPackagesAndErrors was never passed a top-level package with
an error before, it doesn't currently propagate such an error to the
packages it synthesizes (even though it will propagate errors in
imported packages). Fix this by copying the error from the top-level
package into the synthesized test package while we're copying
everything else.

For #62067.

Change-Id: Icd563a3d9912256b53afd998050995e5260ebe5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/558637
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
5 months agocmd/go: implement "go build -json"
Austin Clements [Thu, 24 Aug 2023 17:29:39 +0000 (13:29 -0400)]
cmd/go: implement "go build -json"

This adds support for a "-json" flag in all build-related go
subcommands. This causes build output and build failures to be
reported to stdout in a machine-readable way.

For #62067.
Fixes #23037.

Change-Id: Id045c5bd5dde9d16cc09dde6248a4b9637896a30
Reviewed-on: https://go-review.googlesource.com/c/go/+/536397
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Russ Cox <rsc@golang.org>
5 months agocmd/go: add Printer interface and use for error reporting
Austin Clements [Tue, 17 Oct 2023 20:30:36 +0000 (16:30 -0400)]
cmd/go: add Printer interface and use for error reporting

This replaces the existing Shell print function callback. The
interface also gives us a way to report build failures, which is the
other type of event that will appear in the build -json output.

This CL hooks up error reporting in two places:

- In Builder.Do, where all builder errors are reported.

- In load.CheckPackageErrors, where most loading errors are reported.

For #62067.

Change-Id: Id66a31b0d2c3786559c7d2bb376fffeffc9a66ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/536396
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agocrypto/internal/fips/check: remove unreachable return
Tim King [Fri, 15 Nov 2024 23:53:13 +0000 (15:53 -0800)]
crypto/internal/fips/check: remove unreachable return

return statement is immediately after a panic. This is unreachable
code and vet complains.

Change-Id: I1e483a552b0d1d543414c1e173c1140aa32b5b26
Reviewed-on: https://go-review.googlesource.com/c/go/+/628555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>

5 months agocrypto/sha512: improve performance of riscv64 assembly
Joel Sing [Wed, 14 Aug 2024 14:39:59 +0000 (00:39 +1000)]
crypto/sha512: improve performance of riscv64 assembly

Implement optimised versions of Maj and Ch, which reduce the number of
instructions required per round. Reorder instructions for better
interleaving.

This gives around a 10% gain on a StarFive VisionFive 2:

                    │  sha512.1   │              sha512.2               │
                    │   sec/op    │   sec/op     vs base                │
Hash8Bytes/New-4      9.310µ ± 0%   8.564µ ± 0%   -8.01% (p=0.000 n=10)
Hash8Bytes/Sum384-4   8.833µ ± 0%   7.980µ ± 0%   -9.66% (p=0.000 n=10)
Hash8Bytes/Sum512-4   9.293µ ± 0%   8.162µ ± 0%  -12.17% (p=0.000 n=10)
Hash1K/New-4          49.60µ ± 0%   44.33µ ± 0%  -10.63% (p=0.000 n=10)
Hash1K/Sum384-4       48.93µ ± 0%   43.78µ ± 0%  -10.53% (p=0.000 n=10)
Hash1K/Sum512-4       49.48µ ± 0%   43.96µ ± 0%  -11.15% (p=0.000 n=10)
Hash8K/New-4          327.9µ ± 0%   292.6µ ± 0%  -10.78% (p=0.000 n=10)
Hash8K/Sum384-4       327.3µ ± 0%   292.0µ ± 0%  -10.77% (p=0.000 n=10)
Hash8K/Sum512-4       327.8µ ± 0%   292.2µ ± 0%  -10.85% (p=0.000 n=10)
geomean               52.87µ        47.31µ       -10.51%

                    │   sha512.1   │               sha512.2               │
                    │     B/s      │     B/s       vs base                │
Hash8Bytes/New-4      839.8Ki ± 0%   908.2Ki ± 0%   +8.14% (p=0.000 n=10)
Hash8Bytes/Sum384-4   888.7Ki ± 1%   976.6Ki ± 0%   +9.89% (p=0.000 n=10)
Hash8Bytes/Sum512-4   839.8Ki ± 0%   957.0Ki ± 0%  +13.95% (p=0.000 n=10)
Hash1K/New-4          19.69Mi ± 0%   22.03Mi ± 0%  +11.86% (p=0.000 n=10)
Hash1K/Sum384-4       19.96Mi ± 0%   22.31Mi ± 0%  +11.75% (p=0.000 n=10)
Hash1K/Sum512-4       19.74Mi ± 0%   22.21Mi ± 0%  +12.51% (p=0.000 n=10)
Hash8K/New-4          23.82Mi ± 0%   26.70Mi ± 0%  +12.09% (p=0.000 n=10)
Hash8K/Sum384-4       23.87Mi ± 0%   26.75Mi ± 0%  +12.07% (p=0.000 n=10)
Hash8K/Sum512-4       23.83Mi ± 0%   26.73Mi ± 0%  +12.16% (p=0.000 n=10)
geomean               7.334Mi        8.184Mi       +11.59%

Change-Id: I66e359e96b25b38efbc4d840e6b2d6a1e5d417ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/605495
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
5 months agoruntime/pprof: reduce label overhead
Felix Geisendörfer [Tue, 26 Mar 2024 19:23:42 +0000 (20:23 +0100)]
runtime/pprof: reduce label overhead

Switch labelMap from map[string]string to use LabelSet as a data
structure. Optimize Labels() for the case where the keys are given in
sorted order without duplicates.

This is primarily motivated by reducing the overhead of distributed
tracing systems that use pprof labels. We have encountered cases where
users complained about the overhead relative to the rest of our
distributed tracing library code. Additionally, we see this as an
opportunity to free up hundreds of CPU cores across our fleet.

A secondary motivation is eBPF profilers that try to access pprof
labels. The current map[string]string requires them to implement Go map
access in eBPF, which is non-trivial. With the enablement of swiss maps,
this complexity is only increasing. The slice data structure introduced
in this CL will greatly lower the implementation complexity for eBPF
profilers in the future. But to be clear: This change does not imply
that the pprof label mechanism is now a stable ABI. They are still an
implementation detail and may change again in the future.

goos: darwin
goarch: arm64
pkg: runtime/pprof
cpu: Apple M1 Max
                                   │ baseline.txt │             patch1.txt              │
                                   │    sec/op    │   sec/op     vs base                │
Labels/set-one-10                    153.50n ± 3%   75.00n ± 1%  -51.14% (p=0.000 n=10)
Labels/merge-one-10                   187.8n ± 1%   128.8n ± 1%  -31.42% (p=0.000 n=10)
Labels/overwrite-one-10               193.1n ± 2%   102.0n ± 1%  -47.18% (p=0.000 n=10)
Labels/ordered/set-many-10            502.6n ± 4%   146.1n ± 2%  -70.94% (p=0.000 n=10)
Labels/ordered/merge-many-10          516.3n ± 2%   238.1n ± 1%  -53.89% (p=0.000 n=10)
Labels/ordered/overwrite-many-10      569.3n ± 4%   247.6n ± 2%  -56.51% (p=0.000 n=10)
Labels/unordered/set-many-10          488.9n ± 2%   308.3n ± 3%  -36.94% (p=0.000 n=10)
Labels/unordered/merge-many-10        523.6n ± 1%   258.5n ± 1%  -50.64% (p=0.000 n=10)
Labels/unordered/overwrite-many-10    571.4n ± 1%   412.1n ± 2%  -27.89% (p=0.000 n=10)
geomean                               366.8n        186.9n       -49.05%

                                   │ baseline.txt │             patch1b.txt              │
                                   │     B/op     │     B/op      vs base                │
Labels/set-one-10                      424.0 ± 0%     104.0 ± 0%  -75.47% (p=0.000 n=10)
Labels/merge-one-10                    424.0 ± 0%     200.0 ± 0%  -52.83% (p=0.000 n=10)
Labels/overwrite-one-10                424.0 ± 0%     136.0 ± 0%  -67.92% (p=0.000 n=10)
Labels/ordered/set-many-10            1344.0 ± 0%     392.0 ± 0%  -70.83% (p=0.000 n=10)
Labels/ordered/merge-many-10          1184.0 ± 0%     712.0 ± 0%  -39.86% (p=0.000 n=10)
Labels/ordered/overwrite-many-10      1056.0 ± 0%     712.0 ± 0%  -32.58% (p=0.000 n=10)
Labels/unordered/set-many-10          1344.0 ± 0%     712.0 ± 0%  -47.02% (p=0.000 n=10)
Labels/unordered/merge-many-10        1184.0 ± 0%     712.0 ± 0%  -39.86% (p=0.000 n=10)
Labels/unordered/overwrite-many-10   1.031Ki ± 0%   1.008Ki ± 0%   -2.27% (p=0.000 n=10)
geomean                                843.1          405.1       -51.95%

                                   │ baseline.txt │             patch1b.txt              │
                                   │  allocs/op   │ allocs/op   vs base                  │
Labels/set-one-10                      5.000 ± 0%   3.000 ± 0%  -40.00% (p=0.000 n=10)
Labels/merge-one-10                    5.000 ± 0%   5.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels/overwrite-one-10                5.000 ± 0%   4.000 ± 0%  -20.00% (p=0.000 n=10)
Labels/ordered/set-many-10             8.000 ± 0%   3.000 ± 0%  -62.50% (p=0.000 n=10)
Labels/ordered/merge-many-10           8.000 ± 0%   5.000 ± 0%  -37.50% (p=0.000 n=10)
Labels/ordered/overwrite-many-10       7.000 ± 0%   4.000 ± 0%  -42.86% (p=0.000 n=10)
Labels/unordered/set-many-10           8.000 ± 0%   4.000 ± 0%  -50.00% (p=0.000 n=10)
Labels/unordered/merge-many-10         8.000 ± 0%   5.000 ± 0%  -37.50% (p=0.000 n=10)
Labels/unordered/overwrite-many-10     7.000 ± 0%   5.000 ± 0%  -28.57% (p=0.000 n=10)
geomean                                6.640        4.143       -37.60%
¹ all samples are equal

Change-Id: Ie68e960a25c2d97bcfb6239dc481832fa8a39754
Reviewed-on: https://go-review.googlesource.com/c/go/+/574516
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
5 months agoos: add and use ignoringEINTR2
Tobias Klauser [Fri, 15 Nov 2024 19:41:33 +0000 (20:41 +0100)]
os: add and use ignoringEINTR2

Copy ignoringEINTR2 from internal/poll and make use of it to remove
open-coded implementations.

Change-Id: I8802862f2012980f2af445b75eb45bb5a97bcc2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/627479
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agofmt: document nil receiver handling for GoStringer
Sean Liao [Wed, 13 Nov 2024 15:04:37 +0000 (15:04 +0000)]
fmt: document nil receiver handling for GoStringer

Fixes #70305

Change-Id: I8ae4e6dae3327a54039d470c8c8545e2cc6de98f
Reviewed-on: https://go-review.googlesource.com/c/go/+/627495
Reviewed-by: Rob Pike <r@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
5 months agoruntime: implement Stop for AddCleanup
Carlos Amedee [Thu, 14 Nov 2024 14:56:49 +0000 (09:56 -0500)]
runtime: implement Stop for AddCleanup

This change adds the implementation for AddCleanup.Stop. It allows the
caller to cancel the call to execute the cleanup. Cleanup will not be
stopped if the cleanup has already been queued for execution.

For #67535

Change-Id: I494b77d344e54d772c41489d172286773c3814e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/627975
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>

5 months agoruntime: implement AddCleanup
Carlos Amedee [Wed, 13 Nov 2024 20:25:41 +0000 (15:25 -0500)]
runtime: implement AddCleanup

This change introduces AddCleanup to the runtime package. AddCleanup attaches
a cleanup function to an pointer to an object.

The Stop method on Cleanups will be implemented in a followup CL.

AddCleanup is intended to be an incremental improvement over
SetFinalizer and will result in SetFinalizer being deprecated.

For #67535

Change-Id: I99645152e3fdcee85fcf42a4f312c6917e8aecb1
Reviewed-on: https://go-review.googlesource.com/c/go/+/627695
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
5 months agoruntime/pprof: continued attempt to deflake the VMInfo test.
Cosmos Nicolaou [Sat, 3 Feb 2024 01:12:27 +0000 (17:12 -0800)]
runtime/pprof: continued attempt to deflake the VMInfo test.

This PR will use test.Skip to bypass a test run for which the vmmap
subprocess appears to hang before the test times out.
In addition it catches a different error message from vmmap that can
occur due to transient resource shortages and triggers a retry for
this additional case.

Fixes #62352

Change-Id: I3ae749e5cd78965c45b1b7c689b896493aa37ba0
Reviewed-on: https://go-review.googlesource.com/c/go/+/560935
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

5 months agoruntime: improve CALLFN macro for loong64
Xiaolin Zhao [Tue, 12 Nov 2024 07:50:03 +0000 (15:50 +0800)]
runtime: improve CALLFN macro for loong64

The previous CALLFN macro was copying a single byte at a time
which is inefficient on loong64. In this CL, according to the
argsize, copy 16 bytes or 8 bytes at a time, and copy 1 byte
a time for the rest.

benchmark in reflect on 3A5000 and 3A6000:

goos: linux
goarch: loong64
pkg: reflect
cpu: Loongson-3A6000 @ 2500.00MHz
                       |  bench.old   |              bench.new              |
                       |    sec/op    |   sec/op     vs base                |
CallArgCopy/size=128      360.2n ± 0%   266.9n ± 0%  -25.90% (p=0.000 n=20)
CallArgCopy/size=256      473.2n ± 0%   277.5n ± 0%  -41.35% (p=0.000 n=20)
CallArgCopy/size=1024    1128.0n ± 0%   332.9n ± 0%  -70.49% (p=0.000 n=20)
CallArgCopy/size=4096    3743.0n ± 0%   672.6n ± 0%  -82.03% (p=0.000 n=20)
CallArgCopy/size=65536   58.888µ ± 0%   9.667µ ± 0%  -83.58% (p=0.000 n=20)
geomean                   2.116µ        693.4n       -67.22%

                       |  bench.old   |               bench.new                |
                       |     B/s      |      B/s       vs base                 |
CallArgCopy/size=128     338.9Mi ± 0%    457.3Mi ± 0%   +34.94% (p=0.000 n=20)
CallArgCopy/size=256     516.0Mi ± 0%    879.8Mi ± 0%   +70.52% (p=0.000 n=20)
CallArgCopy/size=1024    865.5Mi ± 0%   2933.6Mi ± 0%  +238.94% (p=0.000 n=20)
CallArgCopy/size=4096    1.019Gi ± 0%    5.672Gi ± 0%  +456.52% (p=0.000 n=20)
CallArgCopy/size=65536   1.036Gi ± 0%    6.313Gi ± 0%  +509.13% (p=0.000 n=20)
geomean                  699.6Mi         2.085Gi       +205.10%

goos: linux
goarch: loong64
pkg: reflect
cpu: Loongson-3A5000 @ 2500.00MHz
                       |  bench.old   |              bench.new              |
                       |    sec/op    |   sec/op     vs base                |
CallArgCopy/size=128      466.6n ± 0%   368.7n ± 0%  -20.98% (p=0.000 n=20)
CallArgCopy/size=256      579.4n ± 0%   384.6n ± 0%  -33.62% (p=0.000 n=20)
CallArgCopy/size=1024    1273.0n ± 0%   492.0n ± 0%  -61.35% (p=0.000 n=20)
CallArgCopy/size=4096    4049.0n ± 0%   978.1n ± 0%  -75.84% (p=0.000 n=20)
CallArgCopy/size=65536    69.01µ ± 0%   14.50µ ± 0%  -78.99% (p=0.000 n=20)
geomean                   2.492µ        997.9n       -59.96%

                       |  bench.old   |               bench.new                |
                       |     B/s      |      B/s       vs base                 |
CallArgCopy/size=128     261.6Mi ± 0%    331.0Mi ± 0%   +26.54% (p=0.000 n=20)
CallArgCopy/size=256     421.4Mi ± 0%    634.8Mi ± 0%   +50.66% (p=0.000 n=20)
CallArgCopy/size=1024    767.2Mi ± 0%   1985.0Mi ± 0%  +158.75% (p=0.000 n=20)
CallArgCopy/size=4096    964.8Mi ± 0%   3993.8Mi ± 0%  +313.95% (p=0.000 n=20)
CallArgCopy/size=65536   905.7Mi ± 0%   4310.6Mi ± 0%  +375.97% (p=0.000 n=20)
geomean                  593.9Mi         1.449Gi       +149.76%

Change-Id: I9570395af80b2e4b760058098a1b5b07d4b37ad7
Reviewed-on: https://go-review.googlesource.com/c/go/+/627175
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
5 months agoruntime: use ABIInternal for calls to sigtrampgo on linux/loong64
Guoqi Chen [Fri, 17 May 2024 10:05:11 +0000 (18:05 +0800)]
runtime: use ABIInternal for calls to sigtrampgo on linux/loong64

Change-Id: I13fd5a96daff66a2ecb54f5bafa3d6e5c60f3879
Reviewed-on: https://go-review.googlesource.com/c/go/+/586357
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: sophie zhao <zhaoxiaolin@loongson.cn>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Qiqi Huang <huangqiqi@loongson.cn>