cmd/go: add global ignore mechanism

This CL adds the ignore directive which enables users to tell the Go
Command to skip traversing into a given directory.

This behaves similar to how '_' or 'testdata' are currently treated.
This mainly has benefits for go list and go mod tidy.
This does not affect what is packed into a module.

Fixes: #42965
Change-Id: I232e27c1a065bb6eb2d210dbddad0208426a1fdd
Reviewed-on: https://go-review.googlesource.com/c/go/+/643355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@google.com>

runtime: prevent cleanup goroutines from missing work

Currently, there's a window of time where each cleanup goroutine has
committed to going to sleep (immediately after full.pop() == nil) but
hasn't yet marked itself as asleep (state.sleep()). If new work arrives
in this window, it might get missed. This is what we see in #73642, and
I can reproduce it with stress2.

Side-note: even if the work gets missed by the existing sleeping
goroutines, needg is incremented. So in theory a new goroutine will
handle the work. Right now that doesn't happen in tests like the one
running in #73642, where there might never be another call to AddCleanup
to create the additional goroutine. Also, if we've hit the maximum on
cleanup goroutines and all of them are in this window simultaneously, we
can still end up missing work, it's just more rare. So this is still a
problem even if we choose to just be more aggressive about creating new
cleanup goroutines.

This change fixes the problem and also aims to make the cleanup
wake/sleep code clearer. The way this change fixes this problem is to
have cleanup goroutines re-check the work list before going to sleep,
but after having already marked themselves as sleeping. This way, if new
work comes in before the cleanup goroutine marks itself as going to
sleep, we can rely on the re-check to pick up that work. If new work
comes after the goroutine marks itself as going to sleep and after the
re-check, we can rely on the scheduler noticing that the goroutine is
asleep and waking it up. If work comes in between a goroutine marking
itself as sleeping and the re-check, then the re-check will catch that
piece of work. However, the scheduler might now get a false signal that
the goroutine is asleep and try to wake it up. This is OK. The sleeping
signal is now mutated and double-checked under the queue lock, so the
scheduler will grab the lock, may notice there are no sleeping
goroutines, and go on its way. This may cause spurious lock acquisitions
but it should be very rare. The window between a cleanup goroutine
marking itself as going to sleep and re-checking the work list is a
handful of instructions at most.

This seems subtle but overall it's a simplification of the code. We
rely more on the lock, which is easier to reason about, and we track two
separate atomic variables instead of the merged cleanupSleepState: the
length of the full list, and the number of cleanup goroutines that are
asleep. The former is now the primary way to acquire work. Cleanup
goroutines must decrement the length successfully to obtain an item off
the full list. The number of cleanup goroutines asleep, meanwhile, is
now only updated with the queue lock held. It can be checked without the
lock held, and the invariant to make that safe is simple: it must always
be an overestimate of the number of sleeping cleanup goroutines.

The changes here do change some other behaviors.

First, since we're tracking the length of the full list instead of the
abstract concept of a wake-up, the waker can't consume wake-ups anymore.
This means that cleanup goroutines may be created more aggressively. If
two threads in the scheduler see that there are goroutines that are
asleep, only one will win the race, but the other will observe zero
asleep goroutines but potentially many work units available. This will
cause it to signal many goroutines to be created. This is OK since we
have a cap on the number of cleanup goroutines, and the race should be
relatively rare.

Second, because cleanup goroutines can now fail to go to sleep if any
units of work come in, they might spend more time contended on the lock.
For example, if we have N cleanup goroutines and work comes in at *just*
the wrong rate, in the worst case we'll have each of G goroutines loop
N times for N blocks, resulting in O(G*N) thread time to handle each
block in the worst case. To paint a picture, imagine each goroutine
trying to go to sleep, fail because a new block of work came in, and
only one goroutine will get that block. Then once that goroutine is
done, we all try again, fail because a new block of work came in, and so
on and so forth. This case is unlikely, though, and probably not worth
worrying about until it actually becomes a problem. (A similar problem
exists with parking (and exists before this change, too) but at least in
that case each goroutine parks, so it doesn't block the thread.)

Fixes #73642.

Change-Id: I6bbe1b789e7eb7e8168e56da425a6450fbad9625
Reviewed-on: https://go-review.googlesource.com/c/go/+/671676
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>

cmd/internal/obj/loong64: change the plan9 format of the prefetch instruction PRELDX

before:
    MOVV    $n + $offset, Roff
    PRELDX  (Rbase)(Roff), $hint
after:
    PRELDX  offset(Rbase), $n, $hint

This instruction is supported in CL 671875, but is not actually used

Change-Id: I943d488ea6dc77781cd796ef480a89fede666bab
Reviewed-on: https://go-review.googlesource.com/c/go/+/673155
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: sophie zhao <zhaoxiaolin@loongson.cn>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

os: set FILE_FLAG_BACKUP_SEMANTICS when opening without I/O access

FILE_FLAG_BACKUP_SEMANTICS is necessary to open directories on Windows,
and to enable backup applications do extended operations on files if
they hold the SE_BACKUP_NAME and SE_RESTORE_NAME privileges.

os.OpenFile currently sets FILE_FLAG_BACKUP_SEMANTICS for all supported
cases except when the file is opened with O_WRONLY | O_RDWR (that is,
access mode 3). This access mode doesn't correspond to any of the
standard POSIX access modes, but some OSes special case it to mean
different things. For example, on Linux, O_WRONLY | O_RDWR means check
for read and write permission on the file and return a file descriptor
that can't be used for reading or writing.

On Windows, os.OpenFile has historically mapped O_WRONLY | O_RDWR to a
0 access mode, which Windows internally interprets as
FILE_READ_ATTRIBUTES. Additionally, it doesn't prepare the file for I/O,
given that the read attributes permission doesn't allow reading or
writing (not that this is similar to what happens on Linux). This
makes opening the file around 50% faster, and one can still use the
handle to stat it, so some projects have been using this behavior
to open files without I/O access.

This CL updates os.OpenFile so that directories can also be opened
without I/O access. This effectively closes #23312, as all the remaining
cases where we don't set FILE_FLAG_BACKUP_SEMANTICS imply opening
with O_WRONLY or O_RDWR, and that's not allowed by Unix's open.

Closes #23312.

Change-Id: I77c4f55e1ca377789aef75bd8a9bce2b7499f91d
Reviewed-on: https://go-review.googlesource.com/c/go/+/673035
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

crypto/tls: fix bogo IgnoreClientVersionOrder skip reason

The BoGo IgnoreClientVersionOrder test checks that a client that sends
a supported_versions extension with the list [TLS 1.2, TLS 1.3] ends up
negotiating TLS 1.3.

However, the crypto/tls module treats this list as being in client
preference order, and so negotiates TLS 1.2, failing the test.

Our behaviour appears to be the correct handling based on RFC 8446
§4.2.1 where it says:
  The extension contains a list of supported versions in preference
  order, with the most preferred version first.

This commit updates the reason we skip this test to cite the RFC instead
of saying it's something to be fixed.

Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/671415
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>

cmd/compile: allow load-op merging in additional situations

x += *p

We want to do this with a single load+add operation on amd64.
The tricky part is that we don't want to combine if there are
other uses of x after this instruction.

Implement a simple detector that seems to capture a common situation -
x += *p is in a loop, and the other use of x is after loop exit.
In that case, it does not hurt to do the load+add combo.

Change-Id: I466174cce212e78bde83f908cc1f2752b560c49c
Reviewed-on: https://go-review.googlesource.com/c/go/+/672957
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

strings,bytes: make benchmark work deterministic

It's hard to compare two different runs of a benchmark if they
are doing different amounts of work.

Change-Id: I5d6845f3d11bb10136f745e6207d5f683612276d
Reviewed-on: https://go-review.googlesource.com/c/go/+/672895
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/compile: schedule induction variable increments late

for ..; ..; i++ {
 ...
}

We want to schedule the i++ late in the block, so that all other
uses of i in the block are scheduled first. That way, i++ can
happen in place in a register instead of requiring a temporary register.

Change-Id: Id777407c7e67a5ddbd8e58251099b0488138c0df
Reviewed-on: https://go-review.googlesource.com/c/go/+/672998
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@google.com>

cmd/compile: use OpMove instead of memmove more on arm64

OpMove is faster for small moves of fixed size.

For safety, we have to rewrite the Move rewrite rules a bit so that
all the loads are done before any stores happen.

Also use an 8-byte move instead of a 16-byte move if the tail is
at most 8 bytes.

Change-Id: I7f6c7496ac6d5eb2e0706fd59ca4b5d797c51101
Reviewed-on: https://go-review.googlesource.com/c/go/+/672997
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>

cmd/compile: create LSym for closures with type conversion

Follow-up to #54959 with another failing case.

The linker needs FuncInfo metadata for all inlined functions. CL 436240 explicitly creates LSym for direct closure calls to ensure we keep the FuncInfo metadata.

However, CL 436240 won't work if the direct closure call is wrapped by a no-effect type conversion, even if that closure could be inlined.

This commit should fix such case.

Fixes #73716

Change-Id: Icda6024da54c8d933f87300e691334c080344695
GitHub-Last-Rev: e9aed02eb6ef343e4ed2d8a79f6426abf917ab0e
GitHub-Pull-Request: golang/go#73718
Reviewed-on: https://go-review.googlesource.com/c/go/+/672855
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

testing: fix panic in t.Log

If a testing.TB is no longer on the stack, t.Log would panic because
its outputWriter is nil. Check for nil and drop the write, which
is the previous behavior.

Change-Id: Ifde97997a3aa26ae604ac9c218588c1980110cbf
Reviewed-on: https://go-review.googlesource.com/c/go/+/673215
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>

doc: add a release note for greenteagc GOEXPERIMENT

For #71661.

Change-Id: I802b0c36cac3bbd87b35ff216f06822e87fb7b5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/671439
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

runtime/pprof: include PCs for deduplication in TestMutexBlockFullAggregation

TestMutexBlockFullAggregation aggregates stacks by function, file, and
line number. But there can be multiple function calls on the same line,
giving us different sequences of PCs. This causes the test to spuriously
fail in some cases. Include PCs in the stacks for this test.

Also pick up a small "range over int" modernize suggestion while we're
looking at the test.

Fixes #73641

Change-Id: I50489e19fcf920e27b9eebd9d4b35feb89981cbc
Reviewed-on: https://go-review.googlesource.com/c/go/+/673115
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/link: fix outdated output mmap check

Outbuf.View used to perform a mmap check by default
and return an error if the check failed,
this behavior has been changed so that now
the View never returns any error,
so the usage needs to be modified accordingly.

Change-Id: I76ffcda5476847f6fed59856a5a5161734f47562
GitHub-Last-Rev: 6449f2973d28c3b4a5c9e289c38dfcc38f83b3d9
GitHub-Pull-Request: golang/go#73730
Reviewed-on: https://go-review.googlesource.com/c/go/+/673095
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

net: avoid windows hang in TestCloseWrite

On Windows, reading from a socket at the same time as the other
end is closing it will occasionally hang. This is a Windows issue, not
a Go issue, similar to what happens in macOS (see #49352).

Work around this condition by adding a brief sleep before the read.

Fixes #73140.

Change-Id: I24e457a577e507d0d69924af6ffa1aa24c4aaaa6
Reviewed-on: https://go-review.googlesource.com/c/go/+/671457
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Commit-Queue: Alan Donovan <adonovan@google.com>
Reviewed-by: Damien Neil <dneil@google.com>

os: don't fallback to the Stat slow path if file doesn't exist on Windows

os.Stat and os.Lstat first try stating the file without opening it. If
that fails, then they open the file and try again, operations that tends
to be slow. There is no point in trying the slow path if the file
doesn't exist, we should just return an error immediately.

This CL makes stating a non-existent file on Windows 50% faster:

goos: windows
goarch: amd64
pkg: os
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
                │   old.txt    │                new.txt                 │
                │    sec/op    │    sec/op     vs base                  │
StatNotExist-12   43.65µ ± 15%   20.02µ ± 10%  -54.14% (p=0.000 n=10+7)

                │  old.txt   │             new.txt              │
                │    B/op    │    B/op     vs base              │
StatNotExist-12   224.0 ± 0%   224.0 ± 0%  ~ (p=1.000 n=10+7) ¹
¹ all samples are equal

                │  old.txt   │             new.txt              │
                │ allocs/op  │ allocs/op   vs base              │
StatNotExist-12   2.000 ± 0%   2.000 ± 0%  ~ (p=1.000 n=10+7) ¹

Updates #72992.

Change-Id: Iaeb9596d0d18e5a5a1bd1970e296a3480501af78
Reviewed-on: https://go-review.googlesource.com/c/go/+/671458
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jake Bailey <jacob.b.bailey@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

net: use closesocket when closing socket os.File's on Windows

The WSASocket documentation states that the returned socket must be
closed by calling closesocket instead of CloseHandle. The different
File methods on the net package return an os.File that is not aware
that it should use closesocket. Ideally, os.NewFile should detect that
the passed handle is a socket and use the appropriate close function,
but there is no reliable way to detect that a handle is a socket on
Windows (see CL 671455).

To work around this, we add a hidden function to the os package that
can be used to return an os.File that uses closesocket. This approach
is the same as used on Unix, which also uses a hidden function for other
purposes.

While here, fix a potential issue with FileConn, which was using File.Fd
rather than File.SyscallConn to get the handle. This could result in the
File being closed and garbage collected before the syscall was made.

Fixes #73683.

Change-Id: I179405f34c63cbbd555d8119e0f77157c670eb3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/672195
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

sync: use blockUntilCleanupQueueEmpty instead of busy-looping in tests

testPool currently does the old-style busy loop to wait until cleanups
have executed. Clean this up by using the linkname'd
blockUntilCleanupQueueEmpty.

For #73642.

Change-Id: Ie0c2614db858a984f25b33a805dc52948069eb52
Reviewed-on: https://go-review.googlesource.com/c/go/+/671675
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

runtime: help the race detector detect possible concurrent cleanups

This change makes it so that cleanup goroutines, in race mode, create a
fake race context and switch to it, emulating cleanups running on new
goroutines. This helps in catching races between cleanups that might run
concurrently.

Change-Id: I4c4e33054313798d4ac4e5d91ff2487ea3eb4b16
Reviewed-on: https://go-review.googlesource.com/c/go/+/652635
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>

runtime: improve scan inner loop

On every arch except amd64, it is faster to do x&(x-1) than x^(1<<n).

Most archs need 3 instructions for the latter: MOV $1, R; SLL n, R;
ANDN R, x. Maybe 4 if there's no ANDN.

Most archs need only 2 instructions to do x&(x-1). It takes 3 on
x86/amd64 because NEG only works in place.

Only amd64 can do x^(1<<n) in a single instruction.
(We could on 386 also, but that's currently not implemented.)

Change-Id: I3b74b7a466ab972b20a25dbb21b572baf95c3467
Reviewed-on: https://go-review.googlesource.com/c/go/+/672956
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/compile: fold negation into addition/subtraction on loong64

This change also avoid double negation, and add loong64 codegen for arithmetic tests.
Reduce the number of go toolchain instructions on loong64 as follows.

    file      before    after     Δ       %
    addr2line 279972    279896  -76    -0.0271%
    asm       556390    556310  -80    -0.0144%
    buildid   272376    272300  -76    -0.0279%
    cgo       481534    481550  +16    +0.0033%
    compile   2457992   2457396 -596   -0.0242%
    covdata   323488    323404  -84    -0.0260%
    cover     518630    518490  -140   -0.0270%
    dist      340894    340814  -80    -0.0235%
    distpack  282568    282484  -84    -0.0297%
    doc       790224    789984  -240   -0.0304%
    fix       324408    324348  -60    -0.0185%
    link      704910    704666  -244   -0.0346%
    nm        277220    277144  -76    -0.0274%
    objdump   508026    507878  -148   -0.0291%
    pack      221810    221786  -24    -0.0108%
    pprof     1470284   1469880 -404   -0.0275%
    test2json 254896    254852  -44    -0.0173%
    trace     1100390   1100074 -316   -0.0287%
    vet       781398    781142  -256   -0.0328%
    go        1529668   1529128 -540   -0.0353%
    gofmt     318668    318568  -100   -0.0314%
    total     13795746 13792094 -3652  -0.0265%

Change-Id: I88d1f12cfc4be0e92687c48e06a57213aa484aca
Reviewed-on: https://go-review.googlesource.com/c/go/+/672555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

crypto/internal/fips140/subtle: add assembly implementation of xorBytes for arm

goos: linux
goarch: arm
pkg: crypto/subtle
                                     │       o        │                  n                  │
                                     │     sec/op     │    sec/op     vs base               │
ConstantTimeByteEq-4                    5.353n ±  88%   4.012n ± 67%        ~ (p=0.381 n=8)
ConstantTimeEq-4                        4.151n ±   1%   4.078n ±  0%   -1.76% (p=0.000 n=8)
ConstantTimeLessOrEq-4                  4.010n ±  15%   4.154n ±  3%        ~ (p=0.584 n=8)
XORBytes/8Bytes-4                       85.69n ±  13%   44.02n ±  1%  -48.64% (p=0.000 n=8)
XORBytes/128Bytes-4                    164.85n ±   9%   84.62n ±  5%  -48.67% (p=0.000 n=8)
XORBytes/2048Bytes-4                   1374.0n ±   1%   741.2n ± 15%  -46.05% (p=0.000 n=8)
XORBytes/8192Bytes-4                    4.357µ ±   0%   2.801µ ±  0%  -35.71% (p=0.000 n=8)
XORBytes/32768Bytes-4                   16.67µ ±   0%   11.96µ ±  0%  -28.26% (p=0.000 n=8)
XORBytesAlignment/8Bytes0Offset-4       83.28n ±   0%   42.77n ±  1%  -48.65% (p=0.000 n=8)
XORBytesAlignment/8Bytes1Offset-4       61.52n ±   1%   50.30n ± 16%  -18.24% (p=0.000 n=8)
XORBytesAlignment/8Bytes2Offset-4       61.75n ±   1%   42.72n ±  1%  -30.82% (p=0.000 n=8)
XORBytesAlignment/8Bytes3Offset-4       61.53n ±   1%   42.70n ±  1%  -30.60% (p=0.000 n=8)
XORBytesAlignment/8Bytes4Offset-4       83.28n ±   0%   42.71n ±  1%  -48.72% (p=0.000 n=8)
XORBytesAlignment/8Bytes5Offset-4       61.53n ±   0%   42.73n ±  1%  -30.55% (p=0.000 n=8)
XORBytesAlignment/8Bytes6Offset-4       61.58n ±   0%   42.69n ±  1%  -30.68% (p=0.000 n=8)
XORBytesAlignment/8Bytes7Offset-4       61.63n ±   1%   42.70n ±  1%  -30.72% (p=0.000 n=8)
XORBytesAlignment/128Bytes0Offset-4    154.15n ±   4%   83.48n ±  0%  -45.84% (p=0.000 n=8)
XORBytesAlignment/128Bytes1Offset-4    265.25n ±   0%   91.70n ±  8%  -65.43% (p=0.000 n=8)
XORBytesAlignment/128Bytes2Offset-4    265.20n ±   0%   98.09n ± 13%  -63.01% (p=0.000 n=8)
XORBytesAlignment/128Bytes3Offset-4    265.20n ±   0%   85.48n ±  0%  -67.77% (p=0.000 n=8)
XORBytesAlignment/128Bytes4Offset-4    150.05n ±   0%   83.52n ± 15%  -44.34% (p=0.000 n=8)
XORBytesAlignment/128Bytes5Offset-4    265.20n ±   0%   85.48n ± 15%  -67.77% (p=0.000 n=8)
XORBytesAlignment/128Bytes6Offset-4    265.20n ±   0%   96.16n ± 11%  -63.74% (p=0.000 n=8)
XORBytesAlignment/128Bytes7Offset-4    265.20n ±   0%   85.49n ±  0%  -67.76% (p=0.000 n=8)
XORBytesAlignment/2048Bytes0Offset-4   1114.0n ±   0%   739.5n ±  0%  -33.62% (p=0.000 n=8)
XORBytesAlignment/2048Bytes1Offset-4   3285.0n ±  15%   783.5n ±  0%  -76.15% (p=0.000 n=8)
XORBytesAlignment/2048Bytes2Offset-4   3288.0n ±  15%   783.6n ± 25%  -76.17% (p=0.000 n=8)
XORBytesAlignment/2048Bytes3Offset-4   3286.0n ±   0%   783.5n ±  0%  -76.15% (p=0.000 n=8)
XORBytesAlignment/2048Bytes4Offset-4   1116.0n ± 115%   742.9n ±  0%  -33.43% (p=0.000 n=8)
XORBytesAlignment/2048Bytes5Offset-4   3285.0n ±   0%   785.0n ±  0%  -76.10% (p=0.000 n=8)
XORBytesAlignment/2048Bytes6Offset-4   3284.0n ±   0%   784.8n ±  0%  -76.10% (p=0.000 n=8)
XORBytesAlignment/2048Bytes7Offset-4   3283.0n ±   0%   784.9n ±  0%  -76.09% (p=0.000 n=8)
geomean                                 269.5n          129.5n        -51.93%

                                     │       o       │                   n                    │
                                     │      B/s      │      B/s        vs base                │
XORBytes/8Bytes-4                      89.08Mi ± 11%   173.34Mi ±  1%   +94.58% (p=0.000 n=8)
XORBytes/128Bytes-4                    741.9Mi ± 10%   1442.6Mi ± 13%   +94.45% (p=0.000 n=8)
XORBytes/2048Bytes-4                   1.388Gi ±  0%    2.573Gi ± 13%   +85.40% (p=0.000 n=8)
XORBytes/8192Bytes-4                   1.751Gi ±  1%    2.724Gi ±  0%   +55.57% (p=0.000 n=8)
XORBytes/32768Bytes-4                  1.830Gi ±  0%    2.551Gi ±  0%   +39.38% (p=0.000 n=8)
XORBytesAlignment/8Bytes0Offset-4      91.61Mi ±  0%   178.40Mi ±  1%   +94.75% (p=0.000 n=8)
XORBytesAlignment/8Bytes1Offset-4      124.0Mi ±  1%    152.2Mi ± 18%   +22.73% (p=0.000 n=8)
XORBytesAlignment/8Bytes2Offset-4      123.6Mi ±  1%    178.6Mi ± 14%   +44.54% (p=0.000 n=8)
XORBytesAlignment/8Bytes3Offset-4      124.0Mi ±  1%    178.6Mi ±  1%   +44.10% (p=0.000 n=8)
XORBytesAlignment/8Bytes4Offset-4      91.61Mi ±  0%   178.65Mi ±  1%   +95.01% (p=0.000 n=8)
XORBytesAlignment/8Bytes5Offset-4      124.0Mi ±  1%    178.5Mi ±  1%   +43.98% (p=0.000 n=8)
XORBytesAlignment/8Bytes6Offset-4      123.9Mi ±  1%    178.7Mi ±  1%   +44.23% (p=0.000 n=8)
XORBytesAlignment/8Bytes7Offset-4      123.8Mi ±  6%    178.7Mi ±  1%   +44.33% (p=0.000 n=8)
XORBytesAlignment/128Bytes0Offset-4    792.5Mi ±  4%   1462.3Mi ± 13%   +84.51% (p=0.000 n=8)
XORBytesAlignment/128Bytes1Offset-4    460.2Mi ±  0%   1337.2Mi ±  8%  +190.56% (p=0.000 n=8)
XORBytesAlignment/128Bytes2Offset-4    460.2Mi ±  0%   1244.6Mi ± 15%  +170.42% (p=0.000 n=8)
XORBytesAlignment/128Bytes3Offset-4    460.3Mi ±  0%   1428.1Mi ±  0%  +210.27% (p=0.000 n=8)
XORBytesAlignment/128Bytes4Offset-4    813.5Mi ±  0%   1461.6Mi ± 13%   +79.67% (p=0.000 n=8)
XORBytesAlignment/128Bytes5Offset-4    460.3Mi ±  0%   1428.0Mi ± 13%  +210.25% (p=0.000 n=8)
XORBytesAlignment/128Bytes6Offset-4    460.3Mi ±  0%   1285.1Mi ± 11%  +179.16% (p=0.000 n=8)
XORBytesAlignment/128Bytes7Offset-4    460.2Mi ±  0%   1427.9Mi ± 18%  +210.25% (p=0.000 n=8)
XORBytesAlignment/2048Bytes0Offset-4   1.711Gi ±  0%    2.579Gi ±  0%   +50.71% (p=0.000 n=8)
XORBytesAlignment/2048Bytes1Offset-4   594.5Mi ± 13%   2493.0Mi ± 20%  +319.35% (p=0.000 n=8)
XORBytesAlignment/2048Bytes2Offset-4   594.0Mi ± 13%   2492.7Mi ± 20%  +319.63% (p=0.000 n=8)
XORBytesAlignment/2048Bytes3Offset-4   594.4Mi ± 53%   2492.8Mi ±  0%  +319.35% (p=0.000 n=8)
XORBytesAlignment/2048Bytes4Offset-4   1.710Gi ± 53%    2.567Gi ±  0%   +50.17% (p=0.000 n=8)
XORBytesAlignment/2048Bytes5Offset-4   594.5Mi ±  0%   2487.9Mi ±  0%  +318.47% (p=0.000 n=8)
XORBytesAlignment/2048Bytes6Offset-4   594.8Mi ±  0%   2488.6Mi ±  0%  +318.41% (p=0.000 n=8)
XORBytesAlignment/2048Bytes7Offset-4   594.9Mi ±  0%   2488.3Mi ±  0%  +318.28% (p=0.000 n=8)
geomean                                414.2Mi          921.5Mi        +122.46%

Change-Id: I0ac50135de2e69fcf802be31e5175d666c93ad4c
Reviewed-on: https://go-review.googlesource.com/c/go/+/667817
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>

runtime: Improvement in perf of s390x memclr

Memclr routine of s390x architecture is now implemented with vector operations.
And loop unrolling is used for larger sizes.

goos: linux
goarch: s390x
pkg: runtime
                        |    old.txt    |            new_final.txt             |
                        |    sec/op     |    sec/op     vs base                |
Memclr/5                   2.485n ±  5%   2.421n ±  0%   -2.54% (p=0.000 n=10)
Memclr/16                  3.037n ±  2%   2.969n ±  0%   -2.26% (p=0.001 n=10)
Memclr/64                  9.623n ±  0%   4.455n ±  1%  -53.70% (p=0.000 n=10)
Memclr/256                 3.347n ±  3%   3.312n ±  4%        ~ (p=0.670 n=10)
Memclr/4096                15.53n ±  0%   15.54n ±  0%   +0.06% (p=0.000 n=10)
Memclr/65536               329.8n ±  2%   228.4n ±  0%  -30.74% (p=0.000 n=10)
Memclr/1M                  13.09µ ±  0%   12.78µ ±  0%   -2.34% (p=0.000 n=10)
Memclr/4M                  52.33µ ±  0%   51.16µ ±  0%   -2.24% (p=0.000 n=10)
Memclr/8M                  104.6µ ±  0%   102.3µ ±  0%   -2.20% (p=0.000 n=10)
Memclr/16M                 209.4µ ±  0%   204.9µ ±  0%   -2.17% (p=0.000 n=10)
Memclr/64M                 977.8µ ±  0%   967.8µ ±  0%   -1.02% (p=0.000 n=10)
MemclrUnaligned/0_5        3.398n ±  0%   3.657n ±  0%   +7.62% (p=0.000 n=10)
MemclrUnaligned/0_16       3.957n ±  0%   3.958n ±  0%        ~ (p=0.325 n=10)
MemclrUnaligned/0_64      11.550n ±  0%   5.139n ±  0%  -55.51% (p=0.000 n=10)
MemclrUnaligned/0_256      4.288n ±  0%   4.025n ±  4%   -6.14% (p=0.000 n=10)
MemclrUnaligned/0_4096     15.53n ±  0%   15.53n ±  0%        ~ (p=1.000 n=10)
MemclrUnaligned/0_65536    318.3n ±  1%   233.9n ±  0%  -26.52% (p=0.000 n=10)
MemclrUnaligned/1_5        3.398n ±  0%   3.657n ±  0%   +7.62% (p=0.000 n=10)
MemclrUnaligned/1_16       3.965n ±  0%   3.969n ±  0%   +0.10% (p=0.000 n=10)
MemclrUnaligned/1_64      11.550n ±  0%   5.109n ±  0%  -55.76% (p=0.000 n=10)
MemclrUnaligned/1_256      4.385n ±  0%   4.174n ±  1%   -4.80% (p=0.000 n=10)
MemclrUnaligned/1_4096     26.23n ±  0%   26.24n ±  0%   +0.04% (p=0.005 n=10)
MemclrUnaligned/1_65536    570.5n ±  0%   401.3n ±  0%  -29.66% (p=0.000 n=10)
MemclrUnaligned/4_5        3.398n ±  0%   3.657n ±  0%   +7.62% (p=0.000 n=10)
MemclrUnaligned/4_16       3.965n ±  0%   3.973n ±  1%   +0.19% (p=0.000 n=10)
MemclrUnaligned/4_64      11.550n ±  0%   5.131n ±  0%  -55.58% (p=0.000 n=10)
MemclrUnaligned/4_256      4.419n ±  0%   4.187n ±  1%   -5.25% (p=0.000 n=10)
MemclrUnaligned/4_4096     26.23n ±  0%   26.24n ±  0%   +0.04% (p=0.011 n=10)
MemclrUnaligned/4_65536    570.5n ±  0%   401.2n ±  0%  -29.67% (p=0.000 n=10)
MemclrUnaligned/7_5        3.397n ±  0%   3.657n ±  0%   +7.65% (p=0.000 n=10)
MemclrUnaligned/7_16       3.965n ±  0%   3.969n ±  0%   +0.10% (p=0.000 n=10)
MemclrUnaligned/7_64      11.550n ±  0%   5.120n ±  0%  -55.67% (p=0.000 n=10)
MemclrUnaligned/7_256      4.407n ±  0%   4.188n ±  2%   -4.99% (p=0.000 n=10)
MemclrUnaligned/7_4096     26.24n ±  0%   26.24n ±  0%        ~ (p=1.000 n=10)
MemclrUnaligned/7_65536    570.8n ±  0%   401.3n ±  0%  -29.69% (p=0.000 n=10)
MemclrUnaligned/0_1M       13.08µ ±  0%   12.81µ ±  0%   -2.06% (p=0.000 n=10)
MemclrUnaligned/0_4M       52.28µ ±  0%   51.13µ ±  0%   -2.21% (p=0.000 n=10)
MemclrUnaligned/0_8M       104.6µ ±  0%   102.3µ ±  0%   -2.18% (p=0.000 n=10)
MemclrUnaligned/0_16M      209.5µ ±  0%   204.8µ ±  0%   -2.24% (p=0.000 n=10)
MemclrUnaligned/0_64M      977.7µ ±  0%   969.1µ ±  0%   -0.88% (p=0.000 n=10)
MemclrUnaligned/1_1M       17.49µ ±  0%   16.04µ ±  0%   -8.32% (p=0.000 n=10)
MemclrUnaligned/1_4M       69.92µ ±  0%   64.13µ ±  0%   -8.28% (p=0.000 n=10)
MemclrUnaligned/1_8M       139.8µ ±  0%   128.2µ ±  0%   -8.32% (p=0.000 n=10)
MemclrUnaligned/1_16M      279.9µ ±  0%   256.1µ ±  0%   -8.50% (p=0.000 n=10)
MemclrUnaligned/1_64M      1.250m ±  0%   1.216m ±  0%   -2.73% (p=0.000 n=10)
MemclrUnaligned/4_1M       17.50µ ±  0%   16.04µ ±  0%   -8.33% (p=0.000 n=10)
MemclrUnaligned/4_4M       69.93µ ±  0%   64.12µ ±  0%   -8.30% (p=0.000 n=10)
MemclrUnaligned/4_8M       139.8µ ±  0%   128.2µ ±  0%   -8.32% (p=0.000 n=10)
MemclrUnaligned/4_16M      280.2µ ±  0%   256.2µ ±  0%   -8.55% (p=0.000 n=10)
MemclrUnaligned/4_64M      1.250m ±  0%   1.216m ±  0%   -2.73% (p=0.000 n=10)
MemclrUnaligned/7_1M       17.50µ ±  0%   16.04µ ±  0%   -8.35% (p=0.000 n=10)
MemclrUnaligned/7_4M       69.92µ ±  0%   64.13µ ±  0%   -8.28% (p=0.000 n=10)
MemclrUnaligned/7_8M       139.8µ ±  0%   128.2µ ±  0%   -8.34% (p=0.000 n=10)
MemclrUnaligned/7_16M      279.6µ ±  0%   256.2µ ±  0%   -8.35% (p=0.000 n=10)
MemclrUnaligned/7_64M      1.250m ±  0%   1.216m ±  0%   -2.73% (p=0.000 n=10)
MemclrRange/1K_2K          1.053µ ±  0%   1.020µ ±  1%   -3.09% (p=0.000 n=10)
MemclrRange/2K_8K          1.552µ ±  0%   1.570µ ± 12%        ~ (p=0.137 n=10)
MemclrRange/4K_16K         1.283µ ±  0%   1.250µ ±  0%   -2.61% (p=0.000 n=10)
MemclrRange/160K_228K      20.62µ ±  0%   19.86µ ±  0%   -3.70% (p=0.000 n=10)
MemclrKnownSize1           1.732n ±  0%   1.732n ±  0%        ~ (p=1.000 n=10)
MemclrKnownSize2           1.925n ± 34%   1.967n ±  8%        ~ (p=0.080 n=10)
MemclrKnownSize4           1.808n ±  3%   1.732n ±  0%   -4.20% (p=0.000 n=10)
MemclrKnownSize8           2.002n ±  9%   1.773n ±  5%  -11.46% (p=0.000 n=10)
MemclrKnownSize16          2.880n ±  5%   2.461n ±  5%  -14.53% (p=0.000 n=10)
MemclrKnownSize32          8.082n ±  0%   2.838n ±  5%  -64.88% (p=0.000 n=10)
MemclrKnownSize64          8.083n ±  0%   4.960n ±  4%  -38.63% (p=0.000 n=10)
MemclrKnownSize112         8.082n ±  0%   5.533n ±  1%  -31.53% (p=0.000 n=10)
MemclrKnownSize128         8.082n ±  0%   5.534n ±  1%  -31.54% (p=0.000 n=10)
MemclrKnownSize192         8.082n ±  0%   6.833n ±  2%  -15.45% (p=0.000 n=10)
MemclrKnownSize248         8.082n ±  0%   7.165n ±  1%  -11.34% (p=0.000 n=10)
MemclrKnownSize256         2.995n ±  6%   3.226n ±  4%   +7.70% (p=0.006 n=10)
MemclrKnownSize512         3.356n ±  8%   3.595n ±  3%   +7.14% (p=0.007 n=10)
MemclrKnownSize1024        4.664n ±  0%   4.665n ±  0%        ~ (p=0.426 n=10)
MemclrKnownSize4096        15.80n ±  4%   15.15n ±  0%        ~ (p=0.449 n=10)
MemclrKnownSize512KiB      6.543µ ±  0%   6.380µ ±  0%   -2.48% (p=0.000 n=10)
geomean                    327.2n         286.6n        -12.42%

Change-Id: I0f8450743e2f7e736c5ff96a316a8b5d98b27222
Reviewed-on: https://go-review.googlesource.com/c/go/+/662475
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/go/internal/pkg: fail on bad filenames

Unhidden filenames with forbidden characters in subdirectories now
correctly fail the build instead of silently being skipped.
Previously this behavior would only trigger on files in the root of
the embedded directory.

Fixes #54003
Change-Id: I52967d90d6b7929e4ae474b78d3f88732bdd3ac4
Reviewed-on: https://go-review.googlesource.com/c/go/+/670615
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/go: fix not print GCCGO when it's not the default

Fixes #69994

Change-Id: I2a23e5998b7421fd5ae0fdb68303d3244361b341
Reviewed-on: https://go-review.googlesource.com/c/go/+/671635
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

internal/pkgbits, cmd/compile/internal/noder: document string section

To understand this change, we begin with a short description of the UIR
file format.

Every file is a header followed by a series of sections. Each section
has a kind, which determines the type of elements it contains. An
element is just a collection of one or more primitives, as defined by
package pkgbits.

Strings have their own section. Elements in the string section contain
only string primitives. To use a string, elements in other sections
encode a reference to the string section.

To illustrate, consider a simple file which exports nothing at all.

  package p

In the meta section, there is an element representing a package stub.
In that package stub, a string ("p") represents both the path and name
of the package. Again, these are encoded as references.

To manage references, every element begins with a reference table.
Instead of writing the bytes for "p" directly, the package stub encodes
an index in this reference table. At that index, a pair of numbers is
stored, indicating:

  1. which section
  2. which element index within the section

Effectively, elements always use *2* layers of indirection; first to the
reference table, then to the bytes themselves.

With some minor hand-waving, an encoding for the above package is given
below, with (S)ections, (E)lements and (P)rimitives denoted.

+ Header
| + Section Ends                           // each section has 1 element
| | + 1                                    // String is elements [0, 1)
| | + 2                                    // Meta   is elements [1, 2)
| + Element Ends
| | + 1                                    // "p"    is bytes    [0, 1)
| | + 6                                    // stub   is bytes    [1, 6)
+ Payload
| + (S) String
| | + (E) String
| | | + (P) String           { byte } 0x70 // "p"
| + (S) Meta
| | + (E) Package Stub
| | | + Reference Table
| | | | + (P) Entry Count    uvarint  1    // there is a single entry
| | | | + (P) 0th Section    uvarint  0    // to String, 0th section
| | | | + (P) 0th Index      uvarint  0    // to 0th element in String
| | | + Internals
| | | | + (P) Path           uvarint  0    // 0th entry in table
| | | | + (P) Name           uvarint  0    // 0th entry in table

Note that string elements do not have reference tables like other
elements. They behave more like a primitive.

As this is a bit complicated and getting into details of the UIR file
format, we omit some details in the documentation here. The structure
will become clearer as we continue documenting.

Change-Id: I12a5ce9a34251c5358a20f2f2c4d0f9bd497f4d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/671997
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Mark Freeman <mark@golang.org>
TryBot-Bypass: Mark Freeman <mark@golang.org>

os: don't follow symlinks on Windows when O_CREATE|O_EXCL

Match standard Unix behavior: Symlinks are not followed when
O_CREATE|O_EXCL is passed to open.

Thanks to Junyoung Park and Dong-uk Kim of KAIST Hacking Lab
for discovering this issue.

Fixes #73702
Fixes CVE-2025-0913

Change-Id: Ieb46a6780c5e9a6090b09cd34290f04a8e3b0ca5
Reviewed-on: https://go-review.googlesource.com/c/go/+/672396
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>

bytes, strings: speed up Split{,After}Seq

CL 669735 brought a welcome performance boost to splitSeq; however, it rendered explodeSeq ineligible for inlining and failed to update that function's doc comment.

This CL inlines the call to explodeSeq in splitSeq, thereby unlocking a further speedup in the case of an empty separator, and removes function explodeSeq altogether.

Some benchmarks results:

goos: darwin
goarch: amd64
pkg: strings
cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
                                   │     old     │                 new                  │
                                   │   sec/op    │    sec/op     vs base                │
SplitSeqEmptySeparator-8             5.136m ± 6%    3.180m ± 6%  -38.09% (p=0.000 n=20)
SplitSeqSingleByteSeparator-8        995.9µ ± 1%    988.4µ ± 0%   -0.75% (p=0.000 n=20)
SplitSeqMultiByteSeparator-8         593.1µ ± 2%    591.7µ ± 1%        ~ (p=0.253 n=20)
SplitAfterSeqEmptySeparator-8        5.554m ± 3%    3.432m ± 2%  -38.20% (p=0.000 n=20)
SplitAfterSeqSingleByteSeparator-8   997.4µ ± 0%   1000.0µ ± 8%        ~ (p=0.121 n=20)
SplitAfterSeqMultiByteSeparator-8    591.7µ ± 1%    588.9µ ± 0%   -0.48% (p=0.004 n=20)
geomean                              1.466m         1.247m       -14.97%

                                   │     old      │                 new                 │
                                   │     B/op     │    B/op     vs base                 │
SplitSeqEmptySeparator-8             0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
SplitSeqSingleByteSeparator-8        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
SplitSeqMultiByteSeparator-8         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
SplitAfterSeqEmptySeparator-8        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
SplitAfterSeqSingleByteSeparator-8   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
SplitAfterSeqMultiByteSeparator-8    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
geomean                                         ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

Change-Id: I5767b68dc1a4fbcb2ac20683830a49ee3eb1bee1
GitHub-Last-Rev: 344934071f3220a1afea3def306dadfee720d311
GitHub-Pull-Request: golang/go#73685
Reviewed-on: https://go-review.googlesource.com/c/go/+/672175
Reviewed-by: qiu laidongfeng2 <2645477756@qq.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>

cmd/internal/script: fix copying directory when symlink fails

The change fixes `linkOrCopy` to work on systems wihtout symlinks,
when copying directories.  This was originally noticed on Windows
systems when the user did not have admin privs.

Fixes #73692
Change-Id: I8ca66d65e99433ad38e70314abfabafd43794b79
Reviewed-on: https://go-review.googlesource.com/c/go/+/672275
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

runtime: increase freebsd/amd64 pointer size from 48 to 57 bits

Because freebsd is now enabling la57 by default.

Fixes #49405

Change-Id: I30f7bac8b8a9baa85e0c097e06072c19ad474e5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/670715
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/intarnal/obj: add new assembly format for VANDV and VANDB on loong64

In order to make it easier to write in assembly and to be consistent
with the usage of general instructions, a new assembly format is
added for the instructions VANDV and VANDB.

It also works for instructions XVAND{V,B}, [X]V{OR,XOR,NOR,ANDN,ORN}V
and [X]V{OR,XOR,NOR}B.

Change-Id: Ia75d607ac918950e58840ec627aaf0be45d837fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/671316
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

internal/bytealg: optimize the function compare using SIMD on loong64

goos: linux
goarch: loong64
pkg: bytes
cpu: Loongson-3A6000-HV @ 2500.00MHz
                  │      old      │                 new                  │
                  │    sec/op     │    sec/op     vs base                │
BytesCompare/1       7.238n ± 25%    5.204n ± 0%  -28.10% (p=0.001 n=10)
BytesCompare/2       7.242n ±  6%    5.204n ± 0%  -28.14% (p=0.000 n=10)
BytesCompare/4       7.229n ±  5%    4.403n ± 0%  -39.10% (p=0.000 n=10)
BytesCompare/8       7.077n ± 36%    4.403n ± 0%  -37.78% (p=0.000 n=10)
BytesCompare/16      8.373n ±  6%    6.004n ± 0%  -28.30% (p=0.000 n=10)
BytesCompare/32      8.040n ±  3%    4.803n ± 0%  -40.26% (p=0.000 n=10)
BytesCompare/64      8.434n ± 24%   10.410n ± 0%  +23.42% (p=0.014 n=10)
BytesCompare/128    11.530n ± 23%    5.604n ± 0%  -51.40% (p=0.000 n=10)
BytesCompare/256    14.180n ±  0%    7.606n ± 0%  -46.36% (p=0.000 n=10)
BytesCompare/512     26.83n ±  0%    10.81n ± 0%  -59.71% (p=0.000 n=10)
BytesCompare/1024    52.60n ±  0%    17.21n ± 0%  -67.28% (p=0.000 n=10)
BytesCompare/2048   103.70n ±  0%    30.02n ± 0%  -71.05% (p=0.000 n=10)
geomean              13.49n          7.607n       -43.63%

goos: linux
goarch: loong64
pkg: bytes
cpu: Loongson-3A6000-HV @ 2500.00MHz
                                      │     old     │                 new                  │
                                      │   sec/op    │    sec/op     vs base                │
CompareBytesEqual                       5.603n ± 0%   5.604n ±  0%        ~ (p=0.191 n=10)
CompareBytesToNil                       3.202n ± 0%   3.202n ±  0%        ~ (p=1.000 n=10)
CompareBytesEmpty                       2.802n ± 0%   2.802n ±  0%        ~ (p=1.000 n=10)
CompareBytesIdentical                   3.202n ± 0%   2.538n ±  1%  -20.72% (p=0.000 n=10)
CompareBytesSameLength                  8.805n ± 0%   4.803n ±  0%  -45.45% (p=0.000 n=10)
CompareBytesDifferentLength             9.206n ± 0%   4.403n ±  0%  -52.17% (p=0.000 n=10)
CompareBytesBigUnaligned/offset=1       82.04µ ± 0%   45.91µ ±  0%  -44.04% (p=0.000 n=10)
CompareBytesBigUnaligned/offset=2       82.04µ ± 0%   45.91µ ±  0%  -44.04% (p=0.000 n=10)
CompareBytesBigUnaligned/offset=3       82.04µ ± 0%   45.91µ ±  0%  -44.04% (p=0.000 n=10)
CompareBytesBigUnaligned/offset=4       82.04µ ± 0%   45.91µ ±  0%  -44.04% (p=0.000 n=10)
CompareBytesBigUnaligned/offset=5       82.04µ ± 0%   45.91µ ±  0%  -44.04% (p=0.000 n=10)
CompareBytesBigUnaligned/offset=6       82.03µ ± 0%   45.93µ ±  0%  -44.01% (p=0.000 n=10)
CompareBytesBigUnaligned/offset=7       82.04µ ± 0%   45.93µ ±  0%  -44.01% (p=0.000 n=10)
CompareBytesBigBothUnaligned/offset=0   78.76µ ± 0%   45.69µ ±  0%  -41.98% (p=0.000 n=10)
CompareBytesBigBothUnaligned/offset=1   85.32µ ± 0%   46.04µ ±  0%  -46.03% (p=0.000 n=10)
CompareBytesBigBothUnaligned/offset=2   85.31µ ± 0%   46.04µ ±  0%  -46.03% (p=0.000 n=10)
CompareBytesBigBothUnaligned/offset=3   85.32µ ± 0%   46.04µ ±  0%  -46.03% (p=0.000 n=10)
CompareBytesBigBothUnaligned/offset=4   85.32µ ± 0%   46.04µ ±  0%  -46.03% (p=0.000 n=10)
CompareBytesBigBothUnaligned/offset=5   85.32µ ± 0%   46.04µ ±  0%  -46.03% (p=0.000 n=10)
CompareBytesBigBothUnaligned/offset=6   85.31µ ± 0%   46.06µ ±  0%  -46.02% (p=0.000 n=10)
CompareBytesBigBothUnaligned/offset=7   85.32µ ± 0%   52.32µ ±  7%  -38.68% (p=0.000 n=10)
CompareBytesBig                         78.76µ ± 0%   50.20µ ±  6%  -36.26% (p=0.000 n=10)
CompareBytesBigIdentical                3.202n ± 0%   3.442n ± 24%        ~ (p=0.462 n=10)
geomean                                 4.197µ        2.630µ        -37.34%

Change-Id: I621145aef3e6a2c68e7127152f26ed047c6b2ece
Reviewed-on: https://go-review.googlesource.com/c/go/+/671315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>

testing: add Output

Output is a method on T, B and F. It provides an io.Writer that writes
to the same test output stream as TB.Log. The new output writer is
used to refactor the implementation of Log. It maintains the formatting
provided by Log while making call site information optional.
Additionally, it provides buffering of log messages. This fixes and
expands on
https://go-review.googlesource.com/c/go/+/646956.

For #59928.

Change-Id: I08179c35a681f601cf125c0f4aeb648bc10c7a9f
GitHub-Last-Rev: e6e202793c9bc471493187e0556a3a1e7305ff82
GitHub-Pull-Request: golang/go#73703
Reviewed-on: https://go-review.googlesource.com/c/go/+/672395
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>

internal/poll: use runtime.AddCleanup instead of runtime.SetFinalizer

Replace the use of SetFinalizer with AddCleanup.

For #70907

Change-Id: I0cb2c2985eb9285e5f92be9dbcb9d77acc0f59c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/671441
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

reflect: use runtime.AddCleanup instead of runtime.SetFinalizer

Replace a usage of runtime.SetFinalizer with runtime.AddCleanup in
the TestCallReturnsEmpty test. There is an additional use of
SetFinalizer in the reflect package which depends on object
resurrection and needs further refactoring to replace.

Updates #70907

Change-Id: I4c0e56c35745a225776bd611d026945efdaf96f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/667595
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

Revert "testing: add Output"

This reverts commit 8d189f188e225e4919b34c0c097e75dfda255949.

Reason for revert: failing test

Change-Id: I951087eaef7818697acf87e3206003bcc8a81ee2
Reviewed-on: https://go-review.googlesource.com/c/go/+/672335
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

crypto/x509: decouple key usage and policy validation

Disabling key usage validation (by passing ExtKeyUsageAny)
unintentionally disabled policy validation. This change decouples these
two checks, preventing the user from unintentionally disabling policy
validation.

Thanks to Krzysztof Skrzętnicki (@Tener) of Teleport for reporting this
issue.

Fixes #73612
Fixes CVE-2025-22874

Change-Id: Iec8f080a8879a3dd44cb3da30352fa3e7f539d40
Reviewed-on: https://go-review.googlesource.com/c/go/+/670375
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Stapleton Cordasco <graffatcolmingov@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/go: add support for git sha256 hashes

Git's supported SHA 256 object hashes since 2.29[1] in 2021, and Gitlab
now has experimental support for sha256 repos.

Take rsc@'s suggestion of checking the of the length of the hashes from
git ls-remote to determine whether a git repo is using sha256 hashes and
decide whether to pass --object-format=sha256 to git init.

Unfortunately, just passing --object-format=sha256 wasn't quite enough,
though. We also need to decide whether the hash-length is 64 hex bytes
or 40 hex bytes when resolving refs to decide whether we've been passed
a full commit-hash. To that end, we use
git config extensions.objectformat to decide whether the (now guaranteed
local) repo is using sha256 hashes and hence 64-hex-byte strings.

[1]: lost experimental status in 2.42 from Aug 2023
(https://lore.kernel.org/git/xmqqr0nwp8mv.fsf@gitster.g/)

For: #68359
Change-Id: I47f480ab8334128c5d17570fe76722367d0d8ed8
Reviewed-on: https://go-review.googlesource.com/c/go/+/636475
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: David Finkel <david.finkel@gmail.com>

runtime/pprof: return errors from writing profiles

Fixes #73107

Change-Id: I41f3e1bd1fdaca2f0e94151b2320bd569e258a51
Reviewed-on: https://go-review.googlesource.com/c/go/+/671576
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>

cmd/go: support -json flag in go version

It supports features described in the issue:

* add -json flag for 'go version -m' to print json encoding of
  runtime/debug.BuildSetting to standard output.
* report an error when specifying -json flag without -m.
* print build settings on seperated line for each binary

Fixes #69712

Change-Id: I79cba2109f80f7459252d197a74959694c4eea1f
Reviewed-on: https://go-review.googlesource.com/c/go/+/619955
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

crypto: limit md5 or sha256 blocks processed at once in assembly

This change limits the amount of data that can be hashed at once - the
assembly routines are not preemptible and can result in large latency
outliers when part of a larger system.

Benchmarks for sha256 (on an arm64 M1):

name                  old speed      new speed      delta
Hash8Bytes/New-10      178MB/s ± 0%   178MB/s ± 0%  +0.16%  (p=0.002 n=9+8)
Hash8Bytes/Sum224-10   154MB/s ± 0%   154MB/s ± 0%    ~     (p=0.287 n=9+10)
Hash8Bytes/Sum256-10   156MB/s ± 0%   157MB/s ± 0%  +0.13%  (p=0.004 n=9+8)
Hash1K/New-10         2.28GB/s ± 0%  2.28GB/s ± 0%    ~     (p=0.968 n=10+9)
Hash1K/Sum224-10      2.20GB/s ± 0%  2.21GB/s ± 0%  +0.30%  (p=0.001 n=9+9)
Hash1K/Sum256-10      2.21GB/s ± 0%  2.21GB/s ± 0%  +0.26%  (p=0.000 n=9+8)
Hash8K/New-10         2.37GB/s ± 2%  2.40GB/s ± 0%    ~     (p=0.289 n=10+10)
Hash8K/Sum224-10      2.39GB/s ± 0%  2.39GB/s ± 0%    ~     (p=0.983 n=8+9)
Hash8K/Sum256-10      2.39GB/s ± 0%  2.39GB/s ± 0%    ~     (p=0.905 n=9+10)
Hash256K/New-10       2.42GB/s ± 0%  2.42GB/s ± 0%    ~     (p=0.250 n=9+10)
Hash256K/Sum224-10    2.42GB/s ± 0%  2.42GB/s ± 0%    ~     (p=0.093 n=8+9)
Hash256K/Sum256-10    2.42GB/s ± 0%  2.42GB/s ± 0%    ~     (p=0.211 n=10+9)
Hash1M/New-10         2.42GB/s ± 0%  2.42GB/s ± 0%    ~     (p=0.963 n=8+9)
Hash1M/Sum224-10      2.42GB/s ± 0%  2.42GB/s ± 0%    ~     (p=0.173 n=10+8)
Hash1M/Sum256-10      2.42GB/s ± 0%  2.42GB/s ± 0%    ~     (p=0.743 n=9+8)

Note that `Hash8K` shows that a 8K block size is sufficient to achieve
peak bandwidth, so the 64KB maxAsmSize should be plenty.

Benchmarks for md5:

name       old speed     new speed     delta
Hash1M-10  669MB/s ± 0%  669MB/s ± 0%   ~     (p=0.965 n=8+10)
Hash8M-10  667MB/s ± 0%  666MB/s ± 0%   ~     (p=0.356 n=10+9)

Fixes #64417

Change-Id: If7f5e7587b33c65148f49859c9d46ae6f6948db4
GitHub-Last-Rev: 2f83f4255412b533469e953db6c1ef16fa3eb7c2
GitHub-Pull-Request: golang/go#73638
Reviewed-on: https://go-review.googlesource.com/c/go/+/671098
Reviewed-by: Jorropo <jorropo.pgm@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Ian Stapleton Cordasco <graffatcolmingov@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

testing: add Output

Output is a method on T, B and F. It provides an io.Writer that writes
to the same test output stream as TB.Log. The new output writer is
used to refactor the implementation of Log. It maintains the formatting
provided by Log while making call site information optional.
Additionally, it provides buffering of log messages.

Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com>
Fixes #59928.

Change-Id: I29090b3d4f61f7334388b373ec18750d5637aafa
GitHub-Last-Rev: 18af0e15262494f2074d545a6042b079d62301a2
GitHub-Pull-Request: golang/go#71575
Reviewed-on: https://go-review.googlesource.com/c/go/+/646956
Reviewed-by: Arati <artichaut2023@gmail.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

cmd/link: ignore mapping symbols on riscv64

Specified in RISC-V ELF psABI[1], mapping symbols are symbols starting
with "$d" or "$x" with STT_NOTYPE, STB_LOCAL and zero sizes, indicating
boundaries between code and data in the same section.

Let's simply ignore them as they're only markers instead of real symbols.
This fixes linking errors like

sym#63 ("$d"): ignoring symbol in section 4 (".riscv.attributes") (type 0)

when using CGO together with Clang and internal linker, which are caused
by unnecessary (but technically correct) mapping symbols created by LLVM
for various sections.

[1]: https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/87aecf601722171c570120a46003be3c17ad3108/riscv-elf.adoc?plain=1#L1448

Fixes #73516

Change-Id: I02ca90c100ba8a38733fe3b8b8403836b44a3dd1
GitHub-Last-Rev: d7842ceafb840c511cf0c36295c353698898d399
GitHub-Pull-Request: golang/go#73592
Reviewed-on: https://go-review.googlesource.com/c/go/+/669675
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

runtime: only poll network from one P at a time in findRunnable

This change reintroduces CL 564197. It was reverted due to a failing
benchmark. That failure has been resolved.

For #65064

Change-Id: Ic88841d2bc24c2717ad324873f0f52699f21dc66
Reviewed-on: https://go-review.googlesource.com/c/go/+/669235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>

cmd/internal/obj/loong64: Add preld{,x} instructions support

Go asm syntax:
PRELD 16(R4), $8
PRELDX (R4)(R5), $8

Equivalent platform assembler syntax:
preld $8, $r4, 16
preldx $8, $r4, $r5

Change-Id: Ie81d22ebaf4153388a7e9d8fa0f618a0ae7a1c9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/671875
Reviewed-by: sophie zhao <zhaoxiaolin@loongson.cn>
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

internal/pkgbits: rename Reloc* to Section*

This is a basic refactoring. This enumeration refers primarily to
the different sections of a UIR file, so this naming is a bit more
direct.

Change-Id: Ib70ab054e97effaabc035450d246ae4354da8075
Reviewed-on: https://go-review.googlesource.com/c/go/+/671935
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Mark Freeman <mark@golang.org>

internal/pkgbits: write a formal grammar for UIR primitives

This complements the grammar being developed in package noder. It
is unclear how to discuss references in their current state, as
they require knowledge of sections, elements, etc.

Perhaps the references here should refer to indices on the byte
array. This would allow a stronger separation of pkgbits and noder.

Change-Id: Ic0e5ac9c07f0a0b92d6ffd4d4e26dbe5dcf89e57
Reviewed-on: https://go-review.googlesource.com/c/go/+/671440
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Bypass: Mark Freeman <mark@golang.org>
Auto-Submit: Mark Freeman <mark@golang.org>

text/template: clone options when cloning templates

Fixes #43022

Change-Id: I727b86ea0ebfff06f82c909457479c2afb9106dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/671615
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

testing: limit TempDir name length

Fixes #71742

Change-Id: Ibef8f7f0a36b25f181062c4d2f84279a97e467a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/671577
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

net: don't test with leading 0 in ipv4 addresses

Updates #30999
Fixes #73378

Change-Id: I6a96581e78678518473626aa5d108428cfad6619
Reviewed-on: https://go-review.googlesource.com/c/go/+/671795
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

net: fix deadlock in lookupProtocol on Windows

If the context expires before acquireThread(ctx) succeeds, then the goroutine can block like:

net.lookupProtocol.func1()
src/net/lookup_windows.go:58 +0x105
created by net.lookupProtocol in goroutine 2834
src/net/lookup_windows.go:56 +0xda

We saw this in our UTs with a leak detector, confirmed by inspection of the source code.

Change-Id: I9b927f0345a2fa7336b23d95c506a8a0976e28d0
GitHub-Last-Rev: 27af7477a99bdbee0c32697837f1ff261ee70d83
GitHub-Pull-Request: golang/go#73364
Reviewed-on: https://go-review.googlesource.com/c/go/+/664956
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Zxilly Chou <zhouxinyu1001@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

os: remove NewFile socket detection on Windows

NewFile was recently updated (in CL 668195) to detect whether the
handle is a socket or not. This special case is not really necessary,
given that socket handles can be used as if they were normal file
handles on all functions supported by os.File (see https://learn.microsoft.com/en-us/windows/win32/winsock/socket-handles-2).

Not only is not necessary, but is can also be problematic, as there is
no way to reliably detect whether a handle is a socket or not. For
example, the test failure reported in #73630 is caused by a named pipe
wrongly detected as a socket.

This aligns with the Unix NewFile behavior of returning an os.File that
identifies itself as a file handle even if it is a socket. This makes
os.File.Close to always return os.ErrClosed in case of multiple calls
rather than sometimes returning "use of closed network connection".

Updates #10350.
Fixes #73630.

Change-Id: Ia8329783d5c8ef6dac34ef69ed1ce9d2a9862e11
Reviewed-on: https://go-review.googlesource.com/c/go/+/671455
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>

cmd/link: use >4GB base address for 64-bit PE binaries

Windows prefers 64-bit binaries to be loaded at an address above 4GB.

Having a preferred base address below this boundary triggers a
compatibility mode in Address Space Layout Randomization (ASLR) on
recent versions of Windows that reduces the number of locations to which
ASLR may relocate the binary.

The Go internal linker was using a smaller base address due to an issue
with how dynamic cgo symbols were relocated, which has been fixed in
this CL.

Fixes #73561.

Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: Ia8cb35d57d921d9be706a8975fa085af7996f124
Reviewed-on: https://go-review.googlesource.com/c/go/+/671515
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

net/http: add missing ServeTLS on the comment of http.Server.Shutdown

A sentinel error http.ErrServerClosed is returned after Server.Shutdown
and Server.Close but it is not documented on the Server.Shutdown while
other methods such as Server.Serve are documented on it.

Change-Id: Id82886d9d6a1474a514d62e9169b35f3579a9eee
Reviewed-on: https://go-review.googlesource.com/c/go/+/671695
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sean Liao <sean@liao.dev>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Sean Liao <sean@liao.dev>

runtime: add goschedIfBusy to bgsweep to prevent livelock after inlining

gcMarkTermination() ensures that all caches are flushed before continuing the GC cycle, thus preempting all goroutines.
However, inlining calls to lock() in bgsweep makes it non-preemptible for most of the time, leading to livelock.
This change adds explicit preemption to avoid this.

Fixes #73499.

Change-Id: I4abf0d658f3d7a03ad588469cd013a0639de0c8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/668795
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>

crypto/internal/fips140/subtle: add assembly implementation of xorBytes for mipsx

goos: linux
goarch: mipsle
pkg: crypto/subtle
                                     │   osubtle    │              nsubtle               │
                                     │    sec/op    │   sec/op     vs base               │
ConstantTimeByteEq-4                    2.785n ± 0%   2.785n ± 0%        ~ (p=0.876 n=8)
ConstantTimeEq-4                        3.342n ± 0%   3.341n ± 0%        ~ (p=0.258 n=8)
ConstantTimeLessOrEq-4                  3.341n ± 0%   3.340n ± 0%        ~ (p=0.370 n=8)
XORBytes/8Bytes-4                      117.80n ± 0%   27.02n ± 2%  -77.07% (p=0.000 n=8)
XORBytes/128Bytes-4                    176.60n ± 0%   58.42n ± 4%  -66.92% (p=0.000 n=8)
XORBytes/2048Bytes-4                    996.5n ± 0%   462.4n ± 0%  -53.60% (p=0.000 n=8)
XORBytes/8192Bytes-4                    3.568µ ± 0%   1.780µ ± 2%  -50.13% (p=0.000 n=8)
XORBytes/32768Bytes-4                   19.34µ ± 6%   10.52µ ± 5%  -45.60% (p=0.000 n=8)
XORBytesAlignment/8Bytes0Offset-4      127.50n ± 0%   28.31n ± 1%  -77.80% (p=0.000 n=8)
XORBytesAlignment/8Bytes1Offset-4      105.65n ± 1%   28.20n ± 1%  -73.30% (p=0.000 n=8)
XORBytesAlignment/8Bytes2Offset-4      105.55n ± 1%   28.34n ± 1%  -73.15% (p=0.000 n=8)
XORBytesAlignment/8Bytes3Offset-4      105.65n ± 0%   28.45n ± 1%  -73.07% (p=0.000 n=8)
XORBytesAlignment/8Bytes4Offset-4      127.60n ± 0%   28.19n ± 1%  -77.91% (p=0.000 n=8)
XORBytesAlignment/8Bytes5Offset-4      105.45n ± 0%   28.38n ± 1%  -73.09% (p=0.000 n=8)
XORBytesAlignment/8Bytes6Offset-4      105.55n ± 0%   28.27n ± 1%  -73.22% (p=0.000 n=8)
XORBytesAlignment/8Bytes7Offset-4      105.60n ± 0%   28.24n ± 1%  -73.26% (p=0.000 n=8)
XORBytesAlignment/128Bytes0Offset-4    178.25n ± 0%   59.57n ± 0%  -66.58% (p=0.000 n=8)
XORBytesAlignment/128Bytes1Offset-4    313.25n ± 0%   75.32n ± 0%  -75.96% (p=0.000 n=8)
XORBytesAlignment/128Bytes2Offset-4    313.75n ± 0%   75.34n ± 0%  -75.99% (p=0.000 n=8)
XORBytesAlignment/128Bytes3Offset-4    314.25n ± 0%   75.31n ± 0%  -76.04% (p=0.000 n=8)
XORBytesAlignment/128Bytes4Offset-4    178.25n ± 0%   59.57n ± 0%  -66.58% (p=0.000 n=8)
XORBytesAlignment/128Bytes5Offset-4    314.20n ± 0%   75.80n ± 1%  -75.88% (p=0.000 n=8)
XORBytesAlignment/128Bytes6Offset-4    313.30n ± 0%   75.56n ± 0%  -75.88% (p=0.000 n=8)
XORBytesAlignment/128Bytes7Offset-4    313.95n ± 0%   75.45n ± 0%  -75.97% (p=0.000 n=8)
XORBytesAlignment/2048Bytes0Offset-4   1002.5n ± 0%   455.3n ± 0%  -54.58% (p=0.000 n=8)
XORBytesAlignment/2048Bytes1Offset-4   3649.5n ± 0%   731.6n ± 0%  -79.95% (p=0.000 n=8)
XORBytesAlignment/2048Bytes2Offset-4   3645.0n ± 0%   731.5n ± 0%  -79.93% (p=0.000 n=8)
XORBytesAlignment/2048Bytes3Offset-4   3656.0n ± 0%   731.6n ± 0%  -79.99% (p=0.000 n=8)
XORBytesAlignment/2048Bytes4Offset-4   1003.0n ± 0%   455.6n ± 0%  -54.58% (p=0.000 n=8)
XORBytesAlignment/2048Bytes5Offset-4   3651.5n ± 1%   736.6n ± 0%  -79.83% (p=0.000 n=8)
XORBytesAlignment/2048Bytes6Offset-4   3647.5n ± 0%   736.4n ± 0%  -79.81% (p=0.000 n=8)
XORBytesAlignment/2048Bytes7Offset-4   3657.0n ± 1%   736.6n ± 0%  -79.86% (p=0.000 n=8)
geomean                                 313.1n        96.95n       -69.03%

                                     │   osubtle    │                nsubtle                │
                                     │     B/s      │      B/s       vs base                │
XORBytes/8Bytes-4                      64.77Mi ± 0%   282.51Mi ± 2%  +336.18% (p=0.000 n=8)
XORBytes/128Bytes-4                    691.3Mi ± 0%   2092.3Mi ± 4%  +202.66% (p=0.000 n=8)
XORBytes/2048Bytes-4                   1.914Gi ± 0%    4.125Gi ± 0%  +115.51% (p=0.000 n=8)
XORBytes/8192Bytes-4                   2.138Gi ± 0%    4.288Gi ± 2%  +100.54% (p=0.000 n=8)
XORBytes/32768Bytes-4                  1.583Gi ± 6%    2.908Gi ± 5%   +83.61% (p=0.000 n=8)
XORBytesAlignment/8Bytes0Offset-4      59.83Mi ± 0%   269.47Mi ± 1%  +350.37% (p=0.000 n=8)
XORBytesAlignment/8Bytes1Offset-4      72.22Mi ± 0%   270.51Mi ± 1%  +274.56% (p=0.000 n=8)
XORBytesAlignment/8Bytes2Offset-4      72.28Mi ± 1%   269.19Mi ± 1%  +272.41% (p=0.000 n=8)
XORBytesAlignment/8Bytes3Offset-4      72.21Mi ± 0%   268.16Mi ± 1%  +271.38% (p=0.000 n=8)
XORBytesAlignment/8Bytes4Offset-4      59.79Mi ± 0%   270.67Mi ± 1%  +352.74% (p=0.000 n=8)
XORBytesAlignment/8Bytes5Offset-4      72.36Mi ± 0%   268.83Mi ± 1%  +271.49% (p=0.000 n=8)
XORBytesAlignment/8Bytes6Offset-4      72.29Mi ± 0%   269.95Mi ± 1%  +273.44% (p=0.000 n=8)
XORBytesAlignment/8Bytes7Offset-4      72.27Mi ± 0%   270.14Mi ± 1%  +273.79% (p=0.000 n=8)
XORBytesAlignment/128Bytes0Offset-4    684.7Mi ± 0%   2049.1Mi ± 0%  +199.26% (p=0.000 n=8)
XORBytesAlignment/128Bytes1Offset-4    389.7Mi ± 1%   1620.7Mi ± 0%  +315.86% (p=0.000 n=8)
XORBytesAlignment/128Bytes2Offset-4    389.1Mi ± 0%   1620.3Mi ± 0%  +316.41% (p=0.000 n=8)
XORBytesAlignment/128Bytes3Offset-4    388.4Mi ± 1%   1620.9Mi ± 0%  +317.29% (p=0.000 n=8)
XORBytesAlignment/128Bytes4Offset-4    684.8Mi ± 0%   2049.2Mi ± 0%  +199.24% (p=0.000 n=8)
XORBytesAlignment/128Bytes5Offset-4    388.5Mi ± 0%   1610.3Mi ± 1%  +314.47% (p=0.000 n=8)
XORBytesAlignment/128Bytes6Offset-4    389.6Mi ± 0%   1615.4Mi ± 0%  +314.60% (p=0.000 n=8)
XORBytesAlignment/128Bytes7Offset-4    388.9Mi ± 0%   1617.8Mi ± 1%  +316.04% (p=0.000 n=8)
XORBytesAlignment/2048Bytes0Offset-4   1.903Gi ± 3%    4.189Gi ± 3%  +120.18% (p=0.000 n=8)
XORBytesAlignment/2048Bytes1Offset-4   535.1Mi ± 0%   2669.7Mi ± 0%  +398.88% (p=0.000 n=8)
XORBytesAlignment/2048Bytes2Offset-4   535.8Mi ± 0%   2670.1Mi ± 0%  +398.34% (p=0.000 n=8)
XORBytesAlignment/2048Bytes3Offset-4   534.2Mi ± 0%   2669.6Mi ± 0%  +399.71% (p=0.000 n=8)
XORBytesAlignment/2048Bytes4Offset-4   1.902Gi ± 0%    4.187Gi ± 0%  +120.12% (p=0.000 n=8)
XORBytesAlignment/2048Bytes5Offset-4   534.9Mi ± 0%   2651.6Mi ± 0%  +395.73% (p=0.000 n=8)
XORBytesAlignment/2048Bytes6Offset-4   535.5Mi ± 0%   2652.3Mi ± 0%  +395.34% (p=0.000 n=8)
XORBytesAlignment/2048Bytes7Offset-4   534.1Mi ± 1%   2651.6Mi ± 0%  +396.46% (p=0.000 n=8)
geomean                                338.6Mi         1.205Gi       +264.51%

Change-Id: I4d7e759968779cf8470826b8662b9f2018e663bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/666275
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@google.com>

bytes, strings: rename parameters in ExampleCut{Pre,Suf}fix

The old parameter name sep was probably copied from ExampleCut. Change
the parameter names to prefix and suffix, respectivly to make the
examples a bit more readable.

Change-Id: Ie14b0050c2fafe3301c5368efd548a1629a7545f
Reviewed-on: https://go-review.googlesource.com/c/go/+/670955
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Sean Liao <sean@liao.dev>
Reviewed-by: Robert Griesemer <gri@google.com>

bytes, strings: micro-optimize EqualFold

The first loop leaves the lengths of the two arguments unchanged.

Take advantage of this invariant in the loop's condition. Here are some
benchmark results (no change to allocations):

goos: darwin
goarch: amd64
pkg: strings
cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
                          │     old     │                new                 │
                          │   sec/op    │   sec/op     vs base               │
EqualFold/Tests-8           240.0n ± 4%   245.1n ± 5%       ~ (p=0.516 n=20)
EqualFold/ASCII-8           11.50n ± 1%   11.04n ± 0%  -3.96% (p=0.000 n=20)
EqualFold/UnicodePrefix-8   102.1n ± 0%   102.2n ± 0%       ~ (p=0.455 n=20)
EqualFold/UnicodeSuffix-8   90.14n ± 0%   89.80n ± 1%       ~ (p=0.113 n=20)
geomean                     71.00n        70.60n       -0.56%

Change-Id: I1f6d1df8a0398f9493692f59d7369c3f0fbba436
GitHub-Last-Rev: 9508ee26ad3cadcbb5e532a731b2553ba900f2b1
GitHub-Pull-Request: golang/go#73672
Reviewed-on: https://go-review.googlesource.com/c/go/+/671756
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

crypto/sha1: replaced WORD with instruction REVB2W

Change-Id: I37f3db37a318a72a9558c1a728be8099a41a8774
Reviewed-on: https://go-review.googlesource.com/c/go/+/671255
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>

cmd/internal/obj/loong64: add [X]VFCLASS.{S/D} instructions

Go asm syntax:
 VFCLASS{F/D} VJ, VD
XVFCLASS{F/D} XJ, XD

Equivalent platform assembler syntax:
 vfclass.{s/d} vd, vj
xvfclass.{s/d} xd, xj

Change-Id: Iec373f393be315696d1fefc747a4a5882f993195
Reviewed-on: https://go-review.googlesource.com/c/go/+/670256
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dustin Turner <dustin.turner44@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>

crypto/tls: handle client hello version too high

If the client hello legacy version is >= TLS 1.3, and no
supported_versions extension is sent, negotiate TLS 1.2 or lower when
supported.

On the topic of supported version negotiation RFC 8446 4.2.1 indicates
TLS 1.3 implementations MUST send a supported_versions extension with
a list of their supported protocol versions. The crypto/tls package
enforces this when the client hello legacy version indicates TLS 1.3
(0x0304), aborting the handshake with an alertMissingExtension alert if
no supported_versions were received.

However, section 4.2.1 indicates different behaviour should be used when
the extension is not present and TLS 1.2 or prior are supported:

  If this extension is not present, servers which are compliant with
  this specification and which also support TLS 1.2 MUST negotiate
  TLS 1.2 or prior as specified in [RFC5246], even if
  ClientHello.legacy_version is 0x0304 or later.

This commit updates the client hello processing logic to allow this
behaviour. If no supported_versions extension was received we ignore the
legacy version being >= TLS 1.3 and instead negotiate a lower supported
version if the server configuration allows.

This fix in turn allows enabling the BoGo ClientHelloVersionTooHigh,
MinorVersionTolerance, and MajorVersionTolerance tests.

Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/671235
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

crypto/tls: fix TLS <1.3 client cert required alert

Previously for protocol versions older than TLS 1.3 our server handshake
implementation sent an alertBadCertificate alert in the case where the
server TLS config indicates a client cert is required and none was
received.

This commit updates the relevant logic to instead send
alertHandshakeFailure in these circumstances.

For TLS 1.2, RFC 5246 §7.4.6 unambiguously describes this as the correct
alert:
  If the client does not send any certificates, the
  server MAY at its discretion either continue the handshake without
  client authentication, or respond with a fatal handshake_failure
  alert.

The TLS 1.1 and 1.0 specs also describe using this alert (RFC 4346 §7.4.6
and RFC 2246 §7.4.6) both say:
  If client authentication is required by the server for the handshake
  to continue, it may respond with a fatal handshake failure alert.

Making this correction also allows enabling the
RequireAnyClientCertificate-TLS1* bogo tests.

Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/671195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>

crypto/tls: enable more large record bogo tests

Previously a handful of large record tests were in the bogo config
ignore list. The ignored tests were failing because they used
insecure ciphersuites that aren't enabled by default.

This commit adds the non-default insecure ciphersuites to the bogo
TLS configuration and re-enables the tests. Doing this uncovered
a handful of unrelated tests that needed to be fixed, each handled
before this commit.

Updates #72006

Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/669158
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

crypto/tls: skip BadRSAClientKeyExchange-[4,5]

These two bogo tests mutate the version number used for the premaster
secret calculation for a client RSA key exchange, with the expectation
the server rejects the handshake.

Per the comment in the end of rsaKeyAgreement.processClientKeyExchange
we explicitly choose *not* to verify the version number.

This commit adds the two version number tests to the ignore list. They
coincidentally happen to produced the expected failure because they use
a non-default ciphersuite. When we add this ciphersuite to the client
config for the bogo test they will start to fail unless ignored.

Updates #72006

Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/669175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>

crypto/tls: err for unsupported point format configs

If a client or server explicitly offers point formats, and the point
formats don't include the uncompressed format, then error. This matches
BoringSSL and Rustls behaviour and allows enabling the
PointFormat-Client-MissingUncompressed bogo test.

Updates #72006

Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/669157
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

crypto/tls: update TLS 1.3 client compression validation

Unlike in earlier TLS versions, in TLS 1.3 when processing a server
hello the legacy_compression_method MUST have the value 0. It is no
longer a parameter that offers a choice of compression method.

With this in mind, it seems more appropriate to return a decode error
when we encounter a non-zero compression method in a server hello
message. We haven't found a parameter value we reject, we've found
a message that doesn't decode according to its specification.

Making this change also aligns with BoringSSL and allows enabling the
TLS13-HRR-InvalidCompressionMethod bogo test.

Updates #72006

Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/669156
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

crypto/tls: use illegal param alert for bad compression

Previously if the clientHandshakeState for the TLS 1.2 client code
encountered a server helo message that contained a compression method
other than compressionNone, we would emit an unexpected message alert.

Instead, it seems more appropriate to return an illegal parameter alert.
The server hello message _was_ expected, it just contained a bad
parameter option.

Making this change also allows enabling the InvalidCompressionMethod
bogo test.

Updates #72006

Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/669155
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

cmd/go: change go tool to build tools missing from GOROOT/pkg/tool

If a tool in cmd is not installed in $GOROOT/pkg/tool/${GOOS}_${GOARCH},
go tool will build (if it's not cached) and run it in a similar way
(with some changes) to how tools declared with tool directives are built
and run.

The main change in how builtin tools are run as compared to mod tools is
that they are built "in host mode" using the running go command's GOOS
and GOARCH. The "-exec" flag is also ignored and we don't add GOROOT/bin
to the PATH.

A ForceHost function has been added to the cfg package to force the
configuration to runtime.GOOS/runtime.GOARCH. It has to recompute the
BuildContext because it's normally determined at init time but we're
changing it after we realize we're running a builtin tool. (Detecting
that we're running a builtin tool at init time would mean replicating
the cmd line parsing logic so recomputing BuildContext sounds like the
smaller change.)

For #71867

Change-Id: I3b2edf2cb985c1dcf5f845fbf39b7dc11dea4df7
Reviewed-on: https://go-review.googlesource.com/c/go/+/666476
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>

cmd/compile/internal/noder: begin documenting meta section

Meta is the most fundamental section. To flesh this out, we discuss references. Primitives are briefly mentioned by pointing to pkgbits,
where they will be defined using a similar grammar.

Change-Id: I7abd899f38fad4cc5caf87ebfc7aa1b1985b17d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/671176
Auto-Submit: Mark Freeman <mark@golang.org>
TryBot-Bypass: Mark Freeman <mark@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>

crypto/internal/fips140/subtle: add assembly implementation of xorBytes for mips64x

goos: linux
goarch: mips64le
pkg: crypto/subtle
                                     │  oldsubtle   │             newsubtle              │
                                     │    sec/op    │   sec/op     vs base               │
ConstantTimeByteEq-4                    5.011n ± 0%   5.014n ± 0%        ~ (p=0.110 n=8)
ConstantTimeEq-4                        3.342n ± 0%   3.342n ± 0%        ~ (p=0.993 n=8)
ConstantTimeLessOrEq-4                  4.455n ± 0%   4.458n ± 0%        ~ (p=0.182 n=8)
XORBytes/8Bytes-4                       36.48n ± 0%   26.73n ± 0%  -26.74% (p=0.000 n=8)
XORBytes/128Bytes-4                     70.21n ± 0%   50.14n ± 0%  -28.59% (p=0.000 n=8)
XORBytes/2048Bytes-4                    566.1n ± 0%   257.2n ± 0%  -54.58% (p=0.000 n=8)
XORBytes/8192Bytes-4                   2123.0n ± 0%   966.8n ± 0%  -54.46% (p=0.000 n=8)
XORBytes/32768Bytes-4                  13.740µ ± 0%   5.614µ ± 0%  -59.14% (p=0.000 n=8)
XORBytesAlignment/8Bytes0Offset-4       38.98n ± 0%   26.53n ± 0%  -31.95% (p=0.000 n=8)
XORBytesAlignment/8Bytes1Offset-4       43.27n ± 0%   26.54n ± 0%  -38.68% (p=0.000 n=8)
XORBytesAlignment/8Bytes2Offset-4       43.28n ± 0%   26.54n ± 0%  -38.69% (p=0.000 n=8)
XORBytesAlignment/8Bytes3Offset-4       43.32n ± 0%   26.54n ± 0%  -38.74% (p=0.000 n=8)
XORBytesAlignment/8Bytes4Offset-4       43.49n ± 0%   26.53n ± 0%  -38.99% (p=0.000 n=8)
XORBytesAlignment/8Bytes5Offset-4       43.53n ± 0%   26.54n ± 0%  -39.03% (p=0.000 n=8)
XORBytesAlignment/8Bytes6Offset-4       43.48n ± 0%   26.53n ± 0%  -38.98% (p=0.000 n=8)
XORBytesAlignment/8Bytes7Offset-4       43.46n ± 1%   26.53n ± 0%  -38.96% (p=0.000 n=8)
XORBytesAlignment/128Bytes0Offset-4     71.84n ± 0%   47.70n ± 1%  -33.60% (p=0.000 n=8)
XORBytesAlignment/128Bytes1Offset-4    260.60n ± 0%   59.87n ± 0%  -77.03% (p=0.000 n=8)
XORBytesAlignment/128Bytes2Offset-4    260.60n ± 0%   59.81n ± 0%  -77.05% (p=0.000 n=8)
XORBytesAlignment/128Bytes3Offset-4    260.55n ± 0%   59.89n ± 0%  -77.01% (p=0.000 n=8)
XORBytesAlignment/128Bytes4Offset-4    260.60n ± 0%   59.84n ± 0%  -77.04% (p=0.000 n=8)
XORBytesAlignment/128Bytes5Offset-4    260.70n ± 0%   59.82n ± 0%  -77.05% (p=0.000 n=8)
XORBytesAlignment/128Bytes6Offset-4    260.60n ± 0%   59.89n ± 0%  -77.02% (p=0.000 n=8)
XORBytesAlignment/128Bytes7Offset-4    260.70n ± 0%   59.85n ± 0%  -77.04% (p=0.000 n=8)
XORBytesAlignment/2048Bytes0Offset-4    552.2n ± 1%   250.0n ± 0%  -54.73% (p=0.000 n=8)
XORBytesAlignment/2048Bytes1Offset-4   3603.0n ± 0%   548.6n ± 0%  -84.77% (p=0.000 n=8)
XORBytesAlignment/2048Bytes2Offset-4   3602.0n ± 0%   548.6n ± 0%  -84.77% (p=0.000 n=8)
XORBytesAlignment/2048Bytes3Offset-4   3604.0n ± 0%   548.6n ± 0%  -84.78% (p=0.000 n=8)
XORBytesAlignment/2048Bytes4Offset-4   3603.5n ± 0%   548.9n ± 0%  -84.77% (p=0.000 n=8)
XORBytesAlignment/2048Bytes5Offset-4   3603.0n ± 0%   548.8n ± 0%  -84.77% (p=0.000 n=8)
XORBytesAlignment/2048Bytes6Offset-4   3602.0n ± 0%   548.6n ± 0%  -84.77% (p=0.000 n=8)
XORBytesAlignment/2048Bytes7Offset-4   3601.5n ± 0%   548.5n ± 0%  -84.77% (p=0.000 n=8)
geomean                                 220.0n        81.91n       -62.77%

                                     │  oldsubtle   │               newsubtle               │
                                     │     B/s      │      B/s       vs base                │
XORBytes/8Bytes-4                      209.1Mi ± 0%    285.5Mi ± 0%   +36.52% (p=0.000 n=8)
XORBytes/128Bytes-4                    1.698Gi ± 0%    2.378Gi ± 0%   +40.04% (p=0.000 n=8)
XORBytes/2048Bytes-4                   3.369Gi ± 0%    7.418Gi ± 0%  +120.17% (p=0.000 n=8)
XORBytes/8192Bytes-4                   3.594Gi ± 0%    7.892Gi ± 0%  +119.59% (p=0.000 n=8)
XORBytes/32768Bytes-4                  2.221Gi ± 0%    5.436Gi ± 0%  +144.76% (p=0.000 n=8)
XORBytesAlignment/8Bytes0Offset-4      195.7Mi ± 0%    287.6Mi ± 0%   +46.96% (p=0.000 n=8)
XORBytesAlignment/8Bytes1Offset-4      176.3Mi ± 0%    287.5Mi ± 0%   +63.06% (p=0.000 n=8)
XORBytesAlignment/8Bytes2Offset-4      176.3Mi ± 0%    287.4Mi ± 0%   +63.07% (p=0.000 n=8)
XORBytesAlignment/8Bytes3Offset-4      176.1Mi ± 0%    287.5Mi ± 0%   +63.25% (p=0.000 n=8)
XORBytesAlignment/8Bytes4Offset-4      175.5Mi ± 0%    287.6Mi ± 0%   +63.90% (p=0.000 n=8)
XORBytesAlignment/8Bytes5Offset-4      175.3Mi ± 0%    287.5Mi ± 0%   +64.02% (p=0.000 n=8)
XORBytesAlignment/8Bytes6Offset-4      175.5Mi ± 0%    287.6Mi ± 0%   +63.86% (p=0.000 n=8)
XORBytesAlignment/8Bytes7Offset-4      175.5Mi ± 0%    287.6Mi ± 0%   +63.85% (p=0.000 n=8)
XORBytesAlignment/128Bytes0Offset-4    1.659Gi ± 0%    2.499Gi ± 1%   +50.61% (p=0.000 n=8)
XORBytesAlignment/128Bytes1Offset-4    468.4Mi ± 0%   2039.0Mi ± 0%  +335.30% (p=0.000 n=8)
XORBytesAlignment/128Bytes2Offset-4    468.4Mi ± 0%   2040.9Mi ± 0%  +335.73% (p=0.000 n=8)
XORBytesAlignment/128Bytes3Offset-4    468.5Mi ± 0%   2038.1Mi ± 0%  +335.02% (p=0.000 n=8)
XORBytesAlignment/128Bytes4Offset-4    468.4Mi ± 0%   2040.0Mi ± 0%  +335.52% (p=0.000 n=8)
XORBytesAlignment/128Bytes5Offset-4    468.2Mi ± 0%   2040.5Mi ± 0%  +335.82% (p=0.000 n=8)
XORBytesAlignment/128Bytes6Offset-4    468.4Mi ± 0%   2038.2Mi ± 0%  +335.13% (p=0.000 n=8)
XORBytesAlignment/128Bytes7Offset-4    468.2Mi ± 0%   2039.4Mi ± 0%  +335.58% (p=0.000 n=8)
XORBytesAlignment/2048Bytes0Offset-4   3.454Gi ± 1%    7.629Gi ± 0%  +120.90% (p=0.000 n=8)
XORBytesAlignment/2048Bytes1Offset-4   542.1Mi ± 0%   3560.1Mi ± 0%  +556.68% (p=0.000 n=8)
XORBytesAlignment/2048Bytes2Offset-4   542.3Mi ± 0%   3560.1Mi ± 0%  +556.48% (p=0.000 n=8)
XORBytesAlignment/2048Bytes3Offset-4   541.9Mi ± 0%   3560.0Mi ± 0%  +556.93% (p=0.000 n=8)
XORBytesAlignment/2048Bytes4Offset-4   542.0Mi ± 0%   3558.8Mi ± 0%  +556.67% (p=0.000 n=8)
XORBytesAlignment/2048Bytes5Offset-4   542.1Mi ± 3%   3558.8Mi ± 0%  +556.53% (p=0.000 n=8)
XORBytesAlignment/2048Bytes6Offset-4   542.2Mi ± 0%   3560.2Mi ± 0%  +556.57% (p=0.000 n=8)
XORBytesAlignment/2048Bytes7Offset-4   542.3Mi ± 0%   3560.5Mi ± 0%  +556.56% (p=0.000 n=8)
geomean                                514.9Mi         1.496Gi       +197.56%

Change-Id: I649fa6bfca31296d65cccdf5fceb3dcfa0c588a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/666255
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

internal/bytealg: optimize the function indexbyte using SIMD on loong64

goos: linux
goarch: loong64
pkg: bytes
cpu: Loongson-3C5000 @ 2200.00MHz
              │     old      │                 new                 │
              │    sec/op    │   sec/op     vs base                │
IndexByte/10     19.32n ± 0%   11.84n ± 0%  -38.72% (p=0.000 n=10)
IndexByte/32     49.34n ± 0%   14.11n ± 0%  -71.40% (p=0.000 n=10)
IndexByte/4K    5608.0n ± 0%   138.8n ± 0%  -97.52% (p=0.000 n=10)
IndexByte/4M    3822.8µ ± 0%   119.4µ ± 0%  -96.88% (p=0.000 n=10)
IndexByte/64M   61.826m ± 1%   3.812m ± 0%  -93.83% (p=0.000 n=10)
geomean          16.61µ        1.602µ       -90.35%

goos: linux
goarch: loong64
pkg: bytes
cpu: Loongson-3A6000-HV @ 2500.00MHz
              │      old      │                 new                  │
              │    sec/op     │    sec/op     vs base                │
IndexByte/10      6.809n ± 0%   5.804n ±  0%  -14.75% (p=0.000 n=10)
IndexByte/32     16.015n ± 0%   6.404n ±  0%  -60.01% (p=0.000 n=10)
IndexByte/4K    1651.00n ± 0%   52.83n ±  0%  -96.80% (p=0.000 n=10)
IndexByte/4M    1680.76µ ± 0%   91.10µ ±  0%  -94.58% (p=0.000 n=10)
IndexByte/64M    26.878m ± 0%   2.010m ± 27%  -92.52% (p=0.000 n=10)
geomean           6.054µ        815.0n        -86.54%

Change-Id: Ib75b997249708f921c6717eba43543c6650bf376
Reviewed-on: https://go-review.googlesource.com/c/go/+/668055
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: sophie zhao <zhaoxiaolin@loongson.cn>

cmd/internal/obj/loong64: add [X]VF{ADD/SUB/MUL/DIV}.{S/D} instructions

Go asm syntax:
 V{ADD/SUB/MUL/DIV}{F/D} VK, VJ, VD
XV{ADD/SUB/MUL/DIV}{F/D} XK, XJ, XD

Equivalent platform assembler syntax:
 vf{add/sub/mul/div}.{s/d} vd, vj, vk
xvf{add/sub/mul/div}.{s/d} xd, xj, xk

Change-Id: I4607884212167ac97d7b6448ea3c849fc0fdd506
Reviewed-on: https://go-review.googlesource.com/c/go/+/670255
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

crypto/sha512: improve performance of loong64

1. Replaced WORD with instruction REVBV.
2. Simplified the implementation of Ch and Maj by reducing instructions, refer to the implementation of riscv64.

goos: linux
goarch: loong64
pkg: crypto/sha512
cpu: Loongson-3A6000-HV @ 2500.00MHz
                  |  bench.old  |             bench.new              |
                  |   sec/op    |   sec/op     vs base               |
Hash8Bytes/New      415.6n ± 0%   398.9n ± 0%  -4.01% (p=0.000 n=10)
Hash8Bytes/Sum384   427.6n ± 0%   409.7n ± 0%  -4.20% (p=0.000 n=10)
Hash8Bytes/Sum512   432.1n ± 0%   415.3n ± 0%  -3.89% (p=0.000 n=10)
Hash1K/New          3.087µ ± 0%   2.931µ ± 0%  -5.05% (p=0.000 n=10)
Hash1K/Sum384       3.094µ ± 0%   2.938µ ± 0%  -5.04% (p=0.000 n=10)
Hash1K/Sum512       3.102µ ± 0%   2.946µ ± 0%  -5.01% (p=0.000 n=10)
Hash8K/New          21.81µ ± 0%   20.67µ ± 0%  -5.25% (p=0.000 n=10)
Hash8K/Sum384       21.81µ ± 0%   20.66µ ± 0%  -5.26% (p=0.000 n=10)
Hash8K/Sum512       21.82µ ± 0%   20.69µ ± 0%  -5.21% (p=0.000 n=10)
geomean             3.061µ        2.915µ       -4.77%

goos: linux
goarch: loong64
pkg: crypto/sha512
cpu: Loongson-3A5000 @ 2500.00MHz
                  |  bench.old  |             bench.new              |
                  |   sec/op    |   sec/op     vs base               |
Hash8Bytes/New      509.4n ± 0%   484.9n ± 0%  -4.79% (p=0.000 n=10)
Hash8Bytes/Sum384   522.9n ± 0%   498.2n ± 0%  -4.71% (p=0.000 n=10)
Hash8Bytes/Sum512   529.0n ± 0%   504.5n ± 0%  -4.63% (p=0.000 n=10)
Hash1K/New          3.578µ ± 0%   3.364µ ± 0%  -5.98% (p=0.000 n=10)
Hash1K/Sum384       3.593µ ± 0%   3.382µ ± 0%  -5.87% (p=0.000 n=10)
Hash1K/Sum512       3.599µ ± 0%   3.386µ ± 0%  -5.93% (p=0.000 n=10)
Hash8K/New          25.10µ ± 0%   23.56µ ± 0%  -6.14% (p=0.000 n=10)
Hash8K/Sum384       25.12µ ± 0%   23.58µ ± 0%  -6.13% (p=0.000 n=10)
Hash8K/Sum512       25.12µ ± 0%   23.59µ ± 0%  -6.12% (p=0.000 n=10)
geomean             3.607µ        3.405µ       -5.59%

Change-Id: I8307ea0fd2d474671f1eef2da2ba5fe899c645d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/668835
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>

crypto/sha256: improve performance of loong64

1. Replaced WORD with instruction REVB2W.
2. Simplified the implementation of Ch and Maj by reducing instructions, refer to the implementation of riscv64.

goos: linux
goarch: loong64
pkg: crypto/sha256
cpu: Loongson-3A6000-HV @ 2500.00MHz
                  |  bench.old  |             bench.new              |
                  |   sec/op    |   sec/op     vs base               |
Hash8Bytes/New      313.9n ± 0%   293.4n ± 0%  -6.53% (p=0.000 n=10)
Hash8Bytes/Sum224   324.0n ± 0%   304.2n ± 0%  -6.11% (p=0.000 n=10)
Hash8Bytes/Sum256   322.8n ± 0%   301.8n ± 0%  -6.51% (p=0.000 n=10)
Hash1K/New          4.513µ ± 0%   4.183µ ± 0%  -7.31% (p=0.000 n=10)
Hash1K/Sum224       4.522µ ± 0%   4.189µ ± 0%  -7.36% (p=0.000 n=10)
Hash1K/Sum256       4.522µ ± 0%   4.190µ ± 0%  -7.34% (p=0.000 n=10)
Hash8K/New          33.92µ ± 0%   31.42µ ± 0%  -7.38% (p=0.000 n=10)
Hash8K/Sum224       33.94µ ± 0%   31.42µ ± 0%  -7.40% (p=0.000 n=10)
Hash8K/Sum256       33.94µ ± 0%   31.42µ ± 0%  -7.41% (p=0.000 n=10)
geomean             3.662µ        3.404µ       -7.04%

goos: linux
goarch: loong64
pkg: crypto/sha256
cpu: Loongson-3A5000 @ 2500.00MHz
                  |  bench.old  |             bench.new              |
                  |   sec/op    |   sec/op     vs base               |
Hash8Bytes/New      382.2n ± 0%   357.3n ± 0%  -6.51% (p=0.000 n=10)
Hash8Bytes/Sum224   392.3n ± 0%   367.0n ± 0%  -6.45% (p=0.000 n=10)
Hash8Bytes/Sum256   393.9n ± 0%   368.8n ± 0%  -6.37% (p=0.000 n=10)
Hash1K/New          5.173µ ± 0%   4.725µ ± 0%  -8.66% (p=0.000 n=10)
Hash1K/Sum224       5.189µ ± 0%   4.742µ ± 0%  -8.62% (p=0.000 n=10)
Hash1K/Sum256       5.188µ ± 0%   4.742µ ± 0%  -8.60% (p=0.000 n=10)
Hash8K/New          38.75µ ± 0%   35.34µ ± 0%  -8.78% (p=0.000 n=10)
Hash8K/Sum224       38.77µ ± 0%   35.35µ ± 0%  -8.80% (p=0.000 n=10)
Hash8K/Sum256       38.76µ ± 0%   35.35µ ± 0%  -8.80% (p=0.000 n=10)
geomean             4.277µ        3.936µ       -7.96%

Change-Id: I561f6db118d05fe44485af8ea25df85afa6905a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/668775
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>

hash: use testhash.TestHash in all hash functions

For #69521

Change-Id: I4e056253f94ad421fcef12d21edaaaf2517b64c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/670179
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: qiu laidongfeng2 <2645477756@qq.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>

internal/testhash: move cryptotest.TestHash to shared package

This test helper can test any hash.Hash, not just crypto hashes. Move
it to $GOROOT/src/internal/testhash so both crypto and hash can use
it.

For #69521

Change-Id: Iac086cca513d5c03936e35d1ab55b8636f4652f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/670178
Reviewed-by: qiu laidongfeng2 <2645477756@qq.com>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

crypto/tls: use runtime.Gosched instead of time.After in TestCertCache

I noticed a failure of this test on a linux/amd64 builder and reproduced
it locally. I can only really reproduce it in a stress test when I
overload my system (`stress2 ./tls.test -test.run=TestCertCache`) but
this points to the root of the problem: it's possible for a timer to get
delayed and the timeout fires before we ever get the chance to check.

After copious debugging printlns, this is essentially what I'd observed.
There would only be one failed check of the reference count from before
it was updated.

Change the test to be a busy-loop again, but call runtime.Gosched. This
is also what we do for the os.Root tests, and in hindsight should've
been my go-to. This has a much higher likelihood of executing promptly.

We may want to go back and understand why the 1 ms timer would fire so
hilariously late the second time. This might be a real bug. For now,
this change makes the test more stable. It no longer fails when it's
hammered under `stress2`.

Fixes #73637.

Change-Id: I316bd9e30946f4c055e61d179c4efc5fe029c608
Reviewed-on: https://go-review.googlesource.com/c/go/+/671175
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/compile/internal/noder: begin a formal UIR grammar.

The UIR export data format can be reasonably expressed using EBNF.
The noder owns the definition of the export data format, so this
seems like a reasonable place to put this.

Change-Id: I0205ab29a3c5e57d670d7fd3164a8bd604ab8e59
Reviewed-on: https://go-review.googlesource.com/c/go/+/670616
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Bypass: Mark Freeman <mark@golang.org>
Auto-Submit: Mark Freeman <mark@golang.org>

runtime: avoid overflow in mutex delay calculation

If cputicks is in the top quarter of the int64's range, adding two
values together will overflow and confuse the subsequent calculations,
leading to zero-duration contention events in the profile.

This fixes the TestRuntimeLockMetricsAndProfile failures on the
linux-s390x builder.

Change-Id: Icb814c39a8702379dfd71c06a53b2618e3589e07
Reviewed-on: https://go-review.googlesource.com/c/go/+/671115
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Rhys Hiltner <rhys.hiltner@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

runtime: remove ptr/scalar bitmap metric

We don't use this mechanism any more, so the metric will always be zero.
Since CL 616255.

Update #73628

Change-Id: Ic179927a8bc24e6291876c218d88e8848b057c2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/671096
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

runtime: schedule cleanups across multiple goroutines

This change splits the finalizer and cleanup queues and implements a new
lock-free blocking queue for cleanups. The basic design is as follows:

The cleanup queue is organized in fixed-sized blocks. Individual cleanup
functions are queued, but only whole blocks are dequeued.

Enqueuing cleanups places them in P-local cleanup blocks. These are
flushed to the full list as they get full. Cleanups can only be enqueued
by an active sweeper.

Dequeuing cleanups always dequeues entire blocks from the full list.
Cleanup blocks can be dequeued and executed at any time.

The very last active sweeper in the sweep phase is responsible for
flushing all local cleanup blocks to the full list. It can do this
without any synchronization because the next GC can't start yet, so we
can be very certain that nobody else will be accessing the local blocks.

Cleanup blocks are stored off-heap because the need to be allocated by
the sweeper, which is called from heap allocation paths. As a result,
the GC treats cleanup blocks as roots, just like finalizer blocks.

Flushes to the full list signal to the scheduler that cleanup goroutines
should be awoken. Every time the scheduler goes to wake up a cleanup
goroutine and there were more signals than goroutines to wake, it then
forwards this signal to runtime.AddCleanup, so that it creates another
goroutine the next time it is called, up to gomaxprocs goroutines.

The signals here are a little convoluted, but exist because the sweeper
and the scheduler cannot safely create new goroutines.

For #71772.
For #71825.

Change-Id: Ie839fde2b67e1b79ac1426be0ea29a8d923a62cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/650697
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>

crypto/tls: add scheduler call to TestCertCache refcount timeout loop

Currently TestCertCache will busy loop waiting for a cleanup (in the
runtime.AddCleanup sense) to execute. If we ever get into this busy
loop, then on single-threaded platforms like js/wasm, we'll end up
_always_ timing out.

This doesn't happen right now because we're getting lucky. The finalizer
goroutine is scheduled into the runnext slot with 'ready' and is thus
scheduled immediately after the GC call. In a follow-up CL, scheduling
cleanup goroutines becomes less aggressive, and thus this test fails.

Although perhaps that CL should schedule cleanup goroutines more
aggressively, the test is still technically buggy, because it expects
busy loops like this to call into the scheduler, but that won't happen
on certain platforms.

Change-Id: I8efe5975be97f4314aec1c8c6e9e22f396be9c94
Reviewed-on: https://go-review.googlesource.com/c/go/+/670755
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/compile: add 2 phiopt cases

Add 2 more cases:

if a { x = value } else { x = a } => x = a && value
if a { x = a } else { x = value } => x = a || value

AND case goes from:

00006 (8) TESTB AX, AX
00007 (8) JNE 9
00008 (13) MOVL AX, BX
00009 (13) MOVL BX, AX
00010 (13) RET

to:

00006 (13) ANDL BX, AX
00007 (13) RET

OR goes from:

00006 (19) TESTB AX, AX
00007 (19) JNE 9
00008 (24) MOVL BX, AX
00009 (24) RET

to:

00006 (24) ORL BX, AX
00007 (24) RET

compilecmp linux/amd64:

runtime
runtime.lock2 847 -> 869  (+2.60%)
runtime.addspecial 542 -> 517  (-4.61%)
runtime.tracebackPCs changed
runtime.scanstack changed
runtime.mallocinit changed
runtime.traceback2 2238 -> 2206  (-1.43%)

runtime [cmd/compile]
runtime.lock2 860 -> 882  (+2.56%)
runtime.scanstack changed
runtime.addspecial 542 -> 517  (-4.61%)
runtime.traceback2 2238 -> 2206  (-1.43%)
runtime.lockWithRank 870 -> 890  (+2.30%)
runtime.tracebackPCs changed
runtime.mallocinit changed

strconv
strconv.ryuFtoaFixed32 changed
strconv.ryuFtoaFixed64 639 -> 638  (-0.16%)
strconv.readFloat changed
strconv.ryuFtoaShortest changed

strings
strings.(*Replacer).build changed

strconv [cmd/compile]
strconv.readFloat changed
strconv.ryuFtoaFixed64 639 -> 638  (-0.16%)
strconv.ryuFtoaFixed32 changed
strconv.ryuFtoaShortest changed

strings [cmd/compile]
strings.(*Replacer).build changed

regexp
regexp.makeOnePass.func1 changed

regexp [cmd/compile]
regexp.makeOnePass.func1 changed

encoding/json
encoding/json.indirect changed

database/sql
database/sql.driverArgsConnLocked changed

vendor/golang.org/x/text/unicode/norm
vendor/golang.org/x/text/unicode/norm.Form.transform changed

go/doc/comment
go/doc/comment.parseSpans changed

internal/diff
internal/diff.tgs changed

log/slog
log/slog.(*handleState).appendNonBuiltIns 1898 -> 1877  (-1.11%)

testing/fstest
testing/fstest.(*fsTester).checkGlob changed

runtime/pprof
runtime/pprof.(*profileBuilder).build changed

cmd/internal/dwarf
cmd/internal/dwarf.isEmptyInlinedCall 254 -> 244  (-3.94%)

go/printer
go/printer.keepTypeColumn 302 -> 270  (-10.60%)
go/printer.(*printer).binaryExpr changed

cmd/compile/internal/syntax
cmd/compile/internal/syntax.(*scanner).rune changed
cmd/compile/internal/syntax.(*scanner).number 2137 -> 2153  (+0.75%)

Change-Id: I7f95f54b03a35d0b616c40f38b415a7feb71be73
Reviewed-on: https://go-review.googlesource.com/c/go/+/666835
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Jakub Ciolek <jakub@ciolek.dev>
TryBot-Bypass: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/vendor: pull in the x/mod/modfile ignore directive

go get golang.org/x/mod@9d33331
go mod tidy
go mod vendor

For #42965

Change-Id: Ib741c96be53214c35058a564116688e7122a205a
Reviewed-on: https://go-review.googlesource.com/c/go/+/670975
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>

cmd/go: replace backslash systematically in path of command

Using the same method CleanPatterns harmonizes further accepted format of patterns in go command.

Fixes #24233

Change-Id: Idb8176df3a7949b16764cd6ea51d7a8966799e42
Reviewed-on: https://go-review.googlesource.com/c/go/+/669775
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

cmd/go: fix incorrect determining default value of CGO_ENABLED

The default value is the value obtained when
no environment variables are set and go env  -w is not used.

In the past,
we used the current value
(may be modified by an environment variable to a non-default value),
error was used as the default value.

For #69994

Change-Id: Iead3a6cacd04dc51a094ffb9f7bb7553320fcd78
Reviewed-on: https://go-review.googlesource.com/c/go/+/621995
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

pkgbits: consolidate doc.go to only relevant details

The stated goal for pkgbits is to implement encoding / decoding of
primitives. However, pkgbits has knowledge of high-level details like
elements, sections, and file layout.

This change starts to clarify pkgbits by paring back documentation to
only those concepts which pkgbits owns. Further CLs are needed to shift
away logic that pkgbits should not own.

Change-Id: Id93003d080f58ffbd6327e2db1a4878500511619
Reviewed-on: https://go-review.googlesource.com/c/go/+/670176
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Mark Freeman <mark@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Mark Freeman <mark@golang.org>

runtime: fix condition to emit gcpacertrace end-of-sweep line

It's the job of the last sweeper to emit the GC pacer trace. The last
sweeper can identify themselves by reducing the count of sweepers, and
also seeing that there's no more sweep work.

Currently this identification is broken, however, because the last
sweeper doesn't check the state they just transitioned sweeping into,
but rather the state they transitioned from (one sweeper, no sweep work
left). By design, it's impossible to transition *out* of this state,
except for another GC to start, but that doesn't take this codepath.

This means lines like

    pacer: sweep done at heap size ...

were missing from the gcpacertrace output for a long time.

This change fixes this problem by having the last sweeper check the
state they just transitioned sweeping to, instead of the state they
transitioned from.

Change-Id: I44bcd32fe2c8ae6ac6c21ba6feb2e7b9e17f60cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/670735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>

crypto,crypto/x509: implement MessageSigner

And use it in crypto/x509. This allows people to implement single-shot
signers which do the hashing themselves.

Fixes #63405

Change-Id: I038c2e10f77b050b6136c4c0a5b031cb416f59aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/654375
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

cmd/internal/obj/riscv: reject invalid vadc/vsbc encodings

The RISC-V Instruction Set Manual Volume states that "for vadc and
vsbc, the instruction encoding is reserved if the destination vector
register is v0". The assembler currently allows instructions like

VADCVVM V1, V2, V0, V0

to be assembled. It's not clear what the behaviour of such
instructions will be on target hardware so it's best to disallow
them.

For reference, binutils (2.44-3.fc42) allows the instruction

vadc.vvm v0, v4, v8, v0

to be assembled and the instruction actually executes on a Banana PI
F3 without crashing. However, clang (20.1.2) refuses to assemble the
instruction, producing the following error.

error: the destination vector register group cannot be V0
        vadc.vvm v0, v4, v8, v0
                 ^
Change-Id: Ia913cbd864ae8dbcf9227f69b963c93a99481cff
Reviewed-on: https://go-review.googlesource.com/c/go/+/669315
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>

cmd/internal/obj/riscv: fix LMUL encoding for MF2 and MF8

The encodings for the riscv64 special operands SPOP_MF2 and SPOP_MF8
are incorrect, i.e., their values are swapped.  This leads to
incorrect encodings for the VSETVLI and VSETIVLI instructions.  The
assembler currently encodes

VSETVLI X10, E32, MF8, TA, MA, X12

as

VSETVLI X10, E32, MF2, TA, MA, X12

We update the encodings for SPOP_MF2 and SPOP_MF8 so that they match
the LMUL table in section "31.3.4. Vector type register, vtype" of
the "RISC-V Instruction Set Manual Volume 1".

Change-Id: Ic73355533d7c2a901ee060b35c2f7af6d58453e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/670016
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Joel Sing <joel@sing.id.au>

cmd/internal/obj/loong64: add [X]VFRINT[{RNE/RZ/RP/RM}].{S/D} instructions

Go asm syntax:
 VFRINT[{RNE/RZ/RP/RM}]{F/D} VJ, VD
XVFRINT[{RNE/RZ/RP/RM}]{F/D} XJ, XD

Equivalent platform assembler syntax:
 vfrint[{rne/rz/rp/rm}].{s/d} vd, vj
xvfrint[{rne/rz/rp/rm}].{s/d} xd, xj

Change-Id: I4ed8782289ae3329d675239f799d5f75b1adc4ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/670235
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>

cmd/compile/internal/test: verify inlining for mutex fast paths

Change-Id: I17568a898ea8514c7b32d2f48c44365ae37cf898
Reviewed-on: https://go-review.googlesource.com/c/go/+/670195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

runtime: fix tag pointers on aix, take 2

Previous fix in CL 667715 wasn't correct for aix.

Change-Id: I44042786079463967165507b15756cf24b9a213a
Reviewed-on: https://go-review.googlesource.com/c/go/+/668036
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

runtime: use "bubble" terminology for synctest

We've settled on calling the group of goroutines started by
synctest.Run a "bubble". At the time the runtime implementation
was written, I was still calling this a "group". Update the code
to match the current terminology.

Change-Id: I31b757f31d804b5d5f9564c182627030a9532f4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/670135
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

runtime, testing/synctest: stop advancing time when main goroutine exits

Once the goroutine started by synctest.Run exits, stop advancing
the fake clock in its bubble. This avoids confusing situations
where a bubble remains alive indefinitely while a background
goroutine reads from a time.Ticker or otherwise advances the clock.

For #67434

Change-Id: Id608ffe3c7d7b07747b56a21f365787fb9a057d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/662155
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>

internal/runtime/maps: make clear also erase tombstones

This will make future uses of the map faster because the probe
sequences will likely be shorter.

Change-Id: If10f3af49a5feaff7d1b82337bbbfb93bcd9dcb5
Reviewed-on: https://go-review.googlesource.com/c/go/+/633076
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>

runtime: remove GODEBUG=runtimecontentionstacks

Go 1.22 promised to remove the setting in a future release once the
semantics of runtime-internal lock contention matched that of
sync.Mutex. That work is done, remove the setting.

Previously reviewed as https://go.dev/cl/585639.

For #66999

Change-Id: I9fe62558ba0ac12824874a0bb1b41efeb7c0853f
Reviewed-on: https://go-review.googlesource.com/c/go/+/668995
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>

runtime: verify attribution of mutex delay

Have the test use the same clock (cputicks) as the profiler, and use the
test's own measurements as hard bounds on the magnitude to expect in the
profile.

Compare the depiction of two users of the same lock: one where the
critical section is fast, one where it is slow. Confirm that the profile
shows the slow critical section as a large source of delay (with #66999
fixed), rather than showing the fast critical section as a large
recipient of delay.

Previously reviewed as https://go.dev/cl/586237.

For #66999

Change-Id: Ic2d78cc29153d5322577d84abdc448e95ed8f594
Reviewed-on: https://go-review.googlesource.com/c/go/+/667616
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>

runtime: blame unlocker for mutex delay

Correct how the mutex contention profile reports on runtime-internal
mutex values, to match sync.Mutex's semantics.

Decide at the start of unlock2 whether we'd like to collect a contention
sample. If so: Opt in to a slightly slower unlock path which avoids
accidentally accepting blame for delay caused by other Ms. Release the
lock before doing an O(N) traversal of the stack of waiting Ms, to
calculate the total delay to those Ms that our critical section caused.
Report that, with the current callstack, in the mutex profile.

Fixes #66999

Change-Id: I561ed8dc120669bd045d514cb0d1c6c99c2add04
Reviewed-on: https://go-review.googlesource.com/c/go/+/667615
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>