Michael Pratt [Mon, 11 Dec 2023 20:55:17 +0000 (15:55 -0500)]
doc/go1.22: remove compiler TODO
For #61422.
Change-Id: Id4eed9affde20857d6b60c0b6d7d1443ea5e460b
Reviewed-on: https://go-review.googlesource.com/c/go/+/548938 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Tolya Korniltsev [Mon, 4 Dec 2023 10:53:29 +0000 (17:53 +0700)]
runtime/pprof: fix generics function names
profileBuilder is using Frame->Function as key for checking if we already
emitted a function. However for generics functions it has dots there [...],
so sometimes for different functions with different generics types,
the profileBuilder emits wrong functions.
Fixes #64528
Change-Id: I8b39245e0b18f4288ce758c912c6748f87cba39a
Reviewed-on: https://go-review.googlesource.com/c/go/+/546815 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Keith Randall [Fri, 8 Dec 2023 21:50:22 +0000 (13:50 -0800)]
runtime: clear mspan.largeType more carefully in the case of arenas
The pointer stored in mspan.largeType is an invalid pointer when
the span is an arena. We need to make sure that pointer isn't seen
by the garbage collector, as it might barf on it. Make sure we
zero the pointer using a uintptr write so the old value isn't picked
up by the write barrier.
The mspan.largeType field itself is in a NotInHeap struct, so a heap
scan won't find it. The only way we find it is when writing it, or
when reading it and putting it in a GC-reachable location. I think we
might need to audit the runtime to make sure these pointers aren't
being passed in places where the GC might (non-conservatively) scan a
stack frame it lives in. (It might be ok, many such places are either
systemstack or nosplit.)
Change-Id: Ie059d054e0da4d48a4c4b3be88b8e1e46ffa7d10
Reviewed-on: https://go-review.googlesource.com/c/go/+/548535 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Russ Cox [Sat, 9 Dec 2023 19:32:10 +0000 (14:32 -0500)]
os: limit temp file randomness to uint32
CL 516860 accidentally changed the randomness
used in TempFile from 32 to 64 bits on 64-bit platforms,
meaning from 10 to 20 decimal bytes.
This is enough to cause problems in a few tests
because it makes temporary directory names just
a little bit longer.
Limit back down to 32 bits of randomness, which is fine,
and add a test to avoid repeating the mistake.
Robert Griesemer [Fri, 8 Dec 2023 05:08:17 +0000 (21:08 -0800)]
go/parser: fix panic in object resolution for invalid type parameter list
This change restores the original logic in parseParameterList to what
it was before CL 538858 (which caused the issue), not in exact wording
but in identical semantic meaning, and thus restores this function to
a state that we know was working fine.
However, the change keeps the improved error reporting introduced by
CL 538858. To keep the code changes somewhat minimal as we are close
to RC1, the improved error handling exists twice for now even though
it could be factored out.
Fixes #64534.
Change-Id: I0b7bbf74d28811e8aae74f838f2d424f78af1f38
Reviewed-on: https://go-review.googlesource.com/c/go/+/548395 Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Bryan C. Mills [Fri, 8 Dec 2023 17:20:35 +0000 (12:20 -0500)]
cmd/go: relax version regexp from CL 547998
In CL 547998 I relaxed cmd/go's parsing of version lines to allow it
to recognize clang versions with vendor prefixes. To prevent false-positives,
I added a check for a version 3-tuple following the word "version".
However, it appears that some releases of GCC use only a 2-tuple instead.
Keith Randall [Thu, 7 Dec 2023 19:09:29 +0000 (11:09 -0800)]
cmd/compile: fix SCCP propagation into jump tables
We can't delete all the outgoing edges and then add one back in, because
then we've lost the argument of any phi at the target. Instead, move
the important target to the front of the list and delete the rest.
This normally isn't a problem, because there is never normally a phi
at the target of a jump table. But this isn't quite true when in race
build mode, because there is a phi of the result of a bunch of raceread
calls.
The reason this happens is that each case is written like this (where e
is the runtime.eface we're switching on):
if e.type == $type.int32 {
m = raceread(e.data, m1)
}
m2 = phi(m1, m)
if e.type == $type.int32 {
.. do case ..
goto blah
}
so that if e.type is not $type.int32, it falls through to the default
case. This default case will have a memory phi for all the (jumped around
and not actually called) raceread calls.
If we instead did it like
if e.type == $type.int32 {
raceread(e.data)
.. do case ..
goto blah
}
That would paper over this bug, as it is the only way to construct
a jump table whose target is a block with a phi in it. (Yet.)
But we'll fix the underlying bug in this CL. Maybe we can do the
rewrite mentioned above later. (It is an optimization for -race mode,
which isn't particularly important.)
Fixes #64606
Change-Id: I6f6e3c90eb1e2638112920ee2e5b6581cef04ea4
Reviewed-on: https://go-review.googlesource.com/c/go/+/548356 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Russ Cox [Thu, 7 Dec 2023 16:11:01 +0000 (11:11 -0500)]
runtime: put runtime.fastrand back temporarily
Callers should be using math/rand/v2.Uint64 instead,
but there are lots of linkname references to runtime.fastrand
in public code. If we break it all now, that will require people
to use //go:build tags to use rand/v2.Uint64 with Go 1.22
and keep using the linkname for earlier versions.
Instead, leave the linkname working and then we can remove
it in Go 1.24, at which point everyone should be able to use
math/rand/v2.Uint64 unconditionally.
Change-Id: I7287ca4f67c270b009562313661cc28a4c2219a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/548235 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Michael Pratt [Wed, 6 Dec 2023 18:29:03 +0000 (13:29 -0500)]
internal/profile: fully decode proto even if there are no samples
This is a partial revert of CL 483137.
CL 483137 started checking errors in postDecode, which is good. Now we
can catch more malformed pprof protos. However this made
TestEmptyProfile fail, so an early return was added when the profile was
"empty" (no samples).
Unfortunately, this was problematic. Profiles with no samples can still
be valid, but skipping postDecode meant that the resulting Profile was
missing values from the string table. In particular, net/http/pprof
needs to parse empty profiles in order to pass through the sample and
period types to a final output proto. CL 483137 broke this behavior.
internal/profile.Parse is only used in two places: in cmd/compile to
parse PGO pprof profiles, and in net/http/pprof to parse before/after
pprof profiles for delta profiles. In both cases, the input is never
literally empty (0 bytes). Even a pprof proto with no samples still
contains some header fields, such as sample and period type. Upstream
github.com/google/pprof/profile even has an explicit error on 0 byte
input, so `go tool pprof` will not support such an input.
Thus TestEmptyProfile was misleading; this profile doesn't need to
support empty input at all.
Resolve this by removing TestEmptyProfile and replacing it with an
explicit error on empty input, as upstream
github.com/google/pprof/profile has. For non-empty input, always run
postDecode to ensure the string table is processed.
TestConvertCPUProfileEmpty is reverted back to assert the values from
before CL 483137. Note that in this case "Empty" means no samples, not a
0 byte input.
Continue to allow empty files for PGO in order to minimize the chance of
last minute breakage if some users have empty files.
Fixes #64566.
Change-Id: I83a1f0200ae225ac6da0009d4b2431fe215b283f
Reviewed-on: https://go-review.googlesource.com/c/go/+/547996 Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Bryan C. Mills [Wed, 6 Dec 2023 20:19:59 +0000 (15:19 -0500)]
cmd/go: accept clang versions with vendor prefixes
To better diagnose bugs like this one in the future, I think
we should also refuse to use a C compiler if we can't identify
a sensible version for it. I did not do that in this CL because
I want it to be small and low-risk for possible backporting.
Fixes #64423.
Change-Id: I21e44fc55f6fcf76633e4fecf6400c226a742351
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/547998
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Bryan C. Mills [Wed, 6 Dec 2023 14:50:42 +0000 (09:50 -0500)]
cmd/go: unset CC when we remove it from PATH in TestScript/cgo_stale_precompiled
Otherwise, if make.bash produced a relative default CC path but the
user has an absolute path to CC set in their environment, the test
will fail spuriously.
For #64423.
Change-Id: I0f3e1d04851585e1b39266badcda9f17489332d9
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/547997
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Ruinan [Thu, 30 Nov 2023 07:54:06 +0000 (15:54 +0800)]
cmd/asm: print absolute PC for all patterns "off(PC)" in testEndToEnd
Before this CL, testEndToEnd only turns the relative PC to absolute PC
when pattern "off(PC)" is the suffix of an instruction. But there are
some instructions like:
ADR 10(PC), R10
it's also acceptable for the assembler while the pattern "off(PC)" is
not a suffix, which makes the test fail.
This CL fixes this issue by searching the pattern in the whole string
instead of only in the suffix.
Change-Id: I0cffedeb7b3c63abca7697671088cf993aff71ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/547235 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ruinan Sun <Ruinan.Sun@arm.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
Joe Tsai [Wed, 6 Dec 2023 22:03:17 +0000 (14:03 -0800)]
doc: add release notes for changes to encoding/json package
The escaping of certain control characters has been changed.
The change is compliant with the JSON specification.
The JSON package never promised exactly how JSON formatted
and has historically changed its representation over time.
Change-Id: I8b23f503cfff86c460f642693b50dee24038fb0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/548075
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
Michael Pratt [Wed, 6 Dec 2023 17:34:02 +0000 (12:34 -0500)]
runtime/pprof: add section headers to Profile doc
Adding explicit section headers makes it cleaner to split the profile
descriptions into multiple paragraphs, as there is now an explicit
transition from discussion of one profile type to the next.
For #14689.
Change-Id: Ifcff918367e91a165ee5f74423be3935b421972b
Reviewed-on: https://go-review.googlesource.com/c/go/+/547955 Reviewed-by: Rhys Hiltner <rhys@justin.tv> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Michael Pratt [Mon, 4 Dec 2023 22:31:58 +0000 (17:31 -0500)]
doc: add STW metrics to release notes
For #63340.
For #61422.
Change-Id: Ib74bb54b0450e96b7f4b7eb7ba2ae7ac2d40171a
Reviewed-on: https://go-review.googlesource.com/c/go/+/547095 Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Michael Pratt [Mon, 4 Dec 2023 21:45:33 +0000 (16:45 -0500)]
doc: add release notes for runtime-internal contention
For #57071.
For #61422.
Change-Id: I5d546d8828be897cb087e85a1251213c582b3894
Reviewed-on: https://go-review.googlesource.com/c/go/+/547059 Reviewed-by: Rhys Hiltner <rhys@justin.tv>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Michael Pratt [Mon, 4 Dec 2023 20:43:41 +0000 (15:43 -0500)]
doc: document runtimecontentionstacks
For #57071.
Change-Id: Ic1645af57aa589917c67154a5e4ad0b4edd7ba90
Reviewed-on: https://go-review.googlesource.com/c/go/+/547058 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Rhys Hiltner <rhys@justin.tv>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Michael Pratt [Mon, 4 Dec 2023 20:13:13 +0000 (15:13 -0500)]
runtime/pprof: document block and mutex profiles
Amazingly, we seem to have nearly no in-tree documentation on the
semantics of block and mutex profiles. Add brief summaries, including
the new behavior from CL 506415 and CL 544195.
For #14689.
For #44920.
For #57071.
For #61015.
Change-Id: I1a6edce7c434fcb43f17c83eb362b1f9d1a32df1
Reviewed-on: https://go-review.googlesource.com/c/go/+/547057
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Rhys Hiltner <rhys@justin.tv>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Russ Cox [Mon, 20 Nov 2023 00:22:48 +0000 (11:22 +1100)]
iter, runtime: add coroutine support
The exported API is only available with GOEXPERIMENT=rangefunc.
This will let Go 1.22 users who want to experiment with rangefuncs
access an efficient implementation of iter.Pull and iter.Pull2.
For #61897.
Change-Id: I6ef5fa8f117567efe4029b7b8b0f4d9b85697fb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/543319 Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
cui fliter [Sat, 7 Oct 2023 08:13:04 +0000 (16:13 +0800)]
doc/go1.22: mention new #vet analyzer
Change-Id: Ib135101bc8adbdb158c5e98bcca14e13d7ac963b
Reviewed-on: https://go-review.googlesource.com/c/go/+/533555
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Tim King <taking@google.com> Reviewed-by: Tim King <taking@google.com>
Bryan C. Mills [Mon, 4 Dec 2023 23:29:35 +0000 (18:29 -0500)]
cmd/go/internal/modfetch/codehost: add new git tags before statLocal instead of after
gitRepo.statLocal reports tag and version information.
If we are statting a hash that corresponds to a tag, we need to add that tag
before calling statLocal so that it can be included in that information.
Fixes #53955.
Updates #56881.
Change-Id: I69a71428e6ed9096d4cb8ed1bb79531415ff06c1
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/547155
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Michael Pratt [Mon, 4 Dec 2023 20:12:25 +0000 (15:12 -0500)]
runtime/metrics: document runtime-internal locks in /sync/mutex/wait/total:seconds
For #57071.
Change-Id: I7ce6c35bed95a6ea3cdc17007f861c5dd82404d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/547056
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Rhys Hiltner <rhys@justin.tv> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Pratt [Mon, 4 Dec 2023 19:28:30 +0000 (14:28 -0500)]
runtime: rename GODEBUG=profileruntimelocks to runtimecontentionstacks
profileruntimelocks is new in CL 544195, but the name is deceptive. Even
with profileruntimelocks=0, runtime-internal locks are still profiled.
The actual difference is that call stacks are not collected. Instead all
contention is reported at runtime._LostContendedLock.
Rename this setting to runtimecontentionstacks to make its name more
aligned with its behavior.
In addition, for this release the default is profileruntimelocks=0,
meaning that users are fairly likely to encounter
runtime._LostContendedLock. Rename it to
runtime._LostContendedRuntimeLock in an attempt to make it more
intuitive that these are runtime locks, not locks in application code.
For #57071.
Change-Id: I38aac28b2c0852db643d53b1eab3f3bc42a43393
Reviewed-on: https://go-review.googlesource.com/c/go/+/547055 Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Rhys Hiltner <rhys@justin.tv>
Than McIntosh [Mon, 27 Nov 2023 17:24:59 +0000 (12:24 -0500)]
cmd/go: use local go cache for cover_statements script test
Use a test-local directory for GOCACHE in "cover_statements" script
test, as a workaround for issue 64014.
For the portion of this test that verifies that caching works
correctly, the cache should theoretically always behave
reliably/deterministically, however if other tests are concurrently
accessing the cache while this test is running, it can lead to cache
lookup failures, which manifest as a flaky failure. To avoid such
flakes, use a separate isolated GOCACHE for this test.
khr@golang.org [Tue, 5 Dec 2023 22:54:10 +0000 (22:54 +0000)]
doc: document new cmp.Or function
Change-Id: I41ea06018daae2c929edc24a714007cede9296ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/547756
TryBot-Bypass: Keith Randall <khr@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
Robert Griesemer [Tue, 5 Dec 2023 21:40:45 +0000 (13:40 -0800)]
doc: add release notes for changes to encoding packages
For #53693.
Change-Id: I360f5cb9caf5fa77267a100eebcc282955677abe
Reviewed-on: https://go-review.googlesource.com/c/go/+/547755 Reviewed-by: Robert Griesemer <gri@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Robert Griesemer [Tue, 5 Dec 2023 21:01:30 +0000 (13:01 -0800)]
doc: add release note for new go/version package
For #62039.
Change-Id: Id19a4c06489ad24dc44c7caf2405d155d96c6fcb
Reviewed-on: https://go-review.googlesource.com/c/go/+/547695 Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Robert Griesemer [Mon, 4 Dec 2023 21:31:23 +0000 (13:31 -0800)]
doc: re-arrange release notes for go/types
Better order in description of changes to go/types.
Move go/types section up so it's in alphabetical order again.
No changes to actual content.
Change-Id: If2f085b665b412489e5dfdba79b7f93598ff2785
Reviewed-on: https://go-review.googlesource.com/c/go/+/546359 Reviewed-by: Alan Donovan <adonovan@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Robert Griesemer [Mon, 4 Dec 2023 21:24:13 +0000 (13:24 -0800)]
doc: add release note for go/types.Alias type and Unalias function
Also, add some missing <code></code> tags.
For #63223.
Change-Id: I570b82be830b3c124420c5715ab1165ca53725f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/546358
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Alan Donovan <adonovan@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Robert Griesemer [Mon, 4 Dec 2023 20:35:35 +0000 (12:35 -0800)]
doc: add release note for go/types/PkgNameOf
For #62037.
Change-Id: Id1d02f88205e5ea62662e78c8313731ec9e55b1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/546975 Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Robert Griesemer [Mon, 4 Dec 2023 19:38:50 +0000 (11:38 -0800)]
doc: add release note for changes to go/types/SizesFor
For #61035.
Change-Id: I27e2c44f9275b508d9dccc50da80896384a4c8fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/546837
TryBot-Bypass: Robert Griesemer <gri@google.com> Reviewed-by: Alan Donovan <adonovan@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Robert Griesemer [Mon, 4 Dec 2023 19:13:32 +0000 (11:13 -0800)]
doc: add release note for go/types.Info.FileVersions
For #62605.
Change-Id: I3c06b835c874c1be5aa5293e3906bdd06c021d87
Reviewed-on: https://go-review.googlesource.com/c/go/+/546836
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Matthew Dempsky [Tue, 5 Dec 2023 20:56:04 +0000 (12:56 -0800)]
cmd/compile: fix escape analysis of string min/max
When I was plumbing min/max support through the compiler, I was
thinking mostly about numeric argument types. As a result, I forgot
that escape analysis would need to be aware that min/max can operate
on string values, which contain pointers.
Fixes #64565.
Change-Id: I36127ce5a2da942401910fa0f9de922726c9f94d
Reviewed-on: https://go-review.googlesource.com/c/go/+/547715 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Alan Donovan [Tue, 5 Dec 2023 20:16:18 +0000 (15:16 -0500)]
doc/go1.22: document ast.Object deprecation
The api.txt changes were originally recorded by mistake (sorry)
into go1.21.txt; see CL 504915, which made the actual change
in August.
Change-Id: If46b48d9714f01605888a6e975c1a03ccfce3b3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/547637
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Commit-Queue: Alan Donovan <adonovan@google.com>
Alan Donovan [Tue, 5 Dec 2023 19:22:53 +0000 (14:22 -0500)]
doc/go_spec.html: rename golang.org to go.dev
Fixes #64513
Change-Id: I45d6a4ba2170308260f3b8f7965affc9948f3fcf
Reviewed-on: https://go-review.googlesource.com/c/go/+/547476 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Russ Cox [Sun, 6 Aug 2023 03:26:28 +0000 (13:26 +1000)]
math/rand, math/rand/v2: use ChaCha8 for global rand
Move ChaCha8 code into internal/chacha8rand and use it to implement
runtime.rand, which is used for the unseeded global source for
both math/rand and math/rand/v2. This also affects the calculation of
the start point for iteration over very very large maps (when the
32-bit fastrand is not big enough).
The benefit is that misuse of the global random number generators
in math/rand and math/rand/v2 in contexts where non-predictable
randomness is important for security reasons is no longer a
security problem, removing a common mistake among programmers
who are unaware of the different kinds of randomness.
The cost is an extra 304 bytes per thread stored in the m struct
plus 2-3ns more per random uint64 due to the more sophisticated
algorithm. Using PCG looks like it would cost about the same,
although I haven't benchmarked that.
Before this, the math/rand and math/rand/v2 global generator
was wyrand (https://github.com/wangyi-fudan/wyhash).
For math/rand, using wyrand instead of the Mitchell/Reeds/Thompson
ALFG was justifiable, since the latter was not any better.
But for math/rand/v2, the global generator really should be
at least as good as one of the well-studied, specific algorithms
provided directly by the package, and it's not.
(Wyrand is still reasonable for scheduling and cache decisions.)
Good randomness does have a cost: about twice wyrand.
Also rationalize the various runtime rand references.
Russ Cox [Thu, 30 Nov 2023 17:25:43 +0000 (12:25 -0500)]
math/rand/v2: add ChaCha8
This is a replay of CL 516859, after its rollback in CL 543895,
with big-endian systems fixed and the tests disabled on RISC-V
since the compiler is broken there (#64285).
ChaCha8 provides a cryptographically strong generator
alongside PCG, so that people who want stronger randomness
have access to that. On systems with 128-bit vector math
assembly (amd64 and arm64), ChaCha8 runs at about the same
speed as PCG (25% slower on amd64, 2% faster on arm64).
Fixes #64284.
Change-Id: I6290bb8ace28e1aff9a61f805dbe380ccdf25b94
Reviewed-on: https://go-review.googlesource.com/c/go/+/546020 Reviewed-by: Filippo Valsorda <filippo@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Wed, 29 Nov 2023 22:18:32 +0000 (22:18 +0000)]
internal/trace/v2: handle thread ID reuse correctly
To begin with, CL 545515 made the trace parser tolerant of
GoCreateSyscall having a P, but that was wrong. Because dropm trashes
the M's syscalltick, that case should never be possible. So the first
thing this change does is it rewrites the test that CL introduced to
expect a failure instead of a success.
What I'd misinterpreted as a case that should be allowed was actually
the same as the other issues causing #64060, which is that the parser
doesn't correctly implement what happens to Ps when a thread calls back
into Go on non-pthread platforms, and what happens when a thread dies
on pthread platorms (or more succinctly, what the runtime does when it
calls dropm).
Specifically, the GoDestroySyscall event implies that if any P is still
running on that M when it's called, that the P stops running. This is
what is intended by the runtime trashing the M's syscalltick; when it
calls back into Go, the tracer models that thread as obtaining a new P
from scratch.
Handling this incorrectly manifests in one of two ways.
On pthread platforms, GoDestroySyscall is only emitted when a C thread
that previously called into Go is destroyed. However, that thread ID can
be reused. Because we have no thread events, whether it's the same
thread or not is totally ambiguous to the tracer. Therefore, the tracer
may observe a thread that previously died try to start running with a
new P under the same identity. The association to the old P is still
intact because the ID is the same, and the tracer gets confused -- it
appears as if two Ps are running on the same M!
On non-pthread platforms, GoDestroySyscall is emitted on every return to
C from Go code. In this case, the same thread with the same identity is
naturally going to keep calling back into Go. But again, since the
runtime trashes syscalltick in dropm, it's always going to acquire a P
from the tracer's perspective. But if this is a different P than before,
just like the pthread case, the parser is going to get confused, since
it looks like two Ps are running on the same M!
The case that CL 545515 actually handled was actually the non-pthread
case, specifically where the same P is reacquired by an M calling back
into Go. In this case, if we tolerate having a P, then what we'll
observe is the M stealing its own P from itself, then running with it.
Now that we know what the problem is, how do we fix it? This change
addresses the problem by emitting an extra event when encountering a
GoDestroySyscall with an active P in its context. In this case, it emits
an additional ProcSteal event to steal from itself, indicating that the
P stopped running. This removes any association between that M and that
P, resolving any ambiguity in the tracer.
There's one other minor detail that needs to be worked out, and that's
what happens to any *real* ProcSteal event that stole the P we're now
emitting an extra ProcSteal event for. Since, this event is going to
look for an M that may have moved on already and the P at this point is
already idle. Luckily, we have *exactly* the right fix for this. The
handler for GoDestroySyscall now moves any active P it has to the
ProcSyscallAbandoned state, indicating that we've lost information about
the P and that it should be treated as already idle. Conceptually this
all makes sense: this is a P in _Psyscall that has been abandoned by the
M it was previously bound to.
It's unfortunate how complicated this has all ended up being, but we can
take a closer look at that in the future.
Fixes #64060.
Change-Id: Ie9e6eb9cf738607617446e3487392643656069a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/546096 Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Tue, 5 Dec 2023 17:31:34 +0000 (17:31 +0000)]
runtime: add the disablethp GODEBUG setting
Go 1.21.1 and Go 1.22 have ceased working around an issue with Linux
kernel defaults for transparent huge pages that can result in excessive
memory overheads. (https://bugzilla.kernel.org/show_bug.cgi?id=93111)
Many Linux distributions disable huge pages altogether these days, so
this problem isn't quite as far-reaching as it used to be. Also, the
problem only affects Go programs with very particular memory usage
patterns.
That being said, because the runtime used to actively deal with this
problem (but with some unpredictable behavior), it's preventing users
that don't have a lot of control over their execution environment from
upgrading to Go beyond Go 1.20.
This change adds a GODEBUG to smooth over the transition. The GODEBUG
setting disables transparent huge pages for all heap memory on Linux,
which is much more predictable than restoring the old behavior.
Fixes #64332.
Change-Id: I73b1894337f0f0b1a5a17b90da1221e118e0b145
Reviewed-on: https://go-review.googlesource.com/c/go/+/547475 Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Robert Griesemer [Mon, 4 Dec 2023 18:45:50 +0000 (10:45 -0800)]
doc: add release note for math/big.Rat.FloatPrec
For #50489.
Change-Id: I4544a24327196eb3ed62af64ae5ddb1f60441d12
Reviewed-on: https://go-review.googlesource.com/c/go/+/546357 Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Damien Neil [Tue, 7 Nov 2023 18:47:56 +0000 (10:47 -0800)]
net/http: limit chunked data overhead
The chunked transfer encoding adds some overhead to
the content transferred. When writing one byte per
chunk, for example, there are five bytes of overhead
per byte of data transferred: "1\r\nX\r\n" to send "X".
Chunks may include "chunk extensions",
which we skip over and do not use.
For example: "1;chunk extension here\r\nX\r\n".
A malicious sender can use chunk extensions to add
about 4k of overhead per byte of data.
(The maximum chunk header line size we will accept.)
Track the amount of overhead read in chunked data,
and produce an error if it seems excessive.
Fixes #64433
Fixes CVE-2023-39326
Change-Id: I40f8d70eb6f9575fb43f506eb19132ccedafcf39
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2076135 Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/547335 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Keith Randall [Fri, 1 Dec 2023 06:59:04 +0000 (22:59 -0800)]
maps: fix aliasing problems with Clone
Make sure to alloc+copy large keys and values instead of aliasing them,
when they might be updated by a future assignment.
Fixes #64474
Change-Id: Ie2226a81cf3897e4e2ee24472f2966d397ace53f
Reviewed-on: https://go-review.googlesource.com/c/go/+/546515 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Cherry Mui [Sat, 2 Dec 2023 21:37:17 +0000 (16:37 -0500)]
cmd/compile: update PGO profile
Since the default.pgo profile is collected, there has been a lot
of development and refactoring in the compiler. It's time to
refresh the compiler's PGO profile. As we are in the freeze, there
will probably not be many major changes to the compiler, so this
should be good for Go 1.22.
Profile collected by running the cmd/compile/profile.sh script on
the linux-amd64-perf gomote.
Michael Anthony Knyszek [Thu, 30 Nov 2023 19:36:46 +0000 (19:36 +0000)]
doc: add release notes for allocation headers
For #61422.
Change-Id: Ia7cb2721e52d9ccd932a49e0c331868eb7749f94
Reviewed-on: https://go-review.googlesource.com/c/go/+/546027 Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Bryan C. Mills [Thu, 30 Nov 2023 16:00:56 +0000 (11:00 -0500)]
testing: add regression tests for reentrant calls to T.Run
These tests represent two patterns of usage, found in Google-internal
tests, that deadlocked after CL 506755.
TestConcurrentRun is a minor variation on TestParallelSub, with the
additional expectation that the concurrent calls to Run (without
explicit calls to Parallel) proceed without blocking. It replaces
TestParallelSub.
TestParentRun is similar, but instead of calling Run concurrently it
calls Run from within the subtest body. It almost certainly represents
an accidental misuse of T.Run, but since that pattern used to run to
completion we don't want to break it accidentally. (Perhaps it should
be diagnosed with a vet check instead?)
While we are testing concurrency, this also cleans up
TestConcurrentCleanup to use a clearer synchronization pattern.
Fixes #64402.
Change-Id: I14fc7e7085a994c284509eac28190c3a8feb04cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/546019
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
This was released in 1.21, release noted in CL 524675.
(I think it was missed originally and then and backported,
but then somehow incorrectly made it into the 1.22 dragnet.)
Change-Id: I45f3182f14f77f8b92a3b7d5ef0011b71fd3c176
Reviewed-on: https://go-review.googlesource.com/c/go/+/546675 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Keith Randall <khr@google.com>
TryBot-Bypass: Keith Randall <khr@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Cherry Mui [Fri, 1 Dec 2023 19:04:35 +0000 (14:04 -0500)]
cmd/link: update flag doc
Update the go doc for linker flags. Remove flags that no longer
exist. Also remove flags that are intended for debugging the
linker from user docs. Add -aslr to the doc.
The -n flag does nothing except print a nearly useless message on
XCOFF linking. Deprecate it.
Fixes #64476.
Change-Id: I518c9c6cc009eae50b7c11308348524ad6a62b69
Reviewed-on: https://go-review.googlesource.com/c/go/+/546615 Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Joel Sing [Sat, 25 Nov 2023 15:48:05 +0000 (02:48 +1100)]
cmd/compile: correct code generation for right shifts on riscv64
The code generation on riscv64 will currently result in incorrect
assembly when a 32 bit integer is right shifted by an amount that
exceeds the size of the type. In particular, this occurs when an
int32 or uint32 is cast to a 64 bit type and right shifted by a
value larger than 31.
Fix this by moving the SRAW/SRLW conversion into the right shift
rules and removing the SignExt32to64/ZeroExt32to64. Add additional
rules that rewrite to SRAIW/SRLIW when the shift is less than the
size of the type, or replace/eliminate the shift when it exceeds
the size of the type.
Add SSA tests that would have caught this issue. Also add additional
codegen tests to ensure that the resulting assembly is what we
expect in these overflow cases.
Fixes #64285
Change-Id: Ie97b05668597cfcb91413afefaab18ee1aa145ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/545035 Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: M Zhuo <mzh@golangcn.org> Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gopher Robot <gobot@golang.org>
Michael Anthony Knyszek [Thu, 30 Nov 2023 18:13:51 +0000 (18:13 +0000)]
runtime: emit the correct P status from a safepoint in the v2 tracer
The GoSyscallBegin event is a signal for both the P and the G to enter a
syscall state for the trace parser. (Ps can't have their own event
because it's too hard to model. As soon as the P enters _Psyscall it can
get stolen out of it.) But there's a window in time between when that
event is emitted and when the P enters _Psyscall where the P's status
can get emitted. In this window the tracer will emit the wrong status:
Running instead of Syscall. Really any call into the tracer could emit a
status event for the P, but in this particular case it's when running a
safepoint function that explicitly emits an event for the P's status.
The fix is straightforward. The source-of-truth on syscall status is the
G's status, so the function that emits the P's status just needs to
check the status of any G attached to it. If it's in _Gsyscall, then the
tracer should emit a Syscall status for the P if it's in _Prunning.
Fixes #64318.
Change-Id: I3b0fb0d41ff578e62810b04fa5a3ef73e2929b0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/546025
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Keith Randall [Thu, 30 Nov 2023 18:04:16 +0000 (10:04 -0800)]
cmd/compile: fix memcombine pass for big endian, > 1 byte elements
The shift amounts were wrong in this case, leading to miscompilation
of load combining.
Also the store combining was not triggering when it should.
Fixes #64468
Change-Id: Iaeb08972c5fc1d6f628800334789c6af7216e87b
Reviewed-on: https://go-review.googlesource.com/c/go/+/546355 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Bryan C. Mills [Wed, 29 Nov 2023 21:13:07 +0000 (16:13 -0500)]
testing: remove tests from the running log while they are waiting on parallel subtests
The parallel subtests are themselves removed from the running map
while they are blocked on calls to t.Parallel, so it is misleading to
log their parents as if they are running when we know they cannot be
making any kind of meaningful progress.
Alan Donovan [Mon, 20 Nov 2023 22:22:37 +0000 (17:22 -0500)]
go/types: set correct Var.scopePos for parameters/results
Previously, its value was unset (NoPos), but the correct
value is a point after the signature (FuncType.End) and
before the body.
Also, fix a bug in Scope.Innermost whereby it would return
the wrong (outer) scope when the query position was in
the FuncType portion of a Func{Decl,Lit}.
The fix is to set the scope's pos/end to those of the
complete Func{Decl,Lit}. This is now documented at
Info.Scopes, along with other missing information.
Also, fix a bug in the go/types (but not types2) scope
test, in which comments were discarded by the parser,
causing the entire test to be a no-op (!).
Also, make failures of TestScopeLookupParent more
informative.
Also, add a release note about the change in behavior.
Fixes #64292
Fixes #64295
Change-Id: Ib681f59d1b0b43de977666db08302d7524d3305f
Reviewed-on: https://go-review.googlesource.com/c/go/+/544035 Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Cosmos Nicolaou [Wed, 29 Nov 2023 19:04:57 +0000 (11:04 -0800)]
runtime/pprof: retry vmmap invocation if it failed due to a reported temporary resource shortage
As per #62352 the invocation of vmmap may fail (very rarely) due to
a temporary lack of resources on the test runner machine. This PR
allows for retrying the invocation a fixed number of times before
giving up. This is because we suspect the failure is due to
sensible to retry.
Dmitri Shuralyov [Wed, 29 Nov 2023 20:42:53 +0000 (15:42 -0500)]
all: update vendored dependencies
The Go 1.22 code freeze has recently started. This is a time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.
Generated with:
go install golang.org/x/build/cmd/updatestd@latest
go install golang.org/x/tools/cmd/bundle@latest
updatestd -goroot=$(pwd) -branch=master
For #36905.
Change-Id: I76525261b9a954ed21a3bd3cb6c4a12e6c031d80
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-linux-amd64-longtest,gotip-linux-386-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/546055
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Keith Randall [Tue, 28 Nov 2023 21:09:42 +0000 (13:09 -0800)]
doc: release notes for GOARM hardfloat/softfloat change
See CL 514907
Change-Id: Ieba2d7737115c66990b0ea7629033e787a99be93
Reviewed-on: https://go-review.googlesource.com/c/go/+/545655 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Ludi Rehak <ludi317@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org>