Michael Matloob [Wed, 21 May 2025 19:19:16 +0000 (15:19 -0400)]
cmd/doc: add more convenient behavior for go doc -http with no args
If we're in a module, go to the module's page. Outside of a module, but
in a workspace go to the home page, and outside of a module or
workspace, show the stdlib docs.
For #68106
Change-Id: I911a90a0e2b0a2bbb622f56e32827d5bdfa7f2fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/675235 Reviewed-by: Michael Matloob <matloob@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
The crypto/tls package produces the expected error for this test case,
and so it can be enabled.
Looking at the history of the relevant code it appears the TLS 1.3
implementation has always had the correct behaviour for HRR changing to
an unsupported group after the initial hello.
I think this test was skipped initially because at the time of
initial BoGo config commit we hadn't implemented the -curves argument
for the test shim yet, and this test relies on it. We later added
support for that flag alongside X25519Kyber768Draft00 KX and I think we
missed the chance to enable the test then.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/673756 Reviewed-by: Roland Shoemaker <roland@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org> Reviewed-by: David Chase <drchase@google.com>
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net>
Daniel McCarney [Fri, 16 May 2025 17:44:53 +0000 (13:44 -0400)]
crypto/tls: match compression method alert across versions
When a pre-TLS 1.3 server processes a client hello message that
indicates compression methods that don't include the null compression
method, send an illegal parameter alert.
Previously we did this for TLS 1.3 server handshakes only, and the
legacy TLS versions used alertHandshakeFailure for this circumstance. By
switching this to alertIllegalParameter we use a consistent alert across
all TLS versions, and can also enable the NoNullCompression-TLS12 BoGo
test we were skipping.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/673736
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net> Reviewed-by: Roland Shoemaker <roland@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org> Reviewed-by: David Chase <drchase@google.com>
Daniel McCarney [Fri, 16 May 2025 17:08:16 +0000 (13:08 -0400)]
crypto/tls: verify server chooses advertised curve
When a crypto/tls client using TLS < 1.3 sends supported elliptic_curves
in a client hello message the server must limit itself to choosing one
of the supported options from our message. If we process a server key
exchange message that chooses an unadvertised curve, abort the
handshake w/ an error.
Previously we would not note that the server chose a curve we didn't
include in the client hello message, and would proceed with the
handshake as long as the chosen curve was one that we've implemented.
However, RFC 8422 5.1 makes it clear this is a server acting
out-of-spec, as it says:
If a server does not understand the Supported Elliptic Curves
Extension, does not understand the Supported Point Formats Extension,
or is unable to complete the ECC handshake while restricting itself
to the enumerated curves and point formats, it MUST NOT negotiate the
use of an ECC cipher suite.
Changing our behaviour to enforce this also allows enabling the
UnsupportedCurve BoGo test.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/673735
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net> Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net> Reviewed-by: Filippo Valsorda <filippo@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
thepudds [Sat, 1 Mar 2025 19:30:03 +0000 (14:30 -0500)]
reflect: optimize IsZero with a pointer comparison to global zeroVal
Our prior CL 649078 teaches the compiler to use a pointer to
runtime.zeroVal as the data pointer for an interface in cases it where
it can see that a zero value struct or array is being used in
an interface conversion.
This applies to some uses with reflect, such as:
s := S{}
v := reflect.ValueOf(s)
This CL builds on that to do a cheap pointer check in reflect.IsZero
to see if the Value points to runtime.zeroVal, which means it is a zero
value.
An alternative might be to do an initial pointer check in the typ.Equal
function for types where it makes sense to do but doesn't already.
This CL gives a performance boost of -51.71% geomean for
BenchmarkZero/IsZero, with most of the impact there on
arrays of structs. (The left column is CL 649078 and the right column
is this CL).
Note these are of course micro benchmarks with easily predicted
branches. The extra branch we introduce in the CL might hurt if there
was for example a tight loop where 50% of the values used the
global zeroVal and 50% didn't in a way that is not well predicted,
although if the typ.Equal for many types already does an initial
pointer check, it might not matter much.
For the older BenchmarkIsZero in reflect, this change does not help.
(The compiler does not use the global zeroVal as the data word for the
interfaces in this benchmark because values are part of a larger value
that is too big to be used in the global zeroVal, and also a piece of
the larger value is mutated and is not zero).
Change-Id: Ie083853a5bff03856277a293d94532a681f4a8d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/654135 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
thepudds [Wed, 12 Feb 2025 23:45:42 +0000 (18:45 -0500)]
cmd/compile/internal/walk: use global zeroVal in interface conversions for zero values
This is a small-ish adjustment to the change earlier in our
stack in CL 649555, which started creating read-only global storage
for a composite literal used in an interface conversion and setting
the interface data pointer to point to that global storage.
In some cases, there are execution-time performance benefits to point
to runtime.zeroVal in particular. In reflect, pointer checks against
the runtime.zeroVal memory address are used to side-step some work,
such as in reflect.Value.Set and reflect.Value.IsZero.
In this CL, we therefore dig up the zeroVal symbol, and we use the
machinery from earlier in our stack to use a pointer to zeroVal for
the interface data pointer if we see examples like:
sink = S{}
or:
s := S{}
sink = s
CL 649076 (also earlier in our stack) added most of the tests
along with debug diagnostics in convert.go to make it easier
to test this change.
We add a benchmark in reflect to show examples of performance benefit.
The left column is our immediately prior CL 649555, and the right is
this CL. (The arrays of structs here do not seem to benefit, which
we attempt to address in our next CL).
Finally, here are results from the benchmark example from #71323.
Note however that almost all the benefit shown here is from our earlier
CL 649555, which is a more general purpose change and eliminates
the allocation using a different read-only global than this CL.
Change-Id: I64d8cf1a7900f011d2ec59b948388aeda1150676
Reviewed-on: https://go-review.googlesource.com/c/go/+/649078
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com>
thepudds [Fri, 14 Feb 2025 04:49:50 +0000 (23:49 -0500)]
cmd/compile/internal/walk: convert composite literals to interfaces without allocating
Today, this interface conversion causes the struct literal
to be heap allocated:
var sink any
func example1() {
sink = S{1, 1}
}
For basic literals like integers that are directly used in
an interface conversion that would otherwise allocate, the compiler
is able to use read-only global storage (see #18704).
This CL extends that to struct and array literals as well by creating
read-only global storage that is able to represent for example S{1, 1},
and then using a pointer to that storage in the interface
when the interface conversion happens.
A more challenging example is:
func example2() {
v := S{1, 1}
sink = v
}
In this case, the struct literal is not directly part of the
interface conversion, but is instead assigned to a local variable.
To still avoid heap allocation in cases like this, in walk we
construct a cache that maps from expressions used in interface
conversions to earlier expressions that can be used to represent the
same value (via ir.ReassignOracle.StaticValue). This is somewhat
analogous to how we avoided heap allocation for basic literals in
CL 649077 earlier in our stack, though here we also need to do a
little more work to create the read-only global.
CL 649076 (also earlier in our stack) added most of the tests
along with debug diagnostics in convert.go to make it easier
to test this change.
Change-Id: I8924f0c69ff738ea33439bd6af7b4066af493b90
Reviewed-on: https://go-review.googlesource.com/c/go/+/649555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@google.com>
Filippo Valsorda [Wed, 21 May 2025 17:41:06 +0000 (19:41 +0200)]
crypto/internal/fips140: use hash.Hash
Since package hash is just the interface definition, not an
implementation, we can make a good argument that it doesn't impact the
security of the module and can be imported from outside.
For #69521
Change-Id: I6a6a4656b9c3cac8bb9ab8e8df11fa3238dc5d1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/674917 Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Daniel McCarney <daniel@binaryparadox.net> Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Daniel McCarney [Thu, 15 May 2025 17:41:14 +0000 (13:41 -0400)]
crypto/tls: have servers prefer TLS 1.3 when supported
Previously the common Config.mutualVersion() code prioritized the
selected version based on the provided peerVersions being sent in peer
preference order.
Instead we would prefer to see TLS 1.3 used whenever it is
supported, even if the peer would prefer an older protocol version.
This commit updates mutualVersions() to implement this policy change.
Our new behaviour matches the behaviour of other TLS stacks, notably
BoringSSL, and so also allows enabling the IgnoreClientVersionOrder BoGo
test that we otherwise must skip.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/673236
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Roland Shoemaker <roland@golang.org>
Roland Shoemaker [Wed, 7 May 2025 18:37:52 +0000 (11:37 -0700)]
crypto/tls: add GetEncryptedClientHelloKeys
This allows servers to rotate their ECH keys without needing to restart
the server.
Fixes #71920
Change-Id: I55591ab3303d5fde639038541c50edcf1fafc9aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/670655
TryBot-Bypass: Roland Shoemaker <roland@golang.org> Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org> Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Roland Shoemaker [Fri, 14 Mar 2025 00:04:30 +0000 (17:04 -0700)]
crypto: drop pre-AVX2 SHA assembly implementations
Drop the entire pre-AVX2 assembly implementation of SHA-1, SHA-256, and
SHA-512. This also technically impacts the SHA-1 AVX2 implementation,
since it previously called the pre-AVX2 implementation for the last
block if the number of blocks wasn't a multiple of 2.
Instead of keeping the entire implementation just for that case, we
just call the generic implementation for the last block. This will be a
little slower, but still seems like a win.
Updates #69587
Change-Id: Id5234c42910d8c6ec6b8df700a721c0953dff02b
Reviewed-on: https://go-review.googlesource.com/c/go/+/657716 Reviewed-by: Daniel McCarney <daniel@binaryparadox.net> Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Filippo Valsorda <filippo@golang.org>
Julian Zhu [Wed, 21 May 2025 08:36:53 +0000 (16:36 +0800)]
cmd/compile/internal: intrinsify publicationBarrier on mipsx
This enables publicationBarrier to be used as an intrinsic on mipsx.
Change-Id: Ic199f34b84b3058bcfab79aac8f2399ff21a97ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/674856 Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com>
Mateusz Poliwczak [Tue, 20 May 2025 16:21:22 +0000 (16:21 +0000)]
crypto/x509: disallow negative path length
pathLenConstraint is restricted to unsigned integers.
Also the -1 value of cert.MaxPathLength has a special
meaning, so we shouldn't allow unmarshaling -1.
Julian Zhu [Tue, 20 May 2025 02:55:33 +0000 (10:55 +0800)]
cmd/compile/internal: intrinsify publicationBarrier on mips64x
This enables publicationBarrier to be used as an intrinsic on mips64x.
Change-Id: I4030ea65086c37ee1dcc1675d0d5d40ef8683851
Reviewed-on: https://go-review.googlesource.com/c/go/+/674855
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Michael Anthony Knyszek [Wed, 21 May 2025 17:50:15 +0000 (17:50 +0000)]
internal/trace: skip clock snapshot checks on Windows in stress mode
Windows' monotonic and wall clock granularity is just too coarse to get
reasonable values out of stress mode, which is creating new trace
generations constantly.
Fixes #73813.
Change-Id: Id9cb2fed9775ce8d78a736d0164daa7bf45075e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/675096 Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com> Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
thepudds [Wed, 12 Feb 2025 23:55:04 +0000 (18:55 -0500)]
cmd/compile/internal/escape: propagate constants to interface conversions to avoid allocs
Currently, the integer value in the following interface conversion gets
heap allocated:
v := 1000
fmt.Println(v)
In contrast, this conversion does not currently cause the integer value
to be heap allocated:
fmt.Println(1000)
The second example is able to avoid heap allocation because of an
optimization in walk (by Josh in #18704 and related issues) that
recognizes a literal is being used. In the first example, that
optimization is currently thwarted by the literal getting assigned
to a local variable prior to use in the interface conversion.
This CL propagates constants to interface conversions like
in the first example to avoid heap allocations, instead using
a read-only global. The net effect is roughly turning the first example
into the second.
One place this comes up in practice currently is with logging or
debug prints. For example, if we have something like:
func conditionalDebugf(format string, args ...interface{}) {
if debugEnabled {
fmt.Fprintf(io.Discard, format, args...)
}
}
Prior to this CL, this integer is heap allocated, even when the
debugEnabled flag is false, and even when the compiler
inlines conditionalDebugf:
v := 1000
conditionalDebugf("hello %d", v)
With this CL, the integer here is no longer heap allocated, even when
the debugEnabled flag is enabled, because the compiler can now see that
it can use a read-only global.
See the writeup in #71359 for more details.
CL 649076 (earlier in our stack) added most of the tests
along with debug diagnostics in convert.go to make it easier
to test this change.
Change-Id: I19a51e74b36576ebb0b9cf599267cbd2bd847ce4
Reviewed-on: https://go-review.googlesource.com/c/go/+/649079
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@google.com>
cmd/compile/internal/escape: use an ir.ReassignOracle
Using the new-ish ir.ReassignOracle is more efficient than calling
ir.StaticValue repeatedly.
This CL now uses an ir.ReassignOracle for the recent
make constant propagation introduced in CL 649035.
We also pull the main change from CL 649035 into a new function,
which we will update later in our stack. We will also use the
ReassignOracles introduced here later in our stack.
(We originally did most of this work in CL 649077, but we abandoned
that in favor of CL 649035).
We could also use an ir.ReassignOracle in the older processing of
ir.OCALLFUNC in (*escape).call, but for now, we just leave that
as a TODO.
Updates #71359
Change-Id: I6e02eeac269bde3a302622b4dfe0c8dc63ec9ffc
Reviewed-on: https://go-review.googlesource.com/c/go/+/673795 Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
Michael Pratt [Wed, 21 May 2025 14:58:32 +0000 (10:58 -0400)]
cmd/link: require cgo internal linking in TestIssue33979
This was a typo regression in CL 643897, which accidentally dropped the
requirement for cgo internal linking. As a result, this test is
continuously failing on windows-arm64.
For #71395.
Cq-Include-Trybots: luci.golang.try:gotip-windows-arm64
Change-Id: I6a6a636c25fd399cda6649ef94655aa112f10f63
Reviewed-on: https://go-review.googlesource.com/c/go/+/675015
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Roland Shoemaker [Sat, 25 Jan 2025 18:28:02 +0000 (10:28 -0800)]
crypto/tls: replace custom intern cache with weak cache
Uses the new weak package to replace the existing custom intern cache
with a map of weak.Pointers instead. This simplifies the cache, and
means we don't need to store a slice of handles on the Conn anymore.
Change-Id: I5c2bf6ef35fac4255e140e184f4e48574b34174c
Reviewed-on: https://go-review.googlesource.com/c/go/+/644176
TryBot-Bypass: Roland Shoemaker <roland@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
Michael Pratt [Mon, 5 May 2025 17:44:26 +0000 (13:44 -0400)]
runtime: use cgroup CPU limit to set GOMAXPROCS
This CL adds two related features enabled by default via compatibility
GODEBUGs containermaxprocs and updatemaxprocs.
On Linux, containermaxprocs makes the Go runtime consider cgroup CPU
bandwidth limits (quota/period) when setting GOMAXPROCS. If the cgroup
limit is lower than the number of logical CPUs available, then the
cgroup limit takes precedence.
On all OSes, updatemaxprocs makes the Go runtime periodically
recalculate the default GOMAXPROCS value and update GOMAXPROCS if it has
changed. If GOMAXPROCS is set manually, this update does not occur. This
is intended primarily to detect changes to cgroup limits, but it applies
on all OSes because the CPU affinity mask can change as well.
The runtime only considers the limit in the leaf cgroup (the one that
actually contains the process), caching the CPU limit file
descriptor(s), which are periodically reread for updates. This is a
small departure from the original proposed design. It will not consider
limits of parent cgroups (which may be lower than the leaf), and it will
not detection cgroup migration after process start.
We can consider changing this in the future, but the simpler approach is
less invasive; less risk to packages that have some awareness of runtime
internals. e.g., if the runtime periodically opens new files during
execution, file descriptor leak detection is difficult to implement in a
stable way.
For #73193.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Change-Id: I6a6a636c631c1ae577fb8254960377ba91c5dc98
Reviewed-on: https://go-review.googlesource.com/c/go/+/670497
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Roland Shoemaker [Sun, 19 Jan 2025 17:24:50 +0000 (09:24 -0800)]
crypto/sha1: add sha-ni AMD64 implementation
Based on the Intel docs. Provides a ~44% speed-up compared to the AVX
implementation and a ~57% speed-up compared to the generic AMD64
assembly implementation.
Roland Shoemaker [Sat, 22 Mar 2025 00:58:55 +0000 (00:58 +0000)]
runtime: add valgrind instrumentation
Add build tag gated Valgrind annotations to the runtime which let it
understand how the runtime manages memory. This allows for Go binaries
to be run under Valgrind without emitting spurious errors.
Instead of adding the Valgrind headers to the tree, and using cgo to
call the various Valgrind client request macros, we just add an assembly
function which emits the necessary instructions to trigger client
requests.
In particular we add instrumentation of the memory allocator, using a
two-level mempool structure (as described in the Valgrind manual [0]).
We also add annotations which allow Valgrind to track which memory we
use for stacks, which seems necessary to let it properly function.
We describe the memory model to Valgrind as follows: we treat heap
arenas as a "pool" created with VALGRIND_CREATE_MEMPOOL_EXT (so that we
can use VALGRIND_MEMPOOL_METAPOOL and VALGRIND_MEMPOOL_AUTO_FREE).
Within the pool we treat spans as "superblocks", annotated with
VALGRIND_MEMPOOL_ALLOC. We then allocate individual objects within spans
with VALGRIND_MALLOCLIKE_BLOCK.
It should be noted that running binaries under Valgrind can be _quite
slow_, and certain operations, such as running the GC, can be _very
slow_. It is recommended to run programs with GOGC=off. Additionally,
async preemption should be turned off, since it'll cause strange
behavior (GODEBUG=asyncpreemptoff=1).
Running Valgrind with --leak-check=yes will result in some errors
resulting from some things not being marked fully free'd. These likely
need more annotations to rectify, but for now it is recommended to run
with --leak-check=off.
Michael Matloob [Wed, 21 May 2025 16:18:03 +0000 (12:18 -0400)]
cmd/doc: allow go doc -http without package in current directory
go doc tries to find a package to display documentation for. In the case
that no package is provided, it uses "." just like go list does. So if
go doc -http is run without any arguments, it tries to show the
documentation for the package in the current directory. As a special
case, if no arguments are provided, allow no package to match the
current directory and just open the root pkgsite page.
For #68106
Change-Id: I6d65b160a838591db953fac630eced6b09106877
Reviewed-on: https://go-review.googlesource.com/c/go/+/675075 Reviewed-by: Alan Donovan <adonovan@google.com> Reviewed-by: Michael Matloob <matloob@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Tue, 20 May 2025 20:26:56 +0000 (20:26 +0000)]
runtime: guarantee checkfinalizers test allocates in a shared tiny block
Currently the checkfinalizers test (TestDetectCleanupOrFinalizerLeak)
only *tries* to ensure the tiny alloc with a cleanup attached shares a
block with other objects. However, what it does is insufficient, because
it could get unlucky and have the last object allocated be the first
object of a new block.
This change changes the test to guarantee that a tiny object is not at
the start of a fresh block by looking at the alignment of the object's
pointer. If the object's pointer is odd, then that's good enough to know
that it shares a block with something else, since the blocks themselves
are aligned to a much higher power of two.
This fixes a failure I've seen on the builders.
Fixes #73810.
Change-Id: Ieafdbb9cccb0d2dc3659a9a5d9d9233718461635
Reviewed-on: https://go-review.googlesource.com/c/go/+/674655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Mark Freeman [Mon, 19 May 2025 22:00:31 +0000 (18:00 -0400)]
internal/pkgbits: indent productions and hoist some types up
The types being hoisted are those which cannot be referenced; that is,
where Ref[T] is illegal. These are most clearly owned by pkgbits. The
types which follow are those which can be referenced.
Referenceable types are more hazy due to the reference mechanism of UIR
- sections. These are a detail of the UIR file format and are surfaced
directly to importers.
I suspect that pkgbits would benefit from a reference mechanism not
dependent on sections. This would permit us to push down many types
from the noder into pkgbits, reducing the interface surface without
giving up deduplication.
Change-Id: Ifaf5cd9de20c767ad0941413385b308d628aac6c
Reviewed-on: https://go-review.googlesource.com/c/go/+/674635
Auto-Submit: Mark Freeman <mark@golang.org>
TryBot-Bypass: Mark Freeman <mark@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
Felix Geisendörfer [Fri, 28 Feb 2025 16:30:54 +0000 (17:30 +0100)]
internal/trace: add generator tests for sync events
Add generator tests that verify the timestamps for the sync events
emitted in the go1.25 trace format and earlier versions.
Add the ability to configure the properties of the per-generation sync
batches in testgen. Also refactor testgen to produce more realistic
timestamps by keeping track of lastTs and using it for structural
batches that don't have their own timestamps. Otherwise they default to
zero which means the minTs of the generation can't be controlled.
For #69869
Change-Id: I92a49b8281bc4169b63e13c030c1de7720cd6f26
Reviewed-on: https://go-review.googlesource.com/c/go/+/653876
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Felix Geisendörfer [Fri, 28 Feb 2025 16:28:43 +0000 (17:28 +0100)]
internal/trace/internal/testgen: make generated trace version configurable
Replace hard coded references to version.Go122 with the trace version
passed to NewTrace. This allows writing testgen tests for newer trace
versions.
For #69869
Change-Id: Id25350cea1c397a09ca23465526ff259e34a4752
Reviewed-on: https://go-review.googlesource.com/c/go/+/653875
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Felix Geisendörfer [Sat, 3 May 2025 12:56:23 +0000 (15:56 +0300)]
internal/trace: add a validator test for the new clock snapshots
Check that the clock snapshots, when expected to be present, are
non-zero and monotonically increasing.
This required some refactoring to make the validator aware of the
version of the trace it is validating.
Change-Id: I04c4dd10fe6975cbac12bb0ddaebcec3a5284e7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/669715
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com>
Felix Geisendörfer [Fri, 28 Feb 2025 15:07:45 +0000 (16:07 +0100)]
internal/trace: expose clock snapshot timestamps on sync event
Add ClockSnapshot field to the Sync event type and populate it with the
information from the new EvClockSnapshot event when available.
For #69869
Change-Id: I3b24b5bfa15cc7a7dba270f5e6bf189adb096840
Reviewed-on: https://go-review.googlesource.com/c/go/+/653576 Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Felix Geisendörfer [Fri, 28 Feb 2025 15:07:16 +0000 (16:07 +0100)]
runtime,internal/trace: emit clock snapshots at the start of trace generations
Replace the per-generation EvEventBatch containing a lone EvFrequency
event with a per-generation EvEventBatch containing a EvSync header
followed by an EvFrequency and EvClockSnapshot event.
The new EvClockSnapshot event contains trace, mono and wall clock
snapshots taken in close time proximity. Ignoring minor resolution
differences, the trace and mono clock are the same on linux, but not on
windows (which still uses a TSC based trace clock).
Emit the new sync batch at the very beginning of every new generation
rather than the end to be in harmony with the internal/trace reader
which emits a sync event at the beginning of every generation as well
and guarantees monotonically increasing event timestamps.
Bump the version of the trace file format to 1.25 since this change is
not backwards compatible.
Update the internal/trace reader implementation to decode the new
events, but do not expose them to the public reader API yet. This is
done in the next CL.
For #69869
Change-Id: I5bfedccdd23dc0adaf2401ec0970cbcc32363393
Reviewed-on: https://go-review.googlesource.com/c/go/+/653575 Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
The RISC-V integer vector multiply add instructions are not encoded
correctly; the first and second arguments are swapped. For example,
the instruction
This is inconsistent with the argument ordering we use for 3
argument vector instructions, in which the argument order, as given
in the RISC-V specifications, is reversed, and also with the vector
FMA instructions which have the same argument ordering as the vector
integer multiply add instructions in the "The RISC-V Instruction Set
Manual Volume I". For example, in the ISA manual we have the
following instruction definitions
Damien Neil [Wed, 21 May 2025 00:02:11 +0000 (17:02 -0700)]
os: skip TestOpenFileCreateExclDanglingSymlink when no symlinks
Skip this test on plan9, and any other platform that doesn't
have symlinks.
Fixes #73729
Change-Id: I8052db24ed54c3361530bd4f54c96c9d10c4714c
Reviewed-on: https://go-review.googlesource.com/c/go/+/674697
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Commit-Queue: Alan Donovan <adonovan@google.com> Reviewed-by: Richard Miller <millerresearch@gmail.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Xiaolin Zhao [Fri, 16 May 2025 03:05:03 +0000 (11:05 +0800)]
crypto/subtle: optimize function xorBytes using SIMD on loong64
On the Loongson-3A6000-HV and Loongson-3A5000, there has been
a significant improvement in all performance metrics except
for '8Bytes', which has experienced a decline, as follows.
Xiaolin Zhao [Mon, 19 May 2025 09:02:48 +0000 (17:02 +0800)]
cmd/compile: fix the implementation of NORconst on loong64
In the loong64 instruction set, there is no NORI instruction,
so the immediate value in NORconst need to be stored in register
and then use the three-register NOR instruction.
Change-Id: I5ef697450619317218cb3ef47fc07e238bdc2139
Reviewed-on: https://go-review.googlesource.com/c/go/+/673836 Reviewed-by: abner chenc <chenguoqi@loongson.cn> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
thepudds [Wed, 12 Mar 2025 21:29:25 +0000 (17:29 -0400)]
cmd/compile/internal/escape: improve order of work to speed up analyzing many locations
For the package github.com/microsoft/typescript-go/internal/checker,
compilation currently spends most of its time in escape analysis.
Here, we re-order work to be more efficient when analyzing many
locations, and delay visiting some locations to prioritize locations
that might be more likely to reach a terminal point of reaching the
heap and possibly reduce the count of intermediate states for each location.
Action graph reported build times show roughly a 5x improvement for
compilation of the typescript-go/internal/checker package:
go1.24.0: 91.792s
cl-657179-ps1: 17.578s
with timing via:
go build -a -debug-actiongraph=/tmp/actiongraph-cl-657179-ps1 -v github.com/microsoft/typescript-go/internal/checker
There are some additional adjustments to make here, including we can
consider a follow-on CL I have that parallelizes the operations of the
core loop, but this seems to be a nice win as is, and my understanding
is the desire is to merge this as it stands.
Updates #72815
Change-Id: I1753c5354b495b059f68fb97f3103ee7834f9eee
Reviewed-on: https://go-review.googlesource.com/c/go/+/657179 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
khr@golang.org [Tue, 20 May 2025 22:59:02 +0000 (15:59 -0700)]
reflect: turn off allocation test if instrumentation is on
Help fix the asan builders.
Change-Id: I980f5171519643c3543bdefc6ea46fd0fca17c28
Reviewed-on: https://go-review.googlesource.com/c/go/+/674616 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
khr@golang.org [Tue, 20 May 2025 22:56:29 +0000 (15:56 -0700)]
runtime: disable stack allocation test when instrumentation is on
Should fix some asan build failures.
Change-Id: Ic0a816b56a1a278aa0ad541aea962f9fea7b10fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/674696 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Filippo Valsorda [Wed, 8 Jan 2025 10:16:48 +0000 (11:16 +0100)]
crypto/ecdsa,crypto/ed25519: cache FIPS private keys
All private keys need to go through a slow PCT in FIPS-140 mode.
ECDH and RSA keys have places to hide a precomputed value without
causing races, but Ed25519 and ECDSA keys might be constructed by the
application and then used with concurrent Sign calls.
For these, implement an equivalent to crypto/internal/boring/bcache
using weak.Pointer and runtime.AddCleanup.
fips140: latest
goos: linux
goarch: amd64
pkg: crypto/ed25519
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
│ 1a93e4a2cf │ 78a819ea78 │
│ sec/op │ sec/op vs base │
Signing-16 72.72µ ± 0% 16.93µ ± 1% -76.72% (p=0.002 n=6)
fips140: off
goos: linux
goarch: amd64
pkg: crypto/ed25519
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
│ 310bad31e5 │ 310bad31e5-dirty │
│ sec/op │ sec/op vs base │
Signing-16 17.18µ ± 1% 16.95µ ± 1% -1.36% (p=0.002 n=6)
Damien Neil [Tue, 20 May 2025 18:16:23 +0000 (11:16 -0700)]
runtime: record synctest bubble ownership in hchan
Replace the hchan.synctest bool with an hchan.bubble reference
to the synctest bubble that created the chan. I originally used
a bool to avoid increasing the size of hchan, but we have space
in hchan's current size class for another pointer.
This lets us detect one bubble operating on a chan created
in a different bubble.
For #67434
Change-Id: If6cf9ffcb372fe7fb3f8f4ef27b664848578ba5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/674515
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Use a sync.OnceValue rather than a sync.WaitGroup to
coordinate access to encoderCache entries.
The OnceValue better expresses the intent of the code
(we want to initialize the cache entry only once).
However, the motivation for this change is to avoid
testing/synctest incorrectly reporting a deadlock
when multiple bubbles call Marshal at the same time.
Goroutines blocked on WaitGroup.Wait are "durably blocked",
causing confusion when a goroutine in one bubble Waits
for a goroutine in a different bubble. Goroutines blocked
on OnceValue are not durably blocked, avoiding the problem.
Fixes #73733
For #67434
Change-Id: I81cddda80af67cf5c280fd4327620bc37e7a6fe6
Reviewed-on: https://go-review.googlesource.com/c/go/+/673335
Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Matloob [Mon, 19 May 2025 19:31:37 +0000 (15:31 -0400)]
cmd/doc: use golang.org/x/pkgsite/cmd/internal/doc to start server
This change switches the pkgsite command invoked to start a pkgsite
server from golang.org/x/pkgsite/cmd/pkgsite to
golang.org/x/pkgsite/cmd/internal/doc. The doc command is a simplified
version of cmd/pkgsite that changes some options to improve the user
experience. For example, it limits logging informational log messages,
doesn't always expect to find modules (for example if we're outside of a
module getting documentation for the standard library), and it takes the
address of the page to open in the browser (which simplifies waiting for
the server to start listening).
Fixes #68106
Change-Id: I667a49d03823242fa1aff333ecb1c0f198e92412
Reviewed-on: https://go-review.googlesource.com/c/go/+/674158
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: David Chase <drchase@google.com>
Michael Matloob [Mon, 19 May 2025 19:31:37 +0000 (15:31 -0400)]
cmd/doc: use go list to determine import path if it's missing
cmd/doc uses go/build to get information about the packages it's
documenting. In some cases, go/build can return a build.Package that it
couldn't determine an import path for, in which case it sets the import
path to ".". This can happen for relative package paths in in a module:
for relative package paths we don't use the go command to get
information about the module and just open the source files directly
instead, and will be missing the import path. This is usually okay
because go doc doesn't need to print the import path of the package it's
documenting, but for go doc -http, we want to know the import path so we
can open the right page in the browser.
For #68106
Change-Id: Ifba92862ad01d8d63f531c2451f18db2b0d7a3e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/674556
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Michael Matloob <matloob@google.com>
Michael Matloob [Mon, 19 May 2025 19:31:37 +0000 (15:31 -0400)]
cmd/doc: show page for the requested object
This fixes a bug where we start pkgsite for every requested object,
rather than the one that we would have printed the documentation for.
To make things simple, we'll run the logic that prints the
documentation, but with an io.Discard writer. Then we can tell if the
documentation was found based on the return values of those functions.
For #68106
Change-Id: Ibf2ab1720f381d7214fc9239b9c2e915c91f7f7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/674555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
Mateusz Poliwczak [Tue, 20 May 2025 15:39:22 +0000 (15:39 +0000)]
reflect: add TypeAssert[T]
This implementation is zero-alloc when T is a concrete type,
allocates when val contains a method or when T is a interface
and Value was obtained for example through Elem(), in which case
it has to be allocated to avoid sharing the same memory.
Damien Neil [Fri, 9 May 2025 23:18:49 +0000 (16:18 -0700)]
runtime: disallow closing bubbled chans from outside bubble
A chan created within a synctest bubble may not be
operated on from outside the bubble.
We panicked on send and receive, but not close.
Panic on close as well.
For #67434
Change-Id: I98d39e0cf7baa1a679aca1fb325453d69c535308
Reviewed-on: https://go-review.googlesource.com/c/go/+/671960
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Damien Neil [Fri, 9 May 2025 18:07:14 +0000 (11:07 -0700)]
runtime: print blocking status of bubbled goroutines in stacks
For goroutines in a synctest bubble, include whether the goroutine
is "durably blocked" or not in the goroutine status.
Synctest categorizes goroutines in certain states as "durably"
blocked, where the goroutine is not merely idle but can only
be awoken by another goroutine in its bubble. To make it easier
for users to understand why a bubble is or is not idle,
print the state of each bubbled goroutine.
Michael Anthony Knyszek [Fri, 9 May 2025 19:45:25 +0000 (19:45 +0000)]
doc: add release notes for cleanups and the unique package
For #71661.
Change-Id: I74870de3f17a938bc9fd83ccc41e13e64b55b5e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/671438 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
Michael Anthony Knyszek [Wed, 2 Apr 2025 01:46:45 +0000 (01:46 +0000)]
runtime: add package doc for checkfinalizer mode
Fixes #72949.
Change-Id: I114eda73c57bc7d596eb1656e738b80c1cbe5254
Reviewed-on: https://go-review.googlesource.com/c/go/+/662039 Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Fri, 9 May 2025 19:33:22 +0000 (19:33 +0000)]
runtime: report finalizer and cleanup queue length with checkfinalizer>0
This change adds tracking for approximate finalizer and cleanup queue
lengths. These lengths are reported once every GC cycle as a single line
printed to stderr when GODEBUG=checkfinalizer>0.
This change lays the groundwork for runtime/metrics metrics to produce
the same values.
For #72948.
For #72950.
Change-Id: I081721238a0fc4c7e5bee2dbaba6cfb4120d1a33
Reviewed-on: https://go-review.googlesource.com/c/go/+/671437 Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Jake Bailey [Thu, 15 May 2025 05:36:57 +0000 (22:36 -0700)]
unique: add alloc test for Make
This will be useful to show how the next CLs improve things.
Change-Id: I49a691295c1fe3c7455a67c7d19e5c03979f714a
Reviewed-on: https://go-review.googlesource.com/c/go/+/673015 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Wed, 2 Apr 2025 01:37:30 +0000 (01:37 +0000)]
runtime: add scan trace for checkfinalizers>1
This change dumps a scan trace (each pointer marked and where it came
from) for the partial GC cycle performed by checkfinalizers mode when
checkfinalizers>1. This is useful for quickly understanding why certain
values are reachable without having to pull out tools like viewcore.
For #72949.
Change-Id: Ic583f80e9558cdfe1c667d27a1d975008dd39a9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/662038 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Fri, 9 May 2025 18:53:06 +0000 (18:53 +0000)]
runtime: mark and identify tiny blocks in checkfinalizers mode
This change adds support for identifying cleanups and finalizers
attached to tiny blocks to checkfinalizers mode. It also notes a subtle
pitfall, which is that the cleanup arg, if tiny-allocated, could end up
co-located with the object with the cleanup attached! Oops...
For #72949.
Change-Id: Icbe0112f7dcfc63f35c66cf713216796a70121ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/662037
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
Michael Anthony Knyszek [Tue, 1 Apr 2025 19:38:39 +0000 (19:38 +0000)]
runtime: annotate checkfinalizers reports with source and type info
This change adds a new special kind called CheckFinalizer which is used
to annotate finalizers and cleanups with extra information about where
that cleanup or finalizer came from.
For #72949.
Change-Id: I3c1ace7bd580293961b7f0ea30345a6ce956d340
Reviewed-on: https://go-review.googlesource.com/c/go/+/662135 Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Change-Id: I15a349bfbaf7333ec3e4a62ae4d06f3f371dfb1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/673715 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Shibi J M [Tue, 20 May 2025 03:59:15 +0000 (03:59 +0000)]
net: avoid using Windows' TransmitFile on non-server machines
Windows API's TransmitFile function is limited to two concurrent
operations on workstation and client versions of Windows. This change
modifies the net.sendFile function to perform no work in such cases
so that TransmitFile is avoided.
Fixes #73746
Change-Id: Iba70d5d2758bf986e80c78254c8e9e10b39bb368
GitHub-Last-Rev: 315ddc0cd8034f52632dc31baf35057a8bad9bcd
GitHub-Pull-Request: golang/go#73758
Reviewed-on: https://go-review.googlesource.com/c/go/+/673855 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Damien Neil <dneil@google.com>
Michael Matloob [Fri, 16 May 2025 20:40:55 +0000 (16:40 -0400)]
cmd/go: do not try to load 'all' packages with invalid import paths
Before this change, when we tried to compute the set of packages in
'all', we'd add packages with invalid import paths to the set and try to
load them, which would fail. Instead, do not add them to the list of
packages to load in the second iteration of the loader. We'll still
return errors for invalid imports in the importing packages.
Change-Id: I682229011f555ed1d0c827f79100c1c43bf7f93a
Reviewed-on: https://go-review.googlesource.com/c/go/+/673655 Reviewed-by: Michael Matloob <matloob@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Change-Id: Ie4a79a3c410eb79155ce2418ae64fa670d1ccd53
Reviewed-on: https://go-review.googlesource.com/c/go/+/673477 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com>
Michael Anthony Knyszek [Mon, 9 Dec 2024 19:07:40 +0000 (19:07 +0000)]
runtime: add new GODEBUG checkfinalizer
This new debug mode detects cleanup/finalizer leaks using checkmark
mode. It runs a partial GC using only specials as roots. If the GC can
find a path from one of these roots back to the object the special is
attached to, then the object might never be reclaimed. (The cycle could
be broken in the future, but it's almost certainly a bug.)
This debug mode is very barebones. It contains no type information and
no stack location for where the finalizer or cleanup was created.
For #72949.
Change-Id: Ibffd64c1380b51f281950e4cfe61f677385d42a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/634599 Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Jake Bailey [Wed, 14 May 2025 05:01:43 +0000 (22:01 -0700)]
cmd/internal/testdir: filter out errors outside input file set
When an errorcheck test uses -m and instantiates an imported generic
function, the errors will include -m messages from the imported package
(since the new function has not previously been walked). These errors
cannot be matched since we can't write errors in files outside the test
input.
To fix this (and enable the other CLs in this stack), drop any unmatched
errors that occur in files outside those in the input set.
Change-Id: I2fcf0dd4693125d2e5823ea4437011730d8b1b1f
Reviewed-on: https://go-review.googlesource.com/c/go/+/672515 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
khr@golang.org [Tue, 20 May 2025 15:42:24 +0000 (08:42 -0700)]
cmd/dist: pass GO_GCFLAGS to cpuN runtime tests
We want gcflags, which control builder type (e.g. noopt) to be used
for these tests also.
Should fix noopt and maybe other builders.
Change-Id: Iad34beab51714f0c38989ec0fc8778cf79087f72
Reviewed-on: https://go-review.googlesource.com/c/go/+/674455 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
thepudds [Wed, 12 Feb 2025 23:39:08 +0000 (18:39 -0500)]
cmd/compile/internal/escape: additional constant and zero value tests and logging
This adds additional logging for the work that walk does to reduce
how often an interface conversion results in an allocation.
Also, as part of #71359, we will be updating how escape analysis and
walk handle basic literals, composite literals, and zero values,
so add some tests that uses this new logging.
By the end of our CL stack, we address all of these tests.
Updates #71359
Change-Id: I43fde8343d9aacaec1e05360417908014a86c8bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/649076 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
net/http: upon http redirect, copy Request.GetBody in new request
This enable http.RoundTripper implementation to retry POST request (let's
say after a 500) after a 307/308 redirect.
Fixes #73439
Change-Id: I4365ff58b012c7f0d60e0317a08c98b1d48f657e
Reviewed-on: https://go-review.googlesource.com/c/go/+/666735 Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Damien Neil <dneil@google.com>
Michael Anthony Knyszek [Fri, 21 Mar 2025 16:26:15 +0000 (16:26 +0000)]
runtime: prevent unnecessary zeroing of large objects with pointers
CL 614257 refactored mallocgc but lost an optimization: if a span for a
large object is already backed by memory fresh from the OS (and thus
zeroed), we don't need to zero it. CL 614257 unconditionally zeroed
spans for large objects that contain pointers.
This change restores the optimization from before CL 614257, which seems
to matter in some real-world programs.
While we're here, let's also fix a hole with the garbage collector being
able to observe uninitialized memory of the large object is observed
by the conservative scanner before being published. The gory details are
in a comment in heapSetTypeLarge. In short, this change makes
span.largeType an atomic variable, such that the GC can only observe
initialized memory if span.largeType != nil.
Fixes #72991.
Change-Id: I2048aeb220ab363d252ffda7d980b8788e9674dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/659956 Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Michael Anthony Knyszek [Tue, 13 May 2025 20:09:57 +0000 (20:09 +0000)]
runtime: only update freeIndexForScan outside of the mark phase
Currently, it's possible for asynchronous preemption to observe a
partially initialized object. The sequence of events goes like this:
- The GC is in the mark phase.
- Thread T1 is allocating object O1.
- Thread T1 zeroes the allocation, runs the publication barrier, and
updates freeIndexForScan. It has not yet updated the mark bit on O1.
- Thread T2 is conservatively scanning some stack frame.
That stack frame has a dead pointer with the same address as O1.
- T2 picks up the pointer, checks isFree (which checks
freeIndexForScan without an import barrier), and sees that O1 is
allocated. It marks and queues O1.
- T2 then goes to scan O1, and observes uninitialized memory.
Although a publication barrier was executed, T2 did not have an import
barrier. T2 may thus observe T1's writes to zero the object out-of-order
with the write to freeIndexForScan.
Normally this would be impossible if T2 got a pointer to O1 from
somewhere written by T1. The publication barrier guarantees that if the
read side is data-dependent on the write side then we'd necessarily
observe all writes to O1 before T1 published it. However, T2 got the
pointer 'out of thin air' by scanning a stack frame with a dead pointer
on it.
One fix to this problem would be to add the import barrier in the
conservative scanner. We would then also need to put freeIndexForScan
behind the publication barrier, or make the write to freeIndexForScan
exactly that barrier.
However, there's a simpler way. We don't actually care if conservative
scanning observes a stale freeIndexForScan during the mark phase.
Newly-allocated memory is always marked at the point of allocation (the
allocate-black policy part of the GC's design). So it doesn't actually
matter that if the garbage collector scans that memory or not.
This change modifies the allocator to only update freeIndexForScan
outside the mark phase. This means freeIndexForScan is essentially
a snapshot of freeindex at the point the mark phase started. Because
there's no more race between conservative scanning and newly-allocated
objects, the complicated scenario above is no longer a possibility.
One thing we do have to be careful of is other callers of isFree.
Previously freeIndexForScan would always track freeindex, now it no
longer does. This change thus introduces isFreeOrNewlyAllocated which is
used by the conservative scanner, and uses freeIndexForScan. Meanwhile
isFree goes back to using freeindex like it used to. This change also
documents the requirement on isFree that the caller must have obtained
the pointer not 'out of thin air' but after the object was published.
isFree is not currently used anywhere particularly sensitive (heap dump
and checkmark mode, where the world is stopped in both cases) so using
freeindex is both conceptually simple and also safe.
Change-Id: If66b8c536b775971203fb4358c17d711c2944723
Reviewed-on: https://go-review.googlesource.com/c/go/+/672340 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Sam Thanawalla [Mon, 19 May 2025 16:49:57 +0000 (16:49 +0000)]
cmd/go: extend the ignore directive for indexed modules
For modules that have already been indexed, we can skip ignored paths.
We already skip 'testdata' and '_' for this case so we can extend the
ignore directive for this case as well.
Updates: #42965
Change-Id: I076a242ba65c7b905b9dc65dcfb0a0247cbd68d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/674076 Reviewed-by: Michael Matloob <matloob@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
David Finkel [Wed, 14 May 2025 00:54:15 +0000 (20:54 -0400)]
cmd/go: add 2 scripts test for git sha256 fetching
Fast follow to golang.org/cl/636475 with a couple script tests that
build/runs a module that depends on a function inside a git repo using
sha256 hashes. (one with go get of a branch-name and the other
configuring go.mod directly)
Change-Id: Ief6c7efaf6d5c066dc54a3e4a63aad109f625abe
Reviewed-on: https://go-review.googlesource.com/c/go/+/672435 Reviewed-by: Michael Matloob <matloob@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
cmd/go/internal/modload: remove likely vestigial ability to infer module path from Godeps.json and vendor.json
CL 518776 deleted the cmd/go/internal/modconv package and dropped the
ability to import dependency requirements from ~nine or so legacy
pre-module dependency configuration files. Part of the rationale from
Russ in 2023 for dropping that support was that "by now no one is
running into those configs anymore during 'go mod init'".
For two of those legacy file formats, Godeps.json and vendor.json, the
ability to import their listed dependencies was dropped in CL 518776,
but what remained for those two formats was the ability to guess the
resulting module name in the absence of a name being supplied to 'go mod
init'.
This could be explained by the fact that this smaller functionality for
guessing a module name was separate, did not rely on the deleted modconv
package, and instead only relied on simple JSON parsing.
The name guessing was helpful as part of the transition when module
support was initially released, but it was never perfect, including the
various third-party dependency managers did not all have the same naming
rules that were enforced by modules.
In short, it is very unlikely anyone is relying on this now, so we
delete it.
This CL was spawned from discussion in two related documentation CLs
(CL 662675 and CL 662695).
Updates #71537
Change-Id: I9e087aa296580239562a0ecee58913c5edc533ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/664315 Reviewed-by: Michael Matloob <matloob@google.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
Mateusz Poliwczak [Tue, 20 May 2025 07:46:53 +0000 (09:46 +0200)]
testing: use a pattern to match the elapsed time in TestTRun
Fixes #73723
Fixes #73737
Fixes #73739
Change-Id: I1ebd3614614285c3e660d48241389bb0f896be23
Reviewed-on: https://go-review.googlesource.com/c/go/+/674355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Alan Donovan <adonovan@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Keith Randall [Thu, 17 Apr 2025 05:48:06 +0000 (22:48 -0700)]
cmd/compile: align stack-allocated backing stores higher than required
Because that's what mallocgc did and some user code came to rely on it.
Fixes #73199
Change-Id: I45ca00d2ea448e6729ef9ac4cec3c1eb0ceccc89
Reviewed-on: https://go-review.googlesource.com/c/go/+/666116 Reviewed-by: t hepudds <thepudds1460@gmail.com> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
khr@golang.org [Wed, 14 May 2025 00:53:45 +0000 (17:53 -0700)]
cmd/compile: don't preload registers if destination already scheduled
In regalloc, we allocate some values to registers before loop entry,
so that they don't need to be loaded (from spill locations) during
the loop.
But it is pointless if we've already regalloc'd the loop body.
Whatever restores we needed for the body are already generated.
It's not clear if this code is ever useful. No tests fail if I just
remove it. But at least this change is worthwhile. It doesn't help,
and it actively inserts more restores than we really need (mostly
because the desired register list is approximate - I have seen cases
where the loads implicated here end up being dead because the restores
hit the wrong registers and the edge shuffle pass knows it needs
the restores in different registers).
While we are here, might as well have layoutRegallocOrder return
the standard layout order instead of recomputing it.
Change-Id: Ia624d5121de59b6123492603695de50b272b277f
Reviewed-on: https://go-review.googlesource.com/c/go/+/672735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com>