Ian Lance Taylor [Wed, 16 Nov 2022 22:19:47 +0000 (14:19 -0800)]
net: change resolverConfig.dnsConfig to an atomic.Pointer
We were using a RWMutex RLock around a single memory load,
which is not a good use of a RWMutex--it introduces extra work
for the RLock but contention around a single memory load is unlikely.
And, the tryUpdate method was not acquiring the mutex anyhow.
The new atomic.Pointer type is type-safe and easy to use correctly
for a simple use-case like this.
Change-Id: Ib3859c03414c44d2e897f6d15c92c8e4b5c81a11
Reviewed-on: https://go-review.googlesource.com/c/go/+/451416
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Damien Neil [Thu, 22 Sep 2022 23:22:04 +0000 (16:22 -0700)]
archive/tar, archive/zip: return ErrInsecurePath for unsafe paths
Return a distinguishable error when reading an archive file
with a path that is:
- absolute
- escapes the current directory (../a)
- on Windows, a reserved name such as NUL
Users may ignore this error and proceed if they do not need name
sanitization or intend to perform it themselves.
Fixes #25849
Fixes #55356
Change-Id: Ieefa163f00384bc285ab329ea21a6561d39d8096
Reviewed-on: https://go-review.googlesource.com/c/go/+/449937 Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
Damien Neil [Thu, 10 Nov 2022 01:49:44 +0000 (17:49 -0800)]
path/filepath: add IsLocal
IsLocal reports whether a path lexically refers to a location
contained within the directory in which it is evaluated.
It identifies paths that are absolute, escape a directory
with ".." elements, and (on Windows) paths that reference
reserved device names.
For #56219.
Change-Id: I35edfa3ce77b40b8e66f1fc8e0ff73cfd06f2313
Reviewed-on: https://go-review.googlesource.com/c/go/+/449239
Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joedian Reid <joedian@golang.org>
Cherry Mui [Mon, 3 Oct 2022 23:07:39 +0000 (19:07 -0400)]
cmd/go: add PGO auto mode
Add "auto" mode for the -pgo build flag. When -pgo=auto is
specified, if there is a default.pgo file in the directory of the
main package, it will be selected and used for the build.
Currently it requires exactly one main package when -pgo=auto is
specified. (We'll support multiple main packages in the future.)
Also apply to other build-related subcommands, "go install", "go
run", "go test", and "go list".
Bryan C. Mills [Tue, 15 Nov 2022 17:34:31 +0000 (12:34 -0500)]
cmd/fix: disallow cgo errors in tests
The 'cgo' command invoked by 'go fix' was not valid when built with
-trimpath, but the test was not failing because errors from the
command were being logged and ignored instead of causing tests to
fail. Changing the code and test not to ignore the errors revealed
that a number of existing tests were always, unconditionally
triggering cgo errors which were then ignored.
This change updates those tests to no longer produce cgo errors,
and to check their results when cgo is enabled.
Russ Cox [Wed, 16 Nov 2022 03:30:45 +0000 (22:30 -0500)]
cmd/go: automatically disable cgo on systems with no C compiler
The documentation for cgo has always said:
> The cgo tool is enabled by default for native builds
> on systems where it is expected to work.
Following the spirit of that rule, this CL disables cgo by default
on systems where $CC is unset and the default C compiler
(clang or gcc) is not found in $PATH.
This CL makes builds of Go code on systems with no C compiler
installed automatically fall back to non-cgo mode.
For example, if building a Go program using package net
in a stripped down Linux container, that build will now run
with cgo disabled, instead of attempting the build with cgo enabled
and only succeeding if the right pre-compiled .a files happen to
be loaded into the container.
This CL makes it safe to drop the pre-compiled .a files
from the Go distribution. Systems that don't have a C compiler
will simply disable cgo when building new .a files for that system.
In general keeping the pre-compiled .a files working in cgo mode
on systems without C compilers has had only mixed success due
to the precise build cache. Today we have had to disable various
checks in the precise build cache so that distributed .a files look
up-to-date even if the current machine's C compiler is a different
version than the one used when packaging the distribution.
Each time we improve precision we have a decent chance of
re-invalidating the files. This CL, combined with dropping the .a files
entirely, will let us re-enable those checks and ensure that the
.a files used in a build actually match the C compiler being used.
On macOS, the distributed .a files for cgo-dependent packages
have been stale (not actually used by the go command) since the
release of Go 1.14 in February 2020, due to CL 216304 setting
a CGO_CFLAGS environment variable that won't match the default
setting on users machines. (To keep the distributed .a files working,
that CL should have instead changed the default in the go command.)
The effect is that for the past six Go releases (!!!), the go command
has been unable to build basic programs like src/net/http/triv.go
on macOS without either disabling cgo or installing Xcode's C compiler.
This CL fixes that problem by disabling cgo when there's no C compiler.
Now it will once again be possible to build basic programs with just
a Go toolchain installed.
In the past, disabling cgo on macOS would have resulted in subpar
implementations of crypto/x509, net, and os/user, but as of CL 449316
those packages have all been updated to use libc calls directly,
so they now provide the same implementation whether or not cgo is enabled.
In the past, disabling cgo on macOS would also have made the
race detector unusable, but CL 451055 makes the race detector
work on macOS even when cgo is disabled.
On Windows, none of the standard library uses cgo today, so all
the necessary .a files can be rebuilt without a C toolchain,
and there is no loss of functionality in the standard library when
cgo is disabled. After this CL, the race detector won't work on
Windows without a C toolchain installed, but that turns out to be
true already: when linking race-enabled programs, even if the Go linker
does not invoke the host linker, it still attempts to read some of the
host C toolchain's .a files to resolve undefined references.
On Unix systems, disabling cgo when a C compiler is not present
will mean that builds get the pure Go net resolver, which is used
by default even in cgo builds when /etc/resolv.conf is simple enough.
It will also mean they get the pure os/user code, which reads
/etc/passwd and /etc/group instead of using shared libraries,
and therefore it may miss out on other sources of user information
such as LDAP. The race detector also will not work without a C compiler.
This would be dire except that nearly all Unix systems have a C compiler
installed by default, and on those that don't it is trivial to add one.
In particular, the vast majority of Go developers running on Linux
and other Unix systems will already have a C compiler and will be
unaffected.
Russ Cox [Wed, 16 Nov 2022 01:36:10 +0000 (20:36 -0500)]
runtime/race: do not use cgo on macOS
The use of an empty import "C" to trigger cgo in runtime/race
serves two purposes:
1. Cause the runtime to use the C library to create system threads,
because the race syso implementation expects things like
thread-local storage to work correctly.
2. Derive the right set of //go:cgo_import_dynamic comments
to pass to the Go linker, so that it doesn't diagnose them as
undefined references.
On macOS, (1) is unnecessary because using the C library
(via DLL calls) is the only way the runtime ever creates threads.
We can accomplish (2) by writing those comments ourselves.
Having done that in this CL, cgo is no longer needed to run
the race detector on macOS, which means that having a
pre-compiled set of .a files is no longer necessary,
nor is having Xcode for use with cgo when rebuilding those .a files.
Austin Clements [Sat, 12 Nov 2022 02:57:57 +0000 (21:57 -0500)]
cmd/dist: simplify old code
Now that all uses of "go test" have been converted over to the new
abstraction, we can delete the old helpers for building "go test"
commands and simplify some code that's only used by the new
abstraction now.
For #37486.
Change-Id: I770cd457e018160d694abcc0b6ac80f7dc2e8425
Reviewed-on: https://go-review.googlesource.com/c/go/+/450020 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Austin Clements [Fri, 4 Nov 2022 18:54:04 +0000 (14:54 -0400)]
cmd/dist: convert most remaining tests to use goTest
This converts most of the remaining manual "go test" command line
construction in cmd/dist to use the goTest abstraction and
registerTest.
At this point, the only remaining place that directly constructs go
test command lines is runHostTest.
This fixes a bug in the "nolibgcc:os/user" test. It was clearly
supposed to pass "-run=^Test[^CS]", but the logic to override the
"-run" flag for "nolibgcc:net" caused "nolibgcc:os/user" to pass
*both* "-run=^Test[^CS]" and "-run=". This was then rewritten into
just "-run=" by flattenCmdline, which caused all os/user tests to run,
and not actually skip the expensive tests as intended. (This is a
great example of why the new abstraction is much more robust than
command line construction.)
I traced all exec calls from cmd/dist on linux/amd64 and, other than
the fix to nolibgcc:os/user, this makes only no-op changes (such as
re-arranging the order of flags).
For #37486.
Change-Id: Ie8546bacc56640ea39f2804a87795c14a3fe4c7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/450018
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Austin Clements [Fri, 4 Nov 2022 18:49:37 +0000 (14:49 -0400)]
cmd/dist: restructure cgo_test
Currently, dist test has a single test called "cgo_test" that runs a
large number of different "go test"s.
This commit restructures cgo_test into several individual tests, each
of which runs a single "go test" that can be described by a goTest
object and registered with registerTest. Since this lets us raise the
abstraction level of constructing these tests and these tests are
mostly covering the Cartesian product of a small number of orthogonal
dimensions, we pull the common logic for constructing these tests into
a helper function.
For consistency, we now pass -tags=static to the static testtls and
nocgo tests, but this tag doesn't affect the build of these tests at
all. I traced all exec calls from cmd/dist on linux/amd64 and this is
the only non-trivial change.
For #37486.
Change-Id: I53c1efa1c38d785dc71968f05e8d7d636b553e96
Reviewed-on: https://go-review.googlesource.com/c/go/+/450017
Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Austin Clements [Thu, 13 Oct 2022 14:57:13 +0000 (10:57 -0400)]
cmd/dist: make registerTest take a goTest
The overall goal is to make registerTest the primary entry point for
adding dist tests and to convert nearly all dist tests to be
represented by a goTest, registered via registerTest. This will
centralize the logic for creating dist tests corresponding to go tool
tests.
I traced all exec calls from cmd/dist on linux/amd64 and this makes
only no-op changes (such as re-arranging the order of flags).
For #37486.
Change-Id: I4749e6f3666134d3259b54ee6055d76a4235c60c
Reviewed-on: https://go-review.googlesource.com/c/go/+/450016 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Austin Clements [Wed, 2 Nov 2022 18:33:04 +0000 (14:33 -0400)]
cmd/dist: use goTest for manual go test invocations
This CL rewrites everywhere in dist that manually constructs an
exec.Cmd to run "go test" to use the goTest abstraction. All remaining
invocations of "go test" after this CL construct the command line
manually, but ultimately use addCmd to execute it.
I traced all exec calls from cmd/dist on linux/amd64 and this makes
only no-op changes (such as re-arranging the order of flags).
For #37486.
Change-Id: Idc7497e39bac04def7ddaf2010881c9623e76fd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/450015 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Austin Clements [Wed, 2 Nov 2022 18:27:28 +0000 (14:27 -0400)]
cmd/dist: introduce "go test" abstraction
This introduces an abstraction for constructing and running "go test"
commands. Currently, dist test is basically a shell script written in
Go syntax: it mostly just invokes lots of subprocesses, almost all of
which are "go test" invocations, and it constructs those command lines
directly from strings all over the place.
This CL raises the level of abstraction of invoking go test. The
current level of abstraction is not serving us very well: it's
conveniently terse, but the actual logic for constructing a command
line is typically so spread out that it's difficult to predict what
command will actually run. For example, the `gotest` function
constructs the basic command, but many tests want to override at least
some of these flags, so flattenCmdLine has logic specific to `go test`
for eliminating duplicate flags that `go test` itself would reject. At
the same time, the logic for constructing many common flags is
conditional, leading to a bevy of helpers for constructing flags like
`-short` and `-timeout` and `-run` that are scattered throughout
test.go and very easy to forget to call.
This CL centralizes and flattens all of this knowledge into a new
`goTest` type. This type gives dist a single, unified point where we
can change anything about how it invokes "go test".
There's currently some "unnecessary" abstraction in the implementation
of the goTest type to separate "build" and "run" flags. This will
become important later when we convert host tests and to do separate
build and run steps.
The following CLs will convert dist test to use this type rather than
directly constructing "go test" command lines. Finally, we'll strip
out the scattered helper logic for building command lines.
For #37486.
Change-Id: I9f1633fe6c0921696419ce8127ed2ca7b7a4e01b
Reviewed-on: https://go-review.googlesource.com/c/go/+/448802 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Russ Cox [Wed, 16 Nov 2022 17:04:32 +0000 (12:04 -0500)]
math/rand: deprecate Seed
Programs that call Seed and then expect a specific sequence
of results from the global random source (using functions such as Int)
can be broken when a dependency changes how much it consumes
from the global random source. To avoid such breakages, programs
that need a specific result sequence should use NewRand(NewSource(seed))
to obtain a random generator that other packages cannot access.
Fixes #56319.
Change-Id: Idac33991b719d2c71f109f51dacb3467a649e01e
Reviewed-on: https://go-review.googlesource.com/c/go/+/451375 Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Austin Clements [Fri, 11 Nov 2022 16:51:28 +0000 (11:51 -0500)]
cmd/go: make testterminal18153 a normal test
Currently, cmd/go's testterminal18153 is implemented as a special test
that doesn't run as part of cmd/go's regular tests. Because the test
requires stdout and stderr to be a terminal, it is currently run
directly by "dist test" so it can inherit the terminal of all.bash.
This has a few problems. It's yet another special case in dist test.
dist test also has to be careful to not apply its own buffering to
this test, so it can't run in parallel and it limits dist test's own
scheduler. It doesn't run as part of regular "go test", which means it
usually only gets coverage from running all.bash. And since we have to
skip it if all.bash wasn't run at a terminal, I'm sure it often gets
skipped even when running all.bash.
Fix all of this by rewriting this test to create its own PTY and
re-exec "go test" to check that PTY passes through go test. This makes
the test self-contained, so it can be a regular cmd/go test, we can
drop it from dist test, and it's not sensitive to the environment of
all.bash.
The registerTest function has a special case for commands that start
with "time", but we don't use this case anywhere. Delete this special
case and its support code.
Preparation for #37486.
Change-Id: Ica180417e7aa4e4fc260cb97467942bae972fdb6
Reviewed-on: https://go-review.googlesource.com/c/go/+/448801
Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
The cmd/dist cgo_test enumerates a large number of platforms in
various special cases. Some combinations are suspiciously absent. This
CL completes the combinations.
I've confirmed using trybots that the newly-enabled tests pass on
android/* (this is not surprising because the gohostos is never
"android" anyway), windows/arm64, linux/ppc64 (no cgo), linux/loong64
(except for one test, filed #56623), linux/mips*, netbsd/arm (except
for one test, filed #56629), and netbsd/arm64. The windows/arm builder
is out to lunch, so I'm assuming that works. Since netbsd/arm and
arm64 mostly passed these tests, I've also enabled them on netbsd/386
and netbsd/amd64, where they seem to work fine as well.
Filippo Valsorda [Tue, 15 Nov 2022 17:43:56 +0000 (18:43 +0100)]
crypto/x509: add support for PKCS8/PKIX X25519 key encodings
This specifically doesn't add support for X25519 certificates.
Refactored parsePublicKey not to depend on the public PublicKeyAlgorithm
values, and ParseCertificate/ParseCertificateRequest to ignore keys that
don't have a PublicKeyAlgorithm even if parsePublicKey supports them.
Russ Cox [Wed, 16 Nov 2022 05:02:14 +0000 (00:02 -0500)]
test: fix noinit on noopt builder
Fix noopt build break from CL 450136 by not running test.
I can't reproduce the failure locally, but it's entirely reasonable
for this test to fail when optimizations are disabled, so just don't
run it when optimizations are disabled.
Russ Cox [Sun, 13 Nov 2022 14:22:35 +0000 (09:22 -0500)]
cmd/compile: handle simple inlined calls in staticinit
Global variable initializers like
var myErr error = &myError{"msg"}
have been converted to statically initialized data
from the earliest days of Go: there is no init-time
execution or allocation for that line of code.
But if the expression is moved into an inlinable function,
the static initialization no longer happens.
That is, this code has always executed and allocated
at init time, even after we added inlining to the compiler,
which should in theory make this code equivalent to
the original:
This CL makes the static initialization rewriter understand
inlined functions consisting of a single return statement,
like in this example, so that myErr2 can be implemented as
statically initialized data too, just like myErr, with no init-time
execution or allocation.
A real example of code that benefits from this rewrite is
all globally declared errors created with errors.New, like
package io
var EOF = errors.New("EOF")
Package io no longer has to allocate and initialize EOF each
time a program starts.
Another example of code that benefits is any globally declared
godebug setting (using the API from CL 449504), like
package http
var http2server = godebug.New("http2server")
These are no longer allocated and initialized at program startup either.
The list of functions that are inlined into static initializers when
compiling std and cmd (along with how many times each occurs) is:
For the cmd/go binary, this CL cuts the number of init-time
allocations from about 1920 to about 1620 (a 15% reduction).
The total executable code footprint of init functions is reduced
by 24kB, from 137kB to 113kB (an 18% reduction).
The overall binary size is reduced by 45kB,
from 15.335MB to 15.290MB (a 0.3% reduction).
(The binary size savings is larger than the executable code savings
because every byte of executable code also requires corresponding
runtime tables for unwinding, source-line mapping, and so on.)
Also merge test/sinit_run.go, which had stopped testing anything
at all as of CL 161337 (Feb 2019) and initempty.go into a new test
noinit.go.
Fixes #30820.
Change-Id: I52f7275b1ac2a0a32e22c29f9095071c7b1fac20
Reviewed-on: https://go-review.googlesource.com/c/go/+/450136 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Russ Cox [Sat, 12 Nov 2022 17:27:33 +0000 (12:27 -0500)]
cmd/compile: do not emit a few more basic types from every compilation
We already emit types for any and func(error) string in runtime.a
but unlike the other pre-emitted types, we don't then exclude them
from being emitted in other packages. Fix that.
Also add slices of non-func types that we already emit.
Saves 0.3% of .a files in std cmd deps, computed by adding sizes from:
ls -l $(go list -export -f '{{.Export}}' -deps std cmd
The effect is small and not worth doing on its own.
The real improvement is making “what to write always in runtime”
and “what not to write in other packages” more obviously aligned.
Change-Id: Ie5cb5fd7e5a3025d2776d9b4cece775fdf92d3b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/450135 Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@google.com>
Keith Randall [Fri, 11 Nov 2022 23:30:10 +0000 (15:30 -0800)]
cmd/compile: be more careful about pointer incrementing in range loops
For range loops, we use a pointer to the backing store that gets
incremented on each iteration of the loop.
The problem with this scheme is that at the end of the last iteration,
we may briefly have a pointer that points past the end of the backing store
of the slice that is being iterated over. We cannot let the garbage collector
see that pointer.
To fix this problem, have the incremented pointer live briefly as
a uintptr instead of a normal pointer, so it doesn't keep anything
alive. Convert back to a normal pointer just after the loop condition
is checked, but before anything that requires a real pointer representation
(in practice, any call, which is what could cause a GC scan or stack copy).
Fixes #56699
Change-Id: Ia928d23f85a211565357603668bea4e5c534f989
Reviewed-on: https://go-review.googlesource.com/c/go/+/449995 Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Ian Lance Taylor [Tue, 8 Nov 2022 23:08:48 +0000 (15:08 -0800)]
net: rewrite nsswitch.conf parsing to work like other parsers
Seems simpler than having two different parsing mechanisms.
Change-Id: I4f8468bc025f8e03f59ec9c79b17721581b64eed
Reviewed-on: https://go-review.googlesource.com/c/go/+/448855 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
David Chase [Wed, 2 Nov 2022 21:23:47 +0000 (17:23 -0400)]
cmd/compile: add testing-flag guard to package-is-collected assert
On advice of the department of garbage collection, forcing a garbage
collection generally does not improve performance. However,
this-data-is-now-unreachable is a good property to be able to test,
and that requires finalizers and a forced GC. So, to save build time,
this test was removed from the compiler itself, but to verify the
property, it was added to the fma_test (and the end-to-end dependence
on the flag was tested with an inserted failure in testing the
test).
TODO: also turn on the new -d=gccheck=1 debug flag on the ssacheck
builder.
Benchmarking reveals that it is profitable to avoid this GC,
with about 1.5% reduction in both user and wall time.
Russ Cox [Tue, 15 Nov 2022 14:54:39 +0000 (09:54 -0500)]
sync/atomic: allow linked list of atomic pointers again
For #56603, CL 448275 added a _ [0]T field to atomic.Pointer,
so that different kinds of atomic.Pointer are not convertible.
Unfortunately, that breaks code like:
type List struct {
Next atomic.Pointer[List]
}
which should be valid, just as using Next *List is valid.
Instead, we get:
./atomic_test.go:2533:6: invalid recursive type List
./atomic_test.go:2533:6: List refers to
./atomic_test.go:2534:13: "sync/atomic".Pointer refers to
./atomic_test.go:2533:6: List
Cuong Manh Le [Mon, 14 Nov 2022 16:11:30 +0000 (23:11 +0700)]
cmd/compile: fix missing typecheck for static initialization slice
CL 440455 fixed missing walk pass for static initialization slice.
However, slicelit may produce un-typechecked node, thus we need to do
typecheck for sinit before calling walkStmtList.
Fixes #56727
Change-Id: I40730cebcd09f2be4389d71c5a90eb9a060e4ab7
Reviewed-on: https://go-review.googlesource.com/c/go/+/450215 Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Filippo Valsorda [Sat, 15 Oct 2022 10:06:48 +0000 (12:06 +0200)]
crypto/rsa: add a test walking through every key size
We already had some tests for special cases such as PSS with 513 bit
keys. The upcoming backend rewrite also happened to crash at 63 and 504
bits for different reasons. Might as well be systematic about it.
Also, make sure SignPSS returns ErrMessageTooLong like SignPKCS1v15 when
the key is too small, instead of panicking or returning an unnamed error.
Nick Ripley [Wed, 12 Oct 2022 14:05:51 +0000 (10:05 -0400)]
runtime/pprof: scale mutex profile samples when they are recorded
Samples in the mutex profile have their count and duration scaled
according to the probability they were sampled. This is done when the
profile is actually requested. The adjustment is done using to the
current configured sampling rate. However, if the sample rate is changed
after a specific sample is recorded, then the sample will be scaled
incorrectly. In particular, if the sampling rate is changed to 0, all of
the samples in the encoded profile will have 0 count and duration. This
means the profile will be "empty", even if it should have had samples.
This CL scales the samples in the profile when they are recorded, rather
than when the profile is requested. This matches what is currently done
for the block profile.
With this change, neither the block profile nor mutex profile are scaled
when they are encoded, so the logic for scaling the samples can be
removed.
Change-Id: If228cf39284385aa8fb9a2d62492d839e02f027f
Reviewed-on: https://go-review.googlesource.com/c/go/+/443056
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Cherry Mui [Wed, 9 Nov 2022 15:55:54 +0000 (10:55 -0500)]
runtime: make GC see object as allocated after it is initialized
When the GC is scanning some memory (possibly conservatively),
finding a pointer, while concurrently another goroutine is
allocating an object at the same address as the found pointer, the
GC may see the pointer before the object and/or the heap bits are
initialized. This may cause the GC to see bad pointers and
possibly crash.
To prevent this, we make it that the scanner can only see the
object as allocated after the object and the heap bits are
initialized. Currently the allocator uses freeindex to find the
next available slot, and that code is coupled with updating the
free index to a new slot past it. The scanner also uses the
freeindex to determine if an object is allocated. This is somewhat
racy. This CL makes the scanner use a different field, which is
only updated after the object initialization (and a memory
barrier).
Fixes #54596.
Change-Id: I2a57a226369926e7192c253dd0d21d3faf22297c
Reviewed-on: https://go-review.googlesource.com/c/go/+/449017 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Mateusz Poliwczak [Mon, 14 Nov 2022 20:50:02 +0000 (20:50 +0000)]
net: use a consistent dnsConfig in hostLookupOrder
Use the same dnsConfig throughout a DNS lookup operation.
Before this CL it was possible to decide to re-read a
modified resolv.conf file during the DNS lookup,
which could lead to inconsistencies between the lookup order
and the name server list.
Change-Id: I0689749272b8263268d00b9a9cb4458cd68b23eb
GitHub-Last-Rev: 64810a22bc8a7dd5e804b5f5253d11b73942dfe3
GitHub-Pull-Request: golang/go#56690
Reviewed-on: https://go-review.googlesource.com/c/go/+/449337 Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Tim King [Mon, 14 Nov 2022 22:56:13 +0000 (14:56 -0800)]
cmd/vet: enable timeformat analyzer
Fixes #48801
Change-Id: I6230b62f77252a9a694b79caacb38ef15af36e39
Reviewed-on: https://go-review.googlesource.com/c/go/+/450495
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Tim King <taking@google.com>
Tim King [Mon, 14 Nov 2022 21:07:36 +0000 (13:07 -0800)]
cmd: update vendored golang.org/x/tools for loopclosure improvements
Updates vet to report errors where a loop variable escapes the current
loop iteration by a call to testing.T.Run where the subtest body invokes
t.Parallel().
Change-Id: I727f01d0cdd479ff1b5b1e4f1872c530bfefb263
Reviewed-on: https://go-review.googlesource.com/c/go/+/450435
Run-TryBot: Tim King <taking@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Per https://url.spec.whatwg.org/#percent-encoded-bytes an invalid
percent encoding should be handled as ordinary text.
Fixes #56732
Change-Id: Ib0259dfd704922905289eebaacbf722e28f6d636
Reviewed-on: https://go-review.googlesource.com/c/go/+/450375
Run-TryBot: Ian Lance Taylor <iant@google.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Alexander Scheel [Wed, 2 Nov 2022 11:44:41 +0000 (11:44 +0000)]
crypto/x509: remove unused member Raw from certificate
As noticed in the review of the CRL RawIssuer updates (https://go-review.googlesource.com/c/go/+/418834), the Raw field on the internal type certificate of crypto/x509 is unused and could be removed.
From looking at encoding/asn1's implementation, it appears this field would be set on unmarshal but not during marshaling. However, we unmarshal into the x509.Certificate class directly, avoiding this internal class entirely.
Change-Id: I1ab592eb939b6fe701206ba77b6727763deaeaf0
GitHub-Last-Rev: 5272e0d369ae1b9fee350c2731a6084f41011724
GitHub-Pull-Request: golang/go#56524
Reviewed-on: https://go-review.googlesource.com/c/go/+/447215
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
runtime,cmd/internal/obj/x86: use TEB TLS slots on windows/amd64
This CL redesign how we get the TLS pointer on windows/amd64.
We were previously reading it from the [TEB] arbitrary data slot,
located at 0x28(GS), which can only hold 1 TLS pointer.
With this CL, we will read the TLS pointer from the TEB TLS slot array,
located at 0x1480(GS). The TLS slot array can hold multiple
TLS pointers, up to 64, so multiple Go runtimes running on the
same thread can coexists with different TLS.
Each new TLS slot has to be allocated via [TlsAlloc],
which returns the slot index. This index can then be used to get the
slot offset from GS with the following formula: 0x1480 + index*8
The slot index is fixed per Go runtime, so we can store it
in runtime.tls_g and use it latter on to read/update the TLS pointer.
Loading the TLS pointer requires the following asm instructions:
MOVQ runtime.tls_g, AX
MOVQ AX(GS), AX
Notice that this approach is also implemented on windows/arm64.
qmuntal [Tue, 10 May 2022 07:52:20 +0000 (09:52 +0200)]
os,syscall: File.Stat to use file handle for directories on Windows
Updates syscall.Open to support opening directories via CreateFileW.
CreateFileW handles are more versatile than FindFirstFile handles.
They can be used in Win32 APIs like GetFileInformationByHandle and
SetFilePointerEx, which are needed by some Go APIs.
Damien Neil [Mon, 14 Nov 2022 18:21:51 +0000 (10:21 -0800)]
os: don't request read access from CreateFile in Stat
CL 448897 changed os.Stat to request GENERIC_READ access when using
CreateFile to examine a file. This is unnecessary; access flags of 0
will permit examining file metadata even if the file isn't readable.
Revert to the old behavior here.
Paul E. Murphy [Thu, 24 Mar 2022 17:06:27 +0000 (12:06 -0500)]
crypto/aes: rewrite ppc64le asm-cbc routine
This loads the keys once per call, not once per block. This
has the effect of unrolling the inner loop too. This allows
decryption to scale better with available hardware.
Noteably, encryption serializes crypto ops, thus no
performance improvement is seen, but neither is it reduced.
Care is also taken to explicitly clear keys from registers
as was done implicitly in the prior version.
Also, fix a couple of typos from copying the asm used to
load ESPERM.
Performance delta on POWER9:
name old time/op new time/op delta
AESCBCEncrypt1K 1.10µs ± 0% 1.10µs ± 0% +0.55%
AESCBCDecrypt1K 793ns ± 0% 415ns ± 0% -47.70%
Russ Cox [Fri, 11 Nov 2022 17:36:31 +0000 (12:36 -0500)]
internal/godebug: define more efficient API
We have been expanding our use of GODEBUG for compatibility,
and the current implementation forces a tradeoff between
freshness and efficiency. It parses the environment variable
in full each time it is called, which is expensive. But if clients
cache the result, they won't respond to run-time GODEBUG
changes, as happened with x509sha1 (#56436).
This CL changes the GODEBUG API to provide efficient,
up-to-date results. Instead of a single Get function,
New returns a *godebug.Setting that itself has a Get method.
Clients can save the result of New, which is no more expensive
than errors.New, in a global variable, and then call that
variable's Get method to get the value. Get costs only two
atomic loads in the case where the variable hasn't changed
since the last call.
Unfortunately, these changes do require importing sync
from godebug, which will mean that sync itself will never
be able to use a GODEBUG setting. That doesn't seem like
such a hardship. If it was really necessary, the runtime could
pass a setting to package sync itself at startup, with the
caveat that that setting, like the ones used by runtime itself,
would not respond to run-time GODEBUG changes.
Change-Id: I99a3acfa24fb2a692610af26a5d14bbc62c966ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/449504
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Alexander Yastrebov [Fri, 11 Nov 2022 23:23:19 +0000 (23:23 +0000)]
archive/zip: don't read directories containing file data
Fixes #54801
Change-Id: I3d03516792975ddb09835b2621c57e12e7cbad35
GitHub-Last-Rev: 4faa7e14dcc48c05d707f1e137d915da24133e14
GitHub-Pull-Request: golang/go#56714
Reviewed-on: https://go-review.googlesource.com/c/go/+/449955
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Joedian Reid <joedian@golang.org>
Robert Griesemer [Thu, 10 Nov 2022 23:28:03 +0000 (15:28 -0800)]
spec: document the new unsafe functions SliceData, String, and StringData
For #53003.
Change-Id: If5d76c7b8dfcbcab919cad9c333c0225fc155859
Reviewed-on: https://go-review.googlesource.com/c/go/+/449537 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
ruinan [Tue, 8 Nov 2022 08:49:53 +0000 (16:49 +0800)]
math/bits: directly calculate quo/rem when hi is zero in Div64
func Div64(hi, lo, y uint64) (quo, rem uint64) {...}
math/bits.Div64 returns the quotient and remainder of (hi, lo) divided
by y. When hi is zero, we can directly return lo/y, lo%y, which can save
a lot of unnecessary calculations.
The performance is measured on arm64 and the changes will only affect
the arch that doesn't have the intrinsic.
Robert Griesemer [Thu, 10 Nov 2022 21:50:33 +0000 (13:50 -0800)]
spec: clarify struct field and array element comparison order
Fixes #8606.
Change-Id: I64b13b2ed61ecae4641264deb47c9f7653a80356
Reviewed-on: https://go-review.googlesource.com/c/go/+/449536 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Ian Lance Taylor [Fri, 11 Nov 2022 20:42:51 +0000 (12:42 -0800)]
internal/syscall/unix: use runtime.gostring for Gostring
Under the race detector, checkptr flags uses of unsafe.Slice that
result in slices that straddle multiple Go allocations.
Avoid that scenario by calling existing runtime code.
This fixes a failure on the darwin-.*-race builders introduced in
CL 446178.
Change-Id: I6e0fdb37e3c3f38d97939a8799bb4d10f519c5b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/449936 Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>