Robert Griesemer [Mon, 25 Jan 2021 23:39:37 +0000 (15:39 -0800)]
[dev.typeparams] go/constant: choose internal float representations more consistently
go/constant represents a Float constant either as a rational number
(if numerator and denominator are small enough), or, as a "catch-all",
as a arbitrary-precision floating-point number.
This CL cleans up some of these transitions by factoring out more
of the decision logic and documents the rationale between the state
transitions better.
This CL also simplifies some unrelated code that was overly complex.
Updates #43908.
Change-Id: Iccdd2d6b7fb7ed13d68ed5e6d992d1bc56a065bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/286572
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Matthew Dempsky [Tue, 26 Jan 2021 01:26:07 +0000 (17:26 -0800)]
[dev.typeparams] cmd/compile: force untyped constants from types2 to expected kind
Currently, types2 sometimes produces constant.Values with a Kind
different than the untyped constant type's Is{Integer,Float,Complex}
info, which irgen expects to always match.
While we mull how best to proceed in #43891, this CL adapts irgen to
types2's current behavior. In particular, fixedbugs/issue11945.go now
passes with -G=3.
Updates #43891.
Change-Id: I24823a32ff49af6045a032d3903dbb55cbec6bef
Reviewed-on: https://go-review.googlesource.com/c/go/+/286652
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
+ 2021-01-25 bf0f7c9d78 doc/go1.16: mention os.DirFS in os section
+ 2021-01-25 deaf29a8a8 cmd/compile: fix order-of-assignment issue w/ defers
+ 2021-01-25 ad2ca26a52 doc/go1.16: mention os.DirEntry and types moved from os to io/fs
+ 2021-01-25 a51921fa5b doc/go1.16: mention new testing/iotest functions
+ 2021-01-25 e6b6d107f7 doc/go1.16: mention deprecation of io/ioutil
+ 2021-01-25 96a276363b doc/go1.16: mention go/build changes
+ 2021-01-25 3d85c69a0b html/template: revert "avoid race when escaping updates template"
+ 2021-01-25 54514c6b28 cmd/go: fix TestScript/cgo_path, cgo_path_space when CC set
+ 2021-01-25 6de8443f3b doc/asm: add a section on go_asm.h, clean up go_tls.h section
+ 2021-01-25 54b251f542 lib/time, time/tzdata: update tzdata to 2021a
+ 2021-01-25 ff82cc971a os: force consistent mtime before running fstest on directory on Windows
+ 2021-01-25 044f937a73 doc/go1.16: fix WalkDir and Walk links
+ 2021-01-23 b634f5d97a doc/go1.16: add crypto/x509 memory optimization
+ 2021-01-23 9897655c61 doc/go1.16: reword ambiguously parsable sentence
+ 2021-01-23 cd99385ff4 cmd/internal/obj/arm64: fix VMOVQ instruction encoding error
+ 2021-01-23 66ee8b158f runtime: restore cgo_import_dynamic for libc.so on openbsd
+ 2021-01-22 25c39e4fb5 io/ioutil: fix example test for WriteFile to allow it to run in the playground
+ 2021-01-22 eb21b31e48 runtime: define dummy msanmove
+ 2021-01-22 3a778ff50f runtime: check for g0 stack last in signal handler
+ 2021-01-22 a2cef9b544 cmd/go: don't lookup the path for CC when invoking cgo
Ian Lance Taylor [Fri, 22 Jan 2021 01:58:17 +0000 (17:58 -0800)]
doc/go1.16: mention os.DirFS in os section
For #40700
For #41190
Change-Id: I8ade6efd5be09003fc3e5db5a9b91ba6e0f023f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/285593
Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
CL 261677 fixed a logic issue in walk's alias detection, where it was
checking the RHS expression instead of the LHS expression when trying
to determine the kind of assignment. However, correcting this exposed
a latent issue with assigning to result parameters in functions with
defers, where an assignment could become visible earlier than intended
if a later expression could panic.
Fixes #43835.
Change-Id: I061ced125e3896e26d65f45b28c99db2c8a74a8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/285633
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Ian Lance Taylor [Fri, 22 Jan 2021 01:29:36 +0000 (17:29 -0800)]
doc/go1.16: mention os.DirEntry and types moved from os to io/fs
For #40700
For #41467
For #41190
Change-Id: Id94e7511c98c38a22b1f9a55af6e200c9df07fd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/285592
Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Ian Lance Taylor [Fri, 22 Jan 2021 06:47:18 +0000 (22:47 -0800)]
doc/go1.16: mention new testing/iotest functions
For #38781
For #40700
For #41190
Change-Id: I72f1055e51edb517041d3861640734ba6ef5f342
Reviewed-on: https://go-review.googlesource.com/c/go/+/285673
Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Robert Griesemer [Mon, 25 Jan 2021 23:18:56 +0000 (15:18 -0800)]
[dev.typeparams] go/constant: in ToFloat, convert to rational numbers, not floats
Floating-point constants are represented as rational numbers when
possible (i.e., when numerators and denominators are not too large).
If we convert to floats when not necessary, we risk losing precision.
This is the minimal fix for the specific issue, but it's too aggressive:
If the numbers are too large, we still want to convert to floats.
Will address in a separate CL that also does a few related cleanups.
Fixes #43908.
Change-Id: Id575e34fa18361a347c43701cfb4dd7221997f66
Reviewed-on: https://go-review.googlesource.com/c/go/+/286552
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Ian Lance Taylor [Fri, 22 Jan 2021 00:23:18 +0000 (16:23 -0800)]
doc/go1.16: mention deprecation of io/ioutil
For #40025
For #40700
For #42026
Change-Id: Ib51b5e1398c4eb811506df21e3bd56dd84bd1f7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/285377
Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Matthew Dempsky [Mon, 25 Jan 2021 22:14:10 +0000 (14:14 -0800)]
[dev.regabi] cmd/compile: disallow taking address of SSA'd values
Adds some extra validation that the frontend is setting flags like
Addrtaken correctly.
Change-Id: Iffde83e32ba1c4c917ab8cb3fe410a4f623cf635
Reviewed-on: https://go-review.googlesource.com/c/go/+/286434
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Ian Lance Taylor [Thu, 21 Jan 2021 04:41:16 +0000 (20:41 -0800)]
doc/go1.16: mention go/build changes
For #40070
For #41191
For #43469
For #43632
Change-Id: I6dc6b6ea0f35876a4c252e4e287a0280aca9d502
Reviewed-on: https://go-review.googlesource.com/c/go/+/285213
Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Ian Lance Taylor [Fri, 22 Jan 2021 22:54:23 +0000 (14:54 -0800)]
html/template: revert "avoid race when escaping updates template"
This reverts CLs 274450 and 279492, except for the new tests.
The new race test is changed to skip, as it now fails.
We can try again for 1.17.
Original CL descriptions:
html/template: attach functions to namespace
The text/template functions are stored in a data structure shared by
all related templates, so do the same with the original, unwrapped,
functions on the html/template side.
html/template: avoid race when escaping updates template
For #39807
Fixes #43855
Change-Id: I2ce91321ada06ea496a982aefe170eb5af9ba847
Reviewed-on: https://go-review.googlesource.com/c/go/+/285957
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Austin Clements [Thu, 21 Jan 2021 15:19:21 +0000 (10:19 -0500)]
[dev.regabi] cmd/compile/internal: add internal ABI specification
This adds a document specifying the internal ABI (specifically the
calling convention). This document lives in the Go tree (rather than
the proposal repository) because the intent is for it to track the
reality in the compiler.
Austin Clements [Fri, 22 Jan 2021 15:58:12 +0000 (10:58 -0500)]
doc/asm: add a section on go_asm.h, clean up go_tls.h section
Currently the only mention of go_asm.h is buried in a confusing
section about the runtime-specific go_tls.h header. We actually want
people to use go_asm.h, so this CL adds a section with a proper
discussion of this header. As part of this, we remove the discussion
of go_asm.h from the go_tls.h section and clean up what remains.
I stumbled on this when working on the internal ABI specification. I
wanted to refer to stable documentation on how to access struct fields
from assembly and found there was none.
Change-Id: I0d53741e7685e65794611939e76285f7c82e1d65
Reviewed-on: https://go-review.googlesource.com/c/go/+/286052
Trust: Austin Clements <austin@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
This allows more precision and matches types2's behavior.
For backwards compatibility with gcimporter, for now we still need to
write out declared constants as limited-precision floating-point
values. To ensure consistent behavior of constant arithmetic whether
it spans package boundaries or not, we include the full-precision
rational representation in the compiler's extension section of the
export data.
Also, this CL simply uses the math/big.Rat.String text representation
as the encoding. This is inefficient, but because it's only in the
compiler's extension section, we can easily revisit this in the
future.
Declaring exported untyped float and complex constants isn't very
common anyway. Within the standard library, only package math declares
any at all, containing just 15. And those 15 are only imported a total
of 12 times elsewhere in the standard library.
Change-Id: I85ea23ab712e93fd3b68e52d60cbedce9be696a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/286215
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Mon, 25 Jan 2021 05:26:14 +0000 (21:26 -0800)]
[dev.regabi] os: disable TestDirFS until #42637 is fixed
This test is causing nearly every trybot run on dev.regabi and
dev.typeparams to fail. It's already a release blocker for Go 1.16, so
the failures on the development branches is entirely noise; and
because it causes the trybots to short-circuit, it risks masking
actual Windows-specific failures.
This CL disables the test until a proper solution is decided upon and
implemented for Go 1.16.
Updates #42637.
Change-Id: Ibc85edaed591f1c125cf0b210867aa89d2b0a4b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/286213
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Mon, 25 Jan 2021 07:31:25 +0000 (23:31 -0800)]
[dev.typeparams] cmd/compile: fix -G=3 handling of blank methods
Fixes "GO_GCFLAGS=-G=3 go run run.go -- blank.go interface/fail.go".
Change-Id: I669ab06ae29366ce96e2948c89a5c1620afd53db
Reviewed-on: https://go-review.googlesource.com/c/go/+/286214
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> Reviewed-by: Dan Scales <danscales@google.com>
Matthew Dempsky [Mon, 25 Jan 2021 05:04:39 +0000 (21:04 -0800)]
[dev.typeparams] test: enable more errorcheck tests w/ -G=3
Change-Id: I170e4f9c5a1db4bad02a5fe4bddc65d4c75f51e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/286232
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Sun, 24 Jan 2021 20:26:47 +0000 (12:26 -0800)]
[dev.regabi] cmd/go: workaround -race issue on ppc64le
The race detector on ppc64le corrupts command-line arguments lists if
they contain an empty string, and cmd/go often generates compiler
argument lists containing `-D ""`. Since this is equivalent to not
specifying the `-D` flag at all, just do that. This allows using a
race-detector-enabled cmd/compile on ppc64le.
Updates #43883.
Change-Id: Ifac5cd9a44932129438b9b0b3ecc6101ad3716b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/286173
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
David Chase [Thu, 21 Jan 2021 17:02:39 +0000 (12:02 -0500)]
[dev.regabi] cmd/compile: modify abiutils for recently updated ABI
Discovered difficluties posed by earlier design, these modifications
should work better. Updated tests, also added some helper functions
for use in call lowering.
Change-Id: I459f0f71ad8a6730c571244925c3f395e1df28de
Reviewed-on: https://go-review.googlesource.com/c/go/+/285392
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Jason A. Donenfeld [Mon, 18 Jan 2021 14:23:16 +0000 (15:23 +0100)]
os: force consistent mtime before running fstest on directory on Windows
FindFileNext sometimes returns a different mtime than looking at the
file directly, because the MFT on NTFS is written to lazily. In order to
keep these in sync, we use GetFileInformationByHandle to get the actual
mtime, and then write it back to the file explicitly.
Fixes #42637.
Change-Id: I774016d3ac55d0dc9b0f9c1b681516c33ba0d28a
Reviewed-on: https://go-review.googlesource.com/c/go/+/285720
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
+ 2021-01-25 063c72f06d [dev.regabi] cmd/compile: backport changes from dev.typeparams (9456804)
+ 2021-01-23 d05d6fab32 [dev.regabi] cmd/compile: replace ir.Name map with ir.NameSet for SSA 2
+ 2021-01-23 48badc5fa8 [dev.regabi] cmd/compile: fix escape analysis problem with closures
+ 2021-01-23 51e1819a8d [dev.regabi] cmd/compile: scan body of closure in tooHairy to check for disallowed nodes
Matthew Dempsky [Sun, 24 Jan 2021 18:52:00 +0000 (10:52 -0800)]
[dev.regabi] cmd/compile: backport changes from dev.typeparams (9456804)
This CL backports a bunch of changes that landed on dev.typeparams,
but are not dependent on types2 or generics. By backporting, we reduce
the divergence between development branches, hopefully improving test
coverage and reducing risk of merge conflicts.
Updates #43866.
Change-Id: I382510855c9b5fac52b17066e44a00bd07fe86f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/286172
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
The VMOVQ instruction moves a 128-bit constant into a V register, as 128-bit
constant can't be loaded into a register directly, we split it into two 64-bit
constants and load it from constant pool. Currently we add the 128-bit constant
to literal pool by calling the 'addpool' function twice, this is not the right
way because it doesn't guarantee the two DWORD instructions are consecutive,
and the second call of addpool will overwrite the p.Pool field,resulting in a
wrong PC-relative offset value of the Prog.
This CL renames the flag LFROM3 to LFROM128, and adds a new function addpool128
to add a 128-bit constant to the literal pool.
Change-Id: I616f043c99a9a18a663f8768842cc980de2e6f79
Reviewed-on: https://go-review.googlesource.com/c/go/+/282334 Reviewed-by: eric fang <eric.fang@arm.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: eric fang <eric.fang@arm.com>
Trust: eric fang <eric.fang@arm.com>
Baokun Lee [Sat, 23 Jan 2021 09:05:01 +0000 (17:05 +0800)]
[dev.regabi] cmd/compile: replace ir.Name map with ir.NameSet for SSA 2
Same as CL 284897, the last one.
Passes toolstash -cmp.
Updates #43819
Change-Id: I0bd8958b3717fb58a5a6576f1819a85f33b76e2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/285913
Run-TryBot: Baokun Lee <bk@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Baokun Lee <bk@golangcn.org>
Robert Griesemer [Sat, 23 Jan 2021 01:49:57 +0000 (17:49 -0800)]
[dev.typeparams] test: set -G=3 and enable more errorcheck tests in run.go
Change-Id: I9591f7aeab0448aca661560bf3064e057b48293e
Reviewed-on: https://go-review.googlesource.com/c/go/+/286012
Trust: Robert Griesemer <gri@golang.org>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dan Scales <danscales@google.com>
Joel Sing [Fri, 22 Jan 2021 19:18:30 +0000 (06:18 +1100)]
runtime: restore cgo_import_dynamic for libc.so on openbsd
This was removed in change 285692, however we need to explicitly pull libc.so
in when libpthread.so is being used. The current code works on openbsd/amd64
since we pull libc.so in via runtime/sys_openbsd2.go, however openbsd/arm64
does not do this currently.
Change-Id: Ibe93d936a22e69e2fe12620f6d27ccca7a91dba5
Reviewed-on: https://go-review.googlesource.com/c/go/+/285912
Trust: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Dan Scales [Sat, 23 Jan 2021 00:07:00 +0000 (16:07 -0800)]
[dev.regabi] cmd/compile: fix escape analysis problem with closures
In reflect.methodWrapper, we call escape analysis without including the
full batch of dependent functions, including the closure functions.
Because of this, we haven't created locations for the params/local
variables of a closure when we are processing a function that
inlines that closure. (Whereas in the normal compilation of the
function, we do call with the full batch.) To deal with this, I am
creating locations for the params/local variables of a closure when
needed.
Without this fix, the new test closure6.go would fail.
Updates #43818
Change-Id: I5f91cfb6f35efe2937ef88cbcc468e403e0da9ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/285677
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dan Scales <danscales@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Robert Griesemer [Sat, 23 Jan 2021 00:49:21 +0000 (16:49 -0800)]
[dev.typeparams] cmd/compile/internal/types2: use same sort criteria for methods as compiler
Note: This invalidates the implementation of MethodSet further (it
also has not been updated to accomodate for type parameters). But
types2 doesn't make use of it. We should remove it.
Change-Id: Ia2601bdd59b3f3ee0b72bc2512153c42bf5053b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/285994
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Matthew Dempsky [Fri, 22 Jan 2021 21:29:59 +0000 (13:29 -0800)]
[dev.typeparams] cmd/compile: refactor SelectorExpr code into helpers
This CL refactors the SelectorExpr-handling code added in CL 285373
into helper functions that can eventually be reused by iimport.
Change-Id: I15b4a96c242f63cb370d7492ed08168550724f47
Reviewed-on: https://go-review.googlesource.com/c/go/+/285953
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Fri, 22 Jan 2021 23:35:11 +0000 (15:35 -0800)]
[dev.typeparams] all: merge dev.regabi (7e0a81d) into dev.typeparams
As with CL 285875, this required resolving some conflicts around
handling of //go:embed directives. Still further work is needed to
reject uses of //go:embed in files that don't import "embed", so this
is left as a TODO. (When this code was written for dev.typeparams, we
were still leaning towards not requiring the "embed" import.)
Also, the recent support for inlining closures (CL 283112) interacts
poorly with -G=3 mode. There are some known issues with this code
already (#43818), so for now this CL disables inlining of closures
when in -G=3 mode with a TODO to revisit this once closure inlining is
working fully.
Dan Scales [Fri, 22 Jan 2021 22:32:06 +0000 (14:32 -0800)]
[dev.regabi] cmd/compile: scan body of closure in tooHairy to check for disallowed nodes
Several of the bugs in #43818 are because we were not scanning the body
of an possibly inlined closure in tooHairy(). I think this scanning got
lost in the rebase past some of the ir changes. This fixes the issue
related to the SELRECV2 and the bug reported from cuonglm. There is at
least one other bug related to escape analysis which I'll fix in another
change.
Change-Id: I8f38cd12a287881155403bbabbc540ed5fc2248e
Reviewed-on: https://go-review.googlesource.com/c/go/+/285676
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Pantonshire [Mon, 18 Jan 2021 17:41:36 +0000 (17:41 +0000)]
io/ioutil: fix example test for WriteFile to allow it to run in the playground
The example for WriteFile assumed the existence of a testdata/ directory, which is not present on the playground. The example now writes the file to the current working directory, rather than to testdata/.
Fixes #32916
Change-Id: I577caac7e67ba9d9941b2dd19346ad5ff61e78d9
GitHub-Last-Rev: 40f14e0adc4ebc00fb2946fe0cbaf8e0cb99f62c
GitHub-Pull-Request: golang/go#43757
Reviewed-on: https://go-review.googlesource.com/c/go/+/284452
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Cherry Zhang [Fri, 22 Jan 2021 21:35:21 +0000 (16:35 -0500)]
runtime: define dummy msanmove
In msan mode we instrument code with msan* functions, including
msanmove. In some configurations the code is instrumented by the
compiler but msan is not actually linked in, so we need dummy
definitions for those functions so the program links. msanmove is
newly added in CL 270859 but a dummy definition in msan0.go was
not added, causing link failures. Add it.
Change-Id: I91f8e749919f57f1182e90b43412b0282cf4767c
Reviewed-on: https://go-review.googlesource.com/c/go/+/285955
Trust: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Fri, 22 Jan 2021 14:47:59 +0000 (09:47 -0500)]
runtime: check for g0 stack last in signal handler
In the signal handler, we adjust gsingal's stack to the stack
where the signal is delivered. TSAN may deliver signals to the
g0 stack, so we have a special case for the g0 stack. However,
we don't have very good accuracy in determining the g0 stack's
bounds, as it is system allocated and we don't know where it is
exactly. If g0.stack.lo is too low, the condition may be
triggered incorrectly, where we thought the signal is delivered to
the g0 stack but it is actually not. In this case, as the stack
bounds is actually wrong, when the stack grows, it may go below
the (inaccurate) lower bound, causing "morestack on gsignal"
crash.
Check for g0 stack last to avoid this situation. There could still
be false positives, but for those cases we'll crash either way.
(If we could in some way determine the g0 stack bounds accurately,
this would not matter (but probably doesn't hurt).)
Jay Conrod [Fri, 22 Jan 2021 19:27:24 +0000 (14:27 -0500)]
cmd/go: don't lookup the path for CC when invoking cgo
Previously, if CC was a path without separators (like gcc or clang),
we'd look it up in PATH in cmd/go using internal/execabs.LookPath,
then pass the resolved path to cgo in CC.
This caused a regression: if the directory in PATH containing CC has a
space, cgo splits it and interprets it as multiple arguments.
With this change, cmd/go no longer resolves CC before invoking
cgo. cgo does the path lookup on each invocation. This reverts the
security fix CL 284780, but that was redundant with the addition of
internal/execabs (CL 955304), which still protects us.
Fixes #43808
Updates #41400
Change-Id: I65d91a1e303856df8653881eb6e2e75a3bf95c49
Reviewed-on: https://go-review.googlesource.com/c/go/+/285873
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Matthew Dempsky [Fri, 22 Jan 2021 20:00:45 +0000 (12:00 -0800)]
[dev.regabi] all: merge master (dab3e5a) into dev.regabi
This merge had two conflicts to resolve:
1. The embed code on master had somewhat substantially diverged, so
this CL tediously backported the changes to dev.regabi. In particular,
I went through all of the embed changes to gc/{embed,noder,syntax}.go
and made sure the analogous code on dev.regabi in noder/noder.go and
staticdata/embed.go mirrors it.
2. The init-cycle reporting code on master was extended slightly to
track already visited declarations to avoid exponential behavior. The
same fix is applied on dev.regabi, just using ir.NameSet instead of
map[ir.Node]bool.
Matthew Dempsky [Fri, 22 Jan 2021 09:46:42 +0000 (01:46 -0800)]
[dev.typeparams] cmd/compile: refactor irgen's handling of ":="
The previous code was stylized after noder, which was written when it
was more idiomatic to simple create a gc.Node and then populate and
shuffle around its fields as appropriate.
Now with package ir, it's somewhat nicer to compute all the fields up
front and pass them to the constructor functions, rather than passing
nil and populating the fields afterwards.
Net addition of lines of code, but I think the new code is overall
still somewhat simpler, and will be easier to refactor out into code
for helpers.go.
Change-Id: I8c6f6b65e0a8317129655a0fc493d8af75527b97
Reviewed-on: https://go-review.googlesource.com/c/go/+/285732
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dan Scales <danscales@google.com>
Michael Matloob [Sat, 16 Jan 2021 00:41:26 +0000 (19:41 -0500)]
cmd/go: add documentation for test and xtest fields output by go list
The TestEmbedPatterns, TestEmbedFiles, XTestEmbedPatterns, and
XTestEmbedFiles fields were left out of golang.org/cl/282195 which was
supposed to document the embed fields available in the go list
output. Add documentation for them in this CL.
Fixes #43081
Change-Id: Ifc256c476daec7c0f0e2c41f86b82f958b3e2b1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/284258
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Rob Findley [Tue, 19 Jan 2021 22:30:09 +0000 (17:30 -0500)]
[dev.typeparams] go/types: import api_test.go changes from dev.go2go
This CL imports tests for the go/types API from the dev.go2go branch.
Only parse type parameters for packages with a magic prefix, with the
rationale that while generics are in preview, we want existing
(non-generic) tests to exercise the default mode.
Change-Id: I8ae0d8769b997a8a93b708453a1afaecb262244d
Reviewed-on: https://go-review.googlesource.com/c/go/+/284693
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Joel Sing [Fri, 22 Jan 2021 08:32:43 +0000 (19:32 +1100)]
runtime: remove pthread_kill/pthread_self for openbsd
We're now using getthrid() and thrkill() instead.
Updates #36435
Change-Id: I1c6bcfb9b46d149e0a2a10e936a244576489a88e
Reviewed-on: https://go-review.googlesource.com/c/go/+/285692
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Dan Scales [Tue, 19 Jan 2021 21:54:33 +0000 (13:54 -0800)]
[dev.typeparams] cmd/compile: disambiguate OXDOT in noder using types2 Selection info
By using the types2 Selection information, we can create ODOT, ODOTPTR,
OCALLPART, ODOTMETH, ODOTINTER, and OMETHEXPR nodes directly in noder,
so we don't have to do that functionality in typecheck.go. Intermediate
nodes are created as needed for embedded fields. Don't have to typecheck
the results of g.selectorExpr(), because we set the types of all the
needed nodes.
There is one bug remaining in 'go test reflect' that will be fixed when dev.regabi is merged.
Change-Id: I4599d43197783e318610deb2f208137f9344ab63
Reviewed-on: https://go-review.googlesource.com/c/go/+/285373
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dan Scales <danscales@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Damien Neil [Fri, 22 Jan 2021 02:24:55 +0000 (18:24 -0800)]
net/http: fix flaky TestDisableKeepAliveUpgrade
This test hijacks a connection. It was reading from the net.Conn
returned by Hijack, not the bufio.ReadWriter, causing flaky failures
when a read-ahead byte was held in the read buffer.
Fixes #43073.
Change-Id: Ic3e7f704fba9635fd851cb3c0c0c74e312b75f6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/285596
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Anmol Sethi <nhooyr@gmail.com>
Robert Griesemer [Fri, 22 Jan 2021 04:20:22 +0000 (20:20 -0800)]
[dev.typeparams] cmd/compile/internal/types2: adjust errors in branch checking code, fix a bug
The types2.Config.IgnoreBranches flag mistakenly excluded a
set of label-unrelated branch checks. After fixing this and
also adjusting some error messages to match the existing
compiler errors, more errorcheck tests pass now with the -G
option.
Renamed IngnoreBranches to IgnoreLabels since its controlling
label checks, not all branch statement (such as continue, etc)
checks.
Change-Id: I0819f56eb132ce76c9a9628d8942af756691065a
Reviewed-on: https://go-review.googlesource.com/c/go/+/285652
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Ian Lance Taylor [Thu, 21 Jan 2021 02:01:56 +0000 (18:01 -0800)]
doc/go1.16: add notes about package-specific fs.FS changes
For #40700
For #41190
Change-Id: I964d6856d5cad62c859d0f3a7afdd349a8ad87cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/285093
Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Ian Lance Taylor [Thu, 21 Jan 2021 04:09:18 +0000 (20:09 -0800)]
doc/go1.16: mention new debug/elf constants
For #40700
Change-Id: If105d2f043539bb0893f577a984f14ee3e7ca753
Reviewed-on: https://go-review.googlesource.com/c/go/+/285212
Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Roland Shoemaker [Fri, 15 Jan 2021 20:14:06 +0000 (12:14 -0800)]
all: introduce and use internal/execabs
Introduces a wrapper around os/exec, internal/execabs, for use in
all commands. This wrapper prevents exec.LookPath and exec.Command from
running executables in the current directory.
All imports of os/exec in non-test files in cmd/ are replaced with
imports of internal/execabs.
Russ Cox [Mon, 11 Jan 2021 14:43:08 +0000 (09:43 -0500)]
cmd/go: add test case for cgo CC setting
Change-Id: Ied986053a64447c5eac6369f6c9b69ed3d3f94d9
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/949415 Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/284782
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Russ Cox [Mon, 11 Jan 2021 15:01:24 +0000 (10:01 -0500)]
cmd/cgo: report exec errors a bit more clearly
Change-Id: I0e6bebf0e2e6efdef4be880e0c6c7451b938924b
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/949417 Reviewed-by: Katie Hockman <katiehockman@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/284781
Run-TryBot: Roland Shoemaker <roland@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Russ Cox [Mon, 11 Jan 2021 14:41:54 +0000 (09:41 -0500)]
cmd/go: pass resolved CC, GCCGO to cgo
This makes sure the go command and cgo agree about
exactly which compiler is being used.
This issue was reported by RyotaK.
Fixes CVE-2021-3115
Fixes #43783
Change-Id: If171c5c8b2523efb5ea2d957e5ad1380a038149c
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/949416 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/284780
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Filippo Valsorda [Fri, 8 Jan 2021 02:56:58 +0000 (03:56 +0100)]
crypto/elliptic: fix P-224 field reduction
This patch fixes two independent bugs in p224Contract, the function that
performs the final complete reduction in the P-224 field. Incorrect
outputs due to these bugs were observable from a high-level
P224().ScalarMult() call.
The first bug was in the calculation of out3GT. That mask was supposed
to be all ones if the third limb of the value is greater than the third
limb of P (out[3] > 0xffff000). Instead, it was also set if they are
equal. That meant that if the third limb was equal, the value was always
considered greater than or equal to P, even when the three bottom limbs
were all zero. There is exactly one affected value, P - 1, which would
trigger the subtraction by P even if it's lower than P already.
The second bug was more easily hit, and is the one that caused the known
high-level incorrect output: after the conditional subtraction by P, a
potential underflow of the lowest limb was not handled. Any values that
trigger the subtraction by P (values between P and 2^224-1, and P - 1
due to the bug above) but have a zero lowest limb would produce invalid
outputs. Those conditions apply to the intermediate representation
before the subtraction, so they are hard to trace to precise inputs.
This patch also adds a test suite for the P-224 field arithmetic,
including a custom fuzzer that automatically explores potential edge
cases by combining limb values that have various meanings in the code.
contractMatchesBigInt in TestP224Contract finds the second bug in less
than a second without being tailored to it, and could eventually find
the first one too by combining 0, (1 << 28) - 1, and the difference of
(1 << 28) and (1 << 12).
The incorrect P224().ScalarMult() output was found by the
elliptic-curve-differential-fuzzer project running on OSS-Fuzz and
reported by Philippe Antoine (Catena cyber).
Fixes CVE-2021-3114
Fixes #43786
Change-Id: I50176602d544de3da854270d66a293bcaca57ad7
Reviewed-on: https://go-review.googlesource.com/c/go/+/284779
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Roland Shoemaker <roland@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org>
Baokun Lee [Thu, 21 Jan 2021 07:24:38 +0000 (15:24 +0800)]
[dev.regabi] cmd/compile: replace ir.Name map with ir.NameSet for dwarf
Same as CL 284897, but for dwarf.
Passes toolstash -cmp.
Fixes #43819
Change-Id: Icbe43aa2e3cb96e6a6c318523c643247da8e4c74
Reviewed-on: https://go-review.googlesource.com/c/go/+/284899
Run-TryBot: Baokun Lee <bk@golangcn.org>
Trust: Baokun Lee <bk@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Baokun Lee [Thu, 21 Jan 2021 06:13:36 +0000 (14:13 +0800)]
[dev.regabi] cmd/compile: replace ir.Name map with ir.NameSet in inlining
As CL 282212 mentioned, we should clean all map[*ir.Name]bool with
ir.NameSet.
Passes toolstash -cmp.
Updates #43819
Change-Id: I1ce5d2055f88539f807dc021cd8e3941b425bc4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/284897
Run-TryBot: Baokun Lee <bk@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Baokun Lee <bk@golangcn.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Robert Griesemer [Thu, 21 Jan 2021 01:03:36 +0000 (17:03 -0800)]
[dev.typeparams] cmd/compile: use nil instead of syntax.ImplicitOne
Represent x++/-- as x +=/-= with the RHS of the assignment being nil
rather than syntax.ImplicitOne.
Dependent code already had to check for syntax.ImplicitOne, but
then shared some existing code for regular assignment operations.
Now always handle this case fully explicit, which simplifies the
code.
Change-Id: I28c7918153c27cbbf97b041d0c85ff027c58687c
Reviewed-on: https://go-review.googlesource.com/c/go/+/285172
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Cuong Manh Le [Wed, 20 Jan 2021 07:46:38 +0000 (14:46 +0700)]
[dev.regabi] cmd/compile: remove CallExpr.Rargs
Instead, push the temps assignments to init. This does not pass
toolstash, since when before this, the temps were evaluated after
function callee, now we evaluate them before.
Change-Id: Icb9cb10e036925b56c1ef3eec468416a11f4932f
Reviewed-on: https://go-review.googlesource.com/c/go/+/284894
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Matthew Dempsky [Wed, 20 Jan 2021 20:54:23 +0000 (12:54 -0800)]
[dev.typeparams] cmd/compile: directly set some simple expression types
This CL updates irgen to directly set the type for a bunch of basic
expressions that are easy to handle already. Trickier rewrites are
still handled with typecheck.Expr, but responsibility of calling that
is pushed down to the conversion of individual operations.
Change-Id: I774ac6ab4c72ad854860ab5c741867dd42a066b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/285058
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Matthew Dempsky [Thu, 21 Jan 2021 01:03:36 +0000 (17:03 -0800)]
[dev.typeparams] cmd/compile/internal/types2: convert untyped arguments to delete
For the predeclared "delete" function, types2 was checking that the
second argument was assignable to the map's key type, but not actually
updating the Types map as appropriate. So this could leave untyped
constants in the AST.
The error "cannot convert" is somewhat less precise than the previous
"not assignable" error, but it's consistent with how types2 reports
other erroneous assignments of untyped constants.
Change-Id: Ic3ca3a3611ad0e4646c050e93088cdf992234e5f
Reviewed-on: https://go-review.googlesource.com/c/go/+/285059
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Robert Griesemer [Wed, 20 Jan 2021 23:55:39 +0000 (15:55 -0800)]
[dev.typeparams] cmd/compile/internal/types2: make predeclared "any" alias for interface{}
If we ever decide to permit the use of the predeclared identifier
"any" in lieu of interface{}, it must be an alias for interface{}.
Change-Id: Ic751d7f9b61133fb57625f56ce95d99f034b32c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/285132
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Matthew Dempsky [Wed, 20 Jan 2021 22:08:53 +0000 (14:08 -0800)]
[dev.typeparams] cmd/dist: add -G=3 test coverage
Change-Id: Icb85b93f0d98df722fffd70cf9a2554ac2098c60
Reviewed-on: https://go-review.googlesource.com/c/go/+/285052
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dan Scales <danscales@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Dan Scales [Tue, 1 Dec 2020 22:48:03 +0000 (14:48 -0800)]
[dev.regabi] cmd/compile: exporting, importing, and inlining functions with OCLOSURE
I have exporting, importing, and inlining of functions with closures
working in all cases (issue #28727). all.bash runs successfully without
errors.
Approach:
- Write out the Func type, Dcls, ClosureVars, and Body when exporting
an OCLOSURE.
- When importing an OCLOSURE, read in the type, dcls, closure vars,
and body, and then do roughly equivalent code to (*noder).funcLit
- During inlining of a closure within inlined function, create new
nodes for all params and local variables (including closure
variables), so they can have a new Curfn and some other field
values. Must substitute not only on the Nbody of the closure, but
also the Type, Cvars, and Dcl fields.
Fixes #28727
Change-Id: I4da1e2567c3fa31a5121afbe82dc4e5ee32b3170
Reviewed-on: https://go-review.googlesource.com/c/go/+/283112
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Dan Scales <danscales@google.com>
Matthew Dempsky [Wed, 20 Jan 2021 21:54:53 +0000 (13:54 -0800)]
[dev.typeparams] cmd/compile: simplify how irgen handles qualified idents
This CL moves qualified identifier handling into expr0 with other
selector expressions, rather than as a completely separate special
case handled up front. This has a few benefits:
1. It's marginally simpler/cleaner.
2. It allows extra checking for imported objects that they have the
same type that types2 thought they had.
3. For imported, untyped constants, we now instead handle them with
the "tv.Value != nil" case. In particular, this ensures that they've
always already been coerced to the appropriate concrete type by
types2.
Change-Id: Ibf44ae6901db36aa5251f70934616e9fcbd1cbc5
Reviewed-on: https://go-review.googlesource.com/c/go/+/285053
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Dmitri Shuralyov [Sat, 5 Dec 2020 20:03:27 +0000 (15:03 -0500)]
cmd/internal/moddeps: check content of all modules in GOROOT
Expand the scope of the TestAllDependenciesVendored test to check
that all modules in GOROOT are tidy, that packages are vendored,
the vendor content matches the upstream copy exactly, and that
bundled packages are re-generated (using x/tools/cmd/bundle at
the version selected in cmd module; this is deterministic and
guaranteed to be updated over time).
This is done in a conceptually simple way:
1. Make a temporary copy of the entire GOROOT tree (except .git),
one that is safe to modify.
2. Run a list of high-level commands, the same commands we expect
Go developers should be able to run in a normal complete GOROOT
tree to make it clean and tidy.
3. Diff the end result with the original GOROOT tree being tested
to catch any unexpected differences.
The current set of commands that are run require the cmd/go command,
and a functional compiler itself (because re-generating the syscall
package involves a directive like //go:generate go run [...]). As a
result, copying a large majority of the GOROOT tree is a requirement.
Instead of looking for the few files or directories that can we can
get away not copying (e.g., the testdata directories aren't strictly
needed at this time), we opt not to optimize and just do the simple
copy. This is motivated by these reasons:
• We end up having a complete, normal GOROOT tree, one that happens
to be located at another path. There's a very high likelihood that
module management/code generation commands, both the ones we run
today and any additional ones that we might want to add in the
future, will result in correct results even as the Go project
evolves over time.
• Having a completely stand-alone copy of the GOROOT tree without
symlinks minimizes the risk of some of the module management/code
generation commands, either now or in the future, from modifying
the user's original GOROOT tree, something that should not happen
during test execution. Overlays achieved with symlinks work well
when we can guarantee only new files are added, but that isn't
the case here.
• Copying the entire GOROOT (without .git), takes around 5 seconds
on a fairly modern computer with an SSD. The most we can save is
a couple of seconds.
(We make some minor exceptions: the GOROOT/.git directory isn't copied,
and GOROOT/{bin,pkg} are deemed safe to share and thus symlink instead
of copying. If these optimizations cease to be viable to make, we'll
need to remove them.)
Since this functionality is fairly expensive to execute and requires
network access, it runs only when the test is executed without -short
flag. The previous behavior of the TestAllDependenciesVendored test is
kept in -short test mode. all.bash runs package tests with -short flag,
so its behavior is unchanged. The expectation is that the new test will
run on some of the longtest builders to catch problems. Users can invoke
the test manually 'go test cmd/internal/moddeps' (and it's run as part
of 'go test cmd', again, only when -short flag isn't provided).
On a 2017 MacBook Pro, a successful long test takes under 15 seconds,
which should be within scope of all long tests that are selected by
'go test std cmd'. We may further adjust when and where the test runs
by default based on our experience.
Ian Lance Taylor [Wed, 20 Jan 2021 05:30:36 +0000 (21:30 -0800)]
runtime: don't adjust timer pp field in timerWaiting status
Before this CL, the following sequence was possible:
* GC scavenger starts and sets up scavenge.timer
* GC calls readyForScavenger, but sysmon is sleeping
* program calls runtime.GOMAXPROCS to shrink number of processors
* procresize destroys a P, the one that scavenge.timer is on
* (*pp).destroy calls moveTimers, which gets to the scavenger timer
* scavenger timer is timerWaiting, and moveTimers clears t.pp
* sysmon wakes up and calls wakeScavenger
* wakeScavengers calls stopTimer on scavenger.timer, still timerWaiting
* stopTimer calls deltimer which loads t.pp, which is still nil
* stopTimer tries to increment deletedTimers on nil t.pp, and crashes
The point of vulnerability is the time that t.pp is set to nil by
moveTimers and the time that t.pp is set to non-nil by moveTimers,
which is a few instructions at most. So it's not likely and in
particular is quite unlikely on x86. But with a more relaxed memory
model the area of vulnerability can be somewhat larger. This appears
to tbe the cause of two builder failures in a few months on linux-mips.
This CL fixes the problem by making moveTimers change the status from
timerWaiting to timerMoving while t.pp is clear. That will cause
deltimer to wait until the status is back to timerWaiting, at which
point t.pp has been set again.
Fixes #43712
Change-Id: I66838319ecfbf15be66c1fac88d9bd40e2295852
Reviewed-on: https://go-review.googlesource.com/c/go/+/284775
Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Matloob [Thu, 14 Jan 2021 01:58:00 +0000 (20:58 -0500)]
cmd/go: have go mod vendor copy embedded files in subdirs
If a package vendored with go mod vendor depends on embedded
files contained in subdirectories, copy them into the the
corresponding place in the module's vendor tree. (Embeds in
parent directories are disallowed by the embed pattern rules, and
embeds in the same directory are copied because go mod vendor
already copies the non-go files in the package's own directory).
Export the vendor pattern expansion code in internal/load so
internal/modcmd's vendor code can use it.
Fixes #43077
Change-Id: I61edb344d73df590574a6498ffb6069e8d72a147
Reviewed-on: https://go-review.googlesource.com/c/go/+/283641
Trust: Michael Matloob <matloob@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
Rob Findley [Tue, 12 Jan 2021 14:20:22 +0000 (09:20 -0500)]
[dev.typeparams] go/types: add tests from dev.go2go
Add tests from the dev.go2go branch, modified to eliminate support for
parenthesized type embedding and method type parameters. For the most
part these tests were made to pass via the fixes from preceding CLs in
this stack.
While integrating support for type parameters with the changes to
go/types in master, a decision was made to temporarily use an error code
of 0 for new error messages. Now that these messages are actually
emitted during checking of test packages, it is a test failure for them
to have an error code of 0. To satisfy the test, create a new temporary
error code '_Todo', which represents an error code that has not yet been
assigned. _Todo is added only where it was necessary to make tests pass:
many error codes were left as 0, meaning we don't have any tests that
produce them. This marker may help us produce more comprehensive tests
in the future.
Finally, each package checked by testDir was made into a subtest, for
the ease of running individual packages while debugging test failures.
This seemed worth keeping.
Change-Id: Iba421b797e9fb11af664a73902f67d6c4f30ecad
Reviewed-on: https://go-review.googlesource.com/c/go/+/283854
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
Rob Findley [Fri, 15 Jan 2021 16:42:10 +0000 (11:42 -0500)]
[dev.typeparams] go/types: refactor untyped conversion for typeparams
Some logic was missing in the merge from dev.go2go to deal with untyped
conversion of generic types. Part of this was due to the complexity of
the merge, as untyped conversion had been refactored on master.
Rather than back out the refactoring of untyped conversion, in this CL I
have decided to take it one step further. It was always problematic that
isRepresentable and canConvertUntyped mutated their arguments. In
retrospect the refactoring was perhaps too conservative.
This CL performs the following refactoring:
+ Replace 'isRepresentable' with 'representation': a Checker method
produces the rounded representation of an untyped constant operand as
a target type.
+ Make some functions return error codes rather than errors, and factor
out the construction of the error message for invalid conversion.
This avoided some indirect code.
+ Replace implicitType with implicitTypeAndValue, and have it handle
the case of a constant basic operand, returning the rounded value.
+ Eliminate canConvertUntyped, lifting the logic to update expr types
and values to the two callers.
+ Add handling for Sum types in implicitTypeAndValue. Here, the
decision was made to depart from dev.go2go (and types2), and produce
a Sum type as output. This seemed most correct on first principles,
and tests still passed (though some logic for recording types had to
be updated to allow for Sum types).
Change-Id: Ic93901f69e6671b83b14ee2bf185a4ed767e31ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/284256
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
Joel Sing [Fri, 15 Jan 2021 17:38:50 +0000 (04:38 +1100)]
runtime: convert openbsd/amd64 locking to libc
Switch openbsd/amd64 to locking via libc, rather than performing direct
system calls.
Update #36435
Change-Id: I5e92bd70ce557b78ff385577088a9775cc468ea9
Reviewed-on: https://go-review.googlesource.com/c/go/+/270378
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Robert Griesemer [Sat, 16 Jan 2021 02:19:00 +0000 (18:19 -0800)]
[dev.typeparams] cmd/compile/internal/types2: report type of nil based on context
With this CL, the type reported for uses of the predeclared
identifier nil changes from untyped nil to the type of the
context within which nil is used, matching the behaviour of
types2 for other untyped types.
If an untyped nil value is assigned or converted to an
interface, the nil expression is given the interface type.
The predicate TypeAndValue.IsNil doesn't change in behavior,
it still reports whether the relevant expression is a (typed
or untyped) nil value.
Change-Id: Id766468f3f3f2a53e4c55e1e6cd521e459c4a94f
Reviewed-on: https://go-review.googlesource.com/c/go/+/284218
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Rob Findley [Fri, 15 Jan 2021 17:45:11 +0000 (12:45 -0500)]
[dev.typeparams] go/types: fix some merge errors in call.go
Some comments were left unresolved in the merge of call.go. Resolve them
to get tests to pass (tests to be added in a later CL).
Change-Id: Icf894593e7dd5131406c4eece8d43d4cd3170d1c
Reviewed-on: https://go-review.googlesource.com/c/go/+/284255
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
Rob Findley [Fri, 15 Jan 2021 16:30:49 +0000 (11:30 -0500)]
[dev.typeparams] go/types: don't modify Named.underlying in validType
This was fixed on dev.go2go in CL 240901, but accidentally omitted from
the merge.
Change-Id: I9020eb51dac4aa07d57c3de747d33ba84abb6386
Reviewed-on: https://go-review.googlesource.com/c/go/+/284254
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
Rob Findley [Fri, 15 Jan 2021 16:23:28 +0000 (11:23 -0500)]
[dev.typeparams] go/types: strip annotations from errors
Strip annotations from errors before emitting them. This is a partial
merge from dev.go2go: the Error.Full field is omitted for now, and
stripAnnotations is integrated with the updated error handling from
master.
Change-Id: Ia24d66b691a10d90b258b0b688d50c6b176bd629
Reviewed-on: https://go-review.googlesource.com/c/go/+/284253
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>