Russ Cox [Sat, 5 Dec 2020 05:02:46 +0000 (00:02 -0500)]
[dev.regabi] cmd/compile: reorganize ir/fmt.go
This code is a few layer of abstraction stacked up on top
of each other, and they're hard to see all at the same time
because the file is pretty mixed up. As much as I try to avoid
code rearrangement to keep history, this one is long overdue.
A followup CL will cut out some of the layers, and the diff will be
much clearer what's going on with the code ordered with
callers near callees, as it is now.
Russ Cox [Fri, 4 Dec 2020 16:37:54 +0000 (11:37 -0500)]
[dev.regabi] cmd/compile: arrange for typecheck1 to end in switch
Ending typecheck1 in the switch makes it safe for each case
to do an appropriate type assertion. The main change is dropping
the computation of "ok" and using the syntax nodes themselves
to decide what's OK.
Matthew Dempsky [Thu, 3 Dec 2020 22:00:19 +0000 (14:00 -0800)]
[dev.regabi] cmd/compile: silence errors about missing blank methods
If an interface contains a blank method, that's already an error. No
need for useless follow-up error messages about not implementing them.
Fixes #42964.
Change-Id: I5bf53a8f27d75d4c86c61588c5e2e3e95563d320
Reviewed-on: https://go-review.googlesource.com/c/go/+/275294
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Sun, 6 Dec 2020 20:14:46 +0000 (12:14 -0800)]
[dev.regabi] cmd/compile: change iexport to avoid map[ir.Node]
In the past, we had a lot of trouble with misusing *types.Sym
throughout the frontend, so I tried to push us towards always passing
around ONAMEs instead. But for constructing and writing out the symbol
indexes for the indexed export data, keying by *types.Sym is exactly
what we want.
Passes buildall w/ toolstash -cmp.
Updates #42982.
Change-Id: Idd8f1fb057d75a52a34ebc7788d9332fb49caf8d
Reviewed-on: https://go-review.googlesource.com/c/go/+/275755
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Matthew Dempsky [Sun, 6 Dec 2020 20:29:42 +0000 (12:29 -0800)]
[dev.regabi] cmd/compile: replace NodeQueue with NameQueue
Similar to the previous CL, the only two users of NodeQueue only
needed it for tracking objects, not arbitrary AST nodes. So change
it's signature to use *Name instead of Node.
This does require a tweak to the nowritebarrierrec checker, because
previously it was pushing the ODCLFUNC *Func pointers into the queue,
whereas now we push the ONAME/PFUNC *Name pointers instead. However,
it's trivial and safe to flip between them.
Also, this changes a handful of export-related code from Node to
*Name, to avoid introducing type assertions within iexport.go.
Passes buildall w/ toolstash -cmp.
Updates #42982.
Change-Id: I867f9752121509fc3da753978c6a41d5015bc0ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/275753 Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Matthew Dempsky [Sat, 5 Dec 2020 23:49:03 +0000 (15:49 -0800)]
[dev.regabi] cmd/compile: change NodeSet to NameSet
The only user of NodeSet (computing initialization dependencies) only
needs to store *Names in this structure. So change its definition to
match that need, and update the code in initorder.go accordingly.
Passes buildall w/ toolstash -cmp.
Updates #42982.
Change-Id: I181a8aaf9bc71e88f4ac009c4f381a718080e48f
Reviewed-on: https://go-review.googlesource.com/c/go/+/275752
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Matthew Dempsky [Fri, 4 Dec 2020 00:52:45 +0000 (16:52 -0800)]
[dev.regabi] cmd/compile: add SameSource, Uses, and DeclaredBy helpers
Currently, because we use the same *Name to represent both declaration
and uses of an object, it's ambiguous what "n1 == n2" means when
comparing two Node values. It can mean any of: Are these the same
syntactic element? Is n1 a use of declared variable n2? Are n1 and n2
both uses of the same declared variable?
We'd like to introduce a new IdentExpr node to replace use of Name
within the AST, but that means those three cases need to be handled
differently. The first case needs to stay "n1 == n2", but the other
cases need to become "n1.Name() == n2" and "n1.Name() == n2.Name()",
respectively. ("n1.Name() == n2.Name()" also currently works for the
second case, but eventually we'll want to get rid of the Name.Name
method.)
This CL introduces helper functions SameSource and Uses to handle
these cases. It also introduces DeclaredBy, which is another somewhat
common case that the next CL introduces uses of.
The only reason for the DeepCopyNode interface was to
allow the type syntaxes to avoid being constrained by
Left, Right etc. methods. Now those are gone, so the
general traversal methods they implement (doChildren, editChildren)
do the right thing for DeepCopy.
Russ Cox [Fri, 4 Dec 2020 04:56:16 +0000 (23:56 -0500)]
[dev.regabi] cmd/compile: avoid general traversal in deadcode
deadcode is trying to walk the statements it can find,
but it can sweep in other nodes too. Stop doing that:
only walk known statements containing statements.
Otherwise, if we put panics in expression accessors that
shouldn't be used anymore, deadcode can trip them.
deadcode would be a good candidate to rewrite using
EditChildren, but that would certainly cause toolstash
changes, since deadcode is so ad-hoc about exactly
which parts of the function it looks at. For now just
remove the general traversal and leave as is.
Russ Cox [Fri, 4 Dec 2020 02:29:23 +0000 (21:29 -0500)]
[dev.regabi] cmd/compile: implement editChildren for nodes
Put each node in charge of its EditChildren implementation.
This removes the final generic use of Left, SetLeft, Right, SetRight,
and so on in package ir.
Russ Cox [Fri, 4 Dec 2020 02:02:19 +0000 (21:02 -0500)]
[dev.regabi] cmd/compile: implement doChildren for nodes
Put each node in charge of its DoChildren implementation.
This removes a generic use of Left, Right, and so on
in func DoChildren, heading toward removing those even from
being used in package ir.
Passes buildall w/ toolstash -cmp.
Change-Id: Ibdf56f36801217cf24549e063da0078c1820a56b
Reviewed-on: https://go-review.googlesource.com/c/go/+/275375
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Russ Cox [Thu, 3 Dec 2020 23:43:18 +0000 (18:43 -0500)]
[dev.regabi] cmd/compile: implement copy for nodes
Put each node in charge of making copies of its own slices.
This removes a generic use of Body, SetBody, and so on
in func Copy, heading toward removing those even from
being used in package ir.
Russ Cox [Thu, 3 Dec 2020 19:06:41 +0000 (14:06 -0500)]
[dev.regabi] cmd/compile: use ir.EditChildren for inline rewriting
This CL rephrases the general inlining rewriter in terms of ir.EditChildren.
It is the final part of the code that was processing arbitrary nodes using
Left, SetLeft, and so on. After this CL, there should be none left except
for the implementations of DoChildren and EditChildren, which fall next.
Russ Cox [Thu, 3 Dec 2020 17:57:38 +0000 (12:57 -0500)]
[dev.regabi] cmd/compile: introduce IR visitors
This CL introduces the general visitor functionality that will replace
the Left, SetLeft, Right, SetRight, etc methods in the Node interface.
For now, the CL defines the functionality in terms of those methods,
but eventually the Nodes themselves will implement DoChildren
and EditChildren and be relieved of implementing Left, SetLeft, and so on.
The CL also updates Inspect (which moved to visit.go) and DeepCopy
to use the new functionality.
The Find helper is not used in this CL but will be used in a future one.
Russ Cox [Thu, 3 Dec 2020 03:42:24 +0000 (22:42 -0500)]
[dev.regabi] cmd/compile: adjustments to Copy and DeepCopy
DeepCopy is not called DeepSepCopy, so it should use Copy, not SepCopy.
Also, the old gc.treecopy, which became ir.DeepCopy, only copied
the Left, Right, and List fields - not Init, Rlist, Body - and I didn't
notice when I moved it over. A general utility function should of
course copy the whole node, so do that.
Finally, the semantics of Copy should not depend on whether a
particular child node is held directly in a field or in a slice,
so make Copy duplicate the slice backing arrays as well.
(Logically, those backing arrays are part of the node storage.)
Russ Cox [Thu, 3 Dec 2020 20:40:46 +0000 (15:40 -0500)]
[dev.regabi] cmd/compile: swap inlining order of if then vs else blocks
The upcoming general iterators will process nodes in
source code order, meaning that the "then" block comes
before the "else" block. But for an if node, "then" is Body
while "else" is Rlist, and the inliner processes Rlist first.
The order of processing changes the order of inlining decisions,
which can affect which functions are inlined, but in general
won't affect much. (It's not like we know that we should prefer
to inline functions in else bodies over then bodies.)
Swapping these is not safe for toolstash -cmp.
Doing it in a separate CL lets the upcoming CLs all be toolstash-safe.
Change-Id: Id16172849239b0564930d2bbff1260ad6d03d5ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/275308
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Matthew Dempsky [Thu, 3 Dec 2020 19:56:29 +0000 (11:56 -0800)]
[dev.regabi] cmd/compile: eliminate more SetOrig
This CL consolidates and cleans up fmt.go's logic for skipping past
Nodes introduced during typechecking. This allows eliminating SetOrig
on ConvExpr and Name. Also changes ConstExpr.SetOrig to a panic for
good measure.
The only remaining SetOrig uses now are for rewriting multi-value
"f(g())" calls and "return g()" statements, and type-checking
composite literals. It should be possible to eliminate both of those
as well.
Passes buildall w/ toolstash -cmp.
Change-Id: I478aea1a17dfb7a784293b930bf9081637eb2d7a
Reviewed-on: https://go-review.googlesource.com/c/go/+/275179
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Matthew Dempsky [Thu, 3 Dec 2020 05:38:20 +0000 (21:38 -0800)]
[dev.regabi] cmd/compile: store types.Field on {Selector,CallPart}Expr
It's useful to have quick access to the types.Field that a given
selector or method value expression refer to. Previously we abused Opt
for this, but couldn't do that for OCALLPART because escape analysis
uses Opt.
Now that we have more flexibility, we can simply add additional
pointer fields for this. This also allows getting rid of an unneeded
ONAME node for OCALLPART.
Passes buildall w/ toolstash -cmp.
Change-Id: I980d7bdb19abfd0b6f58a232876861b88dee1e47
Reviewed-on: https://go-review.googlesource.com/c/go/+/275034
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Matthew Dempsky [Thu, 3 Dec 2020 03:26:56 +0000 (19:26 -0800)]
[dev.regabi] cmd/compile: add ConstExpr
Currently, we represent constant-folded expressions with Name, which
is suboptimal because Name has a lot of fields to support declared
names (which are irrelevant to constant-folded expressions), while
constant expressions are fairly common.
This CL introduces a new lightweight ConstExpr type that simply wraps
an existing expression and associates it with a value.
Matthew Dempsky [Thu, 3 Dec 2020 07:55:42 +0000 (23:55 -0800)]
[dev.regabi] cmd/compile: refactoring prep for ConstExpr
The next CL adds ConstExpr, which is a more memory efficient
representation for constant expressions than Name. However, currently
a bunch of Val helper methods are defined on Name. This CL changes
them into standalone functions that work with any Node.Val
implementation.
There's also an existing standalone function named Int64Val, which
takes a Type argument to specify what type of integer is expected. So
to avoid collisions, this CL renames it to IntVal.
Passes buildall w/ toolstash -cmp.
[git-generate]
cd src/cmd/compile/internal/ir
rf 'mv Int64Val IntVal'
sed -i -E -e 's/\(n \*Name\) (CanInt64|((I|Ui)nt64|Bool|String)Val)\(/\1(n Node/' name.go
cd ../gc
rf '
ex {
import "cmd/compile/internal/ir"
var n ir.Node
n.CanInt64() -> ir.CanInt64(n)
n.Int64Val() -> ir.Int64Val(n)
n.Uint64Val() -> ir.Uint64Val(n)
n.BoolVal() -> ir.BoolVal(n)
n.StringVal() -> ir.StringVal(n)
}
'
Russ Cox [Tue, 1 Dec 2020 17:02:16 +0000 (12:02 -0500)]
[dev.regabi] cmd/compile: handle OCONVNOP better in ssa
This CL improves handling of OCONVNOP nodes during ssa generation,
so it is not toolstash safe.
An OCONVNOP wrapper is necessary for the "for" condition of
certain compiled range loops, and the boolean evaluator was
not looking through them properly, generating unnecessary
temporaries. That change saved 8k of the (13 MB) go binary.
The other changes just streamline the handling of OCONVNOP
to be more like what OSTMTEXPR will be like. They have no
effect on output size but do tweak the ssa graph a little, which
causes different register decisions and therefore different output.
Change-Id: I9e1dcd413b60944e21554c3e3f2bdc9adcee7634
Reviewed-on: https://go-review.googlesource.com/c/go/+/274598
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Russ Cox [Tue, 1 Dec 2020 04:49:25 +0000 (23:49 -0500)]
[dev.regabi] cmd/compile: use explicit block statements for init
For statements like goto that don't need an init, use an
explicit block statement instead of forcing them to have one.
There is also one call to addinit that is being replaced with
a block. That call is the source of much of my confusion
regarding init statements: walkstmt calls addinit on a statement,
whereas all the other uses of addinit are on expressions.
After this CL, they're all expressions.
Passes buildall w/ toolstash -cmp.
Change-Id: Ifdef9d318c236dc1a7567f9e9ef4a6bedd3fe81f
Reviewed-on: https://go-review.googlesource.com/c/go/+/274597
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Russ Cox [Tue, 1 Dec 2020 02:56:24 +0000 (21:56 -0500)]
[dev.regabi] cmd/compile: merge OBLOCK and OEMPTY
OEMPTY is an empty *statement*, but it confusingly
gets handled as an expression in a few places.
More confusingly, OEMPTY often has an init list,
making it not empty at all. Replace uses and analysis
of OEMPTY with OBLOCK instead.
Russ Cox [Wed, 2 Dec 2020 15:14:53 +0000 (10:14 -0500)]
cmd/go: stop tests from using network during -short
It turned out that "go get" was using the network to look up
https://github.com?go-get=1 while resolving github.com/google/go-cmp,
and that is not the fastest page to load.
Stop that lookup by adjusting the path prefixes in the vcs table.
It also turned out that "go get" was using the network to look up
https://rsc.io?go-get=1 while resolving https://rsc.io/nonexist.svn.
That's a bit more defensible maybe, since rsc.io is not a known VCS host.
But for tests we really want to avoid the network entirely, so this CL
adds a special case in repoRootFromVCSPaths that returns a hard error
for plain "rsc.io" instead of doing the web fetch.
To keep us honest in the future, I added two automatically-set env
variables TESTGONETWORK=panic and TESTGOVCS=panic.
These cause the go command to panic rather than make a network request
or invoke a VCS command.
go test -short cmd/go now passes with these checks.
This reduced the time spent in go test -short cmd/go on my
Google workstation from 154s to 30s. (Yay network firewalls.)
Change-Id: I49207fca7f901fa011765fb984dc9cec8b691f11
Reviewed-on: https://go-review.googlesource.com/c/go/+/274441
Trust: Russ Cox <rsc@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
io/ioutil was a poorly defined collection of helpers.
Proposal #40025 moved out the generic I/O helpers to io.
This CL for proposal #42026 moves the OS-specific helpers to os,
making the entire io/ioutil package deprecated.
Cuong Manh Le [Tue, 1 Dec 2020 15:23:42 +0000 (22:23 +0700)]
doc: update signal.Notify example to use buffered channel
This if follow up of CL 274332.
Updates #9399.
Change-Id: Ic6dd534dc18227a799cbb9577979f2285596b825
Reviewed-on: https://go-review.googlesource.com/c/go/+/274393
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cuong Manh Le [Tue, 1 Dec 2020 05:54:24 +0000 (12:54 +0700)]
cmd/go: fix unbuffered channel passed to signal.Notify
Unbuffered channels passed into signal.Notify can be lost
as the docs for signal.Notify caution with:
Package signal will not block sending to c: the caller must ensure
that c has sufficient buffer space to keep up with the expected signal
rate. For a channel used for notification of just one signal value,
a buffer of size 1 is sufficient.
Found by a static analyzer from Orijtech, Inc. called "sigchanyzer", but
it'll be donated to the Go project soon.
Updates #9399.
Change-Id: Ia0690e447582da028694ed65ace7b97961997b84
Reviewed-on: https://go-review.googlesource.com/c/go/+/274332
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Pseudo branch instructions BGT, BGTU, BLE, and BLEU implemented In
CL 226397 were translated inconsistently compared to other ones due
to the inversion of registers. For instance, while "BLT a, b" generates
"jump if a < b", "BLE a, b" generates "jump if b <= a."
This CL fixes the translation in the assembler and the tests.
Change-Id: Ia757be73e848734ca5b3a790e081f7c4f98c30f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/271911 Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
Matthew Dempsky [Wed, 2 Dec 2020 07:55:03 +0000 (23:55 -0800)]
[dev.regabi] cmd/compile: add ir.NewDeclNameAt
This allows directly creating an ONONAME, which is a primordial Name
before having its Op initialized. Then after an Op is assigned, we
never allow it to be reassigned.
Passes buildall w/ toolstash -cmp.
Change-Id: Ibc2f413dc68c0af6a96abfe653c25ce31b184287
Reviewed-on: https://go-review.googlesource.com/c/go/+/274620
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Matthew Dempsky [Tue, 1 Dec 2020 21:07:35 +0000 (13:07 -0800)]
[dev.regabi] cmd/compile: add Interface, Signature, and Struct constructors
This CL adds the remaining constructors needed to abstract away
construction of Types, and updates the compiler to use them
throughout. There's now just a couple uses within test cases to
remove.
While at it, I also replace the Func.Outnamed field with a simple
helper function, which reduces the size of function types somewhat.
Passes toolstash/buildall.
Change-Id: If1aa1095c98ae34b00380d0b3531bd63c10ce885
Reviewed-on: https://go-review.googlesource.com/c/go/+/274713
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Matthew Dempsky [Tue, 1 Dec 2020 21:38:26 +0000 (13:38 -0800)]
[dev.regabi] cmd/compile: tweak hash bucket type descriptor
There's no need for the bucket type to be precise. The compiler
doesn't actually generate code that references these fields; it just
needs it for size and GC bitmap calculations.
However, changing the type field does alter the runtime type
descriptor and relocations emitted by the compiler, so this change
isn't safe for toolstash.
Change-Id: Icf79d6c4326515889b13435a575d618e3bbfbcd7
Reviewed-on: https://go-review.googlesource.com/c/go/+/274712
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Ian Lance Taylor [Wed, 2 Dec 2020 02:14:40 +0000 (18:14 -0800)]
test: match gofrontend error messages
The gofrontend code doesn't distinguish semicolon and newline,
and it doesn't have special treatment for EOF.
syntax/semi6.go:9:47: error: unexpected semicolon or newline in type declaration
syntax/semi6.go:11:62: error: unexpected semicolon or newline in type declaration
Change-Id: I9996b59a4fc78ad1935e779f354ddf75c0fb44e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/274692
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Cherry Zhang [Sun, 22 Nov 2020 16:43:29 +0000 (11:43 -0500)]
cmd/link: invalidate kernel cache on darwin
Apparently, the darwin kernel may cache the code signature at
mmap. When we mmap the output buffer, it doesn't have a code
signature (as we haven't generated one). Invalidate the kernel
cache after writing the file.
See https://github.com/golang/go/issues/42684#issuecomment-731704900
for more information.
Cherry Zhang [Sun, 22 Nov 2020 02:24:57 +0000 (21:24 -0500)]
cmd/internal/buildid: update Mach-O code signature when rewriting buildid
As the code signature contains hashes of the entire file (except
the signature itself), rewriting buildid will invalidate the
signature. This CL makes it regenerate the signature when
rewriting the buildid. It only does it when the file already has
a code signature, with proper size (darwin/arm64 binaries
generated by the Go linker should have).
Cherry Zhang [Sun, 22 Nov 2020 01:11:03 +0000 (20:11 -0500)]
cmd/link: code-sign on darwin/arm64
This CL lets the linker code-sign output binaries on
darwin/arm64, as the kernel requires binaries must be signed in
order to run.
This signature will likely be invalidated when we stamp the
buildid after linking. We still do it in the linker, for
- plain "go tool link" works.
- the linker generates the LC_CODE_SIGNATURE load command with
the right size and offset, so we don't need to update it when
stamping the buildid.
Jason A. Donenfeld [Mon, 30 Nov 2020 09:41:46 +0000 (10:41 +0100)]
cmd/compile: do not assume TST and TEQ set V on arm
These replacement rules assume that TST and TEQ set V. But TST and
TEQ do not set V. This is a problem because instructions like LT are
actually checking for N!=V. But with TST and TEQ not setting V, LT
doesn't do anything meaningful. It's possible to construct trivial
miscompilations from this, such as:
package main
var x = [4]int32{-0x7fffffff, 0x7fffffff, 2, 4}
func main() {
if x[0] > x[1] {
panic("fail 1")
}
if x[2]&x[3] < 0 {
panic("fail 2") // Fails here
}
}
That first comparison sets V, via the CMP that subtracts the values
causing the overflow. Then the second comparison operation thinks that
it uses the result of TST, when it actually uses the V from CMP.
Before this fix:
TST R0, R1
BLT loc_6C164
After this fix:
TST R0, R1
BMI loc_6C164
The BMI instruction checks the N flag, which TST sets. This commit
fixes the issue by using [LG][TE]noov instead of vanilla [LG][TE], and
also adds a test case for the direct issue.
Fixes #42876.
Change-Id: I13c62c88d18574247ad002b671b38d2d0b0fc6fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/274026
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Reason for revert: Broke many tests inside Google which implies many
tests were broken outside of Google as well. The tests may be brittle
but still would require work to change and it's not clear it's worth
the benefit.
Updates #36221
Fixes #42675
Change-Id: Id3a14eb37e7119f5abe50e80dfbf120fdc44db72
Reviewed-on: https://go-review.googlesource.com/c/go/+/273747
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Cherry Zhang [Sat, 21 Nov 2020 22:43:16 +0000 (17:43 -0500)]
cmd/internal/buildid: exclude Mach-O code signature in hash calculation
The code signature contains hashes of the entire file (except the
signature itself), including the buildid. Therefore, the buildid
cannot depend on the signature. Otherwise updating buildid will
invalidate the signature, and vice versa. As we cannot change the
code-signing algorithm, we can only change buildid calculation.
This CL changes the buildid calculation to exclude the Mach-O
code signature. So updating code signature after stamping the
buildid will not invalidate the buildid.
Cherry Zhang [Sun, 22 Nov 2020 02:33:18 +0000 (21:33 -0500)]
cmd/internal/codesign: new package
On macOS/ARM64, the kernel requires that binaries must have a
valid code signature to run. The C toolchain code-signs the
binary at link time. We do the same.
It is more subtle for Go because we stamp the buildid after
linking. As the signature contains hashes of the entire file
(except the signature itself), we must (re)generate the signature
after stamping the buildid.
This CL adds a new codesign package, which provides
functionality to generate the code signature. It is a separate
internal package so it can be used both in the linker and by the
go command. The next CLs will add code-signing to the linker and
the go command.
Jay Conrod [Tue, 1 Dec 2020 18:51:17 +0000 (13:51 -0500)]
cmd/go: don't print deprecation notice for 'go get exe'
It's difficult for module authors to provide installation instructions
that work in both Go 1.15 and 1.16. We'll wait until 1.17 to print a
deprecation warning for installing executables with 'go get'.
Fixes #42885
Change-Id: I835b447e83e760f48fd664e8a117749e0cb59f83
Reviewed-on: https://go-review.googlesource.com/c/go/+/274552
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Anmol Sethi [Sat, 24 Oct 2020 00:40:41 +0000 (00:40 +0000)]
net/http: allow upgrading non keepalive connections
If one was using http.Transport with DisableKeepAlives and trying
to upgrade a connection against net/http's Server, the Server
would not allow a "Connection: Upgrade" header to be written
and instead override it to "Connection: Close" which would
break the handshake.
This change ensures net/http's Server does not override the
connection header for successful protocol switch responses.
Matthew Dempsky [Tue, 1 Dec 2020 11:52:20 +0000 (03:52 -0800)]
[dev.regabi] cmd/compile: only save ONAMEs on Curfn.Dcl
There's not really any use to tracking function-scoped constants and
types on Curfn.Dcl, and there's sloppy code that assumes all of the
declarations are variables (e.g., cmpstackvarlt).
Matthew Dempsky [Tue, 1 Dec 2020 10:58:41 +0000 (02:58 -0800)]
[dev.regabi] cmd/compile: add NewBasic and cleanup universe
This CL introduces types.NewBasic, for creating the predeclared
universal types, and reorganizes how the universe is initialized so
that all predeclared types are uniformly constructed.
There are now a bunch of Type fields that are no longer assigned
outside of the package, so this CL also introduces some new accessor
methods that a subsequent CL will mechanically introduce uses of.
Matthew Dempsky [Tue, 1 Dec 2020 09:42:47 +0000 (01:42 -0800)]
[dev.regabi] cmd/compile: add NewNamed
The start of abstracting away Type fields. This adds a new constructor
for named types, styled after go/types.NewNamed. Along with helper
methods for SetNod and Pos, this allows hiding Nod.
Matthew Dempsky [Tue, 1 Dec 2020 09:01:59 +0000 (01:01 -0800)]
[dev.regabi] cmd/compile: cleanup type-checking of defined types
The code for type-checking defined types was scattered between
typecheckdef, typecheckdeftype, and setUnderlying. There was redundant
work between them, and setUnderlying also needed to redo a lot of work
because of its brute-force solution of just copying all Type fields.
This CL reorders things so as many of the defined type's fields are
set in advance (in typecheckdeftype), and then setUnderlying only
copies over the details actually needed from the underlying type.
Incidentally, this evidently improves our error handling for an
existing test case, by allowing us to report an additional error.
Passes toolstash/buildall.
Change-Id: Id59a24341e7e960edd1f7366c3e2356da91b9fe7
Reviewed-on: https://go-review.googlesource.com/c/go/+/274432
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Matthew Dempsky [Sun, 29 Nov 2020 22:06:17 +0000 (14:06 -0800)]
[dev.regabi] cmd/compile: process //go:linknames after declarations
Allows emitting errors about ineffectual //go:linkname directives.
In particular, this exposed: a typo in os2_aix.go; redundant (but
harmless) directives for libc_pipe in both os3_solaris.go and
syscall2_solaris.go; and a bunch of useless //go:linkname directives
in macOS wrapper code.
However, because there's also ineffectual directives in the vendored
macOS code from x/sys, that can't be an error just yet. So instead we
print a warning (including a heads up that it will be promoted to an
error in Go 1.17) to prevent backsliding while we fix and re-vendor
that code.
Passes toolstash-check.
Change-Id: I59badeab5df0d8b3abfd14c6066e9bb00e840f73
Reviewed-on: https://go-review.googlesource.com/c/go/+/273986
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Russ Cox [Tue, 1 Dec 2020 02:18:48 +0000 (21:18 -0500)]
[dev.regabi] cmd/compile: use ir.Copy instead of direct use of RawCopy
The ONIL export bug happened because the logic about
maintaining an “implicit” orig pointer in the comments
around ir.Orig only applies to Copy and SepCopy, not to
direct use of RawCopy. I'd forgotten those could exist.
The sole direct use of RawCopy was for the OLITERAL/ONIL case.
The ONIL is now provably indistinguishable from Copy, since
NilExpr does not have an explicit Orig field, so for NilExpr
RawCopy and Copy are the same.
The OLITERAL is not, but we can reconstruct the effect
with Copy+SetOrig to be explicit that we need the orig link.
The next CL will unexport RawCopy.
Also fix a typo in MapType doc comment.
Passes buildall w/ toolstash -cmp.
Change-Id: I876a85ff188e6d1cd4c0dfa385be32482e0de0d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/274292
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Matthew Dempsky [Tue, 1 Dec 2020 04:34:25 +0000 (20:34 -0800)]
[dev.regabi] cmd/compile: improve findTypeLoop
When checking if a defined type is part of a type loop, we can
short-circuit if it was defined in another package. We can assume any
package we import already successfully compiled, so any types it
contains cannot be part of a type loop.
This also allows us to simplify the logic for recursing into the type
used in the type declaration, because any defined type from this
package will have a properly setup node.
Change-Id: Ic024814d95533afd9e59f2103c8ddb22bd87e900
Reviewed-on: https://go-review.googlesource.com/c/go/+/274294
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Joel Sing [Thu, 26 Nov 2020 15:06:08 +0000 (02:06 +1100)]
cmd/link/internal/amd64: always generate R_X86_64_PLT32 for SDYNIMPORT calls
Currently, in the non-DynlinkingGo case with external linking, we generate a
R_X86_64_GOTPCREL relocation for the imported symbol. This results in the
external linker turning this into a R_X86_64_GLOB_DAT relocation, rather
than a R_X86_64_JUMP_SLOT. Always generate R_X86_64_PLT32 for SDYNIMPORT
calls so that these calls work correctly.
Update #36435
Fixes #42671
Change-Id: I8a28884b7853cb4135053ed817bedc919482f4ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/270377
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Cherry Zhang <cherryyz@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Matthew Dempsky [Mon, 30 Nov 2020 08:01:26 +0000 (00:01 -0800)]
[dev.regabi] cmd/compile: remove types.InitSyms
It's not types's responsibility to understand how package
initialization is implemented. Instead, have gc keep track of the
order that packages were imported, and then look for inittask
declarations.
Also, use resolve to force importing of the inittask's export data, so
that we can get the appropriate linker symbol index. (This is also why
this CL doesn't satisfy "toolstash -cmp".)
Change-Id: I5b706497d4a8d1c4439178863b4a8dba4da0f5a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/274006
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Russ Cox [Mon, 30 Nov 2020 19:16:39 +0000 (14:16 -0500)]
[dev.regabi] cmd/compile: clean up ONEW node
The list is no longer needed and can be deleted.
Doing so reduces the inlining cost of any function containing
an explicit call to new by 1 point, so this change is not
toolstash -cmp safe.
Russ Cox [Mon, 30 Nov 2020 02:11:07 +0000 (21:11 -0500)]
[dev.regabi] cmd/compile: clean up in preparation for expression Nodes
Using expression nodes restricts the set of valid SetOp operations,
because you can't SetOp across representation. Rewrite various code
to avoid crossing those as-yet-unintroduced boundaries.
This also includes choosing a single representation for any given Op.
For example, OCLOSE starts out as an OCALL, so it starts with a List
of one node and then moves that node to Left. That's no good with
real data structures, so the code picks a single canonical implementation
and prepares it during the conversion from one Op to the next.
In this case, the conversion of an OCALL to an OCLOSE now creates
a new node with Left initialized from the start. This pattern repeats.
Russ Cox [Sat, 28 Nov 2020 20:28:18 +0000 (15:28 -0500)]
[dev.regabi] cmd/compile: clean up in preparation for statement Nodes
Using statement nodes restricts the set of valid SetOp operations,
because you can't SetOp across representation. Rewrite various
code to avoid crossing those as-yet-unintroduced boundaries.
In particular, code like
x, y := v.(T)
x, y := f()
x, y := m[k]
x, y := <-c
starts out with Op = OAS2, and then it turns into a specific Op
OAS2DOTTYPE, OAS2FUNC, OAS2MAPR, OAS2RECV, and then
later in walk is lowered to an OAS2 again.
In the middle, the specific forms move the right-hand side from
n.Rlist().First() to n.Right(), and then the conversion to OAS2 moves
it back. This is unnecessary and makes it hard for these all to
share an underlying Node implementation.
This CL changes these specific forms to leave the right-hand side
in n.Rlist().First().
Similarly, OSELRECV2 is really just a temporary form of OAS2.
This CL changes it to use same fields too.
Finally, this CL fixes the printing of OAS2 nodes in ir/fmt.go,
which formerly printed n.Right() instead of n.Rlist().
This results in a (correct!) update to cmd/compile/internal/logopt's
expected output: ~R0 = <N> becomes ~R0 = &y.b.
Cherry Zhang [Fri, 27 Nov 2020 05:42:41 +0000 (00:42 -0500)]
runtime: mlock signal stack on macOS/ARM64
Apparently, the macOS ARM64 kernel has a bug where when a signal
arrives and the signal stack is not currently faulted in, it may
kill the program with a SIGILL. Work around it by mlock the
signal stacks.
Matthew Dempsky [Mon, 30 Nov 2020 21:50:05 +0000 (13:50 -0800)]
[dev.regabi] cmd/compile: simplify export data representation of nil
The handling of ONIL and Orig has been a mess for a while, and dates
back to how fmt.go used to print out typed nils. That hasn't applied
for a while, but we've kept dragging it along to appease toolstash
with the intention of someday finally removing it.
Today is that day.
Change-Id: I9a441628e53068ab1993cd2b67b977574d8117b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/274212
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Jay Conrod [Mon, 30 Nov 2020 20:46:33 +0000 (15:46 -0500)]
cmd/go: fix infinite loop in modload.keepSums
Fixes #42891
Change-Id: I0cce4204a1c4959b896188a2ab3719c0507f95e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/274172
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
Andy Pan [Sat, 21 Nov 2020 06:48:26 +0000 (14:48 +0800)]
net: add note about disabling loopback in ListenMulticastUDP()
Fixes #41752
Change-Id: I83520d2303e5fd2e5f6329f092b40e73c13771a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/271908 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Ian Lance Taylor [Sun, 29 Nov 2020 03:10:57 +0000 (19:10 -0800)]
test: update gofrontend expected errors
This matches the error messages after CL 273890.
syntax/semi4.go:11:9: error: unexpected semicolon or newline, expecting ‘{’ after for clause
syntax/semi4.go:10:13: error: reference to undefined name ‘x’
syntax/semi4.go:12:17: error: reference to undefined name ‘z’
Change-Id: Ic88ff6e27d50bf70f5b2114383b84c42c0682f39
Reviewed-on: https://go-review.googlesource.com/c/go/+/273891
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>