cmd/internal/obj/ppc64: Fix ADUFFxxxx generation on aix/ppc64
ADUFFCOPY and ADUFFZERO instructions weren't handled by rewriteToUseTOC.
These instructions are considered as a simple branch except with -dynlink
where they become an indirect call.
text/template/parse: use strings.Builder in String methods
As mentioned in godoc, strings.Builder is more efficient for
concatenating and building strings.
Running a simple bench test on VariableNode.String() gives:
benchmark old ns/op new ns/op delta
BenchmarkParseLarge-8 2567683124453285 -4.77%
BenchmarkVariableString-8 296 115 -61.15%
benchmark old allocs new allocs delta
BenchmarkVariableString-8 8 3 -62.50%
benchmark old bytes new bytes delta
BenchmarkVariableString-8 112 72 -35.71%
Shenghou Ma [Mon, 30 Sep 2019 13:44:37 +0000 (09:44 -0400)]
runtime: fix darwin syscall performance regression
While understanding why syscall.Read is 2x slower on darwin/amd64, I found
out that, contrary to popular belief, the slowdown is not due to the migration
to use libSystem.dylib instead of direct SYSCALLs, i.e., CL 141639 (and #17490),
but due to a subtle change introduced in CL 141639.
Previously, syscall.Read used syscall.Syscall(SYS_READ), whose preamble called
runtime.entersyscall, but after CL 141639, syscall.Read changes to call
runtime.syscall_syscall instead, which in turn calls runtime.entersyscallblock
instead of runtime.entersyscall. And the entire 2x slow down can be attributed
to this change.
I think this is unnecessary as even though syscalls like Read might block, it
does not always block, so there is no need to handoff P proactively for each
Read. Additionally, we have been fine with not handing off P for each Read
prior to Go 1.12, so we probably don't need to change it. This changes restores
the pre-Go 1.12 behavior, where syscall preamble uses runtime.entersyscall,
and we rely on sysmon to take P back from g blocked in syscalls.
Change-Id: If76e97b5a7040cf1c10380a567c4f5baec3121ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/197938
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
John Papandriopoulos [Sun, 29 Sep 2019 23:59:56 +0000 (16:59 -0700)]
cmd/link: load symbols from .syso in external link mode
Fix linking with a package having a .syso file in external link mode,
that would otherwise cause an error before executing the external
linker because it can't find symbols that are exported in the said
.syso file.
Fixes #33139
Change-Id: Id3ee737fba1c6f1e37910593dfedf9c84486d398
Reviewed-on: https://go-review.googlesource.com/c/go/+/186417 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Rob Pike [Mon, 30 Sep 2019 23:54:30 +0000 (09:54 +1000)]
text/template: further simplify building the vars list
Followup to https://golang.org/cl/197997
If you know the number of elements, you don't need append at all.
Either use append to grow, or allocate and index. Here we choose
number 2.
Change-Id: Ic58637231789640ff7b293ece04a95a8de7ccf8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/198097 Reviewed-by: Ian Lance Taylor <iant@golang.org>
An Xiao [Mon, 30 Sep 2019 21:46:32 +0000 (21:46 +0000)]
net: update quotation marks in comment
This change updates the use of quotation marks by replacing `ones' with 'ones'.
Quotation like `this' should not be used any more according to
https://www.cl.cam.ac.uk/~mgk25/ucs/quotes.html
Richard Musiol [Mon, 23 Sep 2019 22:48:39 +0000 (00:48 +0200)]
cmd/link: add producer section to wasm binaries
This change adds an optional "producer" section that reports the source
language and compiler version. See
https://github.com/WebAssembly/tool-conventions/blob/master/ProducersSection.md.
It also removes the now redundant "go.version" section.
Alberto Donizetti [Sat, 28 Sep 2019 18:29:11 +0000 (20:29 +0200)]
doc: remove double mention of custom workspace
In the GOPATH section of the 'How To Write Go Code' document, it is
mentioned two times in the span of a few lines that one can set GOPATH
to a custom workspace path. The two paragraphs say basically the same
thing, and they both link to golang.org/wiki/SettingGOPATH, so I'm
quite sure the duplication is not intentional.
This change deletes the second occurrence.
Change-Id: I16f8bb8657041a23ed272eacf9adbc5637e8e34a
Reviewed-on: https://go-review.googlesource.com/c/go/+/197839 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Alberto Donizetti [Sat, 28 Sep 2019 12:33:10 +0000 (14:33 +0200)]
cmd/compile: use %v for Node formatting
CL 197817 replaced a use of n.Rlist with n.Right in a Fprintf call,
but it left the corresponding format as %.v, which broke the
TestFormats test on the longtest builder.
Since with n.Right is custom to use %v (and not %.v), replace the
format with %v.
Fixes the longtest builder.
Change-Id: Icf5bf820a936c51e633c25ada1a71a1ffb6d28c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/197837
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
David Crawshaw [Tue, 2 Jan 2018 17:43:41 +0000 (09:43 -0800)]
reflect: let StructOf define unexported fields
This was missing from the original StructOf CL because I couldn't
think of a use for it. Now I can: even with types used entirely
by reflect, unexported fields can be set using UnsafeAddr.
Change-Id: I5e7e3d81d16e8817cdd69d85796ce33930ef523b
Reviewed-on: https://go-review.googlesource.com/c/go/+/85661
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd: update x/tools version to enforce only one %w
As mentioned in https://golang.org/issue/34062#issuecomment-529692313
src/cmd refers to older version of golang.org/x/tools.
Hence, not checking if multiple errors are used in the same fmt.Errorf.
Updating golang.org/x/tools version to latest in src/cmd.
Fixes #34062
Change-Id: I358dec2c3d3af2b19add766b8488b919109b81d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/196843
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Daniel Martí [Mon, 23 Sep 2019 22:10:25 +0000 (23:10 +0100)]
cmd/compile: minor simplifications in rulegen
First, be consistent about declaring typ as &b.Func.Config.Types and
not &config.Types. Not particularly better, and it barely changes the
output, but we're more consistent now.
Second, remove a bit of duplication when handling the typ, auxint, and
aux variables.
Third and last, remove a stray canFail assignment; we ended up setting
that in add, not breakf, so it's not necessary to set it manually if we
don't use breakf.
Updates #33644.
Change-Id: I75999cb223a201969266fbfeae043599fa27fac5
Reviewed-on: https://go-review.googlesource.com/c/go/+/196803
Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Matthew Dempsky [Mon, 23 Sep 2019 21:25:22 +0000 (14:25 -0700)]
cmd/compile: add column details to export data
This CL updates the export data format to include column details when
writing out position data. cmd/compile is updated to generate and make
use of the new details, but go/internal/gcimporter only knows how to
read the data. It doesn't yet actually make use of it.
Experimentally across a wide range of packages, this increases export
data size by around 4%. However, it has no impact on binary size.
(Notably, it actually shrinks k8s.io/kubernetes/cmd/kubelet's binary
size by 24kB, but it's unclear to me why at this time.)
Updates #28259.
Change-Id: I351fb340839df8d3adced49b3757c4537fb91b3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/196963
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Justin Nuß [Sun, 26 Nov 2017 10:05:53 +0000 (11:05 +0100)]
html/template: document handling of namespaced and data- attributes
Attributes with a namespace or a data- prefix are handled as if they
had no namespace/data- prefix.
There is also a special case, where attributes with a "xmlns" namespace
are always treated as containing URLs.
This could surprise users of the package, since this behaviour was not
documented anywhere, so this change adds some documentation for all
three cases.
Fixes #12648
Change-Id: If57a2ec49fec91a330fc04795726e8cffa9b75c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/79895
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
Than McIntosh [Thu, 26 Sep 2019 14:01:42 +0000 (10:01 -0400)]
cmd/link: remove reading/processing of function Autom records
Remove linker reading and processing of automs (no longer needed, now
that the compiler is emitting R_USETYPE relocations on functions). So
as to avoid changing the object file format, the object still contains
a count of automs, but this count is required to be zero.
Updates #34554.
Change-Id: I10230e191057c5c5705541eeb06f747d5f73c42d
Reviewed-on: https://go-review.googlesource.com/c/go/+/197500 Reviewed-by: Jeremy Faller <jeremy@golang.org>
Than McIntosh [Thu, 26 Sep 2019 13:53:37 +0000 (09:53 -0400)]
cmd/compile: don't emit autom's into object file
Don't write Autom records when writing a function to the object file;
we no longer need them in the linker for DWARF processing. So as to
keep the object file format unchanged, write out a zero-length list of
automs to the object, as opposed to removing all references.
Updates #34554.
Change-Id: I42a1d67207ea7114ae4f3a315cf37effba57f190
Reviewed-on: https://go-review.googlesource.com/c/go/+/197499 Reviewed-by: Jeremy Faller <jeremy@golang.org>
Than McIntosh [Thu, 26 Sep 2019 12:42:48 +0000 (08:42 -0400)]
cmd/link: create DWARF types for autos based R_USETYPE relocs
Switch the linker over to use dummy R_USETYPE relocations on DWARF
subprogram DIEs as a means of insuring that DWARF types are created
for types of autotmp values used in live functions.
This change is part of a series intended to clean up handling of
autotmp types and remove use of autom's in the compiler and linker.
Updates #34554.
Change-Id: Ic74da6bd723ab7e4d8a16ad46e23228650d4b525
Reviewed-on: https://go-review.googlesource.com/c/go/+/197498
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jeremy Faller <jeremy@golang.org>
Than McIntosh [Thu, 26 Sep 2019 12:38:33 +0000 (08:38 -0400)]
cmd/compile: add R_USETYPE relocs to func syms for autom types
During DWARF processing, keep track of the go type symbols for types
directly or indirectly referenced by auto variables in a function,
and add a set of dummy R_USETYPE relocations to the function's DWARF
subprogram DIE symbol.
This change is not useful on its own, but is part of a series of
changes intended to clean up handling of autom's in the compiler
and linker.
Updates #34554.
Change-Id: I974afa9b7092aa5dba808f74e00aa931249d6fe9
Reviewed-on: https://go-review.googlesource.com/c/go/+/197497
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jeremy Faller <jeremy@golang.org>
Matthew Dempsky [Thu, 26 Sep 2019 23:32:34 +0000 (16:32 -0700)]
cmd/compile: apply constant folding to ORUNESTR
ORUNESTR represents the special case of integer->string conversion. If
the integer is a constant, then the string is a constant too, so
evconst needs to perform constant folding here.
Passes toolstash-check.
Fixes #34563.
Change-Id: Ieab3d76794d8ce570106b6b707a4bcd725d156e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/197677
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Giovanni Bajo [Thu, 19 Sep 2019 20:39:37 +0000 (22:39 +0200)]
cmd/compile: in poset, move all constants to the first DAG
In poset, all constants are always related to each other, so they
are part of the same DAG. Currently, it can be any of the DAGs in
the forest. Since we're about to start visiting that DAG for the
task of calculating bounds, make sure that it's conventionally
always the first, so that we don't need to search for it.
Change-Id: Ia7ca312b52336b4731b070d45cf0d768a0d6aeeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/196599 Reviewed-by: David Chase <drchase@google.com>
Also added a test to ensure that any interactions
between TimeoutHandler and Flusher result in the
correct status code and body, but also that we don't
get superfluous logs from stray writes as was seen
in the bug report.
Matthew Dempsky [Thu, 26 Sep 2019 19:37:36 +0000 (12:37 -0700)]
cmd/compile: remove DDD array types
Currently we handle [...]T array literals by treating [...]T as
special "DDD array" types. However, these array literals are just
composite literal syntax, not a distinct Go type. Moreover,
representing them as Go types contributes to complexity in a number of
unrelated bits of code.
This CL changes OCOMPLIT typechecking to look for the [...]T syntax
and handle it specially, so we can remove DDD arrays.
Passes toolstash-check.
Change-Id: Ibbf701eac4caa7a321e2d10e256658fdfaa8a160
Reviewed-on: https://go-review.googlesource.com/c/go/+/197604
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Thu, 26 Sep 2019 19:29:09 +0000 (12:29 -0700)]
cmd/compile: extract typecheckarraylit function
Typechecking slice literals, array literals, and array literals using
"..." notation all use very similar logic, but tie into the logic for
checking the OCOMPLIT node in slightly different ways.
By refactoring this function out into a separate helper, it makes it
easier to separate slice and array literals, and the subsequent CL
will further separate array literals that do use "..." notation from
those that do not.
Passes toolstash-check.
Change-Id: I4c572e0d9d08bcc86b5c224bd6f9e1c498726c19
Reviewed-on: https://go-review.googlesource.com/c/go/+/197603
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Giovanni Bajo [Mon, 16 Sep 2019 08:25:48 +0000 (10:25 +0200)]
cmd/compile: detect indvars that are bound by other indvars
prove wasn't able to detect induction variables that was bound
by another inducation variable. This happened because an indvar
is a Phi, and thus in case of a dependency, the loop bounding
condition looked as Phi < Phi. This triggered an existing
codepath that checked whether the upper bound was a Phi to
detect loop conditions written in reversed order respect to the
idiomatic way (eg: for i:=0; len(n)>i; i++).
To fix this, we call the indvar pattern matching on both operands
of the loop condition, so that the first operand that matches
will be treated as the indvar.
Updates #24660 (removes a boundcheck from Fannkuch)
Change-Id: Iade83d8deb54f14277ed3f2e37b190e1ed173d11
Reviewed-on: https://go-review.googlesource.com/c/go/+/195220 Reviewed-by: David Chase <drchase@google.com>
Giovanni Bajo [Mon, 16 Sep 2019 08:23:54 +0000 (10:23 +0200)]
cmd/compile: refactor some code in loopbce.go
This CL extracts the logic for pattern-matching an induction
variable into a separate function, in preparation for next CL
where we would need to call it multiple times.
No functional changes, passes toolstash -cmp.
Change-Id: Ic52391e6c1b2e72bae32a0f3f65dfea321caaf4f
Reviewed-on: https://go-review.googlesource.com/c/go/+/195737 Reviewed-by: David Chase <drchase@google.com>
Matthew Dempsky [Wed, 25 Sep 2019 07:21:23 +0000 (00:21 -0700)]
cmd/compile: simplify OPTRLIT handling
Previously, we would recognize &(T{...}) expressions during type
checking, rewrite them into (*T){...}, and then do a lot of extra work
to make sure the user doesn't write (*T){...} themselves and
resynthesizing the OPTRLIT later on.
This CL simply handles &T{...} directly in the straight forward
manner, by changing OADDR directly to OPTRLIT when appropriate.
While here, match go/types's invalid composite literal type error
message.
Passes toolstash-check.
Change-Id: I902b14c7e2cd9fa93e6915dd58272d2352ba38f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/197120
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Joel Sing [Tue, 24 Sep 2019 17:42:45 +0000 (03:42 +1000)]
cmd/internal/obj/riscv: require memory targets for load and store instructions
This allows for `LD 4(X5), X6' rather than `LD $4, X5, X6'. Similar for other
load and store instructions. It is worth noting that none of these are likely
to be used directly once the MOV pseudo-instructions are implemented.
net/textproto: don't normalize headers with spaces before the colon
RFC 7230 is clear about headers with a space before the colon, like
X-Answer : 42
being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.
On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.
For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.
This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn
net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.
context: use fewer goroutines in WithCancel/WithTimeout
If the parent context passed to WithCancel or WithTimeout
is a known context implementation (one created by this package),
we attach the child to the parent by editing data structures directly;
otherwise, for unknown parent implementations, we make a
goroutine that watches for the parent to finish and propagates
the cancellation.
A common problem with this scheme, before this CL, is that
users who write custom context implementations to manage
their value sets cause WithCancel/WithTimeout to start
goroutines that would have not been started before.
This CL changes the way we map a parent context back to the
underlying data structure. Instead of walking up through
known context implementations to reach the *cancelCtx,
we look up parent.Value(&cancelCtxKey) to return the
innermost *cancelCtx, which we use if it matches parent.Done().
This way, a custom context implementation wrapping a
*cancelCtx but not changing Done-ness (and not refusing
to return wrapped keys) will not require a goroutine anymore
in WithCancel/WithTimeout.
Michael Anthony Knyszek [Thu, 5 Sep 2019 16:34:00 +0000 (16:34 +0000)]
runtime: fix lock acquire cycles related to scavenge.lock
There are currently two edges in the lock cycle graph caused by
scavenge.lock: with sched.lock and mheap_.lock. These edges appear
because of the call to ready() and stack growths respectively.
Furthermore, there's already an invariant in the code wherein
mheap_.lock must be acquired before scavenge.lock, hence the cycle.
The fix to this is to bring scavenge.lock higher in the lock cycle
graph, such that sched.lock and mheap_.lock are only acquired once
scavenge.lock is already held.
To faciliate this change, we move scavenger waking outside of
gcSetTriggerRatio such that it doesn't have to happen with the heap
locked. Furthermore, we check scavenge generation numbers with the heap
locked by using gopark instead of goparkunlock, and specify a function
which aborts the park should there be any skew in generation count.
Fixes #34047.
Change-Id: I3519119214bac66375e2b1262b36ce376c820d12
Reviewed-on: https://go-review.googlesource.com/c/go/+/191977
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
cmd/go/internal/robustio: extend filesystem workarounds to darwin platforms
The macOS filesystem seems to have gotten significantly flakier as of
macOS 10.14, so this causes frequently flakes in the 10.14 builders.
We have no reason to believe that it will be fixed any time soon, so
rather than trying to detect the specific macOS version, we'll apply
the same workarounds that we use on Windows: classifying (and
retrying) the errors known to indicate flakiness and relaxing the
success criteria for renameio.TestConcurrentReadsAndWrites.
Fixes #33041
Change-Id: I74d8c15677951d7a0df0d4ebf6ea03e43eebddf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/197517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Jeremy Faller [Thu, 29 Aug 2019 20:35:50 +0000 (16:35 -0400)]
cmd/compile: remove isStmt symbol from FuncInfo
As promised in CL 188238, removing the obsolete symbol.
Here are the latest stats. This is baselined at "e53edafb66" with only
these changes applied, run on magna.cam. The linker looks straight
better (in memory and speed).
There is still a change I'm working on walking the progs to generate the
debug_lines data in the compiler. That will likely result in a compiler
speedup.
Michael Munday [Fri, 13 Sep 2019 12:28:49 +0000 (13:28 +0100)]
cmd/compile: use numeric condition code masks on s390x
Prior to this CL conditional branches on s390x always used an
extended mnemonic such as BNE, BLT and so on to represent branch
instructions with different condition code masks. This CL adds
support for numeric condition code masks to the s390x SSA backend
so that we can encode the condition under which a Block's
successor is chosen as a field in that Block rather than in its
type.
This change will be useful as we come to add support for combined
compare-and-branch instructions. Rather than trying to add extended
mnemonics for every possible combination of mask and compare-and-
branch instruction we can instead use a single mnemonic for each
instruction.
Change-Id: Idb7458f187b50906877d683695c291dff5279553
Reviewed-on: https://go-review.googlesource.com/c/go/+/197178 Reviewed-by: Keith Randall <khr@golang.org>
internal/poll: make SendFile work with large files on Windows
CL 192518 was a minimal simplification to get sendfile
on Windows to work with chunked files, but as I had mentioned,
I would add even more improvements.
This CL improves it by:
* If the reader is not an *io.LimitedReader, since the underlying
reader is anyways an *os.File, we fallback and stat that
file to determine the file size and then also invoke the chunked
sendFile on the underlying reader. This issue existed even
before the prior CL.
* Extracting the chunked TransmitFile logic and moving it directly
into internal/poll.SendFile.
Thus if the callers of net.sendFile don't use *io.LimitedReader,
but have a huge file (>2GiB), we can still invoke the chunked
internal/poll.SendFile on it directly.
The test case is not included in this patch as it requires
creating a 3GiB file, but that if anyone wants to view it, they
can find it at
https://go-review.googlesource.com/c/go/+/194218/13/src/net/sendfile_windows_test.go
Dan Scales [Tue, 24 Sep 2019 00:46:38 +0000 (17:46 -0700)]
misc, runtime, test: extra tests and benchmarks for defer
Add a bunch of extra tests and benchmarks for defer, in preparation for new
low-cost (open-coded) implementation of defers (see #34481),
- New file defer_test.go that tests a bunch more unusual defer scenarios,
including things that might have problems for open-coded defers.
- Additions to callers_test.go actually verifying what the stack trace looks like
for various panic or panic-recover scenarios.
- Additions to crash_test.go testing several more crash scenarios involving
recursive panics.
- New benchmark in runtime_test.go measuring speed of panic-recover
- New CGo benchmark in cgo_test.go calling from Go to C back to Go that
shows defer overhead
Updates #34481
Change-Id: I423523f3e05fc0229d4277dd00073289a5526188
Reviewed-on: https://go-review.googlesource.com/c/go/+/197017
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Michael Munday [Thu, 15 Aug 2019 19:43:46 +0000 (20:43 +0100)]
cmd/asm: add masked branch and conditional load instructions to s390x
The branch-relative-on-condition (BRC) instruction allows us to use
an immediate to specify under what conditions the branch is taken.
For example, `BRC $7, L1` is equivalent to `BNE L1`. It is sometimes
useful to specify branches in this way when either we don't have
an extended mnemonic for a particular mask value or we want to
generate the condition code mask programmatically.
The new load-on-condition (LOCR and LOCGR) and compare-and-branch
(CRJ, CGRJ, CLRJ, CLGRJ, CIJ, CGIJ, CLIJ and CLGIJ) instructions
provide the same flexibility for conditional loads and combined
compare and branch instructions.
Michael Anthony Knyszek [Tue, 25 Jun 2019 19:06:57 +0000 (19:06 +0000)]
runtime: scavenge on growth instead of inline with allocation
Inline scavenging causes significant performance regressions in tail
latency for k8s and has relatively little benefit for RSS footprint.
We disabled inline scavenging in Go 1.12.5 (CL 174102) as well, but
we thought other changes in Go 1.13 had mitigated the issues with
inline scavenging. Apparently we were wrong.
This CL switches back to only doing foreground scavenging on heap
growth, rather than doing it when allocation tries to allocate from
scavenged space.
Fixes #32828.
Change-Id: I1f5df44046091f0b4f89fec73c2cde98bf9448cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/183857
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Austin Clements [Mon, 12 Aug 2019 18:54:28 +0000 (14:54 -0400)]
runtime: grow the heap incrementally
Currently, we map and grow the heap a whole arena (64MB) at a time.
Unfortunately, in order to fix #32828, we need to switch from
scavenging inline with allocation back to scavenging on heap growth,
but heap-growth scavenging happens in large jumps because we grow the
heap in large jumps.
In order to prepare for better heap-growth scavenging, this CL
separates mapping more space for the heap from actually "growing" it
(tracking the new space with spans). Instead, growing the heap keeps
track of the "current arena" it's growing into. It track that with new
spans as needed, and only maps more arena space when the current arena
is inadequate. The effect to the user is the same, but this will let
us scavenge on much smaller increments of heap growth.
There are two slightly subtleties to this change:
1. If an allocation requires mapping a new arena and that new arena
isn't contiguous with the current arena, we don't want to lose the
unused space in the current arena, so we have to immediately track
that with a span.
2. The mapped space must be accounted as released and idle, even
though it isn't actually tracked in a span.
For #32828, since this makes heap-growth scavenging far more
effective, especially at small heap sizes. For example, this change is
necessary for TestPhysicalMemoryUtilization to pass once we remove
inline scavenging.
Change-Id: I300e74a0534062467e4ce91cdc3508e5ef9aa73a
Reviewed-on: https://go-review.googlesource.com/c/go/+/189957
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Tue, 3 Sep 2019 19:54:32 +0000 (19:54 +0000)]
runtime: redefine scavenge goal in terms of heap_inuse
This change makes it so that the scavenge goal is defined primarily in
terms of heap_inuse at the end of the last GC rather than next_gc. The
reason behind this change is that next_gc doesn't take into account
fragmentation, and we can fall into situation where the scavenger thinks
it should have work to do but there's no free and unscavenged memory
available.
In order to ensure the scavenge goal still tracks next_gc, we multiply
heap_inuse by the ratio between the current heap goal and the last heap
goal, which describes whether the heap is growing or shrinking, and by
how much.
Finally, this change updates the documentation for scavenging and
elaborates on why the scavenge goal is defined the way it is.
Fixes #34048.
Updates #32828.
Change-Id: I8deaf87620b5dc12a40ab8a90bf27932868610da
Reviewed-on: https://go-review.googlesource.com/c/go/+/193040
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Keith Randall <khr@golang.org>
I'm also not sure why it ever worked; it looks like it is writing
the arguments to racecallback in the wrong place (the race detector
itself probably still works, it would just have trouble symbolizing
any resulting race report).
At a meta-level, we should really add a ppc64le/race builder.
Otherwise this code will rot, as evidenced by the rot this CL fixes :)
When the -json flag is passed to go mod download,
the sumdb error is embedded in the json Error field.
Other errors for the same command behave this way as
well such as module not found. The fix is done by changing
base.Fatalf into proper error returns.
Fixes #34485
Change-Id: I2727a5c70c7ab03988cad8661894d0f8ec71a768
Reviewed-on: https://go-review.googlesource.com/c/go/+/197062
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Matthew Dempsky [Fri, 20 Sep 2019 22:31:13 +0000 (15:31 -0700)]
cmd/compile: optimize escape graph construction and walking
This CL implements several optimizations for the escape analysis flow
graph:
1. Instead of recognizing heapLoc specially within Escape.outlives,
set heapLoc.escapes = true and recognize any location with escapes
set. This allows us to skip adding edges from the heap to escaped
variables in two cases:
1a. In newLoc, if the location is for a variable or allocation too
large to fit on the stack.
1b. During walkOne, if we discover that an object's address flows
somewhere that naturally outlives it.
2. When recording edges in Escape.flow, if x escapes and we're adding
an edge like "x = &y", we can simply mark that y escapes too.
3. During walkOne, if we reach a location that's marked as escaping,
we can skip visiting it again: we've either already walked from it, or
it's in queue to be walked from again.
On average, reduces the number of visited locations by 15%. Reduces
time spent in escape analysis for particularly hairy packages like
runtime and gc by about 8%. Reduces escape.go's TODO count by 22%.
Matthew Dempsky [Fri, 20 Sep 2019 22:27:14 +0000 (15:27 -0700)]
cmd/compile: use proper work queue for escape graph walking
The old escape analysis code used to repeatedly walk the entire flow
graph until it reached a fixed point. With escape.go, I wanted to
avoid this if possible, so I structured the walking code with two
constraints:
1. Always walk from the heap location last.
2. If an object escapes, ensure it has flow edge to the heap location.
This works, but it precludes some graph construction
optimizations. E.g., if there's an assignment "heap = &x", then we can
immediately tell that 'x' escapes without needing to visit it during
the graph walk. Similarly, if there's a later assignment "x = &y", we
could immediately tell that 'y' escapes too. However, the natural way
to implement this optimization ends up violating the constraints
above.
Further, the constraints above don't guarantee that the 'transient'
flag is handled correctly. Today I think that's handled correctly
because of the order that locations happen to be constructed and
visited based on the AST, but I've felt uneasy about it for a little
while.
This CL changes walkAll to use a proper work queue (technically a work
stack) to track locations that need to be visited, and allows walkOne
to request that a location be re-visited.
Jeremy Faller [Thu, 22 Aug 2019 16:18:28 +0000 (12:18 -0400)]
cmd/compile: update object file format for DWARF file table
In CL 188317, we generate the debug_lines in the compiler, and created a
new symbol to hold the line table. Here we modify the object file format
to output the file table.
Matthew Dempsky [Wed, 25 Sep 2019 06:56:50 +0000 (23:56 -0700)]
cmd/compile: use underlying OCOMPLIT's position for OPTRLIT
Currently, when we create an OPTRLIT node, it defaults to the
OCOMPLIT's final element's position. But it improves error messages to
use the OCOMPLIT's own position instead.
Matthew Dempsky [Wed, 25 Sep 2019 07:14:58 +0000 (00:14 -0700)]
cmd/compile: remove -s flag
This is better handled by tools like cmd/gofmt, which can
automatically rewrite the source code and already supports a syntactic
version of this simplification. (go/types can be used if
type-sensitive simplification is actually necessary.)
Daniel Martí [Tue, 24 Sep 2019 12:11:36 +0000 (13:11 +0100)]
text/template: don't evaluate '.' as a float64
When using a '.' constant literal as a reflect.Value variadic argument,
idealConstant would incorrectly result in a float64. This is because
rune literals can be represented as a float64, and contain a period,
which tricked the logic into thinking the literal must have been a
floating point number.
This also happened with other characters that can be part of a floating
point number, such as 'e' or 'P'.
To fix these edge cases, exit the case sooner if the literal was a rune,
since that should always go to the int case instead.
Finally, add test cases that verify that they behave properly. These
would error before, since eq would receive a mix of int and float64,
which aren't comparable.
Fixes #34483.
Change-Id: Icfcb7803bfa0cf317a1d1adacacad3d69a57eb42
Reviewed-on: https://go-review.googlesource.com/c/go/+/196808
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tom Payne <tom@airmap.com> Reviewed-by: Rob Pike <r@golang.org>
In CL 192980, I tend to think that canSSAType can be used as replacement
for isfat. It is not the truth as @khr points me out that isfat has very
different purpose.
So this CL adds documentation for isfat, also remove outdated TODO.
Change-Id: I15954d638759bd9f6b28a6aa04c1a51129d9ae7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/196499
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Jeremy Faller [Wed, 31 Jul 2019 14:33:11 +0000 (10:33 -0400)]
cmd/compile: generate debug_lines in compiler
This is mostly a copy-paste jobs from the linker to generate the debug
information in the compiler instead of the linker. The new data is
inserted into the debug line numbers symbol defined in CL 188238.
Generating the debug information BEFORE deadcode results in one subtle
difference, and that is that the state machine needs to be reset at the
end of every function's debug line table. The reasoning is that
generating the table AFTER dead code allows the producer and consumer of
the table to agree on the state of the state machine, and since these
blocks will (eventually) be concatenated in the linker, we don't KNOW
the state of the state machine unless we reset it. So,
generateDebugLinesSymbol resets the state machine at the end of every
function.
Right now, we don't do anything with this line information, or the file
table -- we just populate the symbols.
Jeremy Faller [Fri, 9 Aug 2019 15:36:03 +0000 (11:36 -0400)]
cmd/link: add notion of multiple compilation units per package
As we move the debug_line generation into the compiler, we need to
upgrade the notion of compilationUnit to not just be on a per package
basis. That won't be the case as it will be impossible for all
compilationUnits to have the same set of files names used to build the
debug_lines table. (For example, assembled files in a package don't know
about any files but themselves, so the debug_lines table could only
reference themseves. As such, we need to break the 1:1 relationship
between compUnit and package.)
cmd/go: suppress errors in package-to-module queries if the package is already found
In CL 173017, I changed the package-to-module query logic to query all
possible module paths in parallel in order to reduce latency. (For
long package paths, most such paths will not exist and will fail with
little overhead.)
The module resolution algorithm treats various kinds of non-existence
as “soft errors”, to be reported only if package resolution fails, but
treats any remaining errors as hard errors that should fail the query.
Unfortunately, that interacted badly with the +incompatible version
validation added in CL 181881, causing a regression in the 'direct'
fetch path for modules using the “major branch” layout¹ with a post-v1
version on the repository's default branch. Because we did not
interpret a mismatched module path as “no such module”, a go.mod file
specifying the path 'example.com/foo/v2' would cause the search for
module 'example.com/foo' to error out. (That regression was not caught
ahead of time due to a lack of test coverage for 'go get' on a package
within a /vN module.)
The promotion of hard errors during parallel search also made the 'go'
command less tolerant of servers that advertise 'go-import' tags for
nonexistent repositories. CL 194561 mitigated that problem for HTTP
servers that return code 404 or 410 for a nonexistent repository, but
unfortunately a few servers in common use (notably GitLab and
pre-1.9.3 releases of Gitea) do not.
This change mitigates both of those failure modes by ignoring
“miscellaneous” errors from shorter module paths if the requested
package pattern was successfully matched against a module with a
longer path.
Martin Möhrmann [Mon, 9 Sep 2019 15:50:35 +0000 (17:50 +0200)]
compile: prefer an AND instead of SHR+SHL instructions
On modern 64bit CPUs a SHR, SHL or AND instruction take 1 cycle to execute.
A pair of shifts that operate on the same register will take 2 cycles
and needs to wait for the input register value to be available.
Large constants used to mask the high bits of a register with an AND
instruction can not be encoded as an immediate in the AND instruction
on amd64 and therefore need to be loaded into a register with a MOV
instruction.
However that MOV instruction is not dependent on the output register and
on many CPUs does not compete with the AND or shift instructions for
execution ports.
Using a pair of shifts to mask high bits instead of an AND to mask high
bits of a register has a shorter encoding and uses one less general
purpose register but is slower due to taking one clock cycle longer
if there is no register pressure that would make the AND variant need to
generate a spill.
For example the instructions emitted for (x & 1 << 63) before this CL are: 48c1ea3f SHRQ $0x3f, DX 48c1e23f SHLQ $0x3f, DX
after this CL the instructions are the same as GCC and LLVM use: 48b80000000000000080 MOVQ $0x8000000000000000, AX
4821d0 ANDQ DX, AX
Some platforms such as arm64 already have SSA optimization rules to fuse
two shift instructions back into an AND.
Removing the general rule to rewrite AND to SHR+SHL speeds up this benchmark:
var GlobalU uint
func BenchmarkAndHighBits(b *testing.B) {
x := uint(0)
for i := 0; i < b.N; i++ {
x &= 1 << 63
}
GlobalU = x
}
amd64/darwin on Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz:
name old time/op new time/op delta
AndHighBits-4 0.61ns ± 6% 0.42ns ± 6% -31.42% (p=0.000 n=25+25):
'go run run.go -all_codegen -v codegen' passes with following adjustments:
ARM64: The BFXIL pattern ((x << lc) >> rc | y & ac) needed adjustment
since ORshiftRL generation fusing '>> rc' and '|' interferes
with matching ((x << lc) >> rc) to generate UBFX. Previously
ORshiftLL was created first using the shifts generated for (y & ac).
S390X: Add rules for abs and copysign to match use of AND instead of SHIFTs.
cmd/go/internal/modfetch/codehost: work around an apparent bug in 'git fetch --unshallow'
When 'git fetch' is passed the '--unshallow' flag, it assumes that the
local and remote refs are equal.¹ However, we were fetching an
expanded set of refs explicitly in the same command, violating that
assumption.
Now we first expand the set of refs, then unshallow the repo in a
separate fetch. Empirically, this seems to work, whereas the opposite
order does not.
Eli Bendersky [Thu, 1 Aug 2019 22:39:40 +0000 (15:39 -0700)]
cmd/gofmt: fix computation of function header size
Function sizes are computed to determine whether a function
can be kept on one line or should be split to several lines. Part of the
computation is the function header from the FUNC token and until the
opening { token.
Prior to this change, the function header size used distance from the
original source position of the current token, which led to issues when
the source between FUNC and the original source position was rewritten
(such as whitespace being collapsed). Now we take the current output
position into account, so that header size represents the reformatted
source rather than the original source.
The following files in the Go repository are reformatted with this
change:
In both cases the reformatting is minor and seems to be correct given
the heuristic to single-line functions longer than 100 columns to
multiple lines.