Austin Clements [Thu, 22 Mar 2018 16:04:51 +0000 (12:04 -0400)]
cmd/compile: introduce LivenessMap and LivenessIndex
Currently liveness only produces a stack map index at each safe point,
so the information is summarized in a map[*ssa.Value]int. We're about
to have both a stack map index and a register map index, so replace
the int with a LivenessIndex type we can extend, and replace the map
with a LivenessMap that we can also change more easily in the future.
This also gives us an easy hook for defining the value that means "not
a safe point".
The obj package needs to emit the PCDATA to select the entry stack map
before calling morestack. Currently this is copied for every
architecture. Since we're about to change how this works, consolidate
all of these copies into a single helper function.
Austin Clements [Thu, 22 Mar 2018 16:04:51 +0000 (12:04 -0400)]
cmd/compile: don't produce a past-the-end pointer in range loops
Currently, range loops over slices and arrays are compiled roughly
like:
for i, x := range s { b }
⇓
for i, _n, _p := 0, len(s), &s[0]; i < _n; i, _p = i+1, _p + unsafe.Sizeof(s[0]) { b }
⇓
i, _n, _p := 0, len(s), &s[0]
goto cond
body:
{ b }
i, _p = i+1, _p + unsafe.Sizeof(s[0])
cond:
if i < _n { goto body } else { goto end }
end:
The problem with this lowering is that _p may temporarily point past
the end of the allocation the moment before the loop terminates. Right
now this isn't a problem because there's never a safe-point during
this brief moment.
We're about to introduce safe-points everywhere, so this bad pointer
is going to be a problem. We could mark the increment as an unsafe
block, but this inhibits reordering opportunities and could result in
infrequent safe-points if the body is short.
Instead, this CL fixes this by changing how we compile range loops to
never produce this past-the-end pointer. It changes the lowering to
roughly:
i, _n, _p := 0, len(s), &s[0]
if i < _n { goto body } else { goto end }
top:
_p += unsafe.Sizeof(s[0])
body:
{ b }
i++
if i < _n { goto top } else { goto end }
end:
Notably, the increment is split into two parts: we increment the index
before checking the condition, but increment the pointer only *after*
the condition check has succeeded.
The implementation builds on the OFORUNTIL construct that was
introduced during the loop preemption experiments, since OFORUNTIL
places the increment and condition after the loop body. To support the
extra "late increment" step, we further define OFORUNTIL's "List"
field to contain the late increment statements. This makes all of this
a relatively small change.
This depends on the improvements to the prove pass in CL 102603. With
the current lowering, bounds-check elimination knows that i < _n in
the body because the body block is dominated by the cond block. In the
new lowering, deriving this fact requires detecting that i < _n on
*both* paths into body and hence is true in body. CL 102603 made prove
able to detect this.
The code size effect of this is minimal. The cmd/go binary on
linux/amd64 increases by 0.17%. Performance-wise, this actually
appears to be a net win, though it's mostly noise:
Austin Clements [Fri, 23 Mar 2018 21:38:55 +0000 (17:38 -0400)]
cmd/compile: detect OFORUNTIL inductive facts in prove
Currently, we compile range loops into for loops with the obvious
initialization and update of the index variable. In this form, the
prove pass can see that the body is dominated by an i < len condition,
and findIndVar can detect that i is an induction variable and that
0 <= i < len.
GOEXPERIMENT=preemptibleloops compiles range loops to OFORUNTIL and
we're preparing to unconditionally switch to a variation of this for
#24543. OFORUNTIL moves the increment and condition *after* the body,
which makes the bounds on the index variable much less obvious. With
OFORUNTIL, proving anything about the index variable requires
understanding the phi that joins the index values at the top of the
loop body block.
This interferes with both prove's ability to see that i < len (this is
true on both paths that enter the body, but from two different
conditional checks) and with findIndVar's ability to detect the
induction pattern.
Fix this by teaching prove to detect that the index in the pattern
constructed by OFORUNTIL is an induction variable and add both bounds
to the facts table. Currently this is done separately from findIndVar
because it depends on prove's factsTable, while findIndVar runs before
visiting blocks and building the factsTable.
Without any GOEXPERIMENT, this has no effect on std or cmd. However,
with GOEXPERIMENT=preemptibleloops, this change becomes necessary to
prove 90 conditions in std and cmd.
Austin Clements [Fri, 23 Mar 2018 21:25:26 +0000 (17:25 -0400)]
cmd/compile: derive len/cap relations in factsTable.update
Currently, the prove pass derives implicit relations between len and
cap in the code that adds branch conditions. This is fine right now
because that's the only place we can encounter len and cap, but we're
about to add a second way to add assertions to the facts table that
can also produce facts involving len and cap.
Prepare for this by moving the fact derivation from updateRestrictions
(where it only applies on branches) to factsTable.update, which can
derive these facts no matter where the root facts come from.
Austin Clements [Fri, 23 Mar 2018 21:21:33 +0000 (17:21 -0400)]
cmd/compile: teach prove about relations between constants
Currently, we never add a relation between two constants to prove's
fact table because these are eliminated before prove runs, so it
currently doesn't handle facts like this very well even though they're
easy to prove.
We're about to start asserting some conditions that don't appear in
the SSA, but are constructed from existing SSA values that may both be
constants.
Hence, improve the fact table to understand relations between
constants by initializing the constant bounds of constant values to
the value itself, rather than noLimit.
Inlining was refactored to perform tuning experiments,
with the "knobs" now set to also inline functions/methods
that include panic(), and -l=4 (inline calls) now expressed
as a change to costs, rather than scattered if-thens.
The -l=4 inline-calls penalty is chosen to be the best
found during experiments; it makes some programs much
larger and slower (notably, the compiler itself) and is
believed to be risky for machine-generated code in general,
which is why it is not the default. It is also not
well-tested with the debugger and DWARF output.
This change includes an explicit go:noinline applied to the
method that is the largest cause of compiler binary growth
and slowdown for midstack inlining; there are others,
ideally whatever heuristic eventually appears will make
this unnecessary.
Adam Langley [Wed, 16 May 2018 21:35:23 +0000 (14:35 -0700)]
crypto/x509: check EKUs like 1.9.
This change brings back the EKU checking from 1.9. In 1.10, we checked
EKU nesting independent of the requested EKUs so that, after verifying a
certifciate, one could inspect the EKUs in the leaf and trust them.
That, however, was too optimistic. I had misunderstood that the PKI was
/currently/ clean enough to require that, rather than it being
desirable. Go generally does not push the envelope on these sorts of
things and lets the browsers clear the path first.
Elias Naur [Mon, 21 May 2018 13:33:20 +0000 (15:33 +0200)]
runtime: use raise instead of pthread_self and pthread_kill
pthread_self and pthread_kill are not safe to call from a signal
handler. In particular, pthread_self fails in iOS when called from
a signal handler context.
Use raise instead; it is signal handler safe and simpler.
Change-Id: I0cbfe25151aed245f55d7b76719ce06dc78c6a75
Reviewed-on: https://go-review.googlesource.com/113877
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Austin Clements [Wed, 2 May 2018 18:46:00 +0000 (14:46 -0400)]
runtime: fix bitmap copying corner-cases
When an object spans heap arenas, its bitmap is discontiguous, so
heapBitsSetType unrolls the bitmap into the object itself and then
copies it out to the real heap bitmap. Unfortunately, since this code
path is rare, it had two unnoticed bugs related to the head and tail
of the bitmap:
1. At the head of the object, we were using hbitp as the destination
bitmap pointer rather than h.bitp, but hbitp points into the
*temporary* bitmap space (that is, the object itself), so we were
failing to copy the partial bitmap byte at the head of an object.
2. The core copying loop copied all of the full bitmap bytes, but
always drove the remaining word count down to 0, even if there was a
partial bitmap byte for the tail of the object. As a result, we never
wrote partial bitmap bytes at the tail of an object.
I found these by enabling out-of-place unrolling all the time. To
improve our chances of detecting these sorts of bugs in the future,
this CL mimics this by enabling out-of-place mode 50% of the time when
doubleCheck is enabled so that we test both in-place and out-of-place
mode.
Alberto Donizetti [Fri, 4 May 2018 10:08:47 +0000 (12:08 +0200)]
cmd/go: test that Go binaries can be run on QEMU in user-mode
We have a workaround in place in the runtime (see CL 16853 and
CL 111176) to keep arm and arm64 Go binaries working under QEMU
in user-emulation mode (Issue #13024).
This change adds a regression test about arm/arm64 QEMU emulation
to cmd/go.
Change-Id: Ic67f476e7c30a7d7852d9b01834f1dcabfac2ff7
Reviewed-on: https://go-review.googlesource.com/111477 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Martí [Sun, 20 May 2018 15:02:35 +0000 (16:02 +0100)]
cmd/trace: fix a few bugs found by staticcheck
First, the regions sort was buggy, as its last comparison was
ineffective.
Second, the insyscall and insyscallRuntime fields were unsigned, so the
check for them being negative was pointless. Make them signed instead,
to also prevent the possibility of underflows when decreasing numbers
that might realistically be 0.
Third, the color constants were all untyped strings except the first
one. Be consistent with their typing.
Change-Id: I4eb8d08028ed92589493c2a4b9cc5a88d83f769b
Reviewed-on: https://go-review.googlesource.com/113895
Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Munday [Wed, 16 May 2018 10:21:18 +0000 (11:21 +0100)]
cmd/compile: use math/bits functions where possible
Use the math/bits functions to calculate the number of leading/
trailing zeros, bit length and the population count.
The math/bits package is built as part of the bootstrap process
so we do not need to provide an alternative implementation for
Go versions prior to 1.9.
Keith Randall [Tue, 1 May 2018 16:42:04 +0000 (09:42 -0700)]
runtime: use libc for nanotime on Darwin
Use mach_absolute_time and mach_timebase_info to get nanosecond-level
timing information from libc on Darwin.
The conversion code from Apple's arbitrary time unit to nanoseconds is
really annoying. It would be nice if we could replace the internal
runtime "time" with arbitrary units and put the conversion to nanoseconds
only in the places that really need it (so it isn't in every nanotime call).
It's especially annoying because numer==denom==1 for all the machines
I tried. Makes it hard to test the conversion code :(
Update #17490
Change-Id: I6c5d602a802f5c24e35184e33d5e8194aa7afa86
Reviewed-on: https://go-review.googlesource.com/110655
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Thu, 3 May 2018 17:30:31 +0000 (10:30 -0700)]
runtime: fix darwin 386/amd64 stack switches
A few libc_ calls were missing stack switches.
Unfortunately, adding the stack switches revealed a deeper problem.
systemstack() is fundamentally flawed because when you do
systemstack(func() { ... })
There's no way to mark the anonymous function as nosplit. At first I
thought it didn't matter, as that function runs on the g0 stack. But
nosplit is still required, because some syscalls are done when stack
bounds are not set up correctly (e.g. in a signal handler, which runs
on the g0 stack, but g is still pointing at the g stack). Instead use
asmcgocall and funcPC, so we can be nosplit all the way down.
Mid-stack inlining now pushes darwin over the nosplit limit also.
Leaving that as a TODO.
Update #23168
This might fix the cause of occasional darwin hangs.
Update #25181
Update #17490
Change-Id: If9c3ef052822c7679f5a1dd192443f714483327e
Reviewed-on: https://go-review.googlesource.com/111258 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ali Rizvi-Santiago [Fri, 18 May 2018 18:48:44 +0000 (18:48 +0000)]
debug/pe: parse the import directory correctly
This parses the import table properly which allows for debug/pe
to extract import symbols from pecoffs linked with an import
table in a section named something other than ".idata"
The section names in a pecoff object aren't guaranteed to actually
mean anything, so hardcoding a search for the ".idata" section
is not guaranteed to find the import table in all shared libraries.
This resulted in debug/pe being unable to read import symbols
from some libraries.
The proper way to locate the import table is to validate the
number of data directory entries, locate the import entry, and
then use the va to identify the section containing the import
table. This patch does exactly this.
Fixes #16103.
Change-Id: I3ab6de7f896a0c56bb86c3863e504e8dd4c8faf3
GitHub-Last-Rev: ce8077cb154f18ada7a86e152ab03de813937816
GitHub-Pull-Request: golang/go#25193
Reviewed-on: https://go-review.googlesource.com/110555
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Richard Musiol [Thu, 17 May 2018 10:33:01 +0000 (12:33 +0200)]
misc/wasm: make wasm_exec.js more flexible
This commit improves wasm_exec.js to give more control to the
code that uses this helper:
- Allow to load and run more than one Go program at the same time.
- Move WebAssembly.instantiate out of wasm_exec.js so the caller
can optimize for load-time performance, e.g. by using
instantiateStreaming.
- Allow caller to provide argv, env and exit callback.
Robert Griesemer [Fri, 18 May 2018 03:56:11 +0000 (23:56 -0400)]
go/parser: make sure we have a valid AST when 'if' condition is missing
This prevents a crash in go/types due to a nil condition in an 'if'
statement. There's more we can do to make go/types more robust but
this will address the immediate cause and also makes sure that the
parser returns a valid AST in this case.
Fixes #25438.
Change-Id: Ie55dc2c722352a5ecb17af6a16983741e8a8b515
Reviewed-on: https://go-review.googlesource.com/113735
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Dominik Honnef <dominik@honnef.co> Reviewed-by: Alan Donovan <adonovan@google.com>
Elias Naur [Thu, 17 May 2018 13:20:57 +0000 (15:20 +0200)]
net: skip external net tests on iOS
CL 113095 tried to deflake net tests on iOS by skipping the test
that uses the most sockets. That didn't work well enough and will
be reverted in CL 113555.
The flakes appeared after the iOS exec harness started to forward
environment variables, causing testenv.Builder to be non-empty on
the iOS builder. This CL attempts to fix the flakes with the more
conservative strategy of skipping tests that only run on builders.
The skipped tests happen to be those requiring external network
access; it's plausible that the iOS builder network isn't reliable
enough to run the many parallel DNS lookups and dial outs, while
keeping the number of open file descriptors below the 250 limit.
Zhou Peng [Wed, 16 May 2018 12:56:54 +0000 (12:56 +0000)]
runtime: use debugSelect flag to toggle debug code
This block of code once was commented by the original author, but commenting
code looks a little annoying. However, the debugSelect flag is just for the
situation that debug code will be compiled when debuging, when release this
code will be eliminated by the compiler.
Tobias Klauser [Wed, 16 May 2018 12:10:02 +0000 (14:10 +0200)]
doc/contribute.html: remove superfluous article
After CL 113016 the "a" article is no longer necessary.
Change-Id: I5a80a210bd2b9eedd73d5f9f3f338d7f22c29ea6
Reviewed-on: https://go-review.googlesource.com/113355 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Rob Pike [Mon, 14 May 2018 09:41:37 +0000 (19:41 +1000)]
doc/contribute.html: English cleanups
Fixes #24487
Change-Id: Ic523e469f7f67f376edd2fca6e07d35bb11b2db9
Reviewed-on: https://go-review.googlesource.com/113016 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Munday [Tue, 15 May 2018 17:22:52 +0000 (18:22 +0100)]
cmd/compile: improve error message emitted by debug info generation
Before:
unexpected at 2721:load with unexpected source op v3278unexpected at 2775:load with
unexpected source op v3281unexpected at 2249:load with unexpected source op
v3289unexpected at 2875:load with unexpected source op v3278unexpected at 2232:load
with unexpected source op v286unexpected at 2231:load with unexpected source op
v3291unexpected at 2784:load with unexpected source op v3289unexpected at 2785:load
with unexpected source op v3291
After:
debug info generation: v2721: load with unexpected source op: Phi (v3278)
debug info generation: v2775: load with unexpected source op: Phi (v3281)
debug info generation: v2249: load with unexpected source op: Phi (v3289)
debug info generation: v2875: load with unexpected source op: Phi (v3278)
debug info generation: v2232: load with unexpected source op: Phi (v286)
debug info generation: v2231: load with unexpected source op: Phi (v3291)
debug info generation: v2784: load with unexpected source op: Phi (v3289)
debug info generation: v2785: load with unexpected source op: Phi (v3291)
An import of golang.org/x/sys/cpu was replaced with an import of
internal/cpu as required by
https://github.com/golang/go/issues/24843#issuecomment-383194779.
The following bash command can be used to replicate this import
update:
find `pwd` -name '*.go' -exec sed -i 's/golang\.org\/x\/sys\/cpu/internal\/cpu/g' '{}' \;
isharipo [Tue, 15 May 2018 16:09:49 +0000 (19:09 +0300)]
mime: do a pre-allocation in encodeWord
The preallocated memory size is comparable to bytes.Buffer bootstraping
array length (bytes.Buffer was used before rewrite to strings.Builder).
Without preallocation, encodeWord does more than one allocation for
almost any possible input.
The regression happens because bytes.Buffer did a 80-bytes allocation
at the beginning of encodeWord while strings.Builder did several
smaller allocations (started with cap=0).
Comparison with reported regression:
name old time/op new time/op delta
QEncodeWord-4 781ns ± 1% 593ns ± 1% -24.08% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
QEncodeWord-4 152B ± 0% 80B ± 0% -47.37% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
QEncodeWord-4 5.00 ± 0% 2.00 ± 0% -60.00% (p=0.008 n=5+5)
Comparison with buffer solution (like before strings.Builder, but
without sync pool for buffer re-using):
name old time/op new time/op delta
QEncodeWord-4 595ns ± 1% 593ns ± 1% ~ (p=0.460 n=5+5)
name old alloc/op new alloc/op delta
QEncodeWord-4 160B ± 0% 80B ± 0% -50.00% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
QEncodeWord-4 2.00 ± 0% 2.00 ± 0% ~ (all equal)
Richard Musiol [Mon, 14 May 2018 22:52:18 +0000 (00:52 +0200)]
misc/wasm: fix passing large negative integers from JS to Go
This commit addresses a FIXME left in the code of wasm_exec.js to
properly get the upper 32 bit of a JS number to be stored as an
64-bit integer. A bitshift operation is not possible, because in
JavaScript bitshift operations only operate on the lower 32 bits.
Richard Musiol [Wed, 9 May 2018 13:18:59 +0000 (15:18 +0200)]
misc/wasm: pollute global JS namespace less
This commit changes wasm_exec.js so it only puts the single
name "go" into the global namespace. Other names became private
or were turned into a property/method of "go".
Elias Naur [Mon, 14 May 2018 18:02:22 +0000 (20:02 +0200)]
net: skip socket hungry test on iOS
The iOS builder recently gained access to the GO_BUILDER_NAME
environment variable, which in turn enabled some net tests that
were previously guarded by testenv.Builder() == "". Some such tests
have been disabled because they don't work; others have increased
the pressure on open file descriptors, pushing the low iOS limit of
250.
Since many net tests run with t.Parallel(), the "too many open files"
error hit many different tests, so instead of playing whack-a-mole,
lower the file descriptor demand by skipping the most file
descriptor hungry test, TestTCPSpuriousConnSetupCompletionWithCancel.
Before:
$ GO_BUILDER_NAME=darwin-arm64 GOARCH=arm64 go test -short -v net
...
Socket statistical information:
...
(inet4, stream, default): opened=5245 connected=193 listened=75 accepted=177 closed=5399 openfailed=0 connectfailed=5161 listenfailed=0 acceptfailed=143 closefailed=0
...
After:
$ GO_BUILDER_NAME=darwin-arm64 GOARCH=arm64 go test -short -v net
...
Socket statistical information:
...
(inet4, stream, default): opened=381 connected=194 listened=75 accepted=169 closed=547 openfailed=0 connectfailed=297 listenfailed=0 acceptfailed=134 closefailed=0
...
Fixes #25365 (Hopefully).
Change-Id: I8343de1b687ffb79001a846b1211df7aadd0535b
Reviewed-on: https://go-review.googlesource.com/113095
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
David Chase [Mon, 14 May 2018 20:12:59 +0000 (16:12 -0400)]
cmd/compile: remove now-irrelevant test
This test measures "line churn" which was minimized to help
improve the debugger experience. With proper is_stmt markers,
this is no longer necessary, and it is more accurate (for
profiling) to allow line numbers to vary willy-nilly.
"Debugger experience" is now better measured by
cmd/compile/internal/ssa/debug_test.go
This CL made the obsoleting change:
https://go-review.googlesource.com/c/go/+/102435
testing: allow marking subtest and subbenchmark functions as Helpers
Since subtests and subbenchmarks run in a separate goroutine, and thus
a separate stack, this entails capturing the stack trace at the point
tb.Run is called. The work of getting the file and line information from
this stack is only done when needed, however.
Continuing the search into the parent test also requires temporarily
holding its mutex. Since Run does not hold it while waiting for the
subtest to complete, there should be no risk of a deadlock due to this.
Fixes #24128
Change-Id: If0bb169f3ac96bd48794624e619ade7edb599f83
Reviewed-on: https://go-review.googlesource.com/108658
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
Alberto Donizetti [Wed, 2 May 2018 07:45:40 +0000 (09:45 +0200)]
testing: only compute b.N once when passed -count > 1
When running a benchmark multiple times, instead of re-computing the
value of b.N each time, use the value found by the first run.
For
go test -bench=. -benchtime 3s -count 2 p_test.go
on the benchmark in the linked issue; before:
BenchmarkBenchmark-4 500 10180593 ns/op
--- BENCH: BenchmarkBenchmark-4
p_test.go:13: single call took 10.111079ms
p_test.go:13: single call took 1.017298685s
p_test.go:13: single call took 5.090096124s
BenchmarkBenchmark-4 500 10182164 ns/op
--- BENCH: BenchmarkBenchmark-4
p_test.go:13: single call took 10.098169ms
p_test.go:13: single call took 1.017712905s
p_test.go:13: single call took 5.090898517s
PASS
ok command-line-arguments 12.244s
and after:
BenchmarkBenchmark-4 500 10177076 ns/op
--- BENCH: BenchmarkBenchmark-4
p_test.go:13: single call took 10.091301ms
p_test.go:13: single call took 1.016943125s
p_test.go:13: single call took 5.088376028s
BenchmarkBenchmark-4 500 10171497 ns/op
--- BENCH: BenchmarkBenchmark-4
p_test.go:13: single call took 10.140245ms
p_test.go:13: single call took 5.085605921s
PASS
ok command-line-arguments 11.218s
Giovanni Bajo [Sat, 12 May 2018 20:13:44 +0000 (22:13 +0200)]
cmd/compile: improve undo of poset
prove uses the poset datastructure in a DFS walk, and always undoes
it back to its pristine status. Before this CL, poset's undo of
a new node creation didn't fully deallocate the node, which means
that at the end of prove there was still some allocated memory pending.
This was not a problem until now because the posets used by prove
were discarded after each function, but it would prevent recycling
them between functions (as a followup CL does).
Change-Id: I1c1c99c03fe19ad765395a43958cb256f686765a
Reviewed-on: https://go-review.googlesource.com/112976
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: David Chase <drchase@google.com>
David Chase [Tue, 1 May 2018 03:52:14 +0000 (23:52 -0400)]
cmd/compile: plumb prologueEnd into DWARF
This marks the first instruction after the prologue for
consumption by debuggers, specifically Delve, who asked
for it. gdb appears to ignore it, lldb appears to use it.
The bits for end-of-prologue and beginning-of-epilogue
are added to Pos (reducing maximum line number by 4x, to 1048575). They're added in cmd/internal/obj/<ARCH>.go
(currently x86 only), so the compiler-proper need not
deal with them.
The linker currently does nothing with beginning-of-epilogue,
but the plumbing exists to make it easier in the future.
This also upgrades the line number table to DWARF version 3.
This CL includes a regression in the coverage for
testdata/i22558.gdb-dbg.nexts, this appears to be a gdb
artifact but the fix would be in the preceding CL in the
stack.
David Chase [Sat, 24 Mar 2018 02:46:06 +0000 (22:46 -0400)]
cmd/compile: assign and preserve statement boundaries.
A new pass run after ssa building (before any other
optimization) identifies the "first" ssa node for each
statement. Other "noise" nodes are tagged as being never
appropriate for a statement boundary (e.g., VarKill, VarDef,
Phi).
Rewrite, deadcode, cse, and nilcheck are modified to move
the statement boundaries forward whenever possible if a
boundary-tagged ssa value is removed; never-boundary nodes
are ignored in this search (some operations involving
constants are also tagged as never-boundary and also ignored
because they are likely to be moved or removed during
optimization).
Code generation treats all nodes except those explicitly
marked as statement boundaries as "not statement" nodes,
and floats statement boundaries to the beginning of each
same-line run of instructions found within a basic block.
Line number html conversion was modified to make statement
boundary nodes a bit more obvious by prepending a "+".
The code in fuse.go that glued together the value slices
of two blocks produced a result that depended on the
former capacities (not lengths) of the two slices. This
causes differences in the 386 bootstrap, and also can
sometimes put values into an order that does a worse job
of preserving statement boundaries when values are removed.
Portions of two delve tests that had caught problems were
incorporated into ssa/debug_test.go. There are some
opportunities to do better with optimized code, but the
next-ing is not lying or overly jumpy.
Over 4 CLs, compilebench geomean measured binary size
increase of 3.5% and compile user time increase of 3.8%
(this is after optimization to reuse a sparse map instead
of creating multiple maps.)
This CL worsens the optimized-debugging experience with
Delve; we need to work with the delve team so that
they can use the is_stmt marks that we're emitting now.
The reference output changes from time to time depending
on other changes in the compiler, sometimes better,
sometimes worse.
This CL now includes a test ensuring that 99+% of the lines
in the Go command itself (a handy optimized binary) include
is_stmt markers.
Michael Munday [Sun, 13 May 2018 07:13:56 +0000 (08:13 +0100)]
sync: deflake TestWaitGroupMisuse2
We need to yield to the runtime every now and again to avoid
deadlock. This doesn't show up on most machines because the test
only runs when you have 5 or more CPUs.
Fixes #20072.
Change-Id: Ibf5ed370e919943395f3418487188df0b2be160b
Reviewed-on: https://go-review.googlesource.com/112978
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Dmitri Shuralyov [Fri, 13 Apr 2018 21:59:16 +0000 (17:59 -0400)]
path/filepath: make Abs("") return working directory on Windows
The current Abs docs say:
// If the path is not absolute it will be joined with the current
// working directory to turn it into an absolute path.
The empty string is not an absolute path, so the docs suggest that the
empty string should be joined with the current working directory to
turn it into an absolute path. This was already the case on all
platforms other than Windows. Per the decision in issue #24441,
this change makes it work on Windows too.
Since the empty string is not a valid path for the purposes of calling
os.Stat on it, we can't simply add the empty string test case to
absTests, which TestAbs uses. It would error when trying to do:
info, err := os.Stat(path)
I didn't find a good way to modify TestAbs to handle this situation
without significantly complicating its code and compromising the test.
So, a separate test is created for testing Abs on empty string input.
Fixes #24441.
Change-Id: I11d8ae2f6e6e358f3e996372ee2a0449093898d2
Reviewed-on: https://go-review.googlesource.com/112935 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Rob Pike [Mon, 14 May 2018 03:53:08 +0000 (13:53 +1000)]
doc/contribute.html: clean up HTML and formatting
Mostly just formatting and minor cleanup:
- regularize HTML (add </p> etc.)
- remove all errors caught by tidy
- start all sentences on new line for easy editing
Some wording changes, but there will be more to come.
It seemed there were already enough edits to send it out.
Update #24487
Change-Id: I613ce206b1e8e3e522ecb0bbcd2acb11c4ff5bae
Reviewed-on: https://go-review.googlesource.com/113015 Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime: unify fetching of locals and arguments maps
Currently we have two nearly identical copies of the code that fetches
the locals and arguments liveness maps for a frame, plus a third
that's a poor knock-off. Unify these all into a single function.
Alessandro Arzilli [Sun, 13 May 2018 13:17:40 +0000 (15:17 +0200)]
cmd/link: writelines should keep is_stmt in sync with what it's writing
For all functions but the last one if the function ends on a
non-statement instruction the statement flag in debug_line is changed
but is_stmt is not updated to match.
Change-Id: I03c275c5e261ea672ce4da7baca2458810708326
Reviewed-on: https://go-review.googlesource.com/112979 Reviewed-by: David Chase <drchase@google.com>
Martin Möhrmann [Thu, 10 May 2018 08:10:36 +0000 (10:10 +0200)]
cmd/compile: ensure init of memclr happens after growslice in extendslice
Using the extendslice init node list to add the init nodes for the memclr
call could add init nodes for memclr function before the growslice call
created by extendslice.
As all arguments of the memclr were explicitly set in OAS nodes before
the memclr call this does not change the generated code currently.
./all.bash runs fine when replacing memclr init with nil suggesting there
are currently no additional nodes added to the init of extendslice by
the memclr call.
Add the init nodes for the memclr call directly before the node of the
memclr call to prevent additional future init nodes for function calls
and argument evaluations to be evaluated too early when other compiler
code is added.
Alex Brainman [Sat, 5 May 2018 07:03:21 +0000 (17:03 +1000)]
net: stop multiple sends into single capacity channel in lookupIP
ch is of size 1, and has only one read. But current code can
write to ch more than once. This makes goroutines that do network
name lookups block forever. Only 500 goroutines are allowed, and
we eventually run out of goroutines.
Elias Naur [Fri, 11 May 2018 15:46:00 +0000 (17:46 +0200)]
misc/ios: forward SIGQUIT to the iOS program
When running tests that fails to complete within the test timeout,
the go tool sends the test program a SIGQUIT signal to print
backtraces. However, for tests running with an exec wrapper, the
resulting backtraces will come from the exec wrapper process and
not the test program.
Change the iOS exec wrapper to forward SIGQUIT signals to the lldb
python driver and change the driver to forward the signals to the
running test on the device.
Before:
$ GOARCH=arm64 go test forever_test.go
lldb: running program
SIGQUIT: quit
PC=0x10816fe m=0 sigcode=0
Daniel Theophanes [Fri, 20 Apr 2018 21:22:18 +0000 (14:22 -0700)]
database/sql: add additional Stats to DBStats
Provide better statistics for the database pool. Add counters
for waiting on the pool and closes. Too much waiting or too many
connection closes could indicate a problem.
Robert Griesemer [Thu, 10 May 2018 20:33:47 +0000 (13:33 -0700)]
go/types: adopt spec terminology, use 'embedded' rather then 'anonyous' field
Commit f8b4123613a (https://go-review.googlesource.com/35108) adjusted
the spec to uniformly use 'embedded' rather than 'anonymous' for struct
embedded fields. Adjust go/types' internal terminology.
Provide an additional accessor Var.IsEmbedded().
This is essentially a rename of an internal field and adjustments of
documentation.
Change-Id: Icd07aa192bc5df7a2ee103185fa7e9c55e8f1ac3
Reviewed-on: https://go-review.googlesource.com/112716
Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Alan Donovan <adonovan@google.com>
Daniel Theophanes [Sun, 25 Mar 2018 23:58:27 +0000 (16:58 -0700)]
database/sql: check for nil connRequest.conn before use
The connRequest may return a nil conn value. However in a rare
case that is difficult to test for it was being passed to
DB.putConn without a nil check. This was an error as this
made no sense if the driverConn is nil. This also caused
a panic in putConn.
A test for this would be nice, but didn't find a sane
way to test for this condition.
Robert Griesemer [Thu, 10 May 2018 18:20:35 +0000 (11:20 -0700)]
cmd/compile: use 'not defined' rather than 'unnamed' in error message
A receiver type may have an (alias type) name and thus be 'named'
even though the name doesn't refer to a defined type. Adjust the
error message to make this clearer.
Change-Id: I969bf8d1ba3db8820f67f6ecd6d5cfe564c5b80d
Reviewed-on: https://go-review.googlesource.com/112638 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/go/internal/work: cache cgo invocations for vet, build modes
Even if we had an up-to-date package binary, we reran cgo anyway if
(1) we needed a header file for buildmode c-archive or c-shared, or
(2) we needed cgo-translated files source files for input to go vet.
Cache those outputs too, so that we can avoid cgo if possible.
Working toward exposing the cgo-generated files in go list.
cmd/go: add support for 'go run pkg' or 'go run .'
To date, go run has required a list of .go files.
This CL allows in place of that list a single import path
or a directory name or a pattern matching a single patckage.
This allows 'go run pkg' or 'go run dir', most importantly 'go run .'.
The discussion in #22726 gives more motivation.
The basic idea is that you can already run 'go test .'
but if you're developing a command it's pretty awkward
to iterate at the same speed. This lets you do that,
by using 'go run . [args]'.
Keith Randall [Thu, 10 May 2018 17:15:52 +0000 (10:15 -0700)]
cmd/compile: fix zero extend after float->int conversion
Don't do direct loads from argument slots if the sizes don't match.
This prevents us from loading from a float32 using a uint64 load
during expressions like uint64(math.float32Bits(f)) where f is a float32 arg.
Agniva De Sarker [Wed, 9 May 2018 07:00:43 +0000 (12:30 +0530)]
net/http/pprof: update the /debug/pprof endpoint
- Documented the duration parameter in Profile() to match with Trace().
- Properly handling the error from strconv.ParseInt to match with Trace().
- Updated the profiles tables to include additional handlers exposed from
net/http/pprof. Added a separate section at the bottom to explain what
the profiles are and how to use them.
Fixes #24380
Change-Id: I8b7e100d6826a4feec81f29f918e7a7f7ccc71a0
Reviewed-on: https://go-review.googlesource.com/112495 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Lynn Boger [Wed, 9 May 2018 21:35:10 +0000 (17:35 -0400)]
cmd/compile/internal/ssa: initialize t.UInt in SetTypPtrs()
Initialization of t.UInt is missing from SetTypPtrs in config.go,
preventing rules that use it from matching when they should.
This adds the initialization to allow those rules to work.
Updated test/codegen/rotate.go to test for this case, which
appears in math/bits RotateLeft32 and RotateLeft64. There had been
a testcase for this in go 1.10 but that went away when asm_test.go
was removed.
Richard Musiol [Mon, 7 May 2018 14:18:19 +0000 (16:18 +0200)]
cmd/internal/obj/wasm: avoid invalid offsets for Load/Store
Offsets for Load and Store instructions have type i32. Bad index
expression offsets can cause an offset to be larger than MaxUint32,
which is not allowed. One example for this is the test test/index0.go.
Generate valid code by adding a guard to the responsible rewrite rule.
Also emit a proper error when using such a bad index in assembly code.
This changes decoder.Read to always return io.ErrUnexpectedEOF if the input
contains surplus padding or unexpected content. Previously the error could
be io.EOF or io.ErrUnexpectedEOF depending on how the input was chunked.
Fixes #25296
Change-Id: I07c36c35e6c83e795c3991bfe45647a35aa58aa4
GitHub-Last-Rev: 818dfda90b0edf9fc415da4579c5810268c1cdba
GitHub-Pull-Request: golang/go#25319
Reviewed-on: https://go-review.googlesource.com/112516
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
spec: clarify “Constant expressions” for untyped operands
This change addresses the grammatical complexity described in
https://groups.google.com/forum/#!topic/golang-dev/RmP-LMC3g58.
Change-Id: Ib292b4ca9c880c7c1c8c992e7c033a0f8f951f2c
Reviewed-on: https://go-review.googlesource.com/106855 Reviewed-by: Robert Griesemer <gri@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Section 2.2 of the referenced spec http://www.xml.com/axml/testaxml.htm
defines 0xD7FF as a (sub)range boundary, not 0xDF77.
Fixes #25172
Change-Id: Ic5a3328cd46ef6474b8e93c4a343dcfba0e6511f
Reviewed-on: https://go-review.googlesource.com/109495
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: If431dfa59496b86f58f2ba2a83ca544a28a2a972
Reviewed-on: https://go-review.googlesource.com/112435 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ian Lance Taylor [Fri, 4 May 2018 22:15:47 +0000 (15:15 -0700)]
go/build, cmd/go: don't expect gccgo to have GOROOT packages
When using gccgo the standard library sources are not available in
GOROOT. Don't expect them to be there. In the gccgo build, use a set
of standard library packages generated at build time.
Change-Id: Id133022604d9b7e778e73e8512f9080c61462fba
Reviewed-on: https://go-review.googlesource.com/111595
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Ian Lance Taylor [Fri, 4 May 2018 21:32:59 +0000 (14:32 -0700)]
cmd/go: force untranslated output when running GCC/clang driver
When we look for the tool ID to use for a compiler, force untranslated
output so that we can match the literal string "version".
Fixes https://gcc.gnu.org/PR84765
Change-Id: I607df445dbd3c5a7c3a6907601adcb039ac16fc1
Reviewed-on: https://go-review.googlesource.com/111575
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Ian Lance Taylor [Wed, 2 May 2018 17:02:41 +0000 (10:02 -0700)]
cmd/go: fix testsuite for gccgo
A number of cmd/go tests can never work with gccgo, for various
different reasons. Skip those tests when using gccgo. Adjust some
other tests to pass when using gccgo. Adjust one test to not skip when
using gccgo, since it does work.
Change-Id: I33b09558581a1e304416cf1c05a96f9526abba0e
Reviewed-on: https://go-review.googlesource.com/110915
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Ian Lance Taylor [Fri, 4 May 2018 22:29:09 +0000 (15:29 -0700)]
cmd/go: for gccgo, don't edit cgo header when using -o
This change was made to the gccgo sources as part of CL 47037.
It is required to make the testcarchive and testcshared tests work.
Otherwise using `go build -mode=c-archive -o libgo.a` will cause the
header file to be named go.h rather than libgo.h.
Change-Id: I2db1d7b0f575368b31273cc01097447a0471efd6
Reviewed-on: https://go-review.googlesource.com/111615
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Michael Munday [Sun, 29 Apr 2018 14:12:50 +0000 (15:12 +0100)]
cmd/compile: convert memmove call into Move when arguments are disjoint
Move ops can be faster than memmove calls because the number of bytes
to be moved is fixed and they don't incur the overhead of a call.
This change allows memmove to be converted into a Move op when the
arguments are disjoint.
The optimization is only enabled on s390x at the moment, however
other architectures may also benefit from it in the future. The
memmove inlining rule triggers an extra 12 times when compiling the
standard library. It will most likely make more of a difference as the
disjoint function is improved over time (to recognize fresh heap
allocations for example).
Elias Naur [Tue, 8 May 2018 20:59:35 +0000 (22:59 +0200)]
misc/ios: inject the -u device_id option before any other arguments
The idevicedebugserverproxy command takes a port number without a
flag, like so:
idevicedebugserverproxy 3222
If the -u <device_id> flag is added afterwards, it is ignored and
the command use an arbitrary device. Instead, always inject the -u
flag before any other idevice command arguments.
While here, also kill any leftover idevicedebugserverproxy instance
previous (failed) runs might have left running.
Daniel Martí [Wed, 9 May 2018 04:11:49 +0000 (11:11 +0700)]
cmd/vet: assume that no builtin funcs are pure
That was the intention with the existing code, but it was buggy; builtin
functions aren't treated as values by types.TypeAndVal. Thus, we should
use the IsBuiltin method instead of IsValue.
Teaching vet what builtin funcs are pure is already being tracked as a
separate issue, #22851.
While at it, also add a test with methods, just to be sure that the
current logic doesn't break with that edge case either.
Fixes #25303.
Change-Id: Ic18402b22cceeabf76641c02f575b194b9a536cc
Reviewed-on: https://go-review.googlesource.com/112177
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>