Keith Randall [Sun, 19 Feb 2017 17:37:22 +0000 (09:37 -0800)]
cmd/compile: fix disassembly of invalid instructions
Make sure that if we encode an explicit base register, we print it.
That will ensure that if we make an Addr with an auto variable but
a base that isn't SP, then it will be obvious from the disassembly.
Brad Fitzpatrick [Wed, 1 Mar 2017 17:44:11 +0000 (17:44 +0000)]
net/http: deflake TestClientRedirect308NoGetBody
In an unrelated CL I found a way to increase the likelihood of latent
flaky tests and found this one.
This is just like yesterday's https://golang.org/cl/37624 and dozens
before it (all remnants from the great net/http test parallelization
of Nov 2016 in https://golang.org/cl/32684).
Lynn Boger [Wed, 22 Feb 2017 18:32:54 +0000 (13:32 -0500)]
runtime, cmd/go: roll back stale message, test detail
Some debugging code was recently added to:
1) provide more detail for the stale reason when it is
determined that a package is stale
2) provide file and package time and date information when
it is determined that runtime.a is stale
Lynn Boger [Fri, 10 Feb 2017 20:40:44 +0000 (15:40 -0500)]
cmd/compile: use reg moves for int <-> float conversions on ppc64x
This makes a change in the SSA code generated for OpPPC64Xf2i64
and OpPPC64Xi2f64 to use register based instructions to convert
between float and integer. This will require at least power8.
Currently the conversion is done by storing to and loading
from memory, which is more expensive.
Change-Id: I8e1cf14d3880d7cd720dc5188dd174cba1f7fef7
Reviewed-on: https://go-review.googlesource.com/36725 Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com> Reviewed-by: Keith Randall <khr@golang.org>
Kevin Burke [Wed, 1 Mar 2017 00:28:34 +0000 (16:28 -0800)]
os: add OpenFile example for appending data
Fixes #19329.
Change-Id: I6d8bb112a56d751a6d3ea9bd6021803cb9f59234
Reviewed-on: https://go-review.googlesource.com/37619 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Heschi Kreinick [Mon, 27 Feb 2017 23:26:33 +0000 (18:26 -0500)]
testing: fix Benchmark() to start at 1 iteration, not 100
The run1 call removed in golang.org/cl/36990 was necessary to
initialize the duration of the benchmark. With it gone, the math in
launch() starts from 100. This doesn't work out well for second-long
benchmark methods. Put it back.
Alex Brainman [Wed, 8 Feb 2017 01:47:43 +0000 (12:47 +1100)]
cmd/link: write dwarf sections
Also stop skipping TestExternalLinkerDWARF and
TestDefaultLinkerDWARF.
Fixes #10776.
Change-Id: Ia596a684132e3cdee59ce5539293eedc1752fe5a
Reviewed-on: https://go-review.googlesource.com/36983 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Wed, 8 Feb 2017 01:30:30 +0000 (12:30 +1100)]
cmd/link: write dwarf relocations
For #10776.
Change-Id: I11dd441d8e5d6316889ffa8418df8b58c57c677d
Reviewed-on: https://go-review.googlesource.com/36982 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Matthew Dempsky [Tue, 28 Feb 2017 23:51:29 +0000 (15:51 -0800)]
cmd/compile/internal/gc: separate builtin and real runtime packages
The builtin runtime package definitions intentionally diverge from the
actual runtime package's, but this only works as long as they never
overlap.
To make it easier to expand the builtin runtime package, this CL now
loads their definitions into a logically separate "go.runtime"
package. By resetting the package's Prefix field to "runtime", any
references to builtin definitions will still resolve against the real
package runtime.
Fixes #14482.
Passes toolstash -cmp.
Change-Id: I539c0994deaed4506a331f38c5b4d6bc8c95433f
Reviewed-on: https://go-review.googlesource.com/37538
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Heschi Kreinick [Fri, 17 Feb 2017 21:52:16 +0000 (16:52 -0500)]
cmd/compile, cmd/asm: remove Link.Plists
Link.Plists never contained more than one Plist, and sometimes none.
Passing around the Plist being worked on is straightforward and makes
the data flow easier to follow.
Robert Griesemer [Sat, 25 Feb 2017 02:13:29 +0000 (18:13 -0800)]
compress/flate: use math/bits.Reverse8/16 instead of local implementation
No measurable impact on performance (specifically, no degradation).
Reverse is used in Huffman en/de-coding. For completeness, here are
all the speed-related benchmark results:
Martin Möhrmann [Tue, 28 Feb 2017 20:21:45 +0000 (21:21 +0100)]
strings: fix handling of invalid UTF-8 sequences in Map
The new Map implementation introduced in golang.org/cl/33201
did not differentiate if an invalid UTF-8 sequence was decoded
or the RuneError rune. It would therefore always advance by
3 bytes (which is the length of the RuneError rune) instead
of 1 for an invalid sequences. This cl adds a check to correctly
determine the length of bytes needed to advance to the next rune.
Fixes #19330.
Change-Id: I1e7f9333f3ef6068ffc64015bb0a9f32b0b7111d
Reviewed-on: https://go-review.googlesource.com/37597
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Keith Randall [Tue, 28 Feb 2017 22:01:59 +0000 (14:01 -0800)]
cmd/compile: simplify load+op rules
There's no need to use @block rules, as canMergeLoad makes sure that
the load and op are already in the same block.
With no @block needed, we also don't need to set the type explicitly.
It can just be inherited from the op being rewritten.
runtime: evacuate old map buckets more consistently
During map growth, buckets are evacuated in two ways.
When a value is altered, its containing bucket is evacuated.
Also, an evacuation mark is maintained and advanced every time.
Prior to this CL, the evacuation mark was always incremented,
even if the next bucket to be evacuated had already been evacuated.
This CL changes evacuation mark advancement to skip previously
evacuated buckets. This has the effect of making map evacuation both
more aggressive and more consistent.
Aggressive map evacuation is good. While the map is growing,
map accesses must check two buckets, which may be far apart in memory.
Map growth also delays garbage collection.
And if map evacuation is not aggressive enough, there is a risk that
a populate-once read-many map may be stuck permanently in map growth.
This CL does not eliminate that possibility, but it shrinks the window.
philhofer [Mon, 20 Feb 2017 16:43:54 +0000 (08:43 -0800)]
cmd/compile/ssa: more aggressive constant folding
Add rewrite rules that canonicalize the location
of constants in expressions, and fold conststants
that appear in operations that can be trivially
reassociated.
After this change, the compiler constant-folds
expressions like "4 + x - 1" and "4 & x & 1"
cmd/compile, runtime: specialize convT2x, don't alloc for zero vals
Prior to this CL, all runtime conversions
from a concrete value to an interface went
through one of two runtime calls: convT2E or convT2I.
However, in practice, basic types are very common.
Specializing convT2x for those basic types allows
for a more efficient implementation for those types.
For basic scalars and strings, allocation and copying
can use the same methods as normal code.
For pointer-free types, allocation can occur without
zeroing, and copying can take place without GC calls.
For slices, copying is cheaper and simpler.
While compiling make.bash, 93% of all convT2x calls
are now to one of these specialized convT2x call.
Within specialized convT2x routines, it is cheap to check
for a zero value, in a way that it is not in general.
When we detect a zero value there, we return a pointer
to zeroVal, rather than allocating.
Cherry Zhang [Mon, 20 Feb 2017 04:40:24 +0000 (23:40 -0500)]
cmd/compile: update signature of runtime.memclr*
runtime.memclr* functions have signatures
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)
func memclrHasPointers(ptr unsafe.Pointer, n uintptr)
Update compiler's copy. Also teach gc/mkbuiltin.go to handle
unsafe.Pointer. The import statement and its support is not
really necessary, but just to make it look like real Go code.
cmd/vet/all: move suspicious shift whitelists to 64 bit
This is an inconsequential consequence of updating
math/big to use math/bits.
Better would be to teach the vet shift test
to size int/uint/uintptr to the platform in use,
eliminating the whole category of "might be too small".
Filed #19321 for that.
Michael Munday [Mon, 13 Feb 2017 03:12:12 +0000 (22:12 -0500)]
cmd/compile: emit fused multiply-{add,subtract} instructions on s390x
Explcitly block fused multiply-add pattern matching when a cast is used
after the multiplication, for example:
- (a * b) + c // can emit fused multiply-add
- float64(a * b) + c // cannot emit fused multiply-add
float{32,64} and complex{64,128} casts of matching types are now kept
as OCONV operations rather than being replaced with OCONVNOP operations
because they now imply a rounding operation (and therefore aren't a
no-op anymore).
Operations (for example, multiplication) on complex types may utilize
fused multiply-add and -subtract instructions internally. There is no
way to disable this behavior at the moment.
Improves the performance of the floating point implementation of
poly1305:
Carlo Alberto Ferraris [Fri, 24 Feb 2017 23:48:00 +0000 (08:48 +0900)]
bytes: make bytes.Buffer cache-friendly
During benchmark of an internal tool we found out that (*Buffer).Reset() was
surprisingly showing up in CPU profiles.
This CL contains two related changes aimed at speeding up Reset():
1. Create a fast path for Truncate(0) by moving the logic to Reset()
(this makes Reset() a simple leaf func that gets inlined since it
gets compiled to 3 MOVx instructions). Accordingly change calls in
the rest of the Buffer methods to call Reset() instead of Truncate(0).
2. Reorder the fields in the Buffer struct so that frequently accessed
fields are packed together (buf, off, lastRead). This also make them
likely to be in the same cacheline.
Ideally it would be advisable to have Buffer{} cacheline-aligned, but I
couldn't find a way to do this without changing the size of the bootstrap
array (but this will cause some regressions, because it will make duffcopy
show up in CPU profiles where it wasn't showing up before).
go1 benchmarks are not really affected, but some other benchmarks that
exercise Buffer more show improvements:
Change-Id: I86d7d9d2cac65335baf62214fbb35ba0fd8f9528
Reviewed-on: https://go-review.googlesource.com/37416
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Munday [Tue, 21 Feb 2017 20:20:38 +0000 (15:20 -0500)]
cmd/compile: fix merging of s390x conditional moves into branch conditions
A type conversion inserted between MOVD{LT,LE,GT,GE,EQ,NE} and CMPWconst
by CL 36256 broke the rewrite rule designed to merge the two.
This results in simple for loops (e.g. for i := 0; i < N; i++ {})
emitting two comparisons instead of one, plus a conditional move.
This CL explicitly types the input to CMPWconst so that the type conversion
can be omitted. It also adds a test to check that conditional moves aren't
emitted for loops with 'less than' conditions (i.e. i < N) on s390x.
Fixes #19227.
Change-Id: Ia39e806ed723791c3c755951aef23f957828ea3e
Reviewed-on: https://go-review.googlesource.com/37334 Reviewed-by: Keith Randall <khr@golang.org>
Joe Tsai [Thu, 2 Feb 2017 08:14:25 +0000 (00:14 -0800)]
net/url: document the package better
Changes made:
* Adjust the documented form for a URL to make it more obvious what
happens when the scheme is missing.
* Remove references to Go1.5. We are sufficiently far along enough
that this distinction no longer matters.
* Remove the "Opaque" example which provides a hacky and misleading
use of the Opaque field. This workaround is no longer necessary
since RawPath was added in Go1.5 and the obvious approach just works:
// The raw string "/%2f/" will be sent as expected.
req, _ := http.NewRequest("GET", "https://example.com/%2f/")
The comments in cmd/internal/obj/funcdata.go are identical to the
comments in runtime/funcdata.h, but the majority of the definitions
they refer to don't apply to Go sources and have been stripped out of
funcdata.go.
Remove these stale comments from funcdata.go and clean up the
references to other copies of the PCDATA and FUNCDATA indexes.
Change-Id: I5d6e49a6e586cc9aecd7c3ce1567679f2a605884
Reviewed-on: https://go-review.googlesource.com/37330 Reviewed-by: Keith Randall <khr@golang.org>
Kevin Burke [Wed, 30 Nov 2016 18:03:09 +0000 (10:03 -0800)]
os/user: add Go implementation of LookupGroup, LookupGroupId
If cgo is not available, parse /etc/group in Go to find the name/gid
we need. This does not consult the Network Information System (NIS),
/etc/nsswitch.conf or any other libc extensions to /etc/group.
cmd/compile: ignore some dead code during escape analysis
This is the escape analysis analog of CL 37499.
Fixes #12397
Fixes #16871
The only "moved to heap" decisions eliminated by this
CL in std+cmd are:
cmd/compile/internal/gc/const.go:1514: moved to heap: ac
cmd/compile/internal/gc/const.go:1515: moved to heap: bd
cmd/compile/internal/gc/const.go:1516: moved to heap: bc
cmd/compile/internal/gc/const.go:1517: moved to heap: ad
cmd/compile/internal/gc/const.go:1546: moved to heap: ac
cmd/compile/internal/gc/const.go:1547: moved to heap: bd
cmd/compile/internal/gc/const.go:1548: moved to heap: bc
cmd/compile/internal/gc/const.go:1549: moved to heap: ad
cmd/compile/internal/gc/const.go:1550: moved to heap: cc_plus
cmd/compile/internal/gc/export.go:162: moved to heap: copy
cmd/compile/internal/gc/mpfloat.go:66: moved to heap: b
cmd/compile/internal/gc/mpfloat.go:97: moved to heap: b
Change-Id: I0d420b69c84a41ba9968c394e8957910bab5edea
Reviewed-on: https://go-review.googlesource.com/37508 Reviewed-by: David Chase <drchase@google.com>
Keep liveness bit vectors as simple live-variable vectors during
liveness analysis. We can defer expanding them into runtime heap
bitmaps until we're actually writing out the symbol data, and then we
only need temporary memory to expand one bitmap at a time.
This is logically cleaner (e.g., we no longer depend on stack frame
layout during analysis) and saves a little bit on allocations.
cmd/internal/obj/x86: improve static branch prediction for wrapper prologue
Static branch prediction assumes that forward branches are not taken.
The existing wrapper prologue almost always takes the first forward
branch.
Move the rare case to the end of the function.
This CL is amd64 only. Other architectures will be done in separate CLs.
cmd/compile: ignore some dead code when deciding whether to inline
Constant evaluation provides some rudimentary
knowledge of dead code at inlining decision time.
Use it.
This CL addresses only dead code inside if statements.
For statements are never inlined anyway,
and dead code inside for statements is rare.
Analyzing switch statements is worth doing,
but it is more complicated, since we would have
to evaluate each case; leave it for later.
Fixes #9274
After this CL, the following functions in std+cmd
can be newly inlined:
cmd/internal/obj/x86/asm6.go:3122: can inline subreg
cmd/vendor/golang.org/x/arch/x86/x86asm/decode.go:172: can inline instPrefix
cmd/vendor/golang.org/x/arch/x86/x86asm/decode.go:202: can inline truncated
go/constant/value.go:234: can inline makeFloat
go/types/labels.go:52: can inline (*block).insert
math/big/float.go:231: can inline (*Float).Sign
math/bits/bits.go:57: can inline OnesCount
net/http/server.go:597: can inline (*Server).newConn
runtime/hashmap.go:1165: can inline reflect_maplen
runtime/proc.go:207: can inline os_beforeExit
runtime/signal_unix.go:55: can inline init.5
runtime/stack.go:1081: can inline gostartcallfn
Change-Id: I4c92fb96aa0c3d33df7b3f2da548612e79b56b5b
Reviewed-on: https://go-review.googlesource.com/37499 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
runtime/pprof: handle empty stack traces in Profile.Add
If the caller passes a large number to Profile.Add,
the list of pcs is empty, which results in junk
(a nil pc) being recorded. Check for that explicitly,
and replace such stack traces with a lostProfileEvent.
Kevin Burke [Mon, 27 Feb 2017 10:28:24 +0000 (02:28 -0800)]
os/user: rename group cgo file
In another CL, I'll add a pure Go implementation of lookupGroup and
lookupGroupId in lookup_unix.go, but attempting that in one CL makes
the diff too difficult to read.
David Chase [Fri, 24 Feb 2017 22:21:54 +0000 (17:21 -0500)]
cmd/compile: added cheapexpr call to simplify operand of CONVIFACE
New special case for booleans and byte-sized integer types
converted to interfaces needs to ensure that the operand is
not too complex, if it were to appear in a parameter list
for example.
Added test, also increased the recursive node dump depth to
a level that was actually useful for an actual bug.
Michael Munday [Thu, 23 Feb 2017 21:09:05 +0000 (16:09 -0500)]
os/exec: deflake TestPipeLookPathLeak
The number of open file descriptors reported by lsof is unreliable
because it depends on whether the parent process (the test) closed
the file descriptors it passed into the child process (lsof) before
lsof runs.
Reading /proc/self/fd directly on Linux appears to be much more
reliable and still detects any file descriptor leaks originating
from attempting to run an executable that cannot be found (issue
#5071). If /proc/self/fd is not available (e.g. on Darwin) then we
fall back to lsof and tolerate small differences in open file
descriptor counts.
Fixes #19243.
Change-Id: I052b0c129e609010f1083e43a9911cba154117bf
Reviewed-on: https://go-review.googlesource.com/37343
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Lorenzo Masini [Mon, 20 Feb 2017 16:17:28 +0000 (17:17 +0100)]
cmd/compile: speed up TestAssembly
TestAssembly was very slow, leading to it being skipped by default.
This is not surprising, it separately invoked the compiler and
parsed the result many times.
Now the test assembles one source file for arch/os combination,
containing the relevant functions.
Tests for each arch/os run in parallel.
Now the test runs approximately 10x faster on my Intel(R) Core(TM)
i5-6600 CPU @ 3.30GHz.
Russ Cox [Fri, 17 Feb 2017 20:27:12 +0000 (15:27 -0500)]
runtime: check that pprof accepts but doesn't need executable
The profiles are self-contained now.
Check that they work by themselves in the tests that invoke pprof,
but also keep checking that the old command lines work.
Russ Cox [Fri, 17 Feb 2017 05:10:39 +0000 (00:10 -0500)]
runtime/pprof: add streaming protobuf encoder
The existing code builds a full profile in memory.
Then it translates that profile into a data structure (in memory).
Then it marshals that data structure into a protocol buffer (in memory).
Then it gzips that marshaled form into the underlying writer.
So there are three copies of the full profile data in memory
at the same time before we're done. This is obviously dumb.
This CL implements a fully streaming conversion from
the original in-memory profile to the underlying writer.
There is now only one copy of the profile in memory.
For the non-CPU profiles, this is optimal, since we have to
have a full copy in memory to start with.
For the CPU profiles, we could still try to bound the profile
size stored in memory and stream fragments out during
the actual profiling, as Go 1.7 did (with a simpler format),
but so far that hasn't been necessary.
cmd/compile: evaluate zero-sized values converted to interfaces
CL 35562 substituted zerobase for the pointer for
interfaces containing zero-sized values.
However, it failed to evaluate the zero-sized value
expression for side-effects. Fix that.
The other similar interface value optimizations
are not affected, because they all actually use the
value one way or another.
Robert Griesemer [Wed, 22 Feb 2017 21:43:23 +0000 (13:43 -0800)]
cmd/compile/internal/parser: improved a couple of error messages
The new syntax tree introduced with 1.8 represents send statements
(ch <- x) as statements; the old syntax tree represented them as
expressions (and parsed them as such) but complained if they were
used in expression context. As a consequence, some of the errors
that in the past were of the form "ch <- x used as value" now look
like "unexpected <- ..." because a "<-" is not valid according to
Go syntax in those situations. Accept the new error message.
Also: Fine-tune handling of misformed for loop headers.
Also: Minor cleanups/better comments.
Fixes #17590.
Change-Id: Ia541dea1f2f015c1b21f5b3ae44aacdec60a8aba
Reviewed-on: https://go-review.googlesource.com/37386 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Sean Christopherson [Wed, 22 Feb 2017 14:56:26 +0000 (06:56 -0800)]
run.bash: set GOPATH to $GOROOT/nil before running tests
Set $GOPATH to a semantically valid, non-empty string that cannot
conflict with $GOROOT to avoid false test failures that occur when
$GOROOT resides under $GOPATH. Unsetting GOPATH is no longer viable
as Go now defines a default $GOPATH that may conflict with $GOROOT.
Fixes #19237
Change-Id: I376a2ad3b18e9c4098211b988dde7e76bc4725d2
Reviewed-on: https://go-review.googlesource.com/37396 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox [Thu, 9 Feb 2017 20:57:57 +0000 (15:57 -0500)]
runtime/pprof: use new profile buffers for CPU profiling
This doesn't change the functionality of the current code,
but it sets us up for exporting the profiling labels into the profile.
The old code had a hash table of profile samples maintained
during the signal handler, with evictions going into a log.
The new code just logs every sample directly, leaving the
hash-based deduplication to an ordinary goroutine.
The new code also avoids storing the entire profile in two
forms in memory, an unfortunate regression introduced
when binary profile support was added. After this CL the
entire profile is only stored once in memory. We'd still like
to get back down to storing it zero times (streaming it to
the underlying io.Writer).
Russ Cox [Fri, 17 Feb 2017 15:17:42 +0000 (10:17 -0500)]
runtime: do not allocate on every time.Sleep
It's common for some goroutines to loop calling time.Sleep.
Allocate once per goroutine, not every time.
This comes up in runtime/pprof's background reader.
Change-Id: I89d17dc7379dca266d2c9cd3aefc2382f5bdbade
Reviewed-on: https://go-review.googlesource.com/37162 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Joe Tsai [Thu, 12 Jan 2017 00:53:49 +0000 (16:53 -0800)]
cmd/doc: truncate long lists of arguments
Some field-lists (especially in generated code) can be excessively long.
In the one-line printout, it does not make sense to print all elements
of the list if line-wrapping causes the "one-line" to become multi-line.