Reason for revert: There was a potential problem spotted in the original
fix, which resulted in it being rolled back from release-branch.go1.12
and not included in Go 1.12.1 release. We intend to improve the fix and
include it in Go 1.12.2 instead. To make the fix easier to backport,
revert this change before re-applying the improved fix (next commit).
Change-Id: If6c785f58482d2531b5927c5ea7002f548c21c7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/168402 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Than McIntosh [Thu, 21 Mar 2019 13:20:11 +0000 (09:20 -0400)]
cmd/link: add optional sanity checking for duplicate symbols
Introduce a new linker command line option "-strictdups", which
enables sanity checking of "ok to duplicate" symbols, especially
DWARF info symbols. Acceptable values are 0 (no checking) 1 (issue
warnings) and 2 (issue a fatal error checks fail).
Currently if we read a DWARF symbol (such as "go.info.PKG.FUNCTION")
from one object file, and then encounter the same symbol later on
while reading another object file, we simply discard the second one
and move on with the link, since the two should in theory be
identical.
If as a result of a compiler bug we wind up with symbols that are not
identical, this tends to (silently) result in incorrect DWARF
generation, which may or may not be discovered depending on who is
consuming the DWARF and what's being done with it.
When this option is turned on, at the point where a duplicate
symbol is detected in the object file reader, we check to make sure
that the length/contents of the symbol are the same as the previously
read symbol, and print a descriptive warning (or error) if not.
For the time being this can be used for one-off testing to find
problems; at some point it would be nice if we can enable it by
default.
Updates #30908.
Change-Id: I64c4e07c326b4572db674ff17c93307e2eec607c
Reviewed-on: https://go-review.googlesource.com/c/go/+/168410
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Daniel Martí [Wed, 13 Mar 2019 11:57:29 +0000 (11:57 +0000)]
cmd/compile: reduce rulegen's for loop verbosity
A lot of the naked for loops begin like:
for {
v := b.Control
if v.Op != OpConstBool {
break
}
...
return true
}
Instead, write them out in a more compact and readable way:
for v.Op == OpConstBool {
...
return true
}
This requires the addition of two bytes.Buffer writers, as this helps us
make a decision based on future pieces of generated code. This probably
makes rulegen slightly slower, but that's not noticeable; the code
generation still takes ~3.5s on my laptop, excluding build time.
The "v := b.Control" declaration can be moved to the top of each
function. Even though the rules can modify b.Control when firing, they
also make the function return, so v can't be used again.
While at it, remove three unnecessary lines from the top of each
rewriteBlock func.
In total, this results in ~4k lines removed from the generated code, and
a slight improvement in readability.
Iskander Sharipov [Fri, 22 Mar 2019 12:30:30 +0000 (15:30 +0300)]
sort: replace Errorf+FailNow with Fatalf
Change-Id: I4f8d0178e780b86d1f551b367e2ddac3789be5aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/168880
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Fri, 22 Mar 2019 11:54:37 +0000 (12:54 +0100)]
cmd: enable -buildmode=pie on aix/ppc64
Change-Id: I939518462c931ba9feb125b2f299ef0706b124ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/168879
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Fri, 22 Mar 2019 11:48:31 +0000 (12:48 +0100)]
cmd/compile, misc/cgo: fix fortran tests on aix/ppc64
Enable pattern lib.a/shared.so.X in cgo_import_dynamic as on AIX,
archive files (.a) often have shared objects (.so) inside them.
Change-Id: I21096c75eb7fbcc7064b0b832bfa8ed862142051
Reviewed-on: https://go-review.googlesource.com/c/go/+/168877
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Andrei Tudor Călin [Thu, 21 Mar 2019 15:04:25 +0000 (16:04 +0100)]
image/png: add Fuzz function
Add a Fuzz function to package png, under the gofuzz build
tag. This function is based on the png/png.go code, from
github.com/dvyukov/go-fuzz-corpus, modified to use direct
comparison of image bounds rather than reflect.DeepEqual.
Richard Musiol [Sun, 17 Mar 2019 14:51:30 +0000 (15:51 +0100)]
cmd/link/internal/wasm: optimize data section in wasm binary
This change optimizes the data section in the wasm binary by
omitting blocks of zeroes and instead emitting data segments
with offsets skipping the zeroes.
This optimization is inspired by the memory-packing pass of the
wasm-opt tool and reduces the wasm binary size of "hello world" by 14%.
Nigel Tao [Thu, 21 Mar 2019 04:08:41 +0000 (15:08 +1100)]
image/gif: don't assume Encode src's origin is (0, 0)
When gif.Encode is given an "m image.Image" argument that isn't an
*image.Paletted, it creates a temporary *image.Paletted (called pm) that
is intended to be a copy of this image, only with fewer colors.
That creation process, and specifically the opts.Drawer.Draw call that
does the copy, incorrectly assumed that m.Bounds().Min is the zero point
(0, 0). This commit fixes that.
Cherry Zhang [Thu, 21 Mar 2019 17:40:28 +0000 (13:40 -0400)]
cmd/compile: copy volatile values before emitting write barrier call
It is possible that a "volatile" value (one that can be clobbered
by preparing args of a call) to be used in multiple write barrier
calls. We used to copy the volatile value right before each call.
But this doesn't work if the value is used the second time, after
the first call where it is already clobbered. Copy it before
emitting any call.
Dmitri Shuralyov [Mon, 11 Mar 2019 13:15:38 +0000 (09:15 -0400)]
cmd/go/internal/modfetch: update comment referring to old Import function
The Import function was moved from modfetch/repo.go to modload/import.go
in vgo CL 122880, and its semantics have changed in vgo CL 123095 to do
more than just searching for a module. Both of these changes were ported
to cmd/go in CL 123576.
Delete the mention of the old Import function from the modfetch/repo.go
comment, since what it refers to does not exist anymore.
Change-Id: I6dc6984128152cf9611d30fbc4e6418e91a7641f
Reviewed-on: https://go-review.googlesource.com/c/go/+/166597 Reviewed-by: Jay Conrod <jayconrod@google.com>
Elias Naur [Thu, 21 Mar 2019 14:15:40 +0000 (15:15 +0100)]
misc/ios: don't override GOCACHE on the device
The iOS exec wrapper copies the environment variables to the binary
running on the device. However, some variables such as HOME, TMPDIR and
GOCACHE refer to directories that might not be valid on the device.
The wrapper already ignores HOME and TMPDIR, but after GO111MODULE
was flipped to on for the standard library, cmd/go tests started
failing without a valid and writable GOCACHE.
It took a while to reproduce because go test does not set an explicit
GOCACHE but cmd/dist test does.
Elias Naur [Tue, 19 Mar 2019 05:59:51 +0000 (06:59 +0100)]
misc/android: pass on GOPROXY to device environment
Android devices don't have git available, so many go tool commands
that fetch dependencies fail. Builders already have a GOPROXY
available, so pass that along to the device environment.
Elias Naur [Tue, 19 Mar 2019 11:19:48 +0000 (13:19 +0200)]
cmd/link/internal/ld: copy Mach-O platform version commands to go.o
To build for watchOS and tvOS the Apple toolchain requires a Mach-O
load command that matches the platform for all object files in a build.
The go.o object file produced by the Go linker contains no such command.
The loader commands are mutually exclusive so we need to pick the
right one. Fortunately, cgo must be enabled for watchOS and tvOS to
be useful, so we can copy the first loader command we find in the object
files produced by the host compiler.
Add a test that builds a small program for tvOS to test both this CL
and the previous CL that added bitcode support.
We're going to need the ability to extract the LC_VERSION_MIN_* and
LC_BUILD_VERSION load commands. This CL adds peekMachoPlatform to do
that and in the process simplifies machoCombineDwarf.
While here, disable DWARF combining for Apple platforms other than
macOS (watchOS, tvOS, bridgeOS), not just iOS.
Updates #22395
Change-Id: I4862b0f15ccc87b7be1a6532b4d37b47c8f7f243
Reviewed-on: https://go-review.googlesource.com/c/go/+/168459 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Elias Naur [Sun, 17 Mar 2019 16:48:57 +0000 (17:48 +0100)]
cmd/link/internal/ld: enable bitcode builds for iOS, tvOS, watchOS
The Go toolchain cannot output bitcode, but there is a trick where
object code can be marked with an __asm section, persuading the
Apple toolchain to include our object code in bitcode builds.
This enables Go builds with bitcode enabled; the next CL adds
the necessary plumbing for building on tvOS and watchOS.
Thanks to Aman Gupta for the trick.
Test is added two CLs from here.
Fixes #22395 (at least until Apple tightens bitcode requirements.)
The original goal of rounding to readable b.N
was to make it easier to eyeball times.
However, proper analysis requires tooling
(such as benchstat) anyway.
Instead, take b.N as it comes.
This will reduce the impact of external noise
such as GC on benchmarks.
This requires reworking our iteration estimates.
We used to calculate the estimated ns/op
and then divide our target ns by that estimate.
However, this order of operations was destructive
when the ns/op was very small; rounding could
hide almost an order of magnitude of variation.
Instead, multiply first, then divide.
Also, make n an int64 to avoid overflow.
Prior to this change, we attempted to cap b.N at 1e9.
Due to rounding up, it was possible to get b.N as high as 2e9.
This change consistently enforces the 1e9 cap.
This change also reduces the wall time required to run benchmarks.
Here's the impact of this change on the wall time to run
all benchmarks once with benchtime=1s on some std packages:
This is part of a general effort to shrink walk.
In an ideal world, we'd have an SSA op for allocation,
but we don't yet have a good mechanism for introducing
function calling during SSA compilation.
In the meantime, SSA conversion is a better place for it.
This also makes it easier to introduce new optimizations;
instead of doing the typecheck walk dance,
we can simply write what we want the backend to do.
I introduced a new opcode in this change because:
(a) It avoids a class of bugs involving correctly detecting
whether this ONEW is a "before walk" ONEW or an "after walk" ONEW.
It also means that using ONEW or ONEWOBJ in the wrong context
will generally result in a faster failure.
(b) Opcodes are cheap.
(c) It provides a better place to put documentation.
This change also is also marginally more performant:
Damien Neil [Tue, 19 Feb 2019 21:03:55 +0000 (13:03 -0800)]
os: make errors.Is work with ErrPermission et al.
As proposed in Issue #29934, update errors produced by the os package to
work with errors.Is sentinel tests. For example,
errors.Is(err, os.ErrPermission) is equivalent to os.IsPermission(err)
with added unwrapping support.
Move the definition for os.ErrPermission and others into the syscall
package. Add an Is method to syscall.Errno and others. Add an Unwrap
method to os.PathError and others.
Updates #30322
Updates #29934
Change-Id: I95727d26c18a5354c720de316dff0bffc04dd926
Reviewed-on: https://go-review.googlesource.com/c/go/+/163058 Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
Clément Chigot [Thu, 21 Feb 2019 09:56:35 +0000 (10:56 +0100)]
cmd/dist, misc/cgo: enable tests for aix/ppc64
Some cgo tests aren't yet available on aix/ppc64.
-shared and -static don't work as expected and will be fixed latter.
Updates #30565
Change-Id: Ic59cabe685cb1cbdf89a8d1d1a1d2c4b0e8ef442
Reviewed-on: https://go-review.googlesource.com/c/go/+/164018
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Thu, 21 Feb 2019 09:50:29 +0000 (10:50 +0100)]
cmd/nm: fix cgo tests for aix/ppc64
This commit handles AIX cgo in cmd/nm tests.
Change-Id: I6753a0102e4f2c4c7bd4d7c999f62a0cb3d2183c
Reviewed-on: https://go-review.googlesource.com/c/go/+/164017
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
LE Manh Cuong [Sun, 17 Mar 2019 04:56:29 +0000 (11:56 +0700)]
os: fix windows Lstat missing name for some files
On Windows, GetFileAttributesEx fails with ERROR_SHARING_VIOLATION for
some files, like c:\pagefile.sys. In this case,
newFileStatFromWin32finddata is used to fill file info, but it does not fill
name and path.
After getting file stat from newFileStatFromWin32finddata, just set file info
name and path before return fixes the issue.
Fixes #30883
Change-Id: I654e96c634e8a9bf5ce7e1aaa93968e88953620d
Reviewed-on: https://go-review.googlesource.com/c/go/+/167779
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Mikio Hara [Tue, 19 Mar 2019 09:32:11 +0000 (18:32 +0900)]
runtime: disable event scanning error reporting on solaris
It seems like we need to pay special attention to capturing error
condition on the event port of SmartOS. The previous attempt CL 167777
works on Oracle Solaris but doesn't work on SmartOS for the uncertain
reason. It's better to disable the reporting for now.
Updates #30624.
Fixes #30840.
Change-Id: Ieca5dac4fceb7e8c9cb4db149bb4c2e79691588c
Reviewed-on: https://go-review.googlesource.com/c/go/+/167782
Run-TryBot: Mikio Hara <mikioh.public.networking@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
erifan01 [Mon, 14 Jan 2019 09:36:18 +0000 (09:36 +0000)]
cmd/compile: intrinsify math/bits.Add64 for arm64
This CL instrinsifies Add64 with arm64 instruction sequence ADDS, ADCS
and ADC, and optimzes the case of carry chains.The CL also changes the
test code so that the intrinsic implementation can be tested.
* Reduce the scope of q.
* Remove duplicate handling of AADJSP.
* Move ab declaration closer to use.
* Collapse nested if statements.
* Change declaration of n for increased readability in context.
* Simplify AADJSP handling.
cmd/compile: move flagalloc op splitting to rewrite rules
Flagalloc has the unenviable task of splitting
flag-generating ops that have been merged with loads
when the flags need to "spilled" (i.e. regenerated).
Since there weren't very many of them, there was a hard-coded list
of ops and bespoke code written to split them.
This change migrates load splitting into rewrite rules,
to make them easier to maintain.
Jay Conrod [Mon, 18 Mar 2019 22:35:44 +0000 (18:35 -0400)]
cmd/go: avoid link error when -coverpkg covers main packages (more)
This fixes two problems missed in CL 164877.
First, p.Internal.BuildInfo is now part of the cache key. This is
important since p.Internal.BuildInfo causes the build action to
synthesize a new source file, which affects the output.
Second, recompileForTest is always called for test
packages. Previously, it was only called when there were internal test
sources, so the fix in CL 164877 did not apply to packages that only
had external tests.
Fixes #30374
Change-Id: Iac2d7e8914f0313f9ab4222299a866f67889eb2e
Reviewed-on: https://go-review.googlesource.com/c/go/+/168200
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Thu, 21 Feb 2019 09:46:47 +0000 (10:46 +0100)]
runtime: fix sigfwd for aix/ppc64
This commit fixes runtime.sigfwd for aix/ppc64. fn is a function
descriptor and not a function. R2 must be saved and restored.
Change-Id: Ie506b0bdde562ca37202d19973ba1d537c3d64e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/164015 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Mikio Hara [Thu, 14 Mar 2019 20:10:08 +0000 (05:10 +0900)]
runtime, internal/poll: report only critical event scanning error
This change makes the runtime-integrated network poller report only
critical event scanning errors.
In the previous attempt, CL 166497, we treated any combination of error
events as event scanning errors and it caused false positives in event
waiters because platform-dependent event notification mechanisms allow
event originators to use various combination of events.
To avoid false positives, this change makes the poller treat an
individual error event as a critical event scanning error by the
convention of event notification mechanism implementations.
Updates #30624.
Fixes #30817.
Fixes #30840.
Change-Id: I906c9e83864527ff73f636fd02bab854d54684ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/167777
Run-TryBot: Mikio Hara <mikioh.public.networking@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Thu, 21 Feb 2019 09:45:40 +0000 (10:45 +0100)]
runtime: enable external linker tests for aix/ppc64
Change-Id: Icc42843adb15c2aba1cfea854fad049c6704344b
Reviewed-on: https://go-review.googlesource.com/c/go/+/164014
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Wèi Cōngruì [Mon, 4 Mar 2019 10:07:07 +0000 (10:07 +0000)]
internal/poll, os: cancel pending I/O when closing pipes on Windows
When closing a pipe, use CancelIoEx to cancel pending I/O.
This makes concurrent Read and Write calls return os.ErrClosed.
This change also enables some pipe tests on Windows.
Fixes #28477
Fixes #25835
Change-Id: If52bb7d80895763488a61632e4682a78336e8420
Reviewed-on: https://go-review.googlesource.com/c/go/+/164721
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Thu, 21 Feb 2019 09:14:46 +0000 (10:14 +0100)]
runtime: disable pprof test with cgo on aix/ppc64
This commit disables new cgo pprof tests and adds an handler in sigtramp
to refuse SIGPROF signal.
Updates #28555
Change-Id: I152a871f8636e93328d411329104c6f023bd1691
Reviewed-on: https://go-review.googlesource.com/c/go/+/164013
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Wed, 20 Feb 2019 16:01:15 +0000 (17:01 +0100)]
runtime: fix TestSigStackSwapping on aix/ppc64
This commit fixes TestSigStackSwapping by increasing the signal stack
size. This is needed because SIGSTKSZ is too small when VMX is used on
AIX.
Change-Id: Ic2b5faa65745228d0768383b3d6ebd4b6f9f532c
Reviewed-on: https://go-review.googlesource.com/c/go/+/164012
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Wed, 20 Feb 2019 15:52:35 +0000 (16:52 +0100)]
cmd, runtime: fix C trampolines on aix/ppc64
C trampolines are made by fixup CSECTS which are added between two
symbols. If such CSECTS is added inside Go functions, all method
offsets stored in moduledatas will be wrong.
In order to prevent this, every C code is moved at the end of the
executable and long calls are created for GO functions called by C
code.
The main function can't longer be made in Go as AIX __start isn't using
a long call to branch on it. Therefore, a main is defined on
runtime/cgo.
Change-Id: I214b18decdb83107cf7325b298609eef9f9d1330
Reviewed-on: https://go-review.googlesource.com/c/go/+/164010
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Sat, 5 Jan 2019 01:34:33 +0000 (17:34 -0800)]
cmd/compile: better write barrier removal when initializing new objects
When initializing a new object, we're often writing
1) to a location that doesn't have a pointer to a heap object
2) a pointer that doesn't point to a heap object
When both those conditions are true, we can avoid the write barrier.
This CL detects case 1 by looking for writes to known-zeroed
locations. The results of runtime.newobject are zeroed, and we
perform a simple tracking of which parts of that object are written so
we can determine what part remains zero at each write.
This CL detects case 2 by looking for addresses of globals (including
the types and itabs which are used in interfaces) and for nil pointers.
Makes cmd/go 0.3% smaller. Some particular cases, like the slice
literal in #29573, can get much smaller.
TODO: we can remove actual zero writes also with this mechanism.
Philipp Stephani [Mon, 18 Mar 2019 20:13:17 +0000 (20:13 +0000)]
cmd/cgo: use C exact-width integer types to represent Go types
The exact-width integer types are required to use two’s complement
representation and may not have padding bits, cf. §7.20.1.1/1 in the C11
standard or https://en.cppreference.com/w/c/types/integer. This ensures that
they have the same domain and representation as the corresponding Go types.
Fixes #29878
Change-Id: Ie8a51e91666dfd89731c7859abe47356c94ca1be
GitHub-Last-Rev: 546a2cc3f1e22dc282757f73c01c91b00899d911
GitHub-Pull-Request: golang/go#29907
Reviewed-on: https://go-review.googlesource.com/c/go/+/159258
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Mon, 18 Mar 2019 20:17:35 +0000 (13:17 -0700)]
cmd/compile: rename init function from init.ializers back to init
The name change init -> init.ializers was initially required for
initialization code.
With CL 161337 there's no wrapper code any more, there's a data
structure instead (named .inittask). So we can go back to just
plain init appearing in tracebacks.
Keith Randall [Wed, 6 Feb 2019 00:22:38 +0000 (16:22 -0800)]
cmd/compile: reorganize init functions
Instead of writing an init function per package that does the same
thing for every package, just write that implementation once in the
runtime. Change the compiler to generate a data structure that encodes
the required initialization operations.
Reduces cmd/go binary size by 0.3%+. Most of the init code is gone,
including all the corresponding stack map info. The .inittask
structures that replace them are quite a bit smaller.
Most usefully to me, there is no longer an init function in every -S output.
(There is an .inittask global there, but it's much less distracting.)
After this CL we could change the name of the "init.ializers" function
back to just "init".
Matthew Dempsky [Mon, 18 Mar 2019 19:19:26 +0000 (12:19 -0700)]
cmd/compile: fix importing rewritten f(g()) calls
golang.org/cl/166983 started serializing the Ninit field of OCALL
nodes within function inline bodies (necessary to fix a regression in
building crypto/ecdsa with -gcflags=-l=4), but this means the Ninit
field needs to be typechecked when the imported function body is used.
It's unclear why this wasn't necessary for the crypto/ecdsa
regression.
Fixes #30907.
Change-Id: Id5f0bf3c4d17bbd6d5318913b859093c93a0a20c
Reviewed-on: https://go-review.googlesource.com/c/go/+/168199
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Keith Randall [Wed, 6 Feb 2019 22:12:36 +0000 (14:12 -0800)]
cmd/compile,runtime: provide index information on bounds check failure
A few examples (for accessing a slice of length 3):
s[-1] runtime error: index out of range [-1]
s[3] runtime error: index out of range [3] with length 3
s[-1:0] runtime error: slice bounds out of range [-1:]
s[3:0] runtime error: slice bounds out of range [3:0]
s[3:-1] runtime error: slice bounds out of range [:-1]
s[3:4] runtime error: slice bounds out of range [:4] with capacity 3
s[0:3:4] runtime error: slice bounds out of range [::4] with capacity 3
Note that in cases where there are multiple things wrong with the
indexes (e.g. s[3:-1]), we report one of those errors kind of
arbitrarily, currently the rightmost one.
An exhaustive set of examples is in issue30116[u].out in the CL.
The message text has the same prefix as the old message text. That
leads to slightly awkward phrasing but hopefully minimizes the chance
that code depending on the error text will break.
Increases the size of the go binary by 0.5% (amd64). The panic functions
take arguments in registers in order to keep the size of the compiled code
as small as possible.
Fixes #30116
Change-Id: Idb99a827b7888822ca34c240eca87b7e44a04fdd
Reviewed-on: https://go-review.googlesource.com/c/go/+/161477
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Martin Möhrmann [Wed, 7 Nov 2018 08:33:12 +0000 (09:33 +0100)]
runtime: replace division by span element size by multiply and shifts
Divisions are generally slow. The compiler can optimize a division
to use a sequence of faster multiplies and shifts (magic constants)
if the divisor is not know at compile time.
The value of the span element size in mcentral.grow is not known at
compile time but magic constants to compute n / span.elementsize
are already stored in class_to_divmagic and mspan.
They however need to be adjusted to work for
(0 <= n <= span.npages * pagesize) instead of
(0 <= n < span.npages * pagesize).
Change-Id: Ieea59f1c94525a88d012f2557d43691967900deb
Reviewed-on: https://go-review.googlesource.com/c/go/+/148057
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Daniel Martí [Sun, 17 Mar 2019 22:45:30 +0000 (22:45 +0000)]
encoding/json: fix performance regression in the decoder
In golang.org/cl/145218, a feature was added where the JSON decoder
would keep track of the entire path to a field when reporting an
UnmarshalTypeError.
However, we all failed to check if this affected the benchmarks - myself
included, as a reviewer. Below are the numbers comparing the CL's parent
with itself, once it was merged:
name old time/op new time/op delta
CodeDecoder-8 12.9ms ± 1% 28.2ms ± 2% +119.33% (p=0.002 n=6+6)
name old speed new speed delta
CodeDecoder-8 151MB/s ± 1% 69MB/s ± 3% -54.40% (p=0.002 n=6+6)
name old alloc/op new alloc/op delta
CodeDecoder-8 2.74MB ± 0% 109.39MB ± 0% +3891.83% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
CodeDecoder-8 77.5k ± 0% 168.5k ± 0% +117.30% (p=0.004 n=6+5)
The reason why the decoder got twice as slow is because it now allocated
~40x as many objects, which puts a lot of pressure on the garbage
collector.
The reason is that the CL concatenated strings every time a nested field
was decoded. In other words, practically every field generated garbage
when decoded. This is hugely wasteful, especially considering that the
vast majority of JSON decoding inputs won't return UnmarshalTypeError.
Instead, use a stack of fields, and make sure to always use the same
backing array, to ensure we only need to grow the slice to the maximum
depth once.
The original CL also introduced a bug. The field stack string wasn't
reset to its original state when reaching "d.opcode == scanEndObject",
so the last field in a decoded struct could leak. For example, an added
test decodes a list of structs, and encoding/json before this CL would
fail:
got: cannot unmarshal string into Go struct field T.Ts.Y.Y.Y of type int
want: cannot unmarshal string into Go struct field T.Ts.Y of type int
To fix that, simply reset the stack after decoding every field, even if
it's the last.
Below is the original performance versus this CL. There's a tiny
performance hit, probably due to the append for every decoded field, but
at least we're back to the usual ~150MB/s.
name old time/op new time/op delta
CodeDecoder-8 12.9ms ± 1% 13.0ms ± 1% +1.25% (p=0.009 n=6+6)
name old speed new speed delta
CodeDecoder-8 151MB/s ± 1% 149MB/s ± 1% -1.24% (p=0.009 n=6+6)
name old alloc/op new alloc/op delta
CodeDecoder-8 2.74MB ± 0% 2.74MB ± 0% +0.00% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
CodeDecoder-8 77.5k ± 0% 77.5k ± 0% +0.00% (p=0.002 n=6+6)
Finally, make all of these benchmarks report allocs by default. The
decoder ones are pretty sensitive to generated garbage, so ReportAllocs
would have made the performance regression more obvious.
Tobias Klauser [Sun, 17 Mar 2019 21:39:42 +0000 (22:39 +0100)]
internal/bytealg: share code for IndexByte functions on arm
Move the shared code of IndexByte and IndexByteString into
indexbytebody. This will allow to implement optimizations (e.g.
for #29001) in a single function.
Elias Naur [Sun, 17 Mar 2019 15:01:27 +0000 (16:01 +0100)]
os: only fallback to root directory if $HOME fails for UserHomeDir
UserHomeDir always returns "/" for platforms where the home directory
is not always well defined. However, the user might set HOME before
running a Go program on those platforms and on at least iOS, HOME
is actually set to something useful (the root of the app specific
writable directory).
This CL changes UserHomeDir to use the root directory "/" only if
$HOME is empty.
The 5-12% decrease in performance on Equal/{6,9,15} are due to the
benchmarks splitting the bytes buffer in half. The b argument to Equal
then ends up being unaligned and thus the fast word-wise compare doesn't
kick in.
Elias Naur [Sun, 17 Mar 2019 15:07:25 +0000 (16:07 +0100)]
cmd/go,misc/ios: fix tests on iOS
Now that modules are always on, cmd/go tests require a valid
GOCACHE. However, on iOS where the go tool is not available, the
cmd/go test driver ends up setting GOCACHE to the empty string.
Fix it by falling back to the builtin default cache directory.
The iOS exec wrapper passes the environment variables to the app
on the device, including $HOME used for the default cache directory.
Skip $HOME to let the device specific and writable $HOME be used
instead.
Should fix cmd/go on the iOS builders that broke when GO111MODULE
defaulted to on.
Jay Conrod [Tue, 5 Mar 2019 19:53:03 +0000 (14:53 -0500)]
cmd/go: list directories in module cache replacements
"go list" has allowed listing directory paths to packages in the
module cache since CL 126715. This is sometimes necessary for tools
gathering package information about source files in imported packages.
With this change, we only allow directories in the module cache for
modules in the build list after replacements are applied. Previously,
we ignored replacements when expanding file system path patterns while
constructing the build list.
Fixes #29548
Change-Id: Ic7f89122c4656c8967c14545cb7117f98e89e721
Reviewed-on: https://go-review.googlesource.com/c/go/+/165381
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Austin Clements [Fri, 15 Mar 2019 17:29:40 +0000 (13:29 -0400)]
runtime: fix write barrier on wasm
The current wasm write barrier implementation incorrectly implements
the "deletion" part of the barrier. It correctly greys the new value
of the pointer, but rather than also greying the old value of the
pointer, it greys the object containing the slot (which, since the old
value was just overwritten, is not going to contain the old value).
This can lead to unmarked, reachable objects.
Often, this is masked by other marking activity, but one specific
sequence that can lead to an unmarked object because of this bug is:
1. Initially, GC is off, object A is reachable from just one pointer
in the heap.
2. GC starts and scans the stack of goroutine G.
3. G copies the pointer to A on to its stack and overwrites the
pointer to A in the heap. (Now A is reachable only from G's stack.)
4. GC finishes while A is still reachable from G's stack.
With a functioning deletion barrier, step 3 causes A to be greyed.
Without a functioning deletion barrier, nothing causes A to be greyed,
so A will be freed even though it's still reachable from G's stack.
Than McIntosh [Fri, 18 Jan 2019 20:16:11 +0000 (15:16 -0500)]
debug/dwarf: more graceful handling of unsupported types
Enhance the type decoder to do a better job handling unknown type
tags. DWARF has a number of type DIEs that this package doesn't handle
(things like "pointer to member" types in C++); avoid crashing for
such types, but instead return a placeholder "UnsupportedType" object
(this idea suggested by Austin). This provides a compromise between
implementing the entire kitchen sink and simply returning an error
outright on any unknown type DIE.
Clément Chigot [Wed, 20 Feb 2019 15:48:43 +0000 (16:48 +0100)]
cmd/link: fix trampoline generation for aix/ppc64
This commit fixes trampoline generation on aix/ppc64 which must use TOC
symbols.
It also adds a size to runtime.text.X symbols to prevent ld from moving
them, like runtime.text.
Change-Id: Ida033ec20ad8d7b7fb3faeb0ec4fa7bc4ce86b7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/164009
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Thu, 14 Mar 2019 01:56:37 +0000 (18:56 -0700)]
runtime: introduce and consistently use setNsec for timespec
The general code for setting a timespec value sometimes used set_nsec
and sometimes used a combination of set_sec and set_nsec. Standardize
on a setNsec function that takes a number of nanoseconds and splits
them up to set the tv_sec and tv_nsec fields. Consistently mark
setNsec as go:nosplit, since it has to be that way on some systems
including Darwin and GNU/Linux. Consistently use timediv on 32-bit
systems to help stay within split-stack limits on processors that
don't have a 64-bit division instruction.
Change-Id: I6396bb7ddbef171a96876bdeaf7a1c585a6d725b
Reviewed-on: https://go-review.googlesource.com/c/go/+/167389
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Matthew Dempsky [Thu, 14 Mar 2019 20:54:05 +0000 (13:54 -0700)]
cmd/compile: move Strongly Connected Components code into new file
This logic is used by the current escape analysis pass, but otherwise
logically independent. Move (unchanged) into a separate file to make
that clearer, and to make it easier to replace esc.go later.
Updates #23109.
Change-Id: Iec8c0c47ea04c0008165791731c11d9104d5a474
Reviewed-on: https://go-review.googlesource.com/c/go/+/167715 Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Wed, 12 Dec 2018 19:15:37 +0000 (11:15 -0800)]
cmd/compile: rewrite f(g()) for multi-value g() during typecheck
This is a re-attempt at CL 153841, which caused two regressions:
1. crypto/ecdsa failed to build with -gcflags=-l=4. This was because
when "t1, t2, ... := g(); f(t1, t2, ...)" was exported, we were losing
the first assignment from the call's Ninit field.
2. net/http/pprof failed to run with -gcflags=-N. This is due to a
conflict with CL 159717: as of that CL, package-scope initialization
statements are executed within the "init.ializer" function, rather
than the "init" function, and the generated temp variables need to be
moved accordingly too.
[Rest of description is as before.]
This CL moves order.go's copyRet logic for rewriting f(g()) into t1,
t2, ... := g(); f(t1, t2, ...) earlier into typecheck. This allows the
rest of the compiler to stop worrying about multi-value functions
appearing outside of OAS2FUNC nodes.
This changes compiler behavior in a few observable ways:
1. Typechecking error messages for builtin functions now use general
case error messages rather than unnecessarily differing ones.
2. Because f(g()) is rewritten before inlining, saved inline bodies
now see the rewritten form too. This could be addressed, but doesn't
seem worthwhile.
3. Most notably, this simplifies escape analysis and fixes a memory
corruption issue in esc.go. See #29197 for details.
Fixes #15992.
Fixes #29197.
Change-Id: I930b10f7e27af68a0944d6c9bfc8707c3fab27a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/166983
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>