Keith Randall [Fri, 28 Dec 2018 20:43:48 +0000 (12:43 -0800)]
cmd/compile: fix racewalk{enter,exit} removal
We can't remove race instrumentation unless there are no calls,
not just no static calls. Closure and interface calls also count.
The problem in issue 29329 is that there was a racefuncenter, an
InterCall, and a racefuncexit. The racefuncenter was removed, then
the InterCall was rewritten to a StaticCall. That prevented the
racefuncexit from being removed. That caused an imbalance in
racefuncenter/racefuncexit calls, which made the race detector barf.
Work involved in getting a stack trace is divided between
runtime.Callers and runtime.CallersFrames.
Before this CL, runtime.Callers returns a pc per runtime frame.
runtime.CallersFrames is responsible for expanding a runtime frame
into potentially multiple user frames.
After this CL, runtime.Callers returns a pc per user frame.
runtime.CallersFrames just maps those to user frame info.
Entries in the result of runtime.Callers are now pcs
of the calls (or of the inline marks), not of the instruction
just after the call.
Fixes #29007
Fixes #28640
Update #26320
Change-Id: I1c9567596ff73dc73271311005097a9188c3406f
Reviewed-on: https://go-review.googlesource.com/c/152537
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
This rule is problematic. 1<<(32-uint32(d)) <= int32(c) meant to
say that it is true if c is greater than the largest possible
value of the right shift. But when d==1, 1<<(32-1) is negative
and results in the wrong comparison.
Max Ushakov [Fri, 21 Dec 2018 14:53:33 +0000 (17:53 +0300)]
time: return ENOENT if a zoneinfo zip file is not found
Updates #20969
Change-Id: Ibcf0bf932d5b1de67c22c63dd8514ed7a5d198fb
Reviewed-on: https://go-review.googlesource.com/c/155538
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
LE Manh Cuong [Fri, 21 Dec 2018 04:21:02 +0000 (11:21 +0700)]
path/filepath: walkSymlinks: return correct error for file with trailing slash
Rather than return os.ErrNotExist for /path/to/existing_file/,
walkSymLinks now returns syscall.ENOTDIR.
This is consistent with behavior of os.Lstat.
Fixes #29372
Change-Id: Id5c471d901db04b2f35d60f60a81b2a0be93cae9
Reviewed-on: https://go-review.googlesource.com/c/155597
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Martí [Thu, 20 Dec 2018 23:04:18 +0000 (00:04 +0100)]
cmd/go: add regression test for cryptic vcs errors
On Go 1.11.x, if one ran 'go build' on a main package within a module,
while a needed vcs program like git was missing, a confusing error would
show up:
build testmod: cannot find module for path rsc.io/quote
The error should instead point at the source of the problem, which is
the missing vcs program. Thankfully, Go 1.12 doesn't have this bug, even
though it doesn't seem like the bug was fixed directly and
intentionally.
To ensure that this particular edge case isn't broken again, add a
regression test. Piggyback on mod_vcs_missing, since it already requires
a missing vcs program and network access.
I double-checked that Go 1.11 fails this test via /usr/bin/go, which is
1.11.3 on my system:
$ PATH=~/tip/bin go test -v -run Script/mod_vcs_missing
[...]
> exec /usr/bin/go build
[stderr]
build m: cannot find module for path launchpad.net/gocheck
Fixes #28948.
Change-Id: Iff1bcf77d9f7c11d15935cb87d6f58d7981d33d2
Reviewed-on: https://go-review.googlesource.com/c/155537
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Keith Randall [Sat, 22 Dec 2018 00:36:45 +0000 (16:36 -0800)]
cmd/compile: pad zero-sized stack variables
If someone takes a pointer to a zero-sized stack variable, it can
be incorrectly interpreted as a pointer to the next object in the
stack frame. To avoid this, add some padding after zero-sized variables.
We only need to pad if the next variable in memory (which is the
previous variable in the order in which we allocate variables to the
stack frame) has pointers. If the next variable has no pointers, it
won't hurt to have a pointer to it.
Because we allocate all pointer-containing variables before all
non-pointer-containing variables, we should only have to pad once per
frame.
Fixes #24993
Change-Id: Ife561cdfdf964fdbf69af03ae6ba97d004e6193c
Reviewed-on: https://go-review.googlesource.com/c/155698
Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Keith Randall [Fri, 21 Dec 2018 23:41:28 +0000 (15:41 -0800)]
cmd/compile: fix line number for implicitly declared method expressions
Method expressions where the method is implicitly declared have no
line number. The Error method of the built-in error type is one such
method. We leave the line number at the use of the method expression
in this case.
Fixes #29389
Change-Id: I29c64bb47b1a704576abf086599eb5af7b78df53
Reviewed-on: https://go-review.googlesource.com/c/155639
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jay Conrod [Fri, 21 Dec 2018 18:47:20 +0000 (13:47 -0500)]
cmd/go: use cached source files in "go list -find -compiled"
When "go list" is invoked with -find, it clears the list of imports
for each package matched on the command line. This affects action IDs,
since they incorporate dependencies' action IDs. Consequently, the
build triggered by -compiled won't find sources cached by
"go build".
We can still safely cache compiled sources from multiple runs of
"go list -find -compiled" though, since cgo generated sources are not
affected by imported dependencies. This change adds a second look into
the cache in this situation.
Fixes #29371
Change-Id: Ia0ae5a403ab5d621feaa16f521e6a65ac0ae6d9a
Reviewed-on: https://go-review.googlesource.com/c/155481 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Jay Conrod [Fri, 21 Dec 2018 17:23:18 +0000 (12:23 -0500)]
cmd/go: fix -n output in runtime/internal/atomic
When building runtime/internal/atomic, the toolchain writes a symabis2
file. This file is read back in, filtered, and appended to the symabis
file. This breaks with -n, since the symabis2 file is never written.
With this change, when -n is used, an equivalent "grep" command is
printed instead. The output for -x is unchanged.
Fixes #29346
Change-Id: Id25e06e06364fc6689e71660d000f09c649c4f0c
Reviewed-on: https://go-review.googlesource.com/c/155480 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Michael Anthony Knyszek [Thu, 20 Dec 2018 20:21:45 +0000 (20:21 +0000)]
runtime: skip TestLockOSThreadAvoidsStatePropagation if one can't unshare
This change splits a testprog out of TestLockOSThreadExit and makes it
its own test. Then, this change makes the testprog exit prematurely with
a special message if unshare fails with EPERM because not all of the
builders allow the user to call the unshare syscall.
Also, do some minor cleanup on the TestLockOSThread* tests.
Matthew Dempsky [Thu, 20 Dec 2018 19:10:06 +0000 (11:10 -0800)]
cmd/compile: fix ICE due to bad rune width
It was possible that
var X interface{} = 'x'
could cause a compilation failure due to having not calculated rune's
width yet. typecheck.go normally calculates the width of things, but
it doesn't for implicit conversions to default type. We already
compute the width of all of the standard numeric types in universe.go,
but we failed to calculate it for the rune alias type. So we could
later crash if the code never otherwise explicitly mentioned 'rune'.
While here, explicitly compute widths for 'byte' and 'error' for
consistency.
Kevin Burke [Tue, 25 Sep 2018 16:22:09 +0000 (09:22 -0700)]
os: clearer doc for Interrupt
I was confused by the juxtaposition of os.Interrupt docs, which are
"guaranteed to exist on all platforms" in one sentence and then
"not implemented" in the next sentence. Reading the code reveals
"not implemented" refers specifically to the implementation of
os.Process.Signal on Windows, not to the os.Interrupt variable itself.
Reword the doc to make this distinction clearer.
Fixes #27854.
Change-Id: I5fe7cddea61fa1954cef2006dc51b8fa8ece4d6e
Reviewed-on: https://go-review.googlesource.com/c/137336 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Keith Randall [Thu, 13 Dec 2018 17:31:21 +0000 (09:31 -0800)]
cmd/compile: ignore out-of-bounds reads from readonly constants
Out-of-bounds reads of globals can happen in dead code. For code
like this:
s := "a"
if len(s) == 3 {
load s[0], s[1], and s[2]
}
The out-of-bounds loads are dead code, but aren't removed yet
when lowering. We need to not panic when compile-time evaluating
those loads. This can only happen for dead code, so the result
doesn't matter.
Fixes #29215
Change-Id: I7fb765766328b9524c6f2a1e6ab8d8edd9875097
Reviewed-on: https://go-review.googlesource.com/c/154057
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com>
Tobias Klauser [Thu, 20 Dec 2018 08:38:37 +0000 (09:38 +0100)]
doc/go1.12: fix GOARCH value in Syscall18 link
Currently the link works also with the non-existing GOARCH armd64, but
let's correct in anyhow.
Change-Id: Ida647b8f9dd2f8460b019f5a23759f10a6da8e60
Reviewed-on: https://go-review.googlesource.com/c/155277 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 18 Dec 2018 23:10:38 +0000 (15:10 -0800)]
net: don't accept timeouts in TestUDPZeroBytePayload
Before this CL we accepted timeouts in TestUDPZeroBytePayload to avoid
flakiness and because, according to CL 9194, the test didn't work on
some platforms. On Windows, before CL 132781, the read would always
timeout, and so since the test accepted timeouts it would pass
incorrectly. CL 132781 fixed Windows, and changed the test to not
accept timeouts in the ReadFrom case.
However, the timeout was short, and so on a loaded system the Read
might timeout not due to an error in the code, but just because the
read was not delivered. So ignoring timeouts made the test flaky, as
reported in issue #29225.
This CL tries to get to a better state by increasing the timeout to a
large value and not permitting timeouts at all. If there are systems
where the test fails, we will need to explicitly skip the test on
those systems.
Fixes #29225
Change-Id: I26863369898a69cac866b34fcb5b6ffbffab31f6
Reviewed-on: https://go-review.googlesource.com/c/154759
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Alex Brainman [Mon, 17 Dec 2018 09:39:48 +0000 (20:39 +1100)]
os: adjust TempDir for Z:\
If TMP environment variable is set to Z:\, TempDir returns Z:.
But Z: refers to current directory on Z:, while Z:\ refers to root
directory on Z:. Adjust TempDir to return Z:\.
Jordan Rhee [Wed, 19 Dec 2018 00:41:57 +0000 (16:41 -0800)]
runtime: use QPC to implement cputicks() on windows/arm
Tracing uses cputicks() to generate trace event timestamps. cputicks()
is expected to be a high resolution clock source. On Windows/ARM,
call QueryPerformanceCounter() which is the highest resolution clock
source available.
Michael Anthony Knyszek [Fri, 7 Dec 2018 00:07:43 +0000 (00:07 +0000)]
runtime: don't clear lockedExt on locked M when G exits
When a locked M has its G exit without calling UnlockOSThread, then
lockedExt on it was getting cleared. Unfortunately, this meant that
during P handoff, if a new M was started, it might get forked (on
most OSs besides Windows) from the locked M, which could have kernel
state attached to it.
To solve this, just don't clear lockedExt. At the point where the
locked M has its G exit, it will also exit in accordance with the
LockOSThread API. So, we can safely assume that it's lockedExt state
will no longer be used. For the case of the main thread where it just
gets wedged instead of exiting, it's probably better for it to keep
the locked marker since it more accurately represents its state.
Michael Anthony Knyszek [Wed, 19 Dec 2018 17:32:22 +0000 (17:32 +0000)]
runtime: disable TestArenaCollision on Darwin in race mode
This change disables the test TestArenaCollision on Darwin in race mode
to deal with the fact that Darwin 10.10 must use MAP_FIXED in race mode
to ensure we retain our heap in a particular portion of the address
space which the race detector needs. The test specifically checks to
make sure a manually mapped region's space isn't re-used, which is
definitely possible with MAP_FIXED because it replaces whatever mapping
already exists at a given address.
This change then also makes it so that MAP_FIXED is only used in race
mode and on Darwin, not all BSDs, because using MAP_FIXED breaks this
test for FreeBSD in addition to Darwin.
Jordan Rhee [Wed, 19 Dec 2018 00:32:09 +0000 (16:32 -0800)]
Revert "runtime: use QPC for nanotime and time.now on windows/arm"
This reverts change https://golang.org/cl/154758.
Restore the previous implementations of nanotime and time.now, which
are sufficiently high resolution and more efficient than
QueryPerformanceCounter. The intent of the change was to improve
resolution of tracing timestamps, but the change was overly broad
as it was only necessary to fix cputicks(). cputicks() is fixed in
a subsequent change.
Agniva De Sarker [Wed, 19 Dec 2018 05:03:42 +0000 (10:33 +0530)]
doc/go1.12: fix minor grammatical error
Change-Id: I767bf77aeab62f2d42239fac9d601a8e04fe860f
Reviewed-on: https://go-review.googlesource.com/c/154957 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Tobias Klauser [Wed, 19 Dec 2018 08:36:11 +0000 (09:36 +0100)]
doc/go1.12: fix typos and code formatting
Fix two typos and don't indent the go vet example.
Change-Id: Iccec56ca5decfbae45547a00115500ed13b703e1
Reviewed-on: https://go-review.googlesource.com/c/154721 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I3c8ba5fdb05b6b1324648622656cc10071c70a34
Reviewed-on: https://go-review.googlesource.com/c/154997 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Tue, 18 Dec 2018 20:04:53 +0000 (20:04 +0000)]
runtime: avoid write barrier in startpanic_m
startpanic_m could be called correctly in a context where there's a
valid G, a valid M, but no P, for example in a signal handler which
panics. Currently, startpanic_m has write barriers enabled because
write barriers are permitted if a G's M is dying. However, all the
current write barrier implementations assume the current G has a P.
Therefore, in this change we disable write barriers in startpanic_m,
remove the only pointer write which clears g.writebuf, and fix up gwrite
to ignore the writebuf if the current G's M is dying, rather than
relying on it being nil in the dying case.
Jordan Rhee [Tue, 18 Dec 2018 20:54:23 +0000 (12:54 -0800)]
runtime: use QPC for nanotime and time.now on windows/arm
The previous implementation of nanotime and time.now used a time source
that was updated on the system clock tick, which has a maximum
resolution of about 1ms. On 386 and amd64, this time source maps to
the system performance counter, so has much higher resolution.
On ARM, use QueryPerformanceCounter() to get a high resolution timestamp.
Elias Naur [Tue, 18 Dec 2018 20:32:48 +0000 (21:32 +0100)]
doc/go1.12: note that syscall.Getdirentries is no longer supported on iOS
Change-Id: I4277f4130b460b42c5b51fd5a5e07f6c0e62163b
Reviewed-on: https://go-review.googlesource.com/c/154720 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Kevin Burke [Mon, 17 Dec 2018 00:37:07 +0000 (00:37 +0000)]
os: show how to print permission bits in octal
Permission bits are most commonly viewed in string form (rwx-- etc) or
in octal form (0755), but the latter is relatively rare in Go.
Demonstrate how to print a FileMode in readable octal format.
Change-Id: I41feb801bcecb5077d4eabafdea27c149fc179a1
Reviewed-on: https://go-review.googlesource.com/c/154423
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Michael Anthony Knyszek [Tue, 11 Dec 2018 22:03:04 +0000 (22:03 +0000)]
runtime: call mmap with MAP_FIXED on BSDs in race mode
This change makes it so that reserving more of the address space for the
heap calls mmap with MAP_FIXED in race mode. Race mode requires certain
guarantees on where the heap is located in the address space, and on
Darwin 10.10 it appears that the kernel may end up ignoring the hint
quite often (#26475). Using MAP_FIXED is relatively OK in race mode
because nothing else should be mapped in the memory region provided by
the initial hints.
Cherry Zhang [Sat, 17 Nov 2018 18:58:38 +0000 (13:58 -0500)]
cmd/dist: enable race detector test on Linux/ARM64
CL 138675 added the race detector support on Linux/ARM64, but it
didn't enable the race detector tests in cmd/dist (therefore in
all.bash). Enable them.
Updates #28848
Change-Id: I4306dad2fb4167021d568436076b9f535d7f6e07
Reviewed-on: https://go-review.googlesource.com/c/149967
Run-TryBot: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Tue, 18 Dec 2018 12:00:26 +0000 (13:00 +0100)]
runtime: fix backtrace during C syscalls for aix/ppc64
This commit fixes backtrace if a crash or an exit signal is received
during a C syscall on aix/ppc64.
This is similar to Solaris, Darwin or Windows implementation.
Change-Id: I6040c0b1577a9f5b298f58bd4ee6556258a135ef
Reviewed-on: https://go-review.googlesource.com/c/154718
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Clément Chigot [Mon, 26 Nov 2018 10:14:09 +0000 (11:14 +0100)]
cmd/link: optimize access to data symbols for aix/ppc64
This commit changes the second instruction used to retrieve a symbol on
aix/ppc64 if it is in .data or .bss section.
The previous version always retrieves the symbol address via a load on
its TOC symbol. However, as the TOC is also in .data, the symbol's address
is close enough to be fetched directly and the load instruction can be
replaced by an addi.
Change-Id: Iaf2aa5953b99271361510c69a5ced3371f6c6c20
Reviewed-on: https://go-review.googlesource.com/c/151201
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Austin Clements [Tue, 18 Dec 2018 01:57:34 +0000 (20:57 -0500)]
runtime: flush on every write barrier while debugging
Currently, we flush the write barrier buffer on every write barrier
once throwOnGCWork is set, but not during the mark completion
algorithm itself. As seen in recent failures like
by the time we actually catch a late gcWork put, the write barrier
buffer is full-size again.
As a result, we're probably not catching the actual problematic write
barrier, which is probably somewhere in the buffer.
Fix this by using the gcWork pause generation to also keep the write
barrier buffer small between the mark completion flushes it and when
mark completion is done.
Joel Sing [Sun, 16 Dec 2018 14:18:51 +0000 (01:18 +1100)]
runtime,cmd/dist,cmd/link: add cgo support on openbsd/arm
Add support for cgo on openbsd/arm.The gcc shipped with base OpenBSD armv7
is old/inadequate, so use clang by default.
Change-Id: I945a26d369378952d357727718e69249411e1127
Reviewed-on: https://go-review.googlesource.com/c/154381
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Mon, 17 Dec 2018 22:42:29 +0000 (14:42 -0800)]
cmd/compile: increase nesting depth limit for type descriptors
The formatting routines for types use a depth limit as primitive
mechanism to detect cycles. For now, increase the limit from 100
to 250 and file #29312 so we don't drop this on the floor.
Also, adjust some fatal error messages elsewhere to use
better formatting.
Michael Anthony Knyszek [Thu, 8 Nov 2018 20:06:58 +0000 (20:06 +0000)]
runtime: allocate from free and scav fairly
This change modifies the behavior of span allocations to no longer
prefer the free treap over the scavenged treap.
While there is an additional cost to allocating out of the scavenged
treap, the current behavior of preferring the unscavenged spans can
lead to unbounded growth of a program's virtual memory footprint.
In small programs (low # of Ps, low resident set size, low allocation
rate) this behavior isn't really apparent and is difficult to
reproduce.
However, in relatively large, long-running programs we see this
unbounded growth in free spans, and an unbounded amount of heap
growths.
It still remains unclear how this policy change actually ends up
increasing the number of heap growths over time, but switching the
policy back to best-fit does indeed solve the problem.
Change-Id: Ibb88d24f9ef6766baaa7f12b411974cc03341e7b
Reviewed-on: https://go-review.googlesource.com/c/148979
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Michael Anthony Knyszek [Mon, 26 Nov 2018 23:56:35 +0000 (23:56 +0000)]
runtime: add iterator abstraction for mTreap
This change adds the treapIter type which provides an iterator
abstraction for walking over an mTreap. In particular, the mTreap type
now has iter() and rev() for iterating both forwards (smallest to
largest) and backwards (largest to smallest). It also has an erase()
method for erasing elements at the iterator's current position.
For #28479.
While the expectation is that this change will slow down Go programs,
the impact on Go1 and Garbage is negligible.
Clément Chigot [Fri, 23 Nov 2018 14:12:04 +0000 (15:12 +0100)]
cmd/link: move XCOFF data addresses to an unreachable segment
This commit move data addresses to 0x200000000 for XCOFF executables.
.data and .bss must always be position-independent on AIX. This
modification allows to detect more easily if they aren't, as segfault
will be triggered.
Change-Id: Ied7a5b72b9f4ff9f870a1626cf07c48110635e62
Reviewed-on: https://go-review.googlesource.com/c/151040
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Chase [Mon, 17 Dec 2018 17:01:56 +0000 (12:01 -0500)]
cmd/compile: prevent double-walk of switch for OPRINT/OPRINTN
When a println arg contains a call to an inlineable function
that itself contains a switch, that switch statement will be
walked twice, once by the walkexprlist formerly in the
OPRINT/OPRINTN case, then by walkexprlistcheap in walkprint.
Remove the first walkexprlist, it is not necessary.
walkexprlist =
s[i] = walkexpr(s[i], init)
walkexprlistcheap = {
s[i] = cheapexpr(n, init)
s[i] = walkexpr(s[i], init)
}
Seems like this might be possible in other places, i.e.,
calls to inlineable switch-containing functions.
See also #25776.
Fixes #29220.
Change-Id: I3781e86aad6688711597b8bee9bc7ebd3af93601
Reviewed-on: https://go-review.googlesource.com/c/154497
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Michael Anthony Knyszek [Tue, 11 Dec 2018 18:26:42 +0000 (18:26 +0000)]
runtime: fix sysUsed for Windows
sysUsed on Windows cares about the result from the VirtualAlloc syscall
returning exactly the address that was passed to it. However,
VirtualAlloc aligns the address its given to the kernel's allocation
granularity, so the returned address may not be the same.
Note that this wasn't an issue in the past because we only sysUsed
regions owned by spans, and spans are always a multiple of 8K, which
is a multiple of the allocation granularity on most Windows machines.
Change-Id: I3f5ccd63c6bbbd8b7995945ecedee17573b31667
Reviewed-on: https://go-review.googlesource.com/c/153677
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Rick Hudson <rlh@golang.org>
Austin Clements [Mon, 17 Dec 2018 19:39:05 +0000 (14:39 -0500)]
runtime: capture pause stack for late gcWork put debugging
This captures the stack trace where mark completion observed that each
P had no work, and then dumps this if that P later discovers more
work. Hopefully this will help bound where the work was created.
Austin Clements [Mon, 17 Dec 2018 19:23:56 +0000 (14:23 -0500)]
runtime: make traceback indicate whether _defer was just allocated
Many of the crashes observed in #27993 involve committing the new
_defer object at the end of newdefer. It would be helpful to know if
the _defer was just allocated or was retrieved from the defer pool. In
order to indicate this in the traceback, this CL duplicates the tail
of newdefer so that the PC/line number will tell us whether d is new
or not.
Austin Clements [Mon, 17 Dec 2018 19:17:20 +0000 (14:17 -0500)]
runtime: record extra information in throwOnGCWork crashes
Currently we only know the slot address and the value being written in
the throwOnGCWork crash tracebacks, and we have to infer the old value
from what's dumped by gcWork.checkPut. Sometimes these old values
don't make sense, like when we see a write of a nil pointer to a
freshly-allocated object, yet we observe marking a value (where did
that pointer come from?).
This CL adds the old value of the slot and the first two pointers in
the buffer to the traceback.
Austin Clements [Mon, 17 Dec 2018 20:26:36 +0000 (15:26 -0500)]
doc/go1.12: updates for runtime and compiler
Change-Id: Ifb16fd28105efd05cebbd615b52e45330b77cede
Reviewed-on: https://go-review.googlesource.com/c/154600 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Mon, 17 Dec 2018 19:33:42 +0000 (11:33 -0800)]
cmd/compile: generate interface method expression wrapper for error.Error
A prior optimization (https://golang.org/cl/106175) removed the
generation of unnecessary method expression wrappers, but also
eliminated the generation of the wrapper for error.Error which
was still required.
Austin Clements [Mon, 17 Dec 2018 18:45:14 +0000 (13:45 -0500)]
runtime: poison the write barrier buffer during flushing
Currently we reset the write barrier buffer before processing the
pointers in it. As a result, if there were any write barriers in the
code that processes the buffer, it would corrupt the write barrier
buffer and cause us to mark objects without later scanning them.
As far as I can tell, this shouldn't be happening, but rather than
relying on hope (and incomplete static analysis), this CL changes
wbBufFlush1 to poison the write barrier buffer while processing it,
and only reset it once it's done.
Updates #27993. (Unlike many of the other changes for this issue,
there's no need to roll back this CL. It's a good change in its own
right.)
Alex Brainman [Sat, 15 Dec 2018 07:52:26 +0000 (18:52 +1100)]
os: make Stat work on FAT file system
It appears calling GetFileInformationByHandleEx with
FILE_ATTRIBUTE_TAG_INFO fails on FAT file system. FAT does not
support symlinks, so assume there are no symlnks when
GetFileInformationByHandleEx returns ERROR_INVALID_PARAMETER.
Fixes #29214
Change-Id: If2d9f3288bd99637681ab5fd4e4581c77b578a69
Reviewed-on: https://go-review.googlesource.com/c/154377
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joel Sing [Sat, 15 Dec 2018 14:07:58 +0000 (01:07 +1100)]
cmd/dist: re-enable VFPv3 on openbsd/arm
The OpenBSD armv7 port has working VFPv3 these days - re-enable the VFP
detection code so that GOARM=7 is used by default on openbsd/arm.
Change-Id: I0271d81c048d2d55becd2803c19e5f1542076357
Reviewed-on: https://go-review.googlesource.com/c/154378 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Austin Clements [Fri, 14 Dec 2018 16:55:44 +0000 (11:55 -0500)]
runtime: fix hangs in TestDebugCall*
This fixes a few different issues that led to hangs and general
flakiness in the TestDebugCall* tests.
1. This fixes missing wake-ups in two error paths of the SIGTRAP
signal handler. If the goroutine was in an unknown state, or if
there was an unknown debug call status, we currently don't wake the
injection coordinator. These are terminal states, so this resulted
in a hang.
2. This adds a retry if the target goroutine is in a transient state
that prevents us from injecting a call. The most common failure
mode here is that the target goroutine is in _Grunnable, but this
was previously masked because it deadlocked the test.
3. Related to 2, this switches the "ready" signal from the target
goroutine from a blocking channel send to a non-blocking channel
send. This makes it much less likely that we'll catch this
goroutine while it's in the runtime performing that send.
4. This increases GOMAXPROCS from 2 to 8 during these tests. With the
current setting of 2, we can have at most the non-preemptible
goroutine we're injecting a call in to and the goroutine that's
trying to make it exit. If anything else comes along, it can
deadlock. One particular case I observed was in TestDebugCallGC,
where runtime.GC() returns before the forEachP that prepares
sweeping on all goroutines has finished. When this happens, the
forEachP blocks on the non-preemptible loop, which means we now
have at least three goroutines that need to run.
Elias Naur [Sat, 15 Dec 2018 16:03:37 +0000 (17:03 +0100)]
cmd/fix,cmd/cgo,misc/cgo: map the EGLDisplay C type to uintptr in Go
Similar to to macOS' CF* types and JNI's jobject and derived types,
the EGLDisplay type is declared as a pointer but can contain
non-pointers (see #27054).
Fix it the same way: map EGLDisplay to uintptr in Go.
Fixes #27054
RELNOTE=yes
Change-Id: I6136f8f8162687c5493b30ed324e29efe55a8fd7
Reviewed-on: https://go-review.googlesource.com/c/154417
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Shenghou Ma [Fri, 14 Dec 2018 04:55:22 +0000 (23:55 -0500)]
cmd/link: fix in-package syso linking
CL 146297 ignored archive members with short names that don't have
the .o suffix, however, it also ignored .syso files as well.
This change restores the original .syso behavior and adds a test.
As the test is basically following a shell script, we make use of
the existing cmd/go/testdata/script framework. To support running
C compiler in the script, we added a `cc` command, which runs the
C compiler along with correct platform specific arguments.
Fixes #29253.
Change-Id: If8520151c4d6a74ab9fe84d34bff9a4480688815
Reviewed-on: https://go-review.googlesource.com/c/154109
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Alan Donovan <adonovan@google.com>
Bryan C. Mills [Fri, 14 Dec 2018 21:59:04 +0000 (16:59 -0500)]
cmd/go/internal/cache: save more data from DefaultDir
cmd/go/main.go sets GOCACHE explicitly, so if we don't save some
metadata about how DefaultDir arrived at its answer we will be unable
to reconstruct it later.
Bryan C. Mills [Fri, 14 Dec 2018 02:42:33 +0000 (21:42 -0500)]
cmd/go/internal/get: move wildcard-trimming to before CheckImportPath
Previously, RepoRootForImportPath trimmed certain "..." wildcards from
package patterns (even though its name suggests that the argument must
be an actual import path). It trimmed at the first path element that
was literally "..." (although wildcards in general may appear within a
larger path element), and relied on a subsequent check in
RepoRootForImportPath to catch confusing resolutions.
However, that causes 'go get' with wildcard patterns in fresh paths to
fail as of CL 154101: a wildcard pattern is not a valid import path,
and fails the path check. (The existing Test{Vendor,Go}Get* packages
in go_test.go and vendor_test.go catch the failure, but they are all
skipped when the "-short" flag is set — including in all.bash — and we
had forgotten to run them separately.)
We now trim the path before any element that contains a wildcard, and
perform the path check (and repo resolution) on only that prefix. It
is possible that the expanded path after fetching the repo will be
invalid, but a repository can contain directories that are not valid
import paths in general anyway.
Clément Chigot [Thu, 13 Dec 2018 12:37:57 +0000 (13:37 +0100)]
syscall: remove linknames to runtime symbols for aix/ppc64
Replaces //go:linkname by assembly functions for syscall
functions on aix/ppc64.
Since the new runtime internal ABI, this was triggering an error if
syscall.Syscall6 was called by others packages like x/sys/unix.
This commit should remove every future occurences of this problem.
Martin Möhrmann [Thu, 13 Dec 2018 16:55:52 +0000 (17:55 +0100)]
cmd/compile: fix length overflow when appending elements to a slice
Instead of testing len(slice)+numNewElements > cap(slice) use
uint(len(slice)+numNewElements) > uint(cap(slice)) to test
if a slice needs to be grown in an append operation.
This prevents a possible overflow when len(slice) is near the maximum
int value and the addition of a constant number of new elements
makes it overflow and wrap around to a negative number which is
smaller than the capacity of the slice.
Appending a slice to a slice with append(s1, s2...) already used
a uint comparison to test slice capacity and therefore was not
vulnerable to the same overflow issue.
Fixes: #29190
Change-Id: I41733895838b4f80a44f827bf900ce931d8be5ca
Reviewed-on: https://go-review.googlesource.com/c/154037
Run-TryBot: Martin Möhrmann <moehrmann@google.com> Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Filippo Valsorda [Wed, 5 Dec 2018 03:23:22 +0000 (22:23 -0500)]
crypto/x509: limit number of signature checks for each verification
That number grows quadratically with the number of intermediate
certificates in certain pathological cases (for example if they all have
the same Subject) leading to a CPU DoS. Set a fixed budget that should
fit all real world chains, given we only look at intermediates provided
by the peer.
The algorithm can be improved, but that's left for follow-up CLs:
* the cache logic should be reviewed for correctness, as it seems to
override the entire chain with the cached one
* the equality check should compare Subject and public key, not the
whole certificate
* certificates with the right SKID but the wrong Subject should not
be considered, and in particular should not take priority over
certificates with the right Subject
Bryan C. Mills [Fri, 30 Nov 2018 20:06:40 +0000 (15:06 -0500)]
cmd/go: reject 'get' of paths containing leading dots or unsupported characters
On some platforms, directories beginning with dot are treated as
hidden files, and filenames containing unusual characters can be
confusing for users to manipulate (and delete).
Joe Tsai [Fri, 16 Feb 2018 22:52:10 +0000 (14:52 -0800)]
go/doc: simplify and robustify link detection logic
To fix #5043, we added logic to allow balanced pairs of parenthesis
so that we could match URLs like:
http://example.com/some_resource(foo)
Howewer, such logic breaks when parsing something like the following:
art by [https://example.com/person][Person Name]].
such that the following is considered the link:
https://example.com/person][Person
Since the logic added in #5043 was just a heuristic, we adjust
the heuristic that in addition to requiring balanced pairs,
the first parenthesis must be an opening one.
For further robustness, we apply this heuristic to
parenthesis, braces, and brackets.
Fixes #22285
Change-Id: I23b728a644e35ce3995b05a79129cad2c1e3b1ce
Reviewed-on: https://go-review.googlesource.com/c/94876
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>