Austin Clements [Wed, 26 Feb 2020 20:12:33 +0000 (15:12 -0500)]
[release-branch.go1.14] runtime: fix rounding in materializeGCProg
materializeGCProg allocates a temporary buffer for unrolling a GC
program. Unfortunately, when computing the size of the buffer, it
rounds *down* the number of bytes needed to store bitmap before
rounding up the number of pages needed to store those bytes. The fact
that it rounds up to pages usually mitigates the rounding down, but
the type from #37470 exists right on the boundary where this doesn't
work:
On 64-bit, this GC bitmap is exactly 8 KiB of zeros, followed by three
one bits. Hence, this needs 8193 bytes of storage, but the current
math in materializeGCProg rounds *down* the three one bits to 8192
bytes. Since this is exactly pageSize, the next step of rounding up to
the page size doesn't mitigate this error, and materializeGCProg
allocates a buffer that is one byte too small. runGCProg then writes
one byte past the end of this buffer, causing either a segfault (if
you're lucky!) or memory corruption.
Michael Pratt [Thu, 26 Mar 2020 19:10:21 +0000 (15:10 -0400)]
[release-branch.go1.14] runtime/pprof: increment fake overflow record PC
gentraceback generates PCs which are usually following the CALL
instruction. For those that aren't, it fixes up the PCs so that
functions processing the output can unconditionally decrement the PC.
runtime_expandInlineFrames does this unconditional decrement when
looking up the function. However, the fake stack frame generated for
overflow records fails to meet the contract, and decrementing the PC
results in a PC in the previous function. If that function contains
inlined call, runtime_expandInlineFrames will not short-circuit and will
panic trying to look up a PC that doesn't exist.
Note that the added test does not fail at HEAD. It will only fail (with
a panic) if the function preceeding lostProfileEvent contains inlined
function calls. At the moment (on linux/amd64), that is
runtime/pprof.addMaxRSS, which does not.
Fixes #38118
Change-Id: Iad0819f23c566011c920fd9a5b1254719228da0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/225661 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 4a8b9bd2646a5b297197ffd1c412ef6afebe5c0d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/226077
Tobias Klauser [Sun, 29 Mar 2020 22:38:09 +0000 (00:38 +0200)]
[release-branch.go1.14] cmd/cgo, misc/cgo: only cache anonymous struct typedefs with parent name
CL 181857 broke the translation of certain C types using cmd/cgo -godefs
because it stores each typedef, array and qualified type with their
parent type name in the translation cache.
Fix this by only considering the parent type for typedefs of anonymous
structs which is the only case where types might become ambiguous.
Fixes #37622
Change-Id: I301a749ec89585789cb0d213593bb8b7341beb88
Reviewed-on: https://go-review.googlesource.com/c/go/+/226341
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit a265c2c448497fcee1633d2e2b912da52ea22d3c)
Reviewed-on: https://go-review.googlesource.com/c/go/+/226497
[release-branch.go1.14] os/exec: use environment variables for user token when present
Builds upon the changes from #32000 which supported sourcing environment
variables for a new process from the environment of a Windows user token
when supplied.
But due to the logic of os/exec, the Env field of a process was
always non-nil when it reached that change.
This change moves the logic up to os/exec, specifically when
os.ProcAttr is being built for the os.StartProcess call, this
ensures that if a user token has been supplied and no Env slice has
been provided on the command it will be sourced from the user's
environment.
If no token is provided, or the program is compiled for any other
platform than Windows, the default environment will be sourced from
syscall.Environ().
For #35314
Fixes #37471
Change-Id: I4c1722e90b91945eb6980d5c5928183269b50487
GitHub-Last-Rev: 32216b7291418f9285147a93ed6d0ba028f94ef2
GitHub-Pull-Request: golang/go#37402
Reviewed-on: https://go-review.googlesource.com/c/go/+/220587
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit e0c3ded337e95ded40eb401e7d9e74716e3a445f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/226281
Michael Anthony Knyszek [Wed, 18 Mar 2020 15:09:40 +0000 (15:09 +0000)]
[release-branch.go1.14] runtime: ensure minTriggerRatio never exceeds maxTriggerRatio
Currently, the capping logic for the GC trigger ratio is such that if
gcpercent is low, we may end up setting the trigger ratio far too high,
breaking the promise of SetGCPercent and GOGC has a trade-off knob (we
won't start a GC early enough, and we will use more memory).
This change modifies the capping logic for the trigger ratio by scaling
the minTriggerRatio with gcpercent the same way we scale
maxTriggerRatio.
For #37927.
Fixes #37928.
Change-Id: I2a048c1808fb67186333d3d5a6bee328be2f35da
Reviewed-on: https://go-review.googlesource.com/c/go/+/223937
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
(cherry picked from commit d1ecfcc1e8baa0bb3a9fb504e8c14125a69139ba)
Reviewed-on: https://go-review.googlesource.com/c/go/+/225637 Reviewed-by: David Chase <drchase@google.com>
Bryan C. Mills [Thu, 26 Mar 2020 02:24:44 +0000 (22:24 -0400)]
[release-branch.go1.14] cmd/go: do not append to the global cfg.OrigEnv slice
Appending to a global slice is only safe if its length is already
equal to its capacity. That property is not guaranteed for slices in
general, and empirically does not hold for this one.
This is a minimal fix to make it easier to backport.
A more robust cleanup of the base.EnvForDir function will be sent in a
subsequent CL.
Fixes #38083
Updates #38077
Change-Id: I731d5bbd0e516642c2cf43e713eeea15402604e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/225577
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
(cherry picked from commit bfb1342a40216cba0ff5ae3a1b102823b7603068)
Reviewed-on: https://go-review.googlesource.com/c/go/+/225659
Michael Anthony Knyszek [Wed, 25 Mar 2020 16:41:20 +0000 (16:41 +0000)]
[release-branch.go1.14] runtime: prevent preemption while timer is in timerModifying
Currently if a goroutine is preempted while owning a timer in the
timerModifying state, it could self-deadlock. When the goroutine is
preempted and calls into the scheduler, it could call checkTimers. If
checkTimers encounters the timerModifying timer and calls runtimer on
it, then runtimer will spin, waiting for that timer to leave the
timerModifying state, which it never will.
So far we got lucky that for the most part that there were no preemption
points while timerModifying is happening, however CL 221077 seems to
have introduced one, leading to sporadic self-deadlocks.
This change disables preemption explicitly while a goroutines holds a
timer in timerModifying. Since only checkTimers (and thus runtimer) is
called from the scheduler, this is sufficient to prevent
preemption-based self-deadlocks.
For #38070
Fixes #38072
Change-Id: Idbfac310889c92773023733ff7e2ff87e9896f0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/225497
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit e8be350d78f3fd21b0fab4cc6909c03fe21f1640)
Reviewed-on: https://go-review.googlesource.com/c/go/+/225521
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Sun, 16 Feb 2020 01:01:02 +0000 (12:01 +1100)]
[release-branch.go1.14] runtime: ignore error returned by PowerRegisterSuspendResumeNotification
It appears that PowerRegisterSuspendResumeNotification is not supported
when running inside Docker - see issues #35447, #36557 and #37149.
Our current code relies on error number to determine Docker environment.
But we already saw PowerRegisterSuspendResumeNotification return
ERROR_FILE_NOT_FOUND, ERROR_INVALID_PARAMETERS and ERROR_ACCESS_DENIED
(see issues above). So this approach is not sustainable.
Just ignore PowerRegisterSuspendResumeNotification returned error.
For #37149
Fixes #37699
Change-Id: I2beba9d45cdb8c1efac5e974e747827a6261915a
Reviewed-on: https://go-review.googlesource.com/c/go/+/219657
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
(cherry picked from commit d467f3bbc9c76805ae16ab1924c28ec3be487875)
Reviewed-on: https://go-review.googlesource.com/c/go/+/224586
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Dan Scales [Fri, 20 Mar 2020 16:31:20 +0000 (09:31 -0700)]
[release-branch.go1.14] runtime: fix code so defer record is not added to g0 defer list during panic
newdefer() actually adds the new defer to the current g's defer chain. That
happens even if we are on the system stack, in which case the g will be the g0
stack. For open-coded defers, we call newdefer() (only during panic processing)
while on the system stack, so the new defer is unintentionally added to the
g0._defer defer list. The code later correctly adds the defer to the user g's
defer list.
The g0._defer list is never used. However, that pointer on the g0._defer list can
keep a defer struct alive that is intended to be garbage-collected (smaller defers
use a defer pool, but larger-sized defer records are just GC'ed). freedefer() does
not zero out pointers when it intends that a defer become garbage-collected. So,
we can have the pointers in a defer that is held alive by g0._defer become invalid
(in particular d.link). This is the cause of the bad pointer bug in this issue
The fix is to change newdefer (only used in two places) to not add the new defer
to the gp._defer list. We just do it after the call with the correct gp pointer.
(As mentioned above, this code was already there after the newdefer in
addOneOpenDeferFrame.) That ensures that defers will be correctly
garbage-collected and eliminate the bad pointer.
This fix definitely fixes the original repro. I added a test and tried hard to
reproduce the bug (based on the original repro code), but awasn't actually able to
cause the bug. However, the test is still an interesting mix of heap-allocated,
stack-allocated, and open-coded defers.
For #37688
Fixes #37968
Fixes #37688
Change-Id: I1a481b9d9e9b9ba4e8726ef718a1f4512a2d6faf
Reviewed-on: https://go-review.googlesource.com/c/go/+/224581
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 825ae71e567593d3a28b7dddede8745701273c52)
Reviewed-on: https://go-review.googlesource.com/c/go/+/225279
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dan Scales <danscales@google.com>
Cherry Zhang [Tue, 17 Mar 2020 00:08:00 +0000 (20:08 -0400)]
[release-branch.go1.14] runtime: don't send preemption signal if there is a signal pending
If multiple threads call preemptone to preempt the same M, it may
send many signals to the same M such that it hardly make
progress, causing live-lock problem. Only send a signal if there
isn't already one pending.
Ian Lance Taylor [Thu, 12 Mar 2020 04:51:09 +0000 (21:51 -0700)]
[release-branch.go1.14] runtime: don't crash on mlock failure
Instead, note that mlock has failed, start trying the mitigation of
touching the signal stack before sending a preemption signal, and,
if the program crashes, mention the possible problem and a wiki page
describing the issue (https://golang.org/wiki/LinuxKernelSignalVectorBug).
Tested on a kernel in the buggy version range, but with the patch,
by using `ulimit -l 0`.
For #37436
Fixes #37807
Change-Id: I072aadb2101496dffd655e442fa5c367dad46ce8
Reviewed-on: https://go-review.googlesource.com/c/go/+/223121
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit b851e51160bc8ed412e229152b430b75e7ce56f9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/223417
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dan Scales [Thu, 5 Mar 2020 20:46:04 +0000 (12:46 -0800)]
[release-branch.go1.14] runtime: fix problem with repeated panic/recover/re-panics and open-coded defers
In the open-code defer implementation, we add defer struct entries to the defer
chain on-the-fly at panic time to represent stack frames that contain open-coded
defers. This allows us to process non-open-coded and open-coded defers in the
correct order. Also, we need somewhere to be able to store the 'started' state of
open-coded defers. However, if a recover succeeds, defers will now be processed
inline again (unless another panic happens). Any defer entry representing a frame
with open-coded defers will become stale once we run the corresponding defers
inline and exit the associated stack frame. So, we need to remove all entries for
open-coded defers at recover time.
The current code was only removing the top-most open-coded defer from the defer
chain during recovery. However, with recursive functions that do repeated
panic-recover-repanic, multiple stale entries can accumulate on the chain. So, we
just adjust the loop to process the entire chain. Since this is at panic/recover
case, it is fine to scan through the entire chain (which should usually have few
elements in it, since most defers are open-coded).
The added test fails with a SEGV without the fix, because it tries to run a stale
open-code defer entry (and the stack has changed).
Updates #37664.
Fixes #37782.
Change-Id: I8e3da5d610b5e607411451b66881dea887f7484d
Reviewed-on: https://go-review.googlesource.com/c/go/+/222420
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit fae87a2223e1fa959a20017742455200fe3c35f1)
Reviewed-on: https://go-review.googlesource.com/c/go/+/222818
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Keith Randall [Fri, 6 Mar 2020 22:01:26 +0000 (14:01 -0800)]
[release-branch.go1.14] runtime: make typehash match compiler generated hashes exactly
If typehash (used by reflect) does not match the built-in map's hash,
then problems occur. If a map is built using reflect, and then
assigned to a variable of map type, the hash function can change. That
causes very bad things.
This issue is rare. MapOf consults a cache of all types that occur in
the binary before making a new one. To make a true new map type (with
a hash function derived from typehash) that map type must not occur in
the binary anywhere. But to cause the bug, we need a variable of that
type in order to assign to it. The only way to make that work is to
use a named map type for the variable, so it is distinct from the
unnamed version that MapOf looks for.
Michael Pratt [Fri, 28 Feb 2020 19:16:41 +0000 (14:16 -0500)]
[release-branch.go1.14] runtime/pprof: expand final stack frame to avoid truncation
When generating stacks, the runtime automatically expands inline
functions to inline all inline frames in the stack. However, due to the
stack size limit, the final frame may be truncated in the middle of
several inline frames at the same location.
As-is, we assume that the final frame is a normal function, and emit and
cache a Location for it. If we later receive a complete stack frame, we
will first use the cached Location for the inlined function and then
generate a new Location for the "caller" frame, in violation of the
pprof requirement to merge inlined functions into the same Location.
As a result, we:
1. Nondeterministically may generate a profile with the different stacks
combined or split, depending on which is encountered first. This is
particularly problematic when performing a diff of profiles.
2. When split stacks are generated, we lose the inlining information.
We avoid both of these problems by performing a second expansion of the
last stack frame to recover additional inline frames that may have been
lost. This expansion is a bit simpler than the one done by the runtime
because we don't have to handle skipping, and we know that the last
emitted frame is not an elided wrapper, since it by definition is
already included in the stack.
Fixes #37447
Change-Id: If3ca2af25b21d252cf457cc867dd932f107d4c61
Reviewed-on: https://go-review.googlesource.com/c/go/+/221577
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
(cherry picked from commit fadbf7404d2b1aca63993e289448fcc3b6a23107)
Reviewed-on: https://go-review.googlesource.com/c/go/+/222762
Jay Conrod [Thu, 5 Mar 2020 16:11:47 +0000 (11:11 -0500)]
[release-branch.go1.14] cmd/go: make go test -json report failures for panicking/exiting tests
'go test -json' should report that a test failed if the test binary
did not exit normally with status 0. This covers panics, non-zero
exits, and abnormal terminations.
These tests don't print a final result when run with -test.v (which is
used by 'go test -json'). The final result should be "PASS" or "FAIL"
on a line by itself. 'go test' prints "FAIL" in this case, but
includes error information.
test2json was changed in CL 192104 to report that a test passed if it
does not report a final status. This caused 'go test -json' to report
that a test passed after a panic or non-zero exit.
With this change, test2json treats "FAIL" with error information the
same as "FAIL" on a line by itself. This is intended to be a minimal
fix for backporting, but it will likely be replaced by a complete
solution for #29062.
Change-Id: I63dad8b554353197ae0f29fa2a84f17bffa58557
GitHub-Last-Rev: 5297df32200ea5b52b2e7b52c8ee022d37e44111
GitHub-Pull-Request: golang/go#37661
Reviewed-on: https://go-review.googlesource.com/c/go/+/222037 Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 2b0f481278cc093e9f61945592257e6d651a169c)
Reviewed-on: https://go-review.googlesource.com/c/go/+/222317
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Hana (Hyang-Ah) Kim [Wed, 19 Feb 2020 03:41:20 +0000 (22:41 -0500)]
[release-branch.go1.14] cmd/trace: update to use WebComponents V0 polyfill
Old trace viewer stopped working with Chrome M80+ because the
old trace viewer heavily depended on WebComponents V0 which are deprecated.
Trace viewer recently migrated to use WebComponents V0 polyfill
(crbug.com/1036492). This CL brings in the newly updated trace_viewer_full.html
(sync'd @ 9508452e)
and updates the javascript snippet included in the /trace endpoint
to use the polyfill.
This brings in webcomponents.min.js copied from
https://chromium.googlesource.com/catapult/+/9508452e18f130c98499cb4c4f1e1efaedee8962/third_party/polymer/components/webcomponentsjs/webcomponents.min.js
That is necessary because the /trace endpoint needs to import
the vulcanized trace_viewer_full.html.
It's possible that some features are not working correctly with
this polyfill. In that case, report the issue to crbug.com/1036492.
There will be a warning message in the UI (yellow banner above the timeline)
which can be hidden by clicking the 'hide' button.
This allows to render the trace in browsers other than chrome in theory,
but I observed some buttons and functions still don't work outside
chrome.
Updates #34374.
Fixes #37343.
Change-Id: I0f369b15349dd0f4718c261ec23dfab6a47ace2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/219997
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
(cherry picked from commit 75ea964b3f6073076e1a86a0de2be9a2f159da24)
Reviewed-on: https://go-review.googlesource.com/c/go/+/220323
As far as I can tell, there is no public documentation on this topic,
which cost me several days of debugging.
I am possibly unusual in that I run binaries in production with the
race detector turned on, but I think that others who do the same may
want to be aware of the risk.
Updates #26813.
Updates #37233.
Change-Id: I1f8111bd01d0000596e6057b7cb5ed017d5dc655
Reviewed-on: https://go-review.googlesource.com/c/go/+/220586 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
(cherry picked from commit ba093c4562e7464e95a4bde6505d270b71ed623f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/221019
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
vovapi [Thu, 20 Feb 2020 19:29:22 +0000 (19:29 +0000)]
[release-branch.go1.14] hash/maphash: don't discard data on random seed init
Hash initializes seed on the first usage of seed or state with initSeed.
initSeed uses SetSeed which discards accumulated data.
This causes hash to return different sums for the same data in the first use
and after reset.
This CL fixes this issue by separating the seed set from data discard.
Dmitri Shuralyov [Fri, 21 Feb 2020 19:24:31 +0000 (14:24 -0500)]
[release-branch.go1.14] CONTRIBUTORS: second round of updates for Go 1.14
This update was automatically generated using the updatecontrib command:
cd gotip
go run golang.org/x/build/cmd/updatecontrib
With minor manual changes based on publicly available information
to canonicalize letter case and formatting for a few names.
Actions taken (relative to CONTRIBUTORS at origin/master):
Added Aaron Bieber <deftly@gmail.com>
Added Adam Williams <pwnfactory@gmail.com>
Added Ayke van Laethem <aykevanlaethem@gmail.com>
Added Bradford Lamson-Scribner <brad.lamson@gmail.com>
Added Brian Falk <falk@logicparty.org>
Added Chen Zhihan <energiehund@gmail.com>
Added Christopher Loessl <cloessl+github@gmail.com>
Added Frederik Zipp <fzipp@gmx.de>
Added Fujimoto Kyosuke <kyoro.f@gmail.com>
Added GitHub User jopbrown (6345470) <msshane2008@gmail.com>
Added GitHub User yah01 (12216890) <kagaminehuan@gmail.com>
Added Hiromichi Ema <ema.hiro@gmail.com>
Added Jamal Carvalho <jamal.a.carvalho@gmail.com>
Added Jason Baker <jason-baker@users.noreply.github.com>
Added Kanta Ebihara <kantaebihara@gmail.com>
Added Kirill Tatchihin <kirabsuir@gmail.com>
Added Kévin Dunglas <dunglas@gmail.com>
Added Mariano Cano <mariano@smallstep.com>
Added Sergey Ivanov <ser1325@gmail.com>
Added Thomas Symborski <thomas.symborski@gmail.com>
Added Tomohiro Kusumoto <zabio1192@gmail.com>
Added Xingqang Bai <bxq2011hust@qq.com>
Used GitHub User jopbrown (6345470) form for jopbrown <msshane2008@gmail.com> https://github.com/golang/exp/commit/0405dc7 [exp]
Used GitHub User yah01 (12216890) form for yah01 <kagaminehuan@gmail.com> https://github.com/golang/go/commit/ee55dd6b64 [go]
Used GitHub name "Hiromichi Ema" for emahiro <ema.hiro@gmail.com> https://github.com/golang/tools/commit/b6336cbc [tools]
Used GitHub name "Jamal Carvalho" for Gopher <jamal.a.carvalho@gmail.com> https://github.com/golang/gddo/commit/31dd61d [gddo]
Used GitHub name "Xingqang Bai" for bxq2011hust <bxq2011hust@qq.com> https://github.com/golang/go/commit/79ccbe1b67 [go]
Updates #12042
Change-Id: I13f8ab37f8b38f8f5d0ff71c939ad39d0bc4f985
Reviewed-on: https://go-review.googlesource.com/c/go/+/220363 Reviewed-by: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/220368
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Dmitri Shuralyov [Fri, 21 Feb 2020 20:16:40 +0000 (15:16 -0500)]
[release-branch.go1.14] all: merge master into release-branch.go1.14
1cd724acb6 doc/go1.14: highlight the addition of hash/maphash package a0cf2c872f doc/go1.14: remove TODO comment for CL 200439 a9ea91d571 cmd/link, runtime: skip holes in func table 88e564edb1 doc/go1.14: add missing period at sentence end 6917529cc6 testing: remove obsolete comment in testing.(*T) docs
Dmitri Shuralyov [Tue, 18 Feb 2020 15:03:22 +0000 (10:03 -0500)]
doc/go1.14: highlight the addition of hash/maphash package
Given that it's a package that did not exist before, was a proposal
in issue #28322, got accepted and implemented for 1.14, it seems to
be more than a minor change to the library. Highlight it accordingly.
Also specify the results are 64-bit integers, as done in CL 219340.
Updates #36878
Updates #28322
Change-Id: Idefe63d4c47a02cdcf8be8ab08c40cdb94ff2098
Reviewed-on: https://go-review.googlesource.com/c/go/+/219877
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Toshihiro Shiino <shiino.toshihiro@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dmitri Shuralyov [Tue, 18 Feb 2020 18:28:35 +0000 (18:28 +0000)]
doc/go1.14: remove TODO comment for CL 200439
Based on https://golang.org/issue/36878#issuecomment-587533153
and https://golang.org/issue/36878#issuecomment-587549692,
this is not a CL that needs to be mentioned in the release notes.
Updates #36878
Change-Id: Icaa9153da7481a1d3ebabc237411539dd770cef2
Reviewed-on: https://go-review.googlesource.com/c/go/+/219898
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Sun, 16 Feb 2020 21:18:04 +0000 (16:18 -0500)]
cmd/link, runtime: skip holes in func table
On PPC64 when external linking, for large binaries we split the
text section to multiple sections, so the external linking may
insert trampolines between sections. These trampolines are within
the address range covered by the func table, but not known by Go.
This causes runtime.findfunc to return a wrong function if the
given PC is from such trampolines.
In this CL, we generate a marker between text sections where
there could potentially be a hole in the func table. At run time,
we skip the hole if we see such a marker.
Fixes #37216.
Change-Id: I95ab3875a84b357dbaa65a4ed339a19282257ce0
Reviewed-on: https://go-review.googlesource.com/c/go/+/219717 Reviewed-by: David Chase <drchase@google.com>
Dmitri Shuralyov [Thu, 13 Feb 2020 19:58:56 +0000 (14:58 -0500)]
[release-branch.go1.14] all: merge master into release-branch.go1.14
3eab754cd0 runtime: correct caller PC/SP offsets in walltime1/nanotime1 123f7dd3e1 runtime: zero upper bit of Y registers in asyncPreempt on darwin/amd64 a0c9fb6bd3 hash/maphash: mention the results are 64-bit integers e237df5b53 runtime: fix fallback logic for aeshash on 32/64 bit 363bcd0068 cmd/go: eliminate empty '()' when passing -mod=readonly explicitly to 'go list' 7385947825 cmd/go/internal/modcmd: remove dead function addModFlag 5ce8005990 cmd/go/internal/web: fix a typo d0050e2871 go/build: populate partial package information in importGo 1c241d2879 hash/maphash: mention that hash values do not persist in package docs 25da21ddc9 crypto/elliptic: document the Name and names of each curve ab5d9f5831 doc/go1.14: add a couple minor crypto release notes 6a8164a254 go/doc: clarify that NewFromFiles caller must filter by GOOS/GOARCH dff55c1f76 doc: move doc/modules.md to x/website cfe2ab42e7 doc/go1.14: rearrange in alphabetical order ca8bf63809 doc/go1.14: add link to TempFile in io/ioutil a528215693 doc/go1.14: fix inconsistent markup a6b03c64b2 runtime/race: update reference to compiler-rt sources 60d437f994 runtime: avoid double notewakeup in netpoll stub code b8061825e5 doc: fill in 'go mod init' section of module documentation cb16d26bd6 doc: fill in 'go mod download' section of module documentation 08d41dbb10 doc: fill in 'go list -m' section in module documentation ff091b5fa0 doc: fill in 'Module-aware commands' section in module documentation c7c525a79d doc: add section on module paths to module documentation 153a9e8033 doc: add section on go.mod file syntax 1a37095062 Revert "cmd/link: code cleanup in macho_combine_dwarf.go" 494dd1dddc cmd/link: code cleanup in macho_combine_dwarf.go
Cherry Zhang [Tue, 11 Feb 2020 20:09:39 +0000 (15:09 -0500)]
runtime: correct caller PC/SP offsets in walltime1/nanotime1
In walltime1/nanotime1, we save the caller's PC and SP for stack
unwinding. The code does that assumed zero frame size. Now that
the frame size is not zero, correct the offset. Rewrite it in a
way that doesn't depend on hard-coded frame size.
Cherry Zhang [Tue, 11 Feb 2020 23:54:30 +0000 (18:54 -0500)]
runtime: zero upper bit of Y registers in asyncPreempt on darwin/amd64
Apparently, the signal handling code path in darwin kernel leaves
the upper bits of Y registers in a dirty state, which causes many
SSE operations (128-bit and narrower) become much slower. Clear
the upper bits to get to a clean state.
We do it at the entry of asyncPreempt, which is immediately
following exiting from the kernel's signal handling code, if we
actually injected a call. It does not cover other exits where we
don't inject a call, e.g. failed preemption, profiling signal, or
other async signals. But it does cover an important use case of
async signals, preempting a tight numerical loop, which we
introduced in this cycle.
Running the benchmark in issue #37174:
name old time/op new time/op delta
Fast-8 90.0ns ± 1% 46.8ns ± 3% -47.97% (p=0.000 n=10+10)
Slow-8 188ns ± 5% 49ns ± 1% -73.82% (p=0.000 n=10+9)
There is no more slowdown due to preemption signals.
Kanta Ebihara [Tue, 11 Feb 2020 23:17:30 +0000 (08:17 +0900)]
cmd/go/internal/web: fix a typo
dependenicies -> dependencies
Change-Id: I0b8f06c04cf397c6330ffb43ac3ae5c2f7cf3138
Reviewed-on: https://go-review.googlesource.com/c/go/+/219157 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Dmitri Shuralyov [Mon, 10 Feb 2020 00:51:22 +0000 (19:51 -0500)]
go/build: populate partial package information in importGo
This is a followup to CL 199840 and CL 203820. Cumulatively, they caused
a previously known bug to trigger more often while also nearly fixing it.
This change is a small fixup to CL 199840 that resolves the known bug
and prevents it from causing an additional regression in Go 1.14.
Part 1
The intention in CL 199840 was to return the same error that 'go list'
reported when the package wasn't located, so an early return was added.
However, to determine whether the package was located or not, p.Dir was
unintentionally checked instead of dir.
p is initialized to &Package{ImportPath: path} at top of Context.Import,
and its Dir field is never set before that line in importGo is reached.
So return errors.New(errStr) was always executed whenever errStr != "".
Originally, in CL 125296, the "go list" invocation did not include an
'-e' flag, so it would return a non-zero exit code on packages where
build constraints exclude all Go files, and importGo would return an
error like "go/build: importGo import/path: unexpected output: ...".
CL 199840 added an '-e' flag to the "go list" invocation, but checking
the wrong dir variable caused partial package information to never get
populated, and thus issue #31603 continued to occur, although with a
different error message (which ironically included the location of the
package that was supposedly "not found").
Now that the right dir is checked, issue #31603 is fixed.
Part 2
importGo checks whether it can use the go command to find the directory
of a package. In Go 1.13.x and earlier, one of the conditions to use the
go command was that the source directory must be provided.
CL 203820 made a change such that knowing the source directory was
no longer required:
// To invoke the go command,
-// we must know the source directory,
// ...
That meant build.Import invocations where srcDir is the empty string:
build.Import(path, "", build.FindOnly)
Started using the go command to find the directory of the package, and
started to run into issue #31603 as well. That's the #37153 regression.
Since this change fixes issue #31603, it also fixes issue #37153.
Part 3
There is one more thing. Delete the debugImportGo constant, it's unused.
Filippo Valsorda [Mon, 10 Feb 2020 19:48:58 +0000 (14:48 -0500)]
doc/go1.14: add a couple minor crypto release notes
These were left out of CL 216759 because they are trivial, but I was
advised to be thorough.
Updates #36878
Change-Id: Id4fd3a84866a82265e3f89abfdad6e3d231b507c
Reviewed-on: https://go-review.googlesource.com/c/go/+/218918 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Dmitri Shuralyov [Fri, 7 Feb 2020 16:15:11 +0000 (11:15 -0500)]
go/doc: clarify that NewFromFiles caller must filter by GOOS/GOARCH
The most well known and important build constraints to take into
account when rendering package documentation are the GOOS/GOARCH
values. Make it more clear in the NewFromFiles documentation that
they are a part of all build constraints that the caller is
responsible for filtering out.
Also suggest the "go/build".Context.MatchFile method for performing
file matching. The logic to perform build context file matching is
subtle and has many rules that aren't well known (for example,
taking the gc or gccgo compiler into account). It is currently the
only exported API in the standard library that implements this logic,
and it would be unfortunate if people attempt to re-create it because
they don't realize it is already available.
Updates #23864
Change-Id: I3c5901e7081acf79125b2d429ec3aa3b58416ed7
Reviewed-on: https://go-review.googlesource.com/c/go/+/218477 Reviewed-by: Robert Griesemer <gri@golang.org>
Toshihiro Shiino [Wed, 5 Feb 2020 11:25:01 +0000 (11:25 +0000)]
doc/go1.14: fix inconsistent markup
Unlike the others, the dt tag of reflect is not next to the dl tag.
The dd's closing tags may or may not have been omitted. They were unified without omission.
Updates #36878
Change-Id: I4e24f93fe8763ae8a1e4392db72e0b4818884f44
Reviewed-on: https://go-review.googlesource.com/c/go/+/217701 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Ian Lance Taylor [Fri, 7 Feb 2020 18:49:33 +0000 (10:49 -0800)]
runtime: avoid double notewakeup in netpoll stub code
Otherwise we can see
- goroutine 1 calls netpollBreak, the atomic.Cas succeeds, then suspends
- goroutine 2 calls noteclear, sets netpollBroken to 0
- goroutine 3 calls netpollBreak, the atomic.Cas succeeds, calls notewakeup
- goroutine 1 wakes up calls notewakeup, crashes due to double wakeup
This doesn't happen on Plan 9 because it only runs one thread at a time.
But Fuschia wants to use this code too.
Change-Id: Ib636e4f327bb15e44a2c40fd681aae9a91073a30
Reviewed-on: https://go-review.googlesource.com/c/go/+/218537
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Carlos Amedee [Thu, 6 Feb 2020 19:36:23 +0000 (14:36 -0500)]
[release-branch.go1.14] all: merge master into release-branch.go1.14
ab7c174183 testing: make Cleanup work for benchmarks too. ee3a3717aa doc/go1.14: disable text/template processing in HTML page dd0aa799eb doc/go1.14: quote {{ and }} in text/template note 9ee51745f7 doc/go1.14: mention better error checking in text/template e5b9c10689 doc/go1.14: document io/ioutil.TempDir's predictable prefix+suffix 7a36fa4002 crypto/x509: fix godoc for MarshalPKCS8PrivateKey 921ceadd29 runtime: rewrite a comment in malloc.go 88ae4ccefb math/big: reintroduce pre-Go 1.14 mention in GCD docs 60f11c44c0 doc/go1.14: document http.ServeFile large file fix for Windows 8a4d05cf07 cmd/go/internal/vet: only set work.VetExplicit if the list of explicit flags is non-empty 702226f933 doc/install.html: streamline the “Test your installation” step and make it module-agnostic ffd4e32885 doc/go1.14: add remarks about range inference and check removal
Dmitri Shuralyov [Thu, 6 Feb 2020 00:24:55 +0000 (19:24 -0500)]
doc/go1.14: disable text/template processing in HTML page
HTML pages served by the website have the option to opt-in to template
processing, by including "Template: true" in the page metadata.
This functionality is documented at
https://github.com/golang/tools/blob/403f1254bdfd3da27c92a0e9e37dd180a9a82b3c/godoc/template.go#L5-L30.
Historically, the Go 1 release notes have used template processing
to a great extent, but release notes for all subsequent major Go
releases have not.
Since this feature is generally not used and not very well known,
it tends to do more harm than good by making it possible for errors
in the template to prevent the release notes from showing up at all.
Disable this feature for Go 1.14 release notes and onwards.
We can consider enabling it when there's a stronger need for it.
Fixes #37072
Updates #37070
Change-Id: If93553d52df12544b46c4edcf3aa5eddc2a155ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/218058 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Documents io/ioutil.TempDir's new ability to create
predictable prefixes and suffixes, derived from the argument
'pattern', separated by the last '*' in it.
References: CL 198488
Updates #36878
Change-Id: I92c52fcc7d480ce74746e99e6e85a04efb87294f
Reviewed-on: https://go-review.googlesource.com/c/go/+/217780 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Emmanuel T Odeke [Wed, 5 Feb 2020 19:09:34 +0000 (11:09 -0800)]
doc/go1.14: document http.ServeFile large file fix for Windows
Document that for Windows, net/http.ServeFile can now
correctly serve files >2GB after we fixed internal/poll.SendFile
to transmit files larger than the default Windows limit of 2GB.
References: CL 192518, CL 194218
Updates #36878
Change-Id: Ibefc8b2841bc0cee3a89884a680085f99d2b6928
Reviewed-on: https://go-review.googlesource.com/c/go/+/217779 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Wed, 5 Feb 2020 15:22:07 +0000 (10:22 -0500)]
doc/install.html: streamline the “Test your installation” step and make it module-agnostic
In CL 199417, we updated “How to Write Go Code” to give a basic
introduction to modules and to include module-mode commands.
However, most new users will end up reading “Getting Started”
(doc/install.html) before “How to Write Go Code”, and we forgot to
update the handful of commands there for module mode.
Before this change, the “Test your installation” section also covered
quite a few operations beoyond merely testing the installation: it
included setting up a GOPATH, building a binary, and installing and
cleaning binaries. Those are valuable operations to learn, but they
arguably belong in “How to Write Go Code”, not “Test your
installation” — and having all that extra detail in the install
instructions may well discourage folks from further essential reading.
Rather than updating all of those operations here, I've removed them.
A companion CL will update “How to Write Go Code” to ensure that it
mentions GOPATH (as the location of the module cache and the default
install location for binaries) and 'go clean -i'.
Updates #37042
Change-Id: I157f21ccbe3896575fa1115dc821abf6c71ed15e
Reviewed-on: https://go-review.googlesource.com/c/go/+/217840
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Tyler Bui-Palsulich <tbp@google.com>
Carlos Amedee [Wed, 5 Feb 2020 14:42:17 +0000 (09:42 -0500)]
[release-branch.go1.14] all: merge master into release-branch.go1.14
a864cc7560 doc: rename HTML element IDs to avoid duplicates fb93cd45a6 net: don't check LookupHost error in TestLookupNullByte f770366f6d runtime: don't treat SIGURG as a bad signal e3f2e9ac4e internal/bytealg: fix riscv64 offset names
Dmitri Shuralyov [Wed, 5 Feb 2020 10:23:42 +0000 (05:23 -0500)]
doc: rename HTML element IDs to avoid duplicates
These 3 release notes have had an element ID collision because both
the runtime changes and changes to the package "runtime" used the
same ID. Fix it by using a "pkg-" prefix for the runtime package.
Move the "runtime-again" ID from CL 129635 to a nearby <dt> element
so that existing links to https://golang.org/doc/go1.11#runtime-again
don't break.
Fixes #37036
Updates #36878
Change-Id: Ib68d93acfac802fd84c0a57485937e45dea2064a
Reviewed-on: https://go-review.googlesource.com/c/go/+/217797 Reviewed-by: Toshihiro Shiino <shiino.toshihiro@gmail.com> Reviewed-by: Agniva De Sarker <agniva.quicksilver@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Mon, 3 Feb 2020 23:53:53 +0000 (15:53 -0800)]
runtime: don't treat SIGURG as a bad signal
It's possible for the scheduler to try to preempt a goroutine running
on a thread created by C code just as the goroutine returns from Go code
to C code. If that happens, the goroutine will have a nil g,
which would normally cause us to enter the badsignal code.
The badsignal code will allocate an M, reset the signal handler,
and raise the signal. This is all wasted work for SIGURG,
as the default behavior is for the kernel to ignore the signal.
It also means that there is a period of time when preemption requests
are ignored, because the signal handler is reset to the default.
And, finally, it triggers a bug on 386 OpenBSD 6.2. So stop doing it.
No test because there is no real change in behavior (other than on OpenBSD),
the new code is just more efficient
Katie Hockman [Tue, 4 Feb 2020 17:48:20 +0000 (12:48 -0500)]
doc: remove paragraph break for upgrading to modules
Previously, the release notes broke up the sentences that modules
is now ready for production use and where to file issues for
migration problems into separate paragraphs. This made it look like
the migration paragraph was about upgrading to 1.14, not to modules,
and made the reading a bit confusing. Now the entire idea is in one
paragraph.
Dmitri Shuralyov [Tue, 4 Feb 2020 15:24:20 +0000 (15:24 +0000)]
doc/go1.14: remove TODO about Solaris port
The solaris-amd64-oraclerel builder is passing for the main Go repo
(on tip and release branches for 1.13 and 1.12), and golang.org/x repos
(also on tip and release branches for 1.13 and 1.12).
The builder is still maintained as described at
https://golang.org/issue/15581#issuecomment-550368581.
Updates #36878
Updates #15581
Change-Id: Icc6f7529ca2e05bb34f09ce4363d9582e80829c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/217738 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
Dmitri Shuralyov [Tue, 4 Feb 2020 15:15:39 +0000 (15:15 +0000)]
doc/go1.14: remove TODO about Illumos port
There is an active builder that was added in CL 201597,
and it is passing on Go tip and release-branch.go1.13
(with one failure that appears to be flaky due to being
out of memory). It's also passing on all golang.org/x repos
on tip and release-branch.go1.13. It's not configured to
run on Go 1.12 release branches.
Updates #36878
Updates #15581
Change-Id: I4ed7fc62c11a09743832fca39bd61fa0cf6e7ded
Reviewed-on: https://go-review.googlesource.com/c/go/+/217737 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>