Jay Conrod [Tue, 21 Jan 2020 22:50:36 +0000 (14:50 -0800)]
cmd/go: add -d flag to mod_get_test
'go get all' was run in this test without -d. This caused some std
packages to be reinstalled if the test is run in a slightly different
configuration than make.bash was run. run.bash would fail in some
situations because of this. Nothing in the cmd/go tests should modify
installed std or cmd packages.
Updates #35459
Change-Id: Idd259a27d55502923b7fc54f361a77f0ac11eea2
Reviewed-on: https://go-review.googlesource.com/c/go/+/215721
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Joel Sing [Thu, 16 Jan 2020 15:51:40 +0000 (02:51 +1100)]
cmd/internal/obj/riscv: correctly split immediates for FLW/FLD/FSW/FSD
The FLW/FLD/FSW/FSD instructions can have immediates that exceed 12-bits and
therefore cannot be encoded in the RISCV instruction. Handle these as we do
for other load/store instructions. Also add test coverage for all load/store
instructions with large immediates.
Fixes compilation issue reported by Carlos Eduardo de Paula.
Updates #27532
Change-Id: Ifa62f19493b3acaba5a90ac31d2df209a3afea81
Reviewed-on: https://go-review.googlesource.com/c/go/+/215037 Reviewed-by: Carlos Eduardo de Paula <me@carlosedp.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Kévin Dunglas [Tue, 21 Jan 2020 21:48:41 +0000 (21:48 +0000)]
net/http/httputil: add docs about X-Forwarded-For in ReverseProxy
ReverseProxy automatically sets the X-Forwarded-For header, if the request
already contains a X-Forwarded-For header, the value of the client IP is
appended to the existing header value.
This behavior isn't documented anywhere, and can lead to IP spoofing
security issues is the client is untrusted (the most common situation).
This PR documents this behavior.
For future versions, I proposed #36678 that implements a more secure
default behavior and adds support for other forwarded headers.
Carlos Eduardo Seo [Fri, 17 Jan 2020 20:59:59 +0000 (17:59 -0300)]
runtime: fix wrong offset when calling ppc64x nanotime syscall
There is a wrong offset when getting the results of a clock_gettime
syscall. Although the syscall will never be called in native ppc64x,
QEMU doesn't implement VDSO, so it will return wrong values.
The point of *net.OpError is to add details to an underlying lower
level error. It makes no sense to have an OpError without an Err and
a nil *OpError.Err will cause *OpError.Error() method to panic.
Fixes #33007
Change-Id: If4fb2501e02dad110a095b73e18c47312ffa6015
Reviewed-on: https://go-review.googlesource.com/c/go/+/187677 Reviewed-by: Rob Pike <r@golang.org>
Roger Peppe [Fri, 17 Jan 2020 09:06:13 +0000 (09:06 +0000)]
testing: clarify that Cleanup is run after subtests complete.
It's good to be explicit, as it's not necessarily obvious (and indeed
the behavior has changed recently with https://go-review.googlesource.com/c/go/+/214822)
without an associated doc comment change).
Change-Id: I99d6398bf15b404b1b1b196e712e926e363251e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/215217 Reviewed-by: Paul Jolly <paul@myitcv.org.uk> Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Chase [Fri, 17 Jan 2020 17:03:55 +0000 (12:03 -0500)]
cmd/compile: change the "bogus line" to be 1
The previous value was "too bogus" and caused objdump to crash.
Updated infinite loop test results (only run if -args -f) in ssa/debug_test.go
Probably also fixes #36621 but that bug needs more info to tell for certain.
Fixes #36570
Change-Id: I51144641d25d559308a98d726d87806bd340cc5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/215297
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Jay Conrod [Wed, 15 Jan 2020 21:29:46 +0000 (16:29 -0500)]
cmd/go: trim paths from vendored packages with -trimpath
In CL 199821, we stopped setting the module directory for vendored
packages when -mod=vendor is used. This broke -trimpath, since we
replace the module directory with a string derived from the module
path and version. A comment in CL 202977 makes it clear that the
module directory should not be set.
With this change, -trimpath falls back to replacing the package
directory with the package path if the module directory is not set. We
also fall back to replacing the package directory if the module
version is not set to avoid adding a meaningless @ only for the main
module.
As a consequence of this change, file names in vendored packages will
not have module versions, so file names will be a little different
between -mod=mod and -mod=vendor.
Fixes #36566
Change-Id: I0e9cd76d36a2028a49d0b6697ea9a9b3140d7ff3
Reviewed-on: https://go-review.googlesource.com/c/go/+/214945
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Ian Lance Taylor [Mon, 13 Jan 2020 20:17:26 +0000 (12:17 -0800)]
runtime: keep P's first timer when in new atomically accessed field
This reduces lock contention when only a few P's are running and
checking for whether they need to run timers on the sleeping P's.
Without this change the running P's would get lock contention
while looking at the sleeping P's timers. With this change a single
atomic load suffices to determine whether there are any ready timers.
Change-Id: Ie843782bd56df49867a01ecf19c47498ec827452
Reviewed-on: https://go-review.googlesource.com/c/go/+/214185
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com>
Michael Anthony Knyszek [Mon, 13 Jan 2020 17:18:51 +0000 (17:18 +0000)]
runtime: better approximate total cost of scavenging
Currently, the scavenger is paced according to how long it takes to
scavenge one runtime page's worth of memory. However, this pacing
doesn't take into account the additional cost of actually using a
scavenged page. This operation, "sysUsed," is a counterpart to the
scavenging operation "sysUnused." On most systems this operation is a
no-op, but on some systems like Darwin and Windows we actually make a
syscall. Even on systems where it's a no-op, the cost is implicit: a
more expensive page fault when re-using the page.
On Darwin in particular the cost of "sysUnused" is fairly close to the
cost of "sysUsed", which skews the pacing to be too fast. A lot of
soon-to-be-allocated memory ends up scavenged, resulting in many more
expensive "sysUsed" operations, ultimately slowing down the application.
The way to fix this problem is to include the future cost of "sysUsed"
on a page in the scavenging cost. However, measuring the "sysUsed" cost
directly (like we do with "sysUnused") on most systems is infeasible
because we would have to measure the cost of the first access.
Instead, this change applies a multiplicative constant to the measured
scavenging time which is based on a per-system ratio of "sysUnused" to
"sysUsed" costs in the worst case (on systems where it's a no-op, we
measure the cost of the first access). This ultimately slows down the
scavenger to a more reasonable pace, limiting its impact on performance
but still retaining the memory footprint improvements from the previous
release.
Ian Lance Taylor [Mon, 13 Jan 2020 23:34:53 +0000 (23:34 +0000)]
Revert "os: handle long path in RemoveAll for windows"
This reverts CL 214437.
Does not fix the issue, and the test was wrong so it did not detect that it did not fix the issue.
Updates #36375
Change-Id: I6a4112035a1e90f4fdafed6fdf4ec9dfc718b571
Reviewed-on: https://go-review.googlesource.com/c/go/+/214601 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Rob Pike [Thu, 9 Jan 2020 05:22:31 +0000 (16:22 +1100)]
time: add a sentence about distant times to time.Time.Unix
Since Durations only span 290 years, they are not good for
manipulating very remote times. I bounced off this problem recently
while doing some astronomical calculations and it took me a while to
realize I could get a 64-bit seconds value from time.Time.Unix and
subtract two of them to evaluate the interval.
I thought it worth adding a sentence to make this clear. It didn't
occur to me for quite a while that "Unix time" spans a huge range in
the Go library.
Robert Griesemer [Mon, 13 Jan 2020 21:01:28 +0000 (13:01 -0800)]
spec: uniformly format empty interfaces as "interface{}"
Fixes #36526.
Change-Id: Ic51a287579f139422cc1a7b2fb82d6732114b031
Reviewed-on: https://go-review.googlesource.com/c/go/+/214597 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Constantin Konstantinidis [Sat, 11 Jan 2020 18:53:20 +0000 (19:53 +0100)]
os: handle long path in RemoveAll for windows
Fixes #36375
Change-Id: I407a1db23868880b83e73bc136d274659483fb69
Reviewed-on: https://go-review.googlesource.com/c/go/+/214437
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Thu, 9 Jan 2020 22:58:18 +0000 (14:58 -0800)]
cmd/compile: print recursive types correctly
Change the type printer to take a map of types that we're currently
printing. When we happen upon a type that we're already in the middle
of printing, print a reference to it instead.
A reference to another type is built using the offset of the first
byte of that type's string representation in the result. To facilitate
that computation (and it's probably more efficient, regardless), we
print the type to a buffer as we go, and build the string at the end.
It would be nice to use string.Builder instead of bytes.Buffer, but
string.Builder wasn't around in Go 1.4, and we'd like to bootstrap
from that version.
Fixes #29312
Change-Id: I49d788c1fa20f770df7b2bae3b9979d990d54803
Reviewed-on: https://go-review.googlesource.com/c/go/+/214239 Reviewed-by: Robert Griesemer <gri@golang.org>
Ian Lance Taylor [Fri, 10 Jan 2020 18:10:32 +0000 (10:10 -0800)]
runtime: don't skip timer when adjustTimers sees a modified timer
When adjustTimers sees a timerModifiedEarlier or timerModifiedLater,
it removes it from the heap, leaving a new timer at that position
in the heap. We were accidentally skipping that new timer in our loop.
In some unlikely cases this could cause adjustTimers to look at more
timers than necessary.
Change-Id: Ic71e54c175ab7d86a7fa46f1497aca71ed1c43cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/214338
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Jay Conrod [Thu, 9 Jan 2020 21:22:01 +0000 (16:22 -0500)]
cmd/go: remove references to 1.13 in 'go help modules'
In "Module support" section, there were two mentions of "Go 1.13",
assuming that's the latest version. Rather than update these to 1.14,
this CL changes those to "The go command".
Also, a minor change in wording for finding go.mod files.
Change-Id: Id194be9405b540f221464814e71c361a22cc0f55
Reviewed-on: https://go-review.googlesource.com/c/go/+/214140
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dmitri Shuralyov [Wed, 8 Jan 2020 18:50:30 +0000 (13:50 -0500)]
CONTRIBUTORS: first round of updates for Go 1.14
This update was automatically generated using the updatecontrib command:
cd gotip
go run golang.org/x/build/cmd/updatecontrib
With minor manual changes based on publicly available information
to canonicalize letter case and formatting for a few names.
Actions taken (relative to CONTRIBUTORS at origin/master):
Added Aaron Beitch <aaronb@arista.com>
Added Adam Shelton <aashelt90@gmail.com>
Added Aditya Harindar <aditya.harindar@gmail.com>
Added Alec Benzer <alec.benzer@gmail.com>
Added Alex Buchanan <buchanae@gmail.com>
Added Alex Gaynor <alex@alloy.us>
Added Alex Harford <alex.harford@saucelabs.com>
Added Alex Zhirov <azhirov@google.com>
Added Alexandr Mayorskiy <a.mayorskiy@corp.mail.ru>
Added An Xiao <hac@zju.edu.cn>
Added Andrei Enshin <b1os@bk.ru>
Added Andrew Medvedev <andrew.y.medvedev@gmail.com>
Added Andrew Stormont <astormont@racktopsystems.com>
Added Antonio Huete Jimenez <tuxillo@quantumachine.net>
Added Arda Güçlü <ardaguclu@gmail.com>
Added Artem Alekseev <artem.alekseev@intel.com>
Added Ayan George <ayan@ayan.net>
Added Barnaby Keene <accounts@southcla.ws>
Added Ben Toews <mastahyeti@gmail.com>
Added Benjamin Wuethrich <benjamin.wuethrich@gmail.com>
Added Brandon Philips <brandon@ifup.org>
Added Carlos Amedee <carlos@golang.org>
Added Changkun Ou <hi@changkun.us>
Added Chauncy Cullitan <chauncyc@google.com>
Added Clint J. Edwards <clint.j.edwards@gmail.com>
Added Daisuke Suzuki <daisuzu@gmail.com>
Added Dan Scales <danscales@google.com>
Added David Bond <davidsbond93@gmail.com>
Added Denis Isaev <idenx@yandex.com>
Added Domas Tamašauskas <puerdomus@gmail.com>
Added Duco van Amstel <duco.vanamstel@gmail.com>
Added Eduardo Villaseñor <evillasrmx@gmail.com>
Added Eric Biggers <ebiggers@google.com>
Added Eric Brown <browne@vmware.com>
Added Eric Rutherford <erutherford@gmail.com>
Added Gabriel Rosenhouse <rosenhouse@gmail.com>
Added George Hartzell <hartzell@alerce.com>
Added Gert Cuykens <gert.cuykens@gmail.com>
Added Ghazni Nattarshah <ghazni.nattarshah@gmail.com>
Added GitHub User DQNEO (188741) <dqneoo@gmail.com>
Added GitHub User Matts966 (28551465) <Matts966@users.noreply.github.com>
Added GitHub User ajz01 (4744634) <ajzdenek@gmail.com>
Added GitHub User andig (184815) <cpuidle@gmx.de>
Added GitHub User jinmiaoluo (39730824) <jinmiaoluo@icloud.com>
Added GitHub User maltalex (10195391) <code@bit48.net>
Added GitHub User po3rin (29445112) <abctail30@gmail.com>
Added GitHub User pokutuna (57545) <popopopopokutuna@gmail.com>
Added GitHub User ramenjuniti (32011829) <ramenjuniti@gmail.com>
Added GitHub User skanehira (7888591) <sho19921005@gmail.com>
Added GitHub User witchard (4994659) <witchard@hotmail.co.uk>
Added GitHub User zikaeroh (48577114) <zikaeroh@gmail.com>
Added Günther Noack <gnoack@google.com>
Added Hasit Bhatt <hasit.p.bhatt@gmail.com>
Added Howard Zhang <howard.zhang@arm.com>
Added Huan Du <i@huandu.me>
Added Ignacio Hagopian <jsign.uy@gmail.com>
Added Ilya Sinelnikov <sidhmangh@gmail.com>
Added Irbe Krumina <irbekrm@gmail.com>
Added Isfan Azhabil <isfan.azhabil@tokopedia.com>
Added Ivan Trubach <mr.trubach@icloud.com>
Added Jaap Aarts <jaap.aarts1@gmail.com>
Added Jeremy Faller <jeremy@golang.org>
Added Johan Jansson <johan.jansson@iki.fi>
Added John Papandriopoulos <jpap.code@gmail.com>
Added Jorge L. Fatta <jorge.fatta@auth0.com>
Added Josa Gesell <josa@gesell.me>
Added Julian Tibble <julian.tibble@gmail.com>
Added Jun Zhang <jim.zoumo@gmail.com>
Added Jędrzej Szczepaniak <jbszczepaniak@gmail.com>
Added Kalman Bekesi <kalmanb@google.com>
Added Katharine Berry <ktbry@google.com>
Added Keisuke Kishimoto <keisuke.kishimoto@gmail.com>
Added Kevan Swanberg <kevswanberg@gmail.com>
Added Koki Tomoshige <tomocy.dev@gmail.com>
Added Liz Rice <liz@lizrice.com>
Added Lorenz Brun <lorenz@brun.one>
Added Luke Young <bored-engineer@users.noreply.github.com>
Added Marc Sanmiquel <marcsanmiquel@gmail.com>
Added Marko Kungla <marko.kungla@gmail.com>
Added Mikhail Fesenko <proggga@gmail.com>
Added Mohit Verma <vmohit.93@gmail.com>
Added Moritz Fain <moritz@fain.io>
Added Nathan Dias <nathan.dias@orijtech.com>
Added Nikita Vanyasin <nikita.vanyasin@gmail.com>
Added Nuno Cruces <ncruces@users.noreply.github.com>
Added Oliver Powell <oliverpowell84@gmail.com>
Added Panos Georgiadis <pgeorgiadis@suse.de>
Added Pantelis Sampaziotis <psampaz@gmail.com>
Added Paulo Gomes <paulo.gomes.uk@gmail.com>
Added Prashant Agrawal <prashant.a.vjti@gmail.com>
Added Ricardo Seriani <ricardo.seriani@gmail.com>
Added Rob Findley <rfindley@google.com>
Added Robin Zhong <robin@robinzhong.co>
Added Rohan Challa <rohan@golang.org>
Added Roman Kollár <roman.kollar.0@gmail.com>
Added Ruixin Bao <ruixin.bao@ibm.com>
Added Sardorbek Pulatov <sardorbek.pulatov@outlook.com>
Added Scott Ragan <ragansa@fb.com>
Added Sean Liao <seankhliao@gmail.com>
Added Sebastian Chlopecki <sebsebmc@gmail.com>
Added Sebastian Kinne <skinne@google.com>
Added Sergei Lemeshkin <sergeilem@gmail.com>
Added Serhat Giydiren <serhatgiydiren@gmail.com>
Added Shivashis Padhi <shivashispadhi@gmail.com>
Added Simarpreet Singh <simar@linux.com>
Added Simon Ferquel <simon.ferquel@docker.com>
Added Simon Rozman <simon@rozman.si>
Added Sjoerd Siebinga <sjoerd.siebinga@gmail.com>
Added Spencer Kocot <spencerkocot@gmail.com>
Added Srinidhi Kaushik <shrinidhi.kaushik@gmail.com>
Added Sven Taute <sven.taute@gmail.com>
Added Tao Qingyun <qingyunha@gmail.com>
Added Tianji Wu <the729@gmail.com>
Added Tomas Dabasinskas <tomas@dabasinskas.net>
Added Ville Skyttä <ville.skytta@iki.fi>
Added Vitaly Zdanevich <zdanevich.vitaly@ya.ru>
Added Vladimir Evgrafov <evgrafov.vladimir@gmail.com>
Added Vojtech Bocek <vbocek@gmail.com>
Added Wang Xuerui <git@xen0n.name>
Added William Poussier <william.poussier@gmail.com>
Added Xiangdong Ji <xiangdong.ji@arm.com>
Added Yuichi Nishiwaki <yuichi.nishiwaki@gmail.com>
Added Ziheng Liu <lzhfromustc@gmail.com>
Used GitHub User DQNEO (188741) form for DQNEO <dqneoo@gmail.com> https://github.com/golang/go/commit/f07059d949 [go]
Used GitHub User Matts966 (28551465) form for Matts966 <Matts966@users.noreply.github.com> https://github.com/golang/tools/commit/d89860af [tools]
Used GitHub User ajz01 (4744634) form for ajz01 <ajzdenek@gmail.com> https://github.com/golang/go/commit/57ad6ef15d [go]
Used GitHub User andig (184815) form for andig <cpuidle@gmx.de> https://github.com/golang/go/commit/cf630586ca [go sys]
Used GitHub User jinmiaoluo (39730824) form for jinmiaoluo <jinmiaoluo@icloud.com> https://github.com/golang/go/commit/c7e73ef60a [go]
Used GitHub User maltalex (10195391) form for maltalex <code@bit48.net> https://github.com/golang/sys/commit/c709ea0 [sys]
Used GitHub User po3rin (29445112) form for po3rin <abctail30@gmail.com> https://github.com/golang/go/commit/bf865823ba [go]
Used GitHub User pokutuna (57545) form for pokutuna <popopopopokutuna@gmail.com> https://github.com/golang/go/commit/46e0d724b3 [go]
Used GitHub User ramenjuniti (32011829) form for ramenjuniti <ramenjuniti@gmail.com> https://github.com/golang/go/commit/f9dd99cae3 [go]
Used GitHub User skanehira (7888591) form for skanehira <sho19921005@gmail.com> https://github.com/golang/go/commit/87805c92fd [go]
Used GitHub User utkarsh-extc (53217283) form for utkarsh-extc <53217283+utkarsh-extc@users.noreply.github.com> https://github.com/golang/sys/commit/51ab0e2 [sys]
Used GitHub User witchard (4994659) form for witchard <witchard@hotmail.co.uk> https://github.com/golang/go/commit/42db1da8e9 [go]
Used GitHub User zikaeroh (48577114) form for zikaeroh <zikaeroh@gmail.com> https://github.com/golang/tools/commit/e84277c2 [tools]
Used GitHub name "Adam Shelton" for Adam <aashelt90@gmail.com> https://github.com/golang/exp/commit/c286b88 [exp]
Used GitHub name "Andrew Bonventre" for Andrew <andybons@golang.org> https://github.com/golang/go/commit/8bbfc51d9a [arch blog build crypto debug exp gddo go image lint mobile net oauth2 perf playground proposal.git review sync sys talks term text time tools tour website]
Used GitHub name "David Bond" for davidsbond <davidsbond93@gmail.com> https://github.com/golang/go/commit/b421b85841 [go]
Used GitHub name "Eduardo Villaseñor" for galaxy-designer <evillasrmx@gmail.com> https://github.com/golang/tools/commit/db047d72 [tools]
Used GitHub name "George Hartzell" for hartzell <hartzell@alerce.com> https://github.com/golang/tools/commit/5eefd052 [tools]
Used GitHub name "Ignacio Hagopian" for jsign <jsign.uy@gmail.com> https://github.com/golang/go/commit/4d4ddd862d [go]
Used GitHub name "Ivan Markin" for nogoegst <nogoegst@users.noreply.github.com> https://github.com/golang/go/commit/a1addf15df [go]
Used GitHub name "Jun Zhang" for zoumo <jim.zoumo@gmail.com> https://github.com/golang/tools/commit/81ca6dc7 [tools]
Used GitHub name "Keiji Yoshida" for yosssi <yoshida.keiji.84@gmail.com> https://github.com/golang/lint/commit/ac6833c [lint]
Used GitHub name "Koki Tomoshige" for tomocy <tomocy.dev@gmail.com> https://github.com/golang/go/commit/2f04903fec [go]
Used GitHub name "Michalis Kargakis" for kargakis <mkargaki@redhat.com> https://github.com/golang/go/commit/e243d242d7 [go]
Used GitHub name "Nikita Vanyasin" for nikita-vanyasin <nikita.vanyasin@gmail.com> https://github.com/golang/go/commit/c3e8a20a65 [go]
Used GitHub name "Roberto Clapis" for Roberto <empijei@users.noreply.github.com> https://github.com/golang/go/commit/963776e689 [go]
Used GitHub name "Robin Eklind" for mewmew <rnd0x00@gmail.com> https://github.com/golang/go/commit/b8620afb8d [arch blog go proposal.git]
Used GitHub name "Sergei Lemeshkin" for sergeilem <sergeilem@gmail.com> https://github.com/golang/go/commit/a3a1bdff79 [go]
Used GitHub name "Vladimir Evgrafov" for vovapi <evgrafov.vladimir@gmail.com> https://github.com/golang/go/commit/207a0b7933 [go]
Michael Anthony Knyszek [Fri, 27 Dec 2019 16:48:23 +0000 (16:48 +0000)]
runtime: add scavtrace debug flag and remove scavenge info from gctrace
Currently, scavenging information is printed if the gctrace debug
variable is >0. Scavenging information is also printed naively, for
every page scavenged, resulting in a lot of noise when the typical
expectation for GC trace is one line per GC.
This change adds a new GODEBUG flag called scavtrace which prints
scavenge information roughly once per GC cycle and removes any scavenge
information from gctrace. The exception is debug.FreeOSMemory, which may
force an additional line to be printed.
Bryan C. Mills [Thu, 9 Jan 2020 14:39:48 +0000 (09:39 -0500)]
cmd/go/internal/modload: do not disable Query for -mod=readonly
'go list -m' allows explicit module@version arguments,
which it resolves (using Query) but does not add to the build list.
Similarly, 'go list -u' resolves versions without modifying the build list.
These explicit operations should be allowed even when '-mod=readonly' is set.
Updates #36478
'go list' and 'go mod download' do not
Change-Id: I5d2735729ad573635b9c1902d5d3a8bd960b8a76
Reviewed-on: https://go-review.googlesource.com/c/go/+/214077
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Austin Clements [Wed, 8 Jan 2020 14:58:42 +0000 (09:58 -0500)]
runtime: protect against external code calling ExitProcess
On Windows, we implement asynchronous preemption using SuspendThread
to suspend other threads in our process. However, SuspendThread is
itself actually asynchronous (it enqueues a kernel "asynchronous
procedure call" and returns). Unfortunately, Windows' ExitProcess API
kills all threads except the calling one and then runs APCs. As a
result, if SuspendThread and ExitProcess are called simultaneously,
the exiting thread can be suspended and the suspending thread can be
exited, leaving behind a ghost process consisting of a single thread
that's suspended.
We've already protected against the runtime's own calls to
ExitProcess, but if Go code calls external code, there's nothing
stopping that code from calling ExitProcess. For example, in #35775,
our own call to racefini leads to C code calling ExitProcess and
occasionally causing a deadlock.
This CL fixes this by introducing synchronization between calling
external code on Windows and preemption. It adds an atomic field to
the M that participates in a simple CAS-based synchronization protocol
to prevent suspending a thread running external code. We use this to
protect cgocall (which is used for both cgo calls and system calls on
Windows) and racefini.
Tested by running the flag package's TestParse test compiled in race
mode in a loop. Before this change, this would reliably deadlock after
a few minutes.
Bryan C. Mills [Tue, 7 Jan 2020 20:33:08 +0000 (15:33 -0500)]
cmd/go: make "finding" logging deterministic
In CL 204777, I made the "finding" messages in cachingRepo only print
after a “longish” delay, on the theory that they would help diagnose
slow or stuck fetches.
However, as I've been testing Go 1.14 beta 1, I've found that these
messages are mostly just noise, and the fact that they are so
nondeterministic causes both confusion and test flakes (#35539).
Moreover, it currently triggers once for each candidate module, when
what we're usually after is actually a specific package within the
module.
So let's log the package operation unconditionally instead of the
module fetches nondeterministically.
Fixes #35539
Updates #26152
Change-Id: I41a1c772465b2f0b357d3402bc372b6907773741
Reviewed-on: https://go-review.googlesource.com/c/go/+/213679
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Cherry Zhang [Wed, 8 Jan 2020 23:23:51 +0000 (18:23 -0500)]
runtime: overwrite asyncPreempt PC when injecting sigpanic on Windows
On Windows, it might be possible that SuspendThread suspends a
thread right between when an exception happens and when the
exception handler runs. (This is my guess. I don't know the
implementation detail of Windows exceptions to be sure.) In this
case, we may inject a call to asyncPreempt before the exception
handler runs. The exception handler will inject a sigpanic call,
which will make the stack trace looks like
sigpanic
asyncPreempt
actual panicking function
i.e. it appears asyncPreempt panicked.
Instead, just overwrite the PC, without pushing another frame.
Bryan C. Mills [Fri, 20 Dec 2019 22:12:38 +0000 (17:12 -0500)]
cmd/go: adjust heuristics for skipping +incompatible versions
We know of at least one module (github.com/stripe/stripe-go) that has
a run of +incompatible versions, followed by a run of versions with
go.mod files, followed by another run of +incompatible versions.
We want the heuristics for showing +incompatible versions to reflect
the authors' current intent, and it seems clear that the current
intent of the authors of that module is for users of the unversioned
import path to still be on +incompatible versions.
To respect that intent, we need to keep checking for +incompatible
versions even after we have seen a lower major version with an
explicit go.mod file.
However, we still don't want to download every single version of the
module to check it. A given major version should have a consistent,
canonical import path, so the path (as inferred by the presence or
absence of a go.mod file) should be the same for every release across
that major version.
To avoid unnecessary overhead — and to allow module authors to correct
accidental changes to a major version's import path — we check only
the most recent release of each major version. If a release
accidentally changes the import path in either direction (by deleting
or adding a go.mod file), it can be corrected by issuing a single
subsequent release of that major version to restore the correct path.
I manually verified that, with this change,
github.com/stripe/stripe-go@latest reverts to v68.7.0+incompatible
as it was in Go 1.13.
The other regression tests for #34165 continue to pass.
Updates #34165
Change-Id: I5daff3cd2123f94c7c49519babf4eecd509f169e
Reviewed-on: https://go-review.googlesource.com/c/go/+/212317 Reviewed-by: Jay Conrod <jayconrod@google.com>
Keith Randall [Tue, 7 Jan 2020 01:17:33 +0000 (17:17 -0800)]
cmd/compile: give every really deep type a unique name
This avoids the security problem in #29312 where two very deep, but
distinct, types are given the same name. They both make it to the
linker which chooses one, and the use of the other is now type unsafe.
Instead, give every very deep type its own name. This errs on the
other side, in that very deep types that should be convertible to each
other might now not be. But at least that's not a security hole.
Update #29312.
Change-Id: Iac0ebe73fdc50594fd6fbf7432eef65f9a053126
Reviewed-on: https://go-review.googlesource.com/c/go/+/213517
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Robert Griesemer <gri@golang.org>
Bryan C. Mills [Tue, 7 Jan 2020 17:03:28 +0000 (12:03 -0500)]
net/http: avoid writing to Transport.ProxyConnectHeader
Previously, we accidentally wrote the Proxy-Authorization header for
the initial CONNECT request to the shared ProxyConnectHeader map when
it was non-nil.
Michael Matloob [Thu, 2 Jan 2020 21:42:43 +0000 (16:42 -0500)]
cmd/go: convert TestTestRegexps to the script framework
It's hard to convert this one exactly because I don't think
we can guarantee that the grep command exists to filter stdout,
so I've tried to replicate the intent of the test.
Part of converting all tests to script framework to improve
test parallelism.
Updates #36320
Updates #17751
Change-Id: Ib593799ef7634ce12efb3ff357eb34475e2ea321
Reviewed-on: https://go-review.googlesource.com/c/go/+/213130
Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Joel Sing [Mon, 6 Jan 2020 17:31:23 +0000 (04:31 +1100)]
runtime: correct setNonblock on linux/arm64
The current code uses EOR (exclusive OR), which will result in the O_NONBLOCK
flag being toggled rather than being set. Other implementations use OR, hence
this is likely a bug.
Change-Id: I5dafa9c572452070bd37789c8a731ad6d04a86cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/212766
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dan Scales [Tue, 19 Nov 2019 21:58:28 +0000 (13:58 -0800)]
runtime: avoid potential deadlock when tracing memory code
In reclaimChunk, the runtime is calling traceGCSweepDone() while holding the mheap
lock. traceGCSweepDone() can call traceEvent() and traceFlush(). These functions
not only can get various trace locks, but they may also do memory allocations
(runtime.newobject) that may end up getting the mheap lock. So, there may be
either a self-deadlock or a possible deadlock between multiple threads.
It seems better to release the mheap lock before calling traceGCSweepDone(). It is
fine to release the lock, since the operations to get the index of the chunk of
work to do are atomic. We already release the lock to call sweep, so there is no
new behavior for any of the callers of reclaimChunk.
With this change, mheap is a leaf lock (no other lock is ever acquired while it
is held).
Testing: besides normal all.bash, also ran all.bash with --long enabled, since
it does longer tests of runtime/trace.
Brad Fitzpatrick [Mon, 6 Jan 2020 16:30:48 +0000 (16:30 +0000)]
os: document that File.Seek works on directories, but not portably
Updates #36019
Change-Id: I9fea2c3c5138e2233290979e4724f6e7b91da652
Reviewed-on: https://go-review.googlesource.com/c/go/+/213439 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Fri, 27 Dec 2019 17:02:50 +0000 (12:02 -0500)]
runtime: test memmove writes pointers atomically
In the previous CL we ensures that memmove writes pointers
atomically, so the concurrent GC won't observe a partially
updated pointer. This CL adds a test.
Michael Matloob [Thu, 26 Dec 2019 21:06:49 +0000 (16:06 -0500)]
cmd/go: convert some testcases to the script framework
This change converts TestFatalInBenchmarkCauseNonZeroExitStatus,
TestBenchmarkLabels and TestWildCardsDoNotLookInUselessDirectories
to the script framework so they can run in parallel. These tests
used the testdata directory so they didn't call tg.parallel, because
they couldn't be sure the testdata directory wasn't being
modified while they were being run.
The tests are converted to the script framework instead of being modified
because the script framework tests are easier to read and are self-contained.
Also remove the directory src/cmd/go/testdata/src/badc. The testcase
that used it, "TestDisallowedCSourceFiles" was deleted in
golang.org/cl/19613.
Updates #17751
Change-Id: I0b9b417ae1a9b148067a3e5f8531229f3414f104
Reviewed-on: https://go-review.googlesource.com/c/go/+/212620
Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Michael Matloob [Thu, 26 Dec 2019 20:03:12 +0000 (15:03 -0500)]
cmd/go: parallellize more test cases
Sprinkle in some more calls to tg.Parallel in the cmd/go
tests. These are the easy cases that don't operate in the
testdata directory.
I think the best thing to do for those tests that do operate
in testdata is to use the script tests. They're easier to
read and write, and jump into t.Parallel immediately.
Updates #17751
Change-Id: If8aeb8129cfdc0a9aa91f5a540f179790077fce5
Reviewed-on: https://go-review.googlesource.com/c/go/+/212618
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Joel Sing [Sun, 5 Jan 2020 14:52:19 +0000 (01:52 +1100)]
syscall: resync various riscv64 structs with x/sys/unix
For some reason the existing versions have additional padding - manually resync these
structs with those in x/sys/unix, also bringing in pollFd which does not currently
exist for linux/riscv64.
Cross checked with the output from 'go tool cgo -godefs' on a linux/riscv64 machine.
Joel Sing [Sun, 5 Jan 2020 14:48:43 +0000 (01:48 +1100)]
syscall: provide Renameat as a wrapper around renameat2 on riscv64
linux/riscv64 does not provide a renameat syscall and there is no SYS_RENAMEAT
defined in the syscall package. Provide Renameat as a wrapper around renameat2
instead.
Joel Sing [Sun, 5 Jan 2020 13:18:56 +0000 (00:18 +1100)]
syscall: fix riscv64 syscall assembly
RISCV has no instruction to store an immediate value to memory, or to
subtract an immediate value. An immediate needs to be moved to a register
first, or in the case of $0, the zero value register (ZERO or R0) can be
used instead. Restore the original riscv-go code so that it compiles.
Joel Sing [Fri, 30 Aug 2019 15:02:04 +0000 (01:02 +1000)]
cmd/link: skip symbol references when looking for missing symbols
ErrorUnresolved attempts to find the missing symbol in another ABI,
in order to provide more friendly error messages. However, in doing so
it checks the same ABI and can find the symbol reference for the symbol
that it is currently reporting the unresolved error for. Avoid this by
ignoring SXREF symbols, which is the same behaviour used when linking
is performed.
Fixes #33979
Change-Id: I3cb2477b2ad4baa7c2007323b983eb29404b0aac
Reviewed-on: https://go-review.googlesource.com/c/go/+/192597
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
Joel Sing [Tue, 31 Dec 2019 14:44:54 +0000 (01:44 +1100)]
cmd/internal/obj/riscv: correct CALL reloc symbol handling
Move the CALL reloc symbol back to p.From.Sym, rather than p.To.Sym - while
p.To.Sym is the correct final location, the call to lowerJALR flips the p.To
and p.From addresses (and proves that this form of obj.Prog rewriting is not
a good idea).
Joel Sing [Tue, 31 Dec 2019 14:28:22 +0000 (01:28 +1100)]
cmd/internal/obj/riscv: handle MOV rewrites in separate pass
Handle the rewriting of MOV pseudo-instructions in a separate pass. This allows
AMOV to be more readily used by other code, including AGETCALLERPC and ARET
implementations.
Cherry Zhang [Mon, 30 Dec 2019 16:26:17 +0000 (11:26 -0500)]
runtime: don't use R11 in nanotime1/walltime1 on ARM
R11 a.k.a. REGTMP is the temp register used by the assembler. It
may be clobbered if the assembler needs to synthesize
instructions. In particular, in nanotime1/walltime1, the load of
global variable runtime.iscgo clobbers it. So, avoid using R11
to hold a long-lived value.
Fixes #36309.
Change-Id: Iec2ab9d664532cad8fbf58da17f580e64a744f62
Reviewed-on: https://go-review.googlesource.com/c/go/+/212641 Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Andrew G. Morgan <agm@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Daniel Martí [Sun, 13 Oct 2019 12:18:03 +0000 (13:18 +0100)]
doc: document atexit_sleep_ms flag in race_detector
It was pointed out to me that the thread sanitizer library has many more
flags than the ones documented in this doc page. In particular, I was
able to make use of GORACE=atexit_sleep_ms=10 to reduce the amount that
'go test -race' will sleep before finishing.
I'm sure that this flag will be useful to others in the future, so add
it here. This is still not a complete list, but we can simply add useful
flags that work when we think they deserve a mention.