Bryan C. Mills [Tue, 8 Oct 2019 18:23:43 +0000 (14:23 -0400)]
[release-branch.go1.13] cmd/go: suppress more errors in package-to-module loading
In CL 197059, I suppressed errors if the target package was already found.
However, that does not cover the case of passing a '/v2' module path to
'go get' when the module does not contain a package at its root.
This CL is a minimal fix for that case, intended to be backportable to 1.13.
(Longer term, I intend to rework the version-validation check to treat
all mismatched paths as ErrNotExist.)
Updates #34746
Fixes #34747
Change-Id: Ia963c2ea00fae424812b8f46a4d6c2c668252147
Reviewed-on: https://go-review.googlesource.com/c/go/+/199839
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 421d35cf69f4a18edf96004ba99c01e629a0f79f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/199997
Duco van Amstel [Fri, 4 Oct 2019 13:05:13 +0000 (13:05 +0000)]
[release-branch.go1.13] cmd/go: fix listing of ambiguous paths
Passing ambiguous patterns, ending in `.go`, to `go list` results in them
being interpreted as Go files despite potentially being package references.
This can then result in errors on other package references.
The parsing logic is modified to check for a locally present file
corresponding to any pattern ending in `.go`. If no such file is present
the pattern is considered to be a package reference.
We're also adding a variety of non-regression tests that fail with the
original parsing code but passes after applying the fix.
Updates #34653
Fixes #34694
Change-Id: I073871da0dfc5641a359643f95ac14608fdca09b
GitHub-Last-Rev: 5abc200103ffc122df05422d79cf30c3ba0ee646
GitHub-Pull-Request: golang/go#34663
Reviewed-on: https://go-review.googlesource.com/c/go/+/198459
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit 33683f1d64df0cef2c598a84b741abb5af8abe5e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198957 Reviewed-by: Jay Conrod <jayconrod@google.com>
Jason A. Donenfeld [Fri, 4 Oct 2019 14:48:17 +0000 (16:48 +0200)]
[release-branch.go1.13] syscall: replace mksyscall_windows.go with wrapper to new x/sys home
We replace the existing file with a thin wrapper around its target so
that we don't break anybody's workflow.
This combines CL 198977 and CL 199277.
Fixes #34388
Change-Id: I0d00371c483cb78f4be18fe987df33c79cd40f05
Reviewed-on: https://go-review.googlesource.com/c/go/+/199538
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Tue, 25 Jun 2019 19:06:57 +0000 (19:06 +0000)]
[release-branch.go1.13] runtime: scavenge on growth instead of inline with allocation
Inline scavenging causes significant performance regressions in tail
latency for k8s and has relatively little benefit for RSS footprint.
We disabled inline scavenging in Go 1.12.5 (CL 174102) as well, but
we thought other changes in Go 1.13 had mitigated the issues with
inline scavenging. Apparently we were wrong.
This CL switches back to only doing foreground scavenging on heap
growth, rather than doing it when allocation tries to allocate from
scavenged space.
Fixes #34556
Change-Id: I1f5df44046091f0b4f89fec73c2cde98bf9448cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/183857
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit eb96f8a57444d174bba500b3a5d2a8b21b7e6d1e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198486 Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
Michael Anthony Knyszek [Tue, 3 Sep 2019 19:54:32 +0000 (19:54 +0000)]
[release-branch.go1.13] runtime: redefine scavenge goal in terms of heap_inuse
This change makes it so that the scavenge goal is defined primarily in
terms of heap_inuse at the end of the last GC rather than next_gc. The
reason behind this change is that next_gc doesn't take into account
fragmentation, and we can fall into situation where the scavenger thinks
it should have work to do but there's no free and unscavenged memory
available.
In order to ensure the scavenge goal still tracks next_gc, we multiply
heap_inuse by the ratio between the current heap goal and the last heap
goal, which describes whether the heap is growing or shrinking, and by
how much.
Finally, this change updates the documentation for scavenging and
elaborates on why the scavenge goal is defined the way it is.
Fixes #34149
Change-Id: I8deaf87620b5dc12a40ab8a90bf27932868610da
Reviewed-on: https://go-review.googlesource.com/c/go/+/193040
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 9b30811280427a6d50d2558f316d62210e948656)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198487
Run-TryBot: Andrew Bonventre <andybons@golang.org>
Austin Clements [Mon, 12 Aug 2019 18:54:28 +0000 (14:54 -0400)]
[release-branch.go1.13] runtime: grow the heap incrementally
Currently, we map and grow the heap a whole arena (64MB) at a time.
Unfortunately, in order to fix #32828, we need to switch from
scavenging inline with allocation back to scavenging on heap growth,
but heap-growth scavenging happens in large jumps because we grow the
heap in large jumps.
In order to prepare for better heap-growth scavenging, this CL
separates mapping more space for the heap from actually "growing" it
(tracking the new space with spans). Instead, growing the heap keeps
track of the "current arena" it's growing into. It track that with new
spans as needed, and only maps more arena space when the current arena
is inadequate. The effect to the user is the same, but this will let
us scavenge on much smaller increments of heap growth.
There are two slightly subtleties to this change:
1. If an allocation requires mapping a new arena and that new arena
isn't contiguous with the current arena, we don't want to lose the
unused space in the current arena, so we have to immediately track
that with a span.
2. The mapped space must be accounted as released and idle, even
though it isn't actually tracked in a span.
For #32828, since this makes heap-growth scavenging far more
effective, especially at small heap sizes. For example, this change is
necessary for TestPhysicalMemoryUtilization to pass once we remove
inline scavenging.
Updates #34556
Change-Id: I300e74a0534062467e4ce91cdc3508e5ef9aa73a
Reviewed-on: https://go-review.googlesource.com/c/go/+/189957
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit f18109d7e30c8d1a6e1c87ba3458499ac7ab2a79)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198485
Run-TryBot: Andrew Bonventre <andybons@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Jason A. Donenfeld [Tue, 27 Aug 2019 12:46:16 +0000 (06:46 -0600)]
[release-branch.go1.13] runtime: monitor for suspend/resume to kick timeouts
Starting in Windows 8, the wait functions don't take into account
suspend time, even though the monotonic counters do. This results in
timer buckets stalling on resume. Therefore, this commit makes it so
that on resume, we return from the wait functions and recalculate the
amount of time left to wait.
This is a cherry pick of CL 191957 and its cleanup, CL 198417.
Updates #31528
Fixes #34130
Change-Id: I0db02cc72188cb620954e87a0180e0a3c83f4a56
Reviewed-on: https://go-review.googlesource.com/c/go/+/193607
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Michael Hendricks [Wed, 2 Oct 2019 15:30:51 +0000 (09:30 -0600)]
[release-branch.go1.13] net: avoid an infinite loop in LookupAddr
If a request for a PTR record returned a response with a non-PTR
answer, goLookupPTR would loop forever. Skipping non-PTR answers
guarantees progress through the DNS response.
Michael Anthony Knyszek [Thu, 5 Sep 2019 16:34:00 +0000 (16:34 +0000)]
[release-branch.go1.13] runtime: fix lock acquire cycles related to scavenge.lock
There are currently two edges in the lock cycle graph caused by
scavenge.lock: with sched.lock and mheap_.lock. These edges appear
because of the call to ready() and stack growths respectively.
Furthermore, there's already an invariant in the code wherein
mheap_.lock must be acquired before scavenge.lock, hence the cycle.
The fix to this is to bring scavenge.lock higher in the lock cycle
graph, such that sched.lock and mheap_.lock are only acquired once
scavenge.lock is already held.
To faciliate this change, we move scavenger waking outside of
gcSetTriggerRatio such that it doesn't have to happen with the heap
locked. Furthermore, we check scavenge generation numbers with the heap
locked by using gopark instead of goparkunlock, and specify a function
which aborts the park should there be any skew in generation count.
Fixes #34150.
Change-Id: I3519119214bac66375e2b1262b36ce376c820d12
Reviewed-on: https://go-review.googlesource.com/c/go/+/191977
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 62e415655238a3c0103c1b70e6805edf8193c543)
Reviewed-on: https://go-review.googlesource.com/c/go/+/197501 Reviewed-by: Austin Clements <austin@google.com>
Jay Conrod [Fri, 13 Sep 2019 17:58:58 +0000 (13:58 -0400)]
[release-branch.go1.13] cmd/go: don't include package dir in cache key when -trimpath is set
The '-trimpath' flag tells 'go build' to trim any paths from the
output files that are tied to the current workspace or toolchain. When
this flag is set, we do not need to include the package directory in
the text hashed to construct the action ID for each package.
Updates #33772
Fixes #34326
Change-Id: I20b902d2f58019709b15864ca79aa0d9255ae707
Reviewed-on: https://go-review.googlesource.com/c/go/+/195318
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit aa680c0c49b55722a72ad3772e590cd2f9af541d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198259
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
Also added a test to ensure that any interactions
between TimeoutHandler and Flusher result in the
correct status code and body, but also that we don't
get superfluous logs from stray writes as was seen
in the bug report.
[release-branch.go1.13-security] net/textproto: don't normalize headers with spaces before the colon
RFC 7230 is clear about headers with a space before the colon, like
X-Answer : 42
being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.
On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.
For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.
This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn
net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.
Fixes CVE-2019-16276
Change-Id: I6d272de827e0870da85d93df770d6a0e161bbcf1
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/549719 Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
(cherry picked from commit 1280b868e82bf173ea3e988be3092d160ee66082)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/558935 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Tom Thorogood [Sat, 14 Sep 2019 01:16:20 +0000 (01:16 +0000)]
[release-branch.go1.13] net/http: fix HTTP/2 idle pool tracing
CL 140357 caused HTTP/2 connections to be put in the idle pool, but
failed to properly guard the trace.GotConn call in getConn. dialConn
returns a minimal persistConn with conn == nil for HTTP/2 connections.
This persistConn was then returned from queueForIdleConn and caused the
httptrace.GotConnInfo passed into GotConn to have a nil Conn field.
HTTP/2 connections call GotConn themselves so leave it for HTTP/2 to call
GotConn as is done directly below.
Fixes #34285
Change-Id: If54bfaf6edb14f5391463f908efbef5bb8a5d78e
GitHub-Last-Rev: 2b7d66a1ce66b4424c4d0fca2b8e8b547d874136
GitHub-Pull-Request: golang/go#34283
Reviewed-on: https://go-review.googlesource.com/c/go/+/195237 Reviewed-by: Michael Fraenkel <michael.fraenkel@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 582d5194faec20c475ab93b45cf0520253dec4a9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/196579 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
[release-branch.go1.13] cmd/go/internal/modfetch/codehost: work around an apparent bug in 'git fetch --unshallow'
When 'git fetch' is passed the '--unshallow' flag, it assumes that the
local and remote refs are equal.¹ However, we were fetching an
expanded set of refs explicitly in the same command, violating that
assumption.
Now we first expand the set of refs, then unshallow the repo in a
separate fetch. Empirically, this seems to work, whereas the opposite
order does not.
Jay Conrod [Fri, 13 Sep 2019 15:10:36 +0000 (11:10 -0400)]
[release-branch.go1.13] cmd/go: fix link error for -coverpkg in GOPATH mode
If a generated test main package transitively depends on a main
package, the main package will now always be rebuilt as a library and
will not be compiled with '-p main'.
This expands the fix for #30907, which only applied to packages with
the BuildInfo set (main packages built in module mode). Linking
multiple packages with BuildInfo caused link errors, but it appears
these errors apply to some symbols in GOPATH mode.
Fixes #34223
Change-Id: Ic1e53437942269a950dd7e45d163707922c92edd
Reviewed-on: https://go-review.googlesource.com/c/go/+/195279
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 24781a1faf62ac5c9a553af6f46b787e972f5539)
Reviewed-on: https://go-review.googlesource.com/c/go/+/195281
[release-branch.go1.13] cmd/go: suppress errors in package-to-module queries if the package is already found
In CL 173017, I changed the package-to-module query logic to query all
possible module paths in parallel in order to reduce latency. (For
long package paths, most such paths will not exist and will fail with
little overhead.)
The module resolution algorithm treats various kinds of non-existence
as “soft errors”, to be reported only if package resolution fails, but
treats any remaining errors as hard errors that should fail the query.
Unfortunately, that interacted badly with the +incompatible version
validation added in CL 181881, causing a regression in the 'direct'
fetch path for modules using the “major branch” layout¹ with a post-v1
version on the repository's default branch. Because we did not
interpret a mismatched module path as “no such module”, a go.mod file
specifying the path 'example.com/foo/v2' would cause the search for
module 'example.com/foo' to error out. (That regression was not caught
ahead of time due to a lack of test coverage for 'go get' on a package
within a /vN module.)
The promotion of hard errors during parallel search also made the 'go'
command less tolerant of servers that advertise 'go-import' tags for
nonexistent repositories. CL 194561 mitigated that problem for HTTP
servers that return code 404 or 410 for a nonexistent repository, but
unfortunately a few servers in common use (notably GitLab and
pre-1.9.3 releases of Gitea) do not.
This change mitigates both of those failure modes by ignoring
“miscellaneous” errors from shorter module paths if the requested
package pattern was successfully matched against a module with a
longer path.
Rhys Hiltner [Thu, 29 Aug 2019 16:49:36 +0000 (09:49 -0700)]
[release-branch.go1.13] cmd/cover: skip go list when profile is empty
Only call "go list" when explicitly listing packages. An empty coverage
profile references no packages, and would otherwise lead to "go list"
implicitly looking at the package in "." (which might not exist).
Fixes #33984
Change-Id: I02d4e374405d86f03d105fe14648aa03b4d2284c
Reviewed-on: https://go-review.googlesource.com/c/go/+/192340 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 9d480edadc6144d9f9f5a896d729d1642e46083b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192722
Jay Conrod [Tue, 10 Sep 2019 20:06:12 +0000 (16:06 -0400)]
[release-branch.go1.13] cmd/go: strip trailing slash from versioned arguments
'go get' accepts arguments of the form path@version, and it passes
them through search.CleanPatterns before querying proxies. With this
change, CleanPatterns preserves text after '@' and will strip trailing
slashes from the patn.
Previously, we did not strip trailing slashes when a version was
present, which caused proxy base URL validation to fail. Module paths
that end with ".go" (for example, github.com/nats-io/nats.go) use
trailing slashes to prevent 'go build' and other commands from
interpreting packages as source file names, so this caused unnecessary
problems for them.
Fixes #34243
Change-Id: Id3730c52089e52f1cac446617c20132a3021a808
Reviewed-on: https://go-review.googlesource.com/c/go/+/194600
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit 8875fb97c5cadbc6f02e4ce89efa586023c0a777)
Reviewed-on: https://go-review.googlesource.com/c/go/+/194687
Kevin Gillette [Mon, 26 Aug 2019 03:30:21 +0000 (03:30 +0000)]
[release-branch.go1.13] net/http: make docs refer to Context.Value as a getter instead of context.WithValue
The doc comments of both ServerContextKey and LocalAddrContextKey both suggest that context.WithValue can be used to access (get) properties of the server or connection. This PR fixes those comments to refer to Context.Value instead.
Tom Thorogood [Fri, 2 Aug 2019 07:53:44 +0000 (07:53 +0000)]
[release-branch.go1.13] encoding/json: revert Compact HTML escaping documentation
This partly reverts CL 173417 as it incorrectly documented that Compact
performed HTML escaping and the output was safe to embed inside HTML
<script> tags. This has never been true.
Although Compact does escape U+2028 and U+2029, it doesn't escape <, >
or &. Compact is thus only performing a subset of HTML escaping and it's
output is not safe to embed inside HTML <script> tags.
A more complete fix would be for Compact to either never perform any
HTML escaping, as it was prior to CL 10883045, or to actually perform
the same HTML escaping as HTMLEscape. Neither change is likely safe
enough for go1.13.
Fixes #34006
Updates #30357
Change-Id: I912f0fe9611097d988048b28228c4a5b985080ba
GitHub-Last-Rev: aebababc9233c5705785b225377e80096d4bb8c4
GitHub-Pull-Request: golang/go#33427
Reviewed-on: https://go-review.googlesource.com/c/go/+/188717 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 79669dc705aa8fcddc8f9407c568e4317d619dfe)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192747 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
Dmitri Shuralyov [Wed, 28 Aug 2019 17:34:16 +0000 (13:34 -0400)]
[release-branch.go1.13] CONTRIBUTORS: second round of updates for Go 1.13
This update has been automatically generated using the updatecontrib
command:
cd gotip
go run golang.org/x/build/cmd/updatecontrib
With minor manual changes based on publicly available information
to canonicalize letter case for a few names.
Actions taken (relative to CONTRIBUTORS at origin/master):
Added Albert Teoh <albert.teoh@gmail.com>
Added Allen Li <ayatane@google.com>
Added Anderson Queiroz <contato@andersonq.eti.br>
Added Andrew Todd <andrew.todd@wework.com>
Added Artem Kolin <artemkaxboy@gmail.com>
Added Bharath Thiruveedula <tbharath91@gmail.com>
Added Christian Muehlhaeuser <muesli@gmail.com>
Added Darren McCleary <darren.rmc@gmail.com>
Added David Finkel <david.finkel@gmail.com>
Added Eddie Scholtz <escholtz@google.com>
Added GitHub User tatsumack (4510569) <tatsu.mack@gmail.com>
Added GitHub User utkarsh-extc (53217283) <53217283+utkarsh-extc@users.noreply.github.com>
Added GitHub User yuanhh (1298735) <yuan415030@gmail.com>
Added Illya Yalovyy <yalovoy@gmail.com>
Added James Eady <jmeady@google.com>
Added Jan Steinke <jan.steinke@gmail.com>
Added Javier Revillas <jrevillas@massivedynamic.io>
Added Jordi Martin <jordimartin@gmail.com>
Added Jorge Araya <jorgejavieran@yahoo.com.mx>
Added Kelly Heller <pestophagous@gmail.com>
Added Kevin Gillette <extemporalgenome@gmail.com>
Added Mark Glines <mark@glines.org>
Added Nao Yonashiro <owan.orisano@gmail.com>
Added Pascal Dierich <pascal@pascaldierich.com>
Added Pure White <wu.purewhite@gmail.com>
Added Sam Arnold <sarnold64@bloomberg.net>
Added Sander van Harmelen <sander@vanharmelen.nl>
Added Sergei Zagurskii <gvozdoder@gmail.com>
Added Shivani Singhal <shivani.singhal2804@gmail.com>
Added Stefan Baebler <sbaebler@outbrain.com>
Added Tianon Gravi <admwiggin@gmail.com>
Added Toshihiro Shiino <shiino.toshihiro@gmail.com>
Added Wagner Riffel <wgrriffel@gmail.com>
Used GitHub User tatsumack (4510569) form for tatsumack <tatsu.mack@gmail.com> https://github.com/golang/website/commit/7eeb7ef [website]
Used GitHub User utkarsh-extc (53217283) form for utkarsh-extc <53217283+utkarsh-extc@users.noreply.github.com> https://github.com/golang/sys/commit/51ab0e2 [sys]
Used GitHub User yuanhh (1298735) form for yuanhh <yuan415030@gmail.com> https://github.com/golang/crypto/commit/60c769a [crypto]
Used GitHub name "Akhil Indurti" for smasher164 <aindurti@gmail.com> https://github.com/golang/go/commit/5ca44dc403 [build exp go sys]
Used GitHub name "Artem Kolin" for artemkaxboy <artemkaxboy@gmail.com> https://github.com/golang/go/commit/e881604d1c [go]
Used GitHub name "Ivan Markin" for nogoegst <nogoegst@users.noreply.github.com> https://github.com/golang/go/commit/a1addf15df [go]
Used GitHub name "Keiji Yoshida" for yosssi <yoshida.keiji.84@gmail.com> https://github.com/golang/lint/commit/ac6833c [lint]
Used GitHub name "Michalis Kargakis" for kargakis <mkargaki@redhat.com> https://github.com/golang/go/commit/e243d242d7 [go]
Used GitHub name "Roberto Clapis" for Roberto <empijei@users.noreply.github.com> https://github.com/golang/go/commit/963776e689 [go]
Used GitHub name "Robin Eklind" for mewmew <rnd0x00@gmail.com> https://github.com/golang/go/commit/b8620afb8d [blog go proposal.git]
Updates #12042
Change-Id: I1b21a18138849c537048558dd4324a823ba12a14
Reviewed-on: https://go-review.googlesource.com/c/go/+/192099 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit d0eaec79f95c2ad04d58e42c8e7cb123200cdfd5)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192178 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Bryan C. Mills [Tue, 27 Aug 2019 18:18:48 +0000 (14:18 -0400)]
[release-branch.go1.13] cmd/go/internal/get: remove '--' separator from 'git ls-remote' command
'git ls-remote' started recognizing the '--' separator at some point
after 2.7.4, but git defaults to version 2.7.4 on Ubuntu 16.04 LTS,
which remains supported by Ubuntu until April 2021.
We added '--' tokens to most VCS commands as a defensive measure in
CL 181237, but it isn't strictly necessary here because the 'scheme'
argument to our template is chosen from a predefined list: we can
safely drop it to retain compatibility.
Bryan C. Mills [Mon, 26 Aug 2019 20:44:36 +0000 (16:44 -0400)]
[release-branch.go1.13] net/http: fix wantConnQueue memory leaks in Transport
I'm trying to keep the code changes minimal for backporting to Go 1.13,
so it is still possible for a handful of entries to leak,
but the leaks are now O(1) instead of O(N) in the steady state.
Longer-term, I think it would be a good idea to coalesce idleMu with
connsPerHostMu and clear entries out of both queues as soon as their
goroutines are done waiting.
Cherry Zhang [Fri, 23 Aug 2019 18:40:20 +0000 (14:40 -0400)]
[release-branch.go1.13] cmd/vendor: update vendored x/arch repo to 8a70ba74b3a1
Update vendored x/arch repo to pick up the fix of issue #33802.
This is done with the following commands:
$ cd $GOROOT/src/cmd
$ go get -d golang.org/x/arch@latest
go: finding golang.org/x/arch latest
go: downloading golang.org/x/arch v0.0.0-20190815191158-8a70ba74b3a1
go: extracting golang.org/x/arch v0.0.0-20190815191158-8a70ba74b3a1
$ go mod tidy
$ go mod vendor
Updates #33802.
Fixes #33811.
Change-Id: I0a44f1d83d6f573124cea1f099378b1c851f3feb
Reviewed-on: https://go-review.googlesource.com/c/go/+/191619
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 66ff373911a87140319a6550e4e1b2c6043b1329)
Reviewed-on: https://go-review.googlesource.com/c/go/+/191737
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Emmanuel T Odeke [Wed, 21 Aug 2019 05:09:31 +0000 (23:09 -0600)]
[release-branch.go1.13] doc/go1.13: document _ between digits for math/big, strconv
Document that:
* math/big.Float.Parse
* math/big.Int.SetString
* strconv.ParseFloat
* strconv.ParseInt
* strconv.ParseUint
now accept underscores to group digits only if base = 0,
as per the Go 2 language changes.
Updates #32815
Change-Id: Id45bd803a18442436419739297e8aed0d32ca56c
Reviewed-on: https://go-review.googlesource.com/c/go/+/191077 Reviewed-by: Robert Griesemer <gri@golang.org>
(cherry picked from commit e764432605cb26478067890b49f4f16019c9a9b9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/191477
Run-TryBot: Andrew Bonventre <andybons@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
http2: limit number of control frames in server send queue
An attacker could cause servers to queue an unlimited number of PING
ACKs or RST_STREAM frames by soliciting them and not reading them, until
the program runs out of memory.
Limit control frames in the queue to a few thousands (matching the limit
imposed by other vendors) by counting as they enter and exit the scheduler,
so the protection will work with any WriteScheduler.
Once the limit is exceeded, close the connection, as we have no way to
communicate with the peer.
Change-Id: I842968fc6ed3eac654b497ade8cea86f7267886b
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/525552 Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
This change was generated with cmd/go and cmd/bundle:
$ go get -u golang.org/x/net
$ go mod tidy
$ go mod vendor
$ go generate net/http
Fixes CVE-2019-9512 and CVE-2019-9514
Fixes #33606
Filippo Valsorda [Tue, 20 Aug 2019 21:29:04 +0000 (17:29 -0400)]
Revert "encoding/json: avoid work when unquoting strings"
This reverts CL 151157.
CL 151157 introduced a crash when decoding into ",string" fields. It
came with a moderate speedup, so at this stage of the release cycle
let's just revert it, and reapply it in Go 1.14 with the fix in CL 190659.
Also applied the test cases from CL 190659.
Updates #33728
Change-Id: Ie46e2bc15224b251888580daf6b79d5865f3878e
Reviewed-on: https://go-review.googlesource.com/c/go/+/190909
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
Russ Cox [Wed, 21 Aug 2019 14:43:49 +0000 (10:43 -0400)]
cmd/go: accept GOSUMDB=sum.golang.google.cn
This CL makes the go command understand that
GOSUMDB=sum.golang.google.cn should connect
to that domain but expect to find a checksum database
signed by sum.golang.org there.
The host sum.golang.google.cn is not yet completely
configured; we hope it will be available in a few weeks.
Filippo Valsorda [Mon, 19 Aug 2019 17:17:34 +0000 (13:17 -0400)]
cmd/go: fix "go help build -o" docs
The docs refer to "the last two paragraphs", but in fact should refer to
the first two of the previous three paragraphs. Moved up the out of place
paragraph.
Updates #14295
Change-Id: I066da7a665bc6754d246782b941af214a385017a
Reviewed-on: https://go-review.googlesource.com/c/go/+/190839 Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Wagner Riffel [Mon, 19 Aug 2019 03:41:12 +0000 (00:41 -0300)]
doc: rewrite reference to plan9.bell-labs.com to 9p.io
Change-Id: I75619feced842b8ca509ee08e01b63258c5e87ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/190757 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dmitry Vyukov [Sat, 17 Aug 2019 20:22:22 +0000 (13:22 -0700)]
encoding/json: fix format string in the Fuzz func
Currently test build fails with:
$ go test -tags=gofuzz encoding/json
encoding/json/fuzz.go:36:4: Println call has possible formatting directive %s
FAIL encoding/json [build failed]
Change-Id: I23aef44a421ed0e7bcf48b74ac5a8c6768a4841b
Reviewed-on: https://go-review.googlesource.com/c/go/+/190698
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Fri, 16 Aug 2019 17:36:01 +0000 (10:36 -0700)]
time: update TestSub to avoid future regressions
CL 131196 optimized Time.Sub, but was reverted because
it incorrectly computed the nanoseconds in some edge cases.
This CL adds a test case to enforce the correct behavior
so that a future optimization does not break this again.
Updates #17858
Updates #33677
Change-Id: I596d8302ca6bf721cf7ca11cc6f939639fcbdd43
Reviewed-on: https://go-review.googlesource.com/c/go/+/190524
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Joe Tsai [Fri, 16 Aug 2019 17:42:18 +0000 (10:42 -0700)]
cmd/gofmt: update TestRewrite to avoid future regressions
CL 162337 changed go/ast to better handle block comments,
but was reverted because it introduced an off-by-one bug.
This CL adds a test case to enforce the correct behavior
so that future changes do not break this again.
Updates #18929
Updates #33538
Change-Id: I2d25c139d007f8db1091b7a48b1dd20c584e2699
Reviewed-on: https://go-review.googlesource.com/c/go/+/190523
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Cherry Zhang [Wed, 14 Aug 2019 13:50:51 +0000 (09:50 -0400)]
reflect: align first argument in callMethod
When calling a function obtained from reflect.Value.Method (or
MethodByName), we copy the arguments from the caller frame, which
does not include the receiver, to a new frame to call the actual
method, which does include the receiver. Here we need to align
the first (non-receiver) argument. As the receiver is pointer
sized, it is generally naturally aligned, except on amd64p32,
where the argument can have larger alignment, and this aligning
becomes necessary.
Filippo Valsorda [Tue, 6 Aug 2019 23:32:16 +0000 (19:32 -0400)]
net/url: make Hostname and Port predictable for invalid Host values
When Host is not valid per RFC 3986, the behavior of Hostname and Port
was wildly unpredictable, to the point that Host could have a suffix
that didn't appear in neither Hostname nor Port.
This is a security issue when applications are applying checks to Host
and expecting them to be meaningful for the contents of Hostname.
To reduce disruption, this change only aims to guarantee the following
two security-relevant invariants.
* Host is either Hostname or [Hostname] with Port empty, or
Hostname:Port or [Hostname]:Port.
* Port is only decimals.
The second invariant is the one that's most likely to cause disruption,
but I believe it's important, as it's conceivable an application might
do a suffix check on Host and expect it to be meaningful for the
contents of Hostname (if the suffix is not a valid port).
There are three ways to ensure it.
1) Reject invalid ports in Parse. Note that non-numeric ports are
already rejected if and only if the host starts with "[".
2) Consider non-numeric ports as part of Hostname, not Port.
3) Allow non-numeric ports, and hope they only flow down to net/http,
which will reject them (#14353).
This change adopts both 1 and 2. We could do only the latter, but then
these invalid hosts would flow past port checks, like in
http_test.TestTransportRejectsAlphaPort. Non-numeric ports weren't fully
supported anyway, because they were rejected after IPv6 literals, so
this restores consistency. We could do only the former, but at this
point 2) is free and might help with manually constructed Host values
(or if we get something wrong in Parse).
Note that net.SplitHostPort and net.Dial explicitly accept service names
in place of port numbers, but this is an URL package, and RFC 3986,
Section 3.2.3, clearly specifies ports as a number in decimal.
net/http uses a mix of net.SplitHostPort and url.Parse that would
deserve looking into, but in general it seems that it will still accept
service names in Addr fields as they are passed to net.Listen, while
rejecting them in URLs, which feels correct.
This leaves a number of invalid URLs to reject, which however are not
security relevant once the two invariants above hold, so can be done in
Go 1.14: IPv6 literals without brackets (#31024), invalid IPv6 literals,
hostnames with invalid characters, and more.
Tested with 200M executions of go-fuzz and the following Fuzz function.
u, err := url.Parse(string(data))
if err != nil {
return 0
}
h := u.Hostname()
p := u.Port()
switch u.Host {
case h + ":" + p:
return 1
case "[" + h + "]:" + p:
return 1
case h:
fallthrough
case "[" + h + "]":
if p != "" {
panic("unexpected Port()")
}
return 1
}
panic("Host is not a variant of [Hostname]:Port")
Fixes CVE-2019-14809
Updates #29098
Change-Id: I7ef40823dab28f29511329fa2d5a7fb10c3ec895
Reviewed-on: https://go-review.googlesource.com/c/go/+/189258 Reviewed-by: Ian Lance Taylor <iant@golang.org>
K. "pestophagous" Heller [Mon, 5 Aug 2019 05:46:29 +0000 (22:46 -0700)]
doc/install-source: create distinction between steps that involve "git clone"
Prior doc implied that "git clone" was one way to obtain a go1.4
bootstrap toochain, but it did not state this outright. Further,
the doc did not make it explicit in the "Fetch the repository"
section that one must necessarily "git clone" a second time in
the (presumed-to-be-uncommon) case where "git clone" had already
been perfomed in the "compiler binaries" section.
Updates #33402
Change-Id: Id70a6587b6ee09aca13559d63868b75cb07dff1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/188900 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Thu, 8 Aug 2019 22:09:54 +0000 (18:09 -0400)]
cmd/go: query each path only once in 'go get'
If we don't know whether a path is a module path or a package path,
previously we would first try a module query for it, then fall back to
a package query.
If we are using a sequence of proxies with fallback (as will be the
default in Go 1.13), and the path is not actually a module path, that
initial module query will fail against the first proxy, then
immediately fall back to the next proxy in the sequence — even if the
query could have been satisfied by some other (prefix) module
available from the first proxy.
Instead, we now query the requested path as only one kind of path.
If we query it as a package path but it turns out to only exist as a
module, we can detect that as a PackageNotInModuleError with an
appropriate module path — we do not need to issue a second query to
classify it.
Fixes #31785
Change-Id: I581d44279196e41d1fed27ec25489e75d62654e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/189517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Jay Conrod [Wed, 7 Aug 2019 20:27:39 +0000 (16:27 -0400)]
cmd/go: improve 'go mod download' and 'go list -m' error messages
modload.ListModules now wraps errors as module.ModuleError as
appropriate. The resulting errors always include the module path and
will include the version, if known.
'go mod download' no longer ignores errors reported by ListModules.
Previously, it started requesting module info, go.mod, and zip. Those
requests would fail, overwriting the original failure. They were
usually less descriptive.
'go mod download' with a module not in the build list (and no version
query) is now an error. Previously, this was silently ignored.
Fixes #30743
Change-Id: Icee8c1c6c5240de135a8b6ba42d6bbcdb757cdac
Reviewed-on: https://go-review.googlesource.com/c/go/+/189323
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Alberto Donizetti [Wed, 7 Aug 2019 18:11:46 +0000 (20:11 +0200)]
doc: update list of archs supporting -race
Change-Id: Id0a55674a16671aaee99182d9096a9263f7a80b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/189357 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Document goroutine label inheritance. Goroutine labels are copied upon
goroutine creation and there is a test enforcing this, but it was not
mentioned in the docstrings for `Do` or `SetGoroutineLabels`.
Add notes to both of those functions' docstrings so it's clear that one
does not need to set labels as soon as a new goroutine is spawned if
they want to propagate tags.
Updates #32223
Updates #23458
Change-Id: Idfa33031af0104b884b03ca855ac82b98500c8b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/189317 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jay Conrod [Tue, 6 Aug 2019 16:44:30 +0000 (12:44 -0400)]
cmd/go: improve error message for missing import starting with cmd/
In modload.Import, confirm that the import path does not start with
"cmd/" before calling QueryPackage, which returns a less helpful
error.
In load.loadPackageData, don't wrap errors with "unknown import path".
The wrapped error should always include the import path, and it's also
repeated in the PackageError wrapper.
Fixes #31031
Change-Id: I071efa22e3842c62831d096f888a8006811fe724
Reviewed-on: https://go-review.googlesource.com/c/go/+/189157
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Jonathan Amsterdam [Mon, 5 Aug 2019 14:14:40 +0000 (10:14 -0400)]
doc/go1.13.html: describe error-value additions
A brief description at the top the Standard Library section of the
changes to support error wrapping.
Fixes #33365.
Change-Id: Id5a3b2fe148d9bfb949f2cfc9e5d8a1613a0e219
Reviewed-on: https://go-review.googlesource.com/c/go/+/188798 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Damien Neil [Fri, 2 Aug 2019 18:18:56 +0000 (11:18 -0700)]
os: don't consult Is methods on non-syscall error types
CL #163058 moves interpretation of platform-specific errors to the
syscall package. Package syscall errors implement an Is method which
os.IsPermission etc. consult. This results in an unintended semantic
change to the os package predicate functions: The following program
now prints 'true' where it used to print 'false':
package main
import "os"
type myError struct{ error }
func (e myError) Is(target error) bool { return target == os.ErrPermission }
func main() { println(os.IsPermission(myError{})) }
Change the os package error predicate functions to only examine syscall
errors, avoiding this semantic change.
This CL does retain one minor semantic change: On Plan9, os.IsPermission
used to return true for any error with text containing the string
"permission denied". It now only returns true for a syscall.ErrorString
containing that text.
Change-Id: I6b512b1de6ced46c2f1cc8d264fa2495ae7bf9f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/188817
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>