Michael Hendricks [Wed, 2 Oct 2019 15:30:51 +0000 (09:30 -0600)]
[release-branch.go1.13] net: avoid an infinite loop in LookupAddr
If a request for a PTR record returned a response with a non-PTR
answer, goLookupPTR would loop forever. Skipping non-PTR answers
guarantees progress through the DNS response.
Michael Anthony Knyszek [Thu, 5 Sep 2019 16:34:00 +0000 (16:34 +0000)]
[release-branch.go1.13] runtime: fix lock acquire cycles related to scavenge.lock
There are currently two edges in the lock cycle graph caused by
scavenge.lock: with sched.lock and mheap_.lock. These edges appear
because of the call to ready() and stack growths respectively.
Furthermore, there's already an invariant in the code wherein
mheap_.lock must be acquired before scavenge.lock, hence the cycle.
The fix to this is to bring scavenge.lock higher in the lock cycle
graph, such that sched.lock and mheap_.lock are only acquired once
scavenge.lock is already held.
To faciliate this change, we move scavenger waking outside of
gcSetTriggerRatio such that it doesn't have to happen with the heap
locked. Furthermore, we check scavenge generation numbers with the heap
locked by using gopark instead of goparkunlock, and specify a function
which aborts the park should there be any skew in generation count.
Fixes #34150.
Change-Id: I3519119214bac66375e2b1262b36ce376c820d12
Reviewed-on: https://go-review.googlesource.com/c/go/+/191977
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 62e415655238a3c0103c1b70e6805edf8193c543)
Reviewed-on: https://go-review.googlesource.com/c/go/+/197501 Reviewed-by: Austin Clements <austin@google.com>
Jay Conrod [Fri, 13 Sep 2019 17:58:58 +0000 (13:58 -0400)]
[release-branch.go1.13] cmd/go: don't include package dir in cache key when -trimpath is set
The '-trimpath' flag tells 'go build' to trim any paths from the
output files that are tied to the current workspace or toolchain. When
this flag is set, we do not need to include the package directory in
the text hashed to construct the action ID for each package.
Updates #33772
Fixes #34326
Change-Id: I20b902d2f58019709b15864ca79aa0d9255ae707
Reviewed-on: https://go-review.googlesource.com/c/go/+/195318
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit aa680c0c49b55722a72ad3772e590cd2f9af541d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/198259
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
Also added a test to ensure that any interactions
between TimeoutHandler and Flusher result in the
correct status code and body, but also that we don't
get superfluous logs from stray writes as was seen
in the bug report.
[release-branch.go1.13-security] net/textproto: don't normalize headers with spaces before the colon
RFC 7230 is clear about headers with a space before the colon, like
X-Answer : 42
being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.
On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.
For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.
This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn
net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.
Fixes CVE-2019-16276
Change-Id: I6d272de827e0870da85d93df770d6a0e161bbcf1
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/549719 Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
(cherry picked from commit 1280b868e82bf173ea3e988be3092d160ee66082)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/558935 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Tom Thorogood [Sat, 14 Sep 2019 01:16:20 +0000 (01:16 +0000)]
[release-branch.go1.13] net/http: fix HTTP/2 idle pool tracing
CL 140357 caused HTTP/2 connections to be put in the idle pool, but
failed to properly guard the trace.GotConn call in getConn. dialConn
returns a minimal persistConn with conn == nil for HTTP/2 connections.
This persistConn was then returned from queueForIdleConn and caused the
httptrace.GotConnInfo passed into GotConn to have a nil Conn field.
HTTP/2 connections call GotConn themselves so leave it for HTTP/2 to call
GotConn as is done directly below.
Fixes #34285
Change-Id: If54bfaf6edb14f5391463f908efbef5bb8a5d78e
GitHub-Last-Rev: 2b7d66a1ce66b4424c4d0fca2b8e8b547d874136
GitHub-Pull-Request: golang/go#34283
Reviewed-on: https://go-review.googlesource.com/c/go/+/195237 Reviewed-by: Michael Fraenkel <michael.fraenkel@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 582d5194faec20c475ab93b45cf0520253dec4a9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/196579 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
[release-branch.go1.13] cmd/go/internal/modfetch/codehost: work around an apparent bug in 'git fetch --unshallow'
When 'git fetch' is passed the '--unshallow' flag, it assumes that the
local and remote refs are equal.¹ However, we were fetching an
expanded set of refs explicitly in the same command, violating that
assumption.
Now we first expand the set of refs, then unshallow the repo in a
separate fetch. Empirically, this seems to work, whereas the opposite
order does not.
Jay Conrod [Fri, 13 Sep 2019 15:10:36 +0000 (11:10 -0400)]
[release-branch.go1.13] cmd/go: fix link error for -coverpkg in GOPATH mode
If a generated test main package transitively depends on a main
package, the main package will now always be rebuilt as a library and
will not be compiled with '-p main'.
This expands the fix for #30907, which only applied to packages with
the BuildInfo set (main packages built in module mode). Linking
multiple packages with BuildInfo caused link errors, but it appears
these errors apply to some symbols in GOPATH mode.
Fixes #34223
Change-Id: Ic1e53437942269a950dd7e45d163707922c92edd
Reviewed-on: https://go-review.googlesource.com/c/go/+/195279
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 24781a1faf62ac5c9a553af6f46b787e972f5539)
Reviewed-on: https://go-review.googlesource.com/c/go/+/195281
[release-branch.go1.13] cmd/go: suppress errors in package-to-module queries if the package is already found
In CL 173017, I changed the package-to-module query logic to query all
possible module paths in parallel in order to reduce latency. (For
long package paths, most such paths will not exist and will fail with
little overhead.)
The module resolution algorithm treats various kinds of non-existence
as “soft errors”, to be reported only if package resolution fails, but
treats any remaining errors as hard errors that should fail the query.
Unfortunately, that interacted badly with the +incompatible version
validation added in CL 181881, causing a regression in the 'direct'
fetch path for modules using the “major branch” layout¹ with a post-v1
version on the repository's default branch. Because we did not
interpret a mismatched module path as “no such module”, a go.mod file
specifying the path 'example.com/foo/v2' would cause the search for
module 'example.com/foo' to error out. (That regression was not caught
ahead of time due to a lack of test coverage for 'go get' on a package
within a /vN module.)
The promotion of hard errors during parallel search also made the 'go'
command less tolerant of servers that advertise 'go-import' tags for
nonexistent repositories. CL 194561 mitigated that problem for HTTP
servers that return code 404 or 410 for a nonexistent repository, but
unfortunately a few servers in common use (notably GitLab and
pre-1.9.3 releases of Gitea) do not.
This change mitigates both of those failure modes by ignoring
“miscellaneous” errors from shorter module paths if the requested
package pattern was successfully matched against a module with a
longer path.
Rhys Hiltner [Thu, 29 Aug 2019 16:49:36 +0000 (09:49 -0700)]
[release-branch.go1.13] cmd/cover: skip go list when profile is empty
Only call "go list" when explicitly listing packages. An empty coverage
profile references no packages, and would otherwise lead to "go list"
implicitly looking at the package in "." (which might not exist).
Fixes #33984
Change-Id: I02d4e374405d86f03d105fe14648aa03b4d2284c
Reviewed-on: https://go-review.googlesource.com/c/go/+/192340 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 9d480edadc6144d9f9f5a896d729d1642e46083b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192722
Jay Conrod [Tue, 10 Sep 2019 20:06:12 +0000 (16:06 -0400)]
[release-branch.go1.13] cmd/go: strip trailing slash from versioned arguments
'go get' accepts arguments of the form path@version, and it passes
them through search.CleanPatterns before querying proxies. With this
change, CleanPatterns preserves text after '@' and will strip trailing
slashes from the patn.
Previously, we did not strip trailing slashes when a version was
present, which caused proxy base URL validation to fail. Module paths
that end with ".go" (for example, github.com/nats-io/nats.go) use
trailing slashes to prevent 'go build' and other commands from
interpreting packages as source file names, so this caused unnecessary
problems for them.
Fixes #34243
Change-Id: Id3730c52089e52f1cac446617c20132a3021a808
Reviewed-on: https://go-review.googlesource.com/c/go/+/194600
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit 8875fb97c5cadbc6f02e4ce89efa586023c0a777)
Reviewed-on: https://go-review.googlesource.com/c/go/+/194687
Kevin Gillette [Mon, 26 Aug 2019 03:30:21 +0000 (03:30 +0000)]
[release-branch.go1.13] net/http: make docs refer to Context.Value as a getter instead of context.WithValue
The doc comments of both ServerContextKey and LocalAddrContextKey both suggest that context.WithValue can be used to access (get) properties of the server or connection. This PR fixes those comments to refer to Context.Value instead.
Tom Thorogood [Fri, 2 Aug 2019 07:53:44 +0000 (07:53 +0000)]
[release-branch.go1.13] encoding/json: revert Compact HTML escaping documentation
This partly reverts CL 173417 as it incorrectly documented that Compact
performed HTML escaping and the output was safe to embed inside HTML
<script> tags. This has never been true.
Although Compact does escape U+2028 and U+2029, it doesn't escape <, >
or &. Compact is thus only performing a subset of HTML escaping and it's
output is not safe to embed inside HTML <script> tags.
A more complete fix would be for Compact to either never perform any
HTML escaping, as it was prior to CL 10883045, or to actually perform
the same HTML escaping as HTMLEscape. Neither change is likely safe
enough for go1.13.
Fixes #34006
Updates #30357
Change-Id: I912f0fe9611097d988048b28228c4a5b985080ba
GitHub-Last-Rev: aebababc9233c5705785b225377e80096d4bb8c4
GitHub-Pull-Request: golang/go#33427
Reviewed-on: https://go-review.googlesource.com/c/go/+/188717 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 79669dc705aa8fcddc8f9407c568e4317d619dfe)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192747 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
Dmitri Shuralyov [Wed, 28 Aug 2019 17:34:16 +0000 (13:34 -0400)]
[release-branch.go1.13] CONTRIBUTORS: second round of updates for Go 1.13
This update has been automatically generated using the updatecontrib
command:
cd gotip
go run golang.org/x/build/cmd/updatecontrib
With minor manual changes based on publicly available information
to canonicalize letter case for a few names.
Actions taken (relative to CONTRIBUTORS at origin/master):
Added Albert Teoh <albert.teoh@gmail.com>
Added Allen Li <ayatane@google.com>
Added Anderson Queiroz <contato@andersonq.eti.br>
Added Andrew Todd <andrew.todd@wework.com>
Added Artem Kolin <artemkaxboy@gmail.com>
Added Bharath Thiruveedula <tbharath91@gmail.com>
Added Christian Muehlhaeuser <muesli@gmail.com>
Added Darren McCleary <darren.rmc@gmail.com>
Added David Finkel <david.finkel@gmail.com>
Added Eddie Scholtz <escholtz@google.com>
Added GitHub User tatsumack (4510569) <tatsu.mack@gmail.com>
Added GitHub User utkarsh-extc (53217283) <53217283+utkarsh-extc@users.noreply.github.com>
Added GitHub User yuanhh (1298735) <yuan415030@gmail.com>
Added Illya Yalovyy <yalovoy@gmail.com>
Added James Eady <jmeady@google.com>
Added Jan Steinke <jan.steinke@gmail.com>
Added Javier Revillas <jrevillas@massivedynamic.io>
Added Jordi Martin <jordimartin@gmail.com>
Added Jorge Araya <jorgejavieran@yahoo.com.mx>
Added Kelly Heller <pestophagous@gmail.com>
Added Kevin Gillette <extemporalgenome@gmail.com>
Added Mark Glines <mark@glines.org>
Added Nao Yonashiro <owan.orisano@gmail.com>
Added Pascal Dierich <pascal@pascaldierich.com>
Added Pure White <wu.purewhite@gmail.com>
Added Sam Arnold <sarnold64@bloomberg.net>
Added Sander van Harmelen <sander@vanharmelen.nl>
Added Sergei Zagurskii <gvozdoder@gmail.com>
Added Shivani Singhal <shivani.singhal2804@gmail.com>
Added Stefan Baebler <sbaebler@outbrain.com>
Added Tianon Gravi <admwiggin@gmail.com>
Added Toshihiro Shiino <shiino.toshihiro@gmail.com>
Added Wagner Riffel <wgrriffel@gmail.com>
Used GitHub User tatsumack (4510569) form for tatsumack <tatsu.mack@gmail.com> https://github.com/golang/website/commit/7eeb7ef [website]
Used GitHub User utkarsh-extc (53217283) form for utkarsh-extc <53217283+utkarsh-extc@users.noreply.github.com> https://github.com/golang/sys/commit/51ab0e2 [sys]
Used GitHub User yuanhh (1298735) form for yuanhh <yuan415030@gmail.com> https://github.com/golang/crypto/commit/60c769a [crypto]
Used GitHub name "Akhil Indurti" for smasher164 <aindurti@gmail.com> https://github.com/golang/go/commit/5ca44dc403 [build exp go sys]
Used GitHub name "Artem Kolin" for artemkaxboy <artemkaxboy@gmail.com> https://github.com/golang/go/commit/e881604d1c [go]
Used GitHub name "Ivan Markin" for nogoegst <nogoegst@users.noreply.github.com> https://github.com/golang/go/commit/a1addf15df [go]
Used GitHub name "Keiji Yoshida" for yosssi <yoshida.keiji.84@gmail.com> https://github.com/golang/lint/commit/ac6833c [lint]
Used GitHub name "Michalis Kargakis" for kargakis <mkargaki@redhat.com> https://github.com/golang/go/commit/e243d242d7 [go]
Used GitHub name "Roberto Clapis" for Roberto <empijei@users.noreply.github.com> https://github.com/golang/go/commit/963776e689 [go]
Used GitHub name "Robin Eklind" for mewmew <rnd0x00@gmail.com> https://github.com/golang/go/commit/b8620afb8d [blog go proposal.git]
Updates #12042
Change-Id: I1b21a18138849c537048558dd4324a823ba12a14
Reviewed-on: https://go-review.googlesource.com/c/go/+/192099 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit d0eaec79f95c2ad04d58e42c8e7cb123200cdfd5)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192178 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Bryan C. Mills [Tue, 27 Aug 2019 18:18:48 +0000 (14:18 -0400)]
[release-branch.go1.13] cmd/go/internal/get: remove '--' separator from 'git ls-remote' command
'git ls-remote' started recognizing the '--' separator at some point
after 2.7.4, but git defaults to version 2.7.4 on Ubuntu 16.04 LTS,
which remains supported by Ubuntu until April 2021.
We added '--' tokens to most VCS commands as a defensive measure in
CL 181237, but it isn't strictly necessary here because the 'scheme'
argument to our template is chosen from a predefined list: we can
safely drop it to retain compatibility.
Bryan C. Mills [Mon, 26 Aug 2019 20:44:36 +0000 (16:44 -0400)]
[release-branch.go1.13] net/http: fix wantConnQueue memory leaks in Transport
I'm trying to keep the code changes minimal for backporting to Go 1.13,
so it is still possible for a handful of entries to leak,
but the leaks are now O(1) instead of O(N) in the steady state.
Longer-term, I think it would be a good idea to coalesce idleMu with
connsPerHostMu and clear entries out of both queues as soon as their
goroutines are done waiting.
Cherry Zhang [Fri, 23 Aug 2019 18:40:20 +0000 (14:40 -0400)]
[release-branch.go1.13] cmd/vendor: update vendored x/arch repo to 8a70ba74b3a1
Update vendored x/arch repo to pick up the fix of issue #33802.
This is done with the following commands:
$ cd $GOROOT/src/cmd
$ go get -d golang.org/x/arch@latest
go: finding golang.org/x/arch latest
go: downloading golang.org/x/arch v0.0.0-20190815191158-8a70ba74b3a1
go: extracting golang.org/x/arch v0.0.0-20190815191158-8a70ba74b3a1
$ go mod tidy
$ go mod vendor
Updates #33802.
Fixes #33811.
Change-Id: I0a44f1d83d6f573124cea1f099378b1c851f3feb
Reviewed-on: https://go-review.googlesource.com/c/go/+/191619
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 66ff373911a87140319a6550e4e1b2c6043b1329)
Reviewed-on: https://go-review.googlesource.com/c/go/+/191737
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Emmanuel T Odeke [Wed, 21 Aug 2019 05:09:31 +0000 (23:09 -0600)]
[release-branch.go1.13] doc/go1.13: document _ between digits for math/big, strconv
Document that:
* math/big.Float.Parse
* math/big.Int.SetString
* strconv.ParseFloat
* strconv.ParseInt
* strconv.ParseUint
now accept underscores to group digits only if base = 0,
as per the Go 2 language changes.
Updates #32815
Change-Id: Id45bd803a18442436419739297e8aed0d32ca56c
Reviewed-on: https://go-review.googlesource.com/c/go/+/191077 Reviewed-by: Robert Griesemer <gri@golang.org>
(cherry picked from commit e764432605cb26478067890b49f4f16019c9a9b9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/191477
Run-TryBot: Andrew Bonventre <andybons@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
http2: limit number of control frames in server send queue
An attacker could cause servers to queue an unlimited number of PING
ACKs or RST_STREAM frames by soliciting them and not reading them, until
the program runs out of memory.
Limit control frames in the queue to a few thousands (matching the limit
imposed by other vendors) by counting as they enter and exit the scheduler,
so the protection will work with any WriteScheduler.
Once the limit is exceeded, close the connection, as we have no way to
communicate with the peer.
Change-Id: I842968fc6ed3eac654b497ade8cea86f7267886b
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/525552 Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
This change was generated with cmd/go and cmd/bundle:
$ go get -u golang.org/x/net
$ go mod tidy
$ go mod vendor
$ go generate net/http
Fixes CVE-2019-9512 and CVE-2019-9514
Fixes #33606
Filippo Valsorda [Tue, 20 Aug 2019 21:29:04 +0000 (17:29 -0400)]
Revert "encoding/json: avoid work when unquoting strings"
This reverts CL 151157.
CL 151157 introduced a crash when decoding into ",string" fields. It
came with a moderate speedup, so at this stage of the release cycle
let's just revert it, and reapply it in Go 1.14 with the fix in CL 190659.
Also applied the test cases from CL 190659.
Updates #33728
Change-Id: Ie46e2bc15224b251888580daf6b79d5865f3878e
Reviewed-on: https://go-review.googlesource.com/c/go/+/190909
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
Russ Cox [Wed, 21 Aug 2019 14:43:49 +0000 (10:43 -0400)]
cmd/go: accept GOSUMDB=sum.golang.google.cn
This CL makes the go command understand that
GOSUMDB=sum.golang.google.cn should connect
to that domain but expect to find a checksum database
signed by sum.golang.org there.
The host sum.golang.google.cn is not yet completely
configured; we hope it will be available in a few weeks.
Filippo Valsorda [Mon, 19 Aug 2019 17:17:34 +0000 (13:17 -0400)]
cmd/go: fix "go help build -o" docs
The docs refer to "the last two paragraphs", but in fact should refer to
the first two of the previous three paragraphs. Moved up the out of place
paragraph.
Updates #14295
Change-Id: I066da7a665bc6754d246782b941af214a385017a
Reviewed-on: https://go-review.googlesource.com/c/go/+/190839 Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Wagner Riffel [Mon, 19 Aug 2019 03:41:12 +0000 (00:41 -0300)]
doc: rewrite reference to plan9.bell-labs.com to 9p.io
Change-Id: I75619feced842b8ca509ee08e01b63258c5e87ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/190757 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dmitry Vyukov [Sat, 17 Aug 2019 20:22:22 +0000 (13:22 -0700)]
encoding/json: fix format string in the Fuzz func
Currently test build fails with:
$ go test -tags=gofuzz encoding/json
encoding/json/fuzz.go:36:4: Println call has possible formatting directive %s
FAIL encoding/json [build failed]
Change-Id: I23aef44a421ed0e7bcf48b74ac5a8c6768a4841b
Reviewed-on: https://go-review.googlesource.com/c/go/+/190698
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Fri, 16 Aug 2019 17:36:01 +0000 (10:36 -0700)]
time: update TestSub to avoid future regressions
CL 131196 optimized Time.Sub, but was reverted because
it incorrectly computed the nanoseconds in some edge cases.
This CL adds a test case to enforce the correct behavior
so that a future optimization does not break this again.
Updates #17858
Updates #33677
Change-Id: I596d8302ca6bf721cf7ca11cc6f939639fcbdd43
Reviewed-on: https://go-review.googlesource.com/c/go/+/190524
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Joe Tsai [Fri, 16 Aug 2019 17:42:18 +0000 (10:42 -0700)]
cmd/gofmt: update TestRewrite to avoid future regressions
CL 162337 changed go/ast to better handle block comments,
but was reverted because it introduced an off-by-one bug.
This CL adds a test case to enforce the correct behavior
so that future changes do not break this again.
Updates #18929
Updates #33538
Change-Id: I2d25c139d007f8db1091b7a48b1dd20c584e2699
Reviewed-on: https://go-review.googlesource.com/c/go/+/190523
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Cherry Zhang [Wed, 14 Aug 2019 13:50:51 +0000 (09:50 -0400)]
reflect: align first argument in callMethod
When calling a function obtained from reflect.Value.Method (or
MethodByName), we copy the arguments from the caller frame, which
does not include the receiver, to a new frame to call the actual
method, which does include the receiver. Here we need to align
the first (non-receiver) argument. As the receiver is pointer
sized, it is generally naturally aligned, except on amd64p32,
where the argument can have larger alignment, and this aligning
becomes necessary.
Filippo Valsorda [Tue, 6 Aug 2019 23:32:16 +0000 (19:32 -0400)]
net/url: make Hostname and Port predictable for invalid Host values
When Host is not valid per RFC 3986, the behavior of Hostname and Port
was wildly unpredictable, to the point that Host could have a suffix
that didn't appear in neither Hostname nor Port.
This is a security issue when applications are applying checks to Host
and expecting them to be meaningful for the contents of Hostname.
To reduce disruption, this change only aims to guarantee the following
two security-relevant invariants.
* Host is either Hostname or [Hostname] with Port empty, or
Hostname:Port or [Hostname]:Port.
* Port is only decimals.
The second invariant is the one that's most likely to cause disruption,
but I believe it's important, as it's conceivable an application might
do a suffix check on Host and expect it to be meaningful for the
contents of Hostname (if the suffix is not a valid port).
There are three ways to ensure it.
1) Reject invalid ports in Parse. Note that non-numeric ports are
already rejected if and only if the host starts with "[".
2) Consider non-numeric ports as part of Hostname, not Port.
3) Allow non-numeric ports, and hope they only flow down to net/http,
which will reject them (#14353).
This change adopts both 1 and 2. We could do only the latter, but then
these invalid hosts would flow past port checks, like in
http_test.TestTransportRejectsAlphaPort. Non-numeric ports weren't fully
supported anyway, because they were rejected after IPv6 literals, so
this restores consistency. We could do only the former, but at this
point 2) is free and might help with manually constructed Host values
(or if we get something wrong in Parse).
Note that net.SplitHostPort and net.Dial explicitly accept service names
in place of port numbers, but this is an URL package, and RFC 3986,
Section 3.2.3, clearly specifies ports as a number in decimal.
net/http uses a mix of net.SplitHostPort and url.Parse that would
deserve looking into, but in general it seems that it will still accept
service names in Addr fields as they are passed to net.Listen, while
rejecting them in URLs, which feels correct.
This leaves a number of invalid URLs to reject, which however are not
security relevant once the two invariants above hold, so can be done in
Go 1.14: IPv6 literals without brackets (#31024), invalid IPv6 literals,
hostnames with invalid characters, and more.
Tested with 200M executions of go-fuzz and the following Fuzz function.
u, err := url.Parse(string(data))
if err != nil {
return 0
}
h := u.Hostname()
p := u.Port()
switch u.Host {
case h + ":" + p:
return 1
case "[" + h + "]:" + p:
return 1
case h:
fallthrough
case "[" + h + "]":
if p != "" {
panic("unexpected Port()")
}
return 1
}
panic("Host is not a variant of [Hostname]:Port")
Fixes CVE-2019-14809
Updates #29098
Change-Id: I7ef40823dab28f29511329fa2d5a7fb10c3ec895
Reviewed-on: https://go-review.googlesource.com/c/go/+/189258 Reviewed-by: Ian Lance Taylor <iant@golang.org>
K. "pestophagous" Heller [Mon, 5 Aug 2019 05:46:29 +0000 (22:46 -0700)]
doc/install-source: create distinction between steps that involve "git clone"
Prior doc implied that "git clone" was one way to obtain a go1.4
bootstrap toochain, but it did not state this outright. Further,
the doc did not make it explicit in the "Fetch the repository"
section that one must necessarily "git clone" a second time in
the (presumed-to-be-uncommon) case where "git clone" had already
been perfomed in the "compiler binaries" section.
Updates #33402
Change-Id: Id70a6587b6ee09aca13559d63868b75cb07dff1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/188900 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Thu, 8 Aug 2019 22:09:54 +0000 (18:09 -0400)]
cmd/go: query each path only once in 'go get'
If we don't know whether a path is a module path or a package path,
previously we would first try a module query for it, then fall back to
a package query.
If we are using a sequence of proxies with fallback (as will be the
default in Go 1.13), and the path is not actually a module path, that
initial module query will fail against the first proxy, then
immediately fall back to the next proxy in the sequence — even if the
query could have been satisfied by some other (prefix) module
available from the first proxy.
Instead, we now query the requested path as only one kind of path.
If we query it as a package path but it turns out to only exist as a
module, we can detect that as a PackageNotInModuleError with an
appropriate module path — we do not need to issue a second query to
classify it.
Fixes #31785
Change-Id: I581d44279196e41d1fed27ec25489e75d62654e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/189517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Jay Conrod [Wed, 7 Aug 2019 20:27:39 +0000 (16:27 -0400)]
cmd/go: improve 'go mod download' and 'go list -m' error messages
modload.ListModules now wraps errors as module.ModuleError as
appropriate. The resulting errors always include the module path and
will include the version, if known.
'go mod download' no longer ignores errors reported by ListModules.
Previously, it started requesting module info, go.mod, and zip. Those
requests would fail, overwriting the original failure. They were
usually less descriptive.
'go mod download' with a module not in the build list (and no version
query) is now an error. Previously, this was silently ignored.
Fixes #30743
Change-Id: Icee8c1c6c5240de135a8b6ba42d6bbcdb757cdac
Reviewed-on: https://go-review.googlesource.com/c/go/+/189323
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Alberto Donizetti [Wed, 7 Aug 2019 18:11:46 +0000 (20:11 +0200)]
doc: update list of archs supporting -race
Change-Id: Id0a55674a16671aaee99182d9096a9263f7a80b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/189357 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Document goroutine label inheritance. Goroutine labels are copied upon
goroutine creation and there is a test enforcing this, but it was not
mentioned in the docstrings for `Do` or `SetGoroutineLabels`.
Add notes to both of those functions' docstrings so it's clear that one
does not need to set labels as soon as a new goroutine is spawned if
they want to propagate tags.
Updates #32223
Updates #23458
Change-Id: Idfa33031af0104b884b03ca855ac82b98500c8b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/189317 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jay Conrod [Tue, 6 Aug 2019 16:44:30 +0000 (12:44 -0400)]
cmd/go: improve error message for missing import starting with cmd/
In modload.Import, confirm that the import path does not start with
"cmd/" before calling QueryPackage, which returns a less helpful
error.
In load.loadPackageData, don't wrap errors with "unknown import path".
The wrapped error should always include the import path, and it's also
repeated in the PackageError wrapper.
Fixes #31031
Change-Id: I071efa22e3842c62831d096f888a8006811fe724
Reviewed-on: https://go-review.googlesource.com/c/go/+/189157
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Jonathan Amsterdam [Mon, 5 Aug 2019 14:14:40 +0000 (10:14 -0400)]
doc/go1.13.html: describe error-value additions
A brief description at the top the Standard Library section of the
changes to support error wrapping.
Fixes #33365.
Change-Id: Id5a3b2fe148d9bfb949f2cfc9e5d8a1613a0e219
Reviewed-on: https://go-review.googlesource.com/c/go/+/188798 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Damien Neil [Fri, 2 Aug 2019 18:18:56 +0000 (11:18 -0700)]
os: don't consult Is methods on non-syscall error types
CL #163058 moves interpretation of platform-specific errors to the
syscall package. Package syscall errors implement an Is method which
os.IsPermission etc. consult. This results in an unintended semantic
change to the os package predicate functions: The following program
now prints 'true' where it used to print 'false':
package main
import "os"
type myError struct{ error }
func (e myError) Is(target error) bool { return target == os.ErrPermission }
func main() { println(os.IsPermission(myError{})) }
Change the os package error predicate functions to only examine syscall
errors, avoiding this semantic change.
This CL does retain one minor semantic change: On Plan9, os.IsPermission
used to return true for any error with text containing the string
"permission denied". It now only returns true for a syscall.ErrorString
containing that text.
Change-Id: I6b512b1de6ced46c2f1cc8d264fa2495ae7bf9f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/188817
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Keith Randall [Fri, 2 Aug 2019 17:36:03 +0000 (10:36 -0700)]
cmd/compile: fix crash on write barrier insertion
The compiler can crash if the compiled code tries to
unconditionally read from a nil pointer. This should cause
the generated binary to panic, not the compiler.
which now uses the Request.Body's io.ReaderFrom implementation,
if available, and permits system level optimizations such as
"sendfile" to be used to transmit/upload the Body, which greatly
speeds up file uploads.
Updates #33396
Change-Id: I7b8315c4b3e57ad47bb9be2b0c838857875d4bd5
Reviewed-on: https://go-review.googlesource.com/c/go/+/188457 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Damien Neil [Fri, 2 Aug 2019 16:09:27 +0000 (09:09 -0700)]
all: remove os.ErrTimeout
It is unclear whether the current definition of os.IsTimeout is
desirable or not. Drop ErrTimeout for now so we can consider adding it
(or some other error) in a future release with a corrected definition.
Alberto Donizetti [Fri, 2 Aug 2019 17:09:20 +0000 (19:09 +0200)]
doc: fix Block interface parameters order
The Effective Go document references the crypto/cipher Block
interface, but the parameters' names are swapped. This change
alignes them to the crypto definition, to avoid confusion.
Fixes #33432
Change-Id: I8b9aac4dc6af3eec968bbc8f3ee5366b99016fcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/188797 Reviewed-by: Ian Lance Taylor <iant@golang.org>