zdjones [Fri, 11 Oct 2019 15:04:47 +0000 (16:04 +0100)]
[release-branch.go1.13-security] cmd/compile: make poset use sufficient conditions for OrderedOrEqual
When assessing whether A <= B, the poset's OrderedOrEqual has a passing
condition which permits A <= B, but is not sufficient to infer that A <= B.
This CL removes that incorrect passing condition.
Having identified that A and B are in the poset, the method will report that
A <= B if any of these three conditions are true:
(1) A and B are the same node in the poset.
- This means we know that A == B.
(2) There is a directed path, strict or not, from A -> B
- This means we know that, at least, A <= B, but A < B is possible.
(3) There is a directed path from B -> A, AND that path has no strict edges.
- This means we know that B <= A, but do not know that B < A.
In condition (3), we do not have enough information to say that A <= B, rather
we only know that B == A (which satisfies A <= B) is possible. The way I
understand it, a strict edge shows a known, strictly-ordered relation (<) but
the lack of a strict edge does not show the lack of a strictly-ordered relation.
The difference is highlighted by the example in #34802, where a bounds check is
incorrectly removed by prove, such that negative indexes into a slice
succeed:
n := make([]int, 1)
for i := -1; i <= 0; i++ {
fmt.Printf("i is %d\n", i)
n[i] = 1 // No Bounds check, program runs, assignment to n[-1] succeeds!!
}
When prove is checking the negative/failed branch from the bounds check at n[i],
in the signed domain we learn (0 > i || i >= len(n)). Because prove can't learn
the OR condition, we check whether we know that i is non-negative so we can
learn something, namely that i >= len(n). Prove uses the poset to check whether
we know that i is non-negative. At this point the poset holds the following
relations as a directed graph:
-1 <= i <= 0
-1 < 0
In poset.OrderedOrEqual, we are testing for 0 <= i. In this case, condition (3)
above is true because there is a non-strict path from i -> 0, and that path
does NOT have any strict edges. Because this condition is true, the poset
reports to prove that i is known to be >= 0. Knowing, incorrectly, that i >= 0,
prove learns from the failed bounds check that i >= len(n) in the signed domain.
When the slice, n, was created, prove learned that len(n) == 1. Because i is
also the induction variable for the loop, upon entering the loop, prove previously
learned that i is in [-1,0]. So when prove attempts to learn from the failed
bounds check, it finds the new fact, i > len(n), unsatisfiable given that it
previously learned that i <= 0 and len(n) = 1.
Fixes #34807
Change-Id: I235f4224bef97700c3aa5c01edcc595eb9f13afc
Reviewed-on: https://go-review.googlesource.com/c/go/+/200759
Run-TryBot: Zach Jones <zachj1@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Giovanni Bajo <rasky@develer.com> Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/201060
Run-TryBot: Alexander Rakoczy <alex@golang.org>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/575398 Reviewed-by: Filippo Valsorda <valsorda@google.com>
zdjones [Fri, 30 Aug 2019 13:41:09 +0000 (14:41 +0100)]
[release-branch.go1.13-security] cmd/compile: rename poset method dominates to reaches
The partially ordered set uses a method named 'dominates' to determine whether
two nodes are partially ordered. Dominates does a depth-first search of the
DAG, beginning at the source node, and returns true as soon as it finds a path
to the target node. In the context of the forest-of-DAGs that makes up the
poset, dominates is not necessarily checking dominance, but is checking
reachability. See the issue tracker for a more detailed discussion of the
difference.
Fortunately, reachability is logically correct everywhere dominates is currently
used in poset.go. Reachability within a DAG is sufficient to establish the
partial ordering (source < target).
This CL changes the name of the method (dominates -> reaches) and updates
all the comments in the file accordingly.
Updates #34807
Change-Id: Ia3a34f7b14b363801d75b05099cfc686035f7d96
Reviewed-on: https://go-review.googlesource.com/c/go/+/192617 Reviewed-by: Giovanni Bajo <rasky@develer.com>
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/201059
Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/575397 Reviewed-by: Filippo Valsorda <valsorda@google.com>
Katie Hockman [Mon, 14 Oct 2019 20:42:21 +0000 (16:42 -0400)]
[release-branch.go1.13-security] crypto/dsa: prevent bad public keys from causing panic
dsa.Verify might currently use a nil s inverse in a
multiplication if the public key contains a non-prime Q,
causing a panic. Change this to check that the mod
inverse exists before using it.
[release-branch.go1.13-security] net/textproto: don't normalize headers with spaces before the colon
RFC 7230 is clear about headers with a space before the colon, like
X-Answer : 42
being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.
On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.
For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.
This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn
net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.
Fixes CVE-2019-16276
Change-Id: I6d272de827e0870da85d93df770d6a0e161bbcf1
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/549719 Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
(cherry picked from commit 1280b868e82bf173ea3e988be3092d160ee66082)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/558935 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Kevin Gillette [Mon, 26 Aug 2019 03:30:21 +0000 (03:30 +0000)]
[release-branch.go1.13] net/http: make docs refer to Context.Value as a getter instead of context.WithValue
The doc comments of both ServerContextKey and LocalAddrContextKey both suggest that context.WithValue can be used to access (get) properties of the server or connection. This PR fixes those comments to refer to Context.Value instead.
Tom Thorogood [Fri, 2 Aug 2019 07:53:44 +0000 (07:53 +0000)]
[release-branch.go1.13] encoding/json: revert Compact HTML escaping documentation
This partly reverts CL 173417 as it incorrectly documented that Compact
performed HTML escaping and the output was safe to embed inside HTML
<script> tags. This has never been true.
Although Compact does escape U+2028 and U+2029, it doesn't escape <, >
or &. Compact is thus only performing a subset of HTML escaping and it's
output is not safe to embed inside HTML <script> tags.
A more complete fix would be for Compact to either never perform any
HTML escaping, as it was prior to CL 10883045, or to actually perform
the same HTML escaping as HTMLEscape. Neither change is likely safe
enough for go1.13.
Fixes #34006
Updates #30357
Change-Id: I912f0fe9611097d988048b28228c4a5b985080ba
GitHub-Last-Rev: aebababc9233c5705785b225377e80096d4bb8c4
GitHub-Pull-Request: golang/go#33427
Reviewed-on: https://go-review.googlesource.com/c/go/+/188717 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 79669dc705aa8fcddc8f9407c568e4317d619dfe)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192747 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
Dmitri Shuralyov [Wed, 28 Aug 2019 17:34:16 +0000 (13:34 -0400)]
[release-branch.go1.13] CONTRIBUTORS: second round of updates for Go 1.13
This update has been automatically generated using the updatecontrib
command:
cd gotip
go run golang.org/x/build/cmd/updatecontrib
With minor manual changes based on publicly available information
to canonicalize letter case for a few names.
Actions taken (relative to CONTRIBUTORS at origin/master):
Added Albert Teoh <albert.teoh@gmail.com>
Added Allen Li <ayatane@google.com>
Added Anderson Queiroz <contato@andersonq.eti.br>
Added Andrew Todd <andrew.todd@wework.com>
Added Artem Kolin <artemkaxboy@gmail.com>
Added Bharath Thiruveedula <tbharath91@gmail.com>
Added Christian Muehlhaeuser <muesli@gmail.com>
Added Darren McCleary <darren.rmc@gmail.com>
Added David Finkel <david.finkel@gmail.com>
Added Eddie Scholtz <escholtz@google.com>
Added GitHub User tatsumack (4510569) <tatsu.mack@gmail.com>
Added GitHub User utkarsh-extc (53217283) <53217283+utkarsh-extc@users.noreply.github.com>
Added GitHub User yuanhh (1298735) <yuan415030@gmail.com>
Added Illya Yalovyy <yalovoy@gmail.com>
Added James Eady <jmeady@google.com>
Added Jan Steinke <jan.steinke@gmail.com>
Added Javier Revillas <jrevillas@massivedynamic.io>
Added Jordi Martin <jordimartin@gmail.com>
Added Jorge Araya <jorgejavieran@yahoo.com.mx>
Added Kelly Heller <pestophagous@gmail.com>
Added Kevin Gillette <extemporalgenome@gmail.com>
Added Mark Glines <mark@glines.org>
Added Nao Yonashiro <owan.orisano@gmail.com>
Added Pascal Dierich <pascal@pascaldierich.com>
Added Pure White <wu.purewhite@gmail.com>
Added Sam Arnold <sarnold64@bloomberg.net>
Added Sander van Harmelen <sander@vanharmelen.nl>
Added Sergei Zagurskii <gvozdoder@gmail.com>
Added Shivani Singhal <shivani.singhal2804@gmail.com>
Added Stefan Baebler <sbaebler@outbrain.com>
Added Tianon Gravi <admwiggin@gmail.com>
Added Toshihiro Shiino <shiino.toshihiro@gmail.com>
Added Wagner Riffel <wgrriffel@gmail.com>
Used GitHub User tatsumack (4510569) form for tatsumack <tatsu.mack@gmail.com> https://github.com/golang/website/commit/7eeb7ef [website]
Used GitHub User utkarsh-extc (53217283) form for utkarsh-extc <53217283+utkarsh-extc@users.noreply.github.com> https://github.com/golang/sys/commit/51ab0e2 [sys]
Used GitHub User yuanhh (1298735) form for yuanhh <yuan415030@gmail.com> https://github.com/golang/crypto/commit/60c769a [crypto]
Used GitHub name "Akhil Indurti" for smasher164 <aindurti@gmail.com> https://github.com/golang/go/commit/5ca44dc403 [build exp go sys]
Used GitHub name "Artem Kolin" for artemkaxboy <artemkaxboy@gmail.com> https://github.com/golang/go/commit/e881604d1c [go]
Used GitHub name "Ivan Markin" for nogoegst <nogoegst@users.noreply.github.com> https://github.com/golang/go/commit/a1addf15df [go]
Used GitHub name "Keiji Yoshida" for yosssi <yoshida.keiji.84@gmail.com> https://github.com/golang/lint/commit/ac6833c [lint]
Used GitHub name "Michalis Kargakis" for kargakis <mkargaki@redhat.com> https://github.com/golang/go/commit/e243d242d7 [go]
Used GitHub name "Roberto Clapis" for Roberto <empijei@users.noreply.github.com> https://github.com/golang/go/commit/963776e689 [go]
Used GitHub name "Robin Eklind" for mewmew <rnd0x00@gmail.com> https://github.com/golang/go/commit/b8620afb8d [blog go proposal.git]
Updates #12042
Change-Id: I1b21a18138849c537048558dd4324a823ba12a14
Reviewed-on: https://go-review.googlesource.com/c/go/+/192099 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit d0eaec79f95c2ad04d58e42c8e7cb123200cdfd5)
Reviewed-on: https://go-review.googlesource.com/c/go/+/192178 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Bryan C. Mills [Tue, 27 Aug 2019 18:18:48 +0000 (14:18 -0400)]
[release-branch.go1.13] cmd/go/internal/get: remove '--' separator from 'git ls-remote' command
'git ls-remote' started recognizing the '--' separator at some point
after 2.7.4, but git defaults to version 2.7.4 on Ubuntu 16.04 LTS,
which remains supported by Ubuntu until April 2021.
We added '--' tokens to most VCS commands as a defensive measure in
CL 181237, but it isn't strictly necessary here because the 'scheme'
argument to our template is chosen from a predefined list: we can
safely drop it to retain compatibility.
Bryan C. Mills [Mon, 26 Aug 2019 20:44:36 +0000 (16:44 -0400)]
[release-branch.go1.13] net/http: fix wantConnQueue memory leaks in Transport
I'm trying to keep the code changes minimal for backporting to Go 1.13,
so it is still possible for a handful of entries to leak,
but the leaks are now O(1) instead of O(N) in the steady state.
Longer-term, I think it would be a good idea to coalesce idleMu with
connsPerHostMu and clear entries out of both queues as soon as their
goroutines are done waiting.
Cherry Zhang [Fri, 23 Aug 2019 18:40:20 +0000 (14:40 -0400)]
[release-branch.go1.13] cmd/vendor: update vendored x/arch repo to 8a70ba74b3a1
Update vendored x/arch repo to pick up the fix of issue #33802.
This is done with the following commands:
$ cd $GOROOT/src/cmd
$ go get -d golang.org/x/arch@latest
go: finding golang.org/x/arch latest
go: downloading golang.org/x/arch v0.0.0-20190815191158-8a70ba74b3a1
go: extracting golang.org/x/arch v0.0.0-20190815191158-8a70ba74b3a1
$ go mod tidy
$ go mod vendor
Updates #33802.
Fixes #33811.
Change-Id: I0a44f1d83d6f573124cea1f099378b1c851f3feb
Reviewed-on: https://go-review.googlesource.com/c/go/+/191619
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 66ff373911a87140319a6550e4e1b2c6043b1329)
Reviewed-on: https://go-review.googlesource.com/c/go/+/191737
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Emmanuel T Odeke [Wed, 21 Aug 2019 05:09:31 +0000 (23:09 -0600)]
[release-branch.go1.13] doc/go1.13: document _ between digits for math/big, strconv
Document that:
* math/big.Float.Parse
* math/big.Int.SetString
* strconv.ParseFloat
* strconv.ParseInt
* strconv.ParseUint
now accept underscores to group digits only if base = 0,
as per the Go 2 language changes.
Updates #32815
Change-Id: Id45bd803a18442436419739297e8aed0d32ca56c
Reviewed-on: https://go-review.googlesource.com/c/go/+/191077 Reviewed-by: Robert Griesemer <gri@golang.org>
(cherry picked from commit e764432605cb26478067890b49f4f16019c9a9b9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/191477
Run-TryBot: Andrew Bonventre <andybons@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
http2: limit number of control frames in server send queue
An attacker could cause servers to queue an unlimited number of PING
ACKs or RST_STREAM frames by soliciting them and not reading them, until
the program runs out of memory.
Limit control frames in the queue to a few thousands (matching the limit
imposed by other vendors) by counting as they enter and exit the scheduler,
so the protection will work with any WriteScheduler.
Once the limit is exceeded, close the connection, as we have no way to
communicate with the peer.
Change-Id: I842968fc6ed3eac654b497ade8cea86f7267886b
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/525552 Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
This change was generated with cmd/go and cmd/bundle:
$ go get -u golang.org/x/net
$ go mod tidy
$ go mod vendor
$ go generate net/http
Fixes CVE-2019-9512 and CVE-2019-9514
Fixes #33606
Filippo Valsorda [Tue, 20 Aug 2019 21:29:04 +0000 (17:29 -0400)]
Revert "encoding/json: avoid work when unquoting strings"
This reverts CL 151157.
CL 151157 introduced a crash when decoding into ",string" fields. It
came with a moderate speedup, so at this stage of the release cycle
let's just revert it, and reapply it in Go 1.14 with the fix in CL 190659.
Also applied the test cases from CL 190659.
Updates #33728
Change-Id: Ie46e2bc15224b251888580daf6b79d5865f3878e
Reviewed-on: https://go-review.googlesource.com/c/go/+/190909
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
Russ Cox [Wed, 21 Aug 2019 14:43:49 +0000 (10:43 -0400)]
cmd/go: accept GOSUMDB=sum.golang.google.cn
This CL makes the go command understand that
GOSUMDB=sum.golang.google.cn should connect
to that domain but expect to find a checksum database
signed by sum.golang.org there.
The host sum.golang.google.cn is not yet completely
configured; we hope it will be available in a few weeks.
Filippo Valsorda [Mon, 19 Aug 2019 17:17:34 +0000 (13:17 -0400)]
cmd/go: fix "go help build -o" docs
The docs refer to "the last two paragraphs", but in fact should refer to
the first two of the previous three paragraphs. Moved up the out of place
paragraph.
Updates #14295
Change-Id: I066da7a665bc6754d246782b941af214a385017a
Reviewed-on: https://go-review.googlesource.com/c/go/+/190839 Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Wagner Riffel [Mon, 19 Aug 2019 03:41:12 +0000 (00:41 -0300)]
doc: rewrite reference to plan9.bell-labs.com to 9p.io
Change-Id: I75619feced842b8ca509ee08e01b63258c5e87ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/190757 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dmitry Vyukov [Sat, 17 Aug 2019 20:22:22 +0000 (13:22 -0700)]
encoding/json: fix format string in the Fuzz func
Currently test build fails with:
$ go test -tags=gofuzz encoding/json
encoding/json/fuzz.go:36:4: Println call has possible formatting directive %s
FAIL encoding/json [build failed]
Change-Id: I23aef44a421ed0e7bcf48b74ac5a8c6768a4841b
Reviewed-on: https://go-review.googlesource.com/c/go/+/190698
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Fri, 16 Aug 2019 17:36:01 +0000 (10:36 -0700)]
time: update TestSub to avoid future regressions
CL 131196 optimized Time.Sub, but was reverted because
it incorrectly computed the nanoseconds in some edge cases.
This CL adds a test case to enforce the correct behavior
so that a future optimization does not break this again.
Updates #17858
Updates #33677
Change-Id: I596d8302ca6bf721cf7ca11cc6f939639fcbdd43
Reviewed-on: https://go-review.googlesource.com/c/go/+/190524
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Joe Tsai [Fri, 16 Aug 2019 17:42:18 +0000 (10:42 -0700)]
cmd/gofmt: update TestRewrite to avoid future regressions
CL 162337 changed go/ast to better handle block comments,
but was reverted because it introduced an off-by-one bug.
This CL adds a test case to enforce the correct behavior
so that future changes do not break this again.
Updates #18929
Updates #33538
Change-Id: I2d25c139d007f8db1091b7a48b1dd20c584e2699
Reviewed-on: https://go-review.googlesource.com/c/go/+/190523
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Cherry Zhang [Wed, 14 Aug 2019 13:50:51 +0000 (09:50 -0400)]
reflect: align first argument in callMethod
When calling a function obtained from reflect.Value.Method (or
MethodByName), we copy the arguments from the caller frame, which
does not include the receiver, to a new frame to call the actual
method, which does include the receiver. Here we need to align
the first (non-receiver) argument. As the receiver is pointer
sized, it is generally naturally aligned, except on amd64p32,
where the argument can have larger alignment, and this aligning
becomes necessary.
Filippo Valsorda [Tue, 6 Aug 2019 23:32:16 +0000 (19:32 -0400)]
net/url: make Hostname and Port predictable for invalid Host values
When Host is not valid per RFC 3986, the behavior of Hostname and Port
was wildly unpredictable, to the point that Host could have a suffix
that didn't appear in neither Hostname nor Port.
This is a security issue when applications are applying checks to Host
and expecting them to be meaningful for the contents of Hostname.
To reduce disruption, this change only aims to guarantee the following
two security-relevant invariants.
* Host is either Hostname or [Hostname] with Port empty, or
Hostname:Port or [Hostname]:Port.
* Port is only decimals.
The second invariant is the one that's most likely to cause disruption,
but I believe it's important, as it's conceivable an application might
do a suffix check on Host and expect it to be meaningful for the
contents of Hostname (if the suffix is not a valid port).
There are three ways to ensure it.
1) Reject invalid ports in Parse. Note that non-numeric ports are
already rejected if and only if the host starts with "[".
2) Consider non-numeric ports as part of Hostname, not Port.
3) Allow non-numeric ports, and hope they only flow down to net/http,
which will reject them (#14353).
This change adopts both 1 and 2. We could do only the latter, but then
these invalid hosts would flow past port checks, like in
http_test.TestTransportRejectsAlphaPort. Non-numeric ports weren't fully
supported anyway, because they were rejected after IPv6 literals, so
this restores consistency. We could do only the former, but at this
point 2) is free and might help with manually constructed Host values
(or if we get something wrong in Parse).
Note that net.SplitHostPort and net.Dial explicitly accept service names
in place of port numbers, but this is an URL package, and RFC 3986,
Section 3.2.3, clearly specifies ports as a number in decimal.
net/http uses a mix of net.SplitHostPort and url.Parse that would
deserve looking into, but in general it seems that it will still accept
service names in Addr fields as they are passed to net.Listen, while
rejecting them in URLs, which feels correct.
This leaves a number of invalid URLs to reject, which however are not
security relevant once the two invariants above hold, so can be done in
Go 1.14: IPv6 literals without brackets (#31024), invalid IPv6 literals,
hostnames with invalid characters, and more.
Tested with 200M executions of go-fuzz and the following Fuzz function.
u, err := url.Parse(string(data))
if err != nil {
return 0
}
h := u.Hostname()
p := u.Port()
switch u.Host {
case h + ":" + p:
return 1
case "[" + h + "]:" + p:
return 1
case h:
fallthrough
case "[" + h + "]":
if p != "" {
panic("unexpected Port()")
}
return 1
}
panic("Host is not a variant of [Hostname]:Port")
Fixes CVE-2019-14809
Updates #29098
Change-Id: I7ef40823dab28f29511329fa2d5a7fb10c3ec895
Reviewed-on: https://go-review.googlesource.com/c/go/+/189258 Reviewed-by: Ian Lance Taylor <iant@golang.org>
K. "pestophagous" Heller [Mon, 5 Aug 2019 05:46:29 +0000 (22:46 -0700)]
doc/install-source: create distinction between steps that involve "git clone"
Prior doc implied that "git clone" was one way to obtain a go1.4
bootstrap toochain, but it did not state this outright. Further,
the doc did not make it explicit in the "Fetch the repository"
section that one must necessarily "git clone" a second time in
the (presumed-to-be-uncommon) case where "git clone" had already
been perfomed in the "compiler binaries" section.
Updates #33402
Change-Id: Id70a6587b6ee09aca13559d63868b75cb07dff1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/188900 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Thu, 8 Aug 2019 22:09:54 +0000 (18:09 -0400)]
cmd/go: query each path only once in 'go get'
If we don't know whether a path is a module path or a package path,
previously we would first try a module query for it, then fall back to
a package query.
If we are using a sequence of proxies with fallback (as will be the
default in Go 1.13), and the path is not actually a module path, that
initial module query will fail against the first proxy, then
immediately fall back to the next proxy in the sequence — even if the
query could have been satisfied by some other (prefix) module
available from the first proxy.
Instead, we now query the requested path as only one kind of path.
If we query it as a package path but it turns out to only exist as a
module, we can detect that as a PackageNotInModuleError with an
appropriate module path — we do not need to issue a second query to
classify it.
Fixes #31785
Change-Id: I581d44279196e41d1fed27ec25489e75d62654e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/189517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Jay Conrod [Wed, 7 Aug 2019 20:27:39 +0000 (16:27 -0400)]
cmd/go: improve 'go mod download' and 'go list -m' error messages
modload.ListModules now wraps errors as module.ModuleError as
appropriate. The resulting errors always include the module path and
will include the version, if known.
'go mod download' no longer ignores errors reported by ListModules.
Previously, it started requesting module info, go.mod, and zip. Those
requests would fail, overwriting the original failure. They were
usually less descriptive.
'go mod download' with a module not in the build list (and no version
query) is now an error. Previously, this was silently ignored.
Fixes #30743
Change-Id: Icee8c1c6c5240de135a8b6ba42d6bbcdb757cdac
Reviewed-on: https://go-review.googlesource.com/c/go/+/189323
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Alberto Donizetti [Wed, 7 Aug 2019 18:11:46 +0000 (20:11 +0200)]
doc: update list of archs supporting -race
Change-Id: Id0a55674a16671aaee99182d9096a9263f7a80b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/189357 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Document goroutine label inheritance. Goroutine labels are copied upon
goroutine creation and there is a test enforcing this, but it was not
mentioned in the docstrings for `Do` or `SetGoroutineLabels`.
Add notes to both of those functions' docstrings so it's clear that one
does not need to set labels as soon as a new goroutine is spawned if
they want to propagate tags.
Updates #32223
Updates #23458
Change-Id: Idfa33031af0104b884b03ca855ac82b98500c8b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/189317 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jay Conrod [Tue, 6 Aug 2019 16:44:30 +0000 (12:44 -0400)]
cmd/go: improve error message for missing import starting with cmd/
In modload.Import, confirm that the import path does not start with
"cmd/" before calling QueryPackage, which returns a less helpful
error.
In load.loadPackageData, don't wrap errors with "unknown import path".
The wrapped error should always include the import path, and it's also
repeated in the PackageError wrapper.
Fixes #31031
Change-Id: I071efa22e3842c62831d096f888a8006811fe724
Reviewed-on: https://go-review.googlesource.com/c/go/+/189157
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Jonathan Amsterdam [Mon, 5 Aug 2019 14:14:40 +0000 (10:14 -0400)]
doc/go1.13.html: describe error-value additions
A brief description at the top the Standard Library section of the
changes to support error wrapping.
Fixes #33365.
Change-Id: Id5a3b2fe148d9bfb949f2cfc9e5d8a1613a0e219
Reviewed-on: https://go-review.googlesource.com/c/go/+/188798 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Damien Neil [Fri, 2 Aug 2019 18:18:56 +0000 (11:18 -0700)]
os: don't consult Is methods on non-syscall error types
CL #163058 moves interpretation of platform-specific errors to the
syscall package. Package syscall errors implement an Is method which
os.IsPermission etc. consult. This results in an unintended semantic
change to the os package predicate functions: The following program
now prints 'true' where it used to print 'false':
package main
import "os"
type myError struct{ error }
func (e myError) Is(target error) bool { return target == os.ErrPermission }
func main() { println(os.IsPermission(myError{})) }
Change the os package error predicate functions to only examine syscall
errors, avoiding this semantic change.
This CL does retain one minor semantic change: On Plan9, os.IsPermission
used to return true for any error with text containing the string
"permission denied". It now only returns true for a syscall.ErrorString
containing that text.
Change-Id: I6b512b1de6ced46c2f1cc8d264fa2495ae7bf9f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/188817
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Keith Randall [Fri, 2 Aug 2019 17:36:03 +0000 (10:36 -0700)]
cmd/compile: fix crash on write barrier insertion
The compiler can crash if the compiled code tries to
unconditionally read from a nil pointer. This should cause
the generated binary to panic, not the compiler.
which now uses the Request.Body's io.ReaderFrom implementation,
if available, and permits system level optimizations such as
"sendfile" to be used to transmit/upload the Body, which greatly
speeds up file uploads.
Updates #33396
Change-Id: I7b8315c4b3e57ad47bb9be2b0c838857875d4bd5
Reviewed-on: https://go-review.googlesource.com/c/go/+/188457 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Damien Neil [Fri, 2 Aug 2019 16:09:27 +0000 (09:09 -0700)]
all: remove os.ErrTimeout
It is unclear whether the current definition of os.IsTimeout is
desirable or not. Drop ErrTimeout for now so we can consider adding it
(or some other error) in a future release with a corrected definition.
Alberto Donizetti [Fri, 2 Aug 2019 17:09:20 +0000 (19:09 +0200)]
doc: fix Block interface parameters order
The Effective Go document references the crypto/cipher Block
interface, but the parameters' names are swapped. This change
alignes them to the crypto definition, to avoid confusion.
Fixes #33432
Change-Id: I8b9aac4dc6af3eec968bbc8f3ee5366b99016fcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/188797 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Damien Neil [Wed, 31 Jul 2019 20:13:39 +0000 (13:13 -0700)]
all: remove os.ErrTemporary
As discussed in
https://github.com/golang/go/issues/32463#issuecomment-506833421
the classification of deadline-based timeouts as "temporary" errors is a
historical accident. I/O timeouts used to be duration-based, so they
really were temporary--retrying a timed-out operation could succeed. Now
that they're deadline-based, timeouts aren't temporary unless you reset
the deadline.
Drop ErrTemporary from Go 1.13, since its definition is wrong. We'll
consider putting it back in Go 1.14 with a clear definition and
deprecate net.OpError.Temporary.
Fixes #32463
Change-Id: I70cda664590d8872541e17409a5780da76920891
Reviewed-on: https://go-review.googlesource.com/c/go/+/188398 Reviewed-by: Jonathan Amsterdam <jba@google.com>
Keith Randall [Tue, 30 Jul 2019 23:14:20 +0000 (16:14 -0700)]
cmd/compile: don't eliminate all registers when restricting to desired ones
We shouldn't mask to desired registers if we haven't masked out all the
forbidden registers yet. In this path we haven't masked out the nospill
registers yet. If the resulting mask contains only nospill registers, then
allocReg fails.
This can only happen on resultNotInArgs-marked instructions, which exist
only on the ARM64, MIPS, MIPS64, and PPC64 ports.
Maybe there's a better way to handle resultNotInArgs instructions.
But for 1.13, this is a low-risk fix.
Fixes #33355
Change-Id: I1082f78f798d1371bde65c58cc265540480e4fa4
Reviewed-on: https://go-review.googlesource.com/c/go/+/188178
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Eli Bendersky [Wed, 31 Jul 2019 20:42:16 +0000 (13:42 -0700)]
encoding/json: clarify Marshal behavior for string keys of maps
This is a documentation-only change.
Fixes #28827
Change-Id: Ife9ab997809048784f35872b09905bc209a05eff
Reviewed-on: https://go-review.googlesource.com/c/go/+/188417 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Tue, 30 Jul 2019 23:16:03 +0000 (16:16 -0700)]
spec: add an example of a trivially invalid interface
In preparation for the forthcoming spec changes for #6977.
While at it, modernize existing File example that dates
back all the way to commit 18c5b488a3b.
Change-Id: Id10e4df0513e3de15bd58867222923eefa9473ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/187978 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/go/internal/cache: avoid ioutil.WriteFile for writing cache entries
ioutil.WriteFile always truncates the destination file to 0 before
writing, which is inappropriate for unsynchronized, idempotent,
fixed-size files such as the cache entry files here.
Instead, truncate the file only after writing it, so that a second
write will never (even temporarily!) remove the contents of a
preceding write.
Fixes #29667
Change-Id: I16a53ce79d8a23d23580511cb6abd062f54b65ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/188157
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Michael Knyszek [Fri, 19 Jul 2019 23:35:24 +0000 (19:35 -0400)]
runtime: call sysHugePage less often
Currently when we coalesce memory we make a sysHugePage call
(MADV_HUGEPAGE) to ensure freed and coalesced huge pages are treated as
such so the scavenger's assumptions about performance are more in line
with reality.
Unfortunately we do it way too often because we do it if there was any
change to the huge page count for the span we're coalescing into, not
taking into account that it could coalesce with its neighbors and not
actually create a new huge page.
This change makes it so that it only calls sysHugePage if the original
huge page counts between the span to be coalesced into and its neighbors
do not add up (i.e. a new huge page was created due to alignment). Calls
to sysHugePage will now happen much less frequently, as intended.
Michael Anthony Knyszek [Tue, 16 Jul 2019 20:36:33 +0000 (20:36 +0000)]
runtime: add physHugePageShift
This change adds physHugePageShift which is defined such that
1 << physHugePageShift == physHugePageSize. The purpose of this variable
is to avoid doing expensive divisions in key functions, such as
(*mspan).hugePages.
This change also does a sweep of any place we might do a division or mod
operation with physHugePageSize and turns it into bit shifts and other
bitwise operations.
Finally, this change adds a check to mallocinit which ensures that
physHugePageSize is always a power of two. osinit might choose to ignore
non-powers-of-two for the value and replace it with zero, but mallocinit
will fail if it's not a power of two (or zero). It also derives
physHugePageShift from physHugePageSize.
This change helps improve the performance of most applications because
of how often (*mspan).hugePages is called.
Andrew Todd [Sat, 27 Jul 2019 23:20:49 +0000 (16:20 -0700)]
strings: clarify usage of Title and ToTitle
This is intended to help clear up confusion around the usage of the
Title and ToTitle functions. It includes a link to define title case
to distinguish it from upper case. It also includes an additional
example for the ToTitle function to showcase the difference in behavior
between it and the Title function.
Fixes #33302
Change-Id: I44e62962fb04d0d22966a39eda3a2d16de7a2291
Reviewed-on: https://go-review.googlesource.com/c/go/+/187825 Reviewed-by: Rob Pike <r@golang.org>
Keith Randall [Mon, 29 Jul 2019 19:51:19 +0000 (12:51 -0700)]
runtime: use uintptr instead of int32 for counting to next heap profile sample
Overflow of the comparison caused very large (>=1<<32) allocations to
sometimes not get sampled at all. Use uintptr so the comparison will
never overflow.
Fixes #33342
Tested on the example in 33342. I don't want to check a test in that
needs that much memory, however.
Daniel Martí [Sat, 27 Jul 2019 21:53:54 +0000 (14:53 -0700)]
doc: use <code> for a few file paths that didn't
Spotted while seeing CL 187818 in master.
Change-Id: Ic3f42558f6833fc35580b2b2a15f2c5aba5713c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/187822 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Giovanni Bajo [Sat, 27 Jul 2019 18:55:54 +0000 (11:55 -0700)]
doc: mention that GOPATH/bin should be in PATH
As noticed while mentoring at Gophercon, a common mistake for
beginners is that, after they install git-codereview, it's not
found by git because they haven't put GOPATH/bin in their
environment.