]> Cypherpunks repositories - gostls13.git/log
gostls13.git
6 years agonet/http: remove unnecessary string replace operation in Cookie.String
bronze1man [Mon, 14 Jan 2019 15:41:02 +0000 (15:41 +0000)]
net/http: remove unnecessary string replace operation in Cookie.String

Fixes #29135

Change-Id: I4c10b0395047775e8488b8b0f00f74a7fa01b86c
GitHub-Last-Rev: 120977040506794f00c74383289f913b1e0edd4a
GitHub-Pull-Request: golang/go#29728
Reviewed-on: https://go-review.googlesource.com/c/go/+/157777
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agonet/http: speed up parsing of Cookie headers
sergey [Sun, 24 Feb 2019 11:26:55 +0000 (14:26 +0300)]
net/http: speed up parsing of Cookie headers

Parse the headers without splitting them upfront to reduce
memory allocations.

For non-pathological Cookie headers we can make
a good estimate of the number of cookies in there and preallocate
the slice of cookies

name              old time/op    new time/op    delta
CookieString-4      1.73µs ± 2%    1.70µs ± 5%     ~     (p=0.841 n=5+5)
ReadSetCookies-4    6.09µs ± 3%    5.93µs ± 3%     ~     (p=0.095 n=5+5)
ReadCookies-4       7.63µs ± 1%    6.41µs ± 4%  -15.99%  (p=0.008 n=5+5)

name              old alloc/op   new alloc/op   delta
CookieString-4        360B ± 0%      360B ± 0%     ~     (all equal)
ReadSetCookies-4      976B ± 0%      976B ± 0%     ~     (all equal)
ReadCookies-4       2.17kB ± 0%    1.84kB ± 0%  -15.13%  (p=0.008 n=5+5)

name              old allocs/op  new allocs/op  delta
CookieString-4        5.00 ± 0%      5.00 ± 0%     ~     (all equal)
ReadSetCookies-4      15.0 ± 0%      15.0 ± 0%     ~     (all equal)
ReadCookies-4         16.0 ± 0%      11.0 ± 0%  -31.25%  (p=0.008 n=5+5)

Change-Id: Ica1ca0d40c0d8d275134d1dfafb73f1082115826
Reviewed-on: https://go-review.googlesource.com/c/go/+/163617
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agonet/http: add Server BaseContext & ConnContext fields to control early context
Brad Fitzpatrick [Fri, 15 Mar 2019 14:58:55 +0000 (14:58 +0000)]
net/http: add Server BaseContext & ConnContext fields to control early context

Fixes golang/go#30694

Change-Id: I12a0a870e4aee6576e879d88a4868666ef448298
Reviewed-on: https://go-review.googlesource.com/c/go/+/167681
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: JP Sugarbroad <jpsugar@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agocmd/compile: remove outdate comment of treecopy
LE Manh Cuong [Mon, 15 Apr 2019 18:51:14 +0000 (01:51 +0700)]
cmd/compile: remove outdate comment of treecopy

Since golang.org/cl/32487, treecopy does not handle non-iota ONONAME and
iota ONONAME anymore.

Change-Id: Icd5a81333a0d4d04adef2dbc58db92ce67aa0860
Reviewed-on: https://go-review.googlesource.com/c/go/+/172038
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
6 years agoruntime: remove some unused fields in m and p
Austin Clements [Sat, 13 Apr 2019 14:27:35 +0000 (10:27 -0400)]
runtime: remove some unused fields in m and p

Change-Id: Ie0171f48aaf48d8399ef578f95352445741d83a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/171773
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agocmd/link: deduplicate pctab info in pclntab
Josh Bleecher Snyder [Mon, 15 Apr 2019 16:39:29 +0000 (09:39 -0700)]
cmd/link: deduplicate pctab info in pclntab

The existing pclntab construction took care to re-use strings:
filenames and fully qualified function names.

It did not try to deduplicate pctab information,
perhaps because the author assumed that there
wouldn't be much duplication.

This change introduces that deduplication.
The cache gets a 33% hit rate during make.bash.

This doesn't require any changes to the file format,
and shrinks binaries by about 1%.

Updates #6853

file      before    after     Δ        %
go        14659236  14515876  -143360  -0.978%
addr2line 4272424   4223272   -49152   -1.150%
api       6050808   5993464   -57344   -0.948%
asm       4906416   4869552   -36864   -0.751%
buildid   2861104   2824240   -36864   -1.288%
cgo       4859784   4810632   -49152   -1.011%
compile   25749656  25213080  -536576  -2.084%
cover     5286952   5229608   -57344   -1.085%
dist      3634192   3597328   -36864   -1.014%
doc       4691080   4641928   -49152   -1.048%
fix       3397960   3361096   -36864   -1.085%
link      6113568   6064432   -49136   -0.804%
nm        4221928   4172776   -49152   -1.164%
objdump   4636600   4587448   -49152   -1.060%
pack      2281184   2256608   -24576   -1.077%
pprof     14641204  14485556  -155648  -1.063%
test2json 2814536   2785864   -28672   -1.019%
trace     11602204  11487516  -114688  -0.989%
vet       8399528   8313512   -86016   -1.024%

Change-Id: I59c6aae522700a0d36ddd2cbca6e22ecdf17eea2
Reviewed-on: https://go-review.googlesource.com/c/go/+/172079
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agocmd/link: clean up pclntab
Josh Bleecher Snyder [Mon, 15 Apr 2019 16:25:23 +0000 (09:25 -0700)]
cmd/link: clean up pclntab

More minor cleanup:

* Code simplification
* Move variable declaration closer to use
* Add docs
* Refactor loop

Change-Id: I6f662cb65038b6ad927eb83757b241ac1ef58943
Reviewed-on: https://go-review.googlesource.com/c/go/+/172078
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agoos: don't treat RemoveAll("/x") as RemoveAll("x")
Ian Lance Taylor [Mon, 15 Apr 2019 17:36:17 +0000 (10:36 -0700)]
os: don't treat RemoveAll("/x") as RemoveAll("x")

Fixes #31468

Change-Id: I5c4e61631b8af35bfc14b0cb9bc77feec100e340
Reviewed-on: https://go-review.googlesource.com/c/go/+/172058
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agocrypto/sha512: use math/bits.RotateLeft64 instead of ad-hoc implementation
Udalov Max [Sat, 13 Apr 2019 15:54:03 +0000 (18:54 +0300)]
crypto/sha512: use math/bits.RotateLeft64 instead of ad-hoc implementation

This makes code more readable and idiomatic and slightly increase performance.

Updates #31456

Benchstat:
name          old time/op    new time/op    delta
Hash8Bytes-8     281ns ± 4%     280ns ± 3%    ~     (p=0.640 n=10+10)
Hash1K-8        2.01µs ± 6%    2.02µs ± 3%    ~     (p=0.481 n=10+10)
Hash8K-8        14.2µs ± 6%    13.5µs ± 1%  -4.90%  (p=0.001 n=10+10)

name          old speed      new speed      delta
Hash8Bytes-8  28.5MB/s ± 4%  28.5MB/s ± 3%    ~     (p=0.516 n=10+10)
Hash1K-8       510MB/s ± 6%   507MB/s ± 4%    ~     (p=0.481 n=10+10)
Hash8K-8       576MB/s ± 6%   605MB/s ± 1%  +5.02%  (p=0.001 n=10+10)

Tested on macbook pro 2018 Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz

Change-Id: I1f5b78096dd49d14ffcb9129142c4a4e05b81ff9
Reviewed-on: https://go-review.googlesource.com/c/go/+/171736
Reviewed-by: Filippo Valsorda <filippo@golang.org>
6 years agomath/big: don't clobber shared underlying array in pow5 computation
Robert Griesemer [Thu, 4 Apr 2019 00:12:27 +0000 (17:12 -0700)]
math/big: don't clobber shared underlying array in pow5 computation

Rearranged code slightly to make lifetime of underlying array of
pow5 more explicit in code.

Fixes #31184.

Change-Id: I063081f0e54097c499988d268a23813746592654
Reviewed-on: https://go-review.googlesource.com/c/go/+/170641
Reviewed-by: Filippo Valsorda <filippo@golang.org>
6 years agocmd/compile: add new escape analysis implementation
Matthew Dempsky [Tue, 2 Apr 2019 17:40:12 +0000 (10:40 -0700)]
cmd/compile: add new escape analysis implementation

This CL adds a new escape analysis implementation, which can be
enabled through the -newescape compiler flag.

This implementation focuses on simplicity, but in the process ends up
using less memory, speeding up some compile-times, fixing memory
corruption issues, and overall significantly improving escape analysis
results.

Updates #23109.

Change-Id: I6176d9a7ae9d80adb0208d4112b8a1e1f4c9143a
Reviewed-on: https://go-review.googlesource.com/c/go/+/170322
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
6 years agoruntime: fix GDB tests on s390x running Ubuntu 18.04
Michael Munday [Thu, 28 Mar 2019 12:05:43 +0000 (08:05 -0400)]
runtime: fix GDB tests on s390x running Ubuntu 18.04

On Ubuntu 18.04 I am seeing GDB fail to restore the stack pointer
during this test because stack unwinding can't find the PC. This CL
is essentially a partial revert of CL 23940 and fixes the issue on
s390x.

Change-Id: Ib4c41162dc85dc882eb6e248330f4082c3fa94c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/169857
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
6 years agocmd/link, runtime: mark goexit as the top of the call stack
Michael Munday [Thu, 28 Mar 2019 16:51:30 +0000 (12:51 -0400)]
cmd/link, runtime: mark goexit as the top of the call stack

This CL adds a new attribute, TOPFRAME, which can be used to mark
functions that should be treated as being at the top of the call
stack. The function `runtime.goexit` has been marked this way on
architectures that use a link register.

This will stop programs that use DWARF to unwind the call stack
from unwinding past `runtime.goexit` on architectures that use a
link register. For example, it eliminates "corrupt stack?"
warnings when generating a backtrace that hits `runtime.goexit`
in GDB on s390x.

Similar code should be added for non-link-register architectures
(i.e. amd64, 386). They mark the top of the call stack slightly
differently to link register architectures so I haven't added
that code (they need to mark "rip" as undefined).

Fixes #24385.

Change-Id: I15b4c69ac75b491daa0acf0d981cb80eb06488de
Reviewed-on: https://go-review.googlesource.com/c/go/+/169726
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
6 years agocmd/dist: enable cgo for freebsd/arm
Yuval Pavel Zholkover [Sat, 13 Apr 2019 11:31:07 +0000 (14:31 +0300)]
cmd/dist: enable cgo for freebsd/arm

Change-Id: Icc1a54da848bf446919c0d5470d1e79fad339832
Reviewed-on: https://go-review.googlesource.com/c/go/+/171727
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agogo/token: add IsIdentifier, IsKeyword, and IsExported
Daniel Martí [Sat, 23 Mar 2019 16:20:35 +0000 (16:20 +0000)]
go/token: add IsIdentifier, IsKeyword, and IsExported

Telling whether a string is a valid Go identifier can seem like an easy
task, but it's easy to forget about the edge cases. For example, some
implementations out there forget that an empty string or keywords like
"func" aren't valid identifiers.

Add a simple implementation with proper Unicode support, and start using
it in cmd/cover and cmd/doc. Other pieces of the standard library
reimplement part of this logic, but don't use a "func(string) bool"
signature, so we're leaving them untouched for now.

Add some tests too, to ensure that we actually got these edge cases
correctly.

Since telling whether a string is a valid identifier requires knowing
that it's not a valid keyword, add IsKeyword too. The internal map was
already accessible via Lookup, but "Lookup(str) != IDENT" isn't as easy
to understand as IsKeyword(str). And, as per Josh's suggestion, we could
have IsKeyword (and probably Lookup too) use a perfect hash function
instead of a global map.

Finally, for consistency with these new functions, add IsExported. That
makes go/ast.IsExported a bit redundant, so perhaps it can be deprecated
in favor of go/token.IsExported in the future. Clarify that
token.IsExported doesn't imply token.IsIdentifier, to avoid ambiguity.

Fixes #30064.

Change-Id: I0e0e49215fd7e47b603ebc2b5a44086c51ba57f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/169018
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
6 years agostrings: remove "a copy of the string" from ToUpper/ToLower comments
Максадбек Ахмедов [Sat, 13 Apr 2019 14:38:36 +0000 (17:38 +0300)]
strings: remove "a copy of the string" from ToUpper/ToLower comments

When string letters are all in lower/upper cases, both functions respectively
return original string.

Fixes #30987

Change-Id: Ie8d664f7af5e087f82c1bc156933e9a995645bf4
Reviewed-on: https://go-review.googlesource.com/c/go/+/171735
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agocmd/dist: enable VFPv3 on freebsd/arm
Yuval Pavel Zholkover [Sat, 13 Apr 2019 11:35:23 +0000 (14:35 +0300)]
cmd/dist: enable VFPv3 on freebsd/arm

Since CL 165799 was merged, VFP detection should work properly on FreeBSD.

Updates #27619

Change-Id: I386e856ceb54f0bf6e6bf83bf2d1e19154ba53f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/171728
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
6 years agocmd/compile/internal/arm: merge cases in ssaGenValue
Tobias Klauser [Fri, 12 Apr 2019 12:03:39 +0000 (14:03 +0200)]
cmd/compile/internal/arm: merge cases in ssaGenValue

Merge case statement for OpARMSLL, OpARMSRL and OpARMSRA into an
existing one using the same logic.

Change-Id: Ic4224668228902e5188fb0559b5f1949cfea1381
Reviewed-on: https://go-review.googlesource.com/c/go/+/171724
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
6 years agocrypto/sha1: use math/bits.RotateLeft32 instead of ad-hoc implementation.
Udalov Max [Sat, 13 Apr 2019 13:18:15 +0000 (16:18 +0300)]
crypto/sha1: use math/bits.RotateLeft32 instead of ad-hoc implementation.

This makes code more idiomatic and shows small performance gains of generic benchmarks.

Updates: #31456

name            old time/op    new time/op    delta
Hash8Bytes-8       275ns ± 4%     270ns ± 0%    ~     (p=0.213 n=9+8)
Hash320Bytes-8    1.46µs ± 5%    1.39µs ± 1%  -4.54%  (p=0.000 n=10+10)
Hash1K-8          3.99µs ± 5%    3.86µs ± 1%  -3.38%  (p=0.023 n=10+10)
Hash8K-8          28.9µs ± 0%    28.9µs ± 1%    ~     (p=0.315 n=10+10)

name            old speed      new speed      delta
Hash8Bytes-8    28.8MB/s ± 9%  29.6MB/s ± 0%    ~     (p=0.151 n=10+8)
Hash320Bytes-8   220MB/s ± 5%   230MB/s ± 1%  +4.65%  (p=0.000 n=10+10)
Hash1K-8         257MB/s ± 5%   265MB/s ± 1%  +3.38%  (p=0.023 n=10+10)
Hash8K-8         283MB/s ± 0%   284MB/s ± 1%    ~     (p=0.315 n=10+10)

Change-Id: Iee63aa042614e3bbeda9aaf5236180d4153f03c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/171729
Reviewed-by: Ilya Tokar <tocarip@gmail.com>
Run-TryBot: Ilya Tokar <tocarip@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agoencoding/json: remove a bounds check in readValue
Daniel Martí [Fri, 5 Apr 2019 21:06:49 +0000 (23:06 +0200)]
encoding/json: remove a bounds check in readValue

readValue is a hot function, clocking in at ~13% flat CPU use in
CodeDecoder. In particular, looping over the bytes is slow. That's
partially because the code contains a bounds check at the start of the
loop.

The source of the problem is that scanp is a signed integer, and comes
from a field, so the compiler doesn't know that it's non-negative. Help
it with a simple and comparatively cheap hint.

While at it, use scanp as the index variable directly, removing the need
for a duplicate index variable which is later added back into scanp.

name           old time/op    new time/op    delta
CodeDecoder-8    11.3ms ± 1%    11.2ms ± 1%  -0.98%  (p=0.000 n=9+9)

name           old speed      new speed      delta
CodeDecoder-8   172MB/s ± 1%   174MB/s ± 1%  +0.99%  (p=0.000 n=9+9)

Updates #28923.

Change-Id: I138f83babdf316fc97697cc18f595c3403c1ddb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/170939
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agocmd/go/internal/test: pass default timeout to test programs if not given from command...
OkamotoYuki [Sun, 10 Mar 2019 02:38:12 +0000 (02:38 +0000)]
cmd/go/internal/test: pass default timeout to test programs if not given from command line

Make 'go test' command to pass the default timeout (10m) to test programs if the value is not given from command line.

Fixes #28147

Change-Id: I7856e452224a51a92da03bab8e3a0f9d7c41d32a
GitHub-Last-Rev: 66f9a6f90e9ffe7c58d5c1fe32af84e16ea74ab8
GitHub-Pull-Request: golang/go#30545
Reviewed-on: https://go-review.googlesource.com/c/go/+/164963
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
6 years agoreflect: add Value.IsZero
Romain Baugue [Wed, 10 Apr 2019 14:20:43 +0000 (16:20 +0200)]
reflect: add Value.IsZero

Fixes #7501

Change-Id: Iac7c79cd4b30a90b14ed84bf1eba758972232a6c
Reviewed-on: https://go-review.googlesource.com/c/go/+/171337
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agocmd/link: make Pciter more idiomatic
Josh Bleecher Snyder [Fri, 12 Apr 2019 20:06:52 +0000 (13:06 -0700)]
cmd/link: make Pciter more idiomatic

Rename it to PCIter and convert it to use methods.

Set pcscale once, during construction, to make call sites clearer.

Change some ints to bools.

Use a simple iteration termination condition,
instead of the cap comparison from the c2go translation.

Instead of requiring a Pcdata, which requires one caller
to synthesize a fake Pcdata, just ask for a byte slice.

Passes toolstash-check.

Change-Id: I811da0e929cf4a806bd6d70357ccf2911cd0c737
Reviewed-on: https://go-review.googlesource.com/c/go/+/171770
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agocmd/link, cmd/internal/obj: use encoding/binary for varint
Josh Bleecher Snyder [Fri, 12 Apr 2019 16:29:43 +0000 (09:29 -0700)]
cmd/link, cmd/internal/obj: use encoding/binary for varint

This code was written before the c2go toolchain conversion.
Replace the handwritten varint encoding routines
and the handwritten unsigned-to-signed conversions
with calls to encoding/binary.

Passes toolstash-check.

Change-Id: I30d7f408cde3772ee98a3825e83075c4e1ec96d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/171769
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agocmd/internal/obj: stop plist flushing early on error
Josh Bleecher Snyder [Fri, 12 Apr 2019 19:02:49 +0000 (12:02 -0700)]
cmd/internal/obj: stop plist flushing early on error

If preprocessing or assembling has failed, we should not proceed.
First, there's no point.
Second, I will shortly add some sanity checks to linkpcln
that will fail on malformed input.

Change-Id: I055eeab1c2f3a66b4b2cadb551bbf4ab55d176f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/171886
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agoruntime: add //go:cgo_unsafe_args to AIX syscallX functions
Clément Chigot [Fri, 12 Apr 2019 12:23:13 +0000 (14:23 +0200)]
runtime: add //go:cgo_unsafe_args to AIX syscallX functions

On AIX, syscallX functions are using the first argument to retrieve the
next arguments when calling asmcgocall. Therefore,//go:cgo_unsafe_args
is needed.

Change-Id: I7fe0fbf0c961250a6573c66a8b0eb897dff94bfe
Reviewed-on: https://go-review.googlesource.com/c/go/+/171723
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agoruntime: remove debug prints in netpoll_aix.go
Clément Chigot [Fri, 12 Apr 2019 09:32:48 +0000 (11:32 +0200)]
runtime: remove debug prints in netpoll_aix.go

Change-Id: I80cca386de23cde39ab4ed3be9878374dc7607ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/171721
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agosyscall: add aix to syscall_unix_test.go
Clément Chigot [Wed, 10 Apr 2019 15:16:48 +0000 (17:16 +0200)]
syscall: add aix to syscall_unix_test.go

This file was forgotten during the port of aix/ppc64. In order to make
its tests passed, a few things were added:
- Add termios.h to zerrors
- Add AF_LOCAL = AF_UNIX as this constant doesn't exits natively on AIX
- Fix the alignment in cmsghdr structure.

TestPassFD doesn't work on AIX TL<2 because getsockname isn't working as
expected with unix socket.

Change-Id: I928705bfc78ada29e66df61fe97d8f379f8c739b
Reviewed-on: https://go-review.googlesource.com/c/go/+/171339
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agonet: retrieve if unix network is available only once for AIX
Clément Chigot [Thu, 11 Apr 2019 14:26:38 +0000 (16:26 +0200)]
net: retrieve if unix network is available only once for AIX

The previous version was executing "oslevel -s" everytime testableNetwork
was called with unix/unixgram network. The current version retrieves if
the network is possible only once at the beginning of the tests.

This is clearly faster:
ok      net     74.045s
ok      net     5.098s

Change-Id: I12549da27721f85c007cf17cab5cfdbfeb839cf6
Reviewed-on: https://go-review.googlesource.com/c/go/+/171717
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agodoc/go_faq: fix grammar
Daniel Langner [Fri, 12 Apr 2019 12:21:16 +0000 (12:21 +0000)]
doc/go_faq: fix grammar

Change-Id: Idbd6c97d754e3565aeade4d9e8011a76e8da19c2
GitHub-Last-Rev: 22e917e5ca429d76506346841e5b3c93c2a5f3fb
GitHub-Pull-Request: golang/go#31439
Reviewed-on: https://go-review.googlesource.com/c/go/+/171885
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoruntime: delete unused isscanstatus function
Austin Clements [Thu, 11 Apr 2019 20:48:05 +0000 (16:48 -0400)]
runtime: delete unused isscanstatus function

Change-Id: I693250e980cc60ea151736b42ac6b1426ab801b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/171765
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agobootstrap.bash: fix build if no exec wrapper exists
Tobias Klauser [Fri, 12 Apr 2019 12:20:21 +0000 (14:20 +0200)]
bootstrap.bash: fix build if no exec wrapper exists

Since CL 170941 bootstrap.bash fails if no exec wrapper exists for a
given GOOS/GOARCH yet:

  #### Building ../../go-linux-arm-bootstrap

  Building Go cmd/dist using /usr/local/go.
  Building Go toolchain1 using /usr/local/go.
  Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
  Building Go toolchain2 using go_bootstrap and Go toolchain1.
  Building Go toolchain3 using go_bootstrap and Go toolchain2.
  Building packages and commands for host, linux/amd64.
  Building packages and commands for target, linux/arm.
  rm: cannot remove 'bin/go_linux_arm_exec': No such file or directory

Fix it by using 'rm -f' to ignore nonexisting files.

Change-Id: Ib4b1e19747052fa4dca06319f35cc25a0545c4f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/171722
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Elias Naur <mail@eliasnaur.com>
6 years agocmd/link: fix off-by-one in ftabaddstring
Josh Bleecher Snyder [Fri, 12 Apr 2019 01:05:19 +0000 (18:05 -0700)]
cmd/link: fix off-by-one in ftabaddstring

ftabaddstring adds a string to the pclntab.
The pclntab uses C strings, so the code added 1 to the length.
However, it also added an extraneous 1 in the Grow call. Remove that.

While we're here, simplify, document, remove an unnecessary parameter,
and remove some unnecessary conversions.

Shaves off a few bytes here and there, and thus updates #6853.

file      before    after     Δ       %
go        14671316  14659028  -12288  -0.084%
addr2line 4280552   4276456   -4096   -0.096%
api       6058936   6050744   -8192   -0.135%
buildid   2861040   2856944   -4096   -0.143%
cgo       4867912   4863816   -4096   -0.084%
compile   25770104  25753720  -16384  -0.064%
cover     5286888   5282792   -4096   -0.077%
dist      3634048   3629952   -4096   -0.113%
doc       4691000   4686904   -4096   -0.087%
fix       3393736   3389640   -4096   -0.121%
link      6109280   6105184   -4096   -0.067%
nm        4225960   4221864   -4096   -0.097%
objdump   4636520   4632424   -4096   -0.088%
pack      2285200   2281104   -4096   -0.179%
pprof     14657508  14645220  -12288  -0.084%
test2json 2818568   2814472   -4096   -0.145%
trace     11618524  11610332  -8192   -0.071%
vet       8403544   8395352   -8192   -0.097%

Change-Id: I20b1f541de5d3ed326dd937aad6a43801862df51
Reviewed-on: https://go-review.googlesource.com/c/go/+/171820
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agoruntime: move libcall to stack for syscall package on aix/ppc64
Clément Chigot [Wed, 10 Apr 2019 14:29:57 +0000 (16:29 +0200)]
runtime: move libcall to stack for syscall package on aix/ppc64

Inside syscall_syscall6 function, libcall can directly be on the stack.
This is first function called with //go:nosplit, unlike runtime syscalls
which can be called during the sigtramp or by others //go:nosplit
functions.

Change-Id: Icc28def1a63e525850ec3bfb8184b995dfeaa736
Reviewed-on: https://go-review.googlesource.com/c/go/+/171338
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agoos: don't leak file in ExampleOpenFile_append failure path
james [Thu, 11 Apr 2019 20:18:29 +0000 (16:18 -0400)]
os: don't leak file in ExampleOpenFile_append failure path

Fixes #31424

Change-Id: I8364578cbc77827552bd764c716f68495ec51547
Reviewed-on: https://go-review.googlesource.com/c/go/+/171763
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agocmd/go: log failures to remove workdir
Bryan C. Mills [Thu, 11 Apr 2019 20:35:56 +0000 (16:35 -0400)]
cmd/go: log failures to remove workdir

Failures here don't otherwise affect the build, but they do cause a
slow file leak in the user's temp directory. The user deserves at
least a cursory warning that something may be amiss.

Updates #30789

Change-Id: Id0e72b1967e7f7c88cdc94d532554496653f264b
Reviewed-on: https://go-review.googlesource.com/c/go/+/171764
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
6 years agodoc: document Go 1.12.4 and Go 1.11.9
Brad Fitzpatrick [Thu, 11 Apr 2019 04:57:14 +0000 (04:57 +0000)]
doc: document Go 1.12.4 and Go 1.11.9

Updates #31293

Change-Id: I3d72f732be7b28059310ea6fc134c3bfac81492d
Reviewed-on: https://go-review.googlesource.com/c/go/+/171578
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
6 years agoruntime: abstract initializing and destroying Ps out of procresize
Austin Clements [Thu, 11 Apr 2019 19:43:54 +0000 (15:43 -0400)]
runtime: abstract initializing and destroying Ps out of procresize

procresize is rather ungainly because it includes all of the code to
initialize new Ps and tear down unused Ps. Pull these out into their
own methods on the p type. This also tweaks the initialization loop in
procresize so we only initialize new Ps (and Ps we previously
destroyed) rather than asking each P "have you been initialized?"

This is for #10958 and #24543, but it's also just a nice cleanup.

Change-Id: Ic1242066f572c94a23cea8ea4dc47c918e31d559
Reviewed-on: https://go-review.googlesource.com/c/go/+/171762
Reviewed-by: Michael Knyszek <mknyszek@google.com>
6 years agoruntime: note about improved _Gscan bit handling
Austin Clements [Wed, 3 Apr 2019 20:32:42 +0000 (16:32 -0400)]
runtime: note about improved _Gscan bit handling

Change-Id: I8de5aa64a24e77e0ef876918fcace7668769ebc2
Reviewed-on: https://go-review.googlesource.com/c/go/+/171022
Reviewed-by: Michael Knyszek <mknyszek@google.com>
6 years agotest: add testcases for gccgo bug
Than McIntosh [Thu, 11 Apr 2019 19:47:33 +0000 (15:47 -0400)]
test: add testcases for gccgo bug

Add a couple of testcase for a gccgo type checking bug.

Updates #31412.

Change-Id: I7a813dafde78e4add1432602d2af5fe879415e1c
Reviewed-on: https://go-review.googlesource.com/c/go/+/171761
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agoos: fix aliasing bug in RemoveAllTestHook restoration
Bryan C. Mills [Thu, 11 Apr 2019 19:42:21 +0000 (15:42 -0400)]
os: fix aliasing bug in RemoveAllTestHook restoration

The code to swap RemoveAllTestHook in and out in
TestRemoveAllWithMoreErrorThanReqSize was making a copy of the
RemoveAllTestHook pointer, then attempting to restore by loading from
the copy of that pointer. Since the two copies of the pointer aliased
the same address, the restore operation had no effect, and any
RemoveAll tests that happened to run after
TestRemoveAllWithMoreErrorThanReqSize would fail.

Fixes #31421

Change-Id: I7028475f5ceb3b0a2fa69d22af8d3379508c4531
Reviewed-on: https://go-review.googlesource.com/c/go/+/171777
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoruntime: set itab.fun[0] only on successful conversion
Cherry Zhang [Thu, 11 Apr 2019 18:00:07 +0000 (14:00 -0400)]
runtime: set itab.fun[0] only on successful conversion

For a failed interface conversion not in ",ok" form, getitab
calls itab.init to get the name of the missing method for the
panic message. itab.init will try to find the methods, populate
the method table as it goes. When some method is missing, it sets
itab.fun[0] to 0 before return. There is a small window that
itab.fun[0] could be non-zero.

If concurrently, another goroutine tries to do the same interface
conversion, it will read the same itab's fun[0]. If this happens
in the small window, it sees a non-zero fun[0] and thinks the
conversion succeeded, which is bad.

Fix the race by setting fun[0] to non-zero only when we know the
conversion succeeds. While here, also simplify the syntax
slightly.

Fixes #31419.

Change-Id: Ied34d3043079eb933e330c5877b85e13f98f1916
Reviewed-on: https://go-review.googlesource.com/c/go/+/171759
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
6 years agosyscall: enforce minimum buffer size to call ReadDirent
Keith Randall [Thu, 11 Apr 2019 16:09:39 +0000 (09:09 -0700)]
syscall: enforce minimum buffer size to call ReadDirent

freebsd and netbsd require a minimum buffer size of 1K.

Note this doesn't quite fix freebsd, it has other bugs,
I'll file a separate issue.

Fixes #31403

Change-Id: I9d7e78f6d30859b34715afadc4b8bd3b1ecc606b
Reviewed-on: https://go-review.googlesource.com/c/go/+/171757
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agocmd/link/internal/ld: fix c-archive mach-o compatibility
Aman Gupta [Tue, 2 Apr 2019 11:35:50 +0000 (04:35 -0700)]
cmd/link/internal/ld: fix c-archive mach-o compatibility

These workarounds predate proper DWARF support
and are no longer necessary.

Before this patch, running `/usr/bin/symbols go.o`
using the object in the c-archive would fail, causing
App Store rejections.

Fixes #31022 #28997

Change-Id: I6a210b6369c13038777c6e21e874e81afcb50c2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/170377
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agoruntime: introduce treapForSpan to reduce code duplication
Michael Anthony Knyszek [Fri, 29 Mar 2019 16:02:05 +0000 (16:02 +0000)]
runtime: introduce treapForSpan to reduce code duplication

Currently which treap a span should be inserted into/removed from is
checked by looking at the span's properties. This logic is repeated in
four places. As this logic gets more complex, it makes sense to
de-duplicate this, so introduce treapForSpan instead which captures this
logic by returning the appropriate treap for the span.

For #30333.

Change-Id: I4bd933d93dc50c5fc7c7c7f56ceb95194dcbfbcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/170857
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
6 years agoruntime: add tests for runtime mTreap
Michael Anthony Knyszek [Sat, 9 Feb 2019 00:13:37 +0000 (00:13 +0000)]
runtime: add tests for runtime mTreap

This change exports the runtime mTreap in export_test.go and then adds a
series of tests which check that the invariants of the treap are
maintained under different operations. These tests also include tests
for the treap iterator type.

Also, we note that the find() operation on the treap never actually was
best-fit, so the tests just ensure that it returns an appropriately
sized span.

For #30333.

Change-Id: If81f7c746dda6677ebca925cb0a940134701b894
Reviewed-on: https://go-review.googlesource.com/c/go/+/164100
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
6 years agoruntime: throw if scavenge necessary during coalescing
Michael Anthony Knyszek [Thu, 17 Jan 2019 20:05:20 +0000 (20:05 +0000)]
runtime: throw if scavenge necessary during coalescing

Currently when coalescing if two adjacent spans are scavenged, we
subtract their sizes from memstats and re-scavenge the new combined
span. This is wasteful however, since the realignment semantics make
this case of having to re-scavenge impossible.

In realign() inside of coalesce(), there was also a bug: on systems
where physPageSize > pageSize, we wouldn't realign because a condition
had the wrong sign. This wasteful re-scavenging has been masking this
bug this whole time. So, this change fixes that first.

Then this change gets rid of the needsScavenge logic and instead checks
explicitly for the possibility of unscavenged pages near the physical
page boundary. If the possibility exists, it throws. The intent of
throwing here is to catch changes to the runtime which cause this
invariant to no longer hold, at which point it would likely be
appropriate to scavenge the additional pages (and only the additional
pages) at that point.

Change-Id: I185e3d7b53e36e90cf9ace5fa297a9e8008d75f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/158377
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
6 years agocmd/dist: add BOOT_GO_LDFLAGS - counterpart of BOOT_GO_GCFLAGS
Mihai Moldovan [Tue, 9 Apr 2019 10:25:49 +0000 (10:25 +0000)]
cmd/dist: add BOOT_GO_LDFLAGS - counterpart of BOOT_GO_GCFLAGS

This allows passing custom LDFLAGS while building the bootstrapping
tool.

Afterwards, GO_LDFLAGS will be used as usual.

Change-Id: I1e224e3ce8bf7b2ce1ef8fec1894720338f04396
GitHub-Last-Rev: 17d40dc2dd2f0815331cb2f8de3445f86687cc45
GitHub-Pull-Request: golang/go#31298
Reviewed-on: https://go-review.googlesource.com/c/go/+/171037
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agoos: fix RemoveAll hangs on large directory
LE Manh Cuong [Sun, 7 Apr 2019 06:56:58 +0000 (13:56 +0700)]
os: fix RemoveAll hangs on large directory

golang.org/cl/121255 added close and re-open the directory when looping, prevent
us from missing some if previous iteration deleted files.

The CL introdued a bug. If we can not delete all entries in one request,
the looping never exits, causing RemoveAll hangs.

To fix that, simply discard the entries if we can not delete all of them
in one iteration, then continue reading entries and delete them.

Also make sure removeall_at return first error it encounters.

Fixes #29921

Change-Id: I8ec3a4c822d8d2d95d9f1ab71547879da395bc4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/171099
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agosyscall: store skip count in file descriptor offset
Keith Randall [Wed, 10 Apr 2019 18:06:58 +0000 (11:06 -0700)]
syscall: store skip count in file descriptor offset

Multiple calls to ReadDirent expect to return subsequent
portions of the directory listing. There's no place to store
our progress other than the file descriptor offset.

Fortunately, the file descriptor offset doesn't need to be
a real offset. We can store any int64 we want there.

Fixes #31368

Change-Id: I49e4e0e7ff707d3e96aa5d43e3b0199531013cde
Reviewed-on: https://go-review.googlesource.com/c/go/+/171477
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agoerrors: return false if nil error is passed to As
Ahsun Ahmed [Thu, 21 Mar 2019 16:45:49 +0000 (22:45 +0600)]
errors: return false if nil error is passed to As

Fixes #30970

Change-Id: I333676b55a2364e329fffeafca8fc57d45a0b84b
Reviewed-on: https://go-review.googlesource.com/c/go/+/168598
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agocmd/asm/internal/arch: improve the comment of function IsMIPSMUL
smileeye [Tue, 9 Apr 2019 14:47:38 +0000 (22:47 +0800)]
cmd/asm/internal/arch: improve the comment of function IsMIPSMUL

The check of MADD&MSUB was added to the function IsMIPSMUL in
a previous commit, and the comments should also be updated.

Change-Id: I2d3da055d55b459b908714c542dff99ab5c6cf99
Reviewed-on: https://go-review.googlesource.com/c/go/+/171102
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoruntime: correct the TLS base offset on Android
Elias Naur [Mon, 8 Apr 2019 21:12:52 +0000 (23:12 +0200)]
runtime: correct the TLS base offset on Android

CL 170955 set tlsg to the Android Q free TLS slot offset in the linker
data (16 on amd64, 8 on 386), offsetting all TLS relative access.
We need the 0'th slot (TLS_SLOT_SELF) at initialization, so
compensate with a corresponding negative offset.

Fixes the android/386 and android/amd64 builders broken by CL 170955.

Change-Id: I9882088c0c8bc6a777d2aabc9404cb76f02b6cea
Reviewed-on: https://go-review.googlesource.com/c/go/+/170956
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
6 years agoall: spell "Deprecated: Use etc" consistently
Nigel Tao [Sun, 7 Apr 2019 03:50:06 +0000 (13:50 +1000)]
all: spell "Deprecated: Use etc" consistently

Change-Id: I209b75dc8dc4da881b68e5c5d98cbf08c1032dfc
Reviewed-on: https://go-review.googlesource.com/c/go/+/171098
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
6 years agocmd/internal/obj/x86: allow non-zero offset in TLS reference
Cherry Zhang [Tue, 9 Apr 2019 16:10:27 +0000 (12:10 -0400)]
cmd/internal/obj/x86: allow non-zero offset in TLS reference

An instruction that references TLS, e.g.

MOVQ 0(TLS), AX

on some platforms (e.g. Android), or in shared mode, may be
translated to (assuming TLS offset already loaded to CX)

MOVQ 0(CX)(TLS*1), AX

which in turns translates to

movq %fs:(%rcx), %rax

We have rejected non-zero offset for TLS reference, like 16(TLS).
Actually, the instruction can take offset, i.e. it is a valid
instruction for, e.g.,

movq %fs:16(%rcx),%rcx

So, allow offset in TLS reference.

Change-Id: Iaf1996bad7fe874e0c298ea441af5acb136a4028
Reviewed-on: https://go-review.googlesource.com/c/go/+/171151
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agostrings: add TestIndexByte
Tobias Klauser [Tue, 9 Apr 2019 06:32:22 +0000 (08:32 +0200)]
strings: add TestIndexByte

Add TestIndexByte to package strings similar to the already existing
TestIndexByte in package bytes.

Change-Id: Ib60695cb326156a4fe48138c66393ebbd11e4a25
Reviewed-on: https://go-review.googlesource.com/c/go/+/171197
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agostrings: use Go style character range comparison in ToUpper/ToLower
Tobias Klauser [Tue, 9 Apr 2019 06:35:36 +0000 (08:35 +0200)]
strings: use Go style character range comparison in ToUpper/ToLower

As noted by Brad in CL 170954 for package bytes.

Change-Id: I2772a356299e54ba5b7884d537e6649039adb9be
Reviewed-on: https://go-review.googlesource.com/c/go/+/171198
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoruntime: preempt a goroutine which calls a lot of short system calls
Andrei Vagin [Fri, 29 Mar 2019 17:43:31 +0000 (10:43 -0700)]
runtime: preempt a goroutine which calls a lot of short system calls

A goroutine should be preempted if it runs for 10ms without blocking.
We found that this doesn't work for goroutines which call short system calls.

For example, the next program can stuck for seconds without this fix:

$ cat main.go
package main

import (
"runtime"
"syscall"
)

func main() {
runtime.GOMAXPROCS(1)
c := make(chan int)
go func() {
c <- 1
for {
t := syscall.Timespec{
Nsec: 300,
}
if true {
syscall.Nanosleep(&t, nil)
}
}
}()
<-c
}

$ time go run main.go

real 0m8.796s
user 0m0.367s
sys 0m0.893s

Updates #10958

Change-Id: Id3be54d3779cc28bfc8b33fe578f13778f1ae2a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/170138
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agobytes: optimize ToLower and ToUpper for ASCII-only case
Tobias Klauser [Mon, 8 Apr 2019 13:53:35 +0000 (15:53 +0200)]
bytes: optimize ToLower and ToUpper for ASCII-only case

Follow what CL 68370 and CL 76470 did for the respective functions in
package strings.

Also adjust godoc strings to match the respective strings functions and
mention the special case for ASCII-only byte slices which don't need
conversion.

name                                       old time/op  new time/op  delta
ToUpper/#00-8                              9.35ns ± 3%  6.08ns ± 2%  -35.04%  (p=0.000 n=9+9)
ToUpper/ONLYUPPER-8                        77.7ns ± 1%  16.9ns ± 2%  -78.22%  (p=0.000 n=10+10)
ToUpper/abc-8                              36.5ns ± 1%  22.1ns ± 1%  -39.43%  (p=0.000 n=10+8)
ToUpper/AbC123-8                           56.9ns ± 2%  28.2ns ± 2%  -50.54%  (p=0.000 n=8+10)
ToUpper/azAZ09_-8                          62.3ns ± 1%  26.9ns ± 1%  -56.82%  (p=0.000 n=9+10)
ToUpper/longStrinGwitHmixofsmaLLandcAps-8   219ns ± 2%    63ns ± 2%  -71.17%  (p=0.000 n=10+10)
ToUpper/longɐstringɐwithɐnonasciiⱯchars-8   367ns ± 2%   374ns ± 3%   +2.05%  (p=0.000 n=9+10)
ToUpper/ɐɐɐɐɐ-8                             200ns ± 1%   206ns ± 1%   +2.49%  (p=0.000 n=10+10)
ToUpper/a\u0080\U0010ffff-8                90.4ns ± 1%  93.8ns ± 0%   +3.82%  (p=0.000 n=10+7)
ToLower/#00-8                              9.59ns ± 1%  6.13ns ± 2%  -36.08%  (p=0.000 n=10+10)
ToLower/abc-8                              36.4ns ± 1%  10.4ns ± 1%  -71.50%  (p=0.000 n=10+10)
ToLower/AbC123-8                           55.8ns ± 1%  27.5ns ± 1%  -50.61%  (p=0.000 n=10+10)
ToLower/azAZ09_-8                          61.7ns ± 1%  30.2ns ± 1%  -50.98%  (p=0.000 n=8+10)
ToLower/longStrinGwitHmixofsmaLLandcAps-8   226ns ± 1%    64ns ± 1%  -71.53%  (p=0.000 n=10+9)
ToLower/LONGⱯSTRINGⱯWITHⱯNONASCIIⱯCHARS-8   354ns ± 0%   361ns ± 0%   +2.18%  (p=0.000 n=10+10)
ToLower/ⱭⱭⱭⱭⱭ-8                             180ns ± 1%   186ns ± 0%   +3.45%  (p=0.000 n=10+9)
ToLower/A\u0080\U0010ffff-8                91.7ns ± 0%  94.5ns ± 0%   +2.99%  (p=0.000 n=10+10)

Change-Id: Ifdb8ae328ff9feacd1c170db8eebbf98c399e204
Reviewed-on: https://go-review.googlesource.com/c/go/+/170954
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
6 years agosyscall: gofmt -w -s
Mikio Hara [Tue, 9 Apr 2019 01:10:21 +0000 (10:10 +0900)]
syscall: gofmt -w -s

Change-Id: Ib46f1a528e16cd0c2617defbf4dcd1f1b582cdc2
Reviewed-on: https://go-review.googlesource.com/c/go/+/171101
Run-TryBot: Mikio Hara <mikioh.public.networking@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agocmd/cover: rename temporary directory prefix for consistency
Mikio Hara [Tue, 9 Apr 2019 01:00:38 +0000 (10:00 +0900)]
cmd/cover: rename temporary directory prefix for consistency

This change renames the temporary directory prefix for testing to
go-testcover from gotestcover. It looks like other packages have the
"go-" prefix for temporary directories, such as go-build, go-tool-dist
and go-nettest.

Change-Id: I91ab570d33c4c1bb48e6e01451a811272f6f8b77
Reviewed-on: https://go-review.googlesource.com/c/go/+/171100
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoall: change the old assembly style AX:CX to CX, AX
Neven Sajko [Sat, 30 Mar 2019 19:45:46 +0000 (19:45 +0000)]
all: change the old assembly style AX:CX to CX, AX

Assembly files with "/vendor/" or "testdata" in their paths were ignored.

Change-Id: I3882ff07eb4426abb9f8ee96f82dff73c81cd61f
GitHub-Last-Rev: 51ae8c324d72a12a059272fcf8568e670bfaf21b
GitHub-Pull-Request: golang/go#31166
Reviewed-on: https://go-review.googlesource.com/c/go/+/170197
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agocmd/compile: add ORESULT, remove OINDREGSP
Josh Bleecher Snyder [Fri, 5 Apr 2019 23:43:08 +0000 (16:43 -0700)]
cmd/compile: add ORESULT, remove OINDREGSP

This change is mostly cosmetic.

OINDREGSP was used only for reading the results of a function call.
In recognition of that fact, rename it to ORESULT.
Along the way, trim down our handling of it to the bare minimum,
and rely on the increased clarity of ORESULT to inline nodarg.

Passes toolstash-check.

Change-Id: I25b177df4ea54a8e94b1698d044c297b7e453c64
Reviewed-on: https://go-review.googlesource.com/c/go/+/170705
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
6 years agocmd/go/internal/work: make toolchain builds reproducible when buildmode=pie
Elias Naur [Sun, 7 Apr 2019 13:02:31 +0000 (15:02 +0200)]
cmd/go/internal/work: make toolchain builds reproducible when buildmode=pie

When buildmode=pie, external linking is forced, and our toolchain build id
will be included in the external build id, resulting in the building of
a toolchain tool will never reach a fixed point id.

More importantly, this change will make make.bash converge on self-hosted
Android builds (Android refuses to run non-PIE executables).

Fixes #31320
Updates #18968

Change-Id: Icb5db9f4b1b688afe37f4dafe261ffda580fa4e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/170942
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agoruntime,runtime/cgo: set up TLS storage for Android Q without cgo
Elias Naur [Mon, 8 Apr 2019 15:57:53 +0000 (17:57 +0200)]
runtime,runtime/cgo: set up TLS storage for Android Q without cgo

Android Q frees a static TLS slot for us to use. Use the offset of
that slot as the default for our TLS offset.

As a result, runtime/cgo is no more a requirement for Android Q and
newer.

Updates #31343
Updates #29674

Change-Id: I759049b2e2865bd3d4fdc05a8cfc6db8b0da1f5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/170955
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
6 years agodoc: correct link in 1.12.3 notes
Andrew Bonventre [Mon, 8 Apr 2019 20:09:15 +0000 (16:09 -0400)]
doc: correct link in 1.12.3 notes

Change-Id: I6dd60ea7b8a8756be97503e163c2386af16e4e12
Reviewed-on: https://go-review.googlesource.com/c/go/+/171144
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agodoc: correct link in 1.11.8 notes
Andrew Bonventre [Mon, 8 Apr 2019 20:07:49 +0000 (16:07 -0400)]
doc: correct link in 1.11.8 notes

Change-Id: I09e0c2720ec0a51dc73c24b4550a749448656025
Reviewed-on: https://go-review.googlesource.com/c/go/+/171143
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agodoc: document Go 1.11.8
Andrew Bonventre [Mon, 8 Apr 2019 19:42:30 +0000 (15:42 -0400)]
doc: document Go 1.11.8

Change-Id: Ia06f7005f466e55a22c76bf2b47d74ee8eb77dd1
Reviewed-on: https://go-review.googlesource.com/c/go/+/171139
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agodoc: document Go 1.12.3
Andrew Bonventre [Mon, 8 Apr 2019 19:43:32 +0000 (15:43 -0400)]
doc: document Go 1.12.3

Change-Id: I84c9a8ddbd3101dd478e4a8a4e191863e68c6af6
Reviewed-on: https://go-review.googlesource.com/c/go/+/171140
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoruntime: follow Windows calling convention for _cgo_sys_thread_create
Jingnan Si [Mon, 8 Apr 2019 19:02:40 +0000 (19:02 +0000)]
runtime: follow Windows calling convention for _cgo_sys_thread_create

Windows requires space for four pointers on the stack.

Change-Id: I9f7ba3e09b6c660f86d15139bb51954fffc8ccb1
GitHub-Last-Rev: 76d21bcc2b07edfde6daa45000093d070e2337bc
GitHub-Pull-Request: golang/go#30944
Reviewed-on: https://go-review.googlesource.com/c/go/+/168351
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agocmd/dist: set buildmode=pie on Android
Elias Naur [Sun, 7 Apr 2019 14:58:13 +0000 (16:58 +0200)]
cmd/dist: set buildmode=pie on Android

Android refuses to run non-PIE binaries, a restriction already
encoded in the cmd/go tool's buildModeInit function. This CL adds
the necessary flags to cmd/dist to make ./make.bash run on an
Android device.

Change-Id: I162084f573befaa41dcb47a2b78448bce5b83d35
Reviewed-on: https://go-review.googlesource.com/c/go/+/170943
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
6 years agomisc/cgo/testcshared: use adb exec-out instead of adb shell
Elias Naur [Mon, 8 Apr 2019 13:45:16 +0000 (15:45 +0200)]
misc/cgo/testcshared: use adb exec-out instead of adb shell

Adb exec-out is like adb shell except non-flaky in non-interactive
settings. Don't ask why.

Change-Id: I7ac3c72912883d80bc787c1d0fc101db6bae9c52
Reviewed-on: https://go-review.googlesource.com/c/go/+/170952
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
6 years agocmd/go: do not write output when -o is not specified, but folder with same name exists
Daniel Theophanes [Sat, 6 Apr 2019 16:50:28 +0000 (09:50 -0700)]
cmd/go: do not write output when -o is not specified, but folder with same name exists

Fixes #31296

Change-Id: Ib8850fe22749ca0bf268614ba045ffe3fc68f5cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/171057
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
6 years agocmd/compile: get rid of unnecessary inline marks
Keith Randall [Tue, 2 Apr 2019 22:00:54 +0000 (15:00 -0700)]
cmd/compile: get rid of unnecessary inline marks

If no other instruction mentions an inline mark, we can get rid of it.
This normally happens when the inlined function is empty, or when all
of its code is folded into other instructions.

Also use consistent statement-ness for inline mark positions, so that
more of them can be removed in favor of existing instructions.

Update #29571
Fixes #31172

Change-Id: I71f84d355101f37a27960d9e8528f42f92767496
Reviewed-on: https://go-review.googlesource.com/c/go/+/170445
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
6 years agoruntime: use iterator instead of raw node for treap find
Michael Anthony Knyszek [Mon, 11 Feb 2019 17:20:59 +0000 (17:20 +0000)]
runtime: use iterator instead of raw node for treap find

Right now the mTreap structure exposes the treapNode structure through
only one interface: find. There's no reason (performance or otherwise)
for exposing this, and we get a cleaner abstraction through the
iterators this way. This change also makes it easier to make changes to
the mTreap implementation without violating its interface.

Change-Id: I5ef86b8ac81a47d05d8404df65af9ec5f419dc40
Reviewed-on: https://go-review.googlesource.com/c/go/+/164098
Reviewed-by: Austin Clements <austin@google.com>
6 years agoruntime: merge codepaths in scavengeLargest
Michael Anthony Knyszek [Thu, 31 Jan 2019 16:57:29 +0000 (16:57 +0000)]
runtime: merge codepaths in scavengeLargest

This change just makes the code in scavengeLargest easier to reason
about by reducing the number of exit points to the method. It should
still be correct either way because the condition checked at the end
(released > nbytes) will always be false if we return, but this just
makes the code a little easier to maintain.

Change-Id: If60da7696aca3fab3b5ddfc795d600d87c988238
Reviewed-on: https://go-review.googlesource.com/c/go/+/160617
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
6 years agostrings: unindent Fields
Tobias Klauser [Mon, 8 Apr 2019 07:54:33 +0000 (09:54 +0200)]
strings: unindent Fields

CL 56470 unindented bytes.Fields, but not strings.Fields. Do so now to
make it easier to diff the two functions for potential differences.

Change-Id: Ifef81f50cee64e8277e91efa5ec5521d8d21d3bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/170951
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agoarchive/zip: use Modified in FileHeader.FileInfo
Jannis Andrija Schnitzer [Mon, 8 Apr 2019 07:31:59 +0000 (07:31 +0000)]
archive/zip: use Modified in FileHeader.FileInfo

The Modified field allows representation of extended timestamps, which provide more accuracy than the legacy MS-DOS timestamps.
The FileInfo method provides an implementation of the os.FileInfo interface for files inside archives.

With this change, we make FileInfo use the Modified field, if present, to return more detailed timestamps from its ModTime method.

Fixes #28350

Change-Id: Ia31b5b871a3e61df38a3a1325787ae23ea0b8088
GitHub-Last-Rev: 13e94be3f8ba58717911354146670fc2bc594692
GitHub-Pull-Request: golang/go#28352
Reviewed-on: https://go-review.googlesource.com/c/go/+/144382
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
6 years agomisc/cgo/test: skip Setgid test on Android
Elias Naur [Sun, 7 Apr 2019 22:16:40 +0000 (00:16 +0200)]
misc/cgo/test: skip Setgid test on Android

The setgid syscall is blocked on Android in app context.

Change-Id: I1ff25840bbc25d472ad4e29eb1b51f183a6c4392
Reviewed-on: https://go-review.googlesource.com/c/go/+/170949
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agonet: pass TMPDIR to test client process
Elias Naur [Sun, 7 Apr 2019 18:29:24 +0000 (20:29 +0200)]
net: pass TMPDIR to test client process

Fixes the TestSplice test on Android where the default
TMPDIR (/data/local/tmp) might not be available.

Change-Id: I4f104d11254ba855b1bd2dfa0547d69b7bce4878
Reviewed-on: https://go-review.googlesource.com/c/go/+/170947
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoos/exec: skip unsupported test on Android
Elias Naur [Sun, 7 Apr 2019 17:53:30 +0000 (19:53 +0200)]
os/exec: skip unsupported test on Android

The underlying system call tested by TestCredentialNoSetGroups
is blocked on Android.

Discovered while running all.bash from an Android device; the syscall
is only blocked in an app context.

Change-Id: I16fd2e64636a0958b0ec86820723c0577b8f8f24
Reviewed-on: https://go-review.googlesource.com/c/go/+/170945
Run-TryBot: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agoos: skip Open("/") on Android
Elias Naur [Sun, 7 Apr 2019 18:00:39 +0000 (20:00 +0200)]
os: skip Open("/") on Android

It's not supported in an app context:

$ go test -short os
--- FAIL: TestChdirAndGetwd (0.00s)
    os_test.go:1213: Open /: open /: permission denied

Change-Id: I56b951f925a50fd67715ee2f1de64951ee867e91
Reviewed-on: https://go-review.googlesource.com/c/go/+/170946
Run-TryBot: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agobootstrap.bash: remove exec wrappers
Elias Naur [Sun, 7 Apr 2019 13:11:39 +0000 (15:11 +0200)]
bootstrap.bash: remove exec wrappers

Without this change, building an Android toolchain fails:

$ CGO_ENABLED=1 GOARCH=arm64 GOOS=android ./bootstrap.bash
...
rmdir: failed to remove 'bin/go_android_arm64_exec': Not a directory

Change-Id: Ibc3b1e2fd24b73a63bd3020ce1e813f2b4496125
Reviewed-on: https://go-review.googlesource.com/c/go/+/170941
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoimage: deprecate ZP and ZR
Nigel Tao [Sat, 6 Apr 2019 23:33:36 +0000 (09:33 +1000)]
image: deprecate ZP and ZR

They were added a very long time ago, as a convenience before Go had
struct literals. Today, it is better to use the zero-valued literal. For
example, the compiler cannot prove that ZP or ZR have not been modified.

Change-Id: I7469f1c751e91bf76fe1eab07b5772eccb5d6405
Reviewed-on: https://go-review.googlesource.com/c/go/+/171097
Reviewed-by: Nigel Tao <nigeltao@golang.org>
6 years agocmd/compile: remove AUNDEF opcode
Keith Randall [Sat, 6 Apr 2019 19:42:51 +0000 (12:42 -0700)]
cmd/compile: remove AUNDEF opcode

This opcode was only used to mark unreachable code for plive to use.
plive now uses the SSA representation, so it knows locations are
unreachable because they are ends of Exit blocks. It doesn't need
these opcodes any more.

These opcodes actually used space in the binary, 2 bytes per undef
on x86 and more for other archs.

Makes the amd64 go binary 0.2% smaller.

Change-Id: I64c84c35db7c7949617a3a5830f09c8e5fcd2620
Reviewed-on: https://go-review.googlesource.com/c/go/+/171058
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoencoding/json: use SetBytes in UnmarshalReuse benchmark
Daniel Martí [Fri, 5 Apr 2019 20:28:31 +0000 (22:28 +0200)]
encoding/json: use SetBytes in UnmarshalReuse benchmark

This was the only benchmark missing the SetBytes call, as spotted
earlier by Bryan.

It's not required to make the benchmark useful, but it can still be a
good way to see how its speed is affected by the reduced allocations:

name                  time/op
CodeUnmarshal-8        12.1ms ± 1%
CodeUnmarshalReuse-8   11.4ms ± 1%

name                  speed
CodeUnmarshal-8       161MB/s ± 1%
CodeUnmarshalReuse-8  171MB/s ± 1%

name                  alloc/op
CodeUnmarshal-8        3.28MB ± 0%
CodeUnmarshalReuse-8   1.94MB ± 0%

name                  allocs/op
CodeUnmarshal-8         92.7k ± 0%
CodeUnmarshalReuse-8    77.6k ± 0%

While at it, remove some unnecessary empty lines.

Change-Id: Ib2bd92d5b3237b8f3092e8c6f863dab548fee2f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/170938
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agosyscall: use openat instead of dup to make a really new file descriptor
Keith Randall [Fri, 5 Apr 2019 20:01:58 +0000 (13:01 -0700)]
syscall: use openat instead of dup to make a really new file descriptor

Update #31269

Change-Id: I0e7184420055b8dfd23688dab9f9d8cba1fa2485
Reviewed-on: https://go-review.googlesource.com/c/go/+/170892
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoruntime: separate stack freeing from stack shrinking
Austin Clements [Wed, 3 Apr 2019 18:47:55 +0000 (14:47 -0400)]
runtime: separate stack freeing from stack shrinking

Currently, shrinkstack will free the stack if the goroutine is dead.
There are only two places that call shrinkstack: scanstack, which will
never call it if the goroutine is dead; and markrootFreeGStacks, which
only calls it on dead goroutines.

Clean this up by separating stack freeing out of shrinkstack.

Change-Id: I7d7891e620550c32a2220833923a025704986681
Reviewed-on: https://go-review.googlesource.com/c/go/+/170890
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
6 years agoruntime: use acquirem/releasem more widely
Austin Clements [Tue, 26 Mar 2019 01:16:33 +0000 (21:16 -0400)]
runtime: use acquirem/releasem more widely

We've copy-pasted the pattern of releasem in many places. This CL
replaces almost everywhere that manipulates g.m.locks and g.preempt
with calls to acquirem/releasem. There are a few where we do something
more complicated, like where exitsyscall has to restore the stack
bound differently depending on the preempt flag, which this CL leaves
alone.

Change-Id: Ia7a46c261daea6e7802b80e7eb9227499f460433
Reviewed-on: https://go-review.googlesource.com/c/go/+/170064
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agoruntime: fix sanity check in notetsleep
Austin Clements [Tue, 26 Mar 2019 02:24:03 +0000 (22:24 -0400)]
runtime: fix sanity check in notetsleep

CL 3660 replaced m.gcing with m.preemptoff, but unintentionally
reversed the sense of part of a sanity check in notetsleep.
Originally, notetsleep required that it be called from g0 or with
preemption disabled (specifically from within the garbage collector).
CL 3660 made it require that it be called from g0 or that preemption
be *enabled*.

I'm not sure why it had the original exception for being called from a
user g within the garbage collector, but the current garbage collector
certainly doesn't need that, and the new condition is completely wrong.

Make the sanity check just require that it's called on g0.

Change-Id: I6980d44f5a4461935e10b1b33a981e32b1b7b0c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/170063
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agosync: smooth out Pool behavior over GC with a victim cache
Austin Clements [Sat, 2 Mar 2019 20:16:29 +0000 (15:16 -0500)]
sync: smooth out Pool behavior over GC with a victim cache

Currently, every Pool is cleared completely at the start of each GC.
This is a problem for heavy users of Pool because it causes an
allocation spike immediately after Pools are clear, which impacts both
throughput and latency.

This CL fixes this by introducing a victim cache mechanism. Instead of
clearing Pools, the victim cache is dropped and the primary cache is
moved to the victim cache. As a result, in steady-state, there are
(roughly) no new allocations, but if Pool usage drops, objects will
still be collected within two GCs (as opposed to one).

This victim cache approach also improves Pool's impact on GC dynamics.
The current approach causes all objects in Pools to be short lived.
However, if an application is in steady state and is just going to
repopulate its Pools, then these objects impact the live heap size *as
if* they were long lived. Since Pooled objects count as short lived
when computing the GC trigger and goal, but act as long lived objects
in the live heap, this causes GC to trigger too frequently. If Pooled
objects are a non-trivial portion of an application's heap, this
increases the CPU overhead of GC. The victim cache lets Pooled objects
affect the GC trigger and goal as long-lived objects.

This has no impact on Get/Put performance, but substantially reduces
the impact to the Pool user when a GC happens. PoolExpensiveNew
demonstrates this in the substantially reduction in the rate at which
the "New" function is called.

name                 old time/op     new time/op     delta
Pool-12                 2.21ns ±36%     2.00ns ± 0%     ~     (p=0.070 n=19+16)
PoolOverflow-12          587ns ± 1%      583ns ± 1%   -0.77%  (p=0.000 n=18+18)
PoolSTW-12              5.57µs ± 3%     4.52µs ± 4%  -18.82%  (p=0.000 n=20+19)
PoolExpensiveNew-12     3.69ms ± 7%     1.25ms ± 5%  -66.25%  (p=0.000 n=20+19)

name                 old p50-ns/STW  new p50-ns/STW  delta
PoolSTW-12               5.48k ± 2%      4.53k ± 2%  -17.32%  (p=0.000 n=20+20)

name                 old p95-ns/STW  new p95-ns/STW  delta
PoolSTW-12               6.69k ± 4%      5.13k ± 3%  -23.31%  (p=0.000 n=19+18)

name                 old GCs/op      new GCs/op      delta
PoolExpensiveNew-12       0.39 ± 1%       0.32 ± 2%  -17.95%  (p=0.000 n=18+20)

name                 old New/op      new New/op      delta
PoolExpensiveNew-12       40.0 ± 6%       12.4 ± 6%  -68.91%  (p=0.000 n=20+19)

(https://perf.golang.org/search?q=upload:20190311.2)

Fixes #22950.

Change-Id: If2e183d948c650417283076aacc20739682cdd70
Reviewed-on: https://go-review.googlesource.com/c/go/+/166961
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
6 years agosync: use lock-free structure for Pool stealing
Austin Clements [Fri, 1 Mar 2019 20:33:33 +0000 (15:33 -0500)]
sync: use lock-free structure for Pool stealing

Currently, Pool stores each per-P shard's overflow in a slice
protected by a Mutex. In order to store to the overflow or steal from
another shard, a P must lock that shard's Mutex. This allows for
simple synchronization between Put and Get, but has unfortunate
consequences for clearing pools.

Pools are cleared during STW sweep termination, and hence rely on
pinning a goroutine to its P to synchronize between Get/Put and
clearing. This makes the Get/Put fast path extremely fast because it
can rely on quiescence-style coordination, which doesn't even require
atomic writes, much less locking.

The catch is that a goroutine cannot acquire a Mutex while pinned to
its P (as this could deadlock). Hence, it must drop the pin on the
slow path. But this means the slow path is not synchronized with
clearing. As a result,

1) It's difficult to reason about races between clearing and the slow
path. Furthermore, this reasoning often depends on unspecified nuances
of where preemption points can occur.

2) Clearing must zero out the pointer to every object in every Pool to
prevent a concurrent slow path from causing all objects to be
retained. Since this happens during STW, this has an O(# objects in
Pools) effect on STW time.

3) We can't implement a victim cache without making clearing even
slower.

This CL solves these problems by replacing the locked overflow slice
with a lock-free structure. This allows Gets and Puts to be pinned the
whole time they're manipulating the shards slice (Pool.local), which
eliminates the races between Get/Put and clearing. This, in turn,
eliminates the need to zero all object pointers, reducing clearing to
O(# of Pools) during STW.

In addition to significantly reducing STW impact, this also happens to
speed up the Get/Put fast-path and the slow path. It somewhat
increases the cost of PoolExpensiveNew, but we'll fix that in the next
CL.

name                 old time/op     new time/op     delta
Pool-12                 3.00ns ± 0%     2.21ns ±36%  -26.32%  (p=0.000 n=18+19)
PoolOverflow-12          600ns ± 1%      587ns ± 1%   -2.21%  (p=0.000 n=16+18)
PoolSTW-12              71.0µs ± 2%      5.6µs ± 3%  -92.15%  (p=0.000 n=20+20)
PoolExpensiveNew-12     3.14ms ± 5%     3.69ms ± 7%  +17.67%  (p=0.000 n=19+20)

name                 old p50-ns/STW  new p50-ns/STW  delta
PoolSTW-12               70.7k ± 1%       5.5k ± 2%  -92.25%  (p=0.000 n=20+20)

name                 old p95-ns/STW  new p95-ns/STW  delta
PoolSTW-12               73.1k ± 2%       6.7k ± 4%  -90.86%  (p=0.000 n=18+19)

name                 old GCs/op      new GCs/op      delta
PoolExpensiveNew-12       0.38 ± 1%       0.39 ± 1%   +2.07%  (p=0.000 n=20+18)

name                 old New/op      new New/op      delta
PoolExpensiveNew-12       33.9 ± 6%       40.0 ± 6%  +17.97%  (p=0.000 n=19+20)

(https://perf.golang.org/search?q=upload:20190311.1)

Fixes #22331.
For #22950.

Change-Id: Ic5cd826e25e218f3f8256dbc4d22835c1fecb391
Reviewed-on: https://go-review.googlesource.com/c/go/+/166960
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
6 years agosync: add Pool benchmarks to stress STW and reuse
Austin Clements [Fri, 8 Mar 2019 20:01:34 +0000 (15:01 -0500)]
sync: add Pool benchmarks to stress STW and reuse

This adds two benchmarks that will highlight two problems in Pool that
we're about to address.

The first benchmark measures the impact of large Pools on GC STW time.
Currently, STW time is O(# of items in Pools), and this benchmark
demonstrates 70µs STW times.

The second benchmark measures the impact of fully clearing all Pools
on each GC. Typically this is a problem in heavily-loaded systems
because it causes a spike in allocation. This benchmark stresses this
by simulating an expensive "New" function, so the cost of creating new
objects is reflected in the ns/op of the benchmark.

For #22950, #22331.

Change-Id: I0c8853190d23144026fa11837b6bf42adc461722
Reviewed-on: https://go-review.googlesource.com/c/go/+/166959
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
6 years agosync: internal dynamically sized lock-free queue for sync.Pool
Austin Clements [Fri, 1 Mar 2019 19:54:00 +0000 (14:54 -0500)]
sync: internal dynamically sized lock-free queue for sync.Pool

This adds a dynamically sized, lock-free, single-producer,
multi-consumer queue that will be used in the new Pool stealing
implementation. It's built on top of the fixed-size queue added in the
previous CL.

For #22950, #22331.

Change-Id: Ifc0ca3895bec7e7f9289ba9fb7dd0332bf96ba5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/166958
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
6 years agosync: internal fixed size lock-free queue for sync.Pool
Austin Clements [Fri, 1 Mar 2019 18:16:37 +0000 (13:16 -0500)]
sync: internal fixed size lock-free queue for sync.Pool

This is the first step toward fixing multiple issues with sync.Pool.
This adds a fixed size, lock-free, single-producer, multi-consumer
queue that will be used in the new Pool stealing implementation.

For #22950, #22331.

Change-Id: I50e85e3cb83a2ee71f611ada88e7f55996504bb5
Reviewed-on: https://go-review.googlesource.com/c/go/+/166957
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
6 years agodoc: document Go 1.11.7
Andrew Bonventre [Fri, 5 Apr 2019 18:31:54 +0000 (14:31 -0400)]
doc: document Go 1.11.7

Change-Id: Iec5e69b3ea163f42234d3b73696427a7aa8732e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/170884
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agodoc: document Go 1.12.2
Andrew Bonventre [Fri, 5 Apr 2019 18:29:16 +0000 (14:29 -0400)]
doc: document Go 1.12.2

Change-Id: I990c451ff24844b39dee2477cec4caa9db2e8ebb
Reviewed-on: https://go-review.googlesource.com/c/go/+/170883
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agodatabase/sql: add NullTime
Daniel Theophanes [Fri, 5 Apr 2019 17:23:15 +0000 (10:23 -0700)]
database/sql: add NullTime

This matches NullBool, NullFloat64, and NullInt64.

Fixes #30305

Change-Id: I79bfcf04a3d43b965d2a3159b0ac22f3e8084a53
Reviewed-on: https://go-review.googlesource.com/c/go/+/170699
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
6 years agosyscall: dup the argument to fdopendir
Keith Randall [Fri, 5 Apr 2019 16:03:46 +0000 (09:03 -0700)]
syscall: dup the argument to fdopendir

fdopendir takes ownership of its file descriptor argument.
Getdirentries shouldn't do that, so dup the file descriptor
before passing to fdopendir.

Fixes #31269

Change-Id: Ie36be8fd6c59eb339dcc9f40228d4191fc1e5850
Reviewed-on: https://go-review.googlesource.com/c/go/+/170698
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

6 years agosyscall: allow empty string argument to SetsockoptString
Tobias Klauser [Fri, 5 Apr 2019 15:10:40 +0000 (17:10 +0200)]
syscall: allow empty string argument to SetsockoptString

Don't panic with "index out of range" on empty string argument.

Fixes golang/go#31277

Change-Id: I005f9523caec76337cb2ec87272a6be4736bce18
Reviewed-on: https://go-review.googlesource.com/c/go/+/170937
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>