cmd/go: use t.Fatalf for formatted errors

Change-Id: I657588ccc4a87e075d15acb9f4cd1e417bbd7960
Reviewed-on: https://go-review.googlesource.com/16421
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime/cgo: export _cgo_reginit on ppc64x

This is needed to make external linking work.

Change-Id: I4cf7edb4ea318849cab92a697952f8745eed40c4
Reviewed-on: https://go-review.googlesource.com/14237
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/go: implicitly include math in a shared library on arm

In the same manner in which runtime/cgo is included on other architectures.

Change-Id: I90a5ad8585248b2566d763d33994a600508d89cb
Reviewed-on: https://go-review.googlesource.com/14221
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/compile, cmd/go: enable -buildmode=shared and related flags on linux/arm

Change-Id: I20840632771a250fb279df64d394135994482af8
Reviewed-on: https://go-review.googlesource.com/14186
Reviewed-by: David Crawshaw <crawshaw@golang.org>

runtime: add missing word in comment

Change-Id: Iffe27445e35ec071cf0920a05c81b8a97a3ed712
Reviewed-on: https://go-review.googlesource.com/16431
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime: c-shared entrypoint for linux/arm64

Change-Id: I7dab124842f5209097a8d5a802fcbdde650654fa
Reviewed-on: https://go-review.googlesource.com/16395
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/go: allow -buildmode=pie for android/amd64.

For golang/go#10743 golang/go#10807

Change-Id: I35dedb52e2b47fe7ffc655d01f22ac05fe830bde
Reviewed-on: https://go-review.googlesource.com/16396
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/go: enable android/amd64 build.

For golang/go#10743

Change-Id: Iec047821147a0e28edebf875fefe25993785702b
Reviewed-on: https://go-review.googlesource.com/15994
Reviewed-by: David Crawshaw <crawshaw@golang.org>

runtime, cmd: TLS setup for android/amd64.

Android linker does not handle TLS for us. We set up the TLS slot
for g, as darwin/386,amd64 handle instead. This is disgusting and
fragile. We will eventually fix this ugly hack by taking advantage
of the recent TLS IE model implementation. (Instead of referencing
an GOT entry, make the code sequence look into the TLS variable that
holds the offset.)

The TLS slot for g in android/amd64 assumes a fixed offset from %fs.
See runtime/cgo/gcc_android_amd64.c for details.

For golang/go#10743

Change-Id: I1a3fc207946c665515f79026a56ea19134ede2dd
Reviewed-on: https://go-review.googlesource.com/15991
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/internal/obj, cmd/link: support inital-exec TLS on arm64

Change-Id: Iaf9159a68fa395245bc20ccb4a2a377f89371a7e
Reviewed-on: https://go-review.googlesource.com/13996
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/internal/obj, cmd/link, runtime: native-ish support for tls on arm64

Fixes #10560

Change-Id: Iedffd9c236c4fbb386c3afc52c5a1457f96ef122
Reviewed-on: https://go-review.googlesource.com/13991
Reviewed-by: David Crawshaw <crawshaw@golang.org>

net/http/httputil: add hook for managing io.Copy buffers per request

Adds ReverseProxy.BufferPool for users with sensitive allocation
requirements. Permits avoiding 32 KB of io.Copy garbage per request.

Fixes #10277

Change-Id: I5dfd58fa70a363ead4be56405e507df90d871719
Reviewed-on: https://go-review.googlesource.com/9399
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime: don't use FP when calling nextSample in the Plan 9 sighandler

In the Go signal handler on Plan 9, when a signal with
the _SigThrow flag is received, we call startpanic before
printing the stack trace.

The startpanic function calls systemstack which calls
startpanic_m. In the startpanic_m function, we call
allocmcache to allocate _g_.m.mcache. The problem is
that allocmcache calls nextSample, which does a floating
point operation to return a sampling point for heap profiling.

However, Plan 9 doesn't support floating point in the
signal handler.

This change adds a new function nextSampleNoFP, only
called when in the Plan 9 signal handler, which is
similar to nextSample, but avoids floating point.

Change-Id: Iaa30437aa0f7c8c84d40afbab7567ad3bd5ea2de
Reviewed-on: https://go-review.googlesource.com/16307
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/link: remove unused field LSym.Hash

Change-Id: Idc5c4a69919a8ed9d76d4a9cfd9827fb5c59dd11
Reviewed-on: https://go-review.googlesource.com/16389
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

text/template: remove redundant indirect

Change-Id: I8cc9783fd044bed48347824dcf973c61c78275a5
Reviewed-on: https://go-review.googlesource.com/15833
Reviewed-by: Andrew Gerrand <adg@golang.org>

runtime: invoke vsyscall helper via TCB when dynamic linking on linux/386

The dynamic linker on linux/386 stores the address of the vsyscall helper at a
fixed offset from the %gs register on linux/386 for easy access from PIC code.

Change-Id: I635305cfecceef2289985d62e676e16810ed6b94
Reviewed-on: https://go-review.googlesource.com/16346
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/compile, cmd/go, cmd/link: enable -buildmode=shared and related flags on linux/386

Change-Id: If3417135ca474468a480b08cf46334fda28f79b4
Reviewed-on: https://go-review.googlesource.com/16345
Reviewed-by: Ian Lance Taylor <iant@golang.org>

net/http/httptest: close conns in StateNew on Server close

This part got dropped when we were debating between two solutions
in https://golang.org/cl/15151

Fixes #13032

Change-Id: I820b94f6c0c102ccf9342abf957328ea01f49a26
Reviewed-on: https://go-review.googlesource.com/16313
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

reflect: Comment fix

Change-Id: I86cdd5c1d7b6f76d3474d180e75ea0c732241080
Reviewed-on: https://go-review.googlesource.com/16309
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/internal/obj/x86: remove REGTMP

Nothing uses this.

Change-Id: Ibc13066940bd2ea5c74d955a67f9dc531bef2758
Reviewed-on: https://go-review.googlesource.com/16344
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

runtime: eliminate some unnecessary uintptr conversions

arena_{start,used,end} are already uintptr, so no need to convert them
to uintptr, much less to convert them to unsafe.Pointer and then to
uintptr.  No binary change to pkg/linux_amd64/runtime.a.

Change-Id: Ia4232ed2a724c44fde7eba403c5fe8e6dccaa879
Reviewed-on: https://go-review.googlesource.com/16339
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>

cmd/compile/internal/gc: introduce type for untyped constant kinds

Change-Id: Ia34b6dd099d07d5e1d4bffe775a20fa92705fdb0
Reviewed-on: https://go-review.googlesource.com/16335
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/compile/internal/gc: introduce type for decl contexts/storage classes

Change-Id: I956e27fa07f16060b8f41b986d991c36557f7c12
Reviewed-on: https://go-review.googlesource.com/16332
Reviewed-by: Keith Randall <khr@golang.org>

syscall: define common notes on Plan 9

There is no signal list on Plan 9, since notes
are strings. However, some programs expect
signals to be defined in the syscall package.
Hence, we define a list of the most common notes.

Updates #11975.

Change-Id: I852e14fd98777c9595a406e04125be1cbebed0fb
Reviewed-on: https://go-review.googlesource.com/16301
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime: handle abort note on Plan 9

Implement an abort note on Plan 9, as an
equivalent of the SIGABRT signal on other
operating systems.

Updates #11975.

Change-Id: I010c9b10f2fbd2471aacd1d073368d975a2f0592
Reviewed-on: https://go-review.googlesource.com/16300
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime: fix tiny allocator

When a new tiny block is allocated because we're allocating an object
that won't fit into the current block, mallocgc saves the new block if
it has more space leftover than the old block.  However, the logic for
this was subtly broken in golang.org/cl/2814, resulting in never
saving (or consequently reusing) a tiny block.

Change-Id: Ib5f6769451fb82877ddeefe75dfe79ed4a04fd40
Reviewed-on: https://go-review.googlesource.com/16330
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/compile/internal/gc: escape binary export data so it contains no '$'

Necessary to ensure that subsequent tools can continue to find
then end of the export data section simply by searching for "$$".

Adjusted gcimporter used by go/types accordingly.

Also, fixed a bug in gcimporter related to reading export data
in debug format.

Change-Id: Iaea4ed05edd8a5bab28ebe5b19a4740f5e537d35
Reviewed-on: https://go-review.googlesource.com/16283
Reviewed-by: Chris Manghane <cmang@golang.org>

runtime: partition data and BSS root marking

Currently data and BSS root marking are each a single markroot job.
This makes them difficult to load balance, which can draw out mark
termination time if they are large.

Fix this by splitting both in to 256K chunks. While we're putting in
the infrastructure for dynamic roots, we also replace the fixed
sharding of the span roots with sharding in to fixed sizes. In
addition to helping balance root marking, this also paves the way to
parallelizing concurrent scan and to letting assists help with root
marking.

Updates #10345. This fixes the data and BSS aspects of that bug; it
does not partition scanning of large heap objects.

This has negligible effect on either the go1 benchmarks or the garbage
benchmark:

name              old time/op  new time/op  delta
XBenchGarbage-12  4.90ms ± 1%  4.91ms ± 2%   ~     (p=0.058 n=17+16)

name                      old time/op    new time/op    delta
BinaryTree17-12              3.11s ± 4%     3.12s ± 4%    ~     (p=0.512 n=20+20)
Fannkuch11-12                2.53s ± 2%     2.47s ± 2%  -2.28%  (p=0.000 n=20+18)
FmtFprintfEmpty-12          49.1ns ± 1%    50.0ns ± 4%  +1.68%  (p=0.008 n=18+20)
FmtFprintfString-12          170ns ± 0%     172ns ± 1%  +1.05%  (p=0.000 n=14+19)
FmtFprintfInt-12             174ns ± 1%     162ns ± 1%  -6.81%  (p=0.000 n=18+17)
FmtFprintfIntInt-12          284ns ± 1%     277ns ± 1%  -2.42%  (p=0.000 n=20+19)
FmtFprintfPrefixedInt-12     252ns ± 1%     244ns ± 1%  -2.84%  (p=0.000 n=18+20)
FmtFprintfFloat-12           317ns ± 0%     311ns ± 0%  -1.95%  (p=0.000 n=19+18)
FmtManyArgs-12              1.08µs ± 1%    1.11µs ± 1%  +3.43%  (p=0.000 n=18+19)
GobDecode-12                8.56ms ± 1%    8.61ms ± 1%  +0.50%  (p=0.020 n=20+20)
GobEncode-12                6.58ms ± 1%    6.57ms ± 1%    ~     (p=0.792 n=20+19)
Gzip-12                      317ms ± 3%     317ms ± 2%    ~     (p=0.840 n=19+19)
Gunzip-12                   41.6ms ± 0%    41.6ms ± 0%  +0.07%  (p=0.027 n=18+15)
HTTPClientServer-12         62.2µs ± 1%    62.3µs ± 1%    ~     (p=0.283 n=19+20)
JSONEncode-12               16.5ms ± 2%    16.5ms ± 1%    ~     (p=0.857 n=20+19)
JSONDecode-12               58.5ms ± 1%    61.3ms ± 1%  +4.67%  (p=0.000 n=18+17)
Mandelbrot200-12            3.84ms ± 0%    3.84ms ± 0%    ~     (p=0.259 n=17+17)
GoParse-12                  3.70ms ± 2%    3.74ms ± 2%  +0.96%  (p=0.009 n=19+20)
RegexpMatchEasy0_32-12       100ns ± 1%     100ns ± 0%  +0.31%  (p=0.040 n=19+15)
RegexpMatchEasy0_1K-12       340ns ± 1%     340ns ± 1%    ~     (p=0.411 n=17+19)
RegexpMatchEasy1_32-12      82.7ns ± 2%    82.3ns ± 1%    ~     (p=0.456 n=20+19)
RegexpMatchEasy1_1K-12       498ns ± 2%     495ns ± 0%    ~     (p=0.108 n=19+17)
RegexpMatchMedium_32-12      130ns ± 1%     130ns ± 2%    ~     (p=0.405 n=18+19)
RegexpMatchMedium_1K-12     39.4µs ± 2%    39.1µs ± 1%  -0.64%  (p=0.002 n=20+19)
RegexpMatchHard_32-12       2.03µs ± 2%    2.02µs ± 0%    ~     (p=0.561 n=20+17)
RegexpMatchHard_1K-12       61.1µs ± 2%    60.8µs ± 1%    ~     (p=0.615 n=19+18)
Revcomp-12                   532ms ± 2%     531ms ± 1%    ~     (p=0.470 n=19+19)
Template-12                 68.5ms ± 1%    69.1ms ± 1%  +0.87%  (p=0.000 n=17+17)
TimeParse-12                 344ns ± 2%     344ns ± 1%  +0.25%  (p=0.032 n=19+18)
TimeFormat-12                347ns ± 1%     362ns ± 1%  +4.27%  (p=0.000 n=17+19)
[Geo mean]                  62.3µs         62.3µs       -0.04%

name                      old speed      new speed      delta
GobDecode-12              89.6MB/s ± 1%  89.2MB/s ± 1%  -0.50%  (p=0.019 n=20+20)
GobEncode-12               117MB/s ± 1%   117MB/s ± 1%    ~     (p=0.797 n=20+19)
Gzip-12                   61.3MB/s ± 3%  61.2MB/s ± 2%    ~     (p=0.834 n=19+19)
Gunzip-12                  467MB/s ± 0%   466MB/s ± 0%  -0.07%  (p=0.027 n=18+15)
JSONEncode-12              117MB/s ± 2%   117MB/s ± 1%    ~     (p=0.851 n=20+19)
JSONDecode-12             33.2MB/s ± 1%  31.7MB/s ± 1%  -4.47%  (p=0.000 n=18+17)
GoParse-12                15.6MB/s ± 2%  15.5MB/s ± 2%  -0.95%  (p=0.008 n=19+20)
RegexpMatchEasy0_32-12     321MB/s ± 2%   320MB/s ± 1%  -0.57%  (p=0.002 n=17+17)
RegexpMatchEasy0_1K-12    3.01GB/s ± 1%  3.01GB/s ± 1%    ~     (p=0.132 n=17+18)
RegexpMatchEasy1_32-12     387MB/s ± 2%   389MB/s ± 1%    ~     (p=0.423 n=20+19)
RegexpMatchEasy1_1K-12    2.05GB/s ± 2%  2.06GB/s ± 0%    ~     (p=0.129 n=19+17)
RegexpMatchMedium_32-12   7.64MB/s ± 1%  7.66MB/s ± 1%    ~     (p=0.258 n=18+19)
RegexpMatchMedium_1K-12   26.0MB/s ± 2%  26.2MB/s ± 1%  +0.64%  (p=0.002 n=20+19)
RegexpMatchHard_32-12     15.7MB/s ± 2%  15.8MB/s ± 1%    ~     (p=0.510 n=20+17)
RegexpMatchHard_1K-12     16.8MB/s ± 2%  16.8MB/s ± 1%    ~     (p=0.603 n=19+18)
Revcomp-12                 477MB/s ± 2%   479MB/s ± 1%    ~     (p=0.470 n=19+19)
Template-12               28.3MB/s ± 1%  28.1MB/s ± 1%  -0.85%  (p=0.000 n=17+17)
[Geo mean]                 100MB/s        100MB/s       -0.26%

Change-Id: Ib0bfe0145675ce88c5a8791752f7486ac98805b4
Reviewed-on: https://go-review.googlesource.com/16043
Reviewed-by: Rick Hudson <rlh@golang.org>

androidtest.bash: set GOARM=7

It's the only ARM version we have ever supported on android.
(Not setting it caused some builder timeouts.)

Change-Id: I26061434252ff2a236bb31d95787a1c582d24b3f
Reviewed-on: https://go-review.googlesource.com/16295
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>

runtime: use a 64kb system stack on arm

I went looking for an arm system whose stacks are by default smaller
than 64KB. In fact the smallest common linux target I could find was
Android, which like iOS uses 1MB stacks.

Fixes #11873

Change-Id: Ieeb66ad095b3da18d47ba21360ea75152a4107c6
Reviewed-on: https://go-review.googlesource.com/14602
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-by: Minux Ma <minux@golang.org>

cmd/compile/internal/gc: make embedded unexported structs RO

gc will need to be rebuild.

Package that assume f.PkgPath != nil means a field is unexported and
must be ignored must be revised to check for
f.PkgPath != nil && !f.Anonymous,
so that they do try to walk into the embedded fields to look for
exported fields contained within.

Closes #12367, fixes #7363, fixes #11007, and fixes #7247.

Change-Id: I16402ee21ccfede80f277f84b3995cf26e97433d
Reviewed-on: https://go-review.googlesource.com/14085
Reviewed-by: Russ Cox <rsc@golang.org>

encoding/xml: check for exported fields in embedded structs

Addresses issue #12367.

Must be checked in before CL 14010.

Change-Id: I4523a1de112ed02371504e27882659bce8028a9f
Reviewed-on: https://go-review.googlesource.com/14012
Reviewed-by: Russ Cox <rsc@golang.org>

encoding/json: check for exported fields in embedded structs

Addresses issue #12367.

Must be checked in before CL 14010.

Change-Id: I7233c3a62d4f55d0ac7e8a87df5fc4ee7beb7207
Reviewed-on: https://go-review.googlesource.com/14011
Reviewed-by: Russ Cox <rsc@golang.org>

unicode/utf8: added benchmarks

Cover some functions that weren't benched before and add InString
variants if the underlying implementation is different.

Note: compare (Valid|RuneCount)InString* to their (Valid|RuneCount)*
counterparts. It shows, somewhat unexpectedly, that ranging over
a string is *much* slower than using calls to DecodeRune.

Results:
In order to avoid a discrepancy in measuring the performance
of core we could leave the names of the string-based measurements
unchanged and suffix the added alternatives with Bytes.

Compared to old:
BenchmarkRuneCountTenASCIIChars-8        44.3          12.4          -72.01%
BenchmarkRuneCountTenJapaneseChars-8     167           67.1          -59.82%
BenchmarkEncodeASCIIRune-8               3.37          3.44          +2.08%
BenchmarkEncodeJapaneseRune-8            7.19          7.24          +0.70%
BenchmarkDecodeASCIIRune-8               5.41          5.53          +2.22%
BenchmarkDecodeJapaneseRune-8            8.17          8.41          +2.94%

All benchmarks:
BenchmarkRuneCountTenASCIIChars-8            100000000         12.4 ns/op
BenchmarkRuneCountTenJapaneseChars-8         20000000         67.1 ns/op
BenchmarkRuneCountInStringTenASCIIChars-8    30000000         44.5 ns/op
BenchmarkRuneCountInStringTenJapaneseChars-8 10000000        165 ns/op
BenchmarkValidTenASCIIChars-8                100000000         12.5 ns/op
BenchmarkValidTenJapaneseChars-8             20000000         71.1 ns/op
BenchmarkValidStringTenASCIIChars-8          30000000         50.0 ns/op
BenchmarkValidStringTenJapaneseChars-8       10000000        161 ns/op
BenchmarkEncodeASCIIRune-8                   500000000          3.44 ns/op
BenchmarkEncodeJapaneseRune-8                200000000          7.24 ns/op
BenchmarkDecodeASCIIRune-8                   300000000          5.53 ns/op
BenchmarkDecodeJapaneseRune-8                200000000          8.41 ns/op
BenchmarkFullASCIIRune-8                     500000000          3.91 ns/op
BenchmarkFullJapaneseRune-8                  300000000          4.22 ns/op

Change-Id: I674d2ee4917b975a37717bbfa1082cc84dcd275e
Reviewed-on: https://go-review.googlesource.com/14431
Reviewed-by: Russ Cox <rsc@golang.org>

reflect: adjust access to unexported embedded structs

This CL changes reflect to allow access to exported fields and
methods in unexported embedded structs for gccgo and after gc
has been adjusted to disallow access to embedded unexported structs.

Adresses #12367, #7363, #11007, and #7247.

Change-Id: If80536eab35abcd25300d8ddc2d27d5c42d7e78e
Reviewed-on: https://go-review.googlesource.com/14010
Reviewed-by: Russ Cox <rsc@golang.org>

runtime: copy sqrt normalization bugfix from math

This copies the change from CL 16158 (applied as
22d4c8bf13d5edf4670dbdaf0854d653d9c2b81a).

Updates #13013

Change-Id: Id7d02e63d92806f06a4e064a91b2fb6574fe385f
Reviewed-on: https://go-review.googlesource.com/16291
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

cmd/cgo: clarify C.GoStringN and C.GoBytes docs

Fixes #12427.

Change-Id: I46725620c1e163f3b60ffcd85e5388fa646f074d
Reviewed-on: https://go-review.googlesource.com/15997
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

net/url: fix mentions of URL.EscapedPath method

Fixes #12862.

Change-Id: I6921ae31bd5515f344fd97d08eafc317228b98a0
Reviewed-on: https://go-review.googlesource.com/15590
Reviewed-by: Andrew Gerrand <adg@golang.org>

cmd/compile/internal/gc: remove "goto ret" in walkexpr

Passes go build -a -toolexec 'toolstash -cmp' std cmd.

Change-Id: Ib3d2c50601546495e7f1ab153d2978b1e3774101
Reviewed-on: https://go-review.googlesource.com/14800
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

cmd/go, cmd/link: -buildmode=pie for linux/amd64

Depends on external linking right now. I have no immediate use for
this, but wanted to check how hard it is to support as android/amd64
is coming and it will require PIE.

Change-Id: I65c6b19159f40db4c79cf312cd0368c2b2527bfd
Reviewed-on: https://go-review.googlesource.com/16072
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

math: fix normalization bug in pure-Go sqrt

Fixes #13013

Change-Id: I6cf500eacdce76e303fc1cd92dd1c80eef0986bc
Reviewed-on: https://go-review.googlesource.com/16158
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime: replace is{plan9,solaris,windows} with GOOS tests

Change-Id: I27589395f547c5837dc7536a0ab5bc7cc23a4ff6
Reviewed-on: https://go-review.googlesource.com/10872
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

math/big: fix SetMantExp comment

Change-Id: If30cf9c94b58e18564db46c15c6f5cc14ec1a6fa
Reviewed-on: https://go-review.googlesource.com/16271
Reviewed-by: Robert Griesemer <gri@golang.org>

cmd/compile/internal/gc: update old c-style comments

Update old c-style comments to look like Go comments. Also replace some
lingering references to old .c files that don't exist anymore.

Change-Id: I72b2407a40fc76c23e9048643e0622fd70b4cf90
Reviewed-on: https://go-review.googlesource.com/16190
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

runtime: account for cpu affinity in windows NumCPU

Fixes #11671

Change-Id: Ide1f8d92637dad2a2faed391329f9b6001789b76
Reviewed-on: https://go-review.googlesource.com/14742
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

database/sql: use RWMutex for driver registration

Change-Id: I0f494c9f17cb6bb0cf5e7214cf033fdbd48f27f7
Reviewed-on: https://go-review.googlesource.com/16240
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

regexp: fix slice bounds out of range panics

Regular expressions involving a (x){0} term are
simplified by removing this term from the
expression, just before the expression is compiled.

The number of subexpressions is evaluated before
the simplification. The number of capture instructions
in the compiled expressions is not necessarily in line
with the number of subexpressions.

When the ReplaceAll(String) methods are used, a number
of capture slots (nmatch) is evaluated as 2*(s+1)
(s being the number of subexpressions).

In some case, it can be higher than the number of capture
instructions evaluated at compile time, resulting in a
panic when the internal slices of regexp.machine
are resized to this value.

Fixed by capping the number of capture slots to the number
of capture instructions.

I must say I do not really see the benefits of setting
nmatch lower than re.prog.NumCap using this 2*(s+1) formula,
so perhaps this can be further simplified.

Fixes #11178
Fixes #11176

Change-Id: I21415e8ef2dd5f2721218e9a679f7f6bfb76ae9b
Reviewed-on: https://go-review.googlesource.com/14013
Reviewed-by: Russ Cox <rsc@golang.org>

strings: Add examples for HasPrefix and HasSuffix

These methods didn't had any examples, so added them. Examples makes things more clear
diff --git a/src/strings/example_test.go b/src/strings/example_test.go
index 7243e16..b7763bb 100644
--- a/src/strings/example_test.go
+++ b/src/strings/example_test.go
@@ -223,3 +223,19 @@ func ExampleTrimPrefix() {
  fmt.Print("Hello" + s)
  // Output: Hello, world!
 }
+
+func ExampleHasPrefix() {
+ fmt.Println(strings.HasPrefix("hello", "hell"))
+ fmt.Println(strings.HasPrefix("hello", "heaven"))
+ // Output:
+ // true
+ // false
+}
+
+func ExampleHasSuffix() {
+ fmt.Println(strings.HasSuffix("hello", "llo"))
+ fmt.Println(strings.HasSuffix("hello", "hell"))
+ // Output:
+ // true
+ // false
+}

Change-Id: I5d451c669bd05e19a2afc33ed2ec59b280c2c2d9
Reviewed-on: https://go-review.googlesource.com/12065
Reviewed-by: Russ Cox <rsc@golang.org>

cmd/compile/internal/gc: remove special handling for -0.0

Per the latest spec change, Go doesn't have -0 constants.

Change-Id: Ic2bcdc3bf507d121ed204f30f6744bb8764202c0
Reviewed-on: https://go-review.googlesource.com/16232
Reviewed-by: Chris Manghane <cmang@golang.org>

cmd/compile/internal/gc: compact binary export format

The binary import/export format is significantly more
compact than the existing textual format. It should
also be faster to read and write (to be measured).

Use -newexport to enable, for instance:
export GO_GCFLAGS=-newexport; make.bash

The compiler can import packages using both the old
and the new format ("mixed mode").

Missing: export info for inlined functions bodies
(performance issue, does not affect correctness).

Disabled by default until we have inlined function
bodies and confirmation of no regression and equality
of binaries.

For #6110.
For #1909.

This change depends on:

   https://go-review.googlesource.com/16220
   https://go-review.googlesource.com/16222

(already submitted) for all.bash to work.

Some initial export data sizes for std lib packages. This data
is without exported functions with inlineable function bodies.

Package                                       old      new    new/old

archive/tar.................................13875.....3883    28%
archive/zip.................................19464.....5046    26%
bufio....................................... 7733.....2222    29%
bytes.......................................10342.....3347    32%
cmd/addr2line.................................242.......26    11%
cmd/api.....................................39305....10368    26%
cmd/asm/internal/arch.......................27732.....7939    29%
cmd/asm/internal/asm........................35264....10295    29%
cmd/asm/internal/flags........................629......178    28%
cmd/asm/internal/lex........................39248....11128    28%
cmd/asm.......................................306.......26     8%
cmd/cgo.....................................40197....10570    26%
cmd/compile/internal/amd64...................1106......214    19%
cmd/compile/internal/arm....................27891.....7710    28%
cmd/compile/internal/arm64....................891......153    17%
cmd/compile/internal/big....................21637.....8336    39%
cmd/compile/internal/gc....................109845....29727    27%
cmd/compile/internal/mips64...................972......168    17%
cmd/compile/internal/ppc64....................972......168    17%
cmd/compile/internal/x86.....................1104......195    18%
cmd/compile...................................329.......26     8%
cmd/cover...................................12986.....3749    29%
cmd/dist......................................477.......67    14%
cmd/doc.....................................23043.....6793    29%
cmd/expdump...................................167.......26    16%
cmd/fix......................................1190......208    17%
cmd/go......................................26399.....5629    21%
cmd/gofmt.....................................499.......26     5%
cmd/internal/gcprog..........................1342......490    37%
cmd/internal/goobj...........................2690......980    36%
cmd/internal/obj/arm........................32740....10057    31%
cmd/internal/obj/arm64......................46542....15364    33%
cmd/internal/obj/mips.......................42140....13731    33%
cmd/internal/obj/ppc64......................42140....13731    33%
cmd/internal/obj/x86........................52732....19015    36%
cmd/internal/obj............................36729....11690    32%
cmd/internal/objfile........................36365....10287    28%
cmd/link/internal/amd64.....................45893....12220    27%
cmd/link/internal/arm.........................307.......96    31%
cmd/link/internal/arm64.......................345.......98    28%
cmd/link/internal/ld.......................109300....46326    42%
cmd/link/internal/ppc64.......................344.......99    29%
cmd/link/internal/x86.........................334......107    32%
cmd/link......................................314.......26     8%
cmd/newlink..................................8110.....2544    31%
cmd/nm........................................210.......26    12%
cmd/objdump...................................244.......26    11%
cmd/pack....................................14248.....4066    29%
cmd/pprof/internal/commands..................5239.....1285    25%
cmd/pprof/internal/driver...................37967.....8860    23%
cmd/pprof/internal/fetch....................30962.....7337    24%
cmd/pprof/internal/plugin...................47734.....7719    16%
cmd/pprof/internal/profile..................22286.....6922    31%
cmd/pprof/internal/report...................31187.....7838    25%
cmd/pprof/internal/svg.......................4315......965    22%
cmd/pprof/internal/symbolizer...............30051.....7397    25%
cmd/pprof/internal/symbolz..................28545.....6949    24%
cmd/pprof/internal/tempfile.................12550.....3356    27%
cmd/pprof.....................................563.......26     5%
cmd/trace....................................1455......636    44%
cmd/vendor/golang.org/x/arch/arm/armasm....168035....64737    39%
cmd/vendor/golang.org/x/arch/x86/x86asm.....26871.....8578    32%
cmd/vet.....................................38980.....9913    25%
cmd/vet/whitelist.............................102.......49    48%
cmd/yacc.....................................2518......926    37%
compress/bzip2...............................6326......129     2%
compress/flate...............................7069.....2541    36%
compress/gzip...............................20143.....5069    25%
compress/lzw..................................828......295    36%
compress/zlib...............................10676.....2692    25%
container/heap................................523......181    35%
container/list...............................3517......740    21%
container/ring................................881......229    26%
crypto/aes....................................550......187    34%
crypto/cipher................................1966......825    42%
crypto.......................................1836......646    35%
crypto/des....................................632......235    37%
crypto/dsa..................................18718.....5035    27%
crypto/ecdsa................................23131.....6097    26%
crypto/elliptic.............................20790.....5740    28%
crypto/hmac...................................455......186    41%
crypto/md5...................................1375......171    12%
crypto/rand.................................18132.....4748    26%
crypto/rc4....................................561......240    43%
crypto/rsa..................................22094.....6380    29%
crypto/sha1..................................1416......172    12%
crypto/sha256.................................551......238    43%
crypto/sha512.................................839......378    45%
crypto/subtle................................1153......250    22%
crypto/tls..................................58203....17984    31%
crypto/x509/pkix............................29447.....8161    28%
database/sql/driver..........................3318.....1096    33%
database/sql................................11258.....3942    35%
debug/dwarf.................................18416.....7006    38%
debug/elf...................................57530....21014    37%
debug/gosym..................................4992.....2058    41%
debug/macho.................................23037.....6538    28%
debug/pe....................................21063.....6619    31%
debug/plan9obj...............................2467......802    33%
encoding/ascii85.............................1523......360    24%
encoding/asn1................................1718......527    31%
encoding/base32..............................2642......686    26%
encoding/base64..............................3077......800    26%
encoding/binary..............................4727.....1040    22%
encoding/csv................................12223.....2850    23%
encoding......................................383......217    57%
encoding/gob................................37563....10113    27%
encoding/hex.................................1327......390    29%
encoding/json...............................30897.....7804    25%
encoding/pem..................................595......200    34%
encoding/xml................................37798.....9336    25%
errors........................................274.......36    13%
expvar.......................................3155.....1021    32%
flag........................................19860.....2849    14%
fmt..........................................3137.....1263    40%
go/ast......................................44729....13422    30%
go/build....................................16336.....4657    29%
go/constant..................................3703......846    23%
go/doc.......................................9877.....2807    28%
go/format....................................5472.....1575    29%
go/importer..................................4980.....1301    26%
go/internal/gccgoimporter....................5587.....1525    27%
go/internal/gcimporter.......................8979.....2186    24%
go/parser...................................20692.....5304    26%
go/printer...................................7015.....2029    29%
go/scanner...................................9719.....2824    29%
go/token.....................................7933.....2465    31%
go/types....................................64569....19978    31%
hash/adler32.................................1176......176    15%
hash/crc32...................................1663......360    22%
hash/crc64...................................1587......306    19%
hash/fnv.....................................3964......260     7%
hash..........................................591......278    47%
html..........................................217.......74    34%
html/template...............................69623....12588    18%
image/color/palette...........................315.......98    31%
image/color..................................5565.....1036    19%
image/draw...................................6917.....1028    15%
image/gif....................................8894.....1654    19%
image/internal/imageutil.....................9112.....1476    16%
image/jpeg...................................6647.....1026    15%
image/png....................................6906.....1069    15%
image.......................................28992.....6139    21%
index/suffixarray...........................17106.....4773    28%
internal/singleflight........................1614......506    31%
internal/testenv............................12212.....3152    26%
internal/trace...............................2762.....1323    48%
io/ioutil...................................13502.....3682    27%
io...........................................6765.....2482    37%
log.........................................11620.....3317    29%
log/syslog..................................13516.....3821    28%
math/big....................................21819.....8320    38%
math/cmplx...................................2816......438    16%
math/rand....................................2317......929    40%
math.........................................7511.....2444    33%
mime/multipart..............................12679.....3360    27%
mime/quotedprintable.........................5458.....1235    23%
mime.........................................6076.....1628    27%
net/http/cgi................................59796....17173    29%
net/http/cookiejar..........................14781.....3739    25%
net/http/fcgi...............................57861....16426    28%
net/http/httptest...........................84100....24365    29%
net/http/httputil...........................67763....18869    28%
net/http/internal............................6907......637     9%
net/http/pprof..............................57945....16316    28%
net/http....................................95391....30210    32%
net/internal/socktest........................4555.....1453    32%
net/mail....................................14481.....3608    25%
net/rpc/jsonrpc.............................33335......988     3%
net/rpc.....................................79950....23106    29%
net/smtp....................................57790....16468    28%
net/textproto...............................11356.....3248    29%
net/url......................................3123.....1009    32%
os/exec.....................................20738.....5769    28%
os/signal.....................................437......167    38%
os..........................................24875.....6668    27%
path/filepath...............................11340.....2826    25%
path..........................................778......285    37%
reflect.....................................15469.....5198    34%
regexp......................................13627.....4661    34%
regexp/syntax................................5539.....2249    41%
runtime/debug................................9275.....2322    25%
runtime/pprof................................1355......477    35%
runtime/race...................................39.......17    44%
runtime/trace.................................228.......92    40%
runtime.....................................13498.....1821    13%
sort.........................................2848......842    30%
strconv......................................2947.....1252    42%
strings......................................7983.....2456    31%
sync/atomic..................................2666.....1149    43%
sync.........................................2568......845    33%
syscall.....................................81252....38398    47%
testing/iotest...............................2444......302    12%
testing/quick...............................18890.....5076    27%
testing.....................................16502.....4800    29%
text/scanner.................................6849.....2052    30%
text/tabwriter...............................6607.....1863    28%
text/template/parse.........................22978.....6183    27%
text/template...............................64153....11518    18%
time........................................12103.....3546    29%
unicode......................................9706.....3320    34%
unicode/utf16................................1055......148    14%
unicode/utf8.................................1118......513    46%
vendor/golang.org/x/net/http2/hpack..........8905.....2636    30%

All packages                              3518505  1017774    29%

Change-Id: Id657334f276383ff1e6fa91472d3d1db5a03349c
Reviewed-on: https://go-review.googlesource.com/13937
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Chris Manghane <cmang@golang.org>

cmd/link/internal/ld: export data may be marked with $$ or $$B

Preparation for dealing with binary export format. Accept $$B
as marker for export data. For now, skip that data if found.

Change-Id: I464ba22aaedcf349725379d91070fc900d93b7a2
Reviewed-on: https://go-review.googlesource.com/16222
Reviewed-by: Chris Manghane <cmang@golang.org>

cmd/internal/goobj: permit 0 bytes in export data (prep. for new export format)

Change-Id: Ida9a82a5420a41ef656388866a1cf8fadf12ccc6
Reviewed-on: https://go-review.googlesource.com/16220
Reviewed-by: Chris Manghane <cmang@golang.org>

cmd/compile/internal/gc: fixed comments (cosmetic change)

Change-Id: I9cbbfcf55ce3cccb6f14b9ffe6777567dcdad9c9
Reviewed-on: https://go-review.googlesource.com/16179
Reviewed-by: Chris Manghane <cmang@golang.org>

runtime: add pcvalue cache to improve stack scan speed

The cost of scanning large stacks is currently dominated by the time
spent looking up and decoding the pcvalue table. However, large stacks
are usually large not because they contain calls to many different
functions, but because they contain many calls to the same, small set
of recursive functions. Hence, walking large stacks tends to make the
same pcvalue queries many times.

Based on this observation, this commit adds a small, very simple, and
fast cache in front of pcvalue lookup. We thread this cache down from
operations that make many pcvalue calls, such as gentraceback, stack
scanning, and stack adjusting.

This simple cache works well because it has minimal overhead when it's
not effective. I also tried a hashed direct-map cache, CLOCK-based
replacement, round-robin replacement, and round-robin with lookups
disabled until there had been at least 16 probes, but none of these
approaches had obvious wins over the random replacement policy in this
commit.

This nearly doubles the overall performance of the deep stack test
program from issue #10898:

name        old time/op  new time/op  delta
Issue10898   16.5s ±12%    9.2s ±12%  -44.37%  (p=0.008 n=5+5)

It's a very slight win on the garbage benchmark:

name              old time/op  new time/op  delta
XBenchGarbage-12  4.92ms ± 1%  4.89ms ± 1%  -0.75%  (p=0.000 n=18+19)

It's a wash (but doesn't harm performance) on the go1 benchmarks,
which don't have particularly deep stacks:

name                      old time/op    new time/op    delta
BinaryTree17-12              3.11s ± 2%     3.20s ± 3%  +2.83%  (p=0.000 n=17+20)
Fannkuch11-12                2.51s ± 1%     2.51s ± 1%  -0.22%  (p=0.034 n=19+18)
FmtFprintfEmpty-12          50.8ns ± 3%    50.6ns ± 2%    ~     (p=0.793 n=20+20)
FmtFprintfString-12          174ns ± 0%     174ns ± 1%  +0.17%  (p=0.048 n=15+20)
FmtFprintfInt-12             177ns ± 0%     165ns ± 1%  -6.99%  (p=0.000 n=17+19)
FmtFprintfIntInt-12          283ns ± 1%     284ns ± 0%  +0.22%  (p=0.000 n=18+15)
FmtFprintfPrefixedInt-12     243ns ± 1%     244ns ± 1%  +0.40%  (p=0.000 n=20+19)
FmtFprintfFloat-12           318ns ± 0%     319ns ± 0%  +0.27%  (p=0.001 n=19+20)
FmtManyArgs-12              1.12µs ± 0%    1.14µs ± 0%  +1.74%  (p=0.000 n=19+20)
GobDecode-12                8.69ms ± 0%    8.73ms ± 1%  +0.46%  (p=0.000 n=18+18)
GobEncode-12                6.64ms ± 1%    6.61ms ± 1%  -0.46%  (p=0.000 n=20+20)
Gzip-12                      323ms ± 2%     319ms ± 1%  -1.11%  (p=0.000 n=20+20)
Gunzip-12                   42.8ms ± 0%    42.9ms ± 0%    ~     (p=0.158 n=18+20)
HTTPClientServer-12         63.3µs ± 1%    63.1µs ± 1%  -0.35%  (p=0.011 n=20+20)
JSONEncode-12               16.9ms ± 1%    17.3ms ± 1%  +2.84%  (p=0.000 n=19+20)
JSONDecode-12               59.7ms ± 0%    58.5ms ± 0%  -2.05%  (p=0.000 n=19+17)
Mandelbrot200-12            3.92ms ± 0%    3.91ms ± 0%  -0.16%  (p=0.003 n=19+19)
GoParse-12                  3.79ms ± 2%    3.75ms ± 2%  -0.91%  (p=0.005 n=20+20)
RegexpMatchEasy0_32-12       102ns ± 1%     101ns ± 1%  -0.80%  (p=0.001 n=14+20)
RegexpMatchEasy0_1K-12       337ns ± 1%     346ns ± 1%  +2.90%  (p=0.000 n=20+19)
RegexpMatchEasy1_32-12      84.4ns ± 2%    84.3ns ± 2%    ~     (p=0.743 n=20+20)
RegexpMatchEasy1_1K-12       502ns ± 1%     505ns ± 0%  +0.64%  (p=0.000 n=20+20)
RegexpMatchMedium_32-12      133ns ± 1%     132ns ± 1%  -0.85%  (p=0.000 n=20+19)
RegexpMatchMedium_1K-12     40.1µs ± 1%    39.8µs ± 1%  -0.77%  (p=0.000 n=18+18)
RegexpMatchHard_32-12       2.08µs ± 1%    2.07µs ± 1%  -0.55%  (p=0.001 n=18+19)
RegexpMatchHard_1K-12       62.4µs ± 1%    62.0µs ± 1%  -0.74%  (p=0.000 n=19+19)
Revcomp-12                   545ms ± 2%     545ms ± 3%    ~     (p=0.771 n=19+20)
Template-12                 73.7ms ± 1%    72.0ms ± 0%  -2.33%  (p=0.000 n=20+18)
TimeParse-12                 358ns ± 1%     351ns ± 1%  -2.07%  (p=0.000 n=20+20)
TimeFormat-12                369ns ± 1%     356ns ± 0%  -3.53%  (p=0.000 n=20+18)
[Geo mean]                  63.5µs         63.2µs       -0.41%

name                      old speed      new speed      delta
GobDecode-12              88.3MB/s ± 0%  87.9MB/s ± 0%  -0.43%  (p=0.000 n=18+17)
GobEncode-12               116MB/s ± 1%   116MB/s ± 1%  +0.47%  (p=0.000 n=20+20)
Gzip-12                   60.2MB/s ± 2%  60.8MB/s ± 1%  +1.13%  (p=0.000 n=20+20)
Gunzip-12                  453MB/s ± 0%   453MB/s ± 0%    ~     (p=0.160 n=18+20)
JSONEncode-12              115MB/s ± 1%   112MB/s ± 1%  -2.76%  (p=0.000 n=19+20)
JSONDecode-12             32.5MB/s ± 0%  33.2MB/s ± 0%  +2.09%  (p=0.000 n=19+17)
GoParse-12                15.3MB/s ± 2%  15.4MB/s ± 2%  +0.92%  (p=0.004 n=20+20)
RegexpMatchEasy0_32-12     311MB/s ± 1%   314MB/s ± 1%  +0.78%  (p=0.000 n=15+19)
RegexpMatchEasy0_1K-12    3.04GB/s ± 1%  2.95GB/s ± 1%  -2.90%  (p=0.000 n=19+19)
RegexpMatchEasy1_32-12     379MB/s ± 2%   380MB/s ± 2%    ~     (p=0.779 n=20+20)
RegexpMatchEasy1_1K-12    2.04GB/s ± 1%  2.02GB/s ± 0%  -0.62%  (p=0.000 n=20+20)
RegexpMatchMedium_32-12   7.46MB/s ± 1%  7.53MB/s ± 1%  +0.86%  (p=0.000 n=20+19)
RegexpMatchMedium_1K-12   25.5MB/s ± 1%  25.7MB/s ± 1%  +0.78%  (p=0.000 n=18+18)
RegexpMatchHard_32-12     15.4MB/s ± 1%  15.5MB/s ± 1%  +0.62%  (p=0.000 n=19+19)
RegexpMatchHard_1K-12     16.4MB/s ± 1%  16.5MB/s ± 1%  +0.82%  (p=0.000 n=20+19)
Revcomp-12                 466MB/s ± 2%   466MB/s ± 3%    ~     (p=0.765 n=19+20)
Template-12               26.3MB/s ± 1%  27.0MB/s ± 0%  +2.38%  (p=0.000 n=20+18)
[Geo mean]                97.8MB/s       98.0MB/s       +0.23%

Change-Id: I281044ae0b24990ba46487cacbc1069493274bc4
Reviewed-on: https://go-review.googlesource.com/13614
Reviewed-by: Keith Randall <khr@golang.org>

net/http/httptest: fix comment regarding certificate expiry

In earlier versions of Go, times were only encoded as an ASN.1 UTCTIME and
crypto/tls/generate_cert.go limited times to the maximum UTCTIME value.
Revision 050b60a3 added support for ASN.1 GENERALIZEDTIME, allowing larger
time values to be represented (per RFC 5280).

As a result, when the httptest certificate was regenerated in revision
9b2d84ef, the Not After date changed to Jan 29 16:00:00 2084 GMT. Update
the comment to reflect this.

Change-Id: I1bd66e011f2749f9372b5c7506f52ea34e264ce9
Reviewed-on: https://go-review.googlesource.com/16193
Reviewed-by: Adam Langley <agl@golang.org>

runtime: add mSpanList type to represent lists of mspans

This CL introduces a new mSpanList type to replace the empty mspan
variables that were previously used as list heads.

To be type safe, the previous circular linked list data structure is
now a tail queue instead.  One complication of this is
mSpanList_Remove needs to know the list a span is being removed from,
but this appears to be computable in all circumstances.

As a temporary sanity check, mSpanList_Insert and mSpanList_InsertBack
record the list that an mspan has been inserted into so that
mSpanList_Remove can verify that the correct list was specified.

Whereas mspan is 112 bytes on amd64, mSpanList is only 16 bytes.  This
shrinks the size of mheap from 50216 bytes to 12584 bytes.

Change-Id: I8146364753dbc3b4ab120afbb9c7b8740653c216
Reviewed-on: https://go-review.googlesource.com/15906
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Austin Clements <austin@google.com>

runtime: remove unused printpc and printbyte functions

Change-Id: I40e338f6b445ca72055fc9bac0f09f0dca904e3a
Reviewed-on: https://go-review.googlesource.com/16191
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

os/exec: make Cmd.Output include stderr in ExitError

Change-Id: I3c6649d2f2521ab0843b13308569867d2e5f02da
Reviewed-on: https://go-review.googlesource.com/11415
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

path/filepath: test EvalSymlinks returns canonical path on windows

When you create C:\A.TXT file on windows, you can open it as c:\a.txt.
EvalSymlinks("c:\a.txt") returns C:\A.TXT. This is all EvalSymlinks
did in the past, but recently symlinks functionality been implemented on
some Windows version (where symlinks are supported). So now EvalSymlinks
handles both: searching for file canonical name and resolving symlinks.

Unfortunately TestEvalSymlinks has not been adjusted properly. The test
tests either canonical paths or symlinks, but not both. This CL separates
canonical paths tests into new TestEvalSymlinksCanonicalNames, so all
functionality is covered. Tests are simplified somewhat too.

Also remove EvalSymlinksAbsWindowsTests - it seems not used anywhere.

Change-Id: Id12e9f1441c1e30f15c523b250469978e4511a84
Reviewed-on: https://go-review.googlesource.com/14412
Reviewed-by: Ian Lance Taylor <iant@golang.org>

runtime: prune some dead variables

Change-Id: I7a1c3079b433c4e30d72fb7d59f9594e0d5efe47
Reviewed-on: https://go-review.googlesource.com/16178
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>

runtime: change fixalloc's chunk field to unsafe.Pointer

It's never used as a *byte anyway, so might as well just make it an
unsafe.Pointer instead.

Change-Id: I68ee418781ab2fc574eeac0498f2515b5561b7a8
Reviewed-on: https://go-review.googlesource.com/16175
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

runtime: fix typos

Change-Id: Iffc25fc80452baf090bf8ef15ab798cfaa120b8e
Reviewed-on: https://go-review.googlesource.com/16154
Reviewed-by: Ian Lance Taylor <iant@golang.org>

runtime: split plan9 and solaris's m fields into new embedded mOS type

Reduces the size of m by ~8% on linux/amd64 (1040 bytes -> 960 bytes).

There are also windows-specific fields, but they're currently
referenced in OS-independent source files (but only when
GOOS=="windows").

Change-Id: I13e1471ff585ccced1271f74209f8ed6df14c202
Reviewed-on: https://go-review.googlesource.com/16173
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/go: if -msan, pass -fsanitize=memory to cgo builds

Also fix the msan_fail test.  It was bogus, since it always aborted one
way or another.

Change-Id: Ic693327d1bddb7bc5c7d859ac047fc93cb9b5b1c
Reviewed-on: https://go-review.googlesource.com/16172
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime: replace unsafe pointer arithmetic with array indexing

Change-Id: I313819abebd4cda4a6c30fd0fd6f44cb1d09161f
Reviewed-on: https://go-review.googlesource.com/16167
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

runtime: make iface/eface handling more type safe

Change compiler-invoked interface functions to directly take
iface/eface parameters instead of fInterface/interface{} to avoid
needing to always convert.

For the handful of functions that legitimately need to take an
interface{} parameter, add efaceOf to type-safely convert *interface{}
to *eface.

Change-Id: I8928761a12fd3c771394f36adf93d3006a9fcf39
Reviewed-on: https://go-review.googlesource.com/16166
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

doc: go1.6.txt: -msan option for cmd/{go,compile,link}

Change-Id: I8b41de496e4b58214b98267b529f3525ff6d9745
Reviewed-on: https://go-review.googlesource.com/16171
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/go: add -msan option

The -msan option compiles Go code to use the memory sanitizer.  This is
intended for use when linking with C/C++ code compiled with
-fsanitize=memory.  When memory blocks are passed back and forth between
C/C++ and Go, code in both languages will agree as to whether the memory
is correctly initialized or not, and will report errors for any use of
uninitialized memory.

Change-Id: I2dbdbd26951eacb7d84063cfc7297f88ffadd70c
Reviewed-on: https://go-review.googlesource.com/16169
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/internal/obj: fix PSRLW opcode

The reg-reg version compiled to PSRAW, not PSRLW (arithmetic
instead of logical shift right).

Fixes #13010.

Change-Id: I69a47bd83c8bbe66c7f8d82442ab45e9bf3b94fb
Reviewed-on: https://go-review.googlesource.com/16168
Reviewed-by: Ian Lance Taylor <iant@golang.org>

androidtest.bash: correct the location of compiled packages

For android, gc builds with buildmode=pie by default, and
as a result, the compiled packages are not installed in
the usual pkg/$GOOS_$GOARCH pack. Copy the compiled packages
in pkg/android_$GOARCH_shared into the pkg/android_$GOARCH
in the test device.

Change-Id: I909e4cc7095ac95ef63bdf6ddc4cb2c698f3459e
Reviewed-on: https://go-review.googlesource.com/16151
Reviewed-by: David Crawshaw <crawshaw@golang.org>

runtime, syscall: add calls to msan functions

Add explicit memory sanitizer instrumentation to the runtime and syscall
packages.  The compiler does not instrument the runtime package.  It
does instrument the syscall package, but we need to add a couple of
cases that it can't see.

Change-Id: I2d66073f713fe67e33a6720460d2bb8f72f31394
Reviewed-on: https://go-review.googlesource.com/16164
Reviewed-by: David Crawshaw <crawshaw@golang.org>

runtime: change functype's in and out fields to []*_type

Allows removing a few gratuitous unsafe.Pointer conversions and
parallels the type of reflect.funcType's in and out fields ([]*rtype).

Change-Id: Ie5ca230a94407301a854dfd8782a3180d5054bc4
Reviewed-on: https://go-review.googlesource.com/16163
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

go/build: test code cleanup: remove unnecessary func var

The earlier calls to test(false) in TestDependencies were
removed by https://golang.org/cl/12576

Change-Id: If5c7994172379c2d7f633d2e9c5261e668c754fa
Reviewed-on: https://go-review.googlesource.com/16117
Reviewed-by: Ian Lance Taylor <iant@golang.org>

log/syslog: disable unix/unixgram tests on android

unix/unixgram is not available to standard Android programs.

For golang/go#10807

Change-Id: I6062c3a25cffb86e58cbbd12a07dc90ffbf57185
Reviewed-on: https://go-review.googlesource.com/16114
Reviewed-by: Ian Lance Taylor <iant@golang.org>

runtime, runtime/msan: add msan runtime support

These are the runtime support functions for letting Go code interoperate
with the C/C++ memory sanitizer.  Calls to msanread/msanwrite are now
inserted by the compiler with the -msan option.  Calls to
msanmalloc/msanfree will be from other runtime functions in a subsequent
CL.

Change-Id: I64fb061b38cc6519153face242eccd291c07d1f2
Reviewed-on: https://go-review.googlesource.com/16162
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime: eliminate unused _GCstw phase

Change-Id: Ie94cd17e1975fdaaa418fa6a7b2d3b164fedc135
Reviewed-on: https://go-review.googlesource.com/16057
Reviewed-by: Rick Hudson <rlh@golang.org>

runtime: eliminate unnecessary ragged barrier

The ragged barrier after entering the concurrent mark phase is
vestigial. This used to be the point where we enabled write barriers,
so it was necessary to synchronize all Ps to ensure write barriers
were enabled before any marking occurred. However, we've long since
switched to enabling write barriers during the concurrent scan phase,
so the start-the-world at the beginning of the concurrent scan phase
ensures that all Ps have enabled the write barrier.

Hence, we can eliminate the old "install write barrier" phase.

Fixes #11971.

Change-Id: I8cdcb84b5525cef19927d51ea11ba0a4db991ea8
Reviewed-on: https://go-review.googlesource.com/16044
Reviewed-by: Rick Hudson <rlh@golang.org>

cmd/compile: add -msan option

The -msan option causes the compiler to add instrumentation for the
C/C++ memory sanitizer.  Every memory read/write will be preceded by
a call to msanread/msanwrite.

This CL passes tests but is not usable by itself.  The actual
implementation of msanread/msanwrite in the runtime package, and support
for -msan in the go tool and the linker, and tests, will follow in
subsequent CLs.

Change-Id: I3d517fb3e6e65d9bf9433db070a420fd11f57816
Reviewed-on: https://go-review.googlesource.com/16160
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/link: add -msan option

The -msan option causes the linker to link against the runtime/msan
package in order to use the C/C++ memory sanitizer.

This CL passes tests but is not usable by itself.  The actual
runtime/msan package, and support for -msan in the go tool and the
compiler, and tests, are in separate CLs.

Change-Id: I02c097393b98c5b80e40ee3dbc167a8b4d23efe0
Reviewed-on: https://go-review.googlesource.com/16161
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/vet: fix shadow assignment check with complex rhs

This change fixes shadow assignment check in cases when RHS is not an identifier
or a type assertion.

Fixes #12188

Change-Id: I0940df8d9c237ab8b8d3272eb6895e676c75c115
Reviewed-on: https://go-review.googlesource.com/16038
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

cmd/compile: generalize racewalk to instrument (naming change)

This is mechanical change that is a step toward reusing the racewalk
pass for a more general instrumentation pass.  The first use will be to
add support for the memory sanitizer.

Change-Id: I75b93b814ac60c1db1660e0b9a9a7d7977d86939
Reviewed-on: https://go-review.googlesource.com/16105
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/go: dedup packages in packagesAndErrors

packagesAndErrors function doesn't dedup packages.
As a result, `go list io ./io` prints io package twice.
Same applies to `go build` and `go test`.

* dedup packages.
* add a test for go list

Change-Id: I54d4063979b1c9359e5416e12327cb85c4823a0f
Reviewed-on: https://go-review.googlesource.com/16136
Run-TryBot: Andrew Gerrand <adg@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

cmd/compile: remove a stray word in the go:nosplit documentation

Change-Id: I7a85c0ad8aba2d50032c8faa58c83fb327f360cf
Reviewed-on: https://go-review.googlesource.com/16140
Reviewed-by: Andrew Gerrand <adg@golang.org>

cmd/compile: "invalid variable name x in type switch", where x is a name of a constant

Small fix: looks like a short variable declaration with a type switch
checks to make sure the variable used had valid shape (ONAME, OTYPE, or
ONONAME) and rejects everything else. Then a new variable is declared.
If the symbol contained in the declaration was a named OLITERAL (still a
valid identifier obviously) it would be rejected, even though a new
variable would have been declared.

Fix adds this case to the check.

Added a test case from issue12413.

Fixes #12413

Change-Id: I150dadafa8ee5612c867d58031027f2dca8c6ebc
Reviewed-on: https://go-review.googlesource.com/15760
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

net/http: update copy of http2

Updates to x/net git revision 9946ad7

Change-Id: I95c03daf382667002a5b22f184bd9b7d18144913
Reviewed-on: https://go-review.googlesource.com/16066
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

html/template: include itself while cloning

template.Clone() initialized template set incorrectly:
it didn't include itself.

* include itself in template set while cloning
* add a test

Fixes #12996

Change-Id: I932530e4f7f1bbebf833e12b000a5ce052bc9223
Reviewed-on: https://go-review.googlesource.com/16104
Reviewed-by: Andrew Gerrand <adg@golang.org>

net/http: return error from Serve if http2.ConfigureServer returns an error

In https://golang.org/cl/15860 http2.ConfigureServer was changed to
return an error if explicit CipherSuites are listed and they're not
compliant with the HTTP/2 spec.

This is the net/http side of the change, to look at the return value
from ConfigureServer and propagate it in Server.Serve.

h2_bundle.go will be updated in a future CL. There are too many other
http2 changes pending to be worth updating it now. Instead,
h2_bundle.go is minimally updated by hand in this CL so at least the
net/http change will compile.

Updates #12895

Change-Id: I4df7a097faff2d235742c2d310c333bd3fd5c08e
Reviewed-on: https://go-review.googlesource.com/16065
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime: add stringStructOf helper function

Instead of open-coding conversions from *string to unsafe.Pointer then
to *stringStruct, add a helper function to add some type safety.
Bonus: This caught two **string values being converted to
*stringStruct in heapdump.go.

While here, get rid of the redundant _string type, but add in a
stringStructDWARF type used for generating DWARF debug info.

Change-Id: I8882f8cca66ac45190270f82019a5d85db023bd2
Reviewed-on: https://go-review.googlesource.com/16131
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

runtime: change odd 'print1_write' file names

The '1' part is left over from the C conversion, but no longer makes
sense given that print1.go no longer exists.

Change-Id: Iec171251370d740f234afdbd6fb1a4009fde6696
Reviewed-on: https://go-review.googlesource.com/16036
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

path/filepath: disable symlink tests on android.

Same reason as https://go-review.googlesource.com/#/c/16115/

For golang/go#10807

Change-Id: Id0c404e9feb963f39a111fc317c9787692516ae1
Reviewed-on: https://go-review.googlesource.com/16116
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

os: disable symlink tests on android.

Creating symlinks (/data/local/tmp/*) doesn't seem to work
on android-L (tested on nexus5). I cannot find any official
documentation yet but just guess it's a measure for security
attacks using symlinks.

The tests failed with 'permission denied' errors.

For golang/go#10807

Change-Id: I99a9c401c6ecca3c4adc7b21708adaf3dd601279
Reviewed-on: https://go-review.googlesource.com/16115
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/compile/internal/gc: there are no -0 floating-point constants

Fixes #12577.

Change-Id: Id469cd92f5f9436b0ef948ee1a252ed1842bc7aa
Reviewed-on: https://go-review.googlesource.com/16133
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

spec: remove "untyped bool" oxymorons

The proper term is "untyped boolean".

Change-Id: Id871164190a03c64a8a8987b1ad5d8653a21d96e
Reviewed-on: https://go-review.googlesource.com/16135
Reviewed-by: Robert Griesemer <gri@golang.org>

crypto/cipher: fix CTR infinite loop with large block sizes

Additionally, add a test for CTR mode to cover a range of block sizes.

Fixes #12975

Change-Id: I458aac1616228747e62f92f823768d55e874877a
Reviewed-on: https://go-review.googlesource.com/16050
Reviewed-by: Adam Langley <agl@golang.org>

spec: clarify numeric conversions where IEEE-754 produces -0.0

The spec defines precise numeric constants which do not overflow.
Consequently, +/-Inf and NaN values were excluded. The case was not
clear for -0.0 but they are mostly of interest to determine the sign
of infinities which don't exist.

That said, the conversion rules explicitly say that T(x) (for a numeric
x and floating-point type T) is the value after rounding per IEEE-754.
The result is constant if x is constant. Rounding per IEEE-754 can
produce a -0.0 which we cannot represent as a constant.

Thus, the spec is inconsistent. Attempt to fix the inconsistency by
adjusting the rounding rule rather than letting -0.0 into the language.

For more details, see the issue below.

Open to discussion.

Fixes #12576.

Change-Id: Ibe3c676372ab16d9229f1f9daaf316f761e074ee
Reviewed-on: https://go-review.googlesource.com/14727
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

cmd/compile/internal/gc: remove atoi function (minor cleanup)

Change-Id: I0ad7836c0e8d70ffdc458e125d97b01e85d8a608
Reviewed-on: https://go-review.googlesource.com/16130
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

runtime: add syscalls needed for android/amd64 logging.

access, connect, socket.

In Android-L, logging is done by writing the log messages to the logd
process through a unix domain socket.

Also, changed the arg types of those syscall stubs to match linux
programming APIs.

For golang/go#10743

Change-Id: I66368a03316e253561e9e76aadd180c2cd2e48f3
Reviewed-on: https://go-review.googlesource.com/15993
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/internal/obj/x86: align functions with trap instruction

Align functions with 0xCC (INT $3) - breakpoint instruction,
instead of 0x00, which can disassemble into valid instruction.

Change-Id: Ieda191886efc4aacb86f58bea1169fd1b3b57636
Reviewed-on: https://go-review.googlesource.com/16102
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
Reviewed-by: Gregory Shimansky <gregory.shimansky@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

cmd/compile, syscall: use go:norace comment for forkAndExecInChild

Use a go:norace comment rather than having the compiler know the special
name syscall.forkAndExecInChild.

Change-Id: I69bc6aa6fc40feb2148d23f269ff32453696fb28
Reviewed-on: https://go-review.googlesource.com/16097
Reviewed-by: Minux Ma <minux@golang.org>

runtime: rename _func.frame to make it clear it's deprecated and unused.

When I saw that it was labelled "legacy", I went looking for users of it
to see how it was still used. But there aren't any. Save the next person
the trouble.

Change-Id: I921dd6c57b60331c9816542272555153ac133c02
Reviewed-on: https://go-review.googlesource.com/16035
Reviewed-by: Dave Cheney <dave@cheney.net>
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>