runtime, reflect: support multiple moduledata objects

This changes all the places that consult themoduledata to consult a
linked list of moduledata objects, as will be necessary for
-linkshared to work.

Obviously, as there is as yet no way of adding moduledata objects to
this list, all this change achieves right now is wasting a few
instructions here and there.

Change-Id: I397af7f60d0849b76aaccedf72238fe664867051
Reviewed-on: https://go-review.googlesource.com/8231
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

cmd/go: support -buildmode=default,archive,exe,c-shared

Modelled somewhat on the -race support.

Change-Id: I137037addfc76341f7deb216776fdd18e9af9fe5
Reviewed-on: https://go-review.googlesource.com/8680
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

cmd/internal/obj/x86, cmd/internal/ld, cmd/6l: 6g/asm -dynlink accesses global data via a GOT

Change-Id: I49862e177045369d6c94d6a58afbdace4f13cc96
Reviewed-on: https://go-review.googlesource.com/8237
Reviewed-by: Ian Lance Taylor <iant@golang.org>

runtime: fix arm64 build

Broken by CL 8541.

Change-Id: Ie2e89a22b91748e82f7bc4723660a24ed4135687
Reviewed-on: https://go-review.googlesource.com/8734
Reviewed-by: Minux Ma <minux@golang.org>

runtime: report next_gc for initial heap size in gctrace

Currently, the initial heap size reported in the gctrace line is the
heap_live right before sweep termination. However, we triggered GC
when heap_live reached next_gc, and there may have been significant
allocation between that point and the beginning of sweep
termination. Ideally these would be essentially the same, but
currently there's scheduler delay when readying the GC goroutine as
well as delay from background sweep finalization.

We should fix this delay, but in the mean time, to give the user a
better idea of how much the heap grew during the whole of garbage
collection, report the trigger rather than what the heap size happened
to be after the garbage collector finished rolling out of bed. This
will also be more useful for heap growth plots.

Change-Id: I08476b9fbcfb2de90592405e9c9f434dfb9eb1f8
Reviewed-on: https://go-review.googlesource.com/8512
Reviewed-by: Rick Hudson <rlh@golang.org>

math/big: better doc strings for ErrNaN functionality

Change-Id: Ia0944e7b47193465d3ec37fc8dc46dea9b5dcc6b
Reviewed-on: https://go-review.googlesource.com/8710
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

runtime: add isarchive, set by the linker

According to Go execution modes, a Go program compiled with
-buildmode=c-archive has a main function, but it is ignored on run.
This gives the runtime the information it needs not to run the main.

I have this working with pending linker changes on darwin/amd64.

Change-Id: I49bd7d65aa619ec847c464a872afa5deea7d4d30
Reviewed-on: https://go-review.googlesource.com/8701
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

log: logging an empty string should still print a line

Print("") was printing a header but no line.

Fixes #9665.

Change-Id: Iac783187786065e1389ad6e8d7ef02c579ed7bd8
Reviewed-on: https://go-review.googlesource.com/8665
Reviewed-by: Ian Lance Taylor <iant@golang.org>

encoding/gob: more checks for corrupted data

Also unify the tests where possible to make it easy to add more.

Fixes #10273.

Change-Id: Idfa4f4a5dcaa05974066bafe17bed6cdd2ebedb7
Reviewed-on: https://go-review.googlesource.com/8662
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

runtime: add arm64 runtime.cmpstring and bytes.Compare

Add arm64 assembly implementation of runtime.cmpstring and bytes.Compare.

benchmark                                old ns/op     new ns/op     delta
BenchmarkCompareBytesEqual               98.0          27.5          -71.94%
BenchmarkCompareBytesToNil               9.38          10.0          +6.61%
BenchmarkCompareBytesEmpty               13.3          10.0          -24.81%
BenchmarkCompareBytesIdentical           98.0          27.5          -71.94%
BenchmarkCompareBytesSameLength          43.3          16.3          -62.36%
BenchmarkCompareBytesDifferentLength     43.4          16.3          -62.44%
BenchmarkCompareBytesBigUnaligned        6979680       1360979       -80.50%
BenchmarkCompareBytesBig                 6915995       1381979       -80.02%
BenchmarkCompareBytesBigIdentical        6781440       1327304       -80.43%

benchmark                             old MB/s     new MB/s     speedup
BenchmarkCompareBytesBigUnaligned     150.23       770.46       5.13x
BenchmarkCompareBytesBig              151.62       758.76       5.00x
BenchmarkCompareBytesBigIdentical     154.63       790.01       5.11x

* note, the machine we are benchmarking on has some issues. What is clear is
compared to a few days ago the old MB/s value has increased from ~115 to 150.
I'm less certain about the new MB/s number, which used to be close to 1Gb/s.

Change-Id: I4f31b2c7a06296e13912aacc958525632cb0450d
Reviewed-on: https://go-review.googlesource.com/8541
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

syscall: reduce the set of architecture-dependent files on Plan 9

Change-Id: I98b172181c2fd85aa385341e28bc661dbc274252
Reviewed-on: https://go-review.googlesource.com/2167
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

net/http: handle close/response race more gracefully

There was a logical race in Transport.RoundTrip where a roundtrip with
a pending response would race with the channel for the connection
closing. This usually happened for responses with connection: close
and no body.

We handled this race by reading the close channel, setting a timer
for 100ms and if no response was returned before then, we would then
return an error.

This put a lower bound on how fast a connection could fail. We couldn't
fail a request faster than 100ms.

Reordering the channel operations gets rid of the logical race. If
the readLoop causes the connection to be closed, it would have put
its response into the return channel already and we can fetch it with
a non-blocking receive.

Change-Id: Idf09e48d7a0453d7de0120d3055d0ce5893a5428
Reviewed-on: https://go-review.googlesource.com/1787
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/go: detect which VCS to use with Bitbucket when the API call fails.

      The API call will fail when Bitbucket repositories are private. In
that case, probe for the repository using vcsCmd.ping.

      Fixes #5375

Change-Id: Ia604ecf9014805579dfda4b5c8e627a52783d56e
Reviewed-on: https://go-review.googlesource.com/1910
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/gc: fix escape analysis of closures

Fixes #10353

See test/escape2.go:issue10353. Previously new(int) did not escape to heap,
and so heap-allcated closure was referencing a stack var. This breaks
the invariant that heap must not contain pointers to stack.

Look at the following program:

package main

func main() {
foo(new(int))
bar(new(int))
}

func foo(x *int) func() {
return func() {
println(*x)
}
}

// Models what foo effectively does.
func bar(x *int) *C {
return &C{x}
}

type C struct {
x *int
}

Without this patch escape analysis works as follows:

$ go build -gcflags="-m -m -m -l" esc.go
escflood:1: dst ~r1 scope:foo[0]
escwalk: level:0 depth:0  func literal( l(9) f(1) esc(no) ld(1)) scope:foo[1]
/tmp/live2.go:9: func literal escapes to heap
escwalk: level:0 depth:1   x( l(8) class(PPARAM) f(1) esc(no) ld(1)) scope:foo[1]
/tmp/live2.go:8: leaking param: x to result ~r1

escflood:2: dst ~r1 scope:bar[0]
escwalk: level:0 depth:0  &C literal( l(15) esc(no) ld(1)) scope:bar[1]
/tmp/live2.go:15: &C literal escapes to heap
escwalk: level:-1 depth:1   &C literal( l(15)) scope:bar[0]
escwalk: level:-1 depth:2   x( l(14) class(PPARAM) f(1) esc(no) ld(1)) scope:bar[1]
/tmp/live2.go:14: leaking param: x

/tmp/live2.go:5: new(int) escapes to heap
/tmp/live2.go:4: main new(int) does not escape

new(int) does not escape while being captured by the closure.
With this patch escape analysis of foo and bar works similarly:

$ go build -gcflags="-m -m -m -l" esc.go
escflood:1: dst ~r1 scope:foo[0]
escwalk: level:0 depth:0  &(func literal)( l(9)) scope:foo[0]
escwalk: level:-1 depth:1   func literal( l(9) f(1) esc(no) ld(1)) scope:foo[1]
/tmp/live2.go:9: func literal escapes to heap
escwalk: level:-1 depth:2   x( l(8) class(PPARAM) f(1) esc(no) ld(1)) scope:foo[1]
/tmp/live2.go:8: leaking param: x

escflood:2: dst ~r1 scope:bar[0]
escwalk: level:0 depth:0  &C literal( l(15) esc(no) ld(1)) scope:bar[1]
/tmp/live2.go:15: &C literal escapes to heap
escwalk: level:-1 depth:1   &C literal( l(15)) scope:bar[0]
escwalk: level:-1 depth:2   x( l(14) class(PPARAM) f(1) esc(no) ld(1)) scope:bar[1]
/tmp/live2.go:14: leaking param: x

/tmp/live2.go:4: new(int) escapes to heap
/tmp/live2.go:5: new(int) escapes to heap

Change-Id: Ifd14b7ae3fc11820e3b5eb31eb07f35a22ed0932
Reviewed-on: https://go-review.googlesource.com/8408
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime: make windows exception handler code arch independent

Mainly it is simple copy. But I had to change amd64
lastcontinuehandler return value from uint32 to int32.
I don't remember how it happened to be uint32, but new
int32 is matching better with Windows documentation (LONG).
I don't think it matters one way or the others.

Change-Id: I6935224a2470ad6301e27590f2baa86c13bbe8d5
Reviewed-on: https://go-review.googlesource.com/8686
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

regexp: skip backtracker for long programs

This update makes maxBacktrackLen return 0 if
len(prog.Inst) > maxBacktrackProg. This prevents an attempt to
backtrack against a nil bitstate.

Fixes #10319

Change-Id: Icdbeb2392782ccf66f9d0a70ea57af22fb93f01b
Reviewed-on: https://go-review.googlesource.com/8473
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

net/http: don't send IPv6 zone identifier in outbound request, per RFC 6874

When making a request to an IPv6 address with a zone identifier, for
exmaple [fe80::1%en0], RFC 6874 says HTTP clients must remove the zone
identifier "%en0" before writing the request for security reason.

This change removes any IPv6 zone identifer attached to URI in the Host
header field in requests.

Fixes #9544.

Change-Id: I7406bd0aa961d260d96f1f887c2e45854e921452
Reviewed-on: https://go-review.googlesource.com/3111
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

os: windows Rename should overwrite destination file.

Rename now uses MoveFileEx which was previously not available to
use because it is not supported on Windows 2000.

Change-Id: I583d029c4467c9be6d1574a790c423559b441e87
Reviewed-on: https://go-review.googlesource.com/6140
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>

cmd/internal/obj/arm64, cmd/asm/internal/asm: support CSEL instruction on arm64

Add support for arm64 four operand conditional instructions.

Superceedes CL 8405.

Change-Id: I12da8f4822938feec400bbcc426eeaf884536135
Reviewed-on: https://go-review.googlesource.com/8638
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>

runtime: do not calculate asmstdcall address every time we make syscall

Change-Id: If3c8c9035e12d41647ae4982883f6a979313ea9d
Reviewed-on: https://go-review.googlesource.com/8682
Reviewed-by: Minux Ma <minux@golang.org>

image/jpeg: reject multiple Start-Of-Frame markers.

Fixes #10389

Change-Id: Id1c687122751f9317041d9e425d03b267a26c6de
Reviewed-on: https://go-review.googlesource.com/8681
Reviewed-by: Rob Pike <r@golang.org>

runtime: fix darwin/386, darwin/arm builds

In cl/8652 I broke darwin/arm and darwin/386 because I removed the *g
parameter, which they both expect and use. This CL adjusts both ports
to look for g0 in m, just as darwin/amd64 does.

Tested on darwin{386,arm,amd64}.

Change-Id: Ia56f3d97e126b40d8bbd2e8f677b008e4a1badad
Reviewed-on: https://go-review.googlesource.com/8666
Reviewed-by: Ian Lance Taylor <iant@golang.org>

runtime: use (*context) ip, setip, sp and setsp everywhere on windows

Also move dumpregs into defs_windows_*.go.

Change-Id: Ic077d7dbb133c7b812856e758d696d6fed557afd
Reviewed-on: https://go-review.googlesource.com/4650
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/internal/ld: support for -buildmode=shared

Change-Id: Id4997d611ced29397133f14def6abc88aa9e811e
Reviewed-on: https://go-review.googlesource.com/8252
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/internal/gc, etc: remove dead code

Found with https://github.com/opennota/check.

Change-Id: I50c173382782fb16b15100e02c1c85610bc233a0
Reviewed-on: https://go-review.googlesource.com/7130
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>

cmd/internal/ld: clean up hostlink I/O

Change-Id: I6c3a62403941d357ffd9d0025289c2180139b0bd
Reviewed-on: https://go-review.googlesource.com/8664
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>

runtime: darwin/amd64 library entry point

This is a practice run for darwin/arm.

Similar to the linux/amd64 shared library entry point. With several
pending linker changes I am successfully using this to implement
-buildmode=c-archive on darwin/amd64 with external linking.

The same entry point can be reused to implement -buildmode=c-shared
on darwin/amd64, however that will require further ld changes to
remove all text relocations.

One extra runtime change will follow this. According to the Go
execution modes document, -buildmode=c-archive should ignore the Go
main function. Right now it is being executed (and the process exits
if it doesn't block). I'm still searching for the right way to do
this.

Change-Id: Id97901ddd4d46970996f222bd79731dabff66a3d
Reviewed-on: https://go-review.googlesource.com/8652
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/internal/ld: add -buildmode=c-shared as an alternative to -shared

The linker currently (on some platforms) takes a -shared flag, which means
approximately what -buildmode=c-shared means in the in the proposed "Go
Execution Modes" document. As part of implementing other modes, the term
"shared" becomes horribly overloaded, so this replaces -shared with a
-buildmode argument instead (which currently only handles -buildmode=c-shared
and the default -buildmode=exe -- no new behaviour here).

As the linker support for -shared was in 1.4 this retains it as an alias.

Change-Id: Id2ebb8e05ee07f46208a554bc2622d0e67b47082
Reviewed-on: https://go-review.googlesource.com/8304
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/internal/ld: emit macho .init_array section

Change-Id: Ie75a01e899e68f4f9643410f5e161152a81b8ba0
Reviewed-on: https://go-review.googlesource.com/8655
Reviewed-by: Ian Lance Taylor <iant@golang.org>

encoding/gob: clean up decoderMap after errBadType

When decoding an invalid typeId the associated *decEngine was not
removed from decoderMap. If the decoder was run again on the same input
a nil *decEngine was found in the map and assumed to be initialized,
resulting in a panic.

Fixes #9649

Change-Id: I5bb51808362a21c09228c2705a658f073e5b59b3
Reviewed-on: https://go-review.googlesource.com/3509
Reviewed-by: Rob Pike <r@golang.org>

math/big: make ErrNaN actually implement the error interface (oversight)

There was no way to get to the error message before.

Change-Id: I4aa9d3d9f468c33f9996295bafcbed097de0389f
Reviewed-on: https://go-review.googlesource.com/8660
Reviewed-by: Alan Donovan <adonovan@google.com>

net/rpc: document that the type must be exported, not just the methods

Fixes #10379.

Change-Id: Ia4cdda36ed57a06371f9ace7365ce9e215228487
Reviewed-on: https://go-review.googlesource.com/8654
Reviewed-by: Rob Pike <r@golang.org>

cmd/asm: remove object file if assembly fails.

Just an oversight. Plus the code had an unnecessary call to os.Exit
that now has a purpose.

Fixes #10372.

Change-Id: I456018f3a01ca05b4501c7f8a4961d48ab8c5e16
Reviewed-on: https://go-review.googlesource.com/8651
Reviewed-by: Minux Ma <minux@golang.org>

go/types/internal/gcimporter: update for 7g and 9g

Change-Id: Ied1582d8aabee2eb346e1c23bfd7781e4a091264
Reviewed-on: https://go-review.googlesource.com/8621
Reviewed-by: Robert Griesemer <gri@golang.org>

go/types: more selective disabling of tests

Disable importer-dependent tests on platforms for which the
respective builders don't have access to importable packages.

Fixes #10368.

Change-Id: I8072c59d2bbbc24a43d771fd04fd0b1a678d765a
Reviewed-on: https://go-review.googlesource.com/8611
Reviewed-by: Minux Ma <minux@golang.org>

cmd/internal/ld: change elf64 from int to bool

Change-Id: Iaf2dba7d699a8d52f91ce10222ab0d1a0f1f21fc
Reviewed-on: https://go-review.googlesource.com/8625
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

runtime, cmd/internal/ld: initialize themoduledata slices directly

This CL is quite conservative in some ways.  It continues to define
symbols that have no real purpose (e.g. epclntab).  These could be
deleted if there is no concern that external tools might look for them.

It would also now be possible to make some changes to the pcln data but
I get the impression that would definitely require some thought and
discussion.

Change-Id: Ib33cde07e4ec38ecc1d6c319a10138c9347933a3
Reviewed-on: https://go-review.googlesource.com/7616
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

database/sql: Retry with a fresh connection after maxBadConnRetries

Previously if the connection pool was larger than maxBadConnRetries
and there were a lot of bad connections in the pool (for example if
the database server was restarted), a query might have failed with an
ErrBadConn unnecessarily.  Instead of trying to guess how many times
to retry, try maxBadConnRetries times and then force a fresh
connection to be used for the last attempt.  At the same time, lower
maxBadConnRetries to a smaller value now that it's not that important
to retry so many times from the free connection list.

Fixes #8834

Change-Id: I6542f151a766a658980fb396fa4880ecf5874e3d
Reviewed-on: https://go-review.googlesource.com/2034
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

time: unify formatting of decimals for timestamps

Change function appendUint to appendInt with variable-width 0-padding.

This allows the decimal for the year to be generated without extra code
to handle the wider padding and directly handles negative numbers.

Removes the special casing for numbers with one and two digits.
The special case for 0 was unreachable.

The new version is slightly slower.

benchmark              old ns/op     new ns/op     delta
BenchmarkFormat        444           454           +2.25%
BenchmarkFormatNow     398           415           +4.27%

Change-Id: I4ddef96bf07ad35dca76053321d510441ec6d4f5
Reviewed-on: https://go-review.googlesource.com/2751
Reviewed-by: Robert Griesemer <gri@golang.org>

cmd/7g: reactivate componentgen

Update #10203

When the portable componentgen was introduced in b960263 it
produced broken code on arm64 and was deactivated. In the month since
it looks like the underlying issues have been fixed so componentgen
produces working binaries that are slightly smaller, ~3kb reduction in
size for godoc binary.

Benchmarks are underwhelming, but where visible, trending towards
an improvement (this is with Minux's peep optimiser CL).

benchmark                          old ns/op       new ns/op       delta
BenchmarkBinaryTree17              15336842000     15002766000     -2.18%
BenchmarkFannkuch11                10848984000     10896931000     +0.44%
BenchmarkFmtFprintfEmpty           203             188             -7.39%
BenchmarkFmtFprintfString          753             720             -4.38%
BenchmarkFmtFprintfInt             667             670             +0.45%
BenchmarkFmtFprintfIntInt          1103            1102            -0.09%
BenchmarkFmtFprintfPrefixedInt     981             969             -1.22%
BenchmarkFmtFprintfFloat           1396            1348            -3.44%
BenchmarkFmtManyArgs               4151            4102            -1.18%
BenchmarkGobDecode                 34202360        32933020        -3.71%
BenchmarkGobEncode                 27579180        27438820        -0.51%
BenchmarkGzip                      1296119000      1285096000      -0.85%
BenchmarkGunzip                    291099800       289727200       -0.47%
BenchmarkHTTPClientServer          169476          169803          +0.19%
BenchmarkJSONEncode                70313600        69973400        -0.48%
BenchmarkJSONDecode                227811800       232875200       +2.22%
BenchmarkMandelbrot200             12985600        12996430        +0.08%
BenchmarkGoParse                   14708930        14507320        -1.37%
BenchmarkRegexpMatchEasy0_32       464             460             -0.86%
BenchmarkRegexpMatchEasy0_1K       4516            4517            +0.02%
BenchmarkRegexpMatchEasy1_32       452             454             +0.44%
BenchmarkRegexpMatchEasy1_1K       4664            4669            +0.11%
BenchmarkRegexpMatchMedium_32      602             602             +0.00%
BenchmarkRegexpMatchMedium_1K      172939          172494          -0.26%
BenchmarkRegexpMatchHard_32        9733            9577            -1.60%
BenchmarkRegexpMatchHard_1K        301356          298143          -1.07%
BenchmarkRevcomp                   2754334000      2753874000      -0.02%
BenchmarkTemplate                  315664000       311810800       -1.22%
BenchmarkTimeParse                 1034            989             -4.35%
BenchmarkTimeFormat                1118            1137            +1.70%

benchmark                         old MB/s     new MB/s     speedup
BenchmarkGobDecode                22.44        23.31        1.04x
BenchmarkGobEncode                27.83        27.97        1.01x
BenchmarkGzip                     14.97        15.10        1.01x
BenchmarkGunzip                   66.66        66.98        1.00x
BenchmarkJSONEncode               27.60        27.73        1.00x
BenchmarkJSONDecode               8.52         8.33         0.98x
BenchmarkGoParse                  3.94         3.99         1.01x
BenchmarkRegexpMatchEasy0_32      68.92        69.47        1.01x
BenchmarkRegexpMatchEasy0_1K      226.71       226.65       1.00x
BenchmarkRegexpMatchEasy1_32      70.75        70.42        1.00x
BenchmarkRegexpMatchEasy1_1K      219.55       219.28       1.00x
BenchmarkRegexpMatchMedium_32     1.66         1.66         1.00x
BenchmarkRegexpMatchMedium_1K     5.92         5.94         1.00x
BenchmarkRegexpMatchHard_32       3.29         3.34         1.02x
BenchmarkRegexpMatchHard_1K       3.40         3.43         1.01x
BenchmarkRevcomp                  92.28        92.29        1.00x
BenchmarkTemplate                 6.15         6.22         1.01x

Change-Id: I0b7d95388d6920fcbd7fe305df0c7c630a407726
Reviewed-on: https://go-review.googlesource.com/8636
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/asm/internal/asm: add arm64 end to end tests

Add end to end tests for arm64 to support CL 8405.

There are several instruction forms commented out at the moment
they will be addressed in CL 8405 or later followups.

Change-Id: I6eeeb810c1e03cd49bb3c881bc46a29cdb817822
Reviewed-on: https://go-review.googlesource.com/8631
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Reviewed-by: Rob Pike <r@golang.org>

androidtest.bash: copy pkg for gcimporter tests

The tests for go/types depend on reading gc export data from the
$GOROOT/pkg directory. This is the first use of these files as
testdata, so previously they were not copied to the android device.
Now they are used, copy them.

Fixes android/arm build.

Change-Id: If13bbe603ce0aff697a73a97ae9a7d6b3ea800f9
Reviewed-on: https://go-review.googlesource.com/8624
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

runtime: fix return type for bsdthread_register in comments

The return type for bsdthread_register is int32. See
runtime/os_darwin.go.

This change also rewrites declaration comments for go functions to
use go syntax and fixes vet errors in sys_darwin_amd64.s.

Change-Id: I7482105f7562929e0ede30099efac9e76babd8a3
Reviewed-on: https://go-review.googlesource.com/3260
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>

io: clarify Copy docs regarding error handling

"returns ... the first error" was misleading or at least confusing:
in case a Read results in an error with non-zero bytes read, and the
subsequent Write also results in an error, the error from Write is
returned, which is the second one (in the temporal dimension).

Fixes #9744

Change-Id: If8925a701e4fae820cd9df7446503403fc0785d4
Reviewed-on: https://go-review.googlesource.com/3686
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

doc/go1.5: mention cgo is supported on linux/arm64 (ext. link only)

Change-Id: I1dcca264d9cc900aad2d7737073cc01fe574bf55
Reviewed-on: https://go-review.googlesource.com/8623
Reviewed-by: Minux Ma <minux@golang.org>

go/build: cgo is supported on linux/arm64 (external linking only)

Fixes #10107.

Change-Id: I309e3df7608b9eef9339196fdc50dedf5f9439f6
Reviewed-on: https://go-review.googlesource.com/8453
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>

cmd/internal/ld: force external linking on linux/arm64 with cgo

Update #10373.

Change-Id: I309e3df7608b9eef9339196fdc50dedf5f9439f5
Reviewed-on: https://go-review.googlesource.com/8452
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

misc/cgo/test/issue9400: add arm64 implementation

Change-Id: I309e3df7608b9eef9339196fdc50dedf5f9439f4
Reviewed-on: https://go-review.googlesource.com/8451
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>

runtime: linux/arm64 cgo support

Change-Id: I309e3df7608b9eef9339196fdc50dedf5f9439f3
Reviewed-on: https://go-review.googlesource.com/8450
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>

runtime/cgo: linux/arm64 cgo support

Change-Id: I309e3df7608b9eef9339196fdc50dedf5f9439f2
Reviewed-on: https://go-review.googlesource.com/8439
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>

cmd/internal/obj/arm64, cmd/internal/ld, cmd/7l: remove absolute addressing in .text

This CL introduces R_ADDRARM64, which is similar to R_ADDRPOWER.

Fixes #10112.

Change-Id: I309e3df7608b9eef9339196fdc50dedf5f9439f1
Reviewed-on: https://go-review.googlesource.com/8438
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/internal/ld, cmd/7l: external linking support for linux/arm64

Based on Michael Hudson-Doyle's patch:
https://github.com/4ad/go/commit/b735215ee41b4237ec393d4669f6f55d4b27babc

Change-Id: I309e3df7608b9eef9339196fdc50dedf5f9439f0
Reviewed-on: https://go-review.googlesource.com/8437
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

cmd/api: make the test more robust

Previously, the TestCompareAPI test would fail if runtime.Version()
is "dev", or, more importantly, "go1.5"; because compareAPI depends
on runtime.Version and -allow_new flag. Move that logic out make
its test more robust.

Change-Id: I8f40daa1838b8acd26adac8848762d95315053b0
Reviewed-on: https://go-review.googlesource.com/8622
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/7g: enable peephole optimizer

Based on cmd/9g/peep.go.

Go 1 benchmark comparison:
benchmark                          old ns/op       new ns/op       delta
BenchmarkBinaryTree17              24328574000     18351639000     -24.57%
BenchmarkFannkuch11                17029365000     10817758000     -36.48%
BenchmarkFmtFprintfEmpty           291             223             -23.37%
BenchmarkFmtFprintfString          1073            799             -25.54%
BenchmarkFmtFprintfInt             1024            778             -24.02%
BenchmarkFmtFprintfIntInt          1654            1277            -22.79%
BenchmarkFmtFprintfPrefixedInt     1360            1083            -20.37%
BenchmarkFmtFprintfFloat           2272            1415            -37.72%
BenchmarkFmtManyArgs               5933            4742            -20.07%
BenchmarkGobDecode                 53166003        38584736        -27.43%
BenchmarkGobEncode                 37930156        30074874        -20.71%
BenchmarkGzip                      1880638900      1286832100      -31.57%
BenchmarkGunzip                    386343633       292194480       -24.37%
BenchmarkHTTPClientServer          237077          179776          -24.17%
BenchmarkJSONEncode                101731690       73116925        -28.13%
BenchmarkJSONDecode                344655360       241277600       -29.99%
BenchmarkMandelbrot200             28329778        12950809        -54.29%
BenchmarkGoParse                   21670755        16554244        -23.61%
BenchmarkRegexpMatchEasy0_32       557             484             -13.11%
BenchmarkRegexpMatchEasy0_1K       4687            4832            +3.09%
BenchmarkRegexpMatchEasy1_32       539             483             -10.39%
BenchmarkRegexpMatchEasy1_1K       5100            5080            -0.39%
BenchmarkRegexpMatchMedium_32      796             651             -18.22%
BenchmarkRegexpMatchMedium_1K      233099          182047          -21.90%
BenchmarkRegexpMatchHard_32        13202           9897            -25.03%
BenchmarkRegexpMatchHard_1K        401027          303602          -24.29%
BenchmarkRevcomp                   3837679666      2816546600      -26.61%
BenchmarkTemplate                  440608300       324831040       -26.28%
BenchmarkTimeParse                 1460            1019            -30.21%
BenchmarkTimeFormat                1609            1174            -27.04%

benchmark                         old MB/s     new MB/s     speedup
BenchmarkGobDecode                14.44        19.89        1.38x
BenchmarkGobEncode                20.24        25.52        1.26x
BenchmarkGzip                     10.32        15.08        1.46x
BenchmarkGunzip                   50.23        66.41        1.32x
BenchmarkJSONEncode               19.07        26.54        1.39x
BenchmarkJSONDecode               5.63         8.04         1.43x
BenchmarkGoParse                  2.67         3.50         1.31x
BenchmarkRegexpMatchEasy0_32      57.38        66.05        1.15x
BenchmarkRegexpMatchEasy0_1K      218.47       211.91       0.97x
BenchmarkRegexpMatchEasy1_32      59.29        66.21        1.12x
BenchmarkRegexpMatchEasy1_1K      200.76       201.54       1.00x
BenchmarkRegexpMatchMedium_32     1.26         1.53         1.21x
BenchmarkRegexpMatchMedium_1K     4.39         5.62         1.28x
BenchmarkRegexpMatchHard_32       2.42         3.23         1.33x
BenchmarkRegexpMatchHard_1K       2.55         3.37         1.32x
BenchmarkRevcomp                  66.23        90.24        1.36x
BenchmarkTemplate                 4.40         5.97         1.36x

Fixes #10105.

Change-Id: I353cc9fdf922e431821508c9dbbe4d9a85d64bd4
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/8471
Reviewed-by: Dave Cheney <dave@cheney.net>

cmd/api: re-enable TestGolden on nacl

Fixes #10369.

Change-Id: If0a6d2b33c6862c9f7f862bdc997f2204072c6dc
Reviewed-on: https://go-review.googlesource.com/8620
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/asm/internal/asm: parse arm64 register pairs for LDP/STP

Add test, and while we're at here, also add a test for ARM.

Fixes #10343.

Change-Id: Ic914df8233d4f1f495e2cc0743fbd37b7671bc91
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/8472
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Reviewed-by: Rob Pike <r@golang.org>

go/types: enable disabled test

Change-Id: I58de76c49de6e43befb30b2bf677934e7952b5ab
Reviewed-on: https://go-review.googlesource.com/8610
Reviewed-by: Rob Pike <r@golang.org>

cmd/api: update api checker to use go/types from std repo

The old code checked out a specific version of go/types from the
x/tools repo. With go/types being part of the std repo, this is
not necessary anymore.

Also, for the same reason, the api tool is now built like any
other regular command. There's no need to build it for each run.
Removed the respective +build tags.

Change-Id: I5088e4867223d676957084c24651ec05452ac495
Reviewed-on: https://go-review.googlesource.com/8564
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

go/types: skip failing tests (fix build)

Temporary work-around so we can start using go/types in the std repo.

Change-Id: I661465af791171b01cd23abf34dcb7eea6e26173
Reviewed-on: https://go-review.googlesource.com/8594
Reviewed-by: Rob Pike <r@golang.org>

go/types, go/exact: "vendor" go/types into std repo

This is a first step towards moving go/types from the tools
repo into the std repo. The files were brought over via the
added src/go/types.bash script for reproducability. The
script can be removed once all dependencies on go/types
have moved to the std repo go/types.

The script moved packages as follows:

- x/tools/go/types => go/types (type-checker)
- x/tools/go/exact => go/exact (constants)
- x/tools/go/gcimporter => go/types/internal/gcimporter

The gcimporter is needed to be able to run tests. go/types
should probably have some factory function to provide an
appropriate importer.

Some of the go/types tests fail for a handful of platforms
(windows and nacl). In order to keep this change "clean"
from manual changes, the next change will disable those
tests for now so we can move forward.

Change-Id: I448d8f7faa39ad2e04811911b699f7682627c224
Reviewed-on: https://go-review.googlesource.com/8530
Reviewed-by: Rob Pike <r@golang.org>

image/color: have CMYK.RGBA work in 16-bit color, per the Color interface.

Change-Id: I3621527c924a43724032f80a072505c60d929ab3
Reviewed-on: https://go-review.googlesource.com/8180
Reviewed-by: Rob Pike <r@golang.org>

all: fix race when allocating buffer for some windows syscalls

Fixes #9753

Change-Id: I6c641ed7ef4f687a108e7d937ab4b9c24d5baf5d
Reviewed-on: https://go-review.googlesource.com/4940
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

doc: update go1.5.txt (gc constant arithmetic now based on math/big)

Change-Id: Iff943d15e83e7db1f3c77f509a60e00ee2041d69
Reviewed-on: https://go-review.googlesource.com/8592
Reviewed-by: Robert Griesemer <gri@golang.org>

cmd/internal/gc: use approx. float formatting in error messages

For very out-of-range floating-point constants (1e100000000),
precise formatting of the offending value for error messages
is not needed and potentially extremely slow.

This change resurrects an adjusted variant of the original code
which uses float64 formatting in the common case (in-range values),
and a fast manual approximation for out-of-range values.

Change-Id: I2f6e53040929b8bf924dac4bb27c4d811ede48e2
Reviewed-on: https://go-review.googlesource.com/8470
Reviewed-by: Alan Donovan <adonovan@google.com>

cmd/go: re-run mkdoc.sh

Adds $DOLLAR to doc.go

Change-Id: Ia0ccdd9b5292010297ae95debab4acf50d3c4d9c
Reviewed-on: https://go-review.googlesource.com/8600
Reviewed-by: Ian Lance Taylor <iant@golang.org>

math/big: fix Exp when exponent is 1

Fixed bug that caused Exp(x, y, m) ( i.e. x**y (mod m) ) to return x
instead of x (mod m) when y == 1. See issue page on github for more
details.

Added test case

Fixes #9826

Change-Id: Ibabb58275a20c4231c9474199b7f1c10e54241ce
Reviewed-on: https://go-review.googlesource.com/8409
Reviewed-by: Robert Griesemer <gri@golang.org>

encoding/gob: more cleanups handling slice length

Fix the other places the slice length was being believed, and refactor
the code to use a single function to unify the check.

Fixes #10273.

Change-Id: Ia62b25203fbe87c95d71a70ebc1db8d202eaa4a4
Reviewed-on: https://go-review.googlesource.com/8511
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/internal/gc: fix race build (again)

Add OGETG to the list of ignored operations.

We don't instrument the runtime package, but calls to runtime.getg
can appear in other packages, for example, after inlining
runtime.LockOSThread.

Change-Id: I8d6e91f1f3c8fd1302b596bdead42d588c059911
Reviewed-on: https://go-review.googlesource.com/8553
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>

cmd/internal/gc: fix race build

Add special case for OGETG which should never be instrumented.

Change-Id: I7d082abb8608537f82b03362b687baf2a1d809dc
Reviewed-on: https://go-review.googlesource.com/8551
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>

cmd/internal/gc: inline runtime.getg

This more closely restores what the old C runtime did.
(In C, g was an 'extern register' with the same effective
implementation as in this CL.)

On a late 2012 MacBookPro10,2, best of 5 old vs best of 5 new:

benchmark                          old ns/op      new ns/op      delta
BenchmarkBinaryTree17              4981312777     4463426605     -10.40%
BenchmarkFannkuch11                3046495712     3006819428     -1.30%
BenchmarkFmtFprintfEmpty           89.3           79.8           -10.64%
BenchmarkFmtFprintfString          284            262            -7.75%
BenchmarkFmtFprintfInt             282            262            -7.09%
BenchmarkFmtFprintfIntInt          480            448            -6.67%
BenchmarkFmtFprintfPrefixedInt     382            358            -6.28%
BenchmarkFmtFprintfFloat           529            486            -8.13%
BenchmarkFmtManyArgs               1849           1773           -4.11%
BenchmarkGobDecode                 12835963       11794385       -8.11%
BenchmarkGobEncode                 10527170       10288422       -2.27%
BenchmarkGzip                      436109569      438422516      +0.53%
BenchmarkGunzip                    110121663      109843648      -0.25%
BenchmarkHTTPClientServer          81930          85446          +4.29%
BenchmarkJSONEncode                24638574       24280603       -1.45%
BenchmarkJSONDecode                93022423       85753546       -7.81%
BenchmarkMandelbrot200             4703899        4735407        +0.67%
BenchmarkGoParse                   5319853        5086843        -4.38%
BenchmarkRegexpMatchEasy0_32       151            151            +0.00%
BenchmarkRegexpMatchEasy0_1K       452            453            +0.22%
BenchmarkRegexpMatchEasy1_32       131            132            +0.76%
BenchmarkRegexpMatchEasy1_1K       761            722            -5.12%
BenchmarkRegexpMatchMedium_32      228            224            -1.75%
BenchmarkRegexpMatchMedium_1K      63751          64296          +0.85%
BenchmarkRegexpMatchHard_32        3188           3238           +1.57%
BenchmarkRegexpMatchHard_1K        95396          96756          +1.43%
BenchmarkRevcomp                   661587262      687107364      +3.86%
BenchmarkTemplate                  108312598      104008540      -3.97%
BenchmarkTimeParse                 453            459            +1.32%
BenchmarkTimeFormat                475            441            -7.16%

The garbage benchmark from the benchmarks subrepo gets 2.6% faster as well.

Change-Id: I320aeda332db81012688b26ffab23f6581c59cfa
Reviewed-on: https://go-review.googlesource.com/8460
Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>

net/url: allow Parse, ParseRequestURI to parse ipv6 zone identifiers in URIs

Using IPv6 link-local addresses to make connections between on-link
nodes is useful for small distributed applications but it requires zone
identifiers to distinguish a correct IP link. It's the same for
transports using URI for destination discovery such as HTTP, WebSocket.

This change allows Parse, ParseRequestURI functions and String method of
URL to parse/return a literal IPv6 address followed by a zone identifier
within a URI as described in RFC 6874.

Fixes #6530.

Change-Id: I2936ea65c1446994770cf2ee2c28a1c73faaa0ca
Reviewed-on: https://go-review.googlesource.com/2431
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

encoding/gob: make method doc string match method name

Change-Id: I6859bd9c9dba30fc5eeb9bbc1de90af67984944c
Reviewed-on: https://go-review.googlesource.com/8526
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

net: deflake dual IP stack tests

This change deflakes TestDialerDualStackFDLeak, TestDialerDualStack,
TestResolve{TCP,UDP,IP}Addr by removing external dependencies.

Fixes #8764.

Change-Id: I5cca0a93776cf05652e0e6a4a4ff4af392ccb885
Reviewed-on: https://go-review.googlesource.com/8485
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Revert "misc/cgo/testso: add test for fixed issue 4339"

This reverts commit 9fa9f966e99eb514f9b6f61821f83d5f9178a4a2.

The change has broken darwin and netbsd builders. It needs to be tested properly.

Change-Id: Id9e2d30caa8764c362c9f33890015dfc1aae0dab
Reviewed-on: https://go-review.googlesource.com/8527
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>

doc: add mime/quotedprintable to go1.5.txt

Change-Id: Iace8941c947253b1141f4194c5d2010c420ec220
Reviewed-on: https://go-review.googlesource.com/8540
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

database/sql: close connection if db.numOpen > db.maxOpen

Bug Description:
When reduce db.maxOpen via db.SetMaxOpenConns, the unnecssary
connections won't been released until all other connections are free.

Fixes #9453

Change-Id: I9afb2e4b184139b31029ae53d7f5fd1fdb8d8d7e
Reviewed-on: https://go-review.googlesource.com/2200
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

cmd/dist: add misc/cgo/test with -linkmode=auto

Change-Id: I5939524f75f8fbbd67bd54b7c9e4b8c162337e9d
Reviewed-on: https://go-review.googlesource.com/8525
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

misc/cgo/testso: add test for fixed issue 4339

Update #4339.

Change-Id: Ic1a7535562b8b824ba166777725f7ba5b9623d77
Reviewed-on: https://go-review.googlesource.com/8523
Run-TryBot: Minux Ma <minux@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

net: fix tester goroutine leakage in tests

This change tries to stop various tester goroutines at the end of each
scope for avoiding interference between test cases including benchmarks.
Not yet finished completely but enough to land upcoming changes to Dial
functions. The rest will be fixed later.

Change-Id: Ic38b8681a3a2ddbcd69ba3696f24a61d418a0346
Reviewed-on: https://go-review.googlesource.com/8398
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/internal/gc: convert yet more Node fields to bools

Convert Embedded, Method, and Colas to bools.

I believe that this is the last of the Node fields
that can be trivially converted to bools.

No functional changes. Passes toolstash -cmp.

Change-Id: I81962ee47866596341fc60d24d6959c20cd7fc1c
Reviewed-on: https://go-review.googlesource.com/8440
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

net: move testHookSetKeepAlive into hook.go

Change-Id: I1f2d4e3b0351a7a47c3a6073833a17dbc0c7b05c
Reviewed-on: https://go-review.googlesource.com/8520
Reviewed-by: Ian Lance Taylor <iant@golang.org>

net: add lookup IP hook for testing

This change adds testHookLookIP to enable to inject DNS name to IP
address mappings for Happ{y,yish,ier} Eyeballs dial testing.

Change-Id: I8ac04a594e1e2bd77909528df0552889914a7790
Reviewed-on: https://go-review.googlesource.com/8399
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/internal/ld: remove extraneous parentheses

Ian complained about these in a review and then submitted the change
before I could fix them.

Change-Id: I23d890db2f3648ed1003ed3d13e7247435b913e5
Reviewed-on: https://go-review.googlesource.com/8480
Reviewed-by: Ian Lance Taylor <iant@golang.org>

doc/progs: rewrite test driver in Go

The tests in doc/progs appear to have been originally written
for use with the old test driver. At some later point,
they acquired their own test driver.
Both ran tests in serial.

This CL rewrites the current test driver in Go,
runs tests concurrently, and cleans up
historical artifacts from the old drivers.

The primary motivation is to speed up all.bash.
On my laptop, using tip, this CL reduces doc/progs test
wall time from 26s to 7s. The savings will remain
even when the compiler gets faster. Using Go 1.4,
this CL reduces test wall time from 15s to 4s.

Change-Id: Iae945a8490222beee76e8a2118a0d7956092f543
Reviewed-on: https://go-review.googlesource.com/8410
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

cmd/internal/obj: remove dead fields and code

A quick pass through link.go, mostly removing fields that are not
used on the "creating a single object file" side of the fence.

Change-Id: I35ba41378c2c418f7df2f2f88dce65bc64a1a45d
Reviewed-on: https://go-review.googlesource.com/7672
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/internal/gc, cmd/yacc: implement "expecting" syntax error messages

Bison includes suggestions about what tokens are expected in the
current state when there's only four or fewer of them.  For example:

  syntax error: unexpected literal 2.01, expecting semicolon or newline or }

This CL adds the same functionality to cmd/yacc, which fully restores
the previous error message behavior from Go 1.4.

Updates #9968.

Change-Id: I2c1a1677c6d829a829d812c05e8813aa8829d09c
Reviewed-on: https://go-review.googlesource.com/8494
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

doc/go1.5.txt: add Reader.Size to bytes and strings

Change-Id: Idd42e0f5c6ed55be2e153ac83022439e5272c1a7
Reviewed-on: https://go-review.googlesource.com/8444
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

runtime: add _rt0_arm_android_lib

At the moment this function does nothing, runtime initialization is
still done in android.c:init_go_runtime.

Fixes #10358

Change-Id: I1d762383ba61efcbcf0bbc7c77895f5c1dbf8968
Reviewed-on: https://go-review.googlesource.com/8510
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>

encoding/gob: change panic into error for corrupt input

decBuffer.Drop is called using data provided by the user, don't
panic if it's bogus.

Fixes #10272.

Change-Id: I913ae9c3c45cef509f2b8eb02d1efa87fbd52afa
Reviewed-on: https://go-review.googlesource.com/8496
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

runtime: report marked heap size in gctrace

When the gctrace GODEBUG option is enabled, it will now report three
heap sizes: the heap size at the beginning of the GC cycle, the heap
size at the end of the GC cycle before sweeping, and marked heap size,
which is the amount of heap that will be retained until the next GC
cycle.

Change-Id: Ie13f8a6d5c609bc9cc47c7555960ab55b37b5f1c
Reviewed-on: https://go-review.googlesource.com/8430
Reviewed-by: Rick Hudson <rlh@golang.org>

runtime: make next_gc be heap size to trigger GC at

In the STW collector, next_gc was both the heap size to trigger GC at
as well as the goal heap size.

Early in the concurrent collector's development, next_gc was the goal
heap size, but was also used as the heap size to trigger GC at. This
meant we always overshot the goal because of allocation during
concurrent GC.

Currently, next_gc is still the goal heap size, but we trigger
concurrent GC at 7/8*GOGC heap growth. This complicates
shouldtriggergc, but was necessary because of the incremental
maintenance of next_gc.

Now we simply compute next_gc for the next cycle during mark
termination. Hence, it's now easy to take the simpler route and
redefine next_gc as the heap size at which the next GC triggers. We
can directly compute this with the 7/8 backoff during mark termination
and shouldtriggergc can simply test if the live heap size has grown
over the next_gc trigger.

This will also simplify later changes once we start setting next_gc in
more sophisticated ways.

Change-Id: I872be4ae06b4f7a0d7f7967360a054bd36b90eea
Reviewed-on: https://go-review.googlesource.com/8420
Reviewed-by: Russ Cox <rsc@golang.org>

runtime: introduce heap_live; replace use of heap_alloc in GC

Currently there are two main consumers of memstats.heap_alloc:
updatememstats (aka ReadMemStats) and shouldtriggergc.

updatememstats recomputes heap_alloc from the ground up, so we don't
need to keep heap_alloc up to date for it. shouldtriggergc wants to
know how many bytes were marked by the previous GC plus how many bytes
have been allocated since then, but this *isn't* what heap_alloc
tracks. heap_alloc also includes objects that are not marked and
haven't yet been swept.

Introduce a new memstat called heap_live that actually tracks what
shouldtriggergc wants to know and stop keeping heap_alloc up to date.

Unlike heap_alloc, heap_live follows a simple sawtooth that drops
during each mark termination and increases monotonically between GCs.
heap_alloc, on the other hand, has much more complicated behavior: it
may drop during sweep termination, slowly decreases from background
sweeping between GCs, is roughly unaffected by allocation as long as
there are unswept spans (because we sweep and allocate at the same
rate), and may go up after background sweeping is done depending on
the GC trigger.

heap_live simplifies computing next_gc and using it to figure out when
to trigger garbage collection. Currently, we guess next_gc at the end
of a cycle and update it as we sweep and get a better idea of how much
heap was marked. Now, since we're directly tracking how much heap is
marked, we can directly compute next_gc.

This also corrects bugs that could cause us to trigger GC early.
Currently, in any case where sweep termination actually finds spans to
sweep, heap_alloc is an overestimation of live heap, so we'll trigger
GC too early. heap_live, on the other hand, is unaffected by sweeping.

Change-Id: I1f96807b6ed60d4156e8173a8e68745ffc742388
Reviewed-on: https://go-review.googlesource.com/8389
Reviewed-by: Russ Cox <rsc@golang.org>

runtime: track heap bytes marked by GC

This tracks the number of heap bytes marked by a GC cycle. We'll use
this information to precisely trigger the next GC cycle.

Currently this aggregates the work counter in gcWork and dispose
atomically aggregates this into a global work counter. dispose happens
relatively infrequently, so the contention on the global counter
should be low. If this turns out to be an issue, we can reduce the
number of disposes, and if it's still a problem, we can switch to
per-P counters.

Change-Id: I1bc377cb2e802ef61c2968602b63146d52e7f5db
Reviewed-on: https://go-review.googlesource.com/8388
Reviewed-by: Russ Cox <rsc@golang.org>

cmd/asm/internal/asm: fix comment in ppc64.go

It referred to the wrong architecture.

Fixes #10355.

Change-Id: I5b9d31c9f04f3106b93f94fa68c848b2518b128e
Reviewed-on: https://go-review.googlesource.com/8495
Reviewed-by: Dave Cheney <dave@cheney.net>

cmd/internal/gc/big: update vendored version of math/big

This fixes the formerly extremely slow conversion of floating-point
constants with large exponents (e.g., "const c = 1e1000000000" could
stall the machine).

Change-Id: I36e02158e3334d32b18743ec0c259fec77baa74f
Reviewed-on: https://go-review.googlesource.com/8466
Reviewed-by: Alan Donovan <adonovan@google.com>

net/http, math/big, cmd/internal/gc/big: replaced errors.New(fmt.Sprintf(...)) in favour fmt.Errorf()

Change-Id: I38fc0ab84a374cb9be0234e40665d7cea0e76fc1
Reviewed-on: https://go-review.googlesource.com/8402
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

bytes, strings: add Reader.Size methods

As noted on recently on golang-nuts, there's currently no way to know
the total size of a strings.Reader or bytes.Reader when using ReadAt
on them. Most callers resort to wrapping it in an io.SectionReader to
retain that information.

The SizeReaderAt abstraction (an io.ReaderAt with a Size() int64
method) has proven useful as a way of expressing a concurrency-safe
read-only number of bytes.

As one example, see http://talks.golang.org/2013/oscon-dl.slide#49 and
the rest of that presentation for its use in dl.google.com.

SizeReaderAt is also used in the open source google-api-go-client, and
within Google's internal codebase, where it exists in a public package
created in 2013 with the package comment: "These may migrate to the
standard library after we have enough experience with their feel."

I'm still as happy with the SizeReaderAt abstraction and its
composabilty as I was in 2013, so I'd like to make these two Readers
also be SizeReaderAts.

Fixes #9667

Change-Id: Ie6f145ada419dd116280472d8c029f046d5edf70
Reviewed-on: https://go-review.googlesource.com/3199
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

net: dialChannel should not treat an expired deadline as noDeadline.

Now, only a zero deadline is interpreted as noDeadline.  Any other time
in the past yields an immediate timeout.

TestConnectDeadlineInThePast already covers this case.  We just need to
un-skip it for plan9, where dialChannel is used.

Change-Id: I995fd1a632c31f8004dac772c3d7c43a2a5853b0
Reviewed-on: https://go-review.googlesource.com/8435
Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>

bytes: fix comment

The the has been deleted.

Change-Id: I4290105435d4f1fd10c7014f913a3147ddeb3c2b
Reviewed-on: https://go-review.googlesource.com/8469
Reviewed-by: Ian Lance Taylor <iant@golang.org>

cmd/yacc: fix default action

Previously, a production rule like

A: B C D

would cause yacc to check that A and B have the same declared types,
but then it would generate an implicit action of { $$ = $3 } (i.e.,
copy the value from D), even if A and D have different types.

Fixes #10192.

Change-Id: I51cfd7baa0011557141dca33b7af1d892cc6f49e
Reviewed-on: https://go-review.googlesource.com/7780
Reviewed-by: Russ Cox <rsc@golang.org>