Austin Clements [Sat, 28 May 2016 01:04:40 +0000 (21:04 -0400)]
runtime: bound scanobject to ~100 µs
Currently the time spent in scanobject is proportional to the size of
the object being scanned. Since scanobject is non-preemptible, large
objects can cause significant goroutine (and even whole application)
delays through several means:
1. If a GC assist picks up a large object, the allocating goroutine is
blocked for the whole scan, even if that scan well exceeds that
goroutine's debt.
2. Since the scheduler does not run on the P performing a large object
scan, goroutines in that P's run queue do not run unless they are
stolen by another P (which can take some time). If there are a few
large objects, all of the Ps may get tied up so the scheduler
doesn't run anywhere.
3. Even if a large object is scanned by a background worker and other
Ps are still running the scheduler, the large object scan doesn't
flush background credit until the whole scan is done. This can
easily cause all allocations to block in assists, waiting for
credit, causing an effective STW.
Fix this by splitting large objects into 128 KB "oblets" and scanning
at most one oblet at a time. Since we can scan 1–2 MB/ms, this equates
to bounding scanobject at roughly 100 µs. This improves assist
behavior both because assists can no longer get "unlucky" and be stuck
scanning a large object, and because it causes the background worker
to flush credit and unblock assists more frequently when scanning
large objects. This also improves GC parallelism if the heap consists
primarily of a small number of very large objects by letting multiple
workers scan a large objects in parallel.
Fixes #10345. Fixes #16293.
This substantially improves goroutine latency in the benchmark from
issue #16293, which exercises several forms of very large objects:
Commit 59877bf renamed bitMarked to bitScan, since the bitmap is no
longer used for marking. However, there were several other references
to this strewn about comments and in some other constant names. Fix
these up, too.
David Crawshaw [Fri, 26 Aug 2016 00:29:15 +0000 (20:29 -0400)]
cmd/compile: generate table of main symbol types
For each exported symbol in package main, add its name and type to
go.plugin.tabs symbol. This is used by the runtime when loading a
plugin to return a typed interface{} value.
Change-Id: I23c39583e57180acb8f7a74d218dae4368614f46
Reviewed-on: https://go-review.googlesource.com/27818
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is caused by sqrtsd preserving upper part of destination register.
Which introduces dependency on previous value of X0.
In 1.6 benchmark loop didn't use X0 immediately after call:
Change-Id: I98aa203176f8f2ca2fcca6e334a65bc60d6f824d
Reviewed-on: https://go-review.googlesource.com/28535 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Caio Marcelo de Oliveira Filho [Mon, 5 Sep 2016 18:14:55 +0000 (15:14 -0300)]
cmd/go: use httpGET helper in bug command
Use existing helper function instead of importing "net/http". This
allows the go_bootstrap build to not depend on "net/http" package.
See cmd/go/http.go for details.
The ctxt parameter is always set to 0 on entry into anylit so make this
parameter a literal constant, and where possibly remove ctxt as a parameter
where it is known to be a constant zero.
Passes toolstash -cmp.
This is a re-creation of CL 28221 by Dave Cheney.
That CL was graciously reverted in CL 28480
to make merging other CLs easier.
Dave Cheney [Wed, 31 Aug 2016 04:39:41 +0000 (14:39 +1000)]
cmd/compile/internal/gc: clean up sinit.go
The ctxt parameter is always set to 0 on entry into anylit so make this
parameter a literal constant, and where possibly remove ctxt as a parameter
where it is known to be a constant zero.
net/http/httputil: make ReverseProxy send nil Body requests when possible
The http.Transport's retry can't retry requests with non-nil
bodies. When cloning an incoming server request into an outgoing
client request, nil out the Body field if the ContentLength is 0. (For
server requests, Body is always non-nil, even for GET, HEAD, etc.)
Also, don't use the deprecated CancelRequest and use Context instead.
And don't set Proto, ProtoMajor, ProtoMinor. Those are ignored in
client requests, which was probably a later documentation
clarification.
Fixes #16036
Updates #16696 (remove useless Proto lines)
Joe Tsai [Fri, 2 Sep 2016 21:37:35 +0000 (14:37 -0700)]
archive/tar: reapply Header.Size to regFileReader after merging
The use of PAX headers can modify the overall file size, thus the
formerly created regFileReader may be stale.
The relevant PAX specification for this behavior is:
<<<
Any fields in the preceding optional extended header shall override
the associated fields in this header block for this file.
>>>
Where "optional extended header" refers to the preceding PAX header.
Where "this header block" refers to the subsequent USTAR header.
Dmitry Vyukov [Mon, 8 Aug 2016 14:17:14 +0000 (16:17 +0200)]
runtime: fix global buffer reset in StopTrace
We reset global buffer only if its pos != 0.
We ought to do it always, but queue it only if pos != 0.
This is a latent bug. Currently it does not fire because
whenever we create a global buffer, we increment pos.
runtime: improve message when a bad pointer is found on the stack
Currently this message says "invalid stack pointer", which could be
interpreted as the value of SP being invalid. Change it to "invalid
pointer found on stack" to emphasize that it's a pointer on the stack
that's invalid.
Adam Langley [Thu, 1 Sep 2016 23:00:25 +0000 (16:00 -0700)]
crypto/cipher: enforce message size limits for GCM.
The maximum input plaintext for GCM is 64GiB - 64. Since the GCM
interface is one-shot, it's very hard to hit this in Go (one would need
a 64GiB buffer in memory), but we should still enforce this limit.
Thanks to Quan Nguyen for pointing it out.
Change-Id: Icced47bf8d4d5dfbefa165cf13e893205c9577b8
Reviewed-on: https://go-review.googlesource.com/28410
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
Sina Siadat [Sat, 27 Aug 2016 16:16:25 +0000 (20:46 +0430)]
net/http/httputil: remove proxied headers mentioned in connection-tokens
RFC 2616, section 14.10 says:
>>>
HTTP/1.1 proxies MUST parse the Connection header field before a message
is forwarded and, for each connection-token in this field, remove any
header field(s) from the message with the same name as the
connection-token. Connection options are signaled by the presence of a
connection-token in the Connection header field, not by any
corresponding additional header field(s), since the additional header
field may not be sent if there are no parameters associated with that
connection option.
<<<
The same requirement was included in RFC 7230, section 6.1.
David Glasser [Thu, 4 Aug 2016 23:49:05 +0000 (16:49 -0700)]
math/rand: document that NewSource sources race
While it was previously explicitly documented that "the default Source"
is safe for concurrent use, a careless reader can interpret that as
meaning "the implementation of the Source interface created by functions
in this package" rather than "the default shared Source used by
top-level functions". Be explicit that the Source returned by NewSource
is not safe for use by multiple goroutines.
Matt Layher [Thu, 1 Sep 2016 20:30:15 +0000 (16:30 -0400)]
net/http: omit Content-Length in Response.Write for 1xx or 204 status
Per RFC 7230, Section 3.3.2: "A server MUST NOT send a Content-Length
header field in any response with a status code of 1xx (Informational)
or 204 (No Content).".
Martin Möhrmann [Thu, 1 Sep 2016 06:31:37 +0000 (08:31 +0200)]
cmd/compile: disallow typed non-integer constant len and cap make arguments
make(T, n, m) returns a slice of type T with length n and capacity m
where "The size arguments n and m must be of integer type or untyped."
https://tip.golang.org/ref/spec#Making_slices_maps_and_channels
The failure to reject typed non-integer size arguments in make
during compile time was uncovered after https://golang.org/cl/27851
changed the generation of makeslice calls.
Change-Id: I63af8f0a19ec91f4a2001aa7a2eadcd2232a47df
Reviewed-on: https://go-review.googlesource.com/28348 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Thu, 18 Aug 2016 20:14:30 +0000 (13:14 -0700)]
spec: update language on type switches to match implementations
See the issue below for details.
Fixes #16794.
Change-Id: I7e338089fd80ddcb634fa80bfc658dee2772361c
Reviewed-on: https://go-review.googlesource.com/27356 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Brad Fitzpatrick [Tue, 30 Aug 2016 03:19:01 +0000 (03:19 +0000)]
crypto/tls: add Config.Clone
In Go 1.0, the Config struct consisted only of exported fields.
In Go 1.1, it started to grow private, uncopyable fields (sync.Once,
sync.Mutex, etc).
Ever since, people have been writing their own private Config.Clone
methods, or risking it and doing a language-level shallow copy and
copying the unexported sync variables.
Clean this up and export the Config.clone method as Config.Clone.
This matches the convention of Template.Clone from text/template and
html/template at least.
Fixes #15771
Updates #16228 (needs update in x/net/http2 before fixed)
Updates #16492 (not sure whether @agl wants to do more)
Joe Tsai [Mon, 29 Aug 2016 23:10:32 +0000 (16:10 -0700)]
archive/tar: make Reader error handling consistent
The tar.Reader guarantees stickiness of errors. Ensuring this property means
that the methods of Reader need to be consistent about whose responsibility it
is to actually ensure that errors are sticky.
In this CL, we make it only the responsibility of the exported methods
(Next and Read) to store tr.err. All other methods just return the error as is.
As part of this change, we also check the error value of mergePAX (and test
that it properly detects invalid PAX files). Since the value of mergePAX was
never used before, we change it such that it always returns ErrHeader instead
of strconv.SyntaxError. This keeps it consistent with other usages of strconv
in the same tar package.
Rob Pike [Sat, 27 Aug 2016 02:09:38 +0000 (12:09 +1000)]
doc/faq: explain the meaning of "runtime"
This truly is a common point of confusion that deserves
explanation in the FAQ.
Change-Id: Ie624e31a2042ca99626fe7570d9c8c075aae6a84
Reviewed-on: https://go-review.googlesource.com/28275 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Matthew Dempsky [Tue, 30 Aug 2016 23:31:53 +0000 (16:31 -0700)]
cmd/compile: handle pragmas immediately with -newparser=1
Instead of saving all pragmas and processing them after parsing is
finished, process them immediately during scanning like the current
lexer does.
This is a bit unfortunate because it means we can't use
syntax.ParseFile to concurrently parse files yet, but it fixes how we
report syntax errors in the presence of //line pragmas.
While here, add a bunch more gcCompat entries to syntax/parser.go to
get "go build -toolexec='toolstash -cmp' std cmd" passing. There are
still a few remaining cases only triggered building unit tests, but
this seems like a nice checkpoint.
Change-Id: Iaf3bbcf2849857a460496f31eea228e0c585ce13
Reviewed-on: https://go-review.googlesource.com/28226
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Radu Berinde [Sun, 28 Aug 2016 18:36:06 +0000 (14:36 -0400)]
hash/crc32: cleanup code and improve tests
Major reorganization of the crc32 code:
- The arch-specific files now implement a well-defined interface
(documented in crc32.go). They no longer have the responsibility of
initializing and falling back to a non-accelerated implementation;
instead, that happens in the higher level code.
- The non-accelerated algorithms are moved to a separate file with no
dependencies on other code.
- The "cutoff" optimization for slicing-by-8 is moved inside the
algorithm itself (as opposed to every callsite).
Tests are significantly improved:
- direct tests for the non-accelerated algorithms.
- "cross-check" tests for arch-specific implementations (all archs).
- tests for misaligned buffers for both IEEE and Castagnoli.
Fixes #16909.
Change-Id: I9b6dd83b7a57cd615eae901c0a6d61c6b8091c74
Reviewed-on: https://go-review.googlesource.com/27935 Reviewed-by: Keith Randall <khr@golang.org>
Alex Brainman [Wed, 10 Aug 2016 01:06:46 +0000 (11:06 +1000)]
cmd/link, cmd/go: delay linking of mingwex and mingw32 until very end
cmd/go links mingwex and mingw32 libraries to every package it builds.
This breaks when 2 different packages call same gcc standard library
function pow. gcc linker appends pow implementation to the compiled
package, and names that function "pow". But when these 2 compiled
packages are linked together into the final executable, linker
complains, because it finds two "pow" functions with the same name.
This CL stops linking of mingwex and mingw32 during package build -
that leaves pow function reference unresolved. pow reference gets
resolved as final executable is built, by having both internal and
external linker use mingwex and mingw32 libraries.
Fixes #8756
Change-Id: I50ddc79529ea5463c67118d668488345ecf069bc
Reviewed-on: https://go-review.googlesource.com/26670
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Kevin Burke [Tue, 12 Jul 2016 23:54:36 +0000 (16:54 -0700)]
crypto/sha256: add examples for New, Sum256
The goal for these examples is to show how to mirror the
functionality of the sha256sum Unix utility, a common checksumming
tool, using the Go standard library.
Add a newline at the end of the input, so users will get the same
output if they type `echo 'hello world' | sha256sum`, since the
builtin shell echo appends a newline by default. Also use hex output
(instead of the shorter base64) since this is the default output
encoding for shasum/sha256sum.
Bryan Alexander [Sun, 28 Aug 2016 19:11:31 +0000 (14:11 -0500)]
crypto/x509: Fix bug in UnknownAuthorityError.Error
Fix bug in UnknownAuthorityError.Error that would never allow Org
Name to be inserted into error message if the Common Name was empty.
Create tests for all three paths in UnknownAuthorityError.Error
Cherry Zhang [Tue, 30 Aug 2016 18:46:25 +0000 (14:46 -0400)]
cmd/compile, runtime, etc: get rid of constant FP registers
On ARM64, MIPS64, and PPC64, some floating point registers were
reserved for constants 0, 1, 2, 0.5, etc. This CL removes them.
On ARM64, they are never used. On MIPS64 and PPC64, the only use
case is a multiplication-by-2 in the old backend of the compiler,
which is replaced with an addition.
Keith Randall [Tue, 30 Aug 2016 18:08:47 +0000 (11:08 -0700)]
all: use testing.GoToolPath instead of "go"
This change makes sure that tests are run with the correct
version of the go tool. The correct version is the one that
we invoked with "go test", not the one that is first in our path.
Matthew Dempsky [Mon, 29 Aug 2016 20:29:46 +0000 (13:29 -0700)]
reflect: cleanup wording for type identity/equality
Use terms like "equal" and "identical types" to match the Go spec,
rather than inventing a new explanation. See also discussion on
golang.org/cl/27170.
Updates #16348.
Change-Id: I0fe0bd01c0d1da3c8937a579c2ba44cf1eb16b71
Reviewed-on: https://go-review.googlesource.com/28054 Reviewed-by: Rob Pike <r@golang.org>
cmd/compile: recognize integer ranges in switch statements
Consider a switch statement like:
switch x {
case 1:
// ...
case 2, 3, 4, 5, 6:
// ...
case 5:
// ...
}
Prior to this CL, the generated code treated
2, 3, 4, 5, and 6 independently in a binary search.
With this CL, the generated code checks whether
2 <= x && x <= 6.
walkinrange then optimizes that range check
into a single unsigned comparison.
Experiments suggest that the best min range size
is 2, using binary size as a proxy for optimization.
Matthew Dempsky [Tue, 30 Aug 2016 18:15:38 +0000 (11:15 -0700)]
cmd/compile: get rid of ugly {Recvs,Params,Results}P methods
These were a hack abstraction for before FuncType existed.
The result value from calling FuncType() could be saved, but this
maintains the current idiom of consistently using t.FuncType().foo
everywhere in case we choose to evolve the API further.
Hiroshi Ioka [Fri, 19 Aug 2016 00:37:19 +0000 (09:37 +0900)]
path/filepath: handle ".." in normalizing a path on Windows
Current code assumes there are not ".." in the Clean(path).
That's not true. Clean doesn't handle leading "..", so we need to stop
normalization if we see "..".
Fixes #16793
Change-Id: I0a7901bedac17f1210b134d593ebd9f5e8483775
Reviewed-on: https://go-review.googlesource.com/27410 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Brad Fitzpatrick [Tue, 30 Aug 2016 04:08:10 +0000 (04:08 +0000)]
net/http: fix ordering & data race in TestTransportEventTrace_h2
Ordering fix: this CL swaps the order of the log write and the channel close
in WroteRequest. I could reproduce the bug by putting a sleep between the two
when the channel close was first. It needs to happen after the log.
Data race: use the log buffer's mutex when reading too. Not really
important once the ordering fix above is fixed (since nobody is
concurrently writing anymore), but for consistency.
Fixes #16414
Change-Id: If6657884e67be90b4455c8f5a6f7bc6981999ee4
Reviewed-on: https://go-review.googlesource.com/28078
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Martin Möhrmann [Fri, 26 Aug 2016 13:00:46 +0000 (15:00 +0200)]
cmd/compile: improve string iteration performance
Generate a for loop for ranging over strings that only needs to call
the runtime function charntorune for non ASCII characters.
This provides faster iteration over ASCII characters and slightly
faster iteration for other characters.
The runtime function charntorune is changed to take an index from where
to start decoding and returns the index after the last byte belonging
to the decoded rune.
All call sites of charntorune in the runtime are replaced by a for loop
that will be transformed by the compiler instead of calling the charntorune
function directly.
go binary size decreases by 80 bytes.
godoc binary size increases by around 4 kilobytes.
Robert Griesemer [Tue, 30 Aug 2016 00:56:15 +0000 (17:56 -0700)]
cmd/compile: make internal objects directly print to printer
Internal objects that satisfy the Printable interface can print
directly to a printer w/o going through the conversion to a string
first.
Made printer.f understand and special-case %v so that Printable
objects use the printer directly.
This is work in progress and we may end up doing something else
eventually (perhaps using fmt.Formatter) - or even undo these
changes if this exploration doesn't get us to a significantly
better place.
Allocations numbers relative to commit c85b77c (still up, but
reduced from most recent change):