Aaron Jacobs [Tue, 20 Oct 2015 02:15:12 +0000 (13:15 +1100)]
runtime: rename _func.frame to make it clear it's deprecated and unused.
When I saw that it was labelled "legacy", I went looking for users of it
to see how it was still used. But there aren't any. Save the next person
the trouble.
Change-Id: I921dd6c57b60331c9816542272555153ac133c02
Reviewed-on: https://go-review.googlesource.com/16035 Reviewed-by: Dave Cheney <dave@cheney.net>
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Thorben Krueger [Wed, 14 Oct 2015 19:18:59 +0000 (19:18 +0000)]
fmt: Add support for capital '%X' format verb for scanning
For printing, the format verb '%X' results in a capitalized
hex-representation of the formatted value. Conversely, using
'%X' in a Scanf function should scan a hex-representation
into the given interface{}. The existing implementation
however only supports '%X' for scanning hex values into
integers; strings or byte slices remain empty. On the other
hand, lower-case '%x' supports strings and byte slices just
fine. This is merely an oversight, which this commit fixes.
(Additional tests also included.)
Fixes #12940
Change-Id: I178a7f615bae950dfc014ca8c0a038448cf0452a
Reviewed-on: https://go-review.googlesource.com/15689 Reviewed-by: Andrew Gerrand <adg@golang.org>
Nodir Turakulov [Mon, 19 Oct 2015 21:36:25 +0000 (14:36 -0700)]
net/http/httptest: detect Content-Type in ResponseRecorder
* detect Content-Type on ReponseRecorder.Write[String] call
if header wasn't written yet, Content-Type header is not set and
Transfer-Encoding is not set.
* fix typos in serve_test.go
This is the start of wiring up the HTTP/2 Transport. It is still
disabled in this commit.
This change does two main things:
1) Transport.RegisterProtocol now permits registering "http" or
"https" (they previously paniced), and the semantics of the
registered RoundTripper have been extended to say that the new
sentinel error value (ErrSkipAltProtocol, added in this CL) means
that the Transport's RoundTrip method proceeds as if the alternate
protocol had not been registered. This gives us a place to register
an alternate "https" RoundTripper which gets first dibs on using
HTTP/2 if there's already a cached connection.
2) adds Transport.TLSNextProto, a map keyed by TLS NPN/ALPN protocol
strings, similar in feel to the existing Server.TLSNextProto map.
This map is the glue between the HTTP/1 and HTTP/2 clients, since
we don't know which protocol we're going to speak (and thus which
Transport type to use) until we've already made the TCP connection.
net/http/httptest: change Server to use http.Server.ConnState for accounting
With this CL, httptest.Server now uses connection-level accounting of
outstanding requests instead of ServeHTTP-level accounting. This is
more robust and results in a non-racy shutdown.
This is much easier now that net/http.Server has the ConnState hook.
Burcu Dogan [Mon, 19 Oct 2015 22:27:19 +0000 (15:27 -0700)]
cmd/go: don't override GIT_TERMINAL_PROMPT
This CL keeps disallowing `go get` from falling to the prompt unless
user has set GIT_TERMINAL_PROMPT env variable. If GIT_TERMINAL_PROMPT
is set, go-get will not override its value and will prompt for
username/password in the case of GIT_TERMINAL_PROMPT=1.
Fixes #12706.
Change-Id: Ibd6b1100af6b04fb8114279cdcf608943e7765be
Reviewed-on: https://go-review.googlesource.com/16091 Reviewed-by: Andrew Gerrand <adg@golang.org>
Michael Hudson-Doyle [Mon, 12 Oct 2015 00:20:10 +0000 (13:20 +1300)]
runtime: tweaks to allow -buildmode=shared to work
Building Go shared libraries requires that all functions that have declarations
without bodies have implementations and vice versa, so remove the
implementation of call16 and add a stub implementation of sigreturn.
Change-Id: I4d5a30c8637a5da7991054e151a536611d5bea46
Reviewed-on: https://go-review.googlesource.com/15966 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Hudson-Doyle [Sun, 11 Oct 2015 23:19:20 +0000 (12:19 +1300)]
cmd/link: centralize knowledge of size of fixed part of stack
Shared libraries on ppc64le will require a larger minimum stack frame (because
the ABI mandates that the TOC pointer is available at 24(R1)). Part 2b of
preparing for that is to have all the code in the linker that needs to know
this size of this call a function to find out.
Change-Id: I246363840096db22e44beabbe38b61d60c1f31ad
Reviewed-on: https://go-review.googlesource.com/15675 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Hudson-Doyle [Wed, 23 Sep 2015 09:02:50 +0000 (21:02 +1200)]
cmd/compile: allow -shared/-dynlink on ppc64
Only effect is register related: do not allocate R2 or R12, put function
entrypoint in R12 before indirect call.
Change-Id: I9cdd553bab022601c9cb5bb43c9dc0c368c6fb0a
Reviewed-on: https://go-review.googlesource.com/15961 Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Crawshaw [Mon, 19 Oct 2015 18:41:38 +0000 (14:41 -0400)]
net: android no longer supports unix/unixgram
I cannot find any documentation for this, but these tests no longer run
on the device I have since upgrading to Android L. Presumably it still
works for root, but standard Android programs to not have root access.
Change-Id: I001c8fb5ce22f9ff8d7433f881d0dccbf6ab969d
Reviewed-on: https://go-review.googlesource.com/16056 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Austin Clements [Sun, 18 Oct 2015 03:52:49 +0000 (23:52 -0400)]
runtime: consolidate gcResetGState calls
Currently gcResetGState is called by func gcscan_m for concurrent GC
and directly by func gc for STW GC. Simplify this by consolidating
these two calls in to one call by func gc above where it splits for
concurrent and STW GC.
As a consequence, gcResetGState and gcResetMarkState are always called
together, so the next commit will consolidate these.
Change-Id: Ib62d404c7b32b28f7d3080d26ecf3966cbc4aca0
Reviewed-on: https://go-review.googlesource.com/16040 Reviewed-by: Rick Hudson <rlh@golang.org>
Nodir Turakulov [Fri, 16 Oct 2015 08:33:28 +0000 (01:33 -0700)]
fmt: clarify reflect.Value printing
fmt docs say:
If the operand is a reflect.Value, the concrete value it
holds is printed as if it was the operand.
It implies recursive application of this rule, which is not the case.
Clarify the docs.
Change-Id: I019277c7c6439095bab83e5536aa06403638aa51
Reviewed-on: https://go-review.googlesource.com/15952 Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Crawshaw [Mon, 19 Oct 2015 16:54:38 +0000 (12:54 -0400)]
cmd/go: -buildmode=pie for android/arm
Also make PIE executables the default build mode, as PIE executables
are required as of Android L.
For #10807
Change-Id: I86b7556b9792105cd2531df1b8f3c8f7a8c5d25c
Reviewed-on: https://go-review.googlesource.com/16055 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
Joe Tsai [Fri, 9 Oct 2015 00:37:07 +0000 (17:37 -0700)]
compress/flate: improve inflate speed by reading more bits at a time
The flate library guarantees that the Reader will never read more
bytes than is necessary. This way, the underlying io.Reader will
be left exactly after the last byte of the DEFLATE stream.
Formats like gzip depend on this behavior being true.
As such, inflate conservatively reads the minimum symbol length in
huffSym leading to many individual calls to moreBits. However, if we
take advantage of the fact that every block *must* end with the EOB
symbol, we can choose to read the length of the EOB symbol.
Since the EOB symbol is also the most rare symbol (occuring exactly
once) in a block, we can hypothesize that it is almost as long as
the max symbol length, allowing huffSym to ask for more bits at the
start of every loop. This increases the probabilty that the Huffman
code is decoded on the first iteration of the outer for-loop.
Michael Hudson-Doyle [Thu, 8 Oct 2015 09:49:39 +0000 (22:49 +1300)]
runtime, runtime/cgo: conform to PIC register use rules in ppc64 asm
PIC code on ppc64le uses R2 as a TOC pointer and when calling a function
through a function pointer must ensure the function pointer is in R12. These
rules are easy enough to follow unconditionally in our assembly, so do that.
Change-Id: Icfc4e47ae5dfbe15f581cbdd785cdeed6e40bc32
Reviewed-on: https://go-review.googlesource.com/15526 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Hudson-Doyle [Thu, 8 Oct 2015 09:34:29 +0000 (22:34 +1300)]
reflect, runtime, runtime/cgo: use ppc64 asm constant for fixed frame size
Shared libraries on ppc64le will require a larger minimum stack frame (because
the ABI mandates that the TOC pointer is available at 24(R1)). Part 3 of that
is using a #define in the ppc64 assembly to refer to the size of the fixed
part of the stack (finding all these took me about a week!).
Change-Id: I50f22fe1c47af1ec59da1bd7ea8f84a4750df9b7
Reviewed-on: https://go-review.googlesource.com/15525 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Hudson-Doyle [Thu, 8 Oct 2015 09:13:44 +0000 (22:13 +1300)]
cmd/compile, cmd/internal/obj: centralize knowledge of size of fixed part of stack
Shared libraries on ppc64le will require a larger minimum stack frame (because
the ABI mandates that the TOC pointer is available at 24(R1)). Part 2a of
preparing for that is to have all bits of arch-independent and ppc64-specific
codegen that need to know call a function to find out.
Change-Id: I55899f73037e92227813c491049a3bd6f30bd41f
Reviewed-on: https://go-review.googlesource.com/15524 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Hudson-Doyle [Thu, 8 Oct 2015 08:52:03 +0000 (21:52 +1300)]
runtime: add a constant for the smallest possible stack frame
Shared libraries on ppc64le will require a larger minimum stack frame (because
the ABI mandates that the TOC pointer is available at 24(R1)). So to prepare
for this, make a constant for the fixed part of a stack and use that where
necessary.
Change-Id: I447949f4d725003bb82e7d2cf7991c1bca5aa887
Reviewed-on: https://go-review.googlesource.com/15523 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Hudson-Doyle [Thu, 15 Oct 2015 08:48:11 +0000 (21:48 +1300)]
cmd/link: always disable lazy PLT resolution when dynamically linking Go
Go cannot allow lazy PLT resolution when calling between Go functions because
the lazy resolution can use more stack than is available. Lazy resolution is
disabled by passing -z now to the system linker, but unfortunately was only
passed when linking to a Go shared library. That sounds fine, but the shared
library containing the runtime is not linked to any other Go shared library but
calls main.init and main.main via a PLT, and before this fix this did use lazy
resolution. (For some reason this never caused a problem on intel, but it
breaks on ppc64le). Fortunately the fix is very simple: always pass -z now to
the system linker when dynamically linking Go.
Change-Id: I7806d40aac80dcd1e56b95864d1cfeb1c42614e2
Reviewed-on: https://go-review.googlesource.com/15870 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Yasuharu Goto [Thu, 14 May 2015 15:44:34 +0000 (00:44 +0900)]
net/http: Client support for Expect: 100-continue
Current http client doesn't support Expect: 100-continue request
header(RFC2616-8/RFC7231-5.1.1). So even if the client have the header,
the head of the request body is consumed prematurely.
Those are my intentions to avoid premature consuming body in this change.
- If http.Request header contains body and Expect: 100-continue
header, it blocks sending body until it gets the first response.
- If the first status code to the request were 100, the request
starts sending body. Otherwise, sending body will be cancelled.
- Tranport.ExpectContinueTimeout specifies the amount of the time to
wait for the first response.
Brad Fitzpatrick [Fri, 16 Oct 2015 16:28:42 +0000 (16:28 +0000)]
net: unblock plan9 TCP Read calls after socket close
Fixes #7782
Fixes #9554
Updates #7237 (original metabug, before we switched to specific bugs)
Updates #11932 (plan9 still doesn't have net I/O deadline support)
Change-Id: I96f311b88b1501d884ebc008fd31ad2cf1e16d75
Reviewed-on: https://go-review.googlesource.com/15941 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: David du Colombier <0intro@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ilya Tocar [Thu, 15 Oct 2015 15:59:01 +0000 (18:59 +0300)]
net: use IndexByte implementation from runtime package
In net/parse.go we reimplement bytes.IndexByte and strings.IndexByte,
However those are implemented in runtime/$GOARCH_asm.s.
Using versions from runtime should provide performance advantage,
and keep the same code together.
Nodir Turakulov [Fri, 16 Oct 2015 02:30:02 +0000 (19:30 -0700)]
runtime: rename print1.go -> print.go
It seems that it was called print1.go mistakenly: print.go was deleted
in the same commit:
https://go.googlesource.com/go/+/597b266eafe7d63e9be8da1c1b4813bd2998a11c
Chris Hines [Mon, 14 Sep 2015 07:44:56 +0000 (03:44 -0400)]
database/sql: avoid deadlock waiting for connections
Previously with db.maxOpen > 0, db.maxOpen+n failed connection attempts
started concurrently could result in a deadlock. DB.conn and
DB.openNewConnection did not trigger the DB.connectionOpener go routine
after a failed connection attempt. This omission could leave go routines
waiting for DB.connectionOpener forever.
In addition the logic to track the state of the pool was inconsistent.
db.numOpen was sometimes incremented optimistically and sometimes not.
This change harmonizes the logic and eliminates the db.pendingOpens
variable, making the logic easier to understand and maintain.
Michael Hudson-Doyle [Thu, 15 Oct 2015 20:56:07 +0000 (09:56 +1300)]
runtime, os/signal: use //go:linkname instead of assembly stubs to get access to runtime functions
os/signal depends on a few unexported runtime functions. This removes the
assembly stubs it used to get access to these in favour of using
//go:linkname in runtime to make the functions accessible to os/signal.
This is motivated by ppc64le shared libraries, where you cannot BR to a symbol
defined in a shared library (only BL), but it seems like an improvment anyway.
Change-Id: I09361203ce38070bd3f132f6dc5ac212f2dc6f58
Reviewed-on: https://go-review.googlesource.com/15871
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Minux Ma <minux@golang.org> Reviewed-by: Dave Cheney <dave@cheney.net>
Apply static bounds checking logic during type checking even to
zero-element arrays, but skip synthesized OINDEX nodes that the
compiler has asserted are within bounds (such as the ones generated
while desugaring ORANGE nodes). This matches the logic in walkexpr
that also skips static bounds checking when Bounded is true.
Nodir Turakulov [Wed, 14 Oct 2015 02:09:32 +0000 (19:09 -0700)]
text/template: resolve non-empty interface
Read what a non-empty interface points to.
The deleted lines were added in https://codereview.appspot.com/4810060/,
which attempted to break an infinite loop. That was a long time ago.
If I just delete these lines with current codebase, the test "bug1"
(added in that CL) does not fail.
All new tests fail without this fix.
Fixes #12924
Change-Id: I9370ca44facd6af3019850aa065b936e5a482d37
Reviewed-on: https://go-review.googlesource.com/15809 Reviewed-by: Andrew Gerrand <adg@golang.org>
As correctly mentioned in #11883, encodeState.string and
encodeState.stringBytes never return an error.
This CL removes the error from the function signatures and somewhat
simplifies call sites.
Brad Fitzpatrick [Wed, 14 Oct 2015 20:41:36 +0000 (20:41 +0000)]
net/http: enable automatic HTTP/2 if TLSNextProto is nil
This enables HTTP/2 by default (for https only) if the user didn't
configure anything in their NPN/ALPN map. If they're using SPDY or an
alternate http2 or a newer http2 from x/net/http2, we do nothing
and don't use the standard library's vendored copy of x/net/http2.
David Glasser [Wed, 14 Oct 2015 21:25:00 +0000 (14:25 -0700)]
net/http: don't hang if RemoteAddr() blocks
The PROXY protocol is supported by several proxy servers such as haproxy
and Amazon ELB. This protocol allows services running behind a proxy to
learn the remote address of the actual client connecting to the proxy,
by including a single textual line at the beginning of the TCP
connection.
http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt
There are several Go libraries for this protocol (such as
https://github.com/armon/go-proxyproto), which operate by wrapping a
net.Conn with an implementation whose RemoteAddr method reads the
protocol line before returning. This means that RemoteAddr is a blocking
call.
Before this change, http.Serve called RemoteAddr from the main Accepting
goroutine, not from the per-connection goroutine. This meant that it
would not Accept another connection until RemoteAddr returned, which is
not appropriate if RemoteAddr needs to do a blocking read from the
socket first.
Raul Silvera [Wed, 14 Oct 2015 02:49:40 +0000 (19:49 -0700)]
runtime: Reduce testing for fastlog2 implementation
The current fastlog2 testing checks all 64M values in the domain of
interest, which is too much for platforms with no native floating point.
Reduce testing under testing.Short() to speed up builds for those platforms.
Related to #12620
Change-Id: Ie5dcd408724ba91c3b3fcf9ba0dddedb34706cd1
Reviewed-on: https://go-review.googlesource.com/15830 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Joel Sing <jsing@google.com> Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Hyang-Ah Hana Kim [Tue, 13 Oct 2015 17:58:11 +0000 (13:58 -0400)]
misc/cgo/testcshared: use -pie for android-L.
Also, handle the case where 'read' returns EINVAL instead of EBADF
when the descriptor is not ready. (android 4.4.4/cyanogenmod, nexus7)
Change-Id: I56c5949d27303d44a4fd0de38951b85e20cef167
Reviewed-on: https://go-review.googlesource.com/15810 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 13 Oct 2015 17:05:13 +0000 (10:05 -0700)]
misc/cgo/test: fix go vet warnings
Fixes these warnings from go vet:
buildid_linux.go:25: no formatting directive in Fatalf call
callback.go:180: arg pc[i] for printf verb %p of wrong type: uintptr
env.go:34: possible misuse of unsafe.Pointer
issue7665.go:22: possible misuse of unsafe.Pointer
Change-Id: I83811b9c10c617139713a626b4a34ab05564d4fe
Reviewed-on: https://go-review.googlesource.com/15802 Reviewed-by: David Crawshaw <crawshaw@golang.org>
Burcu Dogan [Mon, 12 Oct 2015 08:02:51 +0000 (01:02 -0700)]
cmd/go: always log dynamic import errors
There is no easy way to understand what user intent was and whether
they wanted to use a dynamic import or not.
If we skip logging such errors, it breaks common use cases such as
https://golang.org/issue/12810.
It's a better approach to expose the underlying mechanism and
be more verbose with the error messages.
Fixes #12810.
Change-Id: I7e922c9e848382690d9d9b006d7046e6cf93223b
Reviewed-on: https://go-review.googlesource.com/15756 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
Brad Fitzpatrick [Mon, 12 Oct 2015 20:41:40 +0000 (20:41 +0000)]
net/http: export the "new" error codes from RFC 6585
These were proposed in the RFC over three years ago, then proposed to
be added to Go in https://codereview.appspot.com/7678043/ 2 years and
7 months ago, and the spec hasn't been updated or retracted the whole
time.
Time to export them.
Of note, HTTP/2 uses code 431 (Request Header Fields Too Large).
Updates #12843
Change-Id: I78c2fed5fab9540a98e845ace73f21c430a48809
Reviewed-on: https://go-review.googlesource.com/15732 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dave Cheney [Sat, 10 Oct 2015 03:21:42 +0000 (14:21 +1100)]
net/url: make *url.Error implement net.Error
Fixes #12866
net/http.Client returns some errors wrapped in a *url.Error. To avoid
the requirement to unwrap these errors to determine if the cause was
temporary or a timeout, make *url.Error implement net.Error directly.
Joe Tsai [Tue, 22 Sep 2015 08:33:11 +0000 (01:33 -0700)]
compress/flate: deprecate ReadError and WriteError
A vast majority of the time, ReadError isn't even returned during
IO operations. Instead, an unwrapped error will be returned because
of the ReadByte call on L705. Because DEFLATE streams are primarily
compressed and require byte for byte Huffman decoding, most of the
data read from a data stream will go through ReadByte.
Although this is technically an API change, any user reliant on
this error would not have worked properly anyways due to the fact
that most IO error are not wrapped. We might as well deprecate
ReadError. It is useless and actually makes clients that do
depend on catching IO errors more difficult.
Fixes #11856
Fixes #12724
Change-Id: Ib5fec5ae215e977c4e85de5701ce6a473d400af8
Reviewed-on: https://go-review.googlesource.com/14834 Reviewed-by: Nigel Tao <nigeltao@golang.org>
Ian Lance Taylor [Fri, 9 Oct 2015 20:02:25 +0000 (13:02 -0700)]
cmd/link: remove -W option
The -W option has not worked since Go 1.3. It is not documented. When
it did work, it generated useful output, but it was for human viewing;
there was no reason to write a script that passes the -W option, so it's
unlikely that anybody is using it today.
Nodir Turakulov [Fri, 9 Oct 2015 18:59:58 +0000 (11:59 -0700)]
cmd/go: print all warnings to stderr
All warnings in cmd/go are printed using fmt.Fprintf(os.Stderr...)
except one in test.go which is printed using log.Printf.
This is a minor inconsistency.
Austin Clements [Mon, 5 Oct 2015 03:56:11 +0000 (20:56 -0700)]
runtime: assist before allocating
Currently, when the mutator allocates, the runtime first allocates the
memory and then, if that G has done "enough" allocation, the runtime
checks whether the G has assist debt to pay off and, if so, pays it
off. This approach leads to under-assisting, where a G can allocate a
large region (or many small regions) before paying for it, or can even
exit with outstanding debt.
This commit flips this around so that a G always acquires enough
credit for an allocation before it can perform that allocation. We
continue to amortize the cost of assists by requiring that they
over-assist when triggered to build up credit for many allocations.
Austin Clements [Mon, 5 Oct 2015 03:16:57 +0000 (20:16 -0700)]
runtime: directly track GC assist balance
Currently we track the per-G GC assist balance as two monotonically
increasing values: the bytes allocated by the G this cycle (gcalloc)
and the scan work performed by the G this cycle (gcscanwork). The
assist balance is hence assistRatio*gcalloc - gcscanwork.
This works, but has two important downsides:
1) It requires floating-point math to figure out if a G is in debt or
not. This makes it inappropriate to check for assist debt in the
hot path of mallocgc, so we only do this when a G allocates a new
span. As a result, Gs can operate "in the red", leading to
under-assist and extended GC cycle length.
2) Revising the assist ratio during a GC cycle can lead to an "assist
burst". If you think of plotting the scan work performed versus
heaps size, the assist ratio controls the slope of this line.
However, in the current system, the target line always passes
through 0 at the heap size that triggered GC, so if the runtime
increases the assist ratio, there has to be a potentially large
assist to jump from the current amount of scan work up to the new
target scan work for the current heap size.
This commit replaces this approach with directly tracking the GC
assist balance in terms of allocation credit bytes. Allocating N bytes
simply decreases this by N and assisting raises it by the amount of
scan work performed divided by the assist ratio (to get back to
bytes).
This will make it cheap to figure out if a G is in debt, which will
let us efficiently check if an assist is necessary *before* performing
an allocation and hence keep Gs "in the black".
This also fixes assist bursts because the assist ratio is now in terms
of *remaining* work, rather than work from the beginning of the GC
cycle. Hence, the plot of scan work versus heap size becomes
continuous: we can revise the slope, but this slope always starts from
where we are right now, rather than where we were at the beginning of
the cycle.
Change-Id: Ia821c5f07f8a433e8da7f195b52adfedd58bdf2c
Reviewed-on: https://go-review.googlesource.com/15408 Reviewed-by: Rick Hudson <rlh@golang.org>
Austin Clements [Thu, 8 Oct 2015 05:37:15 +0000 (22:37 -0700)]
runtime: ensure minimum heap distance via heap goal
Currently we ensure a minimum heap distance of 1MB when computing the
assist ratio. Rather than enforcing this minimum on the heap distance,
it makes more sense to enforce that the heap goal itself is at least
1MB over the live heap size at the beginning of GC. Currently the two
approaches are semantically equivalent, but this will let us switch to
basing the assist ratio on current heap distance rather than the
initial heap distance, since we can't enforce this minimum on the
current heap distance (the GC may never finish because the goal posts
will always be 1MB away).
Change-Id: I0027b1c26a41a0152b01e5b67bdb1140d43ee903
Reviewed-on: https://go-review.googlesource.com/15604 Reviewed-by: Rick Hudson <rlh@golang.org>
Austin Clements [Mon, 5 Oct 2015 03:00:01 +0000 (23:00 -0400)]
runtime: update gcController.scanWork regularly
Currently, gcController.scanWork is updated as lazily as possible
since it is only read at the end of the GC cycle. We're about to read
it during the GC cycle to improve the assist ratio revisions, so
modify gcDrain* to regularly flush to gcController.scanWork in much
the same way as we regularly flush to gcController.bgScanCredit.
One consequence of this is that it's difficult to keep gcw.scanWork
monotonic, so we give up on that and simply return the amount of scan
work done by gcDrainN rather than calculating it in the caller.
Change-Id: I7b50acdc39602f843eed0b5c6d2dacd7e762b81d
Reviewed-on: https://go-review.googlesource.com/15407 Reviewed-by: Rick Hudson <rlh@golang.org>
Austin Clements [Mon, 5 Oct 2015 02:47:27 +0000 (22:47 -0400)]
runtime: control background scan credit flushing with flag
Currently callers of gcDrain control whether it flushes scan work
credit to gcController.bgScanCredit by passing a value other than -1
for the flush threshold. Shortly we're going to make this always flush
scan work to gcController.scanWork and optionally also flush scan work
to gcController.bgScanCredit. This will be much easier if the flush
threshold is simply a constant (which it is in practice) and callers
merely control whether or not the flush includes the background
credit. Hence, replace the flush threshold argument with a flag.
Change-Id: Ia27db17de8a3f1e462a5d7137d4b5dc72f99a04e
Reviewed-on: https://go-review.googlesource.com/15406 Reviewed-by: Rick Hudson <rlh@golang.org>
Austin Clements [Mon, 5 Oct 2015 02:42:43 +0000 (22:42 -0400)]
runtime: consolidate gcDrain and gcDrainUntilPreempt
These functions were nearly identical. Consolidate them by adding a
flags argument. In addition to cleaning up this code, this makes
further changes that affect both functions easier.
Change-Id: I6ec5c947603bbbd3ff4040113b2fbc240e99745f
Reviewed-on: https://go-review.googlesource.com/15405 Reviewed-by: Rick Hudson <rlh@golang.org>
mpl [Sun, 4 Oct 2015 21:32:26 +0000 (23:32 +0200)]
mime/multipart: fix peekBufferSeparatorIndex edge case
The case fixed by this change happens when, in func (pr partReader)
Read, the Peek happens to read so that peek looks like:
"somedata\r\n--Boundary\r"
peekBufferSeparatorIndex was returning (-1, false) because it didn't
find the trailing '\n'.
This was wrong because:
1) It didn't match the documentation: as "\r\n--Boundary" was found, it
should return the index of that pattern, not -1.
2) It lead to an nCopy cut such as:
"somedata\r| |\n--Boundary\r" instead of "somedata| |\r\n--Boundary\r"
which made the subsequent Read miss the boundary, and eventually end
with a "return 0, io.ErrUnexpectedEOF" case, as reported in:
obj.ProgInfo is a field inside obj.Prog, which is currently 320 bytes
on 64bit platforms. By moving the Flags field below the other fields
the size of obj.Prog drops into the 288 byte size class, a saving of
32 bytes per value allocated on the heap.
Matthew Dempsky [Thu, 8 Oct 2015 19:43:57 +0000 (12:43 -0700)]
syscall: fix vet warning in UnixRights
Change-Id: Ic19c94fe0af55e17f6c2fcfd36085ccb1584da6f
Reviewed-on: https://go-review.googlesource.com/15608 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ian Lance Taylor [Thu, 8 Oct 2015 17:22:50 +0000 (10:22 -0700)]
cmd/link: pass -Wl,-z,nodelete when creating an ELF shared library
Go shared libraries do not support dlclose, and there is no likelihood
that they will suppose dlclose in the future. Set the DF_1_NODELETE
flag to tell the dynamic linker to not attempt to remove them from
memory. This makes the shared library act as though every call to
dlopen passed the RTLD_NODELETE flag.
Fixes #12582.
Update #11100.
Update #12873.
Change-Id: Id4b6e90a1b54e2e6fc8355b5fb22c5978fc762b4
Reviewed-on: https://go-review.googlesource.com/15605 Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Keith Randall [Mon, 31 Aug 2015 23:26:12 +0000 (16:26 -0700)]
runtime: make aeshash more DOS-proof
Improve the aeshash implementation to make it harder to engineer collisions.
1) Scramble the seed before xoring with the input string. This
makes it harder to cancel known portions of the seed (like the size)
because it mixes the per-table seed into those other parts.
2) Use table-dependent seeds for all stripes when hashing >16 byte strings.
For small strings this change uses 4 aesenc ops instead of 3, so it
is somewhat slower. The first two can run in parallel, though, so
it isn't 33% slower.
but it turns out that it's necessary to avoid the GC seeing partially written
pointers.
It's of course possible to be more sophisticated (using ldp/stp to move 16
bytes at a time in the core loop and unrolling the tail copying loops being
the obvious ideas) but I wanted something simple and (reasonably) obviously
correct.
Didier Spezia [Wed, 7 Oct 2015 22:25:25 +0000 (22:25 +0000)]
cmd/compile: fix some C to Go translation leftovers
Following the C to Go translation, some useless variables
were left in the code. In fmt.go, this was harmless.
In lex.go, it broke the error message related to
non-canonical import paths.
Fix it, and remove the useless variables.
The added test case is ignored in the go/types tests, since
the behavior of the non-canonical import path check seems
to be different.
Fixes #11362
Change-Id: Ic9129139ede90357dc79ebf167af638cf44536fa
Reviewed-on: https://go-review.googlesource.com/15580 Reviewed-by: Marvin Stenger <marvin.stenger94@gmail.com> Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>