Austin Clements [Tue, 2 Jun 2015 18:48:12 +0000 (14:48 -0400)]
runtime: print start of GC cycle in gctrace, rather than end
Currently the GODEBUG=gctrace=1 trace line includes "@n.nnns" to
indicate the time that the GC cycle ended relative to the time the
program started. This was meant to be consistent with the utilization
as of the end of the cycle, which is printed next on the trace line,
but it winds up just being confusing and unexpected.
Change the trace line to include the time that the GC cycle started
relative to the time the program started.
Rob Pike [Tue, 2 Jun 2015 18:49:24 +0000 (11:49 -0700)]
text/template: clarify the documentation around template definitions
Due to the requirements of parsing template definitions that mention
other templates that are not yet defined, a Template can be in two states:
defined and undefined. Thus, although one calls New, the resulting
template has no definition even though it exists as a data structure.
Thus, for example, will return nil for a template that is named but not
yet defined.
Fixes #10910
Fixes #10926
Clarify the documentation a little to explain this,
Also tidy up the code a little and remove a spurious call to init.
Change-Id: I22cc083291500bca424e83dc12807e0de7b00b7a
Reviewed-on: https://go-review.googlesource.com/10641 Reviewed-by: Andrew Gerrand <adg@golang.org>
Austin Clements [Wed, 20 May 2015 20:30:49 +0000 (16:30 -0400)]
runtime: implement GC stack barriers
This commit implements stack barriers to minimize the amount of
stack re-scanning that must be done during mark termination.
Currently the GC scans stacks of active goroutines twice during every
GC cycle: once at the beginning during root discovery and once at the
end during mark termination. The second scan happens while the world
is stopped and guarantees that we've seen all of the roots (since
there are no write barriers on writes to local stack
variables). However, this means pause time is proportional to stack
size. In particularly recursive programs, this can drive pause time up
past our 10ms goal (e.g., it takes about 150ms to scan a 50MB heap).
Re-scanning the entire stack is rarely necessary, especially for large
stacks, because usually most of the frames on the stack were not
active between the first and second scans and hence any changes to
these frames (via non-escaping pointers passed down the stack) were
tracked by write barriers.
To efficiently track how far a stack has been unwound since the first
scan (and, hence, how much needs to be re-scanned), this commit
introduces stack barriers. During the first scan, at exponentially
spaced points in each stack, the scan overwrites return PCs with the
PC of the stack barrier function. When "returned" to, the stack
barrier function records how far the stack has unwound and jumps to
the original return PC for that point in the stack. Then the second
scan only needs to proceed as far as the lowest barrier that hasn't
been hit.
For deeply recursive programs, this substantially reduces mark
termination time (and hence pause time). For the goscheme example
linked in issue #10898, prior to this change, mark termination times
were typically between 100 and 500ms; with this change, mark
termination times are typically between 10 and 20ms. As a result of
the reduced stack scanning work, this reduces overall execution time
of the goscheme example by 20%.
Fixes #10898.
The effect of this on programs that are not deeply recursive is
minimal:
Austin Clements [Thu, 28 May 2015 16:37:12 +0000 (12:37 -0400)]
runtime: avoid double-scanning of stacks
Currently there's a race between stopg scanning another G's stack and
the G reaching a preemption point and scanning its own stack. When
this race occurs, the G's stack is scanned twice. Currently this is
okay, so this race is benign.
However, we will shortly be adding stack barriers during the first
stack scan, so scanning will no longer be idempotent. To prepare for
this, this change ensures that each stack is scanned only once during
each GC phase by checking the flag that indicates that the stack has
been scanned in this phase before scanning the stack.
Change-Id: Id9f4d5e2e5b839bc3f200ec1723a4a12dd677ab4
Reviewed-on: https://go-review.googlesource.com/10458 Reviewed-by: Rick Hudson <rlh@golang.org>
Austin Clements [Wed, 20 May 2015 20:16:04 +0000 (16:16 -0400)]
runtime: steal space for stack barrier tracking from stack
The stack barrier code will need a bookkeeping structure to keep track
of the overwritten return PCs. This commit introduces and allocates
this structure, but does not yet use the structure.
We don't want to allocate space for this structure during garbage
collection, so this commit allocates it along with the allocation of
the corresponding stack. However, we can't do a regular allocation in
newstack because mallocgc may itself grow the stack (which would lead
to a recursive allocation). Hence, this commit makes the bookkeeping
structure part of the stack allocation itself by stealing the
necessary space from the top of the stack allocation. Since the size
of this bookkeeping structure is logarithmic in the size of the stack,
this has minimal impact on stack behavior.
Austin Clements [Wed, 20 May 2015 19:29:53 +0000 (15:29 -0400)]
runtime: decouple stack bounds and stack allocation size
Currently the runtime assumes that the allocation for the stack is
exactly [stack.lo, stack.hi). We're about to steal a small part of
this allocation for per-stack GC metadata. To prepare for this, this
commit adds a field to the G for the allocated size of the stack.
With this change, stack.lo and stack.hi continue to act as the true
bounds on the stack, but are no longer also used as the bounds on the
stack allocation.
(I also tried this the other way around, where stack.lo and stack.hi
remained the allocation bounds and I introduced a new top of stack.
However, there are far more places that assume stack.hi is the true
top of the stack than there are places that assume it's the top of the
allocation.)
Austin Clements [Thu, 21 May 2015 18:12:29 +0000 (14:12 -0400)]
runtime: clean up signalstack API
Currently signalstack takes a lower limit and a length and all calls
hard-code the passed length. Change the API to take a *stack and
compute the lower limit and length from the passed stack.
This will make it easier for the runtime to steal some space from the
top of the stack since it eliminates the hard-coded stack sizes.
Austin Clements [Sun, 31 May 2015 01:47:00 +0000 (21:47 -0400)]
runtime: increase precision of gctrace times
Currently we truncate gctrace clock and CPU times to millisecond
precision. As a result, many phases are typically printed as 0, which
is fine for user consumption, but makes gathering statistics and
reports over GC traces difficult.
In 1.4, the gctrace line printed times in microseconds. This was
better for statistics, but not as easy for users to read or interpret,
and it generally made the trace lines longer.
This change strikes a balance between these extremes by printing
milliseconds, but including the decimal part to two significant
figures down to microsecond precision. This remains easy to read and
interpret, but includes more precision when it's useful.
For example, where the code currently prints,
gc #29 @1.629s 0%: 0+2+0+12+0 ms clock, 0+2+0+0/12/0+0 ms cpu, 4->4->2 MB, 4 MB goal, 1 P
this prints,
gc #29 @1.629s 0%: 0.005+2.1+0+12+0.29 ms clock, 0.005+2.1+0+0/12/0+0.29 ms cpu, 4->4->2 MB, 4 MB goal, 1 P
Andrew Gerrand [Tue, 2 Jun 2015 18:01:56 +0000 (11:01 -0700)]
net/http: set nosniff header when serving Error
The Error function is a potential XSS vector if a user can control the
error message.
For example, an http.FileServer when given a request for this path
/<script>alert("xss!")</script>
may return a response with a body like this
open <script>alert("xss!")</script>: no such file or directory
Browsers that sniff the content may interpret this as HTML and execute
the script. The nosniff header added by this CL should help, but we
should also try santizing the output entirely.
Mikio Hara [Wed, 20 May 2015 14:20:43 +0000 (23:20 +0900)]
net: fix parsing literal IP addresses in local database
This change fixes incorrect parsing of literal IP addresses in local
database when the addresses contain IPv6 zone identifiers, are in
dotted-decimal notation or in colon-hexadecimal notation with leading
zeros.
https://golang.org/cl/5851 already fixed the code path using getaddrinfo
via cgo. This change fixes the remaining non-cgo code path.
Dave Cheney [Tue, 2 Jun 2015 01:04:45 +0000 (11:04 +1000)]
net: fix undetected set and not used error
Fixes an error where the compiler did not spot that the shadowed err
value was set again after being read. That second assignment was lost
as the value was redeclared in an inner scope.
Spotted by Gordon Klass, https://groups.google.com/forum/#!topic/golang-nuts/MdDLbvOjb4o
Didier Spezia [Wed, 27 May 2015 11:44:19 +0000 (11:44 +0000)]
text/template: fix variadic function call corner case
Executing a template involving variadic functions featuring
a []interface{} slice (such as printf) could result in a
panic in reflect.Value.Call, due to incorrect type checking.
The following expressions failed (with a panic):
{{true|printf}}
{{1|printf}}
{{1.1|printf}}
{{'x'|printf}}
{{1+2i|printf}}
Implemented proper type checks for the fixed parameters of the
variadic functions.
Fixes #10946
Change-Id: Ia75333f651f73b3d2e024cb0c47cc30d90cb6852
Reviewed-on: https://go-review.googlesource.com/10403 Reviewed-by: Rob Pike <r@golang.org>
Didier Spezia [Thu, 21 May 2015 21:35:49 +0000 (21:35 +0000)]
html/template: prevent panic when escaping actions involving chain nodes
The current escape code panics when an action involves chain nodes.
Such nodes can be seen in the following situation:
{{ . | AAA.B }} - AAA being a registered function
The above expression is actually valid, because AAA could return a
map containing a B key. The tests in text/template explicitly
demonstrate this case.
Fix allIdents to cover also chain nodes.
While I was investigating this issue, I realized that the tests
introduced in similar CL 9621 were incorrect. Parse errors were
caught as expected, but for the wrong reason. Fixed them as well.
No changes in text/template code itself.
Fixes #10801
Change-Id: Ic9fe43b63669298ca52c3f499e2725dd2bb818a8
Reviewed-on: https://go-review.googlesource.com/10340 Reviewed-by: Rob Pike <r@golang.org>
Konstantin Shaposhnikov [Sun, 17 May 2015 04:07:04 +0000 (12:07 +0800)]
cmd/doc: try better when looking for package dir
When go doc is invoked with a single package name argument (e.g. go doc pkgname)
it needs to find the directory of the requested package sources in GOPATH.
GOPATH might contain directories with the same name as the requested package
that do no contain any *.go files. This change makes "go doc" ignore such
directories when looking for possible package directories.
This fixes #10882
Change-Id: Ib3d4ea69a25801c34cbe7b044de9870ba12f9aa8
Reviewed-on: https://go-review.googlesource.com/10190 Reviewed-by: Rob Pike <r@golang.org>
Austin Clements [Fri, 29 May 2015 16:13:50 +0000 (12:13 -0400)]
runtime: document that runtime.GC() blocks until GC is complete
runtime.GC() is intentionally very weakly specified. However, it is so
weakly specified that it's difficult to know that it's being used
correctly for its one intended use case: to ensure garbage collection
has run in a test that is garbage-sensitive. In particular, it is
unclear whether it is synchronous or asynchronous. In the old STW
collector this was essentially self-evident; short of queuing up a
garbage collection to run later, it had to be synchronous. However,
with the concurrent collector, there's evidence that people are
inferring that it may be asynchronous (e.g., issue #10986), as this is
both unclear in the documentation and possible in the implementation.
In fact, runtime.GC() runs a fully synchronous STW collection. We
probably don't want to commit to this exact behavior. But we can
commit to the essential property that tests rely on: that runtime.GC()
does not return until the GC has finished.
Change-Id: Ifc3045a505e1898ecdbe32c1f7e80e2e9ffacb5b
Reviewed-on: https://go-review.googlesource.com/10488 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
Adam Langley [Sat, 30 May 2015 16:40:17 +0000 (09:40 -0700)]
encoding/pem: be more permissive about decoding empty blocks.
As noted in bug #10980, an empty PEM block is encoded as
-----BEGIN foo-----
-----END foo-----
However, Decode failed to process this.
RFC 1421 doesn't answer what the encoding of the empty block should be
because PEM messages always contain at least one header. However, PEM
these days is just the encoding format – nobody uses the rest of PEM any
longer.
Having the empty block not contain a newline seems most correct because
https://tools.ietf.org/html/rfc1421#section-9 clearly says that the
optional “pemtext” carries the leading new-line with it. So if omitted,
the new-line should be omitted too.
None the less, this changes makes encoding/pem permissive, accepting any
number of blank lines in an empty PEM block.
Memory usage has been reduced.
The tests are still slow,
but that is issue #10571.
/usr/bin/time shows significant variation
in the peak memory usage compiling with tip.
This is unsurprising, given GC.
Using Go 1.4.2, memory is stable at 410mb.
Using tip at d2ee09298,
memory ranges from 470mb (+15%) to 534mb (+30%),
with a mean of 504mb (+23%), with n=50.
David Chase [Wed, 27 May 2015 21:33:46 +0000 (17:33 -0400)]
cmd/compile: reject p-notation floats in Go source files
Use pkgimport == nil (or not) to distinguish between
parsing .go source files where "p" exponent specifier
is not allowed and parsing .a or .o export data where
it is. Use that to control error when p-exponent is
seen.
Austin Clements [Tue, 26 May 2015 18:32:24 +0000 (14:32 -0400)]
runtime: disable GC around TestGoroutineParallelism
TestGoroutineParallelism can deadlock if the GC runs during the
test. Currently it tries to prevent this by forcing a GC before the
test, but this is best effort and fails completely if GOGC is very low
for testing.
This change replaces this best-effort fix with simply setting GOGC to
off for the duration of the test.
Austin Clements [Wed, 27 May 2015 19:20:49 +0000 (15:20 -0400)]
cmd/compile: avoid temporary in race mode with slice and append
Currently when the race detector is enabled, orderexpr always creates
a temporary for slice and append operations. This used to be necessary
because the race detector had a different code path for slice
assignment that required this temporary. Unfortunately, creating this
temporary inhibits the optimization that eliminates write barriers
when a slice is assigned only to change its length or cap. For most
code, this is bad for performance, and in go:nowritebarrier functions
in the runtime, this can mean the difference between compiling and not
compiling.
Now the race detector uses the regular slice assignment code, so
creating this temporary is no longer necessary.
Austin Clements [Tue, 26 May 2015 19:21:18 +0000 (15:21 -0400)]
runtime: use stripped test environment for TestGdbPython
Most runtime tests that invoke the compiler to build a sub-test binary
do so with a special environment constructed by testEnv that strips
out environment variables that should apply to the test but not to the
build.
Fix TestGdbPython to use this test environment when invoking go build,
like other tests do.
Robert Griesemer [Wed, 27 May 2015 21:43:30 +0000 (14:43 -0700)]
math/big: fix latent decimal conversion bug
A decimal represented 0.0 with a 0-length mantissa and undefined
exponent, but the formatting code assumes a valid zero exponent
if the float value is 0.0. The code worked because we allocate a
new decimal value each time and because there's no rounding that
lead to 0.0.
Change-Id: Ifd771d7709de83b87fdbf141786286b4c3e13d4f
Reviewed-on: https://go-review.googlesource.com/10448 Reviewed-by: Alan Donovan <adonovan@google.com>
Shenghou Ma [Thu, 21 May 2015 21:51:34 +0000 (17:51 -0400)]
cmd/internal/obj: make arm64 use RegTo2 instead of a full fledged Addr To2
It shrinks Prog type from 448 bytes down to 376 bytes on amd64.
It also makes sense, because I don't know of any modern architecture
that have instructions which can write to two destinations, none of
which is a register (even x86 doesn't have such instructions).
Robert Griesemer [Tue, 26 May 2015 21:57:05 +0000 (14:57 -0700)]
math/big: removed TODO, cleanups
- factor out handling of sign
- rename bstring, pstring to fmtB, fmtP consistent with fmtE, fmtF
- move all float-to-string conversion functions into ftoa.go
- no functional changes
Change-Id: I5970ecb874dc9c387630b59147d90bda16a5d8e6
Reviewed-on: https://go-review.googlesource.com/10387 Reviewed-by: Alan Donovan <adonovan@google.com>
David Chase [Sat, 23 May 2015 02:01:01 +0000 (22:01 -0400)]
cmd/compile: propagate correct line numbers in treecopy
Added a lineno parameter to treecopy and listtreecopy
(ignored if = 0). When nodes are copied the copy is
assigned the non-zero lineno (normally this would be
the destination).
Michael Hudson-Doyle [Mon, 25 May 2015 04:13:50 +0000 (16:13 +1200)]
cmd/internal/ld: do not depend on local symbols to read a type's gcdata
We already read the address of a gcmask/gcprog out of the type data, but I
didn't know how many bytes to read. But it turns out that it's easy to
calculate, so change to do that. This means that we no longer depend on the
local symbols being present, allowing me to strip the shared libraries for
distribution and make them a lot smaller.
As a bonus, this makes LSym another 24 bytes smaller, down to 296 bytes now.
Change-Id: I379d359e28d63afae6753efd23efdf1fbb716992
Reviewed-on: https://go-review.googlesource.com/10377 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Hudson-Doyle [Mon, 25 May 2015 02:51:02 +0000 (14:51 +1200)]
cmd/internal/ld: store the libraries a shared library was linked against in a note
The motivation for this is the innocuous looking test case that is added. This
creates a stack exe -> libdep2.so -> libdep.so -> libruntime.so. The problem
comes from the fact that a function from libdep.so gets inlined all the way
into exe. This (unsurprisingly) means that the object file for exe references
symbols from libdep.so, which means that -ldep needs to be passed when linking
exe and it isn't. The fix is simply to pass it -- there is no harm in passing
it when it's not needed.
The thing is, it's not clear at all in the current code to see how the linker
can know that libdep2 is linked against libdep. It could look through the
DT_NEEDED entries in libdep2 and try to guess which are Go libraries, but it
feels better to be explicit. So this adds another SHT_NOTE section that lists
the shared libraries a shared library was linked against, and makes sure the
complete set of depended upon shared libraries is passed to the external
linker.
Change-Id: I79aa6f98b4db4721d657a7eb7b7f062269bf49e2
Reviewed-on: https://go-review.googlesource.com/10376 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Hudson-Doyle [Mon, 25 May 2015 01:59:08 +0000 (13:59 +1200)]
cmd/link/internal/ld: put abi hash into a note
This makes for a more stable API for tools (including cmd/link itself) to
extract the abi hash from a shared library and makes it possible at all for a
library that has had the local symbol table removed.
The existing note-writing code only supports writing notes into the very start
of the object file so they are easy to find in core dumps. This doesn't apply
to the "go" notes and means that all notes have to fit into a fixed size
budget. That's annoying now we have more notes (and the next CL will add
another one) so this does a little bit of work to make adding notes that do not
have to go at the start of the file easier and moves the writing of the package
list note over to that mechanism, which lets me revert a hack that increased
the size budget mentioned above for -buildmode=shared builds.
Change-Id: I6077a68d395c8a2bc43dec8506e73c71ef77d9b9
Reviewed-on: https://go-review.googlesource.com/10375 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Hudson-Doyle [Tue, 26 May 2015 22:30:55 +0000 (10:30 +1200)]
misc/cgo/testshared: do not capture output of go commands in verbose mode
Change-Id: I8694ee5e5642c31815ae63cd414a3b1fcd9c95b0
Reviewed-on: https://go-review.googlesource.com/10411 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Hudson-Doyle [Wed, 27 May 2015 00:04:25 +0000 (12:04 +1200)]
cmd/link: replace interface{} fields with concrete types
The LSym.Section and Section.Elfsect fields were defined as interface{} but
always had the same concrete type (*Section and *ElfShdr respectively) so just
define them with that type. Reduces size of LSym from 328 to 320 bytes and
reduces best-of-10 maxresident size from 246028k to 238036k when linking
libstd.so.
Change-Id: Ie7112c53e4c2c7ce5fe233b81372aa5633f572e8
Reviewed-on: https://go-review.googlesource.com/10410 Reviewed-by: Minux Ma <minux@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ryan Brown [Wed, 8 Apr 2015 19:55:34 +0000 (12:55 -0700)]
cmd/internal/objfile: Skip mach-o debug symbols.
This allows objdump to disassemble gcc generated binaries on OS X 10.6.
Change-Id: I1a5bfbf7c252e78215ef1f122520689d5ce6ddca
Reviewed-on: https://go-review.googlesource.com/10383 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Robert Griesemer [Tue, 26 May 2015 18:44:18 +0000 (11:44 -0700)]
spec: removed TODOs (invisible html comment) in favor of issues
- no "visible" change to spec but for updated date
- retired several outdated TODO items
- filed non-urgent issues 10953, 10954, 10955 for current TODOs
Change-Id: If87ad0fb546c6955a6d4b5801e06e5c7d5695ea2
Reviewed-on: https://go-review.googlesource.com/10382 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Sun, 24 May 2015 18:29:38 +0000 (11:29 -0700)]
cmd/link/internal/ld: if -v, display external linker output
It can be useful when debugging to be able to see what the external
linker is doing even when it succeeds. In particular this permits
passing -v to the external linker to see precisely what it is doing.
Change-Id: Ifed441912d97bbebea20303fdb899e140b380215
Reviewed-on: https://go-review.googlesource.com/10363 Reviewed-by: Minux Ma <minux@golang.org>
Elias Naur [Sat, 23 May 2015 09:26:22 +0000 (11:26 +0200)]
runtime: don't always block all signals on OpenBSD
Implement the changes from CL 10173 on OpenBSD.
Change-Id: I2db1cd8141fd392a34753a1b8113e2e0401173b9
Reviewed-on: https://go-review.googlesource.com/10342
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Mikio Hara [Thu, 14 May 2015 01:18:10 +0000 (10:18 +0900)]
net: adjust dual stack support on dragonfly
As mentioned in
http://gitweb.dragonflybsd.org/dragonfly.git/commit/727ccde8cce813911d885b7f6ed749dcea68a886,
DragonFly BSD is dropping support for IPv6 IPv4-mapped address.
Unfortunately, on some released versions we see the kernels pretend to
support the feature but actually not (unless tweaking some kernel states
via sysctl.)
To avoid unpredictable behavior, the net package assumes that all
DragonFly BSD kernels don't support IPv6 IPv4-mapped address.
Fixes #10764.
Change-Id: Ic7af3651e0372ec03774432fbb6b2eb0c455e994
Reviewed-on: https://go-review.googlesource.com/10071 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Mikio Hara [Fri, 15 May 2015 03:10:10 +0000 (12:10 +0900)]
net: don't show verbose information when -test.v=false
Updates #10845.
Change-Id: I4cec670c7db88c50a6e5619e611744e161d73b3c
Reviewed-on: https://go-review.googlesource.com/10131 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Fri, 22 May 2015 20:58:03 +0000 (13:58 -0700)]
math/big: Always print exponent sign when using 'p' exponent for Floats.
Float.Format supports the 'b' and 'p' format, both of which print
a binary ('p') exponent. The 'b' format always printed a sign ('+'
or '-') for the exponent; the 'p' format only printed a negative
sign for the exponent. This change makes the two consistent. It
also makes the 'p' format easier to read if the exponent is >= 0.
Also:
- Comments added elsewhere.
Change-Id: Ifd2e01bdafb3043345972ca22a90248d055bd29b
Reviewed-on: https://go-review.googlesource.com/10359 Reviewed-by: Alan Donovan <adonovan@google.com>
Elias Naur [Fri, 22 May 2015 22:46:10 +0000 (00:46 +0200)]
misc/cgo/test: fix build for CC=clang
Fix build error when CL=clang introduced by CL 10173.
Change-Id: I8edf210787a9803280c0779ff710c7e634a820d6
Reviewed-on: https://go-review.googlesource.com/10341 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Fri, 22 May 2015 22:21:56 +0000 (15:21 -0700)]
cmd/compile/internal/big: update and apply vendor.bash
Package-external tests must use the vendored math/big package, not
the original one, otherwise tests may fail if there are discrepancies
in the implementation.
Change-Id: Ic5f0489aa6420ffea1f488633453f871ce1f0f66
Reviewed-on: https://go-review.googlesource.com/10380 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Fri, 22 May 2015 00:52:49 +0000 (17:52 -0700)]
math/big: add more Float.Float64 conversion tests
- structure the Float64 conversion tests the same way as for Float32
- add additional test cases, including one that exposes a current issue
(currently disabled, same issue as was fixed for Float32)
The Float64 fix will be in a subsequent change for easier reviewing.
Change-Id: I95dc9e8d1f6b6073a98c7bc2289e6d3248fc3420
Reviewed-on: https://go-review.googlesource.com/10351 Reviewed-by: Alan Donovan <adonovan@google.com>
Robert Griesemer [Fri, 22 May 2015 00:00:37 +0000 (17:00 -0700)]
math/big: fix Float.Float32 conversion for denormal corner cases
The existing code was incorrect for numbers that after rounding would
become the smallest denormal float32 (instead the result was 0). This
caused all.bash to fail if Float32() were used in the compiler for
constant arithmetic (there's currently a work-around - see also issue
10321.
This change fixes the implementation of Float.Float32 and adds
corresponding test cases. Float32 and Float64 diverge at this point.
For ease of review, this change only fixes Float32. Float64 will be
made to match in a subsequent change.
Fixes #10321.
Change-Id: Iccafe37c1593a4946bc552e4ad2045f69be62d80
Reviewed-on: https://go-review.googlesource.com/10350 Reviewed-by: Alan Donovan <adonovan@google.com>
Elias Naur [Mon, 18 May 2015 09:00:24 +0000 (11:00 +0200)]
runtime: don't always unblock all signals
Ian proposed an improved way of handling signals masks in Go, motivated
by a problem where the Android java runtime expects certain signals to
be blocked for all JVM threads. Discussion here
A Go program always needs to have the synchronous signals enabled.
These are the signals for which _SigPanic is set in sigtable, namely
SIGSEGV, SIGBUS, SIGFPE.
A Go program that uses the os/signal package, and calls signal.Notify,
needs to have at least one thread which is not blocking that signal,
but it doesn't matter much which one.
Unix programs do not change signal mask across execve. They inherit
signal masks across fork. The shell uses this fact to some extent;
for example, the job control signals (SIGTTIN, SIGTTOU, SIGTSTP) are
blocked for commands run due to backquote quoting or $().
Our current position on signal masks was not thought out. We wandered
into step by step, e.g., http://golang.org/cl/7323067 .
This CL does the following:
Introduce a new platform hook, msigsave, that saves the signal mask of
the current thread to m.sigsave.
Call msigsave from needm and newm.
In minit grab set up the signal mask from m.sigsave and unblock the
essential synchronous signals, and SIGILL, SIGTRAP, SIGPROF, SIGSTKFLT
(for systems that have it).
In unminit, restore the signal mask from m.sigsave.
The first time that os/signal.Notify is called, start a new thread whose
only purpose is to update its signal mask to make sure signals for
signal.Notify are unblocked on at least one thread.
The effect on Go programs will be that if they are invoked with some
non-synchronous signals blocked, those signals will normally be
ignored. Previously, those signals would mostly be ignored. A change
in behaviour will occur for programs started with any of these signals
blocked, if they receive the signal: SIGHUP, SIGINT, SIGQUIT, SIGABRT,
SIGTERM. Previously those signals would always cause a crash (unless
using the os/signal package); with this change, they will be ignored
if the program is started with the signal blocked (and does not use
the os/signal package).
./all.bash completes successfully on linux/amd64.
OpenBSD is missing the implementation.
Change-Id: I188098ba7eb85eae4c14861269cc466f2aa40e8c
Reviewed-on: https://go-review.googlesource.com/10173 Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Chase [Wed, 20 May 2015 19:16:34 +0000 (15:16 -0400)]
cmd/internal/gc: move check for large-hence-heap-allocated types into escape analysis
Before this change, the check for too-large arrays (and other large
types) occurred after escape analysis. If the data moved off stack
and onto the heap contained any pointers, it would therefore escape,
but because the too-large check occurred after escape analysis this
would not be recorded and a stack pointer would leak to the heap
(see the modified escape_array.go for an example).
Some of these appear to remain, in calls to typecheck from within walk.
Also corrected a few comments in escape_array.go about "BAD"
analysis that is now done correctly.
Enhanced to move aditional EscNone-but-large-so-heap checks into esc.c.
math/big: Simple Montgomery Multiplication to accelerate Mod-Exp
On Haswell I measure anywhere between 2X to 3.5X speedup for RSA.
I believe other architectures will also greatly improve.
In the future may be upgraded by dedicated assembly routine.
Built-in benchmarks i5-4278U turbo off:
benchmark old ns/op new ns/op delta
BenchmarkRSA2048Decrypt 66966493073769 -54.10%
Benchmark3PrimeRSA2048Decrypt 44723401669080 -62.68%
Change-Id: I17df84f85e34208f990665f9f90ea671695b2add
Reviewed-on: https://go-review.googlesource.com/9253 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Adam Langley <agl@golang.org> Reviewed-by: Vlad Krasnov <vlad@cloudflare.com>
Run-TryBot: Adam Langley <agl@golang.org>
Russ Cox [Wed, 20 May 2015 02:58:10 +0000 (22:58 -0400)]
runtime: fix callwritebarrier
Given a call frame F of size N where the return values start at offset R,
callwritebarrier was instructing heapBitsBulkBarrier to scan the block
of memory [F+R, F+R+N). It should only scan [F+R, F+N). The extra N-R
bytes scanned might lead into the next allocated block in memory.
Because the scan was consulting the heap bitmap for type information,
scanning into the next block normally "just worked" in the sense of
not crashing.
Scanning the extra N-R bytes of memory is a problem mainly because
it causes the GC to consider pointers that might otherwise not be
considered, leading it to retain objects that should actually be freed.
This is very difficult to detect.
Luckily, juju turned up a case where the heap bitmap and the memory
were out of sync for the block immediately after the call frame, so that
heapBitsBulkBarrier saw an obvious non-pointer where it expected a
pointer, causing a loud crash.
Why is there a non-pointer in memory that the heap bitmap records as
a pointer? That is more difficult to answer. At least one way that it
could happen is that allocations containing no pointers at all do not
update the heap bitmap. So if heapBitsBulkBarrier walked out of the
current object and into a no-pointer object and consulted those bitmap
bits, it would be misled. This doesn't happen in general because all
the paths to heapBitsBulkBarrier first check for the no-pointer case.
This may or may not be what happened, but it's the only scenario
I've been able to construct.
I tried for quite a while to write a simple test for this and could not.
It does fix the juju crash, and it is clearly an improvement over the
old code.
Austin Clements [Wed, 20 May 2015 15:57:02 +0000 (11:57 -0400)]
runtime: eliminate write barrier from adjustpointers
Currently adjustpointers invokes a write barrier for every stack slot
it updates. This is safe---the write barrier always does nothing
because the new value is never a heap pointer---but it's unnecessary
overhead in performance and complexity.
Fix this by rewriting adjustpointers to work with *uintptrs instead of
*unsafe.Pointers. As an added bonus, this makes the code cleaner.
Russ Cox [Thu, 21 May 2015 17:28:17 +0000 (13:28 -0400)]
all: retire architecture letter in file names, public API
This CL removes the remaining visible uses of the "architecture letter" concept.
(They are no longer in tool names nor in source directory names.)
Because the architecture letter concept is now gone, delete GOCHAR
from "go env" output, and change go/build.ArchChar to return an
error always.
The architecture letter is still used in the compiler and linker sources
as a clumsy architecture enumeration, but that use is not visible to
Go users and can be cleaned up separately.
Change-Id: I4d97a38f372003fb610c9c5241bea440d9dbeb8d
Reviewed-on: https://go-review.googlesource.com/10289 Reviewed-by: Rob Pike <r@golang.org>