Keith Randall [Wed, 7 Jan 2015 00:42:48 +0000 (16:42 -0800)]
runtime: remove size argument from hash and equal algorithms
The equal algorithm used to take the size
equal(p, q *T, size uintptr) bool
With this change, it does not
equal(p, q *T) bool
Similarly for the hash algorithm.
The size is rarely used, as most equal functions know the size
of the thing they are comparing. For instance f32equal already
knows its inputs are 4 bytes in size.
For cases where the size is not known, we allocate a closure
(one for each size needed) that points to an assembly stub that
reads the size out of the closure and calls generic code that
has a size argument.
Reduces the size of the go binary by 0.07%. Performance impact
is not measurable.
Keith Randall [Sun, 28 Dec 2014 03:26:40 +0000 (19:26 -0800)]
runtime: faster version of findfunc
Use a lookup table to find the function which contains a pc. It is
faster than the old binary search. findfunc is used primarily for
stack copying and garbage collection.
benchmark old ns/op new ns/op delta
BenchmarkStackCopy 294746596255400980 -13.35%
(findfunc is one of several tasks done by stack copy, the findfunc
time itself is about 2.5x faster.)
The lookup table is built at link time. The table grows the binary
size by about 0.5% of the text segment.
We impose a lower limit of 16 bytes on any function, which should not
have much of an impact. (The real constraint required is <=256
functions in every 4096 bytes, but 16 bytes/function is easier to
implement.)
Austin Clements [Tue, 16 Dec 2014 23:34:55 +0000 (18:34 -0500)]
cmd/cgo, runtime/cgo: support ppc64
This implements support for calls to and from C in the ppc64 C ABI, as
well as supporting functionality such as an entry point from the
dynamic linker.
Austin Clements [Tue, 16 Dec 2014 18:52:09 +0000 (13:52 -0500)]
cmd/9g: don't use R13
R13 is the C TLS pointer. Once we're calling to and from C code, if
we clobber R13 in our code, sigtramp won't know whether to get the
current g from REGG or from C TLS. The simplest solution is for Go
code to preserve the C TLS pointer. This is equivalent to what other
platforms do, except that on other platforms the TLS pointer is in a
special register.
Austin Clements [Tue, 16 Dec 2014 19:59:59 +0000 (14:59 -0500)]
cmd/9l: support internal linking
This implements the ELF relocations and dynamic linking tables
necessary to support internal linking on ppc64. It also marks ppc64le
ELF files as ABI v2; failing to do this doesn't seem to confuse the
loader, but it does confuse libbfd (and hence gdb, objdump, etc).
Austin Clements [Mon, 22 Dec 2014 19:42:37 +0000 (14:42 -0500)]
cmd/ld: support for relocation variants
Most ppc64 relocations come in six or more variants where the basic
relocation formula is the same, but which bits of the computed value
are installed where changes. Introduce the concept of "variants" for
internal relocations to support this. Since this applies to
architecture-independent relocation types like R_PCREL, we do this in
relocsym.
Currently there is only an identity variant. A later CL that adds
support for ppc64 ELF relocations will introduce more.
Austin Clements [Tue, 16 Dec 2014 19:01:08 +0000 (14:01 -0500)]
cmd/ld: decode local entry offset from ppc64 symbols
ppc64 function symbols have both a global entry point and a local
entry point, where the difference is stashed in sym.other. We'll need
this information to generate calls to ELF ABI functions.
Keith Randall [Sat, 27 Dec 2014 02:10:59 +0000 (18:10 -0800)]
runtime: increase number of stack orders to 4
Cache 2KB, 4KB, 8KB, and 16KB stacks. Larger stacks
will be allocated directly. There is no point in cacheing
32KB+ stacks as we ask for and return 32KB at a time
from the allocator.
Note that the minimum stack is 8K on windows/64bit and 4K on
windows/32bit and plan9. For these os/arch combinations,
the number of stack orders is less so that we have the same
maximum cached size.
Keith Randall [Wed, 7 Jan 2015 04:38:44 +0000 (20:38 -0800)]
runtime: remove trailing empty arrays in structs
The ones at the end of M and G are just used to compute
their size for use in assembly. Generate the size explicitly.
The one at the end of itab is variable-sized, and at least one.
The ones at the end of interfacetype and uncommontype are not
needed, as the preceding slice references them (the slice was
originally added for use by reflect?).
The one at the end of stackmap is already accessed correctly,
and the runtime never allocates one.
Update #9401
Change-Id: Ia75e3aaee38425f038c506868a17105bd64c712f
Reviewed-on: https://go-review.googlesource.com/2420 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Keith Randall [Tue, 6 Jan 2015 17:06:44 +0000 (09:06 -0800)]
runtime: use some startup randomness in the fallback hashes
Fold in some startup randomness to make the hash vary across
different runs. This helps prevent attackers from choosing
keys that all map to the same bucket.
Also, reorganize the hash a bit. Move the *m1 multiply to after
the xor of the current hash and the message. For hash quality
it doesn't really matter, but for DDOS resistance it helps a lot
(any processing done to the message before it is merged with the
random seed is useless, as it is easily inverted by an attacker).
Update #9365
Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24
Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Matthew Dempsky [Tue, 6 Jan 2015 23:08:02 +0000 (15:08 -0800)]
cmd/cgo: update code and docs to reflect post-6c world
The gc toolchain no longer includes a C compiler, so mentions of "6c"
can be removed or replaced by 6g as appropriate. Similarly, some cgo
functions that previously generated C source output no longer need to.
Change-Id: I1ae6b02630cff9eaadeae6f3176c0c7824e8fbe5
Reviewed-on: https://go-review.googlesource.com/2391 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Brad Fitzpatrick [Fri, 2 Jan 2015 18:02:15 +0000 (10:02 -0800)]
bufio: add Reader.Discard
Reader.Discard is the complement to Peek. It discards the next n bytes
of input.
We already have Reader.Buffered to see how many bytes of data are
sitting available in memory, and Reader.Peek to get that that buffer
directly. But once you're done with the Peek'd data, you can't get rid
of it, other than Reading it.
Both Read and io.CopyN(ioutil.Discard, bufReader, N) are relatively
slow. People instead resort to multiple blind ReadByte calls, just to
advance the internal b.r variable.
I've wanted this previously, several people have asked for it in the
past on golang-nuts/dev, and somebody just asked me for it again in a
private email. There are a few places in the standard library we'd use
it too.
Shenghou Ma [Wed, 7 Jan 2015 01:40:16 +0000 (20:40 -0500)]
runtime: fix build for race detector
This CL only fixes the build, there are two failing tests:
RaceMapBigValAccess1 and RaceMapBigValAccess2
in runtime/race tests. I haven't investigated why yet.
Russ Cox [Wed, 31 Dec 2014 03:39:48 +0000 (22:39 -0500)]
runtime: allocate wbshadow at high address
sysReserve doesn't actually reserve the full amount requested on
64-bit systems, because of problems with ulimit. Instead it checks
that it can get the first 64 kB and assumes it can grab the rest as
needed. This doesn't work well with the "let the kernel pick an address"
mode, so don't do that. Pick a high address instead.
Change-Id: I4de143a0e6fdeb467fa6ecf63dcd0c1c1618a31c
Reviewed-on: https://go-review.googlesource.com/2345 Reviewed-by: Rick Hudson <rlh@golang.org>
Russ Cox [Tue, 6 Jan 2015 18:46:13 +0000 (13:46 -0500)]
misc/cgo: disable TestAllocateFromC in wbshadow mode
This test is doing pointer graph manipulation from C, and we
cannot support that with concurrent GC. The wbshadow mode
correctly diagnoses missing write barriers.
Disable the test in that mode for now. There is a bigger issue
behind it, namely SWIG, but for now we are focused on making
all.bash pass with wbshadow enabled.
Change-Id: I55891596d4c763e39b74082191d4a5fac7161642
Reviewed-on: https://go-review.googlesource.com/2346 Reviewed-by: Minux Ma <minux@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
Adam Langley [Fri, 19 Dec 2014 23:14:03 +0000 (15:14 -0800)]
crypto/tls: fix renegotiation extension.
There are two methods by which TLS clients signal the renegotiation
extension: either a special cipher suite value or a TLS extension.
It appears that I left debugging code in when I landed support for the
extension because there's a "+ 1" in the switch statement that shouldn't
be there.
The effect of this is very small, but it will break Firefox if
security.ssl.require_safe_negotiation is enabled in about:config.
(Although almost nobody does this.)
This change fixes the original bug and adds a test. Sadly the test is a
little complex because there's no OpenSSL s_client option that mirrors
that behaviour of require_safe_negotiation.
Russ Cox [Mon, 5 Jan 2015 20:02:09 +0000 (15:02 -0500)]
runtime: fix two garbage collector bugs
First, call clearcheckmarks immediately after changing checkmark,
so that there is less time when the checkmark flag and the bitmap
are inconsistent. The tiny gap between the two lines is fine, because
the world is stopped. Before, the gap was much larger and included
such code as "go bgsweep()", which allocated.
Second, modify gcphase only when the world is stopped.
As written, gcscan_m was changing gcphase from 0 to GCscan
and back to 0 while other goroutines were running.
Another goroutine running at the same time might decide to
sleep, see GCscan, call gcphasework, and start "helping" by
scanning its stack. That's fine, except that if gcphase flips back
to 0 as the goroutine calls scanblock, it will start draining the
work buffers prematurely.
Both of these were found wbshadow=2 (and a lot of hard work).
Eventually that will run automatically, but right now it still
doesn't quite work for all.bash, due to mmap conflicts with
pthread-created threads.
Change-Id: I99aa8210cff9c6e7d0a1b62c75be32a23321897b
Reviewed-on: https://go-review.googlesource.com/2340 Reviewed-by: Rick Hudson <rlh@golang.org>
Russ Cox [Tue, 23 Dec 2014 03:50:42 +0000 (22:50 -0500)]
runtime, sync/atomic: add write barrier for atomic write of pointer
Add write barrier to atomic operations manipulating pointers.
In general an atomic write of a pointer word may indicate racy accesses,
so there is no strictly safe way to attempt to keep the shadow copy
in sync with the real one. Instead, mark the shadow copy as not used.
Redirect sync/atomic pointer routines back to the runtime ones,
so that there is only one copy of the write barrier and shadow logic.
In time we might consider doing this for most of the sync/atomic
functions, but for now only the pointer routines need that treatment.
Found with GODEBUG=wbshadow=1 mode.
Eventually that will run automatically, but right now
it still detects other missing write barriers.
Russ Cox [Tue, 23 Dec 2014 03:43:49 +0000 (22:43 -0500)]
runtime: change Gobuf.g to uintptr, not pointer
The Gobuf.g goroutine pointer is almost always updated by assembly code.
In one of the few places it is updated by Go code - func save - it must be
treated as a uintptr to avoid a write barrier being emitted at a bad time.
Instead of figuring out how to emit the write barriers missing in the
assembly manipulation, change the type of the field to uintptr, so that
it does not require write barriers at all.
Goroutine structs are published in the allg list and never freed.
That will keep the goroutine structs from being collected.
There is never a time that Gobuf.g's contain the only references
to a goroutine: the publishing of the goroutine in allg comes first.
Goroutine pointers are also kept in non-GC-visible places like TLS,
so I can't see them ever moving. If we did want to start moving data
in the GC, we'd need to allocate the goroutine structs from an
alternate arena. This CL doesn't make that problem any worse.
Found with GODEBUG=wbshadow=1 mode.
Eventually that will run automatically, but right now
it still detects other missing write barriers.
Russ Cox [Mon, 22 Dec 2014 15:53:51 +0000 (10:53 -0500)]
runtime: add GODEBUG wbshadow for finding missing write barriers
This is the detection code. It works well enough that I know of
a handful of missing write barriers. However, those are subtle
enough that I'll address them in separate followup CLs.
GODEBUG=wbshadow=1 checks for a write that bypassed the
write barrier at the next write barrier of the same word.
If a bug can be detected in this mode it is typically easy to
understand, since the crash says quite clearly what kind of
word has missed a write barrier.
GODEBUG=wbshadow=2 adds a check of the write barrier
shadow copy during garbage collection. Bugs detected at
garbage collection can be difficult to understand, because
there is no context for what the found word means.
Typically you have to reproduce the problem with allocfreetrace=1
in order to understand the type of the badly updated word.
When constants were declared using unexported constants,
the type information was lost when those constants were filtered out.
This CL propagates the type information of unexported constants
so that it is available for display.
This is a follow-up to CL 144110044, which fixed this problem
specifically for _ constants.
Updates #5397.
Change-Id: I3f0c767a4007d88169a5634ab2870deea4e6a740
Reviewed-on: https://go-review.googlesource.com/2091 Reviewed-by: Robert Griesemer <gri@golang.org>
Brad Fitzpatrick [Mon, 5 Jan 2015 21:14:08 +0000 (13:14 -0800)]
runtime: only check whether the runtime is stale once during tests
Noticed while investigating the speed of the runtime tests, as part
of debugging while Plan 9's runtime tests are timing out on GCE.
Change-Id: I95f5a3d967a0b45ec1ebf10067e193f51db84e26
Reviewed-on: https://go-review.googlesource.com/2283 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fazlul Shahriar [Fri, 2 Jan 2015 03:16:38 +0000 (22:16 -0500)]
math: be consistent in how we document special cases
Change-Id: Ic6bc4af7bcc89b2881b2b9e7290aeb6fd54804e2
Reviewed-on: https://go-review.googlesource.com/2239 Reviewed-by: Ian Lance Taylor <iant@golang.org>
I think it will remain useful to distinguish code that must
run on a system stack from code that can run on either stack,
even if that distinction is no
longer based on the implementation language.
That is, I expect to add a //go:systemstack comment that,
in terms of the old implementation, tells the compiler,
to pretend this function was written in C.
Dave Cheney [Mon, 5 Jan 2015 02:14:04 +0000 (13:14 +1100)]
crypto/x509: split certFiles definition by GOOS
This CL splits the (ever growing) list of ca cert locations by major unix
platforms (darwin, windows and plan9 are already handled seperately).
Although it is clear the unix variants cannot manage to agree on some standard
locations, we can avoid to some extent an artificial ranking of priority
amongst the supported GOOSs.
* Split certFiles definition by GOOS
* Include NetBSD ca cert location
Michael Hudson-Doyle [Thu, 18 Dec 2014 09:27:26 +0000 (22:27 +1300)]
cmd/go: be more careful when linking a test exe with gccgo
Previously, we ended up passing two compiled objects for the package
being tested when linking the test executable. Somewhat by luck, this
worked most of the time but occasionally it did not. This changes the
linking code to not pass two objects for the same ImportPath and to
always pass the object for the test version of the package and removes
some unecessary nil checks.
Change-Id: I7bbd3fc708f14672ee2cc6aed3397421fceb8a38
Reviewed-on: https://go-review.googlesource.com/1840 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Fraenkel [Fri, 2 Jan 2015 02:38:12 +0000 (21:38 -0500)]
reflect: set dir when creating a channel via ChanOf
Fixes #9135
Change-Id: I4d0e4eb52a3d64262f107eb7eae4096a6e47ac08
Reviewed-on: https://go-review.googlesource.com/2238 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Mikio Hara [Wed, 31 Dec 2014 07:46:19 +0000 (16:46 +0900)]
net: add test cases for parsing ipv4-mapped ipv6 address
This CL adds missing ipv4-mapped ipv6 address test cases to TestParseIP.
Change-Id: I3144d2a88d409bd515cf52f8711d407bfa81ed68
Reviewed-on: https://go-review.googlesource.com/2205 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Matthew Dempsky [Tue, 30 Dec 2014 20:31:17 +0000 (12:31 -0800)]
runtime: fix slicecopy return value for zero-width elements
Fixes #8620
Change-Id: Idb49e586919d21d07e94a39ed9ebb0562f403460
Reviewed-on: https://go-review.googlesource.com/2221 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Martin Möhrmann [Wed, 31 Dec 2014 20:18:59 +0000 (21:18 +0100)]
os: replace itod on posix with general itoa and fix possible infinite recursion
Remove use of itod on posix systems and replace with call to itoa.
Build and use same itoa function on all systems.
Fix infinite recursion in iota function for the case -1<<63.
Shenghou Ma [Thu, 1 Jan 2015 07:23:55 +0000 (02:23 -0500)]
include: remove unnecessary stuff on windows
Our definition of struct timespec used to cause problems with
certain versions of mingw-rt. However, as it turns out, we don't
actually need those definitions and prototypes, so remove them.
Fixes #9472.
Change-Id: Ie0880f0d58be112625140f73d0bed71f98b7cf05
Reviewed-on: https://go-review.googlesource.com/2236 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Mikio Hara [Wed, 31 Dec 2014 01:08:51 +0000 (10:08 +0900)]
net: don't return io.EOF on reading data from datagram, raw sockets on windows
Preventing returning io.EOF on non-connection oriented sockets is
already applied to Unix variants. This CL applies it to Windows.
Update #4856.
Change-Id: I82071d40f617e2962d0540b9d1d6a10ea4cdb2ec
Reviewed-on: https://go-review.googlesource.com/2203 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Mikio Hara [Wed, 31 Dec 2014 06:45:46 +0000 (15:45 +0900)]
net: remove redundant test case for lookupIP with threadLimit
There is no reason to have the redundant test case TestDNSThreadLimt
because TestLookupIPDeadline does cover what we need to test with
-dnsflood flag and more.
Also this CL moves TestLookupIPDeadline into lookup_test.go to avoid
abusing to control the order of test case execution by using file name.
Change-Id: Ib417d7d3411c59d9352c03c996704d584368dc62
Reviewed-on: https://go-review.googlesource.com/2204 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Shenghou Ma [Thu, 1 Jan 2015 01:30:57 +0000 (20:30 -0500)]
runtime/cgo: initialize our pthread_create wrapper earlier on openbsd
This is a genuine bug exposed by our test for issue 9456: our wrapper
for pthread_create is not initialized until we initialize cgo itself,
but it is possible that a static constructor could call pthread_create,
and in that case, it will be calling a nil function pointer.
Fix that by also initializing the sys_pthread_create function pointer
inside our pthread_create wrapper function, and use a pthread_once to
make sure it is only initialized once.
Fix build for openbsd.
Change-Id: Ica4da2c21fcaec186fdd3379128ef46f0e767ed7
Reviewed-on: https://go-review.googlesource.com/2232 Reviewed-by: David Crawshaw <crawshaw@golang.org>
Shenghou Ma [Wed, 31 Dec 2014 00:48:26 +0000 (19:48 -0500)]
cmd/gc: fix filename output format verb for -s
%lL will prepend the current directory to the filename, which is not
what we want here (as the file name is already absolute).
Fixes #9150.
Change-Id: I4c9386be6baf421393b92d9401a264b4692986d0
Reviewed-on: https://go-review.googlesource.com/2231 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Shenghou Ma [Sun, 28 Dec 2014 00:15:38 +0000 (19:15 -0500)]
runtime: ignore SIGPROF to foreign threads before cgocallback is fully initialized
Some libraries, for example, OpenBLAS, create work threads in a global constructor.
If we're doing cpu profiling, it's possible that SIGPROF might come to some of the
worker threads before we make our first cgo call. Cgocallback used to terminate the
process when that happens, but it's better to miss a couple profiling signals than
to abort in this case.
Fixes #9456.
Change-Id: I112b8e1a6e10e6cc8ac695a4b518c0f577309b6b
Reviewed-on: https://go-review.googlesource.com/2141 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Martin Möhrmann [Thu, 25 Dec 2014 18:30:53 +0000 (19:30 +0100)]
strconv: optimize decimal to string conversion
Avoid the decimal lookup in digits array and compute the decimal character value directly.
Reduce calls to 64bit division on 32bit plattforms by splitting conversion into smaller blocks.
Convert value to uintptr type when it can be represented by uintptr.
Shenghou Ma [Mon, 29 Dec 2014 06:08:40 +0000 (01:08 -0500)]
liblink, cmd/ld, runtime: remove stackguard1
Now that we've removed all the C code in runtime and the C compilers,
there is no need to have a separate stackguard field to check for C
code on Go stack.
Remove field g.stackguard1 and rename g.stackguard0 to g.stackguard.
Adjust liblink and cmd/ld as necessary.
Change-Id: I54e75db5a93d783e86af5ff1a6cd497d669d8d33
Reviewed-on: https://go-review.googlesource.com/2144 Reviewed-by: Keith Randall <khr@golang.org>
Emil Hessman [Sat, 27 Dec 2014 19:52:17 +0000 (20:52 +0100)]
encoding/json: address go vet reports
The error message for decoding a unquoted value into a struct field with
the ,string option specified has two arguments when one is needed.
Make the error message take one argument and add a test in order to cover
the case when a unquoted value is specified.
Also add error value as the missing argument for Fatalf call in test.
Fixes the following go vet reports:
decode.go:602: wrong number of args for format in Errorf call: 1 needed but 2 args
decode_test.go:1088: missing argument for Fatalf("%v"): format reads arg 1, have only 0 args
Martin Möhrmann [Sat, 27 Dec 2014 10:53:09 +0000 (11:53 +0100)]
strconv/itoa: add test to generate the longest output string possible by formatBits
The new test case produces the longest string representation possible and thereby uses
all of the 65 bytes in the buffer array used by the formatBits function.