Carl Shapiro [Tue, 3 Dec 2013 22:12:55 +0000 (14:12 -0800)]
runtime: move stack scanning into the parallel mark phase
This change reduces the cost of the stack scanning by frames.
It moves the stack scanning from the serial root enumeration
phase to the parallel tracing phase. The output that follows
are timings for the issue 6482 benchmark
The remaining performance disparity is due to inefficiencies
in gentraceback and its callees. The effect was isolated by
using a parallel stack scan where scanstack was modified to do
a conservative scan of the stack segments without gentraceback
followed by a call of gentrackback with a no-op callback.
The output that follows are the top-10 most frequent tops of
stacks as determined by the Linux perf record facility.
Keith Randall [Tue, 3 Dec 2013 02:03:25 +0000 (18:03 -0800)]
runtime: fix race detector when map keys/values are passed by pointer.
Now that the map implementation is reading the keys and values from
arbitrary memory (instead of from stack slots), it needs to tell the
race detector when it does so.
Fixes #6875.
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/36360043
Keith Randall [Mon, 2 Dec 2013 21:36:50 +0000 (13:36 -0800)]
reflect: prevent the callXX routines from calling makeFuncStub
and methodValueCall directly. Instead, we inline their behavior
inside of reflect.call.
This change is required because otherwise we have a situation where
reflect.callXX calls makeFuncStub, neither of which knows the
layout of the args passed between them. That's bad for
precise gc & stack copying.
Keith Randall [Mon, 2 Dec 2013 21:05:04 +0000 (13:05 -0800)]
runtime: pass key/value to map accessors by reference, not by value.
This change is part of the plan to get rid of all vararg C calls
which are a pain for getting exact stack scanning.
We allocate a chunk of zero memory to return a pointer to when a
map access doesn't find the key. This is simpler than returning nil
and fixing things up in the caller. Linker magic allocates a single
zero memory area that is shared by all (non-reflect-generated) map
types.
Passing things by reference gets rid of some copies, so it speeds
up code with big keys/values.
benchmark old ns/op new ns/op delta
BenchmarkBigKeyMap 34 31 -8.48%
BenchmarkBigValMap 37 30 -18.62%
BenchmarkSmallKeyMap 26 23 -11.28%
Andrew Gerrand [Wed, 20 Nov 2013 20:55:29 +0000 (07:55 +1100)]
doc: update installation instructions
Clarify that GOROOT should only be set when using a custom install path.
Remove NetBSD from binary install page (we don't provide binaries).
Remove "What's next" links from installation instructions.
Emphasize "How to Write Go Code" page.
Fixes #6613.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/28700043
Russ Cox [Thu, 7 Nov 2013 20:24:51 +0000 (15:24 -0500)]
cmd/cgo: fix handling of array of pointers when using clang
Clang does not record the "size" field for pointer types,
so we must insert the size ourselves. We were already
doing this, but only for the case of pointer types.
For an array of pointer types, the setting of the size for
the nested pointer type was happening after the computation
of the size of the array type, meaning that the array type
was always computed as 0 bytes. Delay the size computation.
This bug happens on all Clang systems, not just FreeBSD.
Our test checked that cgo wrote something, not that it was correct.
FreeBSD's default clang rejects array[0] as a C struct field,
so it noticed the incorrect sizes. But the sizes were incorrect
everywhere.
Update testcdefs to check the output has the right semantics.
Russ Cox [Thu, 31 Oct 2013 18:15:55 +0000 (18:15 +0000)]
cmd/5l, runtime: fix divide for profiling tracebacks on ARM
Two bugs:
1. The first iteration of the traceback always uses LR when provided,
which it is (only) during a profiling signal, but in fact LR is correct
only if the stack frame has not been allocated yet. Otherwise an
intervening call may have changed LR, and the saved copy in the stack
frame should be used. Fix in traceback_arm.c.
2. The division runtime call adds 8 bytes to the stack. In order to
keep the traceback routines happy, it must copy the saved LR into
the new 0(SP). Change
SUB $8, SP
into
MOVW 0(SP), R11 // r11 is temporary, for use by linker
MOVW.W R11, -8(SP)
to update SP and 0(SP) atomically, so that the traceback always
sees a saved LR at 0(SP).
Fixes #6681.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/19910044
The CL causes misc/cgo/test to fail randomly.
I suspect that the problem is the use of a division instruction
in usleep, which can be called while trying to acquire an m
and therefore cannot store the denominator in m.
The solution to that would be to rewrite the code to use a
magic multiply instead of a divide, but now we're getting
pretty far off the original code.
Go back to the original in preparation for a different,
less efficient but simpler fix.
««« original CL description
cmd/5l, runtime: make ARM integer division profiler-friendly
The implementation of division constructed non-standard
stack frames that could not be handled by the traceback
routines.
CL 13239052 left the frames non-standard but fixed them
for the specific case of a divide-by-zero panic.
A profiling signal can arrive at any time, so that fix
is not sufficient.
Change the division to store the extra argument in the M struct
instead of in a new stack slot. That keeps the frames bog standard
at all times.
Also fix a related bug in the traceback code: when starting
a traceback, the LR register should be ignored if the current
function has already allocated its stack frame and saved the
original LR on the stack. The stack copy should be used, as the
LR register may have been modified.
Combined, these make the torture test from issue 6681 pass.
Russ Cox [Wed, 30 Oct 2013 18:50:34 +0000 (18:50 +0000)]
cmd/5l, runtime: make ARM integer division profiler-friendly
The implementation of division constructed non-standard
stack frames that could not be handled by the traceback
routines.
CL 13239052 left the frames non-standard but fixed them
for the specific case of a divide-by-zero panic.
A profiling signal can arrive at any time, so that fix
is not sufficient.
Change the division to store the extra argument in the M struct
instead of in a new stack slot. That keeps the frames bog standard
at all times.
Also fix a related bug in the traceback code: when starting
a traceback, the LR register should be ignored if the current
function has already allocated its stack frame and saved the
original LR on the stack. The stack copy should be used, as the
LR register may have been modified.
Combined, these make the torture test from issue 6681 pass.
Russ Cox [Wed, 30 Oct 2013 14:24:42 +0000 (10:24 -0400)]
cmd/cgo: accept extra leading _ on __cgodebug_data for all object formats
The current Windows build breakage appears to be because
the Windows code should be looking for __cgodebug_data
not ___cgodebug_data. Dodge the question everywhere by
accepting both.
Russ Cox [Tue, 29 Oct 2013 21:11:51 +0000 (17:11 -0400)]
time: correct path to time zone zip file on Unix
Most Unix systems have their own time zone data,
so we almost never get far enough in the list to
discover that we cannot fall back to the zip file.
Adjust testing to exercise the final fallback.
Plan 9 and Windows were already correct
(and are the main users of the zip file).
Russ Cox [Tue, 29 Oct 2013 15:50:40 +0000 (11:50 -0400)]
os: do not return Lstat errors from Readdir
This CL restores the Go 1.1.2 semantics for os.File's Readdir method.
The code in Go 1.1.2 was rewritten mainly because it looked buggy.
This new version attempts to be clearer but still provide the 1.1.2 results.
The important diff is not this CL's version against tip but this CL's version
against Go 1.1.2.
Go 1.1.2:
names, err := f.Readdirnames(n)
fi = make([]FileInfo, len(names))
for i, filename := range names {
fip, err := Lstat(dirname + filename)
if err == nil {
fi[i] = fip
} else {
fi[i] = &fileStat{name: filename}
}
}
return fi, err
This CL:
names, err := f.Readdirnames(n)
fi = make([]FileInfo, len(names))
for i, filename := range names {
fip, lerr := lstat(dirname + filename)
if lerr != nil {
fi[i] = &fileStat{name: filename}
continue
}
fi[i] = fip
}
return fi, err
The changes from Go 1.1.2 are stylistic, not semantic:
1. Use lstat instead of Lstat, for testing (done before this CL).
2. Make error handling in loop body look more like an error case.
3. Use separate error variable name in loop body, to be clear
we are not trying to influence the final return result.
Russ Cox [Tue, 29 Oct 2013 14:36:51 +0000 (10:36 -0400)]
debug/dwarf: add DWARF 4 form constants
Some versions of clang generate DWARF 4-format attributes
even when using -gdwarf-2. We don't care much about the
values, but we do need to be able to parse past them.
This fixes a bug in Go 1.2 rc2 reported via private mail using
a near-tip version of clang.
Russ Cox [Tue, 29 Oct 2013 02:21:26 +0000 (22:21 -0400)]
cmd/cgo: stop using -fno-eliminate-unused-debug-types
This flag was added in January 2010, in CL 181102, to fix issue 497.
(Numbers were just shorter back then.) The fix was for OS X machines
and the llvm-gcc frontend.
In July 2011 we had to change the way we get enum values, because
there were no flags available to force Xcode's llvm-gcc to include the
enum names and values in DWARF debug output.
We now use clang, not llvm-gcc, on OS X machines.
Earlier versions of clang printed a warning about not knowing the flag.
Newer versions of clang now make that an error.
That is:
- The flag was added for OS X machines.
- The flag is no longer necessary on OS X machines.
- The flag now breaks some OS X machines.
Remove it.
I have run the original program from issue 497 successfully
without the flag on both OS X and Linux machines.
Russ Cox [Mon, 28 Oct 2013 23:40:40 +0000 (19:40 -0400)]
runtime: relax preemption assertion during stack split
The case can happen when starttheworld is calling acquirep
to get things moving again and acquirep gets preempted.
The stack trace is in golang.org/issue/6644.
It is difficult to build a short test case for this, but
the person who reported issue 6644 confirms that this
solves the problem.
Fixes #6644.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/18740044
Brad Fitzpatrick [Thu, 24 Oct 2013 22:51:19 +0000 (15:51 -0700)]
strings: fix Replacer bug with prefix matches
singleStringReplacer had a bug where if a string was replaced
at the beginning and no output had yet been produced into the
temp buffer before matching ended, an invalid nil check (used
as a proxy for having matched anything) meant it always
returned its input.
Fixes #6659
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/16880043
Adam Langley [Mon, 21 Oct 2013 20:35:09 +0000 (16:35 -0400)]
crypto/tls: advertise support for RSA+SHA1 in TLS 1.2 handshake.
Despite SHA256 support being required for TLS 1.2 handshakes, some
servers are aborting handshakes that don't offer SHA1 support.
This change adds support for signing TLS 1.2 ServerKeyExchange messages
with SHA1. It does not add support for signing TLS 1.2 client
certificates with SHA1 as that would require the handshake to be
buffered.
Fixes #6618.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/15650043
Russ Cox [Fri, 18 Oct 2013 19:56:25 +0000 (15:56 -0400)]
cmd/cgo: stop using compiler error message text to analyze C names
The old approach to determining whether "name" was a type, constant,
or expression was to compile the C program
name;
and scan the errors and warnings generated by the compiler.
This requires looking for specific substrings in the errors and warnings,
which ties the implementation to specific compiler versions.
As compilers change their errors or drop warnings, cgo breaks.
This happens slowly but it does happen.
Clang in particular (now required on OS X) has a significant churn rate.
The new approach compiles a slightly more complex program
that is either valid C or not valid C depending on what kind of
thing "name" is. It uses only the presence or absence of an error
message on a particular line, not the error text itself. The program is:
// error if and only if name is undeclared
void f1(void) { typeof(name) *x; }
// error if and only if name is not a type
void f2(void) { name *x; }
// error if and only if name is not an integer constant
void f3(void) { enum { x = (name)*1 }; }
I had not been planning to do this until Go 1.3, because it is a
non-trivial change, but it fixes a real Xcode 5 problem in Go 1.2,
and the new code is easier to understand than the old code.
It should be significantly more robust.
Dmitriy Vyukov [Fri, 18 Oct 2013 06:45:19 +0000 (10:45 +0400)]
runtime: remove nomemprof
Nomemprof seems to be unneeded now, there is no recursion.
If the recursion will be re-introduced, it will break loudly by deadlocking.
Fixes #6566.
Alberto García Hierro [Thu, 17 Oct 2013 16:02:32 +0000 (09:02 -0700)]
database/sql: make tests repeatable with -cpu=n,n
New test added in CL 14611045 causes a deadlock when
running the tests with -cpu=n,n because the fakedb
driver always waits when opening a new connection after
running TestConnectionLeak. Reset its state after.
Robert Griesemer [Wed, 16 Oct 2013 23:16:54 +0000 (16:16 -0700)]
spec: clarify re-use of underlying arrays in slice operations
Please note the slight rewording for append: The spec now
requires that append reuses the underlying array if it is
sufficiently large. Per majority sentiment.
This is technically a language change but the current
implementation always worked this way.