Dmitriy Vyukov [Tue, 12 Aug 2014 21:02:01 +0000 (01:02 +0400)]
runtime/pprof: fix data race
It's unclear why we do this broken double-checked locking.
The mutex is not held for the whole duration of CPU profiling.
Fixes #8365.
This function does not have a declaration/prototype in a.h, and it is used only
in buf.c, so it is local to it and thus can be marked as private by adding
'static' to it.
Joel Stemmer [Fri, 8 Aug 2014 19:42:20 +0000 (12:42 -0700)]
time: Fix missing colon when formatting time zone offsets with seconds
When formatting time zone offsets with seconds using the stdISO8601Colon
and stdNumColon layouts, the colon was missing between the hour and minute
parts.
Fixes #8497.
LGTM=r
R=golang-codereviews, iant, gobot, r
CC=golang-codereviews
https://golang.org/cl/126840043
Dmitriy Vyukov [Fri, 8 Aug 2014 16:52:11 +0000 (20:52 +0400)]
runtime: bump MaxGcprocs to 32
There was a number of improvements related to GC parallelization:
1. Parallel roots/stacks scanning.
2. Parallel stack shrinking.
3. Per-thread workbuf caches.
4. Workset reduction.
Currently 32 threads work well.
go.benchmarks:garbage benchmark on 2 x Intel Xeon E5-2690 (16 HT cores)
Dmitriy Vyukov [Fri, 8 Aug 2014 16:13:57 +0000 (20:13 +0400)]
runtime: fix data race in stackalloc
Stack shrinking happens during mark phase,
and it assumes that it owns stackcache in mcache.
Stack cache flushing also happens during mark phase,
and it accesses stackcache's w/o any synchronization.
This leads to stackcache corruption:
http://goperfd.appspot.com/log/309af5571dfd7e1817259b9c9cf9bcf9b2c27610
Russ Cox [Fri, 8 Aug 2014 15:20:45 +0000 (11:20 -0400)]
syscall: ignore EINVAL/ENOENT from readdirent on OS X 10.10
On OS X 10.10 Yosemite, if you have a directory that can be returned
in a single getdirentries64 call (for example, a directory with one file),
and you read from the directory at EOF twice, you get EOF both times:
fd = open("dir")
getdirentries64(fd) returns data
getdirentries64(fd) returns 0 (EOF)
getdirentries64(fd) returns 0 (EOF)
But if you remove the file in the middle between the two calls, the
second call returns an error instead.
fd = open("dir")
getdirentries64(fd) returns data
getdirentries64(fd) returns 0 (EOF)
remove("dir/file")
getdirentries64(fd) returns ENOENT/EINVAL
Whether you get ENOENT or EINVAL depends on exactly what was
in the directory. It is deterministic, just data-dependent.
This only happens in small directories. A directory containing more data
than fits in a 4k getdirentries64 call will return EOF correctly.
(It's not clear if the criteria is that the directory be split across multiple
getdirentries64 calls or that it be split across multiple file system blocks.)
We could change package os to avoid the second read at EOF,
and maybe we should, but that's a bit involved.
For now, treat the EINVAL/ENOENT as EOF.
With this CL, all.bash passes on my MacBook Air running
OS X 10.10 (14A299l) and Xcode 6 beta 5 (6A279r).
I tried filing an issue with Apple using "Feedback Assistant", but it was
unable to send the report and lost it.
C program reproducing the issue, also at http://swtch.com/~rsc/readdirbug.c:
Ian Lance Taylor [Thu, 7 Aug 2014 19:38:39 +0000 (12:38 -0700)]
cmd/go: pass --build-id=none when generating a cgo .o
Some systems, like Ubuntu, pass --build-id when linking. The
effect is to put a note in the output file. This is not
useful when generating an object file with the -r option, as
it eventually causes multiple build ID notes in the final
executable, all but one of which are for tiny portions of the
file and are therefore useless.
Disable that by passing an explicit --build-id=none when
linking with -r on systems that might do this.
Dmitriy Vyukov [Thu, 7 Aug 2014 17:39:32 +0000 (21:39 +0400)]
encoding/gob: make benchmarks parallel
There are lots of internal synchronization in gob,
so it makes sense to have parallel benchmarks.
Also add a benchmark with slices and interfaces.
LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/115960043
Russ Cox [Thu, 7 Aug 2014 16:33:19 +0000 (12:33 -0400)]
go/build: look in $GOROOT/src/cmd/foo/bar for import cmd/foo/bar
This lets us have non-main packages like cmd/internal or cmd/nm/internal/whatever.
The src/pkg migration (see golang.org/s/go14mainrepo) will allow this
as a natural side effect. The explicit change here just allows use of the
effect a little sooner.
Russ Cox [Thu, 7 Aug 2014 16:33:06 +0000 (12:33 -0400)]
cmd/addr2line, cmd/nm: factor object reading into cmd/internal/objfile
To do in another CL: make cmd/objdump use cmd/internal/objfile too.
There is a package placement decision in this CL:
cmd/internal/objfile instead of internal/objfile.
I chose to put internal under cmd to make clear (and enforce)
that no standard library packages should use this
(it's a bit dependency-heavy).
Peter Collingbourne [Thu, 7 Aug 2014 13:00:02 +0000 (09:00 -0400)]
cmd/cc, runtime: eliminate use of the unnamed substructure C extension
Eliminating use of this extension makes it easier to port the Go runtime
to other compilers. This CL also disables the extension in cc to prevent
accidental use.
Dmitriy Vyukov [Thu, 7 Aug 2014 09:04:04 +0000 (13:04 +0400)]
runtime: remove mal/malloc/FlagNoGC/FlagNoInvokeGC
FlagNoGC is unused now.
FlagNoInvokeGC is unneeded as we don't invoke GC
on g0 and when holding locks anyway.
mal/malloc have very few uses and you never remember
the exact set of flags they use and the difference between them.
Moreover, eventually we need to give exact types to all allocations,
something what mal/malloc do not support.
Russ Cox [Wed, 6 Aug 2014 22:00:06 +0000 (18:00 -0400)]
encoding/xml: add InputOffset method to Decoder
Among other things, this allows users to match the decoded
pieces with the original XML, which can be necessary for
implementing standards like XML signatures.
Shenghou Ma [Wed, 6 Aug 2014 04:24:55 +0000 (00:24 -0400)]
runtime: get rid of SA_RESTORER on ARM.
The manpages says SA_RESTORER is obsolete, and indeed, not every architecture
support it. However, sadly it's required on x86_64, see http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/x86/kernel/signal.c?id=26bcd8b72563b4c54892c4c2a409f6656fb8ae8b#n430, so only use it on x86.
Rui Ueyama [Wed, 6 Aug 2014 04:10:07 +0000 (21:10 -0700)]
liblink: shorter encoding for zeroing register
Encode MOV $0, %ax as XOR %eax, %eax instead of
XOR %rax, %rax. If an operand register does not
need REX.w bit (i.e. not one of R8-R15), it is
encoded in 2 bytes instead of 3 bytes.
Matthew Dempsky [Wed, 6 Aug 2014 01:16:56 +0000 (18:16 -0700)]
cmd/cgo: fix recursive type mapping
Instead of immediately completing pointer type mappings, add them to
a queue to allow them to be completed later. This fixes issues caused
by Type() returning arbitrary in-progress type mappings.
Matthew Dempsky [Wed, 6 Aug 2014 01:12:32 +0000 (18:12 -0700)]
cmd/cgo: fix handling of defs_linux.go
Instead of including <sys/types.h> to get size_t, instead include
the ISO C standard <stddef.h> header, which defines fewer additional
types at risk of colliding with the user code. In particular, this
prevents collisions between <sys/types.h>'s userspace definitions with
the kernel definitions needed by defs_linux.go.
Also, -cdefs mode uses #pragma pack, so we can keep misaligned fields.
Ian Lance Taylor [Wed, 6 Aug 2014 00:10:15 +0000 (17:10 -0700)]
cmd/cgo: for -godefs, promote first field of anonymous union
Update #6677
When a struct contains an anonymous union, use the type and
name of the first field in the union.
This should make the glibc <sys/resource.h> file work; in that
file struct rusage has fields like
__extension__ union
{
long int ru_maxrss;
__syscall_slong_t __ru_maxrss_word;
};
in which the field that matters is ru_maxrss and
__ru_maxrss_word just exists to advance to the next field on
systems where the kernel uses long long fields but userspace
expects long fields.
Dmitriy Vyukov [Tue, 5 Aug 2014 21:50:37 +0000 (01:50 +0400)]
runtime: cache one GC workbuf in thread-local storage
We call scanblock for lots of small root pieces
e.g. for every stack frame args and locals area.
Every scanblock invocation calls getempty/putempty,
which accesses lock-free stack shared among all worker threads.
One-element local cache allows most scanblock calls
to proceed without accessing the shared stack.