runtime: fix race detector running Go code on g0 of non-main thread
It looks like this has just always been broken:
the race detector handles running Go code on g0 of the main thread
and on g0 of any extra threads created by non-Go code, but it does
not handle running Go code on g0 of non-main threads created by Go.
Handle that.
Should fix the race build failures on the dashboard.
We're running into this now because we are running more
and more Go code on g0.
Alex Brainman [Tue, 2 Sep 2014 01:54:24 +0000 (11:54 +1000)]
time: update generated zoneinfo_abbrs_windows.go
The file in repo has been updated recently, but all these changes
are gone off the web site now. It seems web site gets updated once
in a while, so we'll update our file occasionally.
LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/140780043
runtime/race: better handling of atomic operations
This change fixes the last known false negative of the race detector --
detection of races between mutating atomic operations and non-atomic operations.
Race runtime already has functions for precise modelling of various atomic operations,
so this change just forwards all atomic ops to race runtime
instead of poor man modeling in sync/atomic package.
Performance is also improved -- full sync/atomic tests run in 60s instead of 85s now.
This is needed for callers to be able to keep track of the
writing position within a zip file. Otherwise it's not
possible to compute the size of headers, and the TOC isn't
written until the very end.
runtime: paste mprof.goc into mprof.go as comments
NO CODE CHANGES HERE
The first conversion was not complete.
This CL doesn't make any actual changes,
but it inserts the missing mprof.goc code
as comments so that the next CL will have
useful diffs.
To make the diffs a bit more useful, removed
all semicolons, ->, and runtime· prefixes as well.
Also corrected order of a few functions in mprof.go
to match original order in mprof.goc.
LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews, iant, khr, r
https://golang.org/cl/134200043
Ian Lance Taylor [Mon, 1 Sep 2014 02:59:43 +0000 (22:59 -0400)]
misc/cgo/testcdefs: comment out test of packed structs
The [568]c compilers no longer support packed structs, so
using them with -cdefs no longer works. Just commenting out
the test, rather than removing it, in case this needs to be
handled. It may be that -cdefs can go away entirely in the
future, in which case so can this directory.
Ian Lance Taylor [Sun, 31 Aug 2014 01:15:55 +0000 (18:15 -0700)]
runtime: fix Linux build
Make the definition of the EpollEvent data field consistent
across architectures, adapt the other use of it in
netpoll_epoll for the new definition, and use uint64 rather
than uintptr.
LGTM=dave
R=rsc, dave
CC=golang-codereviews
https://golang.org/cl/137890043
Dmitriy Vyukov [Sat, 30 Aug 2014 18:38:54 +0000 (22:38 +0400)]
runtime/pprof: make CPU profiling tests more robust
Under the race detector most of the samples go into race runtime,
because of that freebsd race builder constantly fails on this test.
Kelsey Hightower [Sat, 30 Aug 2014 05:19:30 +0000 (22:19 -0700)]
net/http: add BasicAuth method to *http.Request
The net/http package supports setting the HTTP Authorization header
using the Basic Authentication Scheme as defined in RFC 2617, but does
not provide support for extracting the username and password from an
authenticated request using the Basic Authentication Scheme.
Add BasicAuth method to *http.Request that returns the username and
password from authenticated requests using the Basic Authentication
Scheme.
Russ Cox [Sat, 30 Aug 2014 04:56:52 +0000 (00:56 -0400)]
runtime: increase nosplit area to 192
In CL 131450043, which raised it to 160,
I'd raise it to 192 if necessary.
Apparently it is necessary on windows/amd64.
One note for those concerned about the growth:
in the old segmented stack world, we wasted this much
space at the bottom of every stack segment.
In the new contiguous stack world, each goroutine has
only one stack segment, so we only waste this much space
once per goroutine. So even raising the limit further might
still be a net savings.
Dmitriy Vyukov [Sat, 30 Aug 2014 04:40:56 +0000 (08:40 +0400)]
runtime: convert type algorithms to Go
Actually it mostly deletes code -- alg.print and alg.copy go away.
There was only one usage of alg.print for debug purposes.
Alg.copy is used in chan.goc, but Keith replaces them with
memcopy during conversion, so alg.copy is not needed as well.
Converting them would be significant amount of work
for no visible benefit.
Alex A Skinner [Sat, 30 Aug 2014 04:12:28 +0000 (13:12 +0900)]
net: implement query-response fast failover in builtin dns stub resolver
Speed improvements via code cleanup, and changes to make go dns behave more like glibc resolver. See https://groups.google.com/forum/#!topic/golang-dev/lV-0aHqxVeo
Fixes #6579.
Benchmark results on linux/amd64
benchmark old ns/op new ns/op delta
BenchmarkGoLookupIP 48319032572937 -46.75%
BenchmarkGoLookupIPNoSuchHost 101141052419641 -76.08%
BenchmarkGoLookupIPWithBrokenNameServer 200077356245004490730 -74.99%
benchmark old allocs new allocs delta
BenchmarkGoLookupIP 287 288 0.35%
BenchmarkGoLookupIPNoSuchHost 204 102 -50.00%
BenchmarkGoLookupIPWithBrokenNameServer 410 358 -12.68%
benchmark old bytes new bytes delta
BenchmarkGoLookupIP 13181 13271 0.68%
BenchmarkGoLookupIPNoSuchHost 17260 8714 -49.51%
BenchmarkGoLookupIPWithBrokenNameServer 28160 22432 -20.34%
Adam Langley [Fri, 29 Aug 2014 19:36:30 +0000 (12:36 -0700)]
crypto: add Signer
Signer is an interface to support opaque private keys.
These keys typically result from being kept in special hardware
(i.e. a TPM) although sometimes operating systems provide a
similar interface using process isolation for security rather
than hardware boundaries.
This changes provides interfaces for representing them and
alters crypto/tls so that client certificates can use
opaque keys.
Rob Pike [Fri, 29 Aug 2014 17:40:45 +0000 (10:40 -0700)]
text/template/parse: restore pointer-only receivers for Type on Dot and Nil
Needless except that the api tool complains. We could fix that issue instead.
Russ Cox [Fri, 29 Aug 2014 17:22:31 +0000 (13:22 -0400)]
runtime: run runtime.init
Run it right before main.init.
There is still some runtime initialization that
happens before runtime.init, and some of that
may call into Go code (for example to acquire locks)
so this timing is not perfect, but I believe it is the
best we can do.
This came up because global variables intialized
to func values are done in the generated init code,
not in the linker.
LGTM=dvyukov
R=dvyukov
CC=golang-codereviews, iant, khr, r
https://golang.org/cl/135210043
(That's nearly exactly what runtime code looked like in C before
I added runtime·printf.)
This CL recognizes the unexported type runtime.hex as an integer
that should be printed in hexadecimal instead of decimal.
It's a little kludgy, but it's restricted to package runtime.
Other packages can define type hex with no effect at all.
Now we can translate that original printf as the more compact:
Rob Pike [Fri, 29 Aug 2014 16:54:00 +0000 (09:54 -0700)]
text/template: add back pointer to Nodes for better error generation
ErrorContext now has all the information it needs from the Node,
rather than depending on the template that contains it. This makes
it easier for html/template to generate correct locations in its
error messages.
Updated html/template to use this ability where it is easy, which is
not everywhere, but more work can probably push it through.
Dmitriy Vyukov [Fri, 29 Aug 2014 14:44:38 +0000 (18:44 +0400)]
runtime: clean up GC code
Remove C version of GC.
Convert freeOSMemory to Go.
Restore g0 check in GC.
Remove unknownGCPercent check in GC,
it's initialized explicitly now.
Dmitriy Vyukov [Fri, 29 Aug 2014 08:44:07 +0000 (12:44 +0400)]
runtime: allow to call stdcall from Go on windows
I've started with just one function with 8 arguments,
but stdcall is called from nosplit functions
and 8 args overflow nosplit area.
Dmitriy Vyukov [Fri, 29 Aug 2014 07:08:10 +0000 (11:08 +0400)]
runtime: convert forcegc helper to Go
Also fix a bunch of bugs:
1. Accesses to last_gc must be atomic (it's int64).
2. last_gc still can be 0 during first checks in sysmon, check for 0.
3. forcegc.g can be unitialized when sysmon accesses it:
forcegc.g is initialized by main goroutine (forcegc.g = newproc1(...)),
and main goroutine is unsynchronized with both sysmon and forcegc goroutine.
Initialize forcegc.g in the forcegc goroutine itself instead.
Matthew Dempsky [Fri, 29 Aug 2014 02:40:57 +0000 (22:40 -0400)]
misc/cgo/test: disable issue 8428 regress test on darwin
Clang 3.2 and older (as shipped with OS X Mountain Lion and older)
outputs ambiguous DWARF debug info that makes it impossible for us to
reconstruct accurate type information as required for this test.
Matthew Dempsky [Thu, 28 Aug 2014 21:23:25 +0000 (14:23 -0700)]
runtime: fix openbsd/386
In revision 05c3fee13eb3, openbsd/386's tfork implementation was
accidentally changed in one instruction from using the "params"
parameter to using the "psize" parameter.
While here, OpenBSD's __tfork system call returns a pid_t which is an
int32 on all OpenBSD architectures, so change runtime.tfork's return
type from int64 to int32 and update the assembly implementations
accordingly.
Robert Griesemer [Thu, 28 Aug 2014 19:16:06 +0000 (12:16 -0700)]
go/token: implement PositionFor accessors
Package addition.
PositionFor permits access to both, positions
adjusted by //line comments (like the Position
accessors), and unadjusted "raw" positions
unaffected by //line comments.
Raw positions are required for correct formatting
of source code via go/printer which until now had
to manually fix adjusted positions.
Robert Griesemer [Thu, 28 Aug 2014 15:53:25 +0000 (08:53 -0700)]
spec: move Method expr/value section near selectors
Preparation for fixing issue 5769 (method selectors
do not auto-dereference): The actual fix may require
some cleanups in all these sections, and syntactically,
method expressions and method values are selector
expressions. Moving them next to each other so that
it's easy to see the actual changes (next CL).
No content changes besides the section moves.
LGTM=iant, rsc
R=r, rsc, iant, ken
CC=golang-codereviews
https://golang.org/cl/132300043
Russ Cox [Thu, 28 Aug 2014 15:45:30 +0000 (11:45 -0400)]
runtime: fix build
'range hash' makes a copy of the hash array in the stack, creating
a very large stack frame. It's just the right amount that it
uses most but not all of the total stack size. If you have a lot
of environment variables, like the builders, then this is too
much and the g0 stack runs out of space.
Russ Cox [Thu, 28 Aug 2014 03:41:43 +0000 (23:41 -0400)]
runtime: fix nacl/386 build
I changed all the NACL_SYSJMP to NACL_SYSCALL in
an earlier CL, but I missed the fact that NACL_SYSCALL
will push another return PC on the stack, so that the
arguments will no longer be in the right place.
Since we have to make our own call, we also have to
copy the arguments. Do that.
Russ Cox [Thu, 28 Aug 2014 03:41:27 +0000 (23:41 -0400)]
cmd/cc: translate C Eface type as Go interface{}
This will allow structs containing Efaces in C to be
manipulated as structs containing real interfaces in Go.
The eface struct is still defined for use by Go code.
LGTM=iant
R=golang-codereviews, iant
CC=dvyukov, golang-codereviews, khr, r
https://golang.org/cl/133980044
Russ Cox [Thu, 28 Aug 2014 03:32:49 +0000 (23:32 -0400)]
runtime: rename Lock to Mutex
Mutex is consistent with package sync, and when in the
unexported Go form it avoids having a conflcit between
the type (now mutex) and the function (lock).
LGTM=iant
R=golang-codereviews, iant
CC=dvyukov, golang-codereviews, r
https://golang.org/cl/133140043
Michael Hudson-Doyle [Thu, 28 Aug 2014 03:18:56 +0000 (20:18 -0700)]
debug/elf: support arm64 relocations
This adds the minimal support for AArch64/arm64 relocations
needed to get cgo to work (when an isomorphic patch is applied
to gccgo) and a test.
This change uses the "AAarch64" name for the architecture rather
than the more widely accepted "arm64" because that's the name that
the relevant docs from ARM such as
NaCl requires the addition of a 32-byte "halt sled" at the end
of the text segment. This means that segtext.len is actually
32 bytes shorter than reality. The computation of the file offset
of the end of the data segment did not take this 32 bytes into
account, so if len and len+32 rounded up (by 64k) to different
values, the symbol table overwrote the last page of the data
segment.
The last page of the data segment is usually the C .string
symbols, which contain the strings used in error prints
by the runtime. So when this happens, your program
probably crashes, and then when it does, you get binary
garbage instead of all the usual prints.
The chance of hitting this with a randomly sized text segment
is 32 in 65536, or 1 in 2048.
If you add or remove ANY code while trying to debug this
problem, you're overwhelmingly likely to bump the text
segment one way or the other and make the bug disappear.
Correct all the computations to use segdata.fileoff+segdata.filelen
instead of trying to rederive segdata.fileoff.
This fixes the failure during the nacl/amd64p32 build.
Russ Cox [Thu, 28 Aug 2014 02:50:08 +0000 (22:50 -0400)]
runtime: fix nacl build
The NaCl "system calls" were assumed to have a compatible
return convention with the C compiler, and we were using
tail jumps to those functions. Don't do that anymore.
Correct mistake introduced in newstackcall duringconversion
from (SP) to (FP) notation. (Actually this fix, in asm_amd64p32.s,
slipped into the C compiler change, but update the name to
match what go vet wants.)
Correct computation of caller stack pointer in morestack:
on amd64p32, the saved PC is the size of a uintreg, not uintptr.
This may not matter, since it's been like this for a while,
but uintreg is the correct one. (And on non-NaCl they are the same.)
This will allow the NaCl build to get much farther.
It will probably still not work completely.
There's a bug in 6l that needs fixing too.
Russ Cox [Wed, 27 Aug 2014 18:08:26 +0000 (14:08 -0400)]
runtime: give nosplit functions 32 more bytes of headroom
The Go calling convention uses more stack space than C.
On 64-bit systems we've been right up against the limit
(128 bytes, so only 16 words) and doing awful things to
our source code to work around it. Instead of continuing
to do awful things, raise the limit to 160 bytes.
I am prepared to raise the limit to 192 bytes if necessary,
but I think this will be enough.
Should fix current link-time stack overflow errors on
- nacl/arm
- netbsd/amd64
- openbsd/amd64
- solaris/amd64
- windows/amd64