Russ Cox [Mon, 24 Mar 2014 23:11:21 +0000 (19:11 -0400)]
doc: allow buffered channel as semaphore without initialization
This rule not existing has been the source of many discussions
on golang-dev and on issues. We have stated publicly that it is
true, but we have never written it down. Write it down.
Rui Ueyama [Mon, 24 Mar 2014 18:48:34 +0000 (11:48 -0700)]
bufio: fix bug that ReadFrom stops before EOF or error
ReadFrom should not return until it receives a non-nil error
or too many contiguous (0, nil)s from a given reader.
Currently it immediately returns if it receives one (0, nil).
Fixes #7611.
David du Colombier [Fri, 21 Mar 2014 18:26:47 +0000 (19:26 +0100)]
cmd/ld: fix warnings on Plan 9
warning: src/cmd/ld/macho.c:595 sign-extended character constant
warning: src/cmd/ld/macho.c:595 sign-extended character constant
warning: src/cmd/ld/symtab.c:63 sign-extended character constant
warning: src/cmd/ld/symtab.c:63 sign-extended character constant
Rob Pike [Fri, 21 Mar 2014 02:59:30 +0000 (13:59 +1100)]
doc/go_faq.html: update description of stack management
They aren't segmented any more, at least with gc.
Also improve the comparison of goroutines and threads.
Fixes #7373.
Rémy Oudompheng [Thu, 20 Mar 2014 21:22:37 +0000 (22:22 +0100)]
cmd/6g, cmd/8g: skip CONVNOP nodes in bgen.
Revision 3ae4607a43ff introduced CONVNOP layers
to fix type checking issues arising from comparisons.
The added complexity made 8g run out of registers
when compiling an equality function in go.net/ipv6.
A similar issue occurred in test/sizeof.go on
amd64p32 with 6g.
Rui Ueyama [Thu, 20 Mar 2014 05:00:34 +0000 (16:00 +1100)]
base64: fix bug that decoder fails to detect corruption
Encoding.Decode() failed to detect trailing garbages if input contains "==" followed by garbage smaller than 3 bytes (for example, it failed to detect "x" in "AA==x"). This patch fixes the bug and adds a few tests.
Rob Pike [Wed, 19 Mar 2014 21:51:06 +0000 (08:51 +1100)]
fmt: make %F a synonym for %f
Rationale:
It already is for scanning.
It is accepted for complexes already, but doesn't work.
It's analogous to %G and %E.
C accepts it too, and we try to be roughly compatible.
Fixes #7518.
Dmitriy Vyukov [Wed, 19 Mar 2014 13:04:51 +0000 (17:04 +0400)]
runtime: fix stack split detection around fork
If runtime_BeforeFork splits stack, it will unsplit it
with spoiled g->stackguard. It leads to check failure in oldstack:
Rick Arnold [Tue, 18 Mar 2014 02:03:03 +0000 (13:03 +1100)]
build: fix race in doc/articles/wiki test
The original test would open a local port and then immediately close it
and use the port number in subsequent tests. Between the port being closed
and reused by the later process, it could be opened by some other program
on the machine.
Changed the test to run the server process directly and have it save the
assigned port to a text file to be used by client processes.
Rob Pike [Tue, 18 Mar 2014 00:25:04 +0000 (11:25 +1100)]
fmt: document GoStringer and explain application of formats to compound objects
%q quotes each element of a string slice; this was never explained in the docs.
Fixes #7015.
Catalin Patulea [Mon, 17 Mar 2014 22:47:16 +0000 (15:47 -0700)]
net/http/fcgi: fix handling of request ID reuse
Request ID reuse is allowed by the FastCGI spec [1]. In particular nginx uses
the same request ID, 1, for all requests on a given connection. Because
serveRequest does not remove the request from conn.requests, this causes it to
treat the second request as a duplicate and drops the connection immediately
after beginRequest. This manifests with nginx option 'fastcgi_keep_conn on' as
the following message in nginx error log:
2014/03/17 01:39:13 [error] 730#0: *109 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: x.x.x.x, server: example.org, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9001", host: "example.org"
Because handleRecord and serveRequest run in different goroutines, access to
conn.requests must now be synchronized.
Nathan John Youngman [Sun, 16 Mar 2014 22:35:04 +0000 (09:35 +1100)]
doc: Revise Contribution Guidelines.
Smooth out the setup process for new contributors.
* Remove references $GOROOT (often not defined).
* Add a note for contributing to subrepositories.
* Emphasize that hg mail also uploads the latest copy.
Dmitriy Vyukov [Fri, 14 Mar 2014 19:32:12 +0000 (23:32 +0400)]
runtime: fix another race in bgsweep
It's possible that bgsweep constantly does not catch up for some reason,
in this case runfinq was not woken at all.
Dmitriy Vyukov [Fri, 14 Mar 2014 19:25:48 +0000 (23:25 +0400)]
runtime: fix spans corruption
The problem was that spans end up in wrong lists after split
(e.g. in h->busy instead of h->central->empty).
Also the span can be non-swept before split,
I don't know what it can cause, but it's safer to operate on swept spans.
Fixes #7544.
Shenghou Ma [Fri, 14 Mar 2014 14:07:51 +0000 (10:07 -0400)]
cmd/gc: replace '·' as '.' in ELF/Mach-O symbol tables
Old versions of DTrace (as those shipped in OS X and FreeBSD)
don't support unicode characters in symbol names. Replace '·'
to '.' to make DTrace happy.
Aram Hăvărneanu [Fri, 14 Mar 2014 13:53:05 +0000 (17:53 +0400)]
runtime: fix use after close race in Solaris network poller
The Solaris network poller uses event ports, which are
level-triggered. As such, it has to re-arm itself after each
wakeup. The arming mechanism (which runs in its own thread) raced
with the closing of a file descriptor happening in a different
thread. When a network file descriptor is about to be closed,
the network poller is awaken to give it a chance to remove its
association with the file descriptor. Because the poller always
re-armed itself, it raced with code that closed the descriptor.
This change makes the network poller check before re-arming if
the file descriptor is about to be closed, in which case it will
ignore the re-arming request. It uses the per-PollDesc lock in
order to serialize access to the PollDesc.
This change also adds extensive documentation describing the
Solaris implementation of the network poller.
Dmitriy Vyukov [Thu, 13 Mar 2014 09:25:59 +0000 (13:25 +0400)]
runtime: harden conditions when runtime panics on crash
This is especially important for SetPanicOnCrash,
but also useful for e.g. nil deref in mallocgc.
Panics on such crashes can't lead to anything useful,
only to deadlocks, hangs and obscure crashes.
This is a copy of broken but already LGTMed
https://golang.org/cl/68540043/
Dmitriy Vyukov [Thu, 13 Mar 2014 09:16:02 +0000 (13:16 +0400)]
runtime: fix stack size check
When we copy stack, we check only new size of the top segment.
This is incorrect, because we can have other segments below it.
Rémy Oudompheng [Thu, 13 Mar 2014 07:14:05 +0000 (08:14 +0100)]
cmd/gc: fix spurious type errors in walkselect.
The lowering to runtime calls introduces hidden pointers to the
arguments of select clauses. When implicit conversions were
involved it could end up with incompatible pointers. Since the
pointed-to types have the same representation, we can introduce a
forced conversion.
Anthony Martin [Thu, 13 Mar 2014 02:41:36 +0000 (19:41 -0700)]
cmd/gc: make the fpu handle all exceptions on Plan 9
The compilers expect to not be interrupted by floating
point exceptions. On Plan 9, every process starts with
interrupts enabled for invalid operation, stack overflow,
and divide by zero exceptions.
Anthony Martin [Thu, 13 Mar 2014 01:12:56 +0000 (18:12 -0700)]
os: relax the way we kill processes on Plan 9
Previously, we wrote "kill" to the process control file
to kill a program. This is problematic because it doesn't
let the program gracefully exit.
This matters especially if the process we're killing is a
Go program. On Unix, sending SIGKILL to a Go program will
automatically kill all runtime threads. On Plan 9, there
are no threads so when the program wants to exit it has to
somehow signal all of the runtime processes. It can't do
this if we mercilessly kill it by writing to it's control
file.
Instead, we now send it a note to invoke it's note handler
and let it perform any cleanup before exiting.
Anthony Martin [Thu, 13 Mar 2014 01:12:25 +0000 (18:12 -0700)]
runtime: use unoptimized memmove and memclr on Plan 9
On Plan 9, the kernel disallows the use of floating point
instructions while handling a note. Previously, we worked
around this by using a simple loop in place of memmove.
When I added that work-around, I verified that all paths
from the note handler didn't end up calling memmove. Now
that memclr is using SSE instructions, the same process
will have to be done again.
Instead of doing that, however, this CL just punts and
uses unoptimized functions everywhere on Plan 9.
Anthony Martin [Thu, 13 Mar 2014 01:10:31 +0000 (18:10 -0700)]
cmd/ld: give acid a fighting chance at unwinding the stack
Acid can't produce a stack trace without .frame symbols.
Of course, it can only unwind through linear stacks but
this is still better than nothing. (I wrote an acid func
to do the full unwind a long time ago but lost it and
haven't worked up the courage to write it again).
Note that these will only be present in the native symbol
table for Plan 9 binaries.
Dmitriy Vyukov [Wed, 12 Mar 2014 06:21:34 +0000 (10:21 +0400)]
runtime: efence support for growable stacks
1. Fix the bug that shrinkstack returns memory to heap.
This causes growslice to misbehave (it manually initialized
blocks, and in efence mode shrinkstack's free leads to
partially-initialized blocks coming out of growslice.
Which in turn causes GC to crash while treating the garbage
as Eface/Iface.
2. Enable efence for stack segments.
Dmitriy Vyukov [Wed, 12 Mar 2014 06:20:58 +0000 (10:20 +0400)]
runtime: temporary weaken a check in test
Currently the test fails as:
$ go test -v -cpu 1,1,1,1 runtime -test.run=TestStack
stack_test.go:1584: Stack inuse: want 4194304, got 18446744073709547520
Russ Cox [Wed, 12 Mar 2014 03:58:39 +0000 (23:58 -0400)]
runtime: fix empty string handling in garbage collector
The garbage collector uses type information to guide the
traversal of the heap. If it sees a field that should be a string,
it marks the object pointed at by the string data pointer as
visited but does not bother to look at the data, because
strings contain bytes, not pointers.
If you save s[len(s):] somewhere, though, the string data pointer
actually points just beyond the string data; if the string data
were exactly the size of an allocated block, the string data
pointer would actually point at the next block. It is incorrect
to mark that next block as visited and not bother to look at
the data, because the next block may be some other type
entirely.
The fix is to ignore strings with zero length during collection:
they are empty and can never become non-empty: the base
pointer will never be used again. The handling of slices already
does this (but using cap instead of len).
This was not a bug in Go 1.2, because until January all string
allocations included a trailing NUL byte not included in the
length, so s[len(s):] still pointed inside the string allocation
(at the NUL).
This bug was causing the crashes in test/run.go. Specifically,
the parsing of a regexp in package regexp/syntax allocated a
[]syntax.Inst with rounded size 1152 bytes. In fact it
allocated many such slices, because during the processing of
test/index2.go it creates thousands of regexps that are all
approximately the same complexity. That takes a long time, and
test/run works on other tests in other goroutines. One such
other test is chan/perm.go, which uses an 1152-byte source
file. test/run reads that file into a []byte and then calls
strings.Split(string(src), "\n"). The string(src) creates an
1152-byte string - and there's a very good chance of it
landing next to one of the many many regexp slices already
allocated - and then because the file ends in a \n,
strings.Split records the tail empty string as the final
element in the slice. A garbage collection happens at this
point, the collection finds that string before encountering
the []syntax.Inst data it now inadvertently points to, and the
[]syntax.Inst data is not scanned for the pointers that it
contains. Each syntax.Inst contains a []rune, those are
missed, and the backing rune arrays are freed for reuse. When
the regexp is later executed, the runes being searched for are
no longer runes at all, and there is no match, even on text
that should match.
On 64-bit machines the pointer in the []rune inside the
syntax.Inst is larger (along with a few other pointers),
pushing the []syntax.Inst backing array into a larger size
class, avoiding the collision with chan/perm.go's
inadvertently sized file.
I expect this was more prevalent on OS X than on Linux or
Windows because those managed to run faster or slower and
didn't overlap index2.go with chan/perm.go as often. On the
ARM systems, we only run one errorcheck test at a time, so
index2 and chan/perm would never overlap.
It is possible that this bug is the root cause of other crashes
as well. For now we only know it is the cause of the test/run crash.
Russ Cox [Wed, 12 Mar 2014 03:58:24 +0000 (23:58 -0400)]
test/run: make errorcheck tests faster
Some of the errorcheck tests have many many identical regexps.
Use a map to avoid storing the compiled form many many times
in memory. Change the filterRe to a simple string to avoid
the expense of those regexps as well.
Cuts the time for run.go on index2.go by almost 50x.
Mikio Hara [Wed, 12 Mar 2014 01:33:09 +0000 (10:33 +0900)]
runtime: make use of THREAD_SHARE userspace mutex on freebsd
For now Note, futexsleep and futexwakeup are designed for threads,
not for processes. The explicit use of UMTX_OP_WAIT_UINT_PRIVATE and
UMTX_OP_WAKE_PRIVATE can avoid unnecessary traversals of VM objects,
to hit undiscovered bugs related to VM system on SMP/SMT/NUMA
environment.