Dmitriy Vyukov [Wed, 29 May 2013 07:49:45 +0000 (11:49 +0400)]
runtime: make notetsleep() return false if timeout happens
This is needed for preemptive scheduler, because during
stoptheworld we want to wait with timeout and re-preempt
M's on timeout.
Carl Shapiro [Wed, 29 May 2013 00:59:10 +0000 (17:59 -0700)]
cmd/5l, cmd/6l, cmd/8l, cmd/gc, runtime: generate and use bitmaps of argument pointer locations
With this change the compiler emits a bitmap for each function
covering its stack frame arguments area. If an argument word
is known to contain a pointer, a bit is set. The garbage
collector reads this information when scanning the stack by
frames and uses it to ignores locations known to not contain a
pointer.
Dmitriy Vyukov [Tue, 28 May 2013 18:14:47 +0000 (22:14 +0400)]
runtime: make mheap statically allocated again
This depends on: 9791044: runtime: allocate page table lazily
Once page table is moved out of heap, the heap becomes small.
This removes unnecessary dereferences during heap access.
No logical changes.
Dmitriy Vyukov [Tue, 28 May 2013 18:04:34 +0000 (22:04 +0400)]
runtime: allocate page table lazily
This removes the 256MB memory allocation at startup,
which conflicts with ulimit.
Also will allow to eliminate an unnecessary memory dereference in GC,
because the page table is usually mapped at known address.
Update #5049.
Update #5236.
Dmitriy Vyukov [Tue, 28 May 2013 17:09:27 +0000 (21:09 +0400)]
os/exec: fix test hang
Currently the test closes random files descriptors,
which leads to hang (in particular if netpoll fd is closed).
Try to open only fd 3, since the parent process expects it to be fd 3 anyway.
Fixes #5571.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9778048
Dmitriy Vyukov [Tue, 28 May 2013 15:17:47 +0000 (19:17 +0400)]
runtime: fix heap corruption during GC
The 'n' variable is used during rescan initiation in GC_END case,
but it's overwritten with chan capacity in GC_CHAN case.
As the result rescan is done with the wrong object size.
Fixes #5554.
««« original CL description
runtime: introduce helper persistentalloc() function
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.
Dmitriy Vyukov [Tue, 28 May 2013 06:47:35 +0000 (10:47 +0400)]
runtime: introduce helper persistentalloc() function
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.
Jan Ziak [Mon, 27 May 2013 06:11:59 +0000 (08:11 +0200)]
runtime: flag static variables as no-pointers
Variables in data sections of 32-bit executables interfere with
garbage collector's ability to free objects and/or unnecessarily
slow down the garbage collector.
This changeset moves some static variables to .noptr sections.
'files' in symtab.c is now allocated dynamically.
Rob Pike [Fri, 24 May 2013 22:49:26 +0000 (15:49 -0700)]
fmt.Printf: introduce notation for random access to arguments.
This text is added to doc.go:
Explicit argument indexes:
In Printf, Sprintf, and Fprintf, the default behavior is for each
formatting verb to format successive arguments passed in the call.
However, the notation [n] immediately before the verb indicates that the
nth one-indexed argument is to be formatted instead. The same notation
before a '*' for a width or precision selects the argument index holding
the value. After processing a bracketed expression [n], arguments n+1,
n+2, etc. will be processed unless otherwise directed.
For example,
fmt.Sprintf("%[2]d %[1]d\n", 11, 22)
will yield "22, 11", while
fmt.Sprintf("%[3]*[2].*[1]f", 12.0, 2, 6),
equivalent to
fmt.Sprintf("%6.2f", 12.0),
will yield " 12.00". Because an explicit index affects subsequent verbs,
this notation can be used to print the same values multiple times
by resetting the index for the first argument to be repeated:
fmt.Sprintf("%d %d %#[1]x %#x", 16, 17)
will yield "16 17 0x10 0x11".
The notation chosen differs from that in C, but I believe it's easier to read
and to remember (we're indexing the arguments), and compatibility with
C's printf was never a strong goal anyway.
While we're here, change the word "field" to "arg" or "argument" in the
code; it was being misused and was confusing.
David du Colombier [Fri, 24 May 2013 20:55:19 +0000 (13:55 -0700)]
build: fix make.rc on Plan 9
Set $status as null to prevent rc from exiting
on the last --no-banner argument checking when
used with rc -e. It allows all.rc to not exit
before executing run.rc
Dmitriy Vyukov [Fri, 24 May 2013 14:35:48 +0000 (18:35 +0400)]
log/syslog: fix deadlock in test
The problem was that server handlers block on done<-,
the goroutine that reads from done blocks on count<-,
and the main goroutine that is supposed to read from count
waits for server handlers to exit.
Fixes #5547.
Dominik Honnef [Thu, 23 May 2013 17:19:03 +0000 (10:19 -0700)]
misc/emacs: Do not modify kill ring when programmatically deleting text
Operations like gofmt and go-remove-unused-imports delete entire
lines of text. Previously this put them on the kill-ring,
negatively affecting user experience.
Daniel Morsing [Thu, 23 May 2013 16:29:19 +0000 (18:29 +0200)]
io: Prioritize WriterTos over ReaderFroms in Copy.
This only affects calls where both ReaderFrom and WriterTo are implemented. WriterTo can issue one large write, while ReaderFrom must Read until EOF, potentially reallocating when out of memory. With one large Write, the Writer only needs to allocate once.
This also helps in ioutil.Discard since we can avoid copying memory when the Reader implements WriterTo.
Dmitriy Vyukov [Wed, 22 May 2013 19:04:46 +0000 (23:04 +0400)]
runtime: properly synchronize GC and finalizer goroutine
This is needed for preemptive scheduler, because the goroutine
can be preempted at surprising points.
Dmitriy Vyukov [Wed, 22 May 2013 18:57:47 +0000 (22:57 +0400)]
runtime: detect deadlocks in programs using cgo
When cgo is used, runtime creates an additional M to handle callbacks on threads not created by Go.
This effectively disabled deadlock detection, which is a right thing, because Go program can be blocked
and only serve callbacks on external threads.
This also disables deadlock detection under race detector, because it happens to use cgo.
With this change the additional M is created lazily on first cgo call. So deadlock detector
works for programs that import "C", "net" or "net/http/pprof" but do not use them in fact.
Also fixes deadlock detector under race detector.
It should be fine to create the M later, because C code can not call into Go before first cgo call,
because C code does not know when Go initialization has completed. So a Go program need to call into C
first either to create an external thread, or notify a thread created in global ctor that Go
initialization has completed.
Fixes #4973.
Fixes #5475.
Dmitriy Vyukov [Wed, 22 May 2013 18:22:57 +0000 (22:22 +0400)]
runtime: fix and speedup malloc stats
Currently per-sizeclass stats are lost for destroyed MCache's. This patch fixes this.
Also, only update mstats.heap_alloc on heap operations, because that's the only
stat that needs to be promptly updated. Everything else needs to be up-to-date only in ReadMemStats().
Dmitriy Vyukov [Wed, 22 May 2013 09:29:17 +0000 (13:29 +0400)]
runtime: simplify MCache
The nlistmin/size thresholds are copied from tcmalloc,
but are unnecesary for Go malloc. We do not do explicit
frees into MCache. For sparse cases when we do (mainly hashmap),
simpler logic will do.
Brad Fitzpatrick [Tue, 21 May 2013 22:51:49 +0000 (15:51 -0700)]
bufio: reuse Writer buffers after Flush
A bufio.Writer.Flush marks the usual end of a Writer's
life. Recycle its internal buffer on those explicit flushes,
but not on normal, as-needed internal flushes.
benchmark old ns/op new ns/op delta
BenchmarkWriterEmpty 1959 727 -62.89%
benchmark old allocs new allocs delta
BenchmarkWriterEmpty 2 1 -50.00%
benchmark old bytes new bytes delta
BenchmarkWriterEmpty 4215 83 -98.03%
Adam Langley [Tue, 21 May 2013 14:47:31 +0000 (10:47 -0400)]
crypto/tls: don't send NPN extension if NextProtos is not set.
This isn't clearly a bug on Go's part, but it triggers a bug in Firefox
which means that crypto/tls and net/http cannot be wired up together
unless NextProtos includes "http/1.1". When net/http sets up the
tls.Config, it does this and so works fine. But anyone setting up the
tls.Config themselves will hit the Firefox bug.
Fixes #5445.
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/9539045
Robert Griesemer [Mon, 20 May 2013 21:01:07 +0000 (14:01 -0700)]
spec: removed old or invalid TODOs
Several old TODOs are either resolved now (e.g. when is a return
needed), or are from a time the language wasn't frozen (^ for uints
only). Consolidated the others.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9599044
Adam Langley [Mon, 20 May 2013 18:20:26 +0000 (14:20 -0400)]
crypto/x509: provide better error messages for X.509 verify failures.
Failures caused by errors like invalid signatures or missing hash
functions cause rather generic, unhelpful error messages because no
trust chain can be constructed: "x509: certificate signed by unknown
authority."
With this change, authority errors may contain the reason why an
arbitary candidate step in the chain was rejected. For example, in the
event of a missing hash function the error looks like:
x509: certificate signed by unknown authority (possibly because of
"crypto/x509: cannot verify signature: algorithm unimplemented" while
trying to verify candidate authority certificate 'Thawte SGC CA')
Fixes 5058.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9104051
Keith Randall [Fri, 17 May 2013 19:53:49 +0000 (12:53 -0700)]
runtime: faster x86 memmove (a.k.a. built-in copy())
REP instructions have a high startup cost, so we handle small
sizes with some straightline code. The REP MOVSx instructions
are really fast for large sizes. The cutover is approximately
1K. We implement up to 128/256 because that is the maximum
SSE register load (loading all data into registers before any
stores lets us ignore copy direction).