Austin Clements [Mon, 11 May 2015 22:53:49 +0000 (18:53 -0400)]
runtime: don't run runq tests on the system stack
Running these tests on the system stack is problematic because they
allocate Ps, which are large enough to overflow the system stack if
they are stack-allocated. It used to be necessary to run these tests
on the system stack because they were written in C, but since this is
no longer the case, we can fix this problem by simply not running the
tests on the system stack.
This also means we no longer need the hack in one of these tests that
forces the allocated Ps to escape to the heap, so eliminate that as
well.
Russ Cox [Tue, 12 May 2015 19:51:22 +0000 (15:51 -0400)]
cmd/5g: fix build
The line in cgen.go was lost during the ginscmp CL.
The ggen.go change is not strictly necessary, but
it makes the 5g -S output for x[0] match what it said
before the ginscmp CL.
Andrew Williams [Sun, 25 Jan 2015 18:53:34 +0000 (12:53 -0600)]
syscall: relocate linux death signal code
Fix bug on Linux SysProcAttr handling: setting both Pdeathsig and
Credential caused Pdeathsig to be ignored. This is because the kernel
clears the deathsignal field when performing a setuid/setgid
system call.
Avoid this by moving Pdeathsig handling after Credential handling.
Fixes #9686
Change-Id: Id01896ad4e979b8c448e0061f00aa8762ca0ac94
Reviewed-on: https://go-review.googlesource.com/3290 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox [Wed, 6 May 2015 16:34:30 +0000 (12:34 -0400)]
cmd/internal/gc: optimize append + write barrier
The code generated for x = append(x, v) is roughly:
t := x
if len(t)+1 > cap(t) {
t = grow(t)
}
t[len(t)] = v
len(t)++
x = t
We used to generate this code as Go pseudocode during walk.
Generate it instead as actual instructions during gen.
Doing so lets us apply a few optimizations. The most important
is that when, as in the above example, the source slice and the
destination slice are the same, the code can instead do:
t := x
if len(t)+1 > cap(t) {
t = grow(t)
x = {base(t), len(t)+1, cap(t)}
} else {
len(x)++
}
t[len(t)] = v
That is, in the fast path that does not reallocate the array,
only the updated length needs to be written back to x,
not the array pointer and not the capacity. This is more like
what you'd write by hand in C. It's faster in general, since
the fast path elides two of the three stores, but it's especially
faster when the form of x is such that the base pointer write
would turn into a write barrier. No write, no barrier.
See next CL for combination with a similar optimization for slice.
The benchmarks that are slower in this CL are still faster overall
with the combination of the two.
Change-Id: I2a7421658091b2488c64741b4db15ab6c3b4cb7e
Reviewed-on: https://go-review.googlesource.com/9812 Reviewed-by: David Chase <drchase@google.com>
David du Colombier [Tue, 12 May 2015 16:20:04 +0000 (18:20 +0200)]
runtime: fix signal handling on Plan 9
Once added to the signal queue, the pointer passed to the
signal handler could no longer be valid. Instead of passing
the pointer to the note string, we recopy the value of the
note string to a static array in the signal queue.
Russ Cox [Fri, 1 May 2015 00:35:47 +0000 (20:35 -0400)]
cmd/internal/gc: avoid turning 'x = f()' into 'tmp = f(); x = tmp' for simple x
This slows down more things than I expected, but it also speeds things up,
and it reduces stack frame sizes and the load on the optimizer, so it's still
likely a net win.
Russ Cox [Mon, 11 May 2015 20:12:01 +0000 (16:12 -0400)]
cmd/internal/gc: detect bad append(f()) during type check
Today's earlier fix can stay, but it's a band-aid over the real problem,
which is that bad code was slipping through the type checker
into the back end (and luckily causing a type error there).
I discovered this because my new append does not use the same
temporaries and failed the test as written.
Fixes #9521.
Change-Id: I7e33e2ea15743406e15c6f3fdf73e1edecda69bd
Reviewed-on: https://go-review.googlesource.com/9921 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jens Frederich [Thu, 19 Feb 2015 20:37:38 +0000 (21:37 +0100)]
cmd/go: "go get" don't ignore git default branch
Any Git branch can be the default branch not only master. Removing
hardwired 'checkout master', and using 'checkout {tag}' is the best
choice. It works with and without a master branch. Furthermore it
resolves the Github default branch issue. Changing Github default
branch is effectively changing HEAD.
Patrick Mezard [Tue, 12 May 2015 06:19:00 +0000 (08:19 +0200)]
time: fix registry zone info lookup on Windows
registry.ReadSubKeyNames requires QUERY access right in addition to
ENUMERATE_SUB_KEYS.
This was making TestLocalZoneAbbr fail on Windows 7 in Paris/Madrid
timezone. It succeeded on Windows 8 because timezone name changed from
"Paris/Madrid" to "Romance Standard Time", the latter being matched by
an abbrs entry.
Alex Brainman [Tue, 12 May 2015 01:34:23 +0000 (11:34 +1000)]
net: relax error checking in TestAcceptIgnoreSomeErrors
TestAcceptIgnoreSomeErrors was created to test that network
accept function ignores some errors. But conditions created
by the test also affects network reads. Change the test to
ignore these read errors when acceptable.
Shenghou Ma [Tue, 12 May 2015 00:59:59 +0000 (20:59 -0400)]
syscall: fix running mkall.sh on linux/{ppc64,ppc64le}
Change-Id: I58c6e914d0e977d5748c87d277e30c933ed86f99
Reviewed-on: https://go-review.googlesource.com/9924 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Hudson-Doyle [Sat, 11 Apr 2015 04:05:21 +0000 (12:05 +0800)]
cmd/internal/ld, runtime: abort on shared library ABI mismatch
This:
1) Defines the ABI hash of a package (as the SHA1 of the __.PKGDEF)
2) Defines the ABI hash of a shared library (sort the packages by import
path, concatenate the hashes of the packages and SHA1 that)
3) When building a shared library, compute the above value and define a
global symbol that points to a go string that has the hash as its value.
4) When linking against a shared library, read the abi hash from the
library and put both the value seen at link time and a reference
to the global symbol into the moduledata.
5) During runtime initialization, check that the hash seen at link time
still matches the hash the global symbol points to.
Change-Id: Iaa54c783790e6dde3057a2feadc35473d49614a5
Reviewed-on: https://go-review.googlesource.com/8773 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
Michael Hudson-Doyle [Mon, 11 May 2015 23:59:14 +0000 (11:59 +1200)]
runtime: fix addmoduledata to follow the platform ABI
addmoduledata is called from a .init_array function and need to follow the
platform ABI. It contains accesses to global data which are rewritten to use
R15 by the assembler, and as R15 is callee-save we need to save it.
Change-Id: I03893efb1576aed4f102f2465421f256f3bb0f30
Reviewed-on: https://go-review.googlesource.com/9941 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Rahul Chaudhry [Fri, 8 May 2015 19:33:30 +0000 (12:33 -0700)]
cmd/dist: de-dup iOS detection
Change-Id: I89778988baec1cf4a35d9342c7dbe8c4c08ff3cd
Reviewed-on: https://go-review.googlesource.com/9893
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Michael Hudson-Doyle [Sun, 10 May 2015 23:24:10 +0000 (11:24 +1200)]
cmd/internal/ld: change Cpos to not flush the output buffer
DWARF generation appears to assume Cpos is cheap and this makes linking godoc
about 8% faster and linking the standard library into a single shared library
about 22% faster on my machine.
Updates #10571
Change-Id: I3f81efd0174e356716e7971c4f59810b72378177
Reviewed-on: https://go-review.googlesource.com/9913 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Daniel Morsing [Mon, 11 May 2015 09:31:24 +0000 (10:31 +0100)]
net/http: silence race detector on client header timeout test
When running the client header timeout test, there is a race between
us timing out and waiting on the remaining requests to be serviced. If
the client times out before the server blocks on the channel in the
handler, we will be simultaneously adding to a waitgroup with the
value 0 and waiting on it when we call TestServer.Close().
This is largely a theoretical race. We have to time out before we
enter the handler and the only reason we would time out if we're
blocked on the channel. Nevertheless, make the race detector happy
by turning the close into a channel send. This turns the defer call
into a synchronization point and we can be sure that we've entered
the handler before we close the server.
Russ Cox [Fri, 8 May 2015 02:40:54 +0000 (22:40 -0400)]
runtime: use heap bitmap for typedmemmove
The current implementation of typedmemmove walks the ptrmask
in the type to find out where pointers are. This led to turning off
GC programs for the Go 1.5 dev cycle, so that there would always
be a ptrmask. Instead of also interpreting the GC programs,
interpret the heap bitmap, which we know must be available and
up to date. (There is no point to write barriers when writing outside
the heap.)
This CL is only about correctness. The next CL will optimize the code.
Russ Cox [Fri, 8 May 2015 02:39:57 +0000 (22:39 -0400)]
runtime: zero entire bitmap for object, even past dead marker
We want typedmemmove to use the heap bitmap to determine
where pointers are, instead of reinterpreting the type information.
The heap bitmap is simpler to access.
In general, typedmemmove will need to be able to look up the bits
for any word and find valid pointer information, so fill even after the
dead marker. Not filling after the dead marker was an optimization
I introduced only a few days ago, when reintroducing the dead marker
code. At the time I said it probably wouldn't last, and it didn't.
Russ Cox [Thu, 7 May 2015 15:03:17 +0000 (11:03 -0400)]
runtime: reorder bits in heap bitmap bytes
The runtime deals with 1-bit pointer bitmaps and 2-bit heap bitmaps
that have entries for both pointers and mark bits.
Each byte in a 1-bit pointer bitmap looks like pppppppp (all pointer bits).
Each byte in a 2-bit heap bitmap looks like mpmpmpmp (mark, pointer, ...).
This means that when converting from 1-bit to 2-bit, as we do
during malloc, we have to pick up 4 bits in pppp form and use
shifts to create the mpmpmpmp form.
This CL changes the 2-bit heap bitmap form to mmmmpppp,
so that 4 bits picked up in 1-bit form can be used directly in
the low bits of the heap bitmap byte, without expansion.
This simplifies the code, and it also happens to be faster.
Russ Cox [Wed, 6 May 2015 16:25:56 +0000 (12:25 -0400)]
cmd/internal/gc: drop unused Reslice field from Node
Dead code.
This field is left over from Go 1.4, when we elided the fake write
barrier in this case. Today, it's unused (always false).
The upcoming append/slice changes handle this case again,
but without needing this field.
Change-Id: Ic6f160b64efdc1bbed02097ee03050f8cd0ab1b8
Reviewed-on: https://go-review.googlesource.com/9789 Reviewed-by: David Chase <drchase@google.com>
Russ Cox [Wed, 6 May 2015 16:22:05 +0000 (12:22 -0400)]
cmd/internal/gc: show register dump before crashing on register left allocated
If you are using -h to get a stack trace at the site of the failure,
Yyerror will never return. Dump the register allocation sites
before calling Yyerror.
Russ Cox [Tue, 5 May 2015 04:26:53 +0000 (00:26 -0400)]
runtime: remove wbshadow mode
The write barrier shadow heap was very useful for
developing the write barriers initially, but it's no longer used,
clunky, and dragging the rest of the implementation down.
The gccheckmark mode will find bugs due to missed barriers
when they result in missed marks; wbshadow mode found the
missed barriers more aggressively, but it required an entire
separate copy of the heap. The gccheckmark mode requires
no extra memory, making it more useful in practice.
It's nice that the microbenchmarks are the ones helped the most,
because those were the ones hurt the most by the conversion from
4-bit to 2-bit heap bitmaps. This CL brings the overall effect of that
process to (compared to CL 9706 patch set 1):
Russ Cox [Tue, 5 May 2015 02:53:54 +0000 (22:53 -0400)]
runtime: reintroduce ``dead'' space during GC scan
Reintroduce an optimization discarded during the initial conversion
from 4-bit heap bitmaps to 2-bit heap bitmaps: when we reach the
place in the bitmap where there are no more pointers, mark that position
for the GC so that it can avoid scanning past that place.
During heapBitsSetType we can also avoid initializing heap bitmap
beyond that location, which gives a bit of a win compared to Go 1.4.
This particular optimization (not initializing the heap bitmap) may not last:
we might change typedmemmove to use the heap bitmap, in which
case it would all need to be initialized. The early stop in the GC scan
will stay no matter what.
Compared to Go 1.4 (github.com/rsc/go, branch go14bench):
name old mean new mean delta
SetTypeNode64 80.7ns × (1.00,1.01) 57.4ns × (1.00,1.01) -28.83% (p=0.000)
SetTypeNode64Dead 80.5ns × (1.00,1.01) 13.1ns × (0.99,1.02) -83.77% (p=0.000)
SetTypeNode64Slice 2.16µs × (1.00,1.01) 1.54µs × (1.00,1.01) -28.75% (p=0.000)
SetTypeNode64DeadSlice 2.16µs × (1.00,1.01) 1.52µs × (1.00,1.00) -29.74% (p=0.000)
Compared to previous CL:
name old mean new mean delta
SetTypeNode64 56.7ns × (1.00,1.00) 57.4ns × (1.00,1.01) +1.19% (p=0.000)
SetTypeNode64Dead 57.2ns × (1.00,1.00) 13.1ns × (0.99,1.02) -77.15% (p=0.000)
SetTypeNode64Slice 1.56µs × (1.00,1.01) 1.54µs × (1.00,1.01) -0.89% (p=0.000)
SetTypeNode64DeadSlice 1.55µs × (1.00,1.01) 1.52µs × (1.00,1.00) -2.23% (p=0.000)
This is the last CL in the sequence converting from the 4-bit heap
to the 2-bit heap, with all the same optimizations reenabled.
Compared to before that process began (compared to CL 9701 patch set 1):
The slowdown of the first few benchmarks seems to be due to the new
atomic operations for certain small size allocations. But the larger
benchmarks mostly improve, probably due to the decreased memory
pressure from having half as much heap bitmap.
CL 9706, which removes the (never used anymore) wbshadow mode,
gets back what is lost in the early microbenchmarks.
Change-Id: I37423a209e8ec2a2e92538b45cac5422a6acd32d
Reviewed-on: https://go-review.googlesource.com/9705 Reviewed-by: Rick Hudson <rlh@golang.org>
Russ Cox [Mon, 4 May 2015 15:30:10 +0000 (11:30 -0400)]
runtime: optimize heapBitsSetType
For the conversion of the heap bitmap from 4-bit to 2-bit fields,
I replaced heapBitsSetType with the dumbest thing that could possibly work:
two atomic operations (atomicand8+atomicor8) per 2-bit field.
This CL replaces that code with a proper implementation that
avoids the atomics whenever possible. Benchmarks vs base CL
(before the conversion to 2-bit heap bitmap) and vs Go 1.4 below.
Compared to Go 1.4, SetTypePtr (a 1-pointer allocation)
is 10ns slower because a race against the concurrent GC requires the
use of an atomicor8 that used to be an ordinary write. This slowdown
was present even in the base CL.
Compared to both Go 1.4 and base, SetTypeNode8 (a 10-word allocation)
is 10ns slower because it too needs a new atomic, because with the
denser representation, the byte on the end of the allocation is now shared
with the object next to it; this was not true with the 4-bit representation.
Excluding these two (fundamental) slowdowns due to the use of atomics,
the new code is noticeably faster than both Go 1.4 and the base CL.
The next CL will reintroduce the ``typeDead'' optimization.
Stats are from 5 runs on a MacBookPro10,2 (late 2012 Core i5).
There are fewer benchmarks vs Go 1.4 because unrolling directly
into the heap bitmap is not yet implemented, so those would not
be meaningful comparisons.
These benchmarks were not present in Go 1.4 as distributed.
The backport to Go 1.4 is in github.com/rsc/go's go14bench branch,
commit 71d5ee5.
Change-Id: I95ed05a22bf484b0fc9efad549279e766c98d2b6
Reviewed-on: https://go-review.googlesource.com/9704 Reviewed-by: Rick Hudson <rlh@golang.org>
Russ Cox [Mon, 4 May 2015 14:19:24 +0000 (10:19 -0400)]
runtime: use 2-bit heap bitmap (in place of 4-bit)
Previous CLs changed the representation of the non-heap type bitmaps
to be 1-bit bitmaps (pointer or not). Before this CL, the heap bitmap
stored a 2-bit type for each word and a mark bit and checkmark bit
for the first word of the object. (There used to be additional per-word bits.)
Reduce heap bitmap to 2-bit, with 1 dedicated to pointer or not,
and the other used for mark, checkmark, and "keep scanning forward
to find pointers in this object." See comments for details.
This CL replaces heapBitsSetType with very slow but obviously correct code.
A followup CL will optimize it. (Spoiler: the new code is faster than Go 1.4 was.)
runtime: use 1-bit pointer bitmaps in type representation
The type information in reflect.Type and the GC programs is now
1 bit per word, down from 2 bits.
The in-memory unrolled type bitmap representation are now
1 bit per word, down from 4 bits.
The conversion from the unrolled (now 1-bit) bitmap to the
heap bitmap (still 4-bit) is not optimized. A followup CL will
work on that, after the heap bitmap has been converted to 2-bit.
The typeDead optimization, in which a special value denotes
that there are no more pointers anywhere in the object, is lost
in this CL. A followup CL will bring it back in the final form of
heapBitsSetType.
Russ Cox [Sun, 3 May 2015 02:59:35 +0000 (22:59 -0400)]
runtime: add benchmark of heapBitsSetType
There was an old benchmark that measured this indirectly
via allocation, but I don't understand how to factor out the
allocation cost when interpreting the numbers.
Replace with a benchmark that only calls heapBitsSetType,
that does not allocate. This was not possible when the
benchmark was first written, because heapBitsSetType had
not been factored out of mallocgc.
Daniel Morsing [Thu, 30 Apr 2015 14:32:54 +0000 (15:32 +0100)]
runtime: enable profiling on g0
Since we now have stack information for code running on the
systemstack, we can traceback over it. To make cpu profiles useful,
add a case in gentraceback to jump over systemstack switches.
Mikio Hara [Sun, 10 May 2015 14:11:04 +0000 (23:11 +0900)]
net: increase timeout in TestWriteTimeoutFluctuation on darwin/arm
On darwin/arm, the test sometimes fails with:
Process 557 resuming
--- FAIL: TestWriteTimeoutFluctuation (1.64s)
timeout_test.go:706: Write took over 1s; expected 0.1s
FAIL
Process 557 exited with status = 1 (0x00000001)
go_darwin_arm_exec: timeout running tests
This change increaes timeout on iOS builders from 1s to 3s as a
temporarily fix.
Updates #10775.
Change-Id: Ifdaf99cf5b8582c1a636a0f7d5cc66bb276efd72
Reviewed-on: https://go-review.googlesource.com/9915 Reviewed-by: Minux Ma <minux@golang.org>
ExpandString correctly loops on the syscall until it reaches the
required buffer size but truncates it before converting it back to
string. The truncation limit is increased to 2^15 bytes which is the
documented maximum ExpandEnvironmentStrings output size.
This fixes TestExpandString on systems where len($PATH) > 1024.
Change-Id: I2a6f184eeca939121b458bcffe1a436a50f3298e
Reviewed-on: https://go-review.googlesource.com/9805 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Didier Spezia [Fri, 8 May 2015 16:38:08 +0000 (16:38 +0000)]
html: simplify and optimize escape/unescape
The html package uses some specific code to escape special characters.
Actually, the strings.Replacer can be used instead, and is much more
efficient. The converse operation is more complex but can still be
slightly optimized.
Credits to Ken Bloom (kabloom@google.com), who first submitted a
similar patch at https://codereview.appspot.com/141930043
Added benchmarks and slightly optimized UnescapeString.
Didier Spezia [Wed, 6 May 2015 22:14:32 +0000 (22:14 +0000)]
html/template: fix pipeline sanitization
Pipelines are altered by inserting sanitizers if they are not
already present. The code makes the assumption that the first
operands of each commands are function identifiers.
This is wrong, since they can also be methods. It results in
a panic with templates such as {{1|print 2|.f 3}}
Adds an extra type assertion to make sure only identifiers
are compared with sanitizers.
Fixes #10673
Change-Id: I3eb820982675231dbfa970f197abc5ef335ce86b
Reviewed-on: https://go-review.googlesource.com/9801 Reviewed-by: Rob Pike <r@golang.org>
Brett Cannon [Fri, 8 May 2015 14:41:15 +0000 (10:41 -0400)]
doc/effective_go: make function signature match function body
In the Slices section of Effective Go, the os package's File.Read
function is used as an example. Unfortunately the function signature
does not match the function's code in the example, nor the os package's
documentation. This change updates the function signature to match
the os package and the pre-existing function code.
Change-Id: Iae9f30c898d3a1ff8d47558ca104dfb3ff07112c
Reviewed-on: https://go-review.googlesource.com/9845 Reviewed-by: Rob Pike <r@golang.org>
Ian Lance Taylor [Thu, 7 May 2015 19:58:43 +0000 (12:58 -0700)]
cmd/cgo: wrap generated exports with extern "C" for C++
This will make it possible for C++ code to #include the export header
file and see the correct declarations.
The preamble remains the user's responsibility. It would not be
appropriate to wrap the preamble in extern "C", because it might
include header files that work with both C and C++. Putting those
header files in an extern "C" block would break them.
Change-Id: Ifb40879d709d26596d5c80b1307a49f1bd70932a
Reviewed-on: https://go-review.googlesource.com/9850 Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox [Thu, 7 May 2015 19:29:48 +0000 (15:29 -0400)]
runtime: fix gccheckmark mode and enable by default
It was testing the mark bits on what roots pointed at,
but not the remainder of the live heap, because in
CL 2991 I accidentally inverted this check during
refactoring.
The next CL will turn it back off by default again,
but I want one run on the builders with the full
checkmark checks.
Rick Hudson [Wed, 6 May 2015 19:58:20 +0000 (15:58 -0400)]
runtime: set heap minimum default based on GOGC
Currently the heap minimum is set to 4MB which prevents our ability to
collect at every allocation by setting GOGC=0. This adjust the
heap minimum to 4MB*GOGC/100 thus reenabling collecting at every allocation.
Fixes #10681
Rob Pike [Thu, 7 May 2015 18:22:43 +0000 (11:22 -0700)]
fmt: fix panic with large precision
The code already handled high widths but not high precisions.
Also make sure it handles the harder cases of %U.
Fixes #10745.
Change-Id: Ib4d394d49a9941eeeaff866dc59d80483e312a98
Reviewed-on: https://go-review.googlesource.com/9769 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Thu, 7 May 2015 05:06:19 +0000 (22:06 -0700)]
cmd/go: install headers for c-archive/c-shared cgo exports
When
using -buildmode=c-archive or c-shared, and
when installing packages that use cgo, and
when those packages export some functions via //export comments,
then
for each such package, install a pkg.h header file that declares the
functions.
This permits C code to #include the header when calling the Go
functions.
This is a little awkward to use when there are multiple packages that
export functions, as you have to "go install" your c-archive/c-shared
object and then pull it out of the package directory. When compiling
your C code you have to -I pkg/$GOOS_$GOARCH. I haven't thought of
any more convenient approach. It's simpler when only the main package
has exported functions.
When using c-shared you currently have to use a _shared suffix in the
-I option; it would be nice to fix that somehow.
Change-Id: I5d8cf08914b7d3c2b194120c77791d2732ffd26e
Reviewed-on: https://go-review.googlesource.com/9798 Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
David Chase [Tue, 5 May 2015 17:25:58 +0000 (13:25 -0400)]
cmd/internal/gc: improve "type *X has no field or method M" message
Try to provide hints for common areas, either *interface
were interface would have been better, and note incorrect
capitalization (but don't be more ambitious than that, at
least not today).
Michael Hudson-Doyle [Wed, 6 May 2015 02:30:28 +0000 (14:30 +1200)]
runtime: check consistency of all module data objects
Current code just checks the consistency (that the functab is correctly
sorted by PC, etc) of the moduledata object that the runtime belongs to.
Change to check all of them.
Change-Id: I544a44c5de7445fff87d3cdb4840ff04c5e2bf75
Reviewed-on: https://go-review.googlesource.com/9773 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
John Dethridge [Sat, 2 May 2015 06:40:21 +0000 (16:40 +1000)]
debug/dwarf: compute ByteSize for more DWARF types
When AttrByteSize is not present for a type, we can still determine the
size in two more cases: when the type is a Typedef referring to another
type, and when the type is a pointer and we know the default address
size.
entry.go: return after setting an error if the offset is out of range.
Ian Lance Taylor [Thu, 7 May 2015 00:53:17 +0000 (17:53 -0700)]
cmd/cgo: add -exportheader option
The -exportheader option tells cgo to generate a header file declaring
expoted functions. The header file is only created if there are, in
fact, some exported functions, so it also serves as a signal as to
whether there were any.
In future CLs the go tool will use this option to install header files
for packages that use cgo and export functions.
Change-Id: I5b04357d453a9a8f0e70d37f8f18274cf40d74c9
Reviewed-on: https://go-review.googlesource.com/9796 Reviewed-by: David Crawshaw <crawshaw@golang.org>
Mikio Hara [Wed, 6 May 2015 01:41:01 +0000 (10:41 +0900)]
net: align temporary file, directory names in tests
Also adds missing temporary file deletion.
Change-Id: Ia644b0898022e05d2f5232af38f51d55e40c6fb5
Reviewed-on: https://go-review.googlesource.com/9772 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Mikio Hara [Wed, 6 May 2015 06:55:22 +0000 (15:55 +0900)]
doc: mention returned error types and values on the net package in go1.5.txt
Change-Id: I70dfc2bad13c513c376c7c41058774b40af73dce
Reviewed-on: https://go-review.googlesource.com/9775 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
Austin Clements [Mon, 4 May 2015 20:55:31 +0000 (16:55 -0400)]
runtime: use heap scan size as estimate of GC scan work
Currently, the GC uses a moving average of recent scan work ratios to
estimate the total scan work required by this cycle. This is in turn
used to compute how much scan work should be done by mutators when
they allocate in order to perform all expected scan work by the time
the allocated heap reaches the heap goal.
However, our current scan work estimate can be arbitrarily wrong if
the heap topography changes significantly from one cycle to the
next. For example, in the go1 benchmarks, at the beginning of each
benchmark, the heap is dominated by a 256MB no-scan object, so the GC
learns that the scan density of the heap is very low. In benchmarks
that then rapidly allocate pointer-dense objects, by the time of the
next GC cycle, our estimate of the scan work can be too low by a large
factor. This in turn lets the mutator allocate faster than the GC can
collect, allowing it to get arbitrarily far ahead of the scan work
estimate, which leads to very long GC cycles with very little mutator
assist that can overshoot the heap goal by large margins. This is
particularly easy to demonstrate with BinaryTree17:
$ GODEBUG=gctrace=1 ./go1.test -test.bench BinaryTree17
gc #1 @0.017s 2%: 0+0+0+0+0 ms clock, 0+0+0+0/0/0+0 ms cpu, 4->262->262 MB, 4 MB goal, 1 P
gc #2 @0.026s 3%: 0+0+0+0+0 ms clock, 0+0+0+0/0/0+0 ms cpu, 262->262->262 MB, 524 MB goal, 1 P
testing: warning: no tests to run
PASS
BenchmarkBinaryTree17 gc #3 @1.906s 0%: 0+0+0+0+7 ms clock, 0+0+0+0/0/0+7 ms cpu, 325->325->287 MB, 325 MB goal, 1 P (forced)
gc #4 @12.203s 20%: 0+0+0+10067+10 ms clock, 0+0+0+0/2523/852+10 ms cpu, 430->2092->1950 MB, 574 MB goal, 1 P
1 9150447353 ns/op
Change this estimate to instead use the *current* scannable heap
size. This has the advantage of being based solely on the current
state of the heap, not on past densities or reachable heap sizes, so
it isn't susceptible to falling behind during these sorts of phase
changes. This is strictly an over-estimate, but it's better to
over-estimate and get more assist than necessary than it is to
under-estimate and potentially spiral out of control. Experiments with
scaling this estimate back showed no obvious benefit for mutator
utilization, heap size, or assist time.
This new estimate has little effect for most benchmarks, including
most go1 benchmarks, x/benchmarks, and the 6g benchmark. It has a huge
effect for benchmarks that triggered the bad pacer behavior:
Austin Clements [Mon, 4 May 2015 20:10:49 +0000 (16:10 -0400)]
runtime: track "scannable" bytes of heap
This tracks the number of scannable bytes in the allocated heap. That
is, bytes that the garbage collector must scan before reaching the
last pointer field in each object.
This will be used to compute a more robust estimate of the GC scan
work.
Austin Clements [Mon, 4 May 2015 19:40:58 +0000 (15:40 -0400)]
runtime: include scalar slots in GC scan work metric
The garbage collector predicts how much "scan work" must be done in a
cycle to determine how much work should be done by mutators when they
allocate. Most code doesn't care what units the scan work is in: it
simply knows that a certain amount of scan work has to be done in the
cycle. Currently, the GC uses the number of pointer slots scanned as
the scan work on the theory that this is the bulk of the time spent in
the garbage collector and hence reflects real CPU resource usage.
However, this metric is difficult to estimate at the beginning of a
cycle.
Switch to counting the total number of bytes scanned, including both
pointer and scalar slots. This is still less than the total marked
heap since it omits no-scan objects and no-scan tails of objects. This
metric may not reflect absolute performance as well as the count of
scanned pointer slots (though it still takes time to scan scalar
fields), but it will be much easier to estimate robustly, which is
more important.
Austin Clements [Fri, 1 May 2015 22:08:44 +0000 (18:08 -0400)]
runtime: dispose gcWork caches before updating controller state
Currently, we only flush the per-P gcWork caches in gcMark, at the
beginning of mark termination. This is necessary to ensure that no
work is held up in these caches.
However, this flush happens after we update the GC controller state,
which depends on statistics about marked heap size and scan work that
are only updated by this flush. Hence, the controller is missing the
bulk of heap marking and scan work. This bug was introduced in commit 1b4025f, which introduced the per-P gcWork caches.
Fix this by flushing these caches before we update the GC controller
state. We continue to flush them at the beginning of mark termination
as well to be robust in case any write barriers happened between the
previous flush and entering mark termination, but this should be a
no-op.
Rob Pike [Mon, 4 May 2015 17:11:27 +0000 (10:11 -0700)]
cmd/doc: if no top-level symbols match, look for methods
Improving the usability further.
Before:
$ go doc bytes.Read
doc: symbol Read not present in package bytes installed in "bytes"
$
After:
$ go doc bytes.Read
func (b *Buffer) Read(p []byte) (n int, err error)
Read reads the next len(p) bytes from the buffer or until the buffer is drained.
The return value n is the number of bytes read. If the buffer has no data to
return, err is io.EOF (unless len(p) is zero); otherwise it is nil.
func (r *Reader) Read(b []byte) (n int, err error)
$
Rob Pike [Wed, 6 May 2015 15:26:51 +0000 (08:26 -0700)]
os: rewrite LookupEnv's test
GOROOT is not dependably set.
When I first wrote this test, I thought it was a waste of time
because the function can't fail if the other environment functions
work, but I didn't want to add functionality without testing it.
Of course, the test broke, and I learned something: GOROOT is not
set on iOS or, to put it more broadly, the world continues to
surprise me with its complexity and horror, such as a version of
cat with syntax coloring.
Rick Hudson [Mon, 4 May 2015 18:03:07 +0000 (14:03 -0400)]
runtime: remove unused GC timers
During development some tracing routines were added that are not
needed in the release. These included GCstarttimes, GCendtimes, and
GCprinttimes.
Fixes #10462
Aram Hăvărneanu [Mon, 23 Mar 2015 12:52:46 +0000 (13:52 +0100)]
net: link with networking libraries when net package is in use
Fixes #10221.
Change-Id: Ib23805494d8af1946360bfea767f9727e2504dc5
Reviewed-on: https://go-review.googlesource.com/7941 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Minux Ma <minux@golang.org>
Aram Hăvărneanu [Tue, 17 Mar 2015 12:50:40 +0000 (13:50 +0100)]
net: try to fix setKeepAlivePeriod on Solaris
Unfortunately Oracle Solaris does not have TCP_KEEPIDLE and
TCP_KEEPINTVL. TCP_KEEPIDLE is equivalent to TCP_KEEPALIVE_THRESHOLD,
but TCP_KEEPINTVL does not have a direct equivalent, so we don't set
TCP_KEEPINTVL any more.
Old Darwin versions also lack TCP_KEEPINTVL, but the code tries to set
it anyway so that it works on newer versions. We can't do that because
Oracle might assign the number illumos uses for TCP_KEEPINTVL to a
constant with a different meaning.
Unfortunately there's nothing we can do if we want to support both
illumos and Oracle Solaris with the same GOOS.
runtime, syscall: link Solaris binaries directly instead of using dlopen/dlsym
Before CL 8214 (use .plt instead of .got on Solaris) Solaris used a
dynamic linking scheme that didn't permit lazy binding. To speed program
startup, Go binaries only used it for a small number of symbols required
by the runtime. Other symbols were resolved on demand on first use, and
were cached for subsequent use. This required some moderately complex
code in the syscall package.
CL 8214 changed the way dynamic linking is implemented, and now lazy
binding is supported. As now all symbols are resolved lazily by the
dynamic loader, there is no need for the complex code in the syscall
package that did the same. This CL makes Go programs link directly
with the necessary shared libraries and deletes the lazy-loading code
implemented in Go.
Change-Id: Ifd7275db72de61b70647242e7056dd303b1aee9e
Reviewed-on: https://go-review.googlesource.com/9184 Reviewed-by: Minux Ma <minux@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Aram Hăvărneanu [Mon, 30 Mar 2015 21:12:04 +0000 (23:12 +0200)]
doc/progs: disable cgo tests that use C.Stdout on Solaris
Solaris, like Windows, NetBSD and OpenBSD, uses macros for stdin, stdout,
and stderr. Cgo can't access them without getters/setters written in
C. Because of this we disable affected tests like for the other platforms.