Russ Cox [Sat, 15 Feb 2014 15:58:55 +0000 (10:58 -0500)]
cmd/gc: correct liveness for fat variables
The VARDEF placement must be before the initialization
but after any final use. If you have something like s = ... using s ...
the rhs must be evaluated, then the VARDEF, then the lhs
assigned.
There is a large comment in pgen.c on gvardef explaining
this in more detail.
This CL also includes Ian's suggestions from earlier CLs,
namely commenting the use of mode in link.h and fixing
the precedence of the ~r check in dcl.c.
This CL enables the check that if liveness analysis decides
a variable is live on entry to the function, that variable must
be a function parameter (not a result, and not a local variable).
If this check fails, it indicates a bug in the liveness analysis or
in the generated code being analyzed.
The race detector generates invalid code for append(x, y...).
The code declares a temporary t and then uses cap(t) before
initializing t. The new liveness check catches this bug and
stops the compiler from writing out the buggy code.
Consequently, this CL disables the race detector tests in
run.bash until the race detector bug can be fixed
(golang.org/issue/7334).
Except for the race detector bug, the liveness analysis check
does not detect any problems (this CL and the previous CLs
fixed all the detected problems).
The net test still fails with GOGC=0 but the rest of the tests
now pass or time out (because GOGC=0 is so slow).
Rob Pike [Sat, 15 Feb 2014 00:26:47 +0000 (16:26 -0800)]
text/template: don't panic when function call evaluates a nil pointer
Catch the error instead and return it to the user. Before this fix,
the template package panicked. Now you get:
template: bug11:1:14: executing "bug11" at <.PS>: dereference of nil pointer of type *string
Extended example at http://play.golang.org/p/uP6pCW3qKT
Adam Langley [Fri, 14 Feb 2014 22:17:19 +0000 (17:17 -0500)]
compress/bzip2: support superfluous Huffman levels.
These should never be found in a bzip2 file but it does appear that
there's a buggy encoder that is producing them. Since the official
bzip2 handles this case, this change makes the Go code do likewise.
With this change, the code produces the same output as the official
bzip2 code on the invalid example given in the bug.
Fixes #7279.
LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/64010043
David du Colombier [Fri, 14 Feb 2014 21:27:47 +0000 (22:27 +0100)]
runtime: fix "invalid address in sys call" on Plan 9
Rfork is not splitting the stack when creating a new thread,
so the parent and child are executing on the same stack.
However, if the parent returns and keeps executing before
the child can read the arguments from the parent stack,
the child will not see the right arguments. The solution
is to load the needed pieces from the parent stack into
register before INT $64.
Ian Lance Taylor [Fri, 14 Feb 2014 19:06:53 +0000 (11:06 -0800)]
runtime: if traceback sees a breakpoint, don't change the PC
Changing the PC confuses gdb, because execution does not
continue where gdb expects it. Not changing the PC has the
potential to confuse a stack dump, but when running under gdb
it seems better to confuse a stack dump than to confuse gdb.
Mikio Hara [Fri, 14 Feb 2014 15:47:28 +0000 (00:47 +0900)]
syscall: make use of include/linux when generating system constants
On Linux include/net directory is just to help porting applications
from BSDs and files under net keep less information than include/linux.
Making use of files under include/linux instead of include/net prevents
lack of information.
Dmitriy Vyukov [Fri, 14 Feb 2014 09:24:48 +0000 (13:24 +0400)]
runtime: remove misleading message during crash
The following checkdead message is false positive:
$ go test -race -c runtime
$ ./runtime.test -test.cpu=2 -test.run=TestSmhasherWindowed -test.v
=== RUN TestSmhasherWindowed-2
checkdead: find g 18 in status 1
SIGABRT: abort
PC=0x42bff1
Dmitriy Vyukov [Fri, 14 Feb 2014 09:20:41 +0000 (13:20 +0400)]
runtime: fix mem profile when both large and small objects are allocated at the same stack
Currently small and large (size>rate) objects are merged into a single entry.
But rate adjusting is required only for small objects.
As a result pprof either incorrectly adjusts large objects
or does not adjust small objects.
With this change objects of different sizes are stored in different buckets.
Russ Cox [Fri, 14 Feb 2014 05:38:24 +0000 (00:38 -0500)]
cmd/gc: correct liveness for various non-returning functions
When the liveness code doesn't know a function doesn't return
(but the generated code understands that), the liveness analysis
invents a control flow edge that is not really there, which can cause
variables to seem spuriously live. This is particularly bad when the
variables are uninitialized.
Dmitriy Vyukov [Fri, 14 Feb 2014 05:20:51 +0000 (09:20 +0400)]
runtime: fix windows cpu profiler
Currently it periodically fails with the following message.
The immediate cause is the wrong base register when obtaining g
in sys_windows_amd64/386.s.
But there are several secondary problems as well.
runtime: unknown pc 0x0 after stack split
panic: invalid memory address or nil pointer dereference
fatal error: panic during malloc
[signal 0xc0000005 code=0x0 addr=0x60 pc=0x42267a]
runtime stack:
runtime.panic(0x7914c0, 0xc862af)
c:/src/perfer/work/windows-amd64-a15f344a9efa/go/src/pkg/runtime/panic.c:217 +0x2c
runtime: unexpected return pc for runtime.externalthreadhandler called from 0x0
Russ Cox [Fri, 14 Feb 2014 04:56:53 +0000 (23:56 -0500)]
cmd/gc: correct liveness for func ending in panic
The registerization code needs the function to end in a RET,
even if that RET is actually unreachable.
The liveness code needs to avoid such unreachable RETs.
It had a special case for final RET after JMP, but no case
for final RET after UNDEF. Instead of expanding the special
cases, let fixjmp - which already knows what is and is not
reachable definitively - mark the unreachable RET so that
the liveness code can identify it.
Russ Cox [Fri, 14 Feb 2014 04:33:20 +0000 (23:33 -0500)]
cmd/gc: correct liveness for wrappers containing tail jumps
A normal RET is treated as using the return values,
but a tail jump RET does not - it is jumping to the
function that is going to fill in the return values.
If a tail jump RET is recorded as using the return values,
since nothing initializes them they will be marked as
live on entry to the function, which is clearly wrong.
Found and tested by the new code in plive.c that looks
for variables that are incorrectly live on entry.
That code is disabled for now because there are other
cases remaining to be fixed. But once it is enabled,
test/live1.go becomes a real test of this CL.
Russ Cox [Fri, 14 Feb 2014 03:45:16 +0000 (22:45 -0500)]
cmd/gc: handle variable initialization by block move in liveness
Any initialization of a variable by a block copy or block zeroing
or by multiple assignments (componentwise copying or zeroing
of a multiword variable) needs to emit a VARDEF. These cases were not.
Russ Cox [Fri, 14 Feb 2014 03:30:35 +0000 (22:30 -0500)]
cmd/5g, cmd/8g: fix build
The test added in CL 63630043 fails on 5g and 8g because they
were not emitting the VARDEF instruction when clearing a fat
value by clearing the components. 6g had the call in the right place.
Mikio Hara [Fri, 14 Feb 2014 03:20:21 +0000 (12:20 +0900)]
net: disable TestDNSThreadLimit even in non-short mode by default
TestDNSThreadLimit creates tons of DNS queries and it occasionally
causes an unintentional traffic jam and/or crash of some virtual
machine software, especially its builtin networking stuff.
We can run TestDNSThreadLimit with -dnsflood flag instead.
LGTM=dave, rsc
R=rsc, dave
CC=golang-codereviews
https://golang.org/cl/63600043
Shenghou Ma [Fri, 14 Feb 2014 02:01:33 +0000 (21:01 -0500)]
cmd/gc: rephrase the invalid indexing operation error message
Old:
prog.go:9: invalid operation: this[i] (index of type int)
New:
prog.go:9: invalid operation: this[i] (type int does not support indexing)
LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/52540043
Russ Cox [Fri, 14 Feb 2014 01:59:39 +0000 (20:59 -0500)]
cmd/gc: distinguish unnamed vs blank-named return variables better
Before, an unnamed return value turned into an ONAME node n with n->sym
named ~anon%d, and n->orig == n.
A blank-named return value turned into an ONAME node n with n->sym
named ~anon%d but n->orig == the original blank n. Code generation and
printing uses n->orig, so that this node formatted as _.
But some code does not use n->orig. In particular the liveness code does
not know about the n->orig convention and so mishandles blank identifiers.
It is possible to fix but seemed better to avoid the confusion entirely.
Now the first kind of node is named ~r%d and the second ~b%d; both have
n->orig == n, so that it doesn't matter whether code uses n or n->orig.
After this change the ->orig field is only used for other kinds of expressions,
not for ONAME nodes.
This requires distinguishing ~b from ~r names in a few places that care.
It fixes a liveness analysis bug without actually changing the liveness code.
Russ Cox [Fri, 14 Feb 2014 00:59:09 +0000 (19:59 -0500)]
cmd/gc: relax address-of escape analysis
Make the loop nesting depth of &x depend on where x is declared,
not on where the &x appears. The latter is only a conservative
estimate of the former. Being more careful can avoid some
variables escaping, and it is easier to reason about.
It would have avoided issue 7313, although that was still a bug
worth fixing.
Not much effect in the tree: one variable in the whole tree
is saved from a heap allocation (something in x509 parsing).
Markus Zimmermann [Fri, 14 Feb 2014 00:43:52 +0000 (16:43 -0800)]
container/list: mark must be an element of the list
The methods MoveAfter and MoveBefore of the container/list package did silently corrupt the interal structure of the list if a mark element is used which is not an element of the list.
Nick Craig-Wood [Fri, 14 Feb 2014 00:19:38 +0000 (16:19 -0800)]
math/big: Optimise ARM assembler
Tweak the ARM assembler to improve its performance.
* Use TEQ instead of CMP which preserves the carry flag. This means
we can avoid saving and restoring CPSR which is very slow.
* Use conditional instructions to read the value of the carry flag.
* Use 3 argument ARM instructions to save instructions
* Improve scheduling for MOVW instructions (LDR)
* Use RSB constant to save an instruction in bitLen
Results of -test.bench 'VV|VW|VU|WW|Bit' -test.benchtime 3s on Samsung
Exynos5 Chromebook.
There are a few small regressions in the benchmarks which I believe to
be noise, perhaps due to different cacheline alignment.
The changes to bitLen are apparently no faster, however less
instructions means less I-cache usage which is a win. I suspect it
will be a win on older ARM processors.
David Symonds [Thu, 13 Feb 2014 23:15:38 +0000 (10:15 +1100)]
net/http: increase panic stack trace buffer size from 4 KB to 64 KB.
4 KB is a bit too small in some situations (e.g. panic during a
template execution), and ends up with an unhelpfully-truncated trace.
64 KB should be much more likely to capture the useful information.
There's not a garbage generation issue, since this code should only
be triggered when there's something seriously wrong with the program.
David du Colombier [Thu, 13 Feb 2014 19:30:42 +0000 (20:30 +0100)]
lib9: fix Solaris build
In file included from src/lib9/utf/utfecpy.c:17:0:
src/lib9/utf/utfdef.h:28:0: error: "nil" redefined [-Werror]
In file included from src/lib9/utf/utfrrune.c:17:0:
src/lib9/utf/utfdef.h:28:0: error: "nil" redefined [-Werror]
Carlos Castillo [Thu, 13 Feb 2014 18:11:44 +0000 (10:11 -0800)]
cmd/go, go/build: support .m files
go/build is changed to list the .m files in a package, and match them for build constraints, adding them to a new field: Package.MFiles.
The go tool is changed to support building .m files and linking in the results during CGO and SWIG builds. This means packages that create a C interface to calls Objective-C code from go are now go-gettable without producing and distributing .syso files. This change is analogous to the one in Go 1.2 made to support C++ built code.
This change doesn't support .mm files (Objective C++).
Also added support for these MFiles to go list's -json mode.
Michael T. Jones [Thu, 13 Feb 2014 16:42:19 +0000 (08:42 -0800)]
math/big: add support for general encoding interfaces
TextMarshaller and TextUnmarshaller to ease transport of
unlimited precision rational numbers.
Fixes #7287.
Consists of encode and decode functions and two test
functions, one using JSON and one using XML. Each
verifies round trips for integers (rationals with
denominator == 1) and for fractional vaues.
Dmitriy Vyukov [Thu, 13 Feb 2014 16:15:19 +0000 (20:15 +0400)]
cmd/gc: fix windows build
c:\src\go\pkg\obj\windows_amd64\libgc.a(lex.o): In function `catcher':
c:/src/go/src/cmd/gc/lex.c:181: undefined reference to `noted'
Dmitriy Vyukov [Thu, 13 Feb 2014 15:36:45 +0000 (19:36 +0400)]
runtime: fix concurrent GC sweep
The issue was that one of the MSpan_Sweep callers
was doing sweep with preemption enabled.
Additional checks are added.
LGTM=rsc
R=rsc, dave
CC=golang-codereviews
https://golang.org/cl/62990043
Nicolas Owens [Thu, 13 Feb 2014 15:26:16 +0000 (10:26 -0500)]
net: only return unique hosts during hostname lookup on plan 9
TestLookupHost expects that no duplicate addresses are returned. when cs is consulted for a name, e.g net!localhost!1, it will possibly return multiple available paths, e.g. via il and tcp. this confuses the tests.
Russ Cox [Thu, 13 Feb 2014 05:06:46 +0000 (05:06 +0000)]
codereview: warn about 'hg upload' not uploading
This seems to be what causes the 'chunk mismatch' errors.
I don't know why the server tells us there's nothing to upload,
but at the least we can warn that it did, so that the user
will try again.
Russ Cox [Thu, 13 Feb 2014 03:54:55 +0000 (03:54 +0000)]
cmd/5g: fix regopt bug in copyprop
copyau1 was assuming that it could deduce the type of the
middle register p->reg from the type of the left or right
argument: in CMPF F1, F2, the p->reg==2 must be a D_FREG
because p->from is F1, and in CMP R1, R2, the p->reg==2 must
be a D_REG because p->from is R1.
This heuristic fails for CMP $0, R2, which was causing copyau1
not to recognize p->reg==2 as a reference to R2, which was
keeping it from properly renaming the register use when
substituting registers.
cmd/5c has the right approach: look at the opcode p->as to
decide the kind of register. It is unclear where 5g's copyau1
came from; perhaps it was an attempt to avoid expanding 5c's
a2type to include new instructions used only by 5g.
Copy a2type from cmd/5c, expand to include additional instructions,
and make it crash the compiler if asked about an instruction
it does not understand (avoid silent bugs in the future if new
instructions are added).
Should fix current arm build breakage.
While we're here, fix the print statements dumping the pred and
succ info in the asm listing to pass an int arg to %.4ud
(Prog.pc is a vlong now, due to the liblink merge).
Dmitriy Vyukov [Wed, 12 Feb 2014 18:31:36 +0000 (22:31 +0400)]
runtime: improve cpu profiles for GC/syscalls/cgo
Current "System->etext" is not very informative.
Add parent "GC" frame.
Replace un-unwindable syscall/cgo frames with Go stack that leads to the call.
Nick Craig-Wood [Wed, 12 Feb 2014 18:24:52 +0000 (13:24 -0500)]
An ARM version of sha1block.go with a big improvement in throughput
(up to 2.8x).
This is a partially unrolled version which performs better for small
hashes and only sacrifices a small amount of ultimate speed to a fully
unrolled version which uses 5k of code.
Code size
Before 1636 bytes
After 1880 bytes
15% larger
Benchmarks on Samsung Exynos 5 ARMv7 Chromebook
benchmark old ns/op new ns/op delta
BenchmarkHash8Bytes 1907 1136 -40.43%
BenchmarkHash1K 20280 7547 -62.79%
BenchmarkHash8K 148469 52576 -64.59%
benchmark old MB/s new MB/s speedup
BenchmarkHash8Bytes 4.19 7.04 1.68x
BenchmarkHash1K 50.49 135.68 2.69x
BenchmarkHash8K 55.18 155.81 2.82x
LGTM=dave, agl
R=dave, bradfitz, agl, adg, nick
CC=golang-codereviews
https://golang.org/cl/56990044
Dmitriy Vyukov [Wed, 12 Feb 2014 18:24:29 +0000 (22:24 +0400)]
runtime: refactor level-triggered IO support
Remove GOOS_solaris ifdef from netpoll code,
instead introduce runtime edge/level triggered IO flag.
Replace armread/armwrite with a single arm(mode) function,
that's how all other interfaces look like and these functions
will need to do roughly the same thing anyway.
Dmitriy Vyukov [Wed, 12 Feb 2014 18:21:38 +0000 (22:21 +0400)]
runtime: refactor chan code
1. Make internal chan functions static.
2. Move selgen local variable instead of a member of G struct.
3. Change "bool *pres/selected" parameter of chansend/chanrecv to "bool block",
which is simpler, faster and less code.
-37 lines total.
Dmitriy Vyukov [Wed, 12 Feb 2014 18:16:42 +0000 (22:16 +0400)]
runtime: concurrent GC sweep
Moves sweep phase out of stoptheworld by adding
background sweeper goroutine and lazy on-demand sweeping.
It turned out to be somewhat trickier than I expected,
because there is no point in time when we know size of live heap
nor consistent number of mallocs and frees.
So everything related to next_gc, mprof, memstats, etc becomes trickier.
At the end of GC next_gc is conservatively set to heap_alloc*GOGC,
which is much larger than real value. But after every sweep
next_gc is decremented by freed*GOGC. So when everything is swept
next_gc becomes what it should be.
For mprof I had to introduce 3-generation scheme (allocs, revent_allocs, prev_allocs),
because by the end of GC we know number of frees for the *previous* GC.
Significant caution is required to not cross yet-unknown real value of next_gc.
This is achieved by 2 means:
1. Whenever I allocate a span from MCentral, I sweep a span in that MCentral.
2. Whenever I allocate N pages from MHeap, I sweep until at least N pages are
returned to heap.
This provides quite strong guarantees that heap does not grow when it should now.
Dmitriy Vyukov [Wed, 12 Feb 2014 17:50:58 +0000 (21:50 +0400)]
encoding/json: fix test failure
$ go test -cpu=1,1,1,1,1,1,1,1,1 encoding/json
--- FAIL: TestIndentBig (0.00 seconds)
scanner_test.go:131: Indent(jsonBig) did not get bigger
On 4-th run initBig generates an empty array.
Daniel Morsing [Tue, 11 Feb 2014 20:25:40 +0000 (20:25 +0000)]
cmd/6g, cmd/8g, cmd/5g: make the undefined instruction have no successors
The UNDEF instruction was listed in the instruction data as having the next instruction in the stream as its successor. This confused the optimizer into adding a load where it wasn't needed, in turn confusing the liveness analysis pass for GC bitmaps into thinking that the variable was live.
Dmitriy Vyukov [Tue, 11 Feb 2014 09:41:46 +0000 (13:41 +0400)]
runtime: do not profile blocked netpoll on windows
There is frequently a thread hanging on GQCS,
currently it skews profiles towards netpoll,
but it is not bad and is not consuming any resources.
Brendan Daniel Tracey [Tue, 11 Feb 2014 01:27:31 +0000 (20:27 -0500)]
cmd/gc: change compile error to "use of package %S without selector"
At present, when a package identifier is used outside of a selector expression, gc gives the error "use of package %S outside selector". However, in the selector expression x.f, the spec defines f as the selector. This change makes the error clearer.
Dmitriy Vyukov [Mon, 10 Feb 2014 16:24:47 +0000 (20:24 +0400)]
runtime: fix crash during cpu profiling
mp->mcache can be concurrently modified by runtime·helpgc.
In such case sigprof can remember mcache=nil, then helpgc sets it to non-nil,
then sigprof restores it back to nil, GC crashes with nil mcache.