cmd/go, testing: streamline direct use of test binaries
Before:
$ go test -c -cover fmt
$ ./fmt.test -test.covermode=set
PASS
coverage: 65.1% of statements in strconv
$
After:
$ go test -c -cover fmt
$ ./fmt.test
PASS
coverage: 65.1% of statements in strconv
$
In addition to being cumbersome, the old flag didn't make sense:
the cover mode cannot be changed after the binary has been built.
Another useful effect of this CL is that if you happen to do
$ go test -c -covermode=atomic fmt
and then forget you did that and run benchmarks,
the final line of the output (the coverage summary) reminds you
that you are benchmarking with coverage enabled, which might
not be what you want.
We put this in because Marshal encoded named []byte but Unmarshal rejected them.
And we noticed that Marshal's behavior was undocumented so we documented it.
But I am starting to think the docs and Unmarshal were correct and Marshal's
behavior was the problem.
Rolling back to give us more time to think.
««« original CL description
json: unmarshal types that are byte slices.
The json package cheerfully would marshal
type S struct {
IP net.IP
}
but would give an error when unmarshalling. This change allows any
type whose concrete type is a byte slice to be unmarshalled from a
string.
encoding/json: coerce invalid UTF-8 to valid UTF-8 during Marshal
In practice, rejecting an entire structure due to a single invalid byte
in a string is just too picky, and too hard to track down.
Be consistent with the bulk of the standard library by converting
invalid UTF-8 into UTF-8 with replacement runes.
Shenghou Ma [Fri, 12 Jul 2013 18:00:07 +0000 (02:00 +0800)]
run.bash: enlarge timeout of runtime tests
Recently addition to runtime test makes it take very close to 720s
of timeout limit on the netbsd-arm-qemu builder.
cmd/5g, cmd/6g, cmd/8g: fix line number of caller of deferred func
Deferred functions are not run by a call instruction. They are run by
the runtime editing registers to make the call start with a caller PC
returning to a
CALL deferreturn
instruction.
That instruction has always had the line number of the function's
closing brace, but that instruction's line number is irrelevant.
Stack traces show the line number of the instruction before the
return PC, because normally that's what started the call. Not so here.
The instruction before the CALL deferreturn could be almost anywhere
in the function; it's unrelated and its line number is incorrect to show.
Fix the line number by inserting a true hardware no-op with the right
line number before the returned-to CALL instruction. That is, the deferred
calls now appear to start with a caller PC returning to the second instruction
in this sequence:
NOP
CALL deferreturn
The traceback will show the line number of the NOP, which we've set
to be the line number of the function's closing brace.
The NOP here is not the usual pseudo-instruction, which would be
elided by the linker. Instead it is the real hardware instruction:
XCHG AX, AX on 386 and amd64, and AND.EQ R0, R0, R0 on ARM.
Fixes #5856.
R=ken2, ken
CC=golang-dev
https://golang.org/cl/11223043
If the stack frame size is larger than the known-unmapped region at the
bottom of the address space, then the stack split prologue cannot use the usual
condition:
SP - size >= stackguard
because SP - size may wrap around to a very large number.
Instead, if the stack frame is large, the prologue tests:
SP - stackguard >= size
(This ends up being a few instructions more expensive, so we don't do it always.)
Preemption requests register by setting stackguard to a very large value, so
that the first test (SP - size >= stackguard) cannot possibly succeed.
Unfortunately, that same very large value causes a wraparound in the
second test (SP - stackguard >= size), making it succeed incorrectly.
To avoid *that* wraparound, we have to amend the test:
cmd/go, testing: remove redundant "for pkg" in coverage message
This is a transcript before this change. I've capitalized the text being removed.
Note that it is always near another line that already says fmt, marked with <<<
$ cd $GOROOT/src/pkg/fmt
$ go test -cover
PASS
coverage FOR FMT: 91.3% of statements
ok fmt 0.040s <<<
$ go test -coverpkg strconv
PASS
coverage FOR FMT: 64.9% of statements in strconv
ok fmt 0.039s <<<
$ go test -cover -c
$ ./fmt.test -test.covermode=set <<<
PASS
coverage FOR FMT: 91.3% of statements
$ go test -coverpkg strconv -c
$ ./fmt.test -test.covermode=set <<<
PASS
coverage FOR FMT: 64.9% of statements in strconv
That the summary printed by 'go test [options] fmt' is unchanged:
$ go test -cover fmt
ok fmt 0.040s coverage: 91.3% of statements
$ go test -coverpkg strconv fmt
ok fmt 0.038s coverage: 64.9% of statements in strconv
The current cas64 definition hard-codes the x86 behavior
of updating *old with the new value when the cas fails.
This is inconsistent with cas32 and casp.
Make it consistent.
This means that the cas64 uses will be epsilon less efficient
than they might be, because they have to do an unnecessary
memory load on x86. But so be it. Code clarity and consistency
is more important.
misc/cgo/test: make test work with -linkmode=internal
The static func named thread in issue5337.go's C snippet
conflicts with the static func named thread in issue3350.go's C snippet.
I don't know why (they're both static) but I also don't care,
because -linkmode=internal only needs to be able to handle
the cgo in the standard library, and it does.
Change the test to avoid this problem.
Fixes build (after run.bash is fixed to detect the breakage).
STRINGSZ (200) is fine for lines generated by things like
instruction dumps, but an error containing a couple file
names can easily exceed that, especially on Macs with
the ridiculous default $TMPDIR.
Shenghou Ma [Thu, 11 Jul 2013 20:35:53 +0000 (04:35 +0800)]
cmd/cgo: silence two gcc warnings for *.cgo2.c
1. "int e;" is unused, generating "unused variable" error.
2. a->e was typed void *[2], but was accessed with *(int *)(a->e), this
generated "dereferencing type-punned pointer will break strict-aliasing rules" error.
runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes #5586.
This is reincarnation of cl/9776044 with the bug fixed.
The bug was due to code added after cl/9776044 was created:
if(tick - (((uint64)tick*0x4325c53fu)>>36)*61 == 0 && runtime·sched.runqsize > 0) {
runtime·lock(&runtime·sched);
gp = globrunqget(m->p, 1);
runtime·unlock(&runtime·sched);
}
If M gets gp from global runq here, it does not reset m->spinning.
runtime: adjust traceback for new preemptive reality
Currently it crashes as follows:
fatal error: unknown pc
...
goroutine 71698 [runnable]:
runtime.racegoend()
src/pkg/runtime/race.c:171
runtime.goexit()
src/pkg/runtime/proc.c:1276 +0x9
created by runtime_test.testConcurrentReadsAfterGrowth
src/pkg/runtime/map_test.go:264 +0x332
Robert Griesemer [Wed, 10 Jul 2013 04:12:53 +0000 (21:12 -0700)]
spec: define notion of named type
The notion of a named type is crucial for the definition
of type identity, assignability, definitions of methods.
Explicitly introduce the notion with an extra sentence.
Rob Pike [Tue, 9 Jul 2013 23:52:36 +0000 (09:52 +1000)]
cmd/go: fix a couple of bugs in coverage tooling
Merging a couple of CLs into one, since they collided in my client
and I'm lazy.
1) Fix up output in "go test -cover" case.
We need to tell the testing package the name of the package being tested
and the name of the package being covered. It can then sort out the report.
2) Filter out the _test.go files from coverage processing. We want to measure
what the tests cover, not what's covered in the tests,
The coverage for encoding/gob goes from 82.2% to 88.4%.
There may be a cleaner way to do this - suggestions welcome - but ça suffit.
Shenghou Ma [Sun, 7 Jul 2013 15:06:30 +0000 (23:06 +0800)]
cmd/go: fix "go get -u" for git repositories.
CL 10869046 changed cmd/go to checkout master branch, so
for "go get -u" to work, it must "git pull" instead of
"git fetch". Added "--ff-only" so that it won't accidentally
overwrite user changes.
Adam Langley [Tue, 2 Jul 2013 23:58:56 +0000 (19:58 -0400)]
crypto/tls: implement TLS 1.2.
This does not include AES-GCM yet. Also, it assumes that the handshake and
certificate signature hash are always SHA-256, which is true of the ciphersuites
that we currently support.
Shenghou Ma [Tue, 2 Jul 2013 16:33:38 +0000 (00:33 +0800)]
runtime: fix runtime.sigreturn_tramp for NetBSD/ARM
using m->tls[0] to save ucontext pointer is not re-entry safe, and
the old code didn't set it before the early return when signal is
received on non-Go threads.
so misc/cgo/test used to hang when testing issue 5337.
Daniel Morsing [Tue, 2 Jul 2013 15:12:08 +0000 (17:12 +0200)]
cmd/gc: fix issue with method wrappers not having escape analysis run on them.
Escape analysis needs the right curfn value on a dclfunc node, otherwise it will not analyze the function.
When generating method value wrappers, we forgot to set the curfn correctly.
Jeff R. Allen [Tue, 2 Jul 2013 01:42:29 +0000 (21:42 -0400)]
time: prevent a panic from leaving the timer mutex held
When deleting a timer, a panic due to nil deref
would leave a lock held, possibly leading to a deadlock
in a defer. Instead return false on a nil timer.
This CL provides stable in-place sorting by use of
bottom up merge sort with in-place merging done by
the SymMerge algorithm from P.-S. Kim and A. Kutzner.
The additional space needed for stable sorting (in the form of
stack space) is logarithmic in the inputs size n.
Number of calls to Less and Swap grow like O(n * log n) and
O(n * log n * log n):
Stable sorting random data uses significantly more calls
to Swap than the unstable quicksort implementation (5 times more
on n=100, 10 times more on n=1e4 and 23 times more on n=1e8).
The number of calls to Less is practically the same for Sort and
Stable.
Stable sorting 1 million random integers takes 5 times longer
than using Sort.
Andrew Gerrand [Mon, 1 Jul 2013 22:44:25 +0000 (08:44 +1000)]
cmd/godoc: set up playground for examples that are already visible
This fixes an issue where linking directly to an example makes it not
runnable and visible only in a tiny window. To see the bug in action, link: http://golang.org/pkg/strings/#example_Map
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10679050
runtime: fix memory leaks due to defers
fn can clearly hold a closure in memory.
argp/pc point into stack and so can hold
in memory a block that was previously
a large stack serment.
Depending on net/http means depending on cgo.
When the tree is in a shaky state it's nice to see sync/atomic
pass even if cgo or net causes broken binaries.
R=golang-dev, dave, r
CC=golang-dev
https://golang.org/cl/10753044
Russ Cox [Fri, 28 Jun 2013 20:37:07 +0000 (13:37 -0700)]
runtime: shorten hash lookup stack frames
On amd64 the frames are very close to the limit for a
nosplit (textflag 7) function, in part because the C compiler
does not make any attempt to reclaim space allocated for
completely registerized variables. Avoid a few short-lived
variables to reclaim two words.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10758043
Dmitriy Vyukov [Fri, 28 Jun 2013 14:37:06 +0000 (18:37 +0400)]
runtime: introduce GODEBUG env var
Currently it replaces GOGCTRACE env var (GODEBUG=gctrace=1).
The plan is to extend it with other type of debug tracing,
e.g. GODEBUG=gctrace=1,schedtrace=100.
Dmitriy Vyukov [Fri, 28 Jun 2013 13:52:17 +0000 (17:52 +0400)]
runtime: preempt goroutines for GC
The last patch for preemptive scheduler,
with this change stoptheworld issues preemption
requests every 100us.
Update #543.
Russ Cox [Thu, 27 Jun 2013 20:51:06 +0000 (16:51 -0400)]
runtime: use gp->sched.sp for stack overflow check
On x86 it is a few words lower on the stack than m->morebuf.sp
so it is a more precise check. Enabling the check requires recording
a valid gp->sched in reflect.call too. This is a good thing in general,
since it will make stack traces during reflect.call work better, and it
may be useful for preemption too.
Dmitriy Vyukov [Thu, 27 Jun 2013 20:49:53 +0000 (00:49 +0400)]
runtime: fix goroutine status corruption
runtime.entersyscall() sets g->status = Gsyscall,
then calls runtime.lock() which causes stack split.
runtime.newstack() resets g->status to Grunning.
This will lead to crash during GC (world is not stopped) or GC will scan stack incorrectly.
On my 64-bit machine, despite being 32-bit code, fixed-base
multiplications are 7.1x faster and arbitary multiplications are 2.6x
faster.
It is difficult to review this change. However, the code is essentially
the same as code that has been open-sourced in Chromium. There it has
been successfully performing P-256 operations for several months on
many machines so the arithmetic of the code should be sound.
««« original CL description
runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes #5586.
Dmitriy Vyukov [Thu, 27 Jun 2013 16:52:12 +0000 (20:52 +0400)]
runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes #5586.