Rob Pike [Tue, 22 Sep 2015 21:23:32 +0000 (14:23 -0700)]
cmd/go: fix processing of flags for test binaries.
The usage message says:
test [-c] [-i] [build and test flags] [packages] [flags for test binary]
but this was not what was implemented. Instead, after packages are named,
flag processing continues, which makes it impossible, for example, to pass
to the binary a flag with the same name as a test flag. This was triggered
by the -v flag in glog.
Attempting to run this test with go test pkg -v=7 would give a usage
message. This change allows it. In fact it allows
go test -v pkg -v=7
The solution is to implement the usage message. One compatibility
issue is that flags after the package name are no longer processed
as test flags, so this no longer works:
go test foo -cover
One must write
go test -cover foo
I do not think this is onerous but it must be called out in the
release notes.
Fixes #12177.
Change-Id: Ib9267884b47a6b0c183efa888ec78333272113aa
Reviewed-on: https://go-review.googlesource.com/14826 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Wed, 23 Sep 2015 17:05:44 +0000 (10:05 -0700)]
math/big: factored out an internal accessor method (cleanup), added benchmark
Current result of DecimalConversion benchmark (for future reference):
BenchmarkDecimalConversion-8 10000 204770 ns/op
Measured on Mac Mini (late 2012) running OS X 10.10.5,
2.3 GHz Intel Core i7, 8 GB 1333 MHz DDR3.
Also: Removed comment suggesting to implement decimal by representing
digits as numbers 0..9 rather than ASCII chars '0'..'9' to avoid
repeated +/-'0' operations. Tried and it appears (per above benchmark)
that the +/-'0' operations are neglibile but the addition conversion
passes around it are not and that it makes things significantly slower.
Change-Id: I6ee033b1172043248093cc5d02abff5fc54c2e7a
Reviewed-on: https://go-review.googlesource.com/14857 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alan Donovan <adonovan@google.com>
The existing comment for regex.Split contains a plain text example,
while many of the other regex functions have runnable examples. This
change provides a runnable example for Split.
Change-Id: I5373f57f532fe843d7d0adcf4b513061ec797047
Reviewed-on: https://go-review.googlesource.com/14737 Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Robert Griesemer [Tue, 22 Sep 2015 16:37:56 +0000 (09:37 -0700)]
test/fixedbugs: update overly restrictive test case
See discussion in https://go-review.googlesource.com/14830 .
Change-Id: I94f25f92b8cdaa509d2c335865a645228425804d
Reviewed-on: https://go-review.googlesource.com/14837 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Sun, 20 Sep 2015 01:24:16 +0000 (18:24 -0700)]
math/big: optimize Float.Parse by reducing powers of 10 to powers of 2 and 5
Instead of computing the final adjustment factor as a power of 10,
it's more efficient to split 10**e into 2**e * 5**e . Powers of 2
are trivially added to the Float exponent, and powers of 5 are
smaller and thus faster to compute.
Also, use a table of uint64 values rather than float64 values for
initial power value. uint64 values appear to be faster to convert
to Floats (useful for small exponents).
Added two small benchmarks to confirm that there's no regresssion.
benchmark old ns/op new ns/op delta
BenchmarkParseFloatSmallExp-8 17543 16220 -7.54%
BenchmarkParseFloatLargeExp-8 60865 59996 -1.43%
Change-Id: I3efd7556b023316f86f334137a67fe0c6d52f8ef
Reviewed-on: https://go-review.googlesource.com/14782 Reviewed-by: Alan Donovan <adonovan@google.com>
This change addresses an integer underflow appearing only on systems
using a 32-bit int type. The patch addresses the problem by limiting the
length of unknown chunks to 0x7fffffff. This value appears to already be
checked for when parsing other chunk types, so the bug shouldn't appear
elsewhere in the package. The PNG spec recommends the maximum size for
any chunk to remain under 2^31, so this shouldn't cause errors with
valid images.
Fixes #12687
Change-Id: I17f0e1683515532c661cf2b0b2bc65309d1b7bb7
Reviewed-on: https://go-review.googlesource.com/14766 Reviewed-by: Nigel Tao <nigeltao@golang.org>
Rob Pike [Mon, 21 Sep 2015 18:06:43 +0000 (11:06 -0700)]
cmd/vet: build the binary only once in the test
Recent changes caused vet to build the binary for each Test function.
This is wasteful and will become only more so as more tests are added.
Use testing.Main to build only once.
Verified that compilation errors still appear if the binary cannot be
built.
Before:
real 0m11.169s
user 0m18.328s
sys 0m2.152s
After:
real 0m5.132s
user 0m9.404s
sys 0m1.168s
Of course if the compiler were fast we might not notice, but vet is
a big program and growing bigger all the time, as are the tests.
The fields step and redoState of struct scanner are now defined as
`func(s *scanner, c byte) int` instead of
`func(s *scanner, c int) int`, since bytes are sufficient.
Further changes improve the consistency in the scanner.go file.
Change-Id: Ifb85f2130d728d2b936d79914d87a1f0b5c6ee7d
Reviewed-on: https://go-review.googlesource.com/14801 Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Sometimes this read is instrumented by compiler when it creates
a temp to take address, but sometimes it is not (e.g. for global vars
compiler takes address of the global directly).
Instrument convT2E/I similarly to chansend and mapaccess.
Dave Cheney [Fri, 18 Sep 2015 08:56:47 +0000 (18:56 +1000)]
cmd/compile/internal/gc: move intLiteral to gc.Node
intLiteral is used by the gins wrappers in arm64, ppc64 and
mips64. Refactor the function to a method on gc.Node and update
the callers to use the common copy.
Change-Id: I2db90d801a9cb18f8526eb921e13daa75ca1cf6f
Reviewed-on: https://go-review.googlesource.com/14744 Reviewed-by: Aram Hăvărneanu <aram@mgk.ro> Reviewed-by: Dave Cheney <dave@cheney.net>
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
runtime: remove flaky TestInvalidptrCrash to fix build
This test fails on arm64 and some amd64 OSs and fails on Linux/amd64
if you remove the first runtime.GC(), which should be unnecessary, and
run it in all.bash (but not if you run it in isolation). I don't
understand any of these failures, so for now just remove this test.
Currently when the GC prints an object for debugging (e.g., for a
failed invalidptr or checkmark check), it dumps the entire object. To
avoid inundating the user with output for really large objects, limit
this to printing just the first 128 words (which are most likely to be
useful in identifying the type of an object) and the 32 words around
the problematic field.
Change-Id: Id94a5c9d8162f8bd9b2a63bf0b1bfb0adde83c68
Reviewed-on: https://go-review.googlesource.com/14764 Reviewed-by: Rick Hudson <rlh@golang.org>
By default, the runtime panics if it detects a pointer to an
unallocated span. At this point, this usually catches bad uses of
unsafe or cgo in user code (though it could also catch runtime bugs).
Unfortunately, the rather cryptic error misleads users, offers users
little help with debugging their own problem, and offers the Go
developers little help with root-causing.
Improve the error message in various ways. First, the wording is
improved to make it clearer what condition was detected and to suggest
that this may be the result of incorrect use of unsafe or cgo. Second,
we add a dump of the object containing the bad pointer so that there's
at least some hope of figuring out why a bad pointer was stored in the
Go heap.
Change-Id: I57b91b12bc3cb04476399d7706679e096ce594b9
Reviewed-on: https://go-review.googlesource.com/14763 Reviewed-by: Rick Hudson <rlh@golang.org>
Rob Pike [Tue, 15 Sep 2015 21:14:44 +0000 (14:14 -0700)]
bufio: allow Scanner to accept a user-provided buffer
Add Scanner.Buffer, which lets the user give a buffer to
the scanner and set the maximum token size.
We call it Buffer not SetBuffer for consistency with Split, which
perhaps should have been called SetSplit; too late regardless.
Both Buffer and Split panic if they are called after Scan. The
panic in Split is new, but the comment on the method already
said it needed to be called first, so we might as well add the
verification while we're doing it for Buffer.
This method allows precise user control of storage.
Fixes #11702.
Change-Id: I80e3d0e3830562fdabd4f7b08f322e1378248c39
Reviewed-on: https://go-review.googlesource.com/14599 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: roger peppe <rogpeppe@gmail.com>
Dave Cheney [Thu, 10 Sep 2015 05:57:39 +0000 (15:57 +1000)]
cmd/compile: convert externdecl to []*Node
This one of a set of changes to make the transition away from NodeList
easier by removing cases in which NodeList doesn't act semi-trivially like a
[]*Node.
This CL was originally prepared by Josh Bleecher Snyder <josharian@gmail.com>.
This change passes go build -toolexec 'toolstash -cmp' -a std.
Shenghou Ma [Fri, 11 Sep 2015 04:26:50 +0000 (00:26 -0400)]
cmd/go: provide full path as os.Args[0] when invoking tools
cmd/dist needs to re-exec or open itself to detect GOARM (CL 3973) and
detect host machine endianness (CL 14460).
Change-Id: If6438831ab0715ba8e236d64bb2c7c1bde1470aa
Reviewed-on: https://go-review.googlesource.com/14476 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Rob Pike [Tue, 15 Sep 2015 16:27:22 +0000 (09:27 -0700)]
text/template: export isTrue
The definition of 'truth' used by if etc. is not trivial to compute, so publish
the implementation to allow custom template functions to have the
same definition as the template language itself.
Fixes #12033.
Change-Id: Icdfd6039722d7d3f984ba0905105eb3253e14831
Reviewed-on: https://go-review.googlesource.com/14593 Reviewed-by: Andrew Gerrand <adg@golang.org>
Rob Pike [Thu, 17 Sep 2015 18:17:02 +0000 (11:17 -0700)]
encoding/gob: document allocation/merge behavior
This is understood, obvious (to me), and well known but has not been clearly documented.
Fixes #11117.
Change-Id: Ib2b1e318924748d1eac0d735ad6286533be7fd39
Reviewed-on: https://go-review.googlesource.com/14693 Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Crawshaw [Thu, 17 Sep 2015 16:13:53 +0000 (12:13 -0400)]
misc/ios: skip revoked certificates
Change-Id: If65e5e55b359a61740d2ef185147bb6df90e0b0c
Reviewed-on: https://go-review.googlesource.com/14654 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Shawn Walker-Salas [Tue, 18 Aug 2015 21:13:38 +0000 (14:13 -0700)]
runtime/trace: fix tracing of blocking system calls
The placement and invocation of traceGoSysCall when using
entersyscallblock() instead of entersyscall() differs enough that the
TestTraceSymbolize test can fail on some platforms.
This change moves the invocation of traceGoSysCall for entersyscall() so
that the same number of "frames to skip" are present in the trace as when
entersyscallblock() is used ensuring system call traces remain identical
regardless of internal implementation choices.
The man page for sigaction(2) on OS X doesn't guarantee that SA_RESTART
will work for open(2) on regular files:
The affected system calls include open(2), read(2), write(2),
sendto(2), recvfrom(2), sendmsg(2) and recvmsg(2) on a
communications channel or a slow device (such as a terminal, but not
a regular file) and during a wait(2) or ioctl(2).
I've never observed EINTR from open(2) for a traditional file system
such as HFS+, but it's easy to observe with a fuse file system that is
slightly slow (cf. https://goo.gl/UxsVgB). After this change, the
problem can no longer be reproduced when calling os.OpenFile.
Fixes #11180.
Change-Id: I967247430e20a7d29a285b3d76bf3498dc4773db
Reviewed-on: https://go-review.googlesource.com/14484 Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/go: indent first test binary flag description for `go test -h`
Fixes #12642
Change-Id: I0b94437055b7d444f5caf7ea310e85357c467bdf
Reviewed-on: https://go-review.googlesource.com/14612 Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Crawshaw [Tue, 15 Sep 2015 17:40:24 +0000 (13:40 -0400)]
runtime: preserve R11 in darwin/arm entrypoint
The _rt0_arm_darwin_lib entrypoint has to conform to the darwin ARMv7
calling convention, which requires functions to preserve the value of
R11. Go uses R11 as the liblink REGTMP register, so save it manually.
Also avoid using R4, which is also callee-save.
Fixes #12590
Change-Id: I9c3b374e330f81ff8fc9c01fa20505a33ddcf39a
Reviewed-on: https://go-review.googlesource.com/14603 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Hudson-Doyle [Wed, 16 Sep 2015 04:20:31 +0000 (16:20 +1200)]
test: move allocation before munmap in recover4
recover4 allocates 16 pages of memory via mmap, makes a 4 page hole in it with
munmap, allocates another 16 pages of memory via normal allocation and then
tries to copy from one to the other. For some reason on arm64 (but no other
platform I have tested) the second allocation sometimes causes the runtime to
ask the kernel for 4 additional pages of memory -- which the kernel satisfies
by remapping the pages that were just unmapped!
Moving the second allocation before the munmap fixes this behaviour, I can run
recover4 tens of thousands of times without failure with this fix vs a failure
rate of ~0.5% before.
Fixes #12549
Change-Id: I490b895b606897e4f7f25b1b51f5d485a366fffb
Reviewed-on: https://go-review.googlesource.com/14632 Reviewed-by: Dave Cheney <dave@cheney.net>
Found with https://github.com/remyoudompheng/go-misc/deadcode:
deadcode: walk.go:2228:1: applywritebarrier_bv is unused
deadcode: subr.go:355:1: gethunk is unused
deadcode: subr.go:1991:1: localexpr is unused
deadcode: dcl.go:82:1: poptodcl is unused
deadcode: swt.go:810:1: dumpcase is unused
deadcode: esc.go:251:1: satAdd8 is unused
deadcode: esc.go:387:1: outputsPerTag is unused
deadcode: obj.go:190:1: duint64 is unused
deadcode: obj.go:287:1: dstringptr is unused
deadcode: plive.go:95:1: xmalloc is unused
deadcode: plive.go:119:1: freeblock is unused
followed by
deadcode: go.go:633:1: hunk is unused
deadcode: go.go:635:1: nhunk is unused
deadcode: go.go:637:1: thunk is unused
after 'gethunk' was removed.
Some dead code in bv.go, mparith3.go, and dcl.go was left as is.
Passes go build -a -toolexec 'toolstash -cmp' std cmd.
Change-Id: Ia63519adedc8650d7095572ddd454fd923d3204d
Reviewed-on: https://go-review.googlesource.com/14610 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Rob Pike [Tue, 15 Sep 2015 17:59:27 +0000 (10:59 -0700)]
internal/obj: protect against nil addr.Sym
This has been the root cause of a number of crashes caused by
fuzz throwing modem noise at the assembler, which in turn attempts
to print diagnostics but instead just gets crashes.
Fixes #12627.
Change-Id: I72c2da79d8eb240e1a37aa6140454c552b05e0f1
Reviewed-on: https://go-review.googlesource.com/14595 Reviewed-by: Ian Lance Taylor <iant@golang.org>
This adds a test that debug/dwarf can read the skeleton DWARF data
from a split DWARF image (though it doesn't currently support piecing
the external DWARF data back together). This should work because
there's nothing particularly different about skeleton DWARF data, but
previously failed because of poor handling of unrecognized attributes.
Updates #12592.
Change-Id: I2fc5f4679883b05ebd7ec9f0b5c398a758181a32
Reviewed-on: https://go-review.googlesource.com/14542 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: jcd . <jcd@golang.org>
debug/dwarf: return ClassUnknown if attribute class cannot be determined
Currently, if the .debug_abbrev section of an ELF file contains
attributes that aren't known to the dwarf package and that have form
formSecOffset, the dwarf package will fail to open the DWARF data with
an error like "decoding dwarf section abbrev at offset 0x17: cannot
determine class of unknown attribute with formSecOffset". For the most
part, the class is implied by the form encoded in the abbrev section,
but formSecOffset can imply many different DWARF classes. Hence,
debug/dwarf disambiguates these using a table of known attributes.
However, it will reject the entire image if it encounters an attribute
it can't determine the class of. This is particularly unfortunate
because the caller may never even uses the offending attribute.
Fix this by introducing a ClassUnknown attribute class to use as a
fallback in these cases. This allows the dwarf package to load the
DWARF data and isolates the problem to just the affected attributes.
Fixes #12592.
Change-Id: I766227b136e9757f8b89c0b3ab8e9ddea899d94f
Reviewed-on: https://go-review.googlesource.com/14541 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: jcd . <jcd@golang.org>
Ian Lance Taylor [Fri, 10 Jul 2015 22:28:01 +0000 (15:28 -0700)]
runtime: on unexpected netpoll error, throw instead of looping
The current code prints an error message and then tries to carry on.
This is not helpful for Go users: they see a message that means
nothing and that they can do nothing about. In the only known case of
this message, in issue 11498, the best guess is that the netpoll code
went into an infinite loop. Instead of doing that, crash the program.
Keith Randall [Tue, 1 Sep 2015 19:53:15 +0000 (12:53 -0700)]
runtime: fix aeshash of empty string
Aeshash currently computes the hash of the empty string as
hash("", seed) = seed. This is bad because the hash of a compound
object with empty strings in it doesn't include information about
where those empty strings were. For instance [2]string{"", "foo"}
and [2]string{"foo", ""} might get the same hash.
Fix this by returning a scrambled seed instead of the seed itself.
With this fix, we can remove the scrambling done by the generated
array hash routines.
The test also rejects hash("", seed) = 0, if we ever thought
it would be a good idea to try that.
Rob Pike [Mon, 14 Sep 2015 18:27:20 +0000 (11:27 -0700)]
text/template: verify that names in FuncMap are valid identifiers
There was no verification in Funcs that the map had valid names,
which meant that the error could only be caught when parsing
the template that tried to use them. Fix this by validating the names
in Funcs and panicking before parsing if there is a bad name.
This is arguably an API change, since it didn't trigger a panic
before, but Funcs did already panic if the function itself was no
good, so I argue it's an acceptable change to add more sanity
checks.
Fixes #9685.
Change-Id: Iabf1d0602c49d830f3ed71ca1ccc7eb9a5521ff5
Reviewed-on: https://go-review.googlesource.com/14562 Reviewed-by: Andrew Gerrand <adg@golang.org>
Cleaning along the way:
-convert variable types from int to bool
-remove unnecessary functions
-remove unnecessary type conversion
-remove unnecessary variable declarations
-transform struct{string,string} with lookup to map[string]string
This change passes go build -toolexec 'toolstash -cmp' -a std.
Change-Id: I259728fe4afd7f23b67f08fab856ce0abee57b21
Reviewed-on: https://go-review.googlesource.com/14435 Reviewed-by: Ian Lance Taylor <iant@golang.org>
net: remove named parameters in Listener.Accept doc signature
They added no value.
Change-Id: I9e690379d2dfd983266de0ea5231f2b57c8b1517
Reviewed-on: https://go-review.googlesource.com/14568 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Dave Cheney [Fri, 7 Aug 2015 01:15:54 +0000 (11:15 +1000)]
cmd/go: skip external tests on linux/arm
CL 13166 skipped external tests on freebsd/arm with the rationale
that the cmd/go tests are not architecture dependent.
This CL does the same for linux/arm to help linux/arm users who are
building Go on platforms like the Raspberry Pi where ./all.bash
frequently times out due to a lack of resources.
Change-Id: Iae1a25b63b74200da3f1b5637da0fa5c2dceeb83
Reviewed-on: https://go-review.googlesource.com/13342 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Thu, 3 Sep 2015 07:48:21 +0000 (17:48 +1000)]
runtime: provide room for first 4 syscall parameters in windows usleep2
Windows amd64 requires all syscall callers to provide room for first
4 parameters on stack. We do that for all our syscalls, except inside
of usleep2. In https://codereview.appspot.com/7563043#msg3 rsc says:
"We don't need the stack alignment and first 4 parameters on amd64
because it's just a system call, not an ordinary function call."
He seems to be wrong on both counts. But alignment is already fixed.
Fix parameter space now too.
Ian Lance Taylor [Fri, 4 Sep 2015 17:58:42 +0000 (10:58 -0700)]
runtime: unblock special glibc signals on each thread
Glibc uses some special signals for special thread operations. These
signals will be used in programs that use cgo and invoke certain glibc
functions, such as setgid. In order for this to work, these signals
need to not be masked by any thread. Before this change, they were
being masked by programs that used os/signal.Notify, because it
carefully masks all non-thread-specific signals in all threads so that a
dedicated thread will collect and report those signals (see ensureSigM
in signal1_unix.go).
This change adds the two glibc special signals to the set of signals
that are unmasked in each thread.
Fixes #12498.
Change-Id: I797d71a099a2169c186f024185d44a2e1972d4ad
Reviewed-on: https://go-review.googlesource.com/14297 Reviewed-by: David Crawshaw <crawshaw@golang.org>
This is a cleanup following cc8f544, which was a minimal change to fix
issue #11617. This consolidates the two places in mSpan_Sweep that
update sweepgen. Previously this was necessary because sweepgen must
be updated before freeing the span, but we freed large spans early.
Now we free large spans later, so there's no need to duplicate the
sweepgen update. This also means large spans can take advantage of the
sweepgen sanity checking performed for other spans.
Change-Id: I23b79dbd9ec81d08575cd307cdc0fa6b20831768
Reviewed-on: https://go-review.googlesource.com/12451 Reviewed-by: Rick Hudson <rlh@golang.org>
Austin Clements [Tue, 4 Aug 2015 14:45:29 +0000 (10:45 -0400)]
runtime: split marking of span roots into 128 subtasks
Marking of span roots can represent a significant fraction of the time
spent in mark termination. Simply traversing the span list takes about
1ms per GB of heap and if there are a large number of finalizers (for
example, for network connections), it may take much longer.
Improve the situation by splitting the span scan into 128 subtasks
that can be executed in parallel and load balanced by the markroots
parallel for. This lets the GC balance this job across the Ps.
A better solution is to do this during concurrent mark, or to improve
it algorithmically, but this is a simple change with a lot of bang for
the buck.
This was suggested by Rhys Hiltner.
Updates #11485.
Change-Id: I8b281adf0ba827064e154a1b6cc32d4d8031c03c
Reviewed-on: https://go-review.googlesource.com/13112 Reviewed-by: Keith Randall <khr@golang.org>
Austin Clements [Thu, 13 Aug 2015 03:39:10 +0000 (23:39 -0400)]
runtime: fix hashing of trace stacks
The call to hash the trace stack reversed the "seed" and "size"
arguments to memhash and, hence, always called memhash with a 0 size,
which dutifully returned a hash value that depended only on the number
of PCs in the stack and not their values. As a result, all stacks were
put in to a very subset of the 8,192 buckets.
Fix this by passing these arguments in the correct order.