Keith Randall [Thu, 1 Dec 2016 00:15:32 +0000 (16:15 -0800)]
cmd/compile: generate frame pointers for otherwise frameless functions
func f() {
g()
}
We mistakenly don't add a frame pointer for f. This means f
isn't seen when walking the frame pointer linked list. That
matters for kernel-gathered profiles, and is an impediment for
issues like #16638.
To fix, allocate a stack frame even for otherwise frameless functions
like f. It is a bit tricky because we need to avoid some runtime
internals that really, really don't want one.
No test at the moment, as only kernel CPU profiles would catch it.
Tests will come with the implementation of #16638.
Quentin Smith [Wed, 30 Nov 2016 20:16:37 +0000 (15:16 -0500)]
crypto/x509: read Darwin trust settings for root CAs
Darwin separately stores bits indicating whether a root certificate
should be trusted; this changes Go to read and use those when
initializing SystemCertPool.
Unfortunately, the trust API is very slow. To avoid a delay of up to
0.5s in initializing the system cert pool, we assume that
the trust settings found in kSecTrustSettingsDomainSystem will always
indicate trust. (That is, all root certs Apple distributes are trusted.)
This is not guaranteed by the API but is true in practice.
In the non-cgo codepath, we do not have that benefit, so we must check
the trust status of every certificate. This causes about 0.5s of delay
in initializing the SystemCertPool.
On OS X 10.11 and older, the "security" command requires a certificate
to be provided in a file and not on stdin, so the non-cgo codepath
creates temporary files for each certificate, further slowing initialization.
Brad Fitzpatrick [Thu, 1 Dec 2016 16:50:00 +0000 (16:50 +0000)]
net/http: clarify Request.Context's lifetime
Reverts https://golang.org/cl/23672 and tweaks the text to clarify
HTTP/2 request cancelations also cancel the context (not just closing
the TCP conn).
Fixes #18143
Change-Id: I9f838e09b906d455c98f676e5bc5559f8f7ecb17
Reviewed-on: https://go-review.googlesource.com/33769 Reviewed-by: Chris Broadfoot <cbro@golang.org>
Brad Fitzpatrick [Thu, 1 Dec 2016 00:48:51 +0000 (00:48 +0000)]
encoding/binary: document the new bool support
Updates #16856
Change-Id: I57af6b0c0d5ecdaf19cf6f969b05ec9ec03058f1
Reviewed-on: https://go-review.googlesource.com/33756 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Austin Clements [Wed, 30 Nov 2016 15:50:40 +0000 (10:50 -0500)]
runtime: fix undead arguments in cgocall
From the garbage collector's perspective, time can move backwards in
cgocall. However, in the midst of this time warp, the pointer
arguments to cgocall can go from dead back to live. If a stack growth
happens while they're dead and then a GC happens when they become live
again, GC can crash with a bad heap pointer.
Specifically, the sequence that leads to a panic is:
1. cgocall calls entersyscall, which saves the PC and SP of its call
site in cgocall. Call this PC/SP "X". At "X" both pointer arguments
are live.
2. cgocall calls asmcgocall. Call the PC/SP of this call "Y". At "Y"
neither pointer argument is live.
3. asmcgocall calls the C code, which eventually calls back into the
Go code.
4. cgocallbackg remembers the saved PC/SP "X" in some local variables,
calls exitsyscall, and then calls cgocallbackg1.
5. The Go code causes a stack growth. This stack unwind sees PC/SP "Y"
in the cgocall frame. Since the arguments are dead at "Y", they are
not adjusted.
6. The Go code returns to cgocallbackg1, which calls reentersyscall
with the recorded saved PC/SP "X", so "X" gets stashed back into
gp.syscallpc/sp.
7. GC scans the stack. It sees there's a saved syscall PC/SP, so it
starts the traceback at PC/SP "X". At "X" the arguments are considered
live, so it scans them, but since they weren't adjusted, the pointers
are bad, so it panics.
This issue started as of commit ca4089ad, when the compiler stopped
marking arguments as live for the whole function.
Since this is a variable liveness issue, fix it by adding KeepAlive
calls that keep the arguments live across this whole time warp.
The existing issue7978 test has all of the infrastructure for testing
this except that it's currently up to chance whether a stack growth
happens in the callback (it currently only happens on the
linux-amd64-noopt builder, for example). Update this test to force a
stack growth, which causes it to fail reliably without this fix.
Shenghou Ma [Tue, 29 Nov 2016 01:31:18 +0000 (20:31 -0500)]
net/http: fix test TestServeMuxHandlerRedirects
The code was intended to test that mux handler should redirect at
most once, but the added loop condition defeated that. Remove the
loop condition and document the intention better.
Ian Lance Taylor [Wed, 30 Nov 2016 01:58:46 +0000 (17:58 -0800)]
doc: add release notes for os and os/signal packages
TBR=See https://golang.org/cl/33244
Updates #17929
Change-Id: I2e5b24fb0b110d833a8b73bccfbf399cb6e37ea2
Reviewed-on: https://go-review.googlesource.com/33681 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Wed, 30 Nov 2016 01:04:31 +0000 (17:04 -0800)]
doc: remove remaining cmd/go entries from go1.8.txt
None of them need to be called out in the release notes.
Change-Id: I143a1879b25063574e4107c1e89264434d45d1d5
Reviewed-on: https://go-review.googlesource.com/33676 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 29 Nov 2016 22:20:58 +0000 (14:20 -0800)]
doc: add note about gccgo go go1.8.html
TBR=See https://golang.org/cl/33244
Updates #17929
Change-Id: I28559724322007d4259810c209a92ec1cc10f338
Reviewed-on: https://go-review.googlesource.com/33668 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 29 Nov 2016 22:17:35 +0000 (14:17 -0800)]
doc: add notes about cgo to go1.8.html
TBR=See https://golang.org/cl/33244
Updates #17929
Change-Id: I0215a7873977be81f2f84374f0b628abaf0e57c1
Reviewed-on: https://go-review.googlesource.com/33667 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Tue, 29 Nov 2016 18:18:06 +0000 (10:18 -0800)]
cmd/compile/internal/syntax: remove unused node field
The doc field is not yet used - remove it for now (we may end up
with a different solution for 1.9). This reduces memory consumption
for parsing all of std lib by about 40MB and makes parsing slightly
faster.
Daniel Theophanes [Fri, 28 Oct 2016 17:10:46 +0000 (10:10 -0700)]
database/sql: do not bypass the driver locks with Context methods
When context methods were initially added it was attempted to unify
behavior between drivers without Context methods and those with
Context methods to always return right away when the Context expired.
However in doing so the driver call could be executed outside of the
scope of the driver connection lock and thus bypassing thread safety.
The new behavior waits until the driver operation is complete. It then
checks to see if the context has expired and if so returns that error.
Dhananjay Nakrani [Sat, 26 Nov 2016 19:21:01 +0000 (11:21 -0800)]
cmd/go: report position info in package errors
Also refactor common position filling code into a function.
Fixes #18011
Change-Id: I76528626da67a7309193fa92af1e361c8e2fcf84
Reviewed-on: https://go-review.googlesource.com/33631
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Austin Clements [Mon, 28 Nov 2016 23:03:16 +0000 (18:03 -0500)]
runtime: fall back to /proc/self/auxv in Android libs
Android's libc doesn't provide access to auxv, so currently the Go
runtime synthesizes a fake, minimal auxv when loaded as a library on
Android. This used to be sufficient, but now we depend on auxv to
retrieve the system physical page size and panic if we can't retrieve
it.
Fix this by falling back to reading auxv from /proc/self/auxv if the
loader-provided auxv is empty and removing the synthetic auxv vectors.
Ian Lance Taylor [Tue, 29 Nov 2016 00:19:03 +0000 (16:19 -0800)]
cmd/link: handle STT_COMMON symbols
Tested by running
GOTRACEBACK=2 CGO_CFLAGS="-Wa,--elf-stt-common=yes" go test -ldflags=-linkmode=internal
in misc/cgo/test. That failed before this CL, succeeded after.
I don't think it's worth doing that as a regular test, though,
especially since only recent versions of the GNU binutils support the
--elf-stt-common option.
Fixes #18088.
Change-Id: I893d86181faee217b1504c054b0ed3f7c8d977d3
Reviewed-on: https://go-review.googlesource.com/33653
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Russ Cox [Tue, 22 Nov 2016 18:31:16 +0000 (13:31 -0500)]
os: fix handling of Windows Unicode console input and ^Z
Go 1.5 worked with Unicode console input but not ^Z.
Go 1.6 did not work with Unicode console input but did handle one ^Z case.
Go 1.7 did not work with Unicode console input but did handle one ^Z case.
The intent of this CL is for Go 1.8 to work with Unicode console input
and also handle all ^Z cases.
Here's a simple test program for reading from the console.
It prints a "> " prompt, calls read, prints what it gets, and repeats.
package main
import (
"fmt"
"os"
)
func main() {
p := make([]byte, 100)
fmt.Printf("> ")
for {
n, err := os.Stdin.Read(p)
fmt.Printf("[%d %q %v]\n> ", n, p[:n], err)
}
}
On Unix, typing a ^D produces a break in the input stream.
If the ^D is at the beginning of a line, then the 0 bytes returned
appear as an io.EOF:
On Windows, the EOF character is ^Z, not ^D, and there has
been a long-standing problem that in Go programs, ^Z on Windows
does not behave in the expected way, namely like ^D on Unix.
Instead, the ^Z come through as literal ^Z characters:
CL 4310 attempted to fix this bug, then known as #6303,
by changing the use of ReadConsole to ReadFile.
This CL was released as part of Go 1.6 and did fix the case
of a ^Z by itself, but not as part of a larger input:
So the fix was incomplete.
Worse, the fix broke Unicode console input.
ReadFile does not handle Unicode console input correctly.
To handle Unicode correctly, programs must use ReadConsole.
Early versions of Go used ReadFile to read the console,
leading to incorrect Unicode handling, which was filed as #4760
and fixed in CL 7312053, which switched to ReadConsole
and was released as part of Go 1.1 and still worked as of Go 1.5:
That is, changing back to ReadFile in Go 1.6 reintroduced #4760,
which has been refiled as #17097. (We have no automated test
for this because we don't know how to simulate console input
in a test: it appears that one must actually type at a keyboard
to use the real APIs. This CL at least adds a comment warning
not to reintroduce ReadFile again.)
CL 29493 attempted to fix #17097, but it was not a complete fix:
the hello world™ example above still fails, as does Shift-JIS input,
which was filed as #17939.
CL 29493 also broke ^Z handling, which was filed as #17427.
This CL attempts the never before successfully performed trick
of simultaneously fixing Unicode console input and ^Z handling.
It changes the console input to use ReadConsole again,
as in Go 1.5, which seemed to work for all known Unicode input.
Then it adds explicit handling of ^Z in the input stream.
(In the case where standard input is a redirected file, ^Z processing
should not happen, and it does not, because this code path is only
invoked when standard input is the console.)
The difference is in the handling of hello^Dworld / hello^Zworld.
On Unix, hello^Dworld terminates the read of hello but does not
result in a zero-length read between reading hello and world.
This is dictated by the tty driver, not any special Go code.
On Windows, in this CL, hello^Zworld inserts a zero length read
result between hello and world, which is treated as an interior EOF.
This is implemented by the Go code in this CL, but it matches the
handling of ^Z on the console in other programs:
C:\>copy con x.txt
hello^Zworld
1 file(s) copied.
C:\>type x.txt
hello
C:\>
A natural question is how to test all this. As noted above, we don't
know how to write automated tests using the actual Windows console.
CL 29493 introduced the idea of substituting a different syscall.ReadFile
implementation for testing; this CL continues that idea but substituting
for syscall.ReadConsole instead. To avoid the regression of putting
ReadFile back, this CL adds a comment warning against that.
Fixes #17427.
Fixes #17939.
Change-Id: Ibaabd0ceb2d7af501d44ac66d53f64aba3944142
Reviewed-on: https://go-review.googlesource.com/33451
Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Quentin Smith <quentin@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Brad Fitzpatrick [Mon, 28 Nov 2016 21:00:29 +0000 (21:00 +0000)]
net/http, net/http/httptest: cross-reference the two NewRequest funcs
Updates #18082
Change-Id: I2e65b115b809c1e1bf813f538989d1a1f96b2876
Reviewed-on: https://go-review.googlesource.com/33636 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Mon, 28 Nov 2016 20:18:29 +0000 (12:18 -0800)]
internal/pprof: don't discard allocations called by reflect.Call
The pprof code discards all heap allocations made by runtime
routines. This caused it to discard heap allocations made by functions
called by reflect.Call, as the calls are made via the functions
`runtime.call32`, `runtime.call64`, etc. Fix the profiler to retain
these heap allocations.
Fixes #18077.
Change-Id: I8962d552f1d0b70fc7e6f7b2dbae8d5bdefb0735
Reviewed-on: https://go-review.googlesource.com/33635
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Michael Munday [Mon, 28 Nov 2016 16:41:48 +0000 (11:41 -0500)]
cmd/asm: fix parsing of the s390x instructions VSTE{G,F,H,B}
The element index needs to be placed in From3. Before this CL it
was impossible to write a VSTE instruction that could be
successfully parsed, so this won't affect existing assembly code.
Fixes #18075.
Change-Id: I5b71be4c6632b1d5a30820a529122f96fd1bc864
Reviewed-on: https://go-review.googlesource.com/33584
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bill O'Farrell <billotosyr@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Martí [Wed, 16 Nov 2016 12:26:23 +0000 (12:26 +0000)]
testing: comment out flag.Parse from example
The TestMain docs explain that flag.Parse() should be called if TestMain
itself depends on command-line flags.
The issue here is that the example implementation does not use any
flags, and thus the flag.Parse call is unnecessary. This leads to people
who use this example as a starting point for their own implementations
to forget that the call is not necessary in most cases.
Comment it out instead of removing the line to keep it as a reminder, as
suggested by Minux Ma.
Change-Id: I6ffc5413e7036366ae3cf0f069b7065e832a3b45
Reviewed-on: https://go-review.googlesource.com/33273 Reviewed-by: Minux Ma <minux@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Munday [Wed, 23 Nov 2016 19:54:12 +0000 (14:54 -0500)]
runtime/cgo: save correct floating point registers on s390x
When transitioning from C code to Go code we must respect the C
calling convention. On s390x this means that r6-r13, r15 and f8-f15
must be saved and restored by functions that use them.
On s390x we were saving the wrong set of floating point registers
(f0, f2, f4 and f6) rather than f8-f15 which means that Go code
could clobber registers that C code expects to be restored. This
CL modifies the crosscall functions on s390x to save/restore the
correct floating point registers.
Fixes #18035.
Change-Id: I5cc6f552c893a4e677669c8891521bf735492e97
Reviewed-on: https://go-review.googlesource.com/33571 Reviewed-by: Ian Lance Taylor <iant@golang.org>
David du Colombier [Tue, 22 Nov 2016 15:19:25 +0000 (16:19 +0100)]
cmd/go: print CC environment variables on Plan 9
This changes makes the output of `go env` the same
as on other operating systems.
Fixes #18013.
Change-Id: I3079e14dcf7b30c75ec3fde6c78cb95721111320
Reviewed-on: https://go-review.googlesource.com/33396 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Munday [Tue, 22 Nov 2016 20:39:51 +0000 (15:39 -0500)]
runtime/pprof/internal/protopprof: fix test on s390x
Applies the fix from CL 32920 to the new test TestSampledHeapAllocProfile
introduced in CL 33422. The test should be skipped rather than fail if
there is only one executable region of memory.
Updates #17852.
Change-Id: Id8c47b1f17ead14f02a58a024c9a04ebb8ec0429
Reviewed-on: https://go-review.googlesource.com/33453
Run-TryBot: Michael Munday <munday@ca.ibm.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox [Sun, 13 Nov 2016 04:01:37 +0000 (23:01 -0500)]
runtime: do not print runtime panic frame at top of user stack
The expected default behavior (no explicit GOTRACEBACK setting)
is for the stack trace to start in user code, eliding unnecessary runtime
frames that led up to the actual trace printing code. The idea was that
the first line number printed was the one that crashed.
For #5832 we added code to show 'panic' frames so that if code panics
and then starts running defers and then we trace from there, the panic
frame can help explain why the code seems to have made a call not
present in the code. But that's only needed for panics between two different
call frames, not the panic at the very top of the stack trace.
Fix the fix to again elide the runtime code at the very top of the stack trace.
The middle panic is important: it explains why main.main ended up calling main.main.func1 on a line that looks like a call to println. The top panic is noise.
After this CL:
panic: runtime error: index out of range
panic: runtime error: index out of range
Ian Lance Taylor [Wed, 16 Nov 2016 05:15:40 +0000 (21:15 -0800)]
runtime: sleep a bit to let a bad signal be delivered
When we raise a signal that was delivered to C code, it's possible that
the kernel will not deliver it immediately. This is especially possible
on Darwin where we use send the signal to the entire process rather than
just the current thread. Sleep for a millisecond after sending the
signal to give it a chance to be delivered before we restore the Go
signal handler. In most real cases the program is going to crash at this
point, so sleeping is kind of irrelevant anyhow.
Fixes #14809.
Change-Id: Ib2c0d2c4e240977fb4535dc1dd2bdc50d430eb85
Reviewed-on: https://go-review.googlesource.com/33300
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Ian Lance Taylor [Tue, 15 Nov 2016 22:48:54 +0000 (14:48 -0800)]
cmd/go: don't clobber `go env GOGCCFLAGS`
When CC is set in the environment, the mkEnv function sets its version
of CC to the first word $CC and sets GOGCCFLAGS to the remainder. That
worked since Go 1 but was broken accidentally by
https://golang.org/cl/6409, which changed the code such that `go env`
calls mkEnv twice. The second call to mkEnv would clobber GOGCCFLAGS
based on the value of CC set by the first call. Go back to the old
handling by only calling mkEnv once.
David Crawshaw [Mon, 21 Nov 2016 21:58:55 +0000 (16:58 -0500)]
cmd/compile, cmd/link: weak relocation for ptrTo
Introduce R_WEAKADDROFF, a "weak" variation of the R_ADDROFF relocation
that will only reference the type described if it is in some other way
reachable.
Use this for the ptrToThis field in reflect type information where it
is safe to do so (that is, types that don't need to be included for
interface satisfaction, and types that won't cause the compiler to
recursively generate an endless series of ptr-to-ptr-to-ptr-to...
types).
Also fix a small bug in reflect, where StructOf was not clearing the
ptrToThis field of new types.
Fixes #17931
Change-Id: I4d3b53cb9c916c97b3b16e367794eee142247281
Reviewed-on: https://go-review.googlesource.com/33427
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Russ Cox [Mon, 10 Oct 2016 20:52:57 +0000 (16:52 -0400)]
math/big: add Baillie-PSW test to (*Int).ProbablyPrime
After x.ProbablyPrime(n) passes the n Miller-Rabin rounds,
add a Baillie-PSW test before declaring x probably prime.
Although the provable error bounds are unchanged, the empirical
error bounds drop dramatically: there are no known inputs
for which Baillie-PSW gives the wrong answer. For example,
before this CL, big.NewInt(443*1327).ProbablyPrime(1) == true.
Now it is (correctly) false.
The new Baillie-PSW test is two pieces: an added Miller-Rabin
round with base 2, and a so-called extra strong Lucas test.
(See the references listed in prime.go for more details.)
The Lucas test takes about 3.5x as long as the Miller-Rabin round,
which is close to theoretical expectations.
However, because the Baillie-PSW test is only added when the old
ProbablyPrime(n) would return true, testing composites runs at
the same speed as before, except in the case where the result
would have been incorrect and is now correct.
In particular, the most important use of this code is for
generating random primes in crypto/rand. That use spends
essentially all its time testing composites, so it is not
slowed down by the new Baillie-PSW check:
name old time/op new time/op delta
Prime 104ms ±22% 111ms ±16% ~ (p=0.165 n=10+10)
Thanks to Serhat Şevki Dinçer for CL 20170, which this CL builds on.
Cherry Zhang [Mon, 21 Nov 2016 23:23:12 +0000 (18:23 -0500)]
runtime/internal/atomic: crash on unaligned 64-bit ops on 32-bit MIPS
This check was originally implemented by Vladimir in
https://go-review.googlesource.com/c/31489/1/src/runtime/internal/atomic/atomic_mipsx.go#30
but removed due to my comment (Sorry!). This CL adds it back.