Alex Brainman [Mon, 6 Jul 2015 00:34:01 +0000 (10:34 +1000)]
runtime: use AddVectoredContinueHandler on Windows XP amd64
Recent change (CL 10370) unexpectedly broke TestRaiseException on
Windows XP amd64. I still do not know why. But reverting old
CL 8165 fixes the problem.
This effectively makes Windows XP amd64 use AddVectoredContinueHandler
instead of SetUnhandledExceptionFilter for exception handling. That is
what we do for all recent Windows versions too.
Fixes #11481
Change-Id: If2e8037711f05bf97e3c69f5a8d86af67c58f6fc
Reviewed-on: https://go-review.googlesource.com/11888
Run-TryBot: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Daniel Theophanes <kardianos@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Charlie Dorian [Wed, 1 Jul 2015 00:14:30 +0000 (20:14 -0400)]
math: Expm1 returns -1 with large negative argument.
Fixes #11442
Change-Id: I2053fe752c6a122924d28565f1338f73e00ed417
Reviewed-on: https://go-review.googlesource.com/11791 Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Instead of silently truncating integers to their expected range, check
that they're within range and emit errors if not. Intended to help
narrow down the cause of issue #11617.
Change-Id: Ia7b577270f8438ca7479262702371e26277f1ea7
Reviewed-on: https://go-review.googlesource.com/12050 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Fri, 10 Jul 2015 12:20:20 +0000 (05:20 -0700)]
runtime, cmd/go: fix tests to work when GOROOT_FINAL is set
When GOROOT_FINAL is set when running all.bash, the tests are run
before the files are copied to GOROOT_FINAL. The tests are run with
GOROOT set, so most work fine. This fixes two cases that do not.
In cmd/go/go_test.go we were explicitly removing GOROOT from the
environment, causing tests that did not themselves explicitly set
GOROOT to fail. There was no need to explicitly remove GOROOT, so
don't do it. If people choose to run "go test cmd/go" with a bad
GOROOT, that is their own lookout.
In the runtime GDB test, the linker has told gdb to find the support
script in GOROOT_FINAL, which will fail. Check for that case, and
skip the test when we see it.
Fixes #11652.
Change-Id: I4d3a32311e3973c30fd8a79551aaeab6789d0451
Reviewed-on: https://go-review.googlesource.com/12021
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Rob Pike [Fri, 10 Jul 2015 01:52:39 +0000 (11:52 +1000)]
cmd/doc: submit to the punched card tyranny
People use 80-column terminals because their grandparents used
punched cards. When I last used a punched card, in 1978, it seemed
antiquated even then. But today, people still set their terminal
widths to 80 to honor the struggles their fallen ancestors made to
endure this painful technology.
We must all stand and salute the 80 column flag, or risk the opprobium
of our peers.
For Pete's sake, I don't even use a fixed-width font. I don't even
believe in columns.
Fixes #11639 with extreme reluctance.
P.S. To avoid the horror of an automatically folded line of text, this commit message has been formatted to fit on an 80-column line, except for this postscript.
Change-Id: Ia2eb2dcf293dabe804c22ee5abb4bbb703f45c33
Reviewed-on: https://go-review.googlesource.com/12011 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Mon, 6 Jul 2015 05:52:33 +0000 (15:52 +1000)]
debug/pe: truncate pe sections to their size in memory
Section.Data returns disk section data, but those are rounded up to
some predefined value. Processing these as is confuses dwarf parser
because of garbage at the end. Truncate Section.Data as per memory
section description.
Sometimes dwarf sections have memory section size of 0
(for pe object files). Keep those to their disk size.
Fixes #11608
Change-Id: I8de0a2271201a24aa9ac8dac44f1e9c8a9285183
Reviewed-on: https://go-review.googlesource.com/11950 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Brad Fitzpatrick [Thu, 25 Jun 2015 10:52:54 +0000 (12:52 +0200)]
net: add mechanisms to force go or cgo lookup, and to debug default strategy
GODEBUG=netdns=1 prints a one-time strategy decision. (cgo or go DNS lookups)
GODEBUG=netdns=2 prints the per-lookup strategy as a function of the hostname.
The new "netcgo" build tag forces cgo DNS lookups.
GODEBUG=netdns=go (or existing build tag "netgo") forces Go DNS resolution.
GODEBUG=netdns=cgo (or new build tag "netcgo") forces libc DNS resolution.
Options can be combined with e.g. GODEBUG=netdns=go+1 or GODEBUG=netdns=2+cgo.
Ross Light [Thu, 9 Jul 2015 17:13:09 +0000 (10:13 -0700)]
CONTRIBUTORS: add Ross Light's google.com email
Change-Id: Ide5b0c81405fbf6b83a6f02527d629898e0fcf02
Reviewed-on: https://go-review.googlesource.com/12000 Reviewed-by: Ian Lance Taylor <iant@golang.org>
sysmon triggers a GC if there has been no GC for two minutes.
Currently, this is a STW GC. There is no reason for this to be STW, so
make it concurrent.
Rob Pike [Wed, 8 Jul 2015 01:17:01 +0000 (11:17 +1000)]
cmd/doc: suppress symbols for commands when showing package docs
Change the default behavior when showing the package docs
for a command to elide the symbols. This makes
go doc somecommand
show the top-level package docs only and hide the symbols,
which are probably irrelevant to the user. This has no effect
on explicit requests for internals, such as
net/http: revert overly-strict part of earlier smuggling defense
The recent https://golang.org/cl/11810 is reportedly a bit too
aggressive.
Apparently some HTTP requests in the wild do contain both a
Transfer-Encoding along with a bogus Content-Length. Instead of
returning a 400 Bad Request error, we should just ignore the
Content-Length like we did before.
David Chase [Mon, 29 Jun 2015 20:30:19 +0000 (16:30 -0400)]
cmd/compile: initialize line number properly for temporaries
The expansion of structure, array, slice, and map literals
does not use the right line number in its introduced assignments
to temporaries, which leads to incorrect line number attribution
for expressions in those literals.
Inlining also incorrectly replaced the line numbers of args to
inlined functions.
This was revealed in CL 9721 because a now-avoided temporary
assignment introduced the correct line number.
I.e. before CL 9721
"tmp_wrongline := expr"
was transformed to
"tmp_rightline := expr; tmp_wrongline := tmp_rightline"
Also includes a repair to CL 10334 involving line numbers
where a spurious -1 remained (should have been 0, now is 0).
Basic randomization of goroutine scheduling for -race mode.
It is probably possible to do much better (there's a paper linked
in the issue that I haven't read, for example), but this suffices
to introduce at least some unpredictability into the scheduling order.
The goal here is to have _something_ for Go 1.5, so that we don't
start hitting more of these scheduling order-dependent bugs
if we change the scheduler order again in Go 1.6.
net/http/httputil: make ReverseProxy support Trailers
Go's continuous build system depends on HTTP trailers for the buildlet
interface.
Andrew rewrote the makerelease tool to work in terms of Go's builder
system (now at x/build/cmd/release), but it previously could only
create GCE-based buildlets, which meant x/build/cmd/release couldn't
build the release for Darwin.
https://golang.org/cl/11901 added support for proxying buildlet
connections via the coordinator, but that exposed the fact that
httputil.ReverseProxy couldn't proxy Trailers. A fork of that code
also wasn't possible because net/http needlessly deleted the "Trailer"
response header in the Transport code. This mistake goes back to
"release-branch.r56" and earlier but was never noticed because nobody
ever uses Trailers, and servers via ResponseWriter never had the
ability to even set trailers before this Go 1.5. Note that setting
trailers requires pre-declaring (in the response header) which
trailers you'll set later (after the response body). Because you could
never set them, before this release you could also never proxy them.
Rob Pike [Sun, 5 Jul 2015 23:01:32 +0000 (09:01 +1000)]
doc: tools for go1.5.html
Don't know why, but git deleted the previous version of this change.
This is the same change as https://go-review.googlesource.com/11884,
which I will now abandon, with a couple of fixes.
Almost all done now. Could use help with the TODOs.
Major missing piece is the trace command. Vendoring
section is also weak, but it's also undocumented elsewhere.
Russ suggests changing the frozon syscall package and obviously it's a
better solution. Perhaps he will also let me know the way how to get the
project owners to agree later.
On some VMs two events can happen at the same time. For examples: 179827399 GoStart p=2 g=11 off=936359 g=11 179827399 GoUnblock p=2 g=0 off=936355 g=11
If we do non-stable sort, the events can be reordered making the trace inconsistent.
Do stable sort instead.
Batches are dumped in FIFO order, so if these same-time events are split into
separate batches, stable sort still works.
Events on different CPUs go into different batches and can be reordered.
But the intention is that causally-related events on different CPUs
will have larger (non-zero) time diff.
Dmitry Vyukov [Thu, 18 Jun 2015 14:27:29 +0000 (16:27 +0200)]
cmd/trace: fix time scale
Integrate the latest trace-viewer changes.
It now handles nanoseconds without any issues (thanks to @egonelbre!).
So change timestamps from microseconds to nanoseconds.
Alan Donovan [Tue, 30 Jun 2015 19:07:20 +0000 (15:07 -0400)]
go/types: change {Type,Object,Selection}String to accept a Qualifier function
The optional Qualifier function determines what prefix to attach to
package-level names, enabling clients to qualify packages in different
ways, for example, using only the package name instead of its complete
path, or using the locally appropriate name for package given a set of
(possibly renaming) imports.
Prior to this change, clients wanting this behavior had to copy
hundreds of lines of complex printing logic.
Fun fact: (*types.Package).Path and (*types.Package).Name are valid
Qualifier functions.
We provide the RelativeTo helper function to create Qualifiers so that
the old behavior remains a one-liner.
Fixes golang/go#11133
This CL is a copy of https://go-review.googlesource.com/#/c/11692/
to the golang.org/x/tools repository.
Change-Id: I26d0f3644d077a26bfe350989f9c545f018eefbf
Reviewed-on: https://go-review.googlesource.com/11790 Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
Russ Cox [Mon, 29 Jun 2015 17:50:30 +0000 (13:50 -0400)]
cmd/compile: allow linker to drop string headers when not needed
Compiling a simple file containing a slice of 100,000 strings,
the size of the resulting binary dropped from 5,896,224 bytes
to 3,495,968 bytes, which is the expected 2,400,000 bytes,
give or take.
Brad Fitzpatrick [Tue, 30 Jun 2015 16:22:41 +0000 (09:22 -0700)]
net/textproto: don't treat spaces as hyphens in header keys
This was originally done in https://codereview.appspot.com/5690059
(Feb 2012) to deal with bad response headers coming back from webcams,
but it presents a potential security problem with HTTP request
smuggling for request headers containing "Content Length" instead of
"Content-Length".
Part of overall HTTP hardening for request smuggling. See RFC 7230.
Dmitry Vyukov [Tue, 30 Jun 2015 12:09:41 +0000 (14:09 +0200)]
cmd/trace: sort procs
If you have more than 10 procs, then currently they are sorted alphabetically as
0, 10, 11, ..., 19, 2, 20, ...
Assign explicit order to procs so that they are sorted numerically.
Russ Cox [Tue, 30 Jun 2015 15:28:29 +0000 (11:28 -0400)]
net/url: only record RawPath when it is needed
RawPath is a hint to the desired encoding of Path.
It is ignored when it is not a valid encoding of Path,
such as when Path has been changed but RawPath has not.
It is not ignored but also not useful when it matches
the url package's natural choice of encoding.
In this latter case, set it to the empty string.
This should help drive home the point that clients
cannot in general depend on it being present and
that they should use the EncodedPath method instead.
This also reduces the impact of the change on tests,
especially tests that use reflect.DeepEqual on parsed URLs.
Roger Peppe [Mon, 29 Jun 2015 11:36:48 +0000 (12:36 +0100)]
encoding/xml: fix xmlns= behavior
When an xmlns="..." attribute was explicitly generated,
it was being ignored because the name space on the
attribute was assumed to have been explicitly set (to the empty
name space) and it's not possible to have an element in the
empty name space when there is a non-empty name space set.
We fix this by recording when a default name space has been
explicitly set and setting the name space of the element to that
so printer.defineNS can do its work correctly.
We do not attempt to add our own xmlns="..." attribute
when one is explicitly set.
We also add tests for EncodeElement, as that's the only way
to attain coverage of some of the changed behaviour.
Some other test coverage is also increased, although
more work remains to be done in this area.
This change was jointly developed with Martin Hilton (mhilton on github).
Aaron Jacobs [Mon, 29 Jun 2015 00:07:31 +0000 (10:07 +1000)]
net/http: add a Request.Cancel channel.
This allows for "race free" cancellation, in the sense discussed in
issue #11013: in contrast to Transport.CancelRequest, the cancellation
will not be lost if the user cancels before the request is put into the
transport's internal map.
Russ Cox [Mon, 29 Jun 2015 15:53:51 +0000 (11:53 -0400)]
path/filepath: document and test behavior of SkipDir on files
This behavior is not what we might have designed from the start,
but it has been present since Go 1. Rather than make a visible
behavioral change that might cause programs to work differently
in Go ≤1.4 vs Go ≥1.5, document what SkipDir on a non-directory
has always meant. If code doesn't want this meaning, it is easy
enough not to return SkipDir on non-directories.
Russ Cox [Mon, 29 Jun 2015 17:12:10 +0000 (13:12 -0400)]
cmd/link: fix -s with external linking
This code used to only be run for ELF, with the predictable
result that using -s with external linking broke on Windows and OS X.
Moving it here should fix Windows and does fix OS X.
CL 10835 also claims to fix the crash on Windows.
I don't know whether it does so correctly, but regardless,
this CL should make that one a no-op.
Fixes #10254.
Change-Id: I2e7b45ab0c28568ddbb1b50581dcc157ae0e7ffe
Reviewed-on: https://go-review.googlesource.com/11695 Reviewed-by: David Crawshaw <crawshaw@golang.org>
Adam Langley [Sat, 27 Jun 2015 21:50:39 +0000 (14:50 -0700)]
crypto/rsa: check for primes ≤ 1 in Validate
Change 7c7126cfeb82894229b9c3d5109e4b04e6cfde0c removed the primality
checking in Validate to save CPU time. That check happened to be
filtering out private keys with primes that were zero or one. Without
that filtering, such primes cause a panic when trying to use such a
private key.
This change specifically checks for and rejects primes ≤ 1 in Validate.
Russ Cox [Mon, 29 Jun 2015 02:26:35 +0000 (22:26 -0400)]
cmd/link: record missing pcdata tables correctly
The old code was recording the current table output offset,
so the table from the next function would be used instead of
the runtime realizing that there was no table at all.
Add debug constant in runtime to check this for every function
at startup. It's too expensive to do that by default, but we can
do the last five functions. The end of the table is usually where
the C symbols end up, so that's where the problems typically are.
Austin Clements [Fri, 26 Jun 2015 17:56:58 +0000 (13:56 -0400)]
runtime: reset mark state before checkmark and gctrace=2 mark
Currently we fail to reset the live heap accounting state before the
checkmark mark and before the gctrace=2 extra mark. As a result, if
either are enabled, at the end of GC it thinks there are 0 bytes of
live heap, which causes the GC controller to initiate a new GC
immediately, regardless of the true heap size.
Fix this by factoring this state reset into a function and calling it
before all three possible marks.
This function should be merged with gcResetGState, but doing so
requires some additional cleanup, so it will wait for after the
freeze. Filed #11427 for this cleanup.
Austin Clements [Mon, 22 Jun 2015 14:24:50 +0000 (10:24 -0400)]
runtime: don't free stack spans during GC
Memory for stacks is manually managed by the runtime and, currently
(with one exception) we free stack spans immediately when the last
stack on a span is freed. However, the garbage collector assumes that
spans can never transition from non-free to free during scan or mark.
This disagreement makes it possible for the garbage collector to mark
uninitialized objects and is blocking us from re-enabling the bad
pointer test in the garbage collector (issue #9880).
For example, the following sequence will result in marking an
uninitialized object:
1. scanobject loads a pointer slot out of the object it's scanning.
This happens to be one of the special pointers from the heap into a
stack. Call the pointer p and suppose it points into X's stack.
2. X, running on another thread, grows its stack and frees its old
stack.
3. The old stack happens to be large or was the last stack in its
span, so X frees this span, setting it to state _MSpanFree.
4. The span gets reused as a heap span.
5. scanobject calls heapBitsForObject, which loads the span containing
p, which is now in state _MSpanInUse, but doesn't necessarily have
an object at p. The not-object at p gets marked, and at this point
all sorts of things can go wrong.
We already have a partial solution to this. When shrinking a stack, we
put the old stack on a queue to be freed at the end of garbage
collection. This was done to address exactly this problem, but wasn't
a complete solution.
This commit generalizes this solution to both shrinking and growing
stacks. For stacks that fit in the stack pool, we simply don't free
the span, even if its reference count reaches zero. It's fine to reuse
the span for other stacks, and this enables that. At the end of GC, we
sweep for cached stack spans with a zero reference count and free
them. For larger stacks, we simply queue the stack span to be freed at
the end of GC. Ideally, we would reuse these large stack spans the way
we can small stack spans, but that's a more invasive change that will
have to wait until after the freeze.
Austin Clements [Thu, 25 Jun 2015 16:24:44 +0000 (12:24 -0400)]
runtime: remove unused _GCsweep state
We don't use this state. _GCoff means we're sweeping in the
background. This makes it clear in the next commit that _GCoff and
only _GCoff means sweeping.
Russ Cox [Mon, 29 Jun 2015 03:04:05 +0000 (23:04 -0400)]
cmd/cgo: fix a problem with 'go build -compiler gccgo'
Port of https://golang.org/cl/154360045 to Git.
Original author is Xia Bin <snyh@snyh.org> (already a contributor).
Fixes #8945.
Change-Id: I28bcaf3348794202ca59fbc3466bd7b9670030e4
Reviewed-on: https://go-review.googlesource.com/11658 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Austin Clements [Wed, 24 Jun 2015 21:13:24 +0000 (17:13 -0400)]
runtime: always clear stack barriers on G exit
Currently the runtime fails to clear a G's stack barriers in gfput if
the G's stack allocation is _FixedStack bytes. This causes the runtime
to panic if the following sequence of events happens:
1) The runtime installs stack barriers on a G.
2) The G exits by calling runtime.Goexit. Since this does not
necessarily return through the stack barriers installed on the G,
there may still be untriggered stack barriers left on the G's stack
in recorded in g.stkbar.
3) The runtime calls gfput to add the exiting G to the free pool. If
the G's stack allocation is _FixedStack bytes, we fail to clear
g.stkbar.
4) A new G starts and allocates the G that was just added to the free
pool.
5) The new G begins to execute and overwrites the stack slots that had
stack barriers in them.
6) The garbage collector enters mark termination, attempts to remove
stack barriers from the new G, and finds that they've been
overwritten.
Fix this by clearing the stack barriers in gfput in the case where it
reuses the stack.
Rob Pike [Sat, 27 Jun 2015 01:07:30 +0000 (11:07 +1000)]
fmt: restore padding for %x on byte slices and strings
Also improve the documentation. A prior fix in this release
changed the properties for empty strings and slices, incorrectly.
Previous behavior is now restored and better documented.
Add lots of tests.
The behavior is that when using a string-like format (%s %q %x %X)
a byte slice is equivalent to a string, and printed as a unit. The padding
applies to the entire object. (The space and sharp flags apply
elementwise.)
Brad Fitzpatrick [Sat, 27 Jun 2015 03:51:13 +0000 (20:51 -0700)]
net/http: fix now-flaky TransportAndServerSharedBodyRace test
TestTransportAndServerSharedBodyRace got flaky after
issue #9662 was fixed by https://golang.org/cl/11412, which made
servers hang up on clients when a Handler stopped reading its body
early.
This test was affected by a race between the the two goroutines in the
test both only reading part of the request, which was an unnecessary
detail for what the test was trying to test (concurrent Read/Close
races on an *http.body)
Also remove an unused remnant from an old test from which this one was
derived. And make the test not deadlock when it fails. (which was why
the test was showing up as 2m timeouts on the dashboard)
Keith Randall [Sat, 27 Jun 2015 19:57:06 +0000 (12:57 -0700)]
cmd/link/internal/ld: exclude only real container symbols from symtab
It looks like the test for whether symbols contain subsymbols is wrong.
In particular, symbols in C libraries are mistakenly considered container
symbols.
Fix the test so only symbols which actually have a subsymbol
are excluded from the symtab. When linking cgo programs the list
of containers is small, something like:
Adam Langley [Sun, 14 Jun 2015 17:48:13 +0000 (10:48 -0700)]
encoding/asn1: be stricter by reserialising parsed times.
The time package does normalisation of times: for example day zero is
converted to the last day of the previous month and the 31st of February
is moved into March etc. This makes the ASN.1 parsing a little
worryingly lax.
This change causes the parser to reserialise parsed times to ensure that
they round-trip correctly and thus were not normalised.
Alex Brainman [Fri, 22 May 2015 00:58:57 +0000 (10:58 +1000)]
runtime: store syscall parameters in m not on stack
Stack can move during callback, so libcall struct cannot be stored on stack.
asmstdcall updates return values and errno in libcall struct parameter, but
these could be at different location when callback returns.
Store these in m, so they are not affected by GC.
Todd Neal [Thu, 18 Jun 2015 12:26:39 +0000 (07:26 -0500)]
net/http: fix race on postPendingDial test hook
The race occurs rarely, but by putting some delays and more reads/writes
of prePendingDial/postPendingDial in the handlePendingDial function I
could reproduce it.
Didier Spezia [Mon, 8 Jun 2015 20:04:14 +0000 (20:04 +0000)]
html/template: prevent panic while escaping pipelines
AFAIK, the documentation does not explicitly state whether
variables can store a callable entity or not. I believe the
current implementation in text/template assumes they cannot
though. The call builtin function is supposed to be used for
this purpose.
Template "{{0|$}}" should generate an error at runtime,
instead of a panic.
Similarly, template "{{0|(nil)}}" should not generate
a panic.
This CL aborts the sanitization process for a given pipeline
when no identifier can be derived from the selected node.
It happens with malformed pipelines.
We now have the following errors:
{{ 0 | $ }}
template: foo:1:10: executing "foo" at <$>: can't give argument to non-function $
{{ 0 | (nil) }}
template: foo:1:11: executing "foo" at <nil>: nil is not a command
Fixes #11118
Fixes #11356
Change-Id: Idae52f806849f4c9ab7aca1b4bb4b59a74723d0e
Reviewed-on: https://go-review.googlesource.com/10823 Reviewed-by: Rob Pike <r@golang.org>