Dmitry Vyukov [Mon, 22 Dec 2014 15:14:00 +0000 (18:14 +0300)]
runtime: simplify procresize
Currently we do very a complex rebalancing of runnable goroutines
between queues, which tries to preserve scheduling fairness.
Besides being complex and error-prone, it also destroys all locality
of scheduling.
This change uses simpler scheme: leave runnable goroutines where
they are, during starttheworld start all Ps with local work,
plus start one additional P in case we have excessive runnable
goroutines in local queues or in the global queue.
The schedler must be able to operate efficiently w/o the rebalancing,
because garbage collections do not have to happen frequently.
The immediate need is execution tracing support: handling of
garabage collection which does stoptheworld/starttheworld several
times becomes exceedingly complex if the current execution can
jump between Ps during starttheworld.
Change-Id: I4fdb7a6d80ca4bd08900d0c6a0a252a95b1a2c90
Reviewed-on: https://go-review.googlesource.com/1951 Reviewed-by: Rick Hudson <rlh@golang.org>
Keith Randall [Tue, 23 Dec 2014 21:45:58 +0000 (13:45 -0800)]
reflect: fix func layout test for nacl build
This test code is ugly. There must be a better way.
But for now, fix the build.
Change-Id: I33064145ea37f11abf040ec97caa87669be1a9fa
Reviewed-on: https://go-review.googlesource.com/2114 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Carlos Castillo [Mon, 8 Dec 2014 15:28:40 +0000 (07:28 -0800)]
go/build: add variable expansion to cgo lines
When go parses #cgo lines, expand ${SRCDIR} into the path to the
source directory. This allows options to be passed to the
compiler and linker that involve file paths relative to the
source code directory. Without the expansion the paths would be
invalid when the current working directory changes.
Fixes #7891
Fixes #5428
Change-Id: I343a145a9771a5ccbaa958e4a1ecd1716fcae52d
Reviewed-on: https://go-review.googlesource.com/1756 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Tue, 23 Dec 2014 18:57:37 +0000 (10:57 -0800)]
reflect: more function layout tests
Test more stuff:
1) flagNoPointers, an incorrect value was the cause of #9425
2) Total function layout size
3) gc program
Change-Id: I73f65fe740215938fa930d2f096febd9db0a0021
Reviewed-on: https://go-review.googlesource.com/2090 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alexandre Cesaro [Thu, 18 Dec 2014 20:33:34 +0000 (21:33 +0100)]
mime/multipart: moved some code to mime/internal/quotedprintable
The code concerning quoted-printable encoding (RFC 2045) and its
variant for MIME headers (RFC 2047) is currently spread in
mime/multipart and net/mail. It is also not exported.
This commit is the first step to fix that issue. It moves the
quoted-printable decoding code from mime/multipart to
mime/internal/quotedprintable. The exposed API is unchanged.
Shenghou Ma [Tue, 23 Dec 2014 04:44:41 +0000 (23:44 -0500)]
runtime: fix build for arm and ppc64/ppc64le
Change-Id: I17ddcb541dfac8b1e48e01ee005563031b6ade2a
Reviewed-on: https://go-review.googlesource.com/2062 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Andrew Gerrand [Tue, 23 Dec 2014 01:09:38 +0000 (12:09 +1100)]
build: give freebsd longer to run tests
The freebsd-386 and freebsd-amd64 builders are timing out sometimes.
This will give them some more breathing room.
Change-Id: Ib65bd172cca046a52861759a4232d7b4b6514fa8
Reviewed-on: https://go-review.googlesource.com/1994 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Tue, 23 Dec 2014 04:00:04 +0000 (15:00 +1100)]
runtime: import unsafe in os_windows.go (fixes windows build)
Change-Id: I55419cb580e6d18cf1c17c3e7bb8777ed6d794e7
Reviewed-on: https://go-review.googlesource.com/1995 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Tue, 23 Dec 2014 03:07:05 +0000 (19:07 -0800)]
reflect: add kindNoPointers if a function layout has no pointers.
malloc checks kindNoPointers and if it is not set and the object
is one pointer in size, it assumes it contains a pointer. So we
must set kindNoPointers correctly; it isn't just a hint.
Russ Cox [Mon, 22 Dec 2014 18:27:53 +0000 (13:27 -0500)]
runtime: remove thunk.s
Replace with uses of //go:linkname in Go files, direct use of name in .s files.
The only one that really truly needs a jump is reflect.call; the jump is now
next to the runtime.reflectcall assembly implementations.
Austin Clements [Tue, 23 Dec 2014 02:18:09 +0000 (21:18 -0500)]
misc/cgo: fix issue 9400 test on 386
issue9400_linux.go did not build on 386 because it used a constant
that was larger than a 32-bit int in a ... argument. Fix this by
casting the constant to uint64 (to match how the constant is being
used).
Change-Id: Ie8cb64c3910382a41c7852be7734a62f0b2d5a21
Reviewed-on: https://go-review.googlesource.com/2060 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Austin Clements [Fri, 19 Dec 2014 21:16:17 +0000 (16:16 -0500)]
runtime: run libc SIGSETXID and SIGCANCEL handlers on signal stack
These signals are used by glibc to broadcast setuid/setgid to all
threads and to send pthread cancellations. Unlike other signals, the
Go runtime does not intercept these because they must invoke the libc
handlers (see issues #3871 and #6997). However, because 1) these
signals may be issued asynchronously by a thread running C code to
another thread running Go code and 2) glibc does not set SA_ONSTACK
for its handlers, glibc's signal handler may be run on a Go stack.
Signal frames range from 1.5K on amd64 to many kilobytes on ppc64, so
this may overflow the Go stack and corrupt heap (or other stack) data.
Fix this by ensuring that these signal handlers have the SA_ONSTACK
flag (but not otherwise taking over the handler).
This has been a problem since Go 1.1, but it's likely that people
haven't encountered it because it only affects setuid/setgid and
pthread_cancel.
Fixes #9600.
Change-Id: I6cf5f5c2d3aa48998d632f61f1ddc2778dcfd300
Reviewed-on: https://go-review.googlesource.com/1887 Reviewed-by: Ian Lance Taylor <iant@golang.org>
George Shammas [Thu, 18 Dec 2014 04:22:49 +0000 (23:22 -0500)]
net/http/cgi: Correctly pass down the REMOTE_PORT value for CGI requests.
Currently when we get a CGI or FCGI request, the remote port of the client
is hard coded to zero, despite nearly every webserver passing down the
REMOTE_PORT variable.
This was likely originally excluded because the CGI RFC (rfc3875) does not
mention anything about the remote port of the client. However every webserver
tested does pass REMOTE_PORT down. This includes Apache 2.2, Apache 2.4,
nginx and lighttpd.
Keith Randall [Mon, 8 Dec 2014 22:18:58 +0000 (14:18 -0800)]
runtime: make stack frames fixed size by modifying goproc/deferproc.
Calls to goproc/deferproc used to push & pop two extra arguments,
the argument size and the function to call. Now, we allocate space
for those arguments in the outargs section so we don't have to
modify the SP.
Defers now use the stack pointer (instead of the argument pointer)
to identify which frame they are associated with.
A followon CL might simplify funcspdelta and some of the stack
walking code.
Mikio Hara [Mon, 22 Dec 2014 03:30:16 +0000 (12:30 +0900)]
syscall: fix the deprecated way of parsing routing message on openbsd
OpenBSD 5.5 changed its kernel ABI and OpenBSD 5.6 enabled it.
This CL works on both 5.5 and 5.6.
Fixes #9102.
Change-Id: I4a295be9ab8acbc99e550d8cb7e8f8dacf3a03c5
Reviewed-on: https://go-review.googlesource.com/1932 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 16 Dec 2014 22:54:20 +0000 (14:54 -0800)]
test: add test case that gccgo failed to link
Gccgo creates a struct to hold the arguments for the deferred
function. In this example the struct holds a type defined in a
different package. The bug was that gccgo tried to create an equality
function for this struct, and it implemented that function by calling
the equality function for the type defined in the other package.
Since that type is not exported, the reference to the equality
function failed at link time. Normally it is impossible for a struct
to directly contain a member that is an unexported type from another
package, but in this specific case it was possible. Fixed in gccgo
with https://codereview.appspot.com/183500043 .
Austin Clements [Tue, 16 Dec 2014 20:59:05 +0000 (15:59 -0500)]
cmd/5l,6l,8l: remove bogus dynsym section indexes
Previously, this code generated bogus section indexes for dynamic
symbols. It turns out this didn't matter, since we only emit these
when generating an executable and in an executable it only matters
whether a symbol is defined or undefined, but it leads to perplexing
code full of mysterious constants.
Unfortunately, this happens too early to put in real section indexes,
so just use section index 1 to distinguish the symbol from an
undefined symbol.
* bug248, bug345, bug369, and bug429 were ported from bash commands to run scripts. bug369 remains disabled.
* bug395 is a test for issue 1909, which is still open. It is marked as skip now and will be usable with compile with run.go when issue 1909 is fixed.
Keith Randall [Thu, 11 Dec 2014 01:02:58 +0000 (17:02 -0800)]
runtime: a better fallback hash
For arm and powerpc, as well as x86 without aes instructions.
Contains a mixture of ideas from cityhash and xxhash.
Compared to our old fallback on ARM, it's ~no slower on
small objects and up to ~50% faster on large objects. More
importantly, it is a much better hash function and thus has
less chance of bad behavior.
Keith Randall [Sat, 20 Dec 2014 04:44:18 +0000 (20:44 -0800)]
runtime: hashmap: move overflow pointer to end of bucket
Pointers to zero-sized values may end up pointing to the next
object in memory, and possibly off the end of a span. This
can cause memory leaks and/or confuse the garbage collector.
By putting the overflow pointer at the end of the bucket, we
make sure that pointers to any zero-sized keys or values don't
accidentally point to the next object in memory.
Bryan Ford [Sat, 13 Dec 2014 18:54:39 +0000 (13:54 -0500)]
encoding/base64: add unpadded encodings, and test all encodings.
Some applications use unpadded base64 format, omitting the trailing
'=' padding characters from the standard base64 format, either to
minimize size or (more justifiably) to avoid use of the '=' character.
Unpadded flavors are standard and documented in section 3.2 of RFC 4648.
To support these unpadded flavors, this change adds two predefined
encoding variables, RawStdEncoding and RawURLEncoding, for unpadded
encodings using the standard and URL character set, respectively.
The change also adds a function WithPadding() to customize the padding
character or disable padding in a custom Encoding.
Finally, I noticed that the existing base64 test-suite was only
exercising the StdEncoding, and not referencing URLEncoding at all.
This change adds test-suite functionality to exercise all four encodings
(the two existing ones and the two new unpadded flavors),
although it still doesn't run *every* test on all four encodings.
Naming: I used the "Raw" prefix because it's more concise than "Unpadded"
and seemed just as expressive, but I have no strong preferences here.
Another short alternative prefix would be "Min" ("minimal" encoding).
Jed Denlea [Tue, 16 Dec 2014 08:24:19 +0000 (00:24 -0800)]
net/http: Fix Range off-by-one error
Given a file of size N, a request for "Range: bytes=N-*" should
return a 416 [1]. Currently, it returns a 206 and a body of 0
bytes, with the illegal Content-Range of "bytes N-(N-1)/N" [2].
[1]: RFC 7233, sec 2.1: "If a valid byte-range-set includes at least one
byte-range-spec with a first-byte-pos that is less than the current
length of the representation, [...]". sec 3.1: "If all of the
preconditions are true, the server supports the Range header field for
the target resource, and the specified range(s) are invalid or
unsatisfiable, the server SHOULD send a 416 (Range Not Satisfiable)
response."
[2]: RFC 7233, sec 4.2: "A Content-Range field value is invalid if it
contains a byte-range-resp that has a last-byte-pos value less than its
first-byte-pos value, [...]"
mischief [Mon, 15 Dec 2014 20:55:00 +0000 (12:55 -0800)]
runtime: fix brk_ signature for plan9
with uintptr, the check for < 0 will never succeed in mem_plan9.go's
sbrk() because the brk_ syscall returns -1 on failure. fixes the plan9/amd64 build.
this failed on plan9/amd64 because of the attempt to allocate 136GB in mallocinit(),
which failed. it was just by chance that on plan9/386 allocations never failed.
Change-Id: Ia3059cf5eb752e20d9e60c9619e591b80e8fb03c
Reviewed-on: https://go-review.googlesource.com/1590 Reviewed-by: Anthony Martin <ality@pbrane.org> Reviewed-by: David du Colombier <0intro@gmail.com> Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Ian Lance Taylor [Sat, 20 Dec 2014 02:17:02 +0000 (18:17 -0800)]
misc/cgo/test: skip test7978 when using gccgo
Gccgo can only get a backtrace for the currently running thread, which
means that it can only get a backtrace for goroutines currently running
Go code. When a goroutine is running C code, gccgo has no way to stop
it and get the backtrace. This test is all about getting a backtrace
of goroutines running C code, so it can't work for gccgo.
Change-Id: I2dff4403841fb544da7396562ab1193875fc14c3
Reviewed-on: https://go-review.googlesource.com/1904 Reviewed-by: Minux Ma <minux@golang.org>
Ian Lance Taylor [Sat, 20 Dec 2014 01:21:14 +0000 (17:21 -0800)]
cmd/cgo: don't #include cgo_export.h in gccgo _cgo_main.c
Instead of relying on the asm names declared in the gccgo version of
cgo_export.h, just emit a dummy symbol with the right asm name. This
is enough to let the _cgo_main link succeed, which is all that matters
here.
Fixes #9294.
Change-Id: I803990705b6b226ed0adf17dc57b58a9f501b213
Reviewed-on: https://go-review.googlesource.com/1901 Reviewed-by: Minux Ma <minux@golang.org>
Ian Lance Taylor [Sat, 20 Dec 2014 01:31:16 +0000 (17:31 -0800)]
testing: style change: omit unnecessary new
This was brought to my attention because a user thought that because
the file was named "example.go" it served as an example of good coding
practice. It's not an example, of course, but may as well use a more
idiomatic style anyhow.
Change-Id: I7aa720f603f09f7d597fb7536dbf46ef09144e28
Reviewed-on: https://go-review.googlesource.com/1902 Reviewed-by: Minux Ma <minux@golang.org>
Matthew Dempsky [Thu, 18 Dec 2014 21:50:57 +0000 (13:50 -0800)]
runtime: simplify cpuprof hash calculation
"x*41" computes the same value as "x*31 + x*7 + x*3" and (when
compiled by gc) requires just one multiply instruction instead of
three.
Alternatively, the expression could be written as "(x<<2+x)<<3 + x" to
use shifts instead of multiplies (which is how GCC optimizes "x*41").
But gc currently emits suboptimal instructions for this expression
anyway (e.g., separate SHL+ADD instructions rather than LEA on
386/amd64). Also, if such an optimization was worthwhile, it would
seem better to implement it as part of gc's strength reduction logic.
Austin Clements [Thu, 18 Dec 2014 16:40:48 +0000 (11:40 -0500)]
liblink: fail for too-large register offset constants
Previously, liblink would silently truncate register offset constants
to 32 bits. For example,
MOVD $0x200000004(R2),R3
would assemble to
addis r31,r2,0
addi r3,r31,4
To fix this, limit C_LACON to 32 bit (signed) offsets and introduce a
new C_DACON operand type for larger register offsets. We don't
implement this currently, but at least liblink will now give an error
if it encounters an address like this.
Change-Id: I8e87def8cc4cc5b75498b0fb543ac7666cf2964e
Reviewed-on: https://go-review.googlesource.com/1758 Reviewed-by: Minux Ma <minux@golang.org>
Austin Clements [Thu, 18 Dec 2014 21:06:57 +0000 (16:06 -0500)]
cmd/ld: set ELF ABI version for ppc64x
On ppc64, there are three ELF ABI versions an ELF file can request.
Previously, we used 0, which means "unspecified". On our test
machines, this meant to use the default (v1 for big endian and v2 for
little endian), but apparently some systems can pick the wrong ABI if
neither is requested. Leaving this as 0 also confuses libbfd, which
confuses gdb, objdump, etc.
Fix these problems by specifying ABI v1 for big endian and v2 for
little endian.
Change-Id: I4d3d5478f37f11baab3681a07daff3da55802322
Reviewed-on: https://go-review.googlesource.com/1800 Reviewed-by: Minux Ma <minux@golang.org>
Keith Randall [Tue, 16 Dec 2014 22:15:49 +0000 (14:15 -0800)]
cmd/gc: resolve static addresses of the form &x.f at link time
When we do y = &x for global variables x and y, y gets initialized
at link time. Do the same for y = &x.f if x is a struct and y=&x[5]
if x is an array.
fixes #9217
fixes #9355
Change-Id: Iea3c0ce2ce1b309e2b760e345608fd95460b5713
Reviewed-on: https://go-review.googlesource.com/1691 Reviewed-by: Minux Ma <minux@golang.org>
Adam Langley [Thu, 18 Dec 2014 19:31:14 +0000 (11:31 -0800)]
crypto/tls: change default minimum version to TLS 1.0.
SSLv3 (the old minimum) is still supported and can be enabled via the
tls.Config, but this change increases the default minimum version to TLS
1.0. This is now common practice in light of the POODLE[1] attack
against SSLv3's CBC padding format.
Ben Burkert [Thu, 18 Dec 2014 18:17:54 +0000 (10:17 -0800)]
crypto/tls: enable TLS_FALLBACK_SCSV in server with default max version
Fix TLS_FALLBACK_SCSV check when comparing the client version to the
default max version. This enables the TLS_FALLBACK_SCSV check by default
in servers that do not explicitly set a max version in the tls config.
Change-Id: I5a51f9da6d71b79bc6c2ba45032be51d0f704b5e
Reviewed-on: https://go-review.googlesource.com/1776 Reviewed-by: Adam Langley <agl@golang.org>
Change-Id: I318ab11be8e3e7489fc1395457c029c8bdb2aa41
Reviewed-on: https://go-review.googlesource.com/1773 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jan Mercl [Thu, 18 Dec 2014 11:24:16 +0000 (12:24 +0100)]
cmd/yacc: Prevent index out of range. Fixes #9099.
Change-Id: I7ef01a738b6ca49af1c148146d439c81b0a33b16
Reviewed-on: https://go-review.googlesource.com/1785 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Austin Clements [Thu, 18 Dec 2014 06:08:09 +0000 (01:08 -0500)]
liblink: generate correct code for MOVD $-n(Rm), x on ppc64
On ppc64, liblink rewrites MOVD's of >32-bit constants by putting the
constant in memory and rewriting the MOVD to load from that memory
address. However, there were two bugs in the condition:
a) owing to an incorrect sign extension, it triggered for all negative
constants, and
b) it could trigger for constant offsets from registers (addresses of
the form $n(Rm) in assembly)
Together, these meant instructions of the form MOVD $-n(Rm), x were
compiled by putting -n in memory and rewriting the MOVD to load this
constant from memory (completely dropping Rm).
Change-Id: I1f6cc980efa3e3d6f164b46c985b2c3b55971cca
Reviewed-on: https://go-review.googlesource.com/1752 Reviewed-by: Minux Ma <minux@golang.org>
Ian Lance Taylor [Wed, 17 Dec 2014 16:52:59 +0000 (08:52 -0800)]
test: add bug494: test case miscompiled by gccgo
Gccgo incorrectly executed functions multiple times when they appeared
in a composite literal that required a conversion between different
interface types.
Kato Kazuyoshi [Tue, 16 Dec 2014 16:22:17 +0000 (08:22 -0800)]
os: handle the sticky bit separately for *BSD and Solaris
open(2) and mkdir(2) won't set the sticky bit on *BSD and Solaris.
This behavior is mentioned on sticky(8).
see also: https://github.com/dotcloud/docker/pull/6587
Fixes #8383.
Change-Id: Ic4733700f9926b9fc2b6fd1f998acec34e518764
Reviewed-on: https://go-review.googlesource.com/1673 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 16 Dec 2014 23:53:41 +0000 (15:53 -0800)]
test: add test that gccgo failed to compile
Gccgo failed to create the type descriptor for the type used to
allocate the nil value passed to append as the second argument when
append is called with only one argument. Calling append with only one
argument is unusual but obviously should not cause a compiler crash.