Michael Matloob [Sun, 21 Aug 2016 22:25:28 +0000 (18:25 -0400)]
cmd/link: use standard library flag package where possible
The obj library's flag functions are (mostly) light wrappers
around the standard library flag package. Use the flag package
directly where possible.
Most uses of the 'count'-type flags (except for -v) only check
against 0, so they can safely be replaced by bools. Only -v
and the flagfns haven't been replaced.
Debug has been turned into a slice of bools rather than ints.
There was a copy of the -v verbosity in ctxt.Debugvlog, so don't use
Debug['v'] and just use ctxt.Debugvlog.
Michael Matloob [Sun, 21 Aug 2016 17:52:23 +0000 (13:52 -0400)]
cmd/link/internal: remove global Ctxt variable
This change threads the *ld.Link Ctxt variable through
code in arch-specific packages. This removes all remaining
uses of Ctxt, so remove the global variable too.
Salman Aljammaz [Sun, 21 Aug 2016 15:59:56 +0000 (16:59 +0100)]
net/http: prepend ./ to directory list hrefs in FileServer
Certain browsers (Chrome 53, Safari 9.1.2, Firefox 46) won't correctly
follow a directory listing's links if the file name begins with a run
of characters then a colon, e.g. "foo:bar". Probably mistaking it for
a URI. However, they are happy to follow "./foo:bar", so this change
prepends "./" to all link hrefs in the directory listing of
FileServer.
Michael Matloob [Sat, 20 Aug 2016 02:40:38 +0000 (22:40 -0400)]
cmd/link/internal: thread *ld.Link through calls
Ctxt is a global defined in cmd/link/internal/ld of type *ld.Link.
Start threading a *ld.Link through function calls instead of
relying on the global variable.
Ctxt is still used as a global by the architecture-specific packages,
but I plan to fix that in a subsequent CL.
Change-Id: I77a3a58bd396fafd959fa1d8b1c83008a9f5a7fb
Reviewed-on: https://go-review.googlesource.com/27408
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Michael Munday [Sun, 21 Aug 2016 01:09:53 +0000 (21:09 -0400)]
hash/crc32: fix optimized s390x implementation
The code wasn't checking to see if the data was still >= 64 bytes
long after aligning it.
Aligning the data is an optimization and we don't actually need
to do it. In fact for smaller sizes it slows things down due to
the overhead of calling the generic function. Therefore for now
I have simply removed the alignment stage. I have also added a
check into the assembly to deliberately trigger a segmentation
fault if the data is too short.
Joe Tsai [Tue, 16 Aug 2016 23:03:00 +0000 (16:03 -0700)]
compress/flate: make huffmanBitWriter errors persistent
For persistent error handling, the methods of huffmanBitWriter have to be
consistent about how they check errors. It must either consistently
check error *before* every operation OR immediately *after* every
operation. Since most of the current logic uses the previous approach,
we apply the same style of error checking to writeBits and all calls
to Write such that they only operate if w.err is already nil going
into them.
The error handling approach is brittle and easily broken by future commits to
the code. In the near future, we should switch the logic to use panic at the
lowest levels and a recover at the edge of the public API to ensure
that errors are always persistent.
encoding/gob: error out instead of panicking on nil dereference
Do not panic when we encounter nil interface values which are
invalid values for gob. Previously this wasn't caught yet
we were calling reflect.*.Type() on reflect.Invalid values
thereby causing panic:
`panic: reflect: call of reflect.Value.Type on zero Value.`
which is a panic not enforced by encoding/gob itself.
We can catch this and send back an error to the caller.
Fixes #16204
Change-Id: Ie646796db297759a74a02eee5267713adbe0c3a0
Reviewed-on: https://go-review.googlesource.com/24989 Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
Dmitry Vyukov [Mon, 27 Jun 2016 10:23:39 +0000 (12:23 +0200)]
runtime: increase malloc size classes
When we calculate class sizes, in some cases we discard considerable
amounts of memory without an apparent reason. For example, we choose
size 8448 with 6 objects in 7 pages. But we can well use object
size 9472, which is also 6 objects in 7 pages but +1024 bytes (+12.12%).
Increase class sizes to the max value that leads to the same
page count/number of objects. Full list of affected size classes:
class 36: pages: 2 size: 1664->1792 +128 (7.69%)
class 39: pages: 1 size: 2560->2688 +128 (5.0%)
class 40: pages: 3 size: 2816->3072 +256 (9.9%)
class 41: pages: 2 size: 3072->3200 +128 (4.16%)
class 42: pages: 3 size: 3328->3456 +128 (3.84%)
class 44: pages: 3 size: 4608->4864 +256 (5.55%)
class 47: pages: 4 size: 6400->6528 +128 (2.0%)
class 48: pages: 5 size: 6656->6784 +128 (1.92%)
class 51: pages: 7 size: 8448->9472 +1024 (12.12%)
class 52: pages: 6 size: 8704->9728 +1024 (11.76%)
class 53: pages: 5 size: 9472->10240 +768 (8.10%)
class 54: pages: 4 size: 10496->10880 +384 (3.65%)
class 57: pages: 7 size: 14080->14336 +256 (1.81%)
class 59: pages: 9 size: 16640->18432 +1792 (10.76%)
class 60: pages: 7 size: 17664->19072 +1408 (7.97%)
class 62: pages: 8 size: 21248->21760 +512 (2.40%)
class 64: pages: 10 size: 24832->27264 +2432 (9.79%)
class 65: pages: 7 size: 28416->28672 +256 (0.90%)
Austin Clements [Fri, 19 Aug 2016 20:03:14 +0000 (16:03 -0400)]
runtime: fix check for vacuous page boundary rounding again
The previous fix for this, commit 336dad2a, had everything right in
the commit message, but reversed the test in the code. Fix the test in
the code.
This reversal effectively disabled the scavenger on large page systems
*except* in the rare cases where this code was originally wrong, which
is why it didn't obviously show up in testing.
Dmitry Vyukov [Mon, 8 Aug 2016 13:41:18 +0000 (15:41 +0200)]
internal/trace: fix analysis of EvGoWaiting/EvGoInSyscall events
When tracing is started in the middle of program execution,
we already have a number of runnable goroutines and a number
of blocked/in syscall goroutines. In order to reflect these
goroutines in the trace, we emit EvGoCreate for all existing
goroutines. Then for blocked/in syscall goroutines we additionally
emit EvGoWaiting/EvGoInSyscall events. These events don't reset g.ev
during trace analysis. So next EvGoStart finds g.ev set to the
previous EvGoCreate. As the result time between EvGoCreate and
EvGoStart is accounted as scheduler latency. While in reality
it is blocking/syscall time.
Properly reset g.ev for EvGoWaiting/EvGoInSyscall events.
Change-Id: I0615ba31ed7567600a0667ebb27458481da73adb
Reviewed-on: https://go-review.googlesource.com/25572 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Brad Fitzpatrick [Fri, 19 Aug 2016 02:21:26 +0000 (19:21 -0700)]
io: fix infinite loop bug in MultiReader
If an io.Reader returned (non-zero, EOF), MultiReader would yield
bytes forever.
This bug has existed before Go 1 (!!), introduced in the original
MultiReader implementation in https://golang.org/cl/1764043 and also
survived basically the only update to this code since then
(https://golang.org/cl/17873, git rev ccdca832c), which was added in
Go 1.7.
This just bit me when writing a test for some unrelated code.
Fixes #16795
Change-Id: I36e6a701269793935d19a47ac12f67b07179fbff
Reviewed-on: https://go-review.googlesource.com/27397
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Adam Langley [Thu, 18 Aug 2016 23:44:08 +0000 (16:44 -0700)]
crypto/x509: allow a leaf certificate to be specified directly as root.
In other systems, putting a leaf certificate in the root store works to
express that exactly that certificate is acceptable. This change makes
that work in Go too.
Matthew Dempsky [Tue, 7 Jun 2016 00:59:05 +0000 (17:59 -0700)]
cmd/compile/internal/syntax: match old parser errors and line numbers
This makes a bunch of changes to package syntax to tweak line numbers
for AST nodes. For example, short variable declaration statements are
now associated with the location of the ":=" token, and function calls
are associated with the location of the final ")" token. These help
satisfy many unit tests that assume the old parser's behavior.
Because many of these changes are questionable, they're guarded behind
a new "gcCompat" const to make them easy to identify and revisit in
the future.
A handful of remaining tests are too difficult to make behave
identically. These have been updated to execute with -newparser=0 and
comments explaining why they need to be fixed.
all.bash now passes with both the old and new parsers.
Change-Id: Iab834b71ca8698d39269f261eb5c92a0d55a3bf4
Reviewed-on: https://go-review.googlesource.com/27199
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Tue, 16 Aug 2016 20:33:29 +0000 (13:33 -0700)]
cmd/compile/internal/syntax: expose additional information for gc
gc needs access to line offsets for Nodes. It also needs access to the
end line offset for function bodies so it knows what line number to
use for things like implicit returns and defer executions.
Lastly, include an extra bool to distinguish between simple and full
slice expressions. This is redundant in valid parse trees, but needed
by gc for producing complete warnings in invalid inputs.
Change-Id: I64baf334a35c72336d26fa6755c67eb9d6f4e93c
Reviewed-on: https://go-review.googlesource.com/27196 Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Adam Langley [Wed, 17 Aug 2016 23:45:47 +0000 (16:45 -0700)]
crypto/tls: support AES-128-CBC cipher suites with SHA-256.
These were new with TLS 1.2 and, reportedly, some servers require it.
Since it's easy, this change adds suport for three flavours of
AES-128-CBC with SHA-256 MACs.
Other testdata/ files have to be updated because this changes the list
of cipher suites offered by default by the client.
Fixes #15487.
Change-Id: I1b14330c31eeda20185409a37072343552c3464f
Reviewed-on: https://go-review.googlesource.com/27315
Run-TryBot: Adam Langley <agl@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Jonathan Rudenberg <jonathan@titanous.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
unsafe: document use of &^ to round/align pointers
Follow-up to CL 27156
Change-Id: I4f1cfced2dced9c9fc8a05bbc00ec4229e85c5c9
Reviewed-on: https://go-review.googlesource.com/27314 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Thu, 18 Aug 2016 01:23:36 +0000 (21:23 -0400)]
cmd/compile: compare size in dead store elimination
Only remove stores that is shadowed by another store with same or
larger size. Normally we don't need this check because we did check
the types, but unsafe pointer casting can get around it.
Michael Matloob [Thu, 28 Jul 2016 17:04:41 +0000 (13:04 -0400)]
cmd: generate DWARF for functions in compile instead of link.
This is a copy of golang.org/cl/22092 by Ryan Brown.
Here's his original comment:
On my machine this increases the average time for 'go build cmd/go' from
2.25s to 2.36s. I tried to measure compile and link separately but saw
no significant change.
Change-Id: If0d2b756d52a0d30d4eda526929c82794d89dd7b
Reviewed-on: https://go-review.googlesource.com/25311
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Crawshaw <crawshaw@golang.org>
fmt treats interfaces as being transparent.
As a result, we cannot say with confidence
that any particular verb is wrong.
This fixes the following vet false positives
in the standard library:
database/sql/sql_test.go:210: arg dep for printf verb %p of wrong type: sql.finalCloser
fmt/fmt_test.go:1663: arg nil for printf verb %s of wrong type: untyped nil
go/ast/commentmap.go:328: arg node for printf verb %p of wrong type: ast.Node
net/http/transport_test.go:120: arg c for printf verb %p of wrong type: net.Conn
net/http/httptest/server.go:198: arg c for printf verb %p of wrong type: net.Conn
net/http/httputil/dump_test.go:258: arg body for printf verb %p of wrong type: io.Reader
reflect/set_test.go:81: arg x for printf verb %p of wrong type: io.Writer
reflect/set_test.go:141: arg bb for printf verb %p of wrong type: io.Reader
Adam Langley [Tue, 5 Jul 2016 20:50:18 +0000 (13:50 -0700)]
crypto/x509: support PSS signatures.
Although the term “RSA” is almost synonymous with PKCS#1 v1.5, that
standard is quite flawed, cryptographically speaking. Bellare and
Rogaway fixed PKCS#1 v1.5 with OAEP (for encryption) and PSS (for
signatures) but they only see a fraction of the use of v1.5.
This change adds support for creating and verifying X.509 certificates
that use PSS signatures. Sadly, every possible dimension of flexibility
seems to have been reflected in the integration of X.509 and PSS
resulting in a huge amount of excess complexity. This change only
supports one “sane” configuration for each of SHA-{256, 384, 512}.
Hopefully this is sufficient because it saves a lot of complexity in the
code.
Although X.509 certificates with PSS signatures are rare, I'm inclined
to look favourably on them because they are sufficiently superior.
Adam Langley [Wed, 17 Aug 2016 22:55:15 +0000 (15:55 -0700)]
crypto/x509: require a NULL parameters for RSA public keys.
The RFC is clear that the Parameters in an AlgorithmIdentifer for an RSA
public key must be NULL. BoringSSL enforces this so we have strong
evidence that this is a widely compatible change.
Embarrassingly enough, the major source of violations of this is us. Go
used to get this correct in only one of two places. This was only fixed
in 2013 (with 4874bc9b). That's why lots of test certificates are
updated in this change.
Adam Langley [Wed, 17 Aug 2016 20:18:43 +0000 (13:18 -0700)]
crypto/hmac: don't test for length equality in Equal.
subtle.ConstantTimeCompare now tests the length of the inputs (although
it didn't when this code was written) so this test in crypto/hmac is now
superfluous.
Radu Berinde [Tue, 16 Aug 2016 12:05:39 +0000 (08:05 -0400)]
hash/crc32: improve the processing of the last bytes in the SSE4.2 code for AMD64
This commit improves the processing of the final few bytes in
castagnoliSSE42: instead of processing one byte at a time, we use all
versions of the CRC32 instruction to process 4 bytes, then 2, then 1.
The difference is only noticeable for small "odd" sized buffers.
We do the similar improvement for processing the first few bytes in
the case of unaligned buffer.
Fixing the test which was not actually verifying the results for
misaligned buffers (WriteString was creating an internal copy which
was aligned).
Adding benchmarks for length 15 (aligned and misaligned), results
below.
If we cannot infer the asm arch from the filename
or the build tags, assume that it is the
current build arch. Assembly files with no
restrictions ought to be usable on all arches.
Cherry Zhang [Tue, 16 Aug 2016 18:17:33 +0000 (14:17 -0400)]
cmd/compile: add more ARM64 optimizations
- Use machine instructions for uint64<->float conversions
- Do not enforce alignment on Zero/Move
ARM64 supports unaligned load/stores, but only aligned offset
or small offset can be encoded into instructions.
- Do combined loads
Change-Id: Iffca7dd0f13070b17b784861ce5a30af584680eb
Reviewed-on: https://go-review.googlesource.com/27086 Reviewed-by: David Chase <drchase@google.com>
Matthew Dempsky [Wed, 17 Aug 2016 15:57:15 +0000 (08:57 -0700)]
cmd/compile/internal/s390x: cleanup betypeinit
The Width{int,ptr,reg} assignments are no longer necessary since
golang.org/cl/21623. The other arch's betypeinit functions were
cleaned up, but apparently this one was missed.
Change-Id: I1c7f074d7864a561659c1f98aef604f57f285fd0
Reviewed-on: https://go-review.googlesource.com/27272
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
math/rand: Document origin of cooked pseudo-random numbers
The Source provided by math/rand relies on an array of cooked
pseudo-random 63bit integers for seeding. The origin of these
numbers is undocumented.
Add a standalone program in math/rand folder that generates
the 63bit integer array as well as a 64bit version supporting
extension of the Source to 64bit pseudo-random number
generation while maintaining the current sequence in the
lower 63bit.
The code is largely based on the initial implementation of the
random number generator in the go repository by Ken Thompson
(revision 399).
Austin Clements [Wed, 17 Aug 2016 02:05:57 +0000 (22:05 -0400)]
runtime: fix check for vacuous page boundary rounding
sysUnused (e.g., madvise MADV_FREE) is only sensible to call on
physical page boundaries, so scavengelist rounds in the bounds of the
region being released to the nearest physical page boundaries.
However, if the region is smaller than a physical page and neither the
start nor end fall on a boundary, then rounding the start up to a page
boundary and the end down to a page boundary will result in end < start.
Currently, we only give up on the region if start == end, so if we
encounter end < start, we'll call madvise with a negative length and
the madvise will fail.
Issue #16644 gives a concrete example of this:
start = 0x1285ac000
end = 0x1285ae000 (1 8K page)
This leads to the rounded values
start = 0x1285b0000
end = 0x1285a0000
which leads to len = -65536.
Fix this by giving up on the region if end <= start, not just if
end == start.
Fixes #16644.
Change-Id: I8300db492dbadc82ac1ad878318b36bcb7c39524
Reviewed-on: https://go-review.googlesource.com/27230 Reviewed-by: Keith Randall <khr@golang.org>
Robert Griesemer [Wed, 17 Aug 2016 00:32:54 +0000 (17:32 -0700)]
cmd/compile: remove conditional code dealing with two export formats
This removes some scaffolding introduced pre-1.7, introduced to
fix an export format bug, and to minimize conflicts with older
formats. The currently deployed and recognized format is "v1",
so don't worry about other versions. This is a step towards a
better scheme for internal export format versioning.
For #16244.
Change-Id: Ic7cf99dd2a24ad5484cc54aed44fa09332c2cf72
Reviewed-on: https://go-review.googlesource.com/27205 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Robert Griesemer <gri@golang.org>
cmd/internal/obj: reduce per-architecture opcode space
s390x took up the last available chunk of int16 opcodes.
There are RISC-V and sparc64 ports in progress out of tree,
and there will likely be other architectures.
Reduce the opcode space to allow more architectures to
fit without increasing to int32.
This is the smallest power of two that accomodates all
existing architectures. All else being equal, smaller is
better--smaller numbers are easier to generate immediates
for and easier on the eyes when debugging.
Keith Randall [Wed, 6 Jul 2016 22:02:49 +0000 (15:02 -0700)]
runtime: fix map iterator concurrent map check
We should check whether there is a concurrent writer at the
start of every mapiternext, not just in mapaccessK (which is
only called during certain map growth situations).
Tests turned off by default because they are inherently flaky.
Keith Randall [Thu, 2 Jun 2016 19:41:42 +0000 (12:41 -0700)]
cmd/compile: use shorter versions of zero-extend ops
Only need to zero-extend to 32 bits and we get the top
32 bits zeroed for free.
Only the WQ change actually generates different code.
The assembler did this optimization for us in the other two cases.
But we might as well do it during SSA so -S output more closely
matches the actual generated instructions.