]> Cypherpunks repositories - gostls13.git/log
gostls13.git
9 years agocmd/compile: use *byte for pointer to string memory
Matthew Dempsky [Wed, 30 Mar 2016 21:26:04 +0000 (14:26 -0700)]
cmd/compile: use *byte for pointer to string memory

Change-Id: I14cff013ef4920d33b5ead0c75d51681bb4f635a
Reviewed-on: https://go-review.googlesource.com/21337
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agotesting: unflake TestBRun
Marcel van Lohuizen [Mon, 4 Apr 2016 17:05:48 +0000 (19:05 +0200)]
testing: unflake TestBRun

by only testing the lower bound of memalloc

Fixes #15063

Change-Id: Iab2fdd75e9ce98c641bfbce57f142fa47176772d
Reviewed-on: https://go-review.googlesource.com/21507
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>

9 years agocmd/compile: constant fold modulo
Alexandru Moșoi [Mon, 4 Apr 2016 14:13:35 +0000 (16:13 +0200)]
cmd/compile: constant fold modulo

Fixes #15079

Change-Id: Ib4dd9eab322da39234008e040100e75cb58761b3
Reviewed-on: https://go-review.googlesource.com/21501
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: fold IsInBounds of modulo.
Alexandru Moșoi [Mon, 4 Apr 2016 14:38:26 +0000 (16:38 +0200)]
cmd/compile: fold IsInBounds of modulo.

In b we only need the division by 0 check.

func b(i uint, v []byte) byte {
    return v[i%uint(len(v))]
}

Updates #15079.

Change-Id: Ic7491e677dd57cd6ba577efbce576dbb6e023cbd
Reviewed-on: https://go-review.googlesource.com/21502
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ahmed Waheed <oneofone@gmail.com>
9 years agoruntime: fix pagesInUse accounting
Austin Clements [Tue, 29 Mar 2016 16:28:24 +0000 (12:28 -0400)]
runtime: fix pagesInUse accounting

When we grow the heap, we create a temporary "in use" span for the
memory acquired from the OS and then free that span to link it into
the heap. Hence, we (1) increase pagesInUse when we make the temporary
span so that (2) freeing the span will correctly decrease it.

However, currently step (1) increases pagesInUse by the number of
pages requested from the heap, while step (2) decreases it by the
number of pages requested from the OS (the size of the temporary
span). These aren't necessarily the same, since we round up the number
of pages we request from the OS, so steps 1 and 2 don't necessarily
cancel out like they're supposed to. Over time, this can add up and
cause pagesInUse to underflow and wrap around to 2^64. The garbage
collector computes the sweep ratio from this, so if this happens, the
sweep ratio becomes effectively infinite, causing the first allocation
on each P in a sweep cycle to sweep the entire heap. This makes
sweeping effectively STW.

Fix this by increasing pagesInUse in step 1 by the number of pages
requested from the OS, so that the two steps correctly cancel out. We
add a test that checks that the running total matches the actual state
of the heap.

Fixes #15022. For 1.6.x.

Change-Id: Iefd9d6abe37d0d447cbdbdf9941662e4f18eeffc
Reviewed-on: https://go-review.googlesource.com/21280
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
9 years agocmd/link: keep types together when dynamic linking
David Crawshaw [Fri, 1 Apr 2016 14:55:21 +0000 (10:55 -0400)]
cmd/link: keep types together when dynamic linking

To refer to types and names by offsets, we want to keep the symbols in
the same sections. Do this by making all types .relro for now.

Once name offsets are further along, name data can move out of relro.

Change-Id: I1cbd2e914bd180cdf25c4aeb13d9c1c734febe69
Reviewed-on: https://go-review.googlesource.com/21394
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agogo/types: better error when assigning to struct field in map
Caio Marcelo de Oliveira Filho [Sat, 2 Apr 2016 15:04:45 +0000 (12:04 -0300)]
go/types: better error when assigning to struct field in map

Identify this assignment case and instead of the more general error

    prog.go:6: cannot assign to students["sally"].age (value of type int)

produce

    prog.go:6: cannot directly assign to struct field students["sally"].age in map

that explains why the assignment is not possible. Used ExprString
instead of String of operand since the type of the field is not relevant
to the error.

Updates #13779.

Change-Id: I581251145ae6336ddd181b9ddd77f657c51b5aff
Reviewed-on: https://go-review.googlesource.com/21463
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: better error when assigning to struct field in map
Caio Marcelo de Oliveira Filho [Sat, 2 Apr 2016 14:27:02 +0000 (11:27 -0300)]
cmd/compile: better error when assigning to struct field in map

Identify this assignment case and instead of the more general error

    prog.go:6: cannot assign to students["sally"].age

produce

    prog.go:6: cannot directly assign to struct field students["sally"].age in map

that explains why the assignment is not possible.

Fixes #13779.

Change-Id: I90c10b445f907834fc1735aa66e44a0f447aa74f
Reviewed-on: https://go-review.googlesource.com/21462
Reviewed-by: David Chase <drchase@google.com>
9 years agoruntime: change osyield to use Windows SwitchToThread
Alex Brainman [Wed, 30 Mar 2016 05:33:52 +0000 (16:33 +1100)]
runtime: change osyield to use Windows SwitchToThread

It appears that windows osyield is just 15ms sleep on my computer
(see benchmarks below). Replace NtWaitForSingleObject in osyield
with SwitchToThread (as suggested by Dmitry).

Also add issue #14790 related benchmarks, so we can track perfomance
changes in CL 20834 and CL 20835 and beyond.

Update #14790

benchmark                             old ns/op     new ns/op     delta
BenchmarkChanToSyscallPing1ms         1953200       1953000       -0.01%
BenchmarkChanToSyscallPing15ms        31562904      31248400      -1.00%
BenchmarkSyscallToSyscallPing1ms      5247          4202          -19.92%
BenchmarkSyscallToSyscallPing15ms     5260          4374          -16.84%
BenchmarkChanToChanPing1ms            474           494           +4.22%
BenchmarkChanToChanPing15ms           468           489           +4.49%
BenchmarkOsYield1ms                   980018        75.5          -99.99%
BenchmarkOsYield15ms                  15625200      75.8          -100.00%

Change-Id: I1b4cc7caca784e2548ee3c846ca07ef152ebedce
Reviewed-on: https://go-review.googlesource.com/21294
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agohtml/template: prefix the internally-used FuncMap values with an underscore
Rob Pike [Mon, 4 Apr 2016 03:45:52 +0000 (20:45 -0700)]
html/template: prefix the internally-used FuncMap values with an underscore

This makes these names even less likely to collide with a real user-defined function.

Fixes #13852.

Change-Id: If5a8562c6797ced19c355c7ab2c86fc4401a8674
Reviewed-on: https://go-review.googlesource.com/21490
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
9 years agocmd/go: fix -buildmode=c-archive should work on windows
Christopher Nelson [Sun, 13 Dec 2015 13:02:29 +0000 (08:02 -0500)]
cmd/go: fix -buildmode=c-archive should work on windows

Add supporting code for runtime initialization, including both
32- and 64-bit x86 architectures.

Add .ctors section on Windows to PE .o files, and INITENTRY to .ctors
section to plug in to the GCC C/C++ startup initialization mechanism.
This allows the Go runtime to initialize itself. Add .text section
symbol for .ctor relocations. Note: This is unlikely to be useful for
MSVC-based toolchains.

Fixes #13494

Change-Id: I4286a96f70e5f5228acae88eef46e2bed95813f3
Reviewed-on: https://go-review.googlesource.com/18057
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>

9 years agocmd/compile: give ChanDir a type
Brad Fitzpatrick [Sun, 3 Apr 2016 22:58:10 +0000 (22:58 +0000)]
cmd/compile: give ChanDir a type

Change-Id: I03621db79637b04982e1f0e7b4268c4ed2db6d22
Reviewed-on: https://go-review.googlesource.com/21484
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/link: simplify readSymName, taking advantage of bufio.Reader
Brad Fitzpatrick [Sun, 3 Apr 2016 18:27:17 +0000 (18:27 +0000)]
cmd/link: simplify readSymName, taking advantage of bufio.Reader

Now that cmd/link uses bufio.Reader, take advantage of it.
I find this new version easier to reason about.

Reduces allocations by 1.1% when linking a basic HTTP server.

Numbers are stable with each round measuring using:
rm prof.mem; go tool link -o foo  -memprofile=prof.mem -memprofilerate=1 foo.a

Before:

65.36MB of 74.53MB total (87.70%)
Dropped 157 nodes (cum <= 0.37MB)
Showing top 10 nodes out of 39 (cum >= 1.47MB)
      flat  flat%   sum%        cum   cum%
   21.48MB 28.81% 28.81%    21.48MB 28.81%  cmd/link/internal/ld.Linklookup
   16.04MB 21.52% 50.33%    16.04MB 21.52%  cmd/link/internal/ld.(*objReader).readSlices
    4.61MB  6.19% 56.52%     4.61MB  6.19%  cmd/link/internal/ld.(*objReader).readSymName
    4.51MB  6.05% 62.57%     6.32MB  8.48%  cmd/link/internal/ld.writelines
    4.50MB  6.03% 68.60%     4.50MB  6.03%  cmd/link/internal/ld.Symgrow
    4.02MB  5.39% 73.99%     4.02MB  5.39%  cmd/link/internal/ld.linknew
    3.98MB  5.34% 79.33%     3.98MB  5.34%  cmd/link/internal/ld.setaddrplus
    2.96MB  3.97% 83.30%    28.78MB 38.62%  cmd/link/internal/ld.(*objReader).readRef
    1.81MB  2.43% 85.73%     1.81MB  2.43%  cmd/link/internal/ld.newcfaoffsetattr
    1.47MB  1.97% 87.70%     1.47MB  1.97%  cmd/link/internal/ld.(*objReader).readSym

After:

64.66MB of 73.87MB total (87.53%)
Dropped 156 nodes (cum <= 0.37MB)
Showing top 10 nodes out of 40 (cum >= 1.47MB)
      flat  flat%   sum%        cum   cum%
   21.48MB 29.08% 29.08%    21.48MB 29.08%  cmd/link/internal/ld.Linklookup
   16.04MB 21.71% 50.79%    16.04MB 21.71%  cmd/link/internal/ld.(*objReader).readSlices
    4.51MB  6.10% 56.90%     6.32MB  8.56%  cmd/link/internal/ld.writelines
    4.50MB  6.09% 62.99%     4.50MB  6.09%  cmd/link/internal/ld.Symgrow
    4.02MB  5.44% 68.42%     4.02MB  5.44%  cmd/link/internal/ld.linknew
    3.98MB  5.38% 73.81%     3.98MB  5.38%  cmd/link/internal/ld.setaddrplus
    3.90MB  5.28% 79.09%     3.90MB  5.28%  cmd/link/internal/ld.(*objReader).readSymName
    2.96MB  4.01% 83.09%    28.08MB 38.01%  cmd/link/internal/ld.(*objReader).readRef
    1.81MB  2.45% 85.55%     1.81MB  2.45%  cmd/link/internal/ld.newcfaoffsetattr
    1.47MB  1.99% 87.53%     1.47MB  1.99%  cmd/link/internal/ld.(*objReader).readSym

Also tested locally with asserts that that the calculated length is
always correct and thus the adjName buf never reallocates.

Change-Id: I19e3e8bfa6a12bcd8b5216f6232f42c122e4f80e
Reviewed-on: https://go-review.googlesource.com/21481
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/link: define a variable for the target platform's elf relocation type
Michael Hudson-Doyle [Sun, 3 Apr 2016 07:32:31 +0000 (19:32 +1200)]
cmd/link: define a variable for the target platform's elf relocation type

Rather than having half a dozen switch statements. Also remove some c2go dregs.

Change-Id: I19af5b64f73369126020e15421c34cad5bbcfbf8
Reviewed-on: https://go-review.googlesource.com/21442
Reviewed-by: Ian Lance Taylor <iant@golang.org>
9 years agosyscall: add support for s390x
Michael Munday [Thu, 31 Mar 2016 03:56:49 +0000 (23:56 -0400)]
syscall: add support for s390x

On s390x char is unsigned. We cannot force it to be signed using
-fsigned-char (see arm64) because the s390x gccgo API is already
public and we need to stick as closely as possible to it to avoid
breaking existing projects. In order to match the gccgo API we
also force the RawSockaddr.Data and RawSockaddrUnix.Path fields
to be signed.

This CL adds a post-processing pass (mkpost.go) to mkall.sh in
order to export the types of fields in PtraceRegs on s390x
without affecting the API on other platforms. The types of these
fields match their counterparts in gccgo. mkpost.go also cleans
up the Pad_cgo* fields and X_* fields (these fields are not
exported by gccgo currently). It could be extended to add build
tags on platforms that need them.

Change-Id: I66bdf5b86ec98af70baf666989027bb354df9e3e
Reviewed-on: https://go-review.googlesource.com/20961
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/link: more idiomatic object reader
Shahar Kohanim [Thu, 31 Mar 2016 15:34:02 +0000 (18:34 +0300)]
cmd/link: more idiomatic object reader

name       old secs    new secs    delta
LinkCmdGo   0.52 ± 3%   0.52 ± 7%    ~     (p=0.325 n=93+100)

name       old MaxRSS  new MaxRSS  delta
LinkCmdGo   120k ± 1%   118k ± 4%  -1.10%   (p=0.000 n=87+96)

Change-Id: I967660b8dc6036d28eeea1b6b30f400fadd57b05
Reviewed-on: https://go-review.googlesource.com/21372
Run-TryBot: Shahar Kohanim <skohanim@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agoall: fix spelling mistakes
Eric Engestrom [Sun, 3 Apr 2016 11:43:27 +0000 (12:43 +0100)]
all: fix spelling mistakes

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Change-Id: I91873aaebf79bdf1c00d38aacc1a1fb8d79656a7
Reviewed-on: https://go-review.googlesource.com/21433
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agopath, path/filepath: add Join example with joined rooted path
Russ Cox [Tue, 15 Mar 2016 00:23:22 +0000 (20:23 -0400)]
path, path/filepath: add Join example with joined rooted path

This makes clear that Go's path.Join and filepath.Join are different
from the Python os.path.join (and perhaps others).

Requested in private mail.

Change-Id: Ie5dfad8a57f9baa5cca31246af1fd4dd5b1a64ee
Reviewed-on: https://go-review.googlesource.com/20711
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/link/internal/s390x: add s390x support
Michael Munday [Sat, 19 Mar 2016 04:23:17 +0000 (00:23 -0400)]
cmd/link/internal/s390x: add s390x support

Supports basic internal linking, external linking and shared library
linking.

Change-Id: Ic9693dd94df561b0f9df9880009b1e8ab9d59b92
Reviewed-on: https://go-review.googlesource.com/20879
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
9 years agoos: make IsExists also recognize syscall.ENOTEMPTY
Brad Fitzpatrick [Sun, 3 Apr 2016 00:44:39 +0000 (00:44 +0000)]
os: make IsExists also recognize syscall.ENOTEMPTY

And adds missing tests.

Fixes #14970

Change-Id: I0dba02603bc245f555498cb5dd3e0a9d87c52353
Reviewed-on: https://go-review.googlesource.com/21467
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agofmt: hold on to all free buffers, regardless of size
Brad Fitzpatrick [Sun, 3 Apr 2016 01:34:29 +0000 (01:34 +0000)]
fmt: hold on to all free buffers, regardless of size

This code made sense before fmt switched to using sync.Pool, but a
sync.Pool clears all items on GC, so not reusing something based on
size is just a waste of memory.

Change-Id: I201312b0ee6c572ff3c0ffaf71e42623a160d23f
Reviewed-on: https://go-review.googlesource.com/21480
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/link/internal/ld: add s390x support
Michael Munday [Fri, 18 Mar 2016 20:57:54 +0000 (16:57 -0400)]
cmd/link/internal/ld: add s390x support

Introduces the new relocation variant RV_390_DBL which indicates
that the relocation value should be shifted right by 1 (to make
it 2-byte aligned).

Change-Id: I03fa96b4759ee19330c5298c3720746622fb1a03
Reviewed-on: https://go-review.googlesource.com/20878
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: add Type.ChanDir
Josh Bleecher Snyder [Sat, 2 Apr 2016 23:26:30 +0000 (16:26 -0700)]
cmd/compile: add Type.ChanDir

Generated with eg.

Passes toolstash -cmp.

Change-Id: I3af35191e73a558080f777a4eed93bcec7dfe1f5
Reviewed-on: https://go-review.googlesource.com/21469
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocompress/gzip: fix Reader.Reset
Joe Tsai [Sat, 2 Apr 2016 22:24:32 +0000 (15:24 -0700)]
compress/gzip: fix Reader.Reset

Rather than specifying every field that should be cleared in Reset,
it is better to just zero the entire struct and only preserve or set the
fields that we actually care about. This ensures that the Header field
is reset for the next use.

Fixes #15077

Change-Id: I41832e506d2d64c62b700aa1986e7de24a577511
Reviewed-on: https://go-review.googlesource.com/21465
Run-TryBot: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: handle non-negatives in prove
Alexandru Moșoi [Sat, 2 Apr 2016 08:29:11 +0000 (10:29 +0200)]
cmd/compile: handle non-negatives in prove

Handle this case:
if 0 <= i && i < len(a) {
        use a[i]
}

Shaves about 5k from pkg/tools/linux_amd64/*.

Change-Id: I6675ff49aa306b0d241b074c5738e448204cd981
Reviewed-on: https://go-review.googlesource.com/21431
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agonet: don't do DNS for onion and local addresses
Brad Fitzpatrick [Thu, 31 Mar 2016 06:08:44 +0000 (23:08 -0700)]
net: don't do DNS for onion and local addresses

Fixes #13705

Change-Id: I86c60c78ce0394f830f904c9cba83ebbf3efc046
Reviewed-on: https://go-review.googlesource.com/21328
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocompress/gzip: cleanup gzip package
Joe Tsai [Sat, 2 Apr 2016 01:11:26 +0000 (18:11 -0700)]
compress/gzip: cleanup gzip package

Changes made:
* Reader.flg is not used anywhere else other than readHeader and
does not need to be stored.
* Store Reader.digest and Writer.digest as uint32s rather than as
a hash.Hash32 and use the crc32.Update function instead. This simplifies
initialization logic since the zero value of uint32 is the initial
CRC-32 value. There are no performance detriments to doing this since
the hash.Hash32 returned by crc32 simply calls crc32.Update as well.
* s/[0:/[:/            Consistently use shorter notation for slicing.
* s/RFC1952/RFC 1952/  Consistently use RFC notation.

Change-Id: I55416a19f4836cbed943adaa3f672538ea5d166d
Reviewed-on: https://go-review.googlesource.com/21429
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Joe Tsai <joetsai@digital-static.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: fix inlining of switch issue
Todd Neal [Sat, 2 Apr 2016 04:28:14 +0000 (23:28 -0500)]
cmd/compile: fix inlining of switch issue

The issue was seen when inlining an exported function that contained
a fallthrough statement.

Fixes #15071

Change-Id: I1e8215ad49d57673dba7e8f8bd2ed8ad290dc452
Reviewed-on: https://go-review.googlesource.com/21452
Reviewed-by: Dave Cheney <dave@cheney.net>
9 years agosyscall: point to x/sys in DLL loading docs, update syscall generator
Brad Fitzpatrick [Sat, 2 Apr 2016 00:20:13 +0000 (00:20 +0000)]
syscall: point to x/sys in DLL loading docs, update syscall generator

Updates the syscall generator for patchset 4 of https://golang.org/cl/21388.

Updates #14959

Change-Id: Icbd6df489887d3dcc076dfc73d4feb1376abaf8b
Reviewed-on: https://go-review.googlesource.com/21428
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
9 years agocmd/compile: eliminate dead code in walkappend
Matthew Dempsky [Fri, 1 Apr 2016 23:43:43 +0000 (16:43 -0700)]
cmd/compile: eliminate dead code in walkappend

The IsStruct case is meant to handle cases like append(f()) where f's
result parameters are something like ([]int, int, int). However, at
this point in the compiler we've already rewritten append(f()) into
"tmp1, tmp2, tmp3 := f(); append(tmp1, tmp2, tmp3)".

As further evidence, the t.Elem() is not a valid method call for a
struct type anyway, which would trigger the Fatalf call in Type.Elem
if this code was ever hit.

Change-Id: Ia066f93df66ee3fadc9a9a0f687be7b5263af163
Reviewed-on: https://go-review.googlesource.com/21427
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agoruntime, syscall: only search for Windows DLLs in the System32 directory
Brad Fitzpatrick [Fri, 25 Mar 2016 06:40:58 +0000 (06:40 +0000)]
runtime, syscall: only search for Windows DLLs in the System32 directory

Make sure that for any DLL that Go uses itself, we only look for the
DLL in the Windows System32 directory, guarding against DLL preloading
attacks.

(Unless the Windows version is ancient and LoadLibraryEx is
unavailable, in which case the user probably has bigger security
problems anyway.)

This does not change the behavior of syscall.LoadLibrary or NewLazyDLL
if the DLL name is something unused by Go itself.

This change also intentionally does not add any new API surface. Instead,
x/sys is updated with a LoadLibraryEx function and LazyDLL.Flags in:
    https://golang.org/cl/21388

Updates #14959

Change-Id: I8d29200559cc19edf8dcf41dbdd39a389cd6aeb9
Reviewed-on: https://go-review.googlesource.com/21140
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agoruntime: allocate mp.cgocallers earlier
Ian Lance Taylor [Fri, 1 Apr 2016 22:06:25 +0000 (15:06 -0700)]
runtime: allocate mp.cgocallers earlier

Fixes #15061.

Change-Id: I71f69f398d1c5f3a884bbd044786f1a5600d0fae
Reviewed-on: https://go-review.googlesource.com/21398
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: rename Node.Int to Node.Int64
Josh Bleecher Snyder [Fri, 1 Apr 2016 21:51:02 +0000 (14:51 -0700)]
cmd/compile: rename Node.Int to Node.Int64

gorename -from '"cmd/compile/internal/gc".Node.Int' -to 'Int64'

Change-Id: I2fe3bf9a26ae6b0600d990d0c981e4b8b53020a4
Reviewed-on: https://go-review.googlesource.com/21426
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: more Isfoo Type cleanups
Matthew Dempsky [Fri, 1 Apr 2016 20:36:24 +0000 (13:36 -0700)]
cmd/compile: more Isfoo Type cleanups

Replace isideal(t) with t.IsUntyped().
Replace Istype(t, k) with t.IsKind(k).
Replace isnilinter(t) with t.IsEmptyInterface().

Also replace a lot of t.IsKind(TFOO) with t.IsFoo().

Replacements prepared mechanically with gofmt -w -r.

Passes toolstash -cmp.

Change-Id: Iba48058f3cc863e15af14277b5ff5e729e67e043
Reviewed-on: https://go-review.googlesource.com/21424
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>

9 years agocmd/compile: add Type.SetNumElem
Josh Bleecher Snyder [Thu, 31 Mar 2016 22:18:39 +0000 (15:18 -0700)]
cmd/compile: add Type.SetNumElem

This removes all access to Type.Bound
from outside type.go.

Update sinit to make a new type rather than
copy and mutate.

Update bimport to create a new slice type
instead of mutating TDDDFIELD.
These are rare, so the extra allocs are nominal.

I’m not happy about having a setter,
but it appears the most practical route
forward at the moment, and it only has a few uses.

Passes toolstash -cmp.

Change-Id: I174f07c8f336afc656904bde4bdbde4f3ef0db96
Reviewed-on: https://go-review.googlesource.com/21423
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: use Node.Int more
Josh Bleecher Snyder [Thu, 31 Mar 2016 21:54:46 +0000 (14:54 -0700)]
cmd/compile: use Node.Int more

Generated by eg.

Passes toolstash -cmp.

Change-Id: I7516c211ca9aacf824f74894671dc62d31763b01
Reviewed-on: https://go-review.googlesource.com/21422
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: use NumElem instead of Type.Bound
Josh Bleecher Snyder [Thu, 31 Mar 2016 21:46:04 +0000 (14:46 -0700)]
cmd/compile: use NumElem instead of Type.Bound

This eliminates all direct reads of Type.Bound
outside type.go.

Change-Id: I0a9a72539f8f4c0de7f5e05e1821936bf7db5eb7
Reviewed-on: https://go-review.googlesource.com/21421
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: cleanup algtype code
Matthew Dempsky [Fri, 1 Apr 2016 18:22:03 +0000 (11:22 -0700)]
cmd/compile: cleanup algtype code

Add AlgKind enum type to represent AFOO values.

Add IsComparable, IsRegularMemory, IncomparableField helper methods to
codify common higher-level idioms.

Passes toolstash -cmp.

Change-Id: I54c544953997a8ccc72396b3058897edcbbea392
Reviewed-on: https://go-review.googlesource.com/21420
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agoruntime/cgo: only build _cgo_callers if x_cgo_callers is defined
Ian Lance Taylor [Fri, 1 Apr 2016 15:33:25 +0000 (08:33 -0700)]
runtime/cgo: only build _cgo_callers if x_cgo_callers is defined

Fixes a problem when using the external linker on Solaris.  The Solaris
external linker still doesn't work due to issue #14957.

The problem is, for example, with `go test cmd/objdump`:

        objdump_test.go:71: go build fmthello.go: exit status 2
                # command-line-arguments
                /var/gcc/iant/go/pkg/tool/solaris_amd64/link: running gcc failed: exit status 1
                Undefined                       first referenced
                 symbol                             in file
                x_cgo_callers                       /tmp/go-link-355600608/go.o
                ld: fatal: symbol referencing errors
                collect2: error: ld returned 1 exit status

Change-Id: I54917cfd5c288ee77ea25c439489bd2c9124fe73
Reviewed-on: https://go-review.googlesource.com/21392
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
9 years agoreflect: implement StructOf
Sebastien Binet [Sat, 5 Mar 2016 12:37:38 +0000 (13:37 +0100)]
reflect: implement StructOf

This change exposes a facility to create new struct types from a slice of
reflect.StructFields.

- reflect: first stab at implementing StructOf
- reflect: tests for StructOf

StructOf creates new struct types in the form of structTypeWithMethods
to accomodate the GC (especially the uncommonType.methods slice field.)

Creating struct types with embedded interfaces with unexported methods
is not supported yet and will panic.
Creating struct types with non-ASCII field names or types is not yet
supported (see #15064.)

Binaries' sizes for linux_amd64:

old=tip (0104a31)

            old bytes     new bytes     delta
bin/go      9911336       9915456       +0.04%
reflect     781704        830048        +6.18%

Updates #5748.
Updates #15064.

Change-Id: I3b8fd4fadd6ce3b1b922e284f0ae72a3a8e3ce44
Reviewed-on: https://go-review.googlesource.com/9251
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>

9 years agocmd/internal/obj: more idiomatic object writer
Shahar Kohanim [Thu, 31 Mar 2016 09:59:05 +0000 (12:59 +0300)]
cmd/internal/obj: more idiomatic object writer

Change-Id: I41722ee605ea76a6b52e8a7e1e10f2293cef1a7a
Reviewed-on: https://go-review.googlesource.com/21371
Reviewed-by: David Crawshaw <crawshaw@golang.org>
9 years agocmd/compile/internal/ssa: BCE for induction variables
Alexandru Moșoi [Wed, 2 Mar 2016 11:58:27 +0000 (12:58 +0100)]
cmd/compile/internal/ssa: BCE for induction variables

There are 5293 loop in the main go repository.
A survey of the top most common for loops:

     18 for __k__ := 0; i < len(sa.Addr); i++ {
     19 for __k__ := 0; ; i++ {
     19 for __k__ := 0; i < 16; i++ {
     25 for __k__ := 0; i < length; i++ {
     30 for __k__ := 0; i < 8; i++ {
     49 for __k__ := 0; i < len(s); i++ {
     67 for __k__ := 0; i < n; i++ {
    376 for __k__ := range __slice__ {
    685 for __k__, __v__ := range __slice__ {
   2074 for __, __v__ := range __slice__ {

The algorithm to find induction variables handles all cases
with an upper limit. It currently doesn't find related induction
variables such as c * ind or c + ind.

842 out of 22954 bound checks are removed for src/make.bash.
1957 out of 42952 bounds checks are removed for src/all.bash.

Things to do in follow-up CLs:
* Find the associated pointer for `for _, v := range a {}`
* Drop the NilChecks on the pointer.
* Replace the implicit induction variable by a loop over the pointer

Generated garbage can be reduced if we share the sdom between passes.

% benchstat old.txt new.txt
name       old time/op     new time/op     delta
Template       337ms ± 3%      333ms ± 3%    ~             (p=0.258 n=9+9)
GoTypes        1.11s ± 2%      1.10s ± 2%    ~           (p=0.912 n=10+10)
Compiler       5.25s ± 1%      5.29s ± 2%    ~             (p=0.077 n=9+9)
MakeBash       33.5s ± 1%      34.1s ± 2%  +1.85%          (p=0.011 n=9+9)

name       old alloc/op    new alloc/op    delta
Template      63.6MB ± 0%     63.9MB ± 0%  +0.52%         (p=0.000 n=10+9)
GoTypes        218MB ± 0%      219MB ± 0%  +0.59%         (p=0.000 n=10+9)
Compiler       978MB ± 0%      985MB ± 0%  +0.69%        (p=0.000 n=10+10)

name       old allocs/op   new allocs/op   delta
Template        582k ± 0%       583k ± 0%  +0.10%        (p=0.000 n=10+10)
GoTypes        1.78M ± 0%      1.78M ± 0%  +0.12%        (p=0.000 n=10+10)
Compiler       7.68M ± 0%      7.69M ± 0%  +0.05%        (p=0.000 n=10+10)

name       old text-bytes  new text-bytes  delta
HelloSize       581k ± 0%       581k ± 0%  -0.08%        (p=0.000 n=10+10)
CmdGoSize      6.40M ± 0%      6.39M ± 0%  -0.08%        (p=0.000 n=10+10)

name       old data-bytes  new data-bytes  delta
HelloSize      3.66k ± 0%      3.66k ± 0%    ~     (all samples are equal)
CmdGoSize       134k ± 0%       134k ± 0%    ~     (all samples are equal)

name       old bss-bytes   new bss-bytes   delta
HelloSize       126k ± 0%       126k ± 0%    ~     (all samples are equal)
CmdGoSize       149k ± 0%       149k ± 0%    ~     (all samples are equal)

name       old exe-bytes   new exe-bytes   delta
HelloSize       947k ± 0%       946k ± 0%  -0.01%        (p=0.000 n=10+10)
CmdGoSize      9.92M ± 0%      9.91M ± 0%  -0.06%        (p=0.000 n=10+10)

Change-Id: Ie74bdff46fd602db41bb457333d3a762a0c3dc4d
Reviewed-on: https://go-review.googlesource.com/20517
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>

9 years agoruntime: support symbolic backtrace of C code in a cgo crash
Ian Lance Taylor [Sat, 12 Dec 2015 01:16:48 +0000 (17:16 -0800)]
runtime: support symbolic backtrace of C code in a cgo crash

The new function runtime.SetCgoTraceback may be used to register stack
traceback and symbolizer functions, written in C, to do a stack
traceback from cgo code.

There is a sample implementation of runtime.SetCgoSymbolizer at
github.com/ianlancetaylor/cgosymbolizer.  Just importing that package is
sufficient to get symbolic C backtraces.

Currently only supported on linux/amd64.

Change-Id: If96ee2eb41c6c7379d407b9561b87557bfe47341
Reviewed-on: https://go-review.googlesource.com/17761
Reviewed-by: Austin Clements <austin@google.com>
9 years agocmd/compile: ignore OXXX nodes in closure captured vars list
David Chase [Wed, 30 Mar 2016 18:14:00 +0000 (14:14 -0400)]
cmd/compile: ignore OXXX nodes in closure captured vars list

Added a debug flag "-d closure" to explain compilation of
closures (should this be done some other way? Should we
rewrite the "-m" flag to "-d escapes"?)  Used this to
discover that cause was an OXXX node in the captured vars
list, and in turn noticed that OXXX nodes are explicitly
ignored in all other processing of captured variables.

Couldn't figure out a reproducer, did verify that this OXXX
was not caused by an unnamed return value (which is one use
of these).  Verified lack of heap allocation by examining -S
output.

Assembly:
(runtime/mgc.go:1371) PCDATA $0, $2
(runtime/mgc.go:1371) CALL "".notewakeup(SB)
(runtime/mgc.go:1377) LEAQ "".gcBgMarkWorker.func1·f(SB), AX
(runtime/mgc.go:1404) MOVQ AX, (SP)
(runtime/mgc.go:1404) MOVQ "".autotmp_2242+88(SP), CX
(runtime/mgc.go:1404) MOVQ CX, 8(SP)
(runtime/mgc.go:1404) LEAQ go.string."GC worker (idle)"(SB), AX
(runtime/mgc.go:1404) MOVQ AX, 16(SP)
(runtime/mgc.go:1404) MOVQ $16, 24(SP)
(runtime/mgc.go:1404) MOVB $20, 32(SP)
(runtime/mgc.go:1404) MOVQ $0, 40(SP)
(runtime/mgc.go:1404) PCDATA $0, $2
(runtime/mgc.go:1404) CALL "".gopark(SB)

Added a check for compiling_runtime to ensure that this is
caught in the future.  Added a test to test the check.
Verified that 1.5.3 did NOT reject the test case when
compiled with -+ flag, so this is not a recently added bug.

Cause of bug is two-part -- there was no leaking closure
detection ever, and instead it relied on capture-of-variables
to trigger compiling_runtime test, but closures improved in
1.5.3 so that mere capture of a value did not also capture
the variable, which thus allowed closures to escape, as well
as this case where the escape was spurious.  In
fixedbugs/issue14999.go, compare messages for f and g;
1.5.3 would reject g, but not f.  1.4 rejects both because
1.4 heap-allocates parameter x for both.

Fixes #14999.

Change-Id: I40bcdd27056810628e96763a44f2acddd503aee1
Reviewed-on: https://go-review.googlesource.com/21322
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
9 years agoall: use bytes.Equal, bytes.Contains and strings.Contains
Dominik Honnef [Fri, 1 Apr 2016 01:49:43 +0000 (03:49 +0200)]
all: use bytes.Equal, bytes.Contains and strings.Contains

Change-Id: Iba82a5bd3846f7ab038cc10ec72ff6bcd2c0b484
Reviewed-on: https://go-review.googlesource.com/21377
Run-TryBot: Dave Cheney <dave@cheney.net>
Reviewed-by: Dave Cheney <dave@cheney.net>
9 years agocrypto/x509: fix build break on darwin/{arm,arm64}
Dave Cheney [Fri, 1 Apr 2016 00:49:25 +0000 (11:49 +1100)]
crypto/x509: fix build break on darwin/{arm,arm64}

Fix build error on darwin/{arm,arm64} caused by a62ae9f6.

Change-Id: I7da4d147a529b11b3e71352a9276a57595ec4b05
Reviewed-on: https://go-review.googlesource.com/21356
Run-TryBot: Dave Cheney <dave@cheney.net>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agonet/http/httputil: deprecate Client+ServerConn harder, use Deprecated syntax
Brad Fitzpatrick [Thu, 31 Mar 2016 07:41:54 +0000 (00:41 -0700)]
net/http/httputil: deprecate Client+ServerConn harder, use Deprecated syntax

Also cleans up return parameter stutter and missing periods.

Change-Id: I47f5c230227ddfd1b105d5e06842f89ffea50760
Reviewed-on: https://go-review.googlesource.com/21362
Reviewed-by: Andrew Gerrand <adg@golang.org>
9 years agonet/http: clean up the Client redirect code, document Body.Close rules more
Brad Fitzpatrick [Thu, 31 Mar 2016 12:36:20 +0000 (05:36 -0700)]
net/http: clean up the Client redirect code, document Body.Close rules more

Issue #8633 (and #9134) noted that we didn't document the rules about
closing the Response.Body when Client.Do returned both a non-nil
*Response and a non-nil error (which can only happen when the user's
CheckRedirect returns an error).

In the process of investigating, I cleaned this code up a bunch, but
no user-visible behavior should have changed, except perhaps some
better error messages in some cases.

It turns out it's always been the case that when a CheckRedirect error
occurs, the Response.Body is already closed. Document that.

And the new code makes that more obvious too.

Fixes #8633

Change-Id: Ibc40cc786ad7fc4e0cf470d66bb559c3b931684d
Reviewed-on: https://go-review.googlesource.com/21364
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
9 years agocmd/compile: s.f aliases itself
Ian Lance Taylor [Thu, 31 Mar 2016 20:58:33 +0000 (13:58 -0700)]
cmd/compile: s.f aliases itself

The change in 20907 fixed varexpr but broke aliased.  After that change,
a reference to a field in a struct would not be seen as aliasing itself.
Before that change, it would, but only because all fields in a struct
aliased everything.

This CL changes the compiler to consider all references to a field as
aliasing all other fields in that struct.  This is imperfect--a
reference to one field does not alias another field--but is a simple fix
for the immediate problem.  A better fix would require tracking the
specific fields as well.

Fixes #15042.

Change-Id: I5c95c0dd7b0699e53022fce9bae2e8f50d6d1d04
Reviewed-on: https://go-review.googlesource.com/21390
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
9 years agonet/http/httputil: make ReverseProxy return 502 Bad Gateway instead of 500
Brad Fitzpatrick [Thu, 31 Mar 2016 11:03:57 +0000 (04:03 -0700)]
net/http/httputil: make ReverseProxy return 502 Bad Gateway instead of 500

... when the backend can't be reached.

Fixes #9864

Change-Id: I9e4814824943060e0101b6236d6dfd8d7822b4c7
Reviewed-on: https://go-review.googlesource.com/21363
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agonet/http: update bundled http2
Brad Fitzpatrick [Thu, 31 Mar 2016 12:49:23 +0000 (05:49 -0700)]
net/http: update bundled http2

Updates x/net/http2 to git rev 31df19d6 for changes since Go 1.6.

The main change was https://go-review.googlesource.com/19726 (move
merging of HEADERS and CONTINUATION into Framer), but there were a few
garbage reduction changes too.

Change-Id: I882443d20749f8638f637a2835efe92538c95d31
Reviewed-on: https://go-review.googlesource.com/21365
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
9 years agonet/http: limit Transport's reading of response header bytes from servers
Brad Fitzpatrick [Thu, 31 Mar 2016 07:06:27 +0000 (00:06 -0700)]
net/http: limit Transport's reading of response header bytes from servers

The default is 10MB, like http2, but can be configured with a new
field http.Transport.MaxResponseHeaderBytes.

Fixes #9115

Change-Id: I01808ac631ce4794ef2b0dfc391ed51cf951ceb1
Reviewed-on: https://go-review.googlesource.com/21329
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
9 years agocmd/compile: constant fold ANDs.
Alexandru Moșoi [Thu, 31 Mar 2016 22:54:13 +0000 (00:54 +0200)]
cmd/compile: constant fold ANDs.

ANDQConst show up occassionally because of right shifting lowering.
ORs and XORs are already folded properly during generic.

Change-Id: I2f9134679555029c641264ce5333d70e167c65f7
Reviewed-on: https://go-review.googlesource.com/21375
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: combine SHLQ into loads and stores
Alexandru Moșoi [Thu, 31 Mar 2016 22:33:29 +0000 (00:33 +0200)]
cmd/compile: combine SHLQ into loads and stores

Very common, cuts about 70k from pkg/tools/linux_amd64/* binaries.

Change-Id: Ied0c049e56e56a56810c781435d79027fbcaf274
Reviewed-on: https://go-review.googlesource.com/21374
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>

9 years agocmd/link: Replace fmt.Sprintf with filepath.Join
Christopher Nelson [Thu, 31 Mar 2016 20:22:54 +0000 (16:22 -0400)]
cmd/link: Replace fmt.Sprintf with filepath.Join

In a number of places the code was joining filepaths explicitly with
"/", instead of using filepath.Join. This may cause problems on Windows
(or other) platforms.

This is in support of https://go-review.googlesource.com/#/c/18057

Change-Id: Ieb1334f35ddb2e125be690afcdadff8d7b0ace10
Reviewed-on: https://go-review.googlesource.com/21369
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocompress/flate: make Reader.Read return io.EOF eagerly
Joe Tsai [Wed, 30 Mar 2016 05:04:03 +0000 (22:04 -0700)]
compress/flate: make Reader.Read return io.EOF eagerly

Rather than checking the block final bit on the next invocation
of nextBlock, we check it at the termination of the current block.
This ensures that we return (n, io.EOF) instead of (0, io.EOF)
more frequently for most streams.

However, there are certain situations where an eager io.EOF is not done:
1) We previously returned from Read because the write buffer of the internal
dictionary was full, and it just so happens that there is no more data
remaining in the stream.
2) There exists a [non-final, empty, raw block] after all blocks that
actually contain uncompressed data. We cannot return io.EOF eagerly here
since it would break flushing semantics.

Both situations happen infrequently, but it is still important to note that
this change does *not* guarantee that flate will *always* return (n, io.EOF).

Furthermore, this CL makes no changes to the pattern of ReadByte calls
to the underlying io.ByteReader.

Below is the motivation for this change, pulling the text from
@bradfitz's CL/21290:

net/http and other things work better when io.Reader implementations
return (n, io.EOF) at the end, instead of (n, nil) followed by (0,
io.EOF). Both are legal, but the standard library has been moving
towards n+io.EOF.

An investigation of net/http connection re-use in
https://github.com/google/go-github/pull/317 revealed that with gzip
compression + http/1.1 chunking, the net/http package was not
automatically reusing the underlying TCP connections when the final
EOF bytes were already read off the wire. The net/http package only
reuses the connection if the underlying Readers (many of them nested
in this case) all eagerly return io.EOF.

Previous related CLs:
    https://golang.org/cl/76400046 - tls.Reader
    https://golang.org/cl/58240043 - http chunked reader

In addition to net/http, this behavior also helps things like
ioutil.ReadAll (see comments about performance improvements in
https://codereview.appspot.com/49570044)

Updates #14867
Updates google/go-github#317

Change-Id: I637c45552efb561d34b13ed918b73c660f668378
Reviewed-on: https://go-review.googlesource.com/21302
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: better job of naming compound types
Keith Randall [Mon, 28 Mar 2016 18:25:17 +0000 (11:25 -0700)]
cmd/compile: better job of naming compound types

Compound AUTO types weren't named previously.  That was because live
variable analysis (plive.go) doesn't handle spilling to compound types.
It can't handle them because there is no valid place to put VARDEFs when
regalloc is spilling compound types.

compound types = multiword builtin types: complex, string, slice, and
interface.

Instead, we split named AUTOs into individual one-word variables.  For
example, a string s gets split into a byte ptr s.ptr and an integer
s.len.  Those two variables can be spilled to / restored from
independently.  As a result, live variable analysis can handle them
because they are one-word objects.

This CL will change how AUTOs are described in DWARF information.
Consider the code:

func f(s string, i int) int {
    x := s[i:i+5]
    g()
    return lookup(x)
}

The old compiler would spill x to two consecutive slots on the stack,
both named x (at offsets 0 and 8).  The new compiler spills the pointer
of x to a slot named x.ptr.  It doesn't spill x.len at all, as it is a
constant (5) and can be rematerialized for the call to lookup.

So compound objects may not be spilled in their entirety, and even if
they are they won't necessarily be contiguous.  Such is the price of
optimization.

Re-enable live variable analysis tests.  One test remains disabled, it
fails because of #14904.

Change-Id: I8ef2b5ab91e43a0d2136bfc231c05d100ec0b801
Reviewed-on: https://go-review.googlesource.com/21233
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
9 years agonet/http, net/http/httputil: rename lk to mu
Brad Fitzpatrick [Thu, 31 Mar 2016 07:31:58 +0000 (00:31 -0700)]
net/http, net/http/httputil: rename lk to mu

The conventional name for a sync.Mutex is "mu".

These "lk" names date back to a time before conventions.

Change-Id: Iee57f9f4423d04269e1125b5d82455c453aac26f
Reviewed-on: https://go-review.googlesource.com/21361
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: don't put SP in index slot
Keith Randall [Thu, 31 Mar 2016 21:09:04 +0000 (14:09 -0700)]
cmd/compile: don't put SP in index slot

For idx1 ops, SP can appear in the index slot.
Swap SP into the base register slot so we can encode
the instruction.

Fixes #15053

Change-Id: I19000cc9d6c86c7611743481e6e2cb78b1ef04eb
Reviewed-on: https://go-review.googlesource.com/21384
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: extend prove pass to handle constant comparisons
Keith Randall [Wed, 23 Mar 2016 17:20:44 +0000 (10:20 -0700)]
cmd/compile: extend prove pass to handle constant comparisons

Find comparisons to constants and propagate that information
down the dominator tree.  Use it to resolve other constant
comparisons on the same variable.

So if we know x >= 7, then a x > 4 condition must return true.

This change allows us to use "_ = b[7]" hints to eliminate bounds checks.

Fixes #14900

Change-Id: Idbf230bd5b7da43de3ecb48706e21cf01bf812f7
Reviewed-on: https://go-review.googlesource.com/21008
Reviewed-by: Alexandru Moșoi <alexandru@mosoi.ro>
9 years agohash/crc64: Add tests for ECMA polynomial
Ilya Tocar [Wed, 30 Mar 2016 14:54:11 +0000 (17:54 +0300)]
hash/crc64: Add tests for ECMA polynomial

Currently we test crc64 only with ISO polynomial.

Change-Id: Ibc5e202db3b960369cbbb18e31eb0fea07b54dba
Reviewed-on: https://go-review.googlesource.com/21309
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: remove Node.Nointerface field
Matthew Dempsky [Thu, 31 Mar 2016 18:42:35 +0000 (11:42 -0700)]
cmd/compile: remove Node.Nointerface field

We already keep the entire pragma bitset in n.Func.Pragma, so there's
no need to track Nointerface separately.

Passes toolstash -cmp.

Change-Id: Ic027ece477fcf63b0c1df128a08b89ef0f34fd58
Reviewed-on: https://go-review.googlesource.com/21381
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: stop generating garbage when checking map key types
Matthew Dempsky [Thu, 31 Mar 2016 19:30:20 +0000 (12:30 -0700)]
cmd/compile: stop generating garbage when checking map key types

Change-Id: Ib500ee92ae1a3d15f7c9f3f46d238b75184b4304
Reviewed-on: https://go-review.googlesource.com/21382
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: dump stack trace in Fatalf during development
Josh Bleecher Snyder [Thu, 31 Mar 2016 16:29:32 +0000 (09:29 -0700)]
cmd/compile: dump stack trace in Fatalf during development

See discussion in #15029.

Change-Id: I5cc8be5737ddb7c1f5e4a6cd92cf557af45e961d
Reviewed-on: https://go-review.googlesource.com/21347
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: include pkgPath on all struct types
David Crawshaw [Wed, 30 Mar 2016 15:15:01 +0000 (11:15 -0400)]
cmd/compile: include pkgPath on all struct types

Fixes #15026.

Change-Id: I61ed71152b99973270d79264d1e8f466f7343c02
Reviewed-on: https://go-review.googlesource.com/21286
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
9 years agocmd/compile: add sliceBound
Josh Bleecher Snyder [Thu, 31 Mar 2016 16:29:39 +0000 (09:29 -0700)]
cmd/compile: add sliceBound

Add a constant for the magic -1 for slice bounds.
Use it.
Enforce more aggressively that bounds must be
slice, ddd, or non-negative.
Remove ad hoc check in plive.go.
Check bounds before constructing an array type
when typechecking.

All changes are manual.

Passes toolstash -cmp.

Change-Id: I9fd9cc789d7d4b4eea3b30b24037a254d3788add
Reviewed-on: https://go-review.googlesource.com/21348
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
9 years agocmd/compile: fix build
Keith Randall [Thu, 31 Mar 2016 17:57:48 +0000 (10:57 -0700)]
cmd/compile: fix build

Pushed from an old client by mistake.  These are the
missing changes.

Change-Id: Ia8d61c5c0bde907369366ea9ea98711823342803
Reviewed-on: https://go-review.googlesource.com/21349
Reviewed-by: Todd Neal <todd@tneal.org>
9 years agocmd/compile: Add more idx1 load/store instructions
Keith Randall [Thu, 31 Mar 2016 16:34:35 +0000 (09:34 -0700)]
cmd/compile: Add more idx1 load/store instructions

Helpful for indexed loads and stores when the stride is not equal to
the size being loaded/stored.

Update #7927

Change-Id: I8714dd4c7b18a96a611bf5647ee21f753d723945
Reviewed-on: https://go-review.googlesource.com/21346
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
9 years agocmd/compile: place combined loads at the location of the last byte load
Keith Randall [Tue, 29 Mar 2016 04:45:33 +0000 (21:45 -0700)]
cmd/compile: place combined loads at the location of the last byte load

We need to make sure all the bounds checks pass before issuing
a load which combines several others.  We do this by issuing the
combined load at the last load's block, where "last" = closest to
the leaf of the dominator tree.

Fixes #15002

Change-Id: I7358116db1e039a072c12c0a73d861f3815d72af
Reviewed-on: https://go-review.googlesource.com/21246
Reviewed-by: Todd Neal <todd@tneal.org>
9 years agocmd/compile: encapsulate Type.Nname
Josh Bleecher Snyder [Wed, 30 Mar 2016 23:59:53 +0000 (16:59 -0700)]
cmd/compile: encapsulate Type.Nname

Generated by eg, manually fixed up.

I’m not thrilled about having a setter,
but given the variety of contexts in which this
gets fiddled with, it is the cleanest
available alternative.

Change-Id: Ibdf23e638fe0bdabded014c9e59d557fab8c955f
Reviewed-on: https://go-review.googlesource.com/21341
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
9 years agonet/http/httptest: clean up unnecessary goroutine
Brad Fitzpatrick [Thu, 31 Mar 2016 07:28:10 +0000 (00:28 -0700)]
net/http/httptest: clean up unnecessary goroutine

Finishes cleanup which was too late to do when discovered during the
Go 1.6 cycle.

Fixes #14291

Change-Id: Idc69fadbba10baf246318a22b366709eff088a75
Reviewed-on: https://go-review.googlesource.com/21360
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
9 years agocmd/compile, runtime: fix pedantic int->string conversions
Matthew Dempsky [Thu, 31 Mar 2016 09:04:12 +0000 (02:04 -0700)]
cmd/compile, runtime: fix pedantic int->string conversions

Previously, cmd/compile rejected constant int->string conversions if
the integer value did not fit into an "int" value. Also, runtime
incorrectly truncated 64-bit values to 32-bit before checking if
they're a valid Unicode code point. According to the Go spec, both of
these cases should instead yield "\uFFFD".

Fixes #15039.

Change-Id: I3c8a3ad9a0780c0a8dc1911386a523800fec9764
Reviewed-on: https://go-review.googlesource.com/21344
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agonet/mail: throw error when multiple addresses are given to ParseAddress
Hiroshi Ioka [Thu, 3 Mar 2016 17:54:51 +0000 (02:54 +0900)]
net/mail: throw error when multiple addresses are given to ParseAddress

Fixes #14610

Change-Id: I3e57dd60b531c1495ea3bc37ef707a1e4e644baa
Reviewed-on: https://go-review.googlesource.com/20180
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocrypto/x509: add SystemCertPool, refactor system cert pool loading
Brad Fitzpatrick [Wed, 30 Mar 2016 05:41:18 +0000 (16:41 +1100)]
crypto/x509: add SystemCertPool, refactor system cert pool loading

This exports the system cert pool.

The system cert loading was refactored to let it be run multiple times
(so callers get a copy, and can't mutate global state), and also to
not discard errors.

SystemCertPool returns an error on Windows. Maybe it's fixable later,
but so far we haven't used it, since the system verifies TLS.

Fixes #13335

Change-Id: I3dfb4656a373f241bae8529076d24c5f532f113c
Reviewed-on: https://go-review.googlesource.com/21293
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
9 years agocmd/link: remove -H elf flag
Matthew Dempsky [Thu, 31 Mar 2016 05:56:00 +0000 (22:56 -0700)]
cmd/link: remove -H elf flag

We create appropriate ELF files automatically based on GOOS. There's
no point in supporting -H elf flag, particularly since we need to emit
different flavors of ELF depending on GOOS anyway.

If that weren't reason enough, -H elf appears to be broken since at
least Go 1.4. At least I wasn't able to find a way to make use of it.

As best I can tell digging through commit history, -H elf is just an
artifact leftover from Plan 9's 6l linker.

Change-Id: I7393caaadbc60107bbd6bc99b976a4f4fe6b5451
Reviewed-on: https://go-review.googlesource.com/21343
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agotest/fixedbugs: add test for divide by zero being optimized away
Brad Fitzpatrick [Thu, 31 Mar 2016 02:51:17 +0000 (13:51 +1100)]
test/fixedbugs: add test for divide by zero being optimized away

This only tests amd64 because it's currently broken on non-SSA
backends.

Fixes #8613

Change-Id: I6bc501c81c395e533bb9c7335789750e0c6b7a8f
Reviewed-on: https://go-review.googlesource.com/21325
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agonet/http: allow Handlers to handle http2 upgrade PRI requests
Brad Fitzpatrick [Thu, 31 Mar 2016 05:11:41 +0000 (22:11 -0700)]
net/http: allow Handlers to handle http2 upgrade PRI requests

The http2 spec defines a magic string which initates an http2 session:

    "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"

It was intentionally chosen to kinda look like an HTTP request, but
just different enough to break things not ready for it. This change
makes Go ready for it.

Notably: Go now accepts the request header (the prefix "PRI *
HTTP/2.0\r\n\r\n") as a valid request, even though it doesn't have a
Host header. But we now mark it as "Connection: close" and teach the
Server to never read a second request from the connection once that's
seen. If the http.Handler wants to deal with the upgrade, it has to
hijack the request, read out the "body", compare it against
"SM\r\n\r\n", and then speak http2. One of the new tests demonstrates
that hijacking.

Fixes #14451
Updates #14141 (h2c)

Change-Id: Ib46142f31c55be7d00c56fa2624ec8a232e00c43
Reviewed-on: https://go-review.googlesource.com/21327
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agonet/http: validate transmitted header fields
Brad Fitzpatrick [Thu, 31 Mar 2016 03:33:46 +0000 (14:33 +1100)]
net/http: validate transmitted header fields

This makes sure the net/http package never attempts to transmit a
bogus header field key or value and instead fails fast with an error
to the user, rather than relying on the server to maybe return an
error.

It's still possible to use x/net/http2.Transport directly to send
bogus stuff. This change only stops h1 & h2 usage via the net/http
package. A future change will update x/net/http2.

This change also moves some code from request.go to lex.go, which in a
separate future change should be moved so it can be shared with http2
to reduce code bloat.

Updates #14048

Change-Id: I0a44ae1ab357fbfcbe037aa4b5d50669a87f2856
Reviewed-on: https://go-review.googlesource.com/21326
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/pprof: use DWARF info to lookup unknown PC addresses
Ian Lance Taylor [Thu, 24 Mar 2016 00:10:18 +0000 (17:10 -0700)]
cmd/pprof: use DWARF info to lookup unknown PC addresses

Test to follow in a separate CL that arranges for the runtime package to
store non-Go addresses in a CPU profile.

Change-Id: I33ce1d66b77340b1e62b54505fc9b1abcec108a9
Reviewed-on: https://go-review.googlesource.com/21055
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>

9 years agoruntime: don't use REP;MOVSB if CPUID doesn't say it is fast
Keith Randall [Wed, 30 Mar 2016 04:25:33 +0000 (21:25 -0700)]
runtime: don't use REP;MOVSB if CPUID doesn't say it is fast

Only use REP;MOVSB if:
 1) The CPUID flag says it is fast, and
 2) The pointers are unaligned
Otherwise, use REP;MOVSQ.

Update #14630

Change-Id: I946b28b87880c08e5eed1ce2945016466c89db66
Reviewed-on: https://go-review.googlesource.com/21300
Reviewed-by: Nigel Tao <nigeltao@golang.org>
9 years agocmd/compile/internal/gc: avoid append when building Type fields
Dave Cheney [Thu, 31 Mar 2016 00:58:19 +0000 (11:58 +1100)]
cmd/compile/internal/gc: avoid append when building Type fields

As a followup to CL 21296, avoid append operations when constructing the
fields of a Type if the length is known beforehand

This also includes some small scoping driveby cleanups, and a change to
tointerface0 to avoid iterating over the field list twice.

compilebench shows a very small reduction in allocations.

 name      old time/op    new time/op    delta
Template     364ms ± 5%     363ms ± 4%    ~     (p=0.945 n=20+19)
Unicode      182ms ±11%     185ms ±12%    ~     (p=0.445 n=20+20)
GoTypes      1.14s ± 2%     1.14s ± 3%    ~     (p=0.221 n=20+20)
Compiler     5.85s ± 2%     5.84s ± 2%    ~     (p=0.369 n=20+20)

name      old alloc/op   new alloc/op   delta
Template    56.7MB ± 0%    56.7MB ± 0%  -0.04%  (p=0.000 n=20+20)
Unicode     38.3MB ± 0%    38.3MB ± 0%    ~     (p=0.728 n=20+19)
GoTypes      180MB ± 0%     180MB ± 0%  -0.02%  (p=0.000 n=20+20)
Compiler     812MB ± 0%     812MB ± 0%  -0.02%  (p=0.000 n=19+20)

name      old allocs/op  new allocs/op  delta
Template      482k ± 0%      480k ± 0%  -0.34%  (p=0.000 n=20+20)
Unicode       377k ± 0%      377k ± 0%  -0.04%  (p=0.010 n=20+20)
GoTypes      1.36M ± 0%     1.35M ± 0%  -0.24%  (p=0.000 n=20+20)
Compiler     5.47M ± 0%     5.46M ± 0%  -0.11%  (p=0.000 n=20+18)

Change-Id: Ibb4c40229fa3816acd8de98ba41d1571a2aabacf
Reviewed-on: https://go-review.googlesource.com/21352
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Dave Cheney <dave@cheney.net>

9 years agocmd/compile/internal/gc: don't let the argument to Fields.Set escape
Dave Cheney [Wed, 30 Mar 2016 09:37:51 +0000 (20:37 +1100)]
cmd/compile/internal/gc: don't let the argument to Fields.Set escape

Apply Robert's optimisation from CL 21241 to Type.Fields. The results
are less impressive, possibly because of the makeup of the test data.

name      old time/op    new time/op    delta
Template     365ms ± 5%     365ms ± 3%    ~     (p=0.888 n=20+16)
Unicode      182ms ±10%     180ms ± 9%    ~     (p=0.883 n=20+20)
GoTypes      1.14s ± 2%     1.13s ± 3%    ~     (p=0.096 n=20+20)
Compiler     5.74s ± 1%     5.76s ± 2%    ~     (p=0.369 n=20+20)

name      old alloc/op   new alloc/op   delta
Template    56.8MB ± 0%    56.7MB ± 0%  -0.15%  (p=0.000 n=19+20)
Unicode     38.3MB ± 0%    38.3MB ± 0%  -0.02%  (p=0.006 n=20+19)
GoTypes      180MB ± 0%     180MB ± 0%  -0.13%  (p=0.000 n=20+20)
Compiler     805MB ± 0%     804MB ± 0%  -0.05%  (p=0.000 n=20+20)

name      old allocs/op  new allocs/op  delta
Template      485k ± 0%      482k ± 0%  -0.54%  (p=0.000 n=19+20)
Unicode       377k ± 0%      377k ± 0%  -0.05%  (p=0.005 n=20+20)
GoTypes      1.37M ± 0%     1.36M ± 0%  -0.53%  (p=0.000 n=20+19)
Compiler     5.42M ± 0%     5.41M ± 0%  -0.21%  (p=0.000 n=20+20)

Change-Id: I6782659fadd605ce9931bf5c737c7058b96a29eb
Reviewed-on: https://go-review.googlesource.com/21296
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agoimage/jpeg: reconstruct progressive images even if incomplete.
Nigel Tao [Thu, 24 Mar 2016 04:34:45 +0000 (15:34 +1100)]
image/jpeg: reconstruct progressive images even if incomplete.

Fixes #14522.

As I said on that issue:

----
This is a progressive JPEG image. There are two dimensions of
progressivity: spectral selection (variables zs and ze in scan.go,
ranging in [0, 63]) and successive approximation (variables ah and al in
scan.go, ranging in [0, 8), from LSB to MSB, although ah=0 implicitly
means ah=8).

For this particular image, there are three components, and the SOS
markers contain this progression:

zs, ze, ah, al:  0  0 0 0 components: 0, 1, 2
zs, ze, ah, al:  1 63 0 0 components: 1
zs, ze, ah, al:  1 63 0 0 components: 2
zs, ze, ah, al:  1 63 0 2 components: 0
zs, ze, ah, al:  1 10 2 1 components: 0
zs, ze, ah, al: 11 63 2 1 components: 0
zs, ze, ah, al:  1 10 1 0 components: 0

The combination of all of these is complete (i.e. spectra 0 to 63 and
bits 8 exclusive to 0) for components 1 and 2, but it is incomplete for
component 0 (the luma component). In particular, there is no data for
component 0, spectra 11 to 63 and bits 1 exclusive to 0.

The image/jpeg code, as of Go 1.6, waits until both dimensions are
complete before performing the de-quantization, IDCT and copy to an
*image.YCbCr. This is the "if zigEnd != blockSize-1 || al != 0 { ...
continue }" code and associated commentary in scan.go.

Almost all progressive JPEG images end up complete in both dimensions
for all components, but this particular image is incomplete for
component 0, so the Go code never writes anything to the Y values of the
resultant *image.YCbCr, which is why the broken output is so dark (but
still looks recognizable in terms of red and blue hues).

My reading of the ITU T.81 JPEG specification (Annex G) doesn't
explicitly say that this is a valid image, but it also doesn't rule it
out.

In any case, the fix is, for progressive JPEG images, to always
reconstruct the decoded blocks (by performing the de-quantization, IDCT
and copy to an *image.YCbCr), regardless of whether or not they end up
complete. Note that, in Go, the jpeg.Decode function does not return
until the entire image is decoded, so we still only want to reconstruct
each block once, not once per SOS (Start Of Scan) marker.
----

A test image was also added, based on video-001.progressive.jpeg. When
decoding that image, inserting a

println("nComp, zs, ze, ah, al:", nComp, zigStart, zigEnd, ah, al)

into decoder.processSOS in scan.go prints:

nComp, zs, ze, ah, al: 3 0 0 0 1
nComp, zs, ze, ah, al: 1 1 5 0 2
nComp, zs, ze, ah, al: 1 1 63 0 1
nComp, zs, ze, ah, al: 1 1 63 0 1
nComp, zs, ze, ah, al: 1 6 63 0 2
nComp, zs, ze, ah, al: 1 1 63 2 1
nComp, zs, ze, ah, al: 3 0 0 1 0
nComp, zs, ze, ah, al: 1 1 63 1 0
nComp, zs, ze, ah, al: 1 1 63 1 0
nComp, zs, ze, ah, al: 1 1 63 1 0

In other words, video-001.progressive.jpeg contains 10 different scans.
This little program below drops half of them (remembering to keep the
"\xff\xd9" End of Image marker):

----
package main

import (
"bytes"
"io/ioutil"
"log"
)

func main() {
sos := []byte{0xff, 0xda}
eoi := []byte{0xff, 0xd9}

src, err := ioutil.ReadFile("video-001.progressive.jpeg")
if err != nil {
log.Fatal(err)
}
b := bytes.Split(src, sos)
println(len(b)) // Prints 11.
dst := bytes.Join(b[:5], sos)
dst = append(dst, eoi...)
if err := ioutil.WriteFile("video-001.progressive.truncated.jpeg", dst, 0666); err != nil {
log.Fatal(err)
}
}
----

The video-001.progressive.truncated.jpeg was converted to png via
libjpeg and ImageMagick:

djpeg -nosmooth video-001.progressive.truncated.jpeg > tmp.tga
convert tmp.tga video-001.progressive.truncated.png
rm tmp.tga

Change-Id: I72b20cd4fb6746d36d8d4d587f891fb3bc641f84
Reviewed-on: https://go-review.googlesource.com/21062
Reviewed-by: Rob Pike <r@golang.org>
9 years agocmd/compile/internal/gc: don't iterate over field list twice
Dave Cheney [Wed, 30 Mar 2016 23:30:04 +0000 (10:30 +1100)]
cmd/compile/internal/gc: don't iterate over field list twice

In tostruct0 and tofunargs we take a list of nodes, transform them into
a slice of Fields, set the fields on a type, then use the IterFields
iterator to iterate over the list again to see if any of them are
broken.

As we know the slice of fielde-we just created it-we can combine these two
interations into one pass over the fields.

Change-Id: I8b04c90fb32fd6c3b1752cfc607128a634ee06c5
Reviewed-on: https://go-review.googlesource.com/21350
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: use t.IsFoo() instead of Isfoo[t.Etype]
Matthew Dempsky [Wed, 30 Mar 2016 22:09:25 +0000 (15:09 -0700)]
cmd/compile: use t.IsFoo() instead of Isfoo[t.Etype]

This allows us to get rid of Isptr and Issigned. Still some code to
clean up for Isint, Isfloat, and Iscomplex.

CL produced mechanically using gofmt -w -r.

Passes toolstash -cmp.

Change-Id: If4f807bb7f2b357288d2547be2380eb511875786
Reviewed-on: https://go-review.googlesource.com/21339
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
9 years agocmd/compile: use t.IsFoo() instead of t.Etype == TFOO
Matthew Dempsky [Wed, 30 Mar 2016 21:56:08 +0000 (14:56 -0700)]
cmd/compile: use t.IsFoo() instead of t.Etype == TFOO

CL produced mechanically using gofmt -w -r.

Passes toolstash -cmp.

Change-Id: Ib2e8710ebd844e2149125b41c335b71a02fcab53
Reviewed-on: https://go-review.googlesource.com/21338
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: generalize strength reduction of mulq
Alexandru Moșoi [Thu, 24 Mar 2016 21:46:37 +0000 (22:46 +0100)]
cmd/compile: generalize strength reduction of mulq

* This is an improved version of an earlier patch.
* Verified with gcc up to 100.
* Limited to two instructions based on costs from
https://gmplib.org/~tege/x86-timing.pdf

Change-Id: Ib7c37de6fd8e0ba554459b15c7409508cbcf6728
Reviewed-on: https://go-review.googlesource.com/21103
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: get rid of redundant Type helper functions
Matthew Dempsky [Wed, 30 Mar 2016 21:45:47 +0000 (14:45 -0700)]
cmd/compile: get rid of redundant Type helper functions

Replace Isfixedarray, Isslice, and Isinter with the IsArray, IsSlice,
and IsInterface methods added for SSA. Rewrite performed mechanically
using gofmt -w -r "Isfoo(t) -> t.IsFoo()".

Because the IsFoo methods panic when given a nil pointer, a handful of
call sites had to be modified to check for nil Type values. These
aren't strictly necessary, because nil Type values should only occur
in invalid Go source programs, so it would be okay if we panicked on
them and gave up type checking the rest of the package. However, there
are a couple regress tests that expect we continue, so add checks to
keep those tests passing. (See #15029.)

Passes toolstash -cmp.

Change-Id: I511c6ac4cfdf3f9cbdb3e52a5fa91b6d09d82f80
Reviewed-on: https://go-review.googlesource.com/21336
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: s/typeX/typX/
Josh Bleecher Snyder [Wed, 30 Mar 2016 17:59:29 +0000 (10:59 -0700)]
cmd/compile: s/typeX/typX/

Apparently I’m having a hard time following my
own naming scheme.

Change-Id: I99c801bef09fa65c1f0e8ecc2fba154a495e9c17
Reviewed-on: https://go-review.googlesource.com/21332
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>

9 years agocmd/compile: add Type.Elem
Josh Bleecher Snyder [Wed, 30 Mar 2016 17:57:47 +0000 (10:57 -0700)]
cmd/compile: add Type.Elem

This removes almost all direct access to
Type’s heavily overloaded Type field.

Mostly generated by eg, manually checked.

Significant manual changes:

* reflect.go's typPkg used Type indiscriminately.
  Use it only for specific etypes.
* gen.go's visitComponents contained a usage of Type
  with structs. Using Type for structs no longer
  occurs, and the Fatal contained therein has not triggered,
  so it has been axed.
* Scary code in cgen.go's cgen_slice is now explicitly scary.

Passes toolstash -cmp.

Change-Id: I2dbfb3c959da7ae239f964d83898c204affcabc6
Reviewed-on: https://go-review.googlesource.com/21331
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

9 years agocmd/compile: add typMap
Josh Bleecher Snyder [Wed, 30 Mar 2016 17:52:13 +0000 (10:52 -0700)]
cmd/compile: add typMap

Also, add two uses of Key and Val that I missed earlier.
As before, direct writes to Down and Type remain in bimport.

Change-Id: I487aa975926b30092db1ad74ace17994697117c1
Reviewed-on: https://go-review.googlesource.com/21330
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
9 years agocmd/compile: use inc/dec for bytes, too
Alexandru Moșoi [Wed, 30 Mar 2016 16:17:45 +0000 (18:17 +0200)]
cmd/compile: use inc/dec for bytes, too

Change-Id: Ib2890ab1983cbef7c1c1ee5a10204ba3ace19b53
Reviewed-on: https://go-review.googlesource.com/21312
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
9 years agointernal/testenv: prefer to find go binary in GOROOT
Brad Fitzpatrick [Wed, 30 Mar 2016 04:13:26 +0000 (15:13 +1100)]
internal/testenv: prefer to find go binary in GOROOT

Partial revert of https://golang.org/cl/20967 which
I can't reproduce and actually breaks me more.

Fixes #14901

Change-Id: I8cce443fbd95f5f6f2a5b6a4b9f2faab36167a12
Reviewed-on: https://go-review.googlesource.com/21292
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
9 years agocmd/compile: rename Type.IsPtr to Type.IsPtrShaped
Matthew Dempsky [Mon, 28 Mar 2016 17:55:44 +0000 (10:55 -0700)]
cmd/compile: rename Type.IsPtr to Type.IsPtrShaped

Previously, t.IsPtr() reported whether t was represented with a
pointer, but some of its callers expected it to report whether t is an
actual Go pointer. Resolve this by renaming t.IsPtr to t.IsPtrShaped
and adding a new t.IsPtr method to report Go pointer types.

Updated a couple callers in gc/ssa.go to use IsPtr instead of
IsPtrShaped.

Passes toolstash -cmp.

Updates #15028.

Change-Id: I0a8154b5822ad8a6ad296419126ad01a3d2a5dc5
Reviewed-on: https://go-review.googlesource.com/21232
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
9 years agocmd/compile: add typPtr
Josh Bleecher Snyder [Tue, 29 Mar 2016 17:18:43 +0000 (10:18 -0700)]
cmd/compile: add typPtr

Passes toolstash -cmp.

Change-Id: I721348ed2122b6a9cd87ad2041b6ee3bf6b2bbb5
Reviewed-on: https://go-review.googlesource.com/21306
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: add typWrapper and Type.Wrapped
Josh Bleecher Snyder [Tue, 29 Mar 2016 17:15:02 +0000 (10:15 -0700)]
cmd/compile: add typWrapper and Type.Wrapped

Passes toolstash -cmp.

Change-Id: I7dffd9bc5bab323590df6fb591bf1e73edf2e465
Reviewed-on: https://go-review.googlesource.com/21305
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: add typChan
Josh Bleecher Snyder [Tue, 29 Mar 2016 17:00:54 +0000 (10:00 -0700)]
cmd/compile: add typChan

Passes toolstash -cmp.

Change-Id: I2c71882f957c44047c7ac83c78236dcc3dfa15a1
Reviewed-on: https://go-review.googlesource.com/21304
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
9 years agocmd/compile: use IsSlice and IsArray instead of checking Bound
Josh Bleecher Snyder [Tue, 29 Mar 2016 16:14:19 +0000 (09:14 -0700)]
cmd/compile: use IsSlice and IsArray instead of checking Bound

Changes generated by eg and manually checked.

Isfixedarray, Isslice, and many other
Type-related functions in subr.go should
either be deleted or moved to type.go.
Later, though; the game now is cleanup via encapsulation.

Passes toolstash -cmp.

Change-Id: I83dd8816f6263b74367d23c2719a08c362e330f9
Reviewed-on: https://go-review.googlesource.com/21303
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
9 years agocmd/compile: ignore dead phis in fuse
Alexandru Moșoi [Wed, 30 Mar 2016 14:19:10 +0000 (16:19 +0200)]
cmd/compile: ignore dead phis in fuse

Happens occasionally for boolean phis was used as a control.

Change-Id: Ie0f2483e9004c1706751d8dfb25ee2e5106d917e
Reviewed-on: https://go-review.googlesource.com/21310
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
9 years agocompress/gzip: fix error handling in Read
Joe Tsai [Wed, 30 Mar 2016 06:37:59 +0000 (23:37 -0700)]
compress/gzip: fix error handling in Read

The Read logic should not assume that only (0, io.EOF) is returned
instead of (n, io.EOF) where n is positive.

The fix done here is very similar to the fix to compress/zlib
in CL/20292.

Change-Id: Icb76258cdcf8cfa386a60bab330fefde46fc071d
Reviewed-on: https://go-review.googlesource.com/21308
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>