Austin Clements [Tue, 24 Oct 2017 21:10:02 +0000 (17:10 -0400)]
cmd/compile: report typedslicecopy write barriers
Most write barrier calls are inserted by SSA, but copy and append are
lowered to runtime.typedslicecopy during walk. Fix these to set
Func.WBPos and emit the "write barrier" warning, as done for the write
barriers inserted by SSA. As part of this, we refactor setting WBPos
and emitting this warning into the frontend so it can be shared by
both walk and SSA.
Adam Langley [Sat, 14 Oct 2017 18:43:17 +0000 (11:43 -0700)]
crypto/{ecdsa,rsa}: rename argument to PrivateKey.Sign.
The crypto.Signer interface takes pre-hased messages for ECDSA and RSA,
but the argument in the implementations was called “msg”, not “digest”,
which is confusing.
This change renames them to help clarify the intended use.
Austin Clements [Sun, 22 Oct 2017 20:36:27 +0000 (16:36 -0400)]
cmd/compile: improve coverage of nowritebarrierrec check
The current go:nowritebarrierrec checker has two problems that limit
its coverage:
1. It doesn't understand that systemstack calls its argument, which
means there are several cases where we fail to detect prohibited write
barriers.
2. It only observes calls in the AST, so calls constructed during
lowering by SSA aren't followed.
This CL completely rewrites this checker to address these issues.
The current checker runs entirely after walk and uses visitBottomUp,
which introduces several problems for checking across systemstack.
First, visitBottomUp itself doesn't understand systemstack calls, so
the callee may be ordered after the caller, causing the checker to
fail to propagate constraints. Second, many systemstack calls are
passed a closure, which is quite difficult to resolve back to the
function definition after transformclosure and walk have run. Third,
visitBottomUp works exclusively on the AST, so it can't observe calls
created by SSA.
To address these problems, this commit splits the check into two
phases and rewrites it to use a call graph generated during SSA
lowering. The first phase runs before transformclosure/walk and simply
records systemstack arguments when they're easy to get. Then, it
modifies genssa to record static call edges at the point where we're
lowering to Progs (which is the latest point at which position
information is conveniently available). Finally, the second phase runs
after all functions have been lowered and uses a direct BFS walk of
the call graph (combining systemstack calls with static calls) to find
prohibited write barriers and construct nice error messages.
Austin Clements [Sun, 22 Oct 2017 22:10:08 +0000 (18:10 -0400)]
runtime: allow write barriers in gchelper
We're about to start tracking nowritebarrierrec through systemstack
calls, which detects that we're calling markroot (which has write
barriers) from gchelper, which is called from the scheduler during STW
apparently without a P.
But it turns out that func helpgc, which wakes up blocked Ms to run
gchelper, installs a P for gchelper to use. This means there *is* a P
when gchelper runs, so it is allowed to have write barriers. Tell the
compiler this by marking gchelper go:yeswritebarrierrec. Also,
document the call to gchelper so I don't have to spend another half a
day puzzling over how on earth this could possibly work before
discovering the spooky action-at-a-distance in helpgc.
Austin Clements [Sun, 22 Oct 2017 22:10:08 +0000 (18:10 -0400)]
runtime: eliminate write barriers from persistentalloc
We're about to start tracking nowritebarrierrec through systemstack
calls, which will reveal write barriers in persistentalloc prohibited
by various callers.
The pointers manipulated by persistentalloc are always to off-heap
memory, so this removes these write barriers statically by introducing
a new go:notinheap type to represent generic off-heap memory.
Austin Clements [Sun, 22 Oct 2017 22:10:08 +0000 (18:10 -0400)]
runtime: allow write barriers in startpanic_m
We're about to start tracking nowritebarrierrec through systemstack
calls, which will reveal write barriers in startpanic_m prohibited by
various callers.
We actually can allow write barriers here because the write barrier is
a no-op when we're panicking. Let the compiler know.
Austin Clements [Thu, 19 Oct 2017 23:57:46 +0000 (19:57 -0400)]
runtime: mark gcWork methods nowritebarrierrec
Currently most of these are marked go:nowritebarrier as a hint, but
it's actually important that these not invoke write barriers
recursively. The danger is that some gcWork method would invoke the
write barrier while the gcWork is in an inconsistent state and that
the write barrier would in turn invoke some other gcWork method, which
would crash or permanently corrupt the gcWork. Simply marking the
write barrier itself as go:nowritebarrierrec isn't sufficient to
prevent this if the write barrier doesn't use the outer method.
Thankfully, this doesn't cause any build failures, so we were getting
this right. :)
Austin Clements [Mon, 23 Oct 2017 01:37:05 +0000 (21:37 -0400)]
runtime: remove write barriers from newstack, gogo
Currently, newstack and gogo have write barriers for maintaining the
context register saved in g.sched.ctxt. This is troublesome, because
newstack can be called from go:nowritebarrierrec places that can't
allow write barriers. It happens to be benign because g.sched.ctxt
will always be nil on entry to newstack *and* it so happens the
incoming ctxt will also always be nil in these contexts (I
think/hope), but this is playing with fire. It's also desirable to
mark newstack go:nowritebarrierrec to prevent any other, non-benign
write barriers from creeping in, but we can't do that right now
because of this one write barrier.
Fix all of this by observing that g.sched.ctxt is really just a saved
live pointer register. Hence, we can shade it when we scan g's stack
and otherwise move it back and forth between the actual context
register and g.sched.ctxt without write barriers. This means we can
save it in morestack along with all of the other g.sched, eliminate
the save from newstack along with its troublesome write barrier, and
eliminate the shenanigans in gogo to invoke the write barrier when
restoring it.
Once we've done all of this, we can mark newstack
go:nowritebarrierrec.
Tobias Klauser [Sat, 28 Oct 2017 11:45:36 +0000 (13:45 +0200)]
.gitignore: ignore src/cmd/dist/dist
In case of a failed/cancelled build, src/cmd/dist/dist might be left in
place.
Change-Id: Id81b5d663476a880101a2eed54fa051c40b0b0bc
Reviewed-on: https://go-review.googlesource.com/74150 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alberto Donizetti [Mon, 9 Oct 2017 07:54:35 +0000 (09:54 +0200)]
cmd/go: add a test to ensure upx works on go binaries
On linux/amd64, for now.
Updates #16706
Change-Id: Ib8c89b6edc73fb88042c06873ff815d387491504
Reviewed-on: https://go-review.googlesource.com/69117
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Russ Cox [Fri, 27 Oct 2017 17:52:59 +0000 (13:52 -0400)]
api: update next.txt
Change-Id: Ib342468f3727be0cd6268ab824ad06d783ee7c94
Reviewed-on: https://go-review.googlesource.com/73993 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Russ Cox [Fri, 27 Oct 2017 20:48:02 +0000 (16:48 -0400)]
cmd/dist: translate /private/var to /var on darwin builders
This is ugly but needed on the builders, because they do not set
PWD/GOROOT consistently, and the new content-based staleness
understands that the setting of GOROOT influences the content in
the linker outputs.
Joshua Rubin [Fri, 27 Oct 2017 07:47:08 +0000 (01:47 -0600)]
net/http: fix minor leak in Header.WriteSubset
Header.WriteSubset uses a sync.Pool but wouldn't Put the sorter back in
the pool if there was an error writing to the io.Writer
I'm not really sure why the sorter is returned to begin with. The
comment says "for possible return to headerSorterCache".
This also doesn't address potential panics that might occur, but the
overhead of doing the Put in a defer would likely be too great.
Change-Id: If3c45a4c3e11f6ec65d187e25b63455b0142d4e3
Reviewed-on: https://go-review.googlesource.com/73910
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tom Bergan <tombergan@google.com>
Than McIntosh [Tue, 24 Oct 2017 20:08:46 +0000 (16:08 -0400)]
cmd/compile, cmd/link: support for DWARF file reference relocations
New relocation flavor R_DWARFFILEREF, to be applied to DWARF attribute
values that correspond to file references (ex: DW_AT_decl_file,
DW_AT_call_file). The LSym for this relocation is the file itself; the
linker replaces the relocation target with the index of the specified
file in the line table's file section.
Note: for testing purposes this patch changes the DWARF function
subprogram DIE abbrev to include DW_AT_decl_file (allowed by DWARF
but not especially useful) so as to have a way to test this
functionality. This attribute will be removed once there are other
file reference attributes (coming as part of inlining support).
Russ Cox [Fri, 27 Oct 2017 14:34:25 +0000 (10:34 -0400)]
cmd/dist: use latest heap, sort packages for compiler bootstrap
The compiler depends on the way heap and sort break ties
in some cases. Instead of trying to find them all, bundle
those packages into the bootstrap compiler builds.
The overall goal is that Go1.4 building cmd/compile during the
bootstrap process produces a semantically equivalent compiler
to cmd/compile compiling itself. After this CL, that property is true,
at least for the compiler compiling itself and the other tools.
Russ Cox [Fri, 27 Oct 2017 14:21:46 +0000 (10:21 -0400)]
sort: split post-Go1.4 code into its own file
This will let us build the latest sort when bootstrapping the compiler.
The compiler depends on the precise tie-breaks used by sort in
some cases, and it's easier to bring sort along than require checking
every sort call ever added to the compiler.
Than McIntosh [Thu, 26 Oct 2017 14:15:29 +0000 (10:15 -0400)]
cmd/link: don't pass -gdwarf-2 to external linker
Don't pass -gdwarf-2 to the external linker when external linkage is
requested. The Go compiler is now emitting DWARF version 4, so this
doesn't seem needed any more.
Fixes #22455
Change-Id: Ic4122c55e946619a266430f2d26f06d6803dd232
Reviewed-on: https://go-review.googlesource.com/73672 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox [Fri, 27 Oct 2017 02:08:23 +0000 (22:08 -0400)]
cmd/go: delay gcc/clang flag support checks
env.MkEnv was computing the full gcc command line to report as
$GOGCCFLAGS in "go env" output, which meant running gcc (or clang)
multiple times to discern which flags are available.
We also set $GOGCCFLAGS in the environment, but nothing actually uses
that as far as I can tell - it was always intended only for debugging.
Move GOGCCFLAGS to env.ExtraEnvVars, which displayed in "go env"
output but not set in child processes and not computed nearly as
often.
The effect is that trivial commands like "go help" or "go env GOARCH"
or "go tool -n compile" now run in about 0.01s instead of 0.1s,
because they no longer run gcc 4 times each.
go test -short cmd/go drops from 81s to 44s (and needs more trimming).
The $GOROOT/test suite drops from 92s to 33s, because the number of
gcc invocation drops from 13,336 to 0.
Overall, all.bash drops from 5m53s to 4m07s wall time.
Change-Id: Ia85abc89e1e2bb126b933aff3bf7c5f6c0984cd5
Reviewed-on: https://go-review.googlesource.com/73850
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Artyom Pervukhin [Sun, 22 Oct 2017 10:33:06 +0000 (13:33 +0300)]
image/draw, image/color: optimize hot path sqDiff function
Function sqDiff is called multiple times in the hot path (x, y loops) of
drawPaletted from the image/draw package; number of sqDiff calls is
between 4×width×height and 4×width×height×len(palette) for each
drawPaletted call.
Simplify this function by removing arguments comparison and relying
instead on signed to unsigned integer conversion rules and properties of
unsigned integer values operations guaranteed by the spec:
> For unsigned integer values, the operations +, -, *, and << are
> computed modulo 2n, where n is the bit width of the unsigned integer's
> type. Loosely speaking, these unsigned integer operations discard high
> bits upon overflow, and programs may rely on ``wrap around''.
image/gif package benchmark that depends on the code updated shows
throughput improvements:
name old time/op new time/op delta
QuantizedEncode-4 788ms ± 2% 468ms ± 9% -40.58% (p=0.000 n=9+10)
name old speed new speed delta
QuantizedEncode-4 1.56MB/s ± 2% 2.63MB/s ± 8% +68.47% (p=0.000 n=9+10)
Closes #22375.
Change-Id: Ic9a540e39ceb21e7741d308af1cfbe61b4ac347b
Reviewed-on: https://go-review.googlesource.com/72373 Reviewed-by: Nigel Tao <nigeltao@golang.org>
isharipo [Fri, 13 Oct 2017 12:37:16 +0000 (15:37 +0300)]
obj/x86: re-organize code to make codegen easier
1. Move AXXX constants (A-enumeration) from "a.out.go" to "aenum.go"
2. Move VEX-encoded optabs from "asm6.go" to "vex_optabs.go"
Also run "go generate" over aenum.go. This explains diff in "anames.go".
Initialization of opindex is split into 2 loops:
one for `vexOptab`, second for `optab`.
Rationale:
when VEX instructions are generated with current structure,
asm6.go is modified, which can lead to merge conflicts and
larger diffs than desired. Same for a.out.go.
This change makes x86avxgen usage possible:
https://go-review.googlesource.com/c/arch/+/66972
David Crawshaw [Sat, 21 Oct 2017 11:29:46 +0000 (07:29 -0400)]
cmd/link, plugin: always encode path
Both the linker and the plugin package were inconsistent
about when they applied the path encoding defined in
objabi.PathToPrefix. As a result, only some symbols from
a package path that required encoding were being found.
So always encoding the path.
Fixes #22295
Change-Id: Ife86c79ca20b2e9307008ed83885e193d32b7dc4
Reviewed-on: https://go-review.googlesource.com/72390
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ben Shi [Fri, 20 Oct 2017 03:50:15 +0000 (03:50 +0000)]
cmd/compile: optimize MOVBS/MOVBU/MOVHS/MOVHU on ARMv6 and ARMv7
MOVBS/MOVBU/MOVHS/MOVHU can be optimized with a single instruction
on ARMv6 and ARMv7, instead of a pair of left/right shifts.
The benchmark tests show big improvement in special cases and a little
improvement in total.
1. A special case gets about 29% improvement.
name old time/op new time/op delta
TypePro-4 3.81ms ± 1% 2.71ms ± 1% -28.97% (p=0.000 n=26+25)
The source code of this case can be found at
https://github.com/benshi001/ugo1/blob/master/typepromotion_test.go
griesemer [Wed, 25 Oct 2017 23:11:51 +0000 (16:11 -0700)]
test: add test cases for method expressions with literal receiver types
For #9060.
Change-Id: Ibd0f047083f3c98cec96c655a3e2024df8e1d2a0
Reviewed-on: https://go-review.googlesource.com/73554 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Artyom Pervukhin [Sat, 21 Oct 2017 06:36:54 +0000 (09:36 +0300)]
image/draw: reduce drawPaletted allocations for special source cases
drawPaletted has to discover R,G,B,A color values of each source image
pixel in a given rectangle. Doing that by calling image.Image.At()
method returning color.Color interface is quite taxing allocation-wise
since interface values go through heap. Introduce special cases for some
concrete source types by fetching color values using type-specific
methods.
name old time/op new time/op delta
Paletted-4 7.62ms ± 4% 3.72ms ± 3% -51.20% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
Paletted-4 480kB ± 0% 0kB ± 0% -99.99% (p=0.000 n=4+5)
name old allocs/op new allocs/op delta
Paletted-4 120k ± 0% 0k ± 0% -100.00% (p=0.008 n=5+5)
Updates #15759.
Change-Id: I0ce1770ff600ac80599541aaad4c2c826855c8fb
Reviewed-on: https://go-review.googlesource.com/72370 Reviewed-by: Nigel Tao <nigeltao@golang.org>
griesemer [Wed, 25 Oct 2017 18:26:02 +0000 (11:26 -0700)]
spec: match syntax for method expressions with implementations
A method expression is of the form T.m where T is a type and m
is a method of that type. The spec restricted T essentially to
a type name. Both cmd/compile and go/types accepted any type
syntactically, and a method expression was really just a form
of a selector expression x.f where x denotes a type.
This CL removes the spec syntax restriction from MethodExpr
to match the actual implementation. It also moves MethodExpr
from Operand to PrimaryExpr, because that's what it is.
It still keeps the separate notion of MethodExpr even though
it looks just like a selector expresion, since a MethodExpr
must start with a type rather than a value, and the spec's
syntax expresses this bit of semantics via distinct productions
(e.g., conversions look like calls but also must start with
a type).
Fixes #9060.
Change-Id: Idd84655b5b4f85d7ee53ebf749f73f0414a05f4a
Reviewed-on: https://go-review.googlesource.com/73233 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Mon, 23 Oct 2017 20:47:15 +0000 (13:47 -0700)]
archive/zip: restrict UTF-8 detection for comment and name fields
CL 39570 added support for automatically setting flag bit 11 to
indicate that the filename and comment fields are encoded in UTF-8,
which is (conventionally) the encoding using for most Go strings.
However, the detection added is too lose for two reasons:
* We need to ensure both fields are at least possibly UTF-8.
That is, if any field is definitely not UTF-8, then we can't set the bit.
* The utf8.ValidRune returns true for utf8.RuneError, which iterating
over a Go string automatically returns for invalid UTF-8.
Thus, we manually check for that value.
Updates #22367
Updates #10741
Change-Id: Ie8aae388432e546e44c6bebd06a00434373ca99e
Reviewed-on: https://go-review.googlesource.com/72791 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Adam Thomason [Wed, 25 Oct 2017 21:36:23 +0000 (14:36 -0700)]
cmd/go: add -shift to go vet's flag whitelist
CL 40112 intended to allow full flag processing in go vet, but missed
vet's -shift flag; this corrects the omission.
Fixes #22442
Change-Id: I47525018306bd8b9aa452fb378d0d45319f8cf11
Reviewed-on: https://go-review.googlesource.com/73553
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Fraenkel [Wed, 25 Oct 2017 16:42:14 +0000 (12:42 -0400)]
io: flatten MultiWriter writers
Replace any nested Writer that is a MultiWriter with its associated
Writers.
Fixes #22431
Change-Id: Ida7c4c83926363c1780689e216cf0c5241a5b8eb
Reviewed-on: https://go-review.googlesource.com/73470
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Lynn Boger [Wed, 25 Oct 2017 19:34:08 +0000 (15:34 -0400)]
cmd/go: enable TestNoteReading on ppc64le
This test was previously disabled when external linking was
unsupported on ppc64le. It should still be disabled on ppc64
since there is no cgo or external linking there, but I removed
the if test for GOARCH=ppc64 since the initial test for cgo
enabled will cause it to be skipped on ppc64.
Fixes #22360
Change-Id: I5a0e3e4a1bd71ac7bf0ed0c792f7b78fb4a5e100
Reviewed-on: https://go-review.googlesource.com/73510
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Henry [Wed, 18 Oct 2017 13:30:08 +0000 (15:30 +0200)]
net/smtp: added Noop to Client
This adds a Noop() function to the net/stmp client.
It allows for testing if a connaction is still healthy.
Fixes #22321
Change-Id: I023b613b1536ea21274cc36d41f5720c9bbdecbc
Reviewed-on: https://go-review.googlesource.com/71650
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Lorenz Bauer [Wed, 18 Oct 2017 10:15:04 +0000 (11:15 +0100)]
net: make {TCP,Unix}Listener implement syscall.Conn
This change adds the syscall.Conn interface to Listener types, with the caveat that only RawConn.Control is supported. Custom socket options can now be set safely.
Updates #19435
Fixes #22065
Change-Id: I7e74780d00318dc54a923d1c628a18a36009acab
Reviewed-on: https://go-review.googlesource.com/71651
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Theophanes [Wed, 25 Oct 2017 18:01:46 +0000 (11:01 -0700)]
database/sql: scan into *time.Time without reflection
Previously scanning time.Time into a *time.Time required reflection.
Now it does not. Scanning already checked if the source value was of
type time.Time. The only addition was checking the destination was
of type *time.Time.
Existing tests already scan time.Time into *time.Time, so no new
tests were added. Linked issue has performance justification.
Fixes #22300
Change-Id: I4eea461c78fad71ce76e7677c8503a1919666931
Reviewed-on: https://go-review.googlesource.com/73232
Run-TryBot: Daniel Theophanes <kardianos@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Daniel Theophanes [Tue, 17 Oct 2017 22:59:56 +0000 (15:59 -0700)]
database/sql: ensure all driver interfaces are called under single lock
Russ pointed out in a previous CL golang.org/cl/65731 that not only
was the locking incomplete, previous changes did not correctly
lock driver calls in other sections. After inspecting
driverConn, driverStmt, driverResult, Tx, and Rows structs
where driver interfaces are stored, I discovered a few more places
that failed to lock driver calls. The largest of these
was the parameter type converter "driverArgs".
driverArgs was typically called right before another call to the
driver in a locked region, so I made the entire driverArgs expect
a locked driver mutex and combined the region. This should not
be a problem because the connection is pulled out of the connection
pool either way so there shouldn't be contention.
Russ Cox [Wed, 25 Oct 2017 15:13:23 +0000 (11:13 -0400)]
runtime: avoid monotonic time zero on systems with low-res timers
Otherwise low-res timers cause problems at call sites that expect to
be able to use 0 as meaning "no time set" and therefore expect that
nanotime never returns 0 itself. For example, sched.lastpoll == 0
means no last poll.
Jason Wangsadinata [Tue, 24 Oct 2017 19:26:46 +0000 (02:26 +0700)]
container/ring: add examples for various Ring functions
Change-Id: I6b16cfcbe824f999c9b64df539deea567e327db7
Reviewed-on: https://go-review.googlesource.com/73090 Reviewed-by: Gabriel Aszalos <gabriel.aszalos@gmail.com> Reviewed-by: Jason Wangsadinata <jwangsadinata@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Gabriel Aszalos <gabriel.aszalos@gmail.com>
Russ Cox [Wed, 25 Oct 2017 01:14:39 +0000 (21:14 -0400)]
cmd/link: only adjust pagezero for iOS on darwin/arm64
The new pagezero_size introduced by CL 72730 breaks
on 32-bit systems, since it is 2³². Restrict the change to
darwin/arm64, since it is intended for iOS only.
We could plausibly allow GOARCH=amd64 as well, but
without a compelling reason, changing the zero page size
doesn't seem worth the risk.
Change-Id: I5d6adcbaff8d0e5b169ff13512f188332cc7ed9a
Reviewed-on: https://go-review.googlesource.com/73250
Run-TryBot: Russ Cox <rsc@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org> Reviewed-by: David Crawshaw <crawshaw@golang.org>
Joe Tsai [Mon, 23 Oct 2017 21:43:51 +0000 (14:43 -0700)]
encoding/csv: forbid certain Comma and Comment runes
The record delimiter (not configurable by user) is "\r\n" or "\n".
It is insensible for the user to set Comma or Comment delimiters
to be some character that conflicts with the record delimiter.
Furthermore, it is insensible for Comma or Comment to be the same rune.
Allowing this leaks implementation details to the user in regards to
the evaluation order of which rune is checked for first.
Fixes #22404
Change-Id: I31e86abc9b3a8fb4584e090477795587740970ae
Reviewed-on: https://go-review.googlesource.com/72793 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox [Tue, 24 Oct 2017 01:57:54 +0000 (21:57 -0400)]
build: move final steps of make.bash, make.bat, make.rc into cmd/dist
This CL expands the job of "dist bootstrap" to be "finish make.bash".
I need to change that logic in upcoming CLs related to cmd/go
changes, and I'd rather not change it in three places in three different
shell script languages.
Change-Id: I545dc215e408289e4d0b28f7c2ffcd849d89ad3b
Reviewed-on: https://go-review.googlesource.com/72870 Reviewed-by: David Crawshaw <crawshaw@golang.org>
Michael Hudson-Doyle [Fri, 28 Apr 2017 01:01:03 +0000 (13:01 +1200)]
cmd/link: replace SSUB with an attribute bit
There is some stuff I don't understand very well involved in SSUB, better words
for the documentation gratefully accepted.
As this is the last use of a bit in SMASK, kill that off too.
Change-Id: Iddff1c9b2af02c9dfb12ac8e668d004e4642f997
Reviewed-on: https://go-review.googlesource.com/42026
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Brian Kessler [Thu, 10 Aug 2017 06:17:07 +0000 (23:17 -0700)]
math/big: implement Lehmer's GCD algorithm
Updates #15833
Lehmer's GCD algorithm uses single precision calculations
to simulate several steps of multiple precision calculations
in Euclid's GCD algorithm which leads to a considerable
speed up. This implementation uses Collins' simplified
testing condition on the single digit cosequences which
requires only one quotient and avoids any possibility of
overflow.
Mark Pulford [Sun, 3 Sep 2017 11:46:51 +0000 (21:46 +1000)]
math: add RoundToEven function
Rounding ties to even is statistically useful for some applications.
This implementation completes IEEE float64 rounding mode support (in
addition to Round, Ceil, Floor, Trunc).
This function avoids subtle faults found in ad-hoc implementations, and
is simple enough to be inlined by the compiler.
Fixes #21748
Change-Id: I09415df2e42435f9e7dabe3bdc0148e9b9ebd609
Reviewed-on: https://go-review.googlesource.com/61211 Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Daniel Theophanes [Mon, 2 Oct 2017 18:16:53 +0000 (11:16 -0700)]
database/sql: add driver.ResetSessioner and add pool support
A single database connection ususally maps to a single session.
A connection pool is logically also a session pool. Most
sessions have a way to reset the session state which is desirable
to prevent one bad query from poisoning another later query with
temp table name conflicts or other persistent session resources.
It also lets drivers provide users with better error messages from
queryies when the underlying transport or query method fails.
Internally the driver connection should now be marked as bad, but
return the actual connection. When ResetSession is called on the
connection it should return driver.ErrBadConn to remove it from
the connection pool. Previously drivers had to choose between
meaningful error messages or poisoning the connection pool.
Lastly update TestPoolExhaustOnCancel from relying on a
WAIT query fixing a flaky timeout issue exposed by this
change.
Fixes #22049
Fixes #20807
Change-Id: I2b5df6d954a38d0ad93bf1922ec16e74c827274c
Reviewed-on: https://go-review.googlesource.com/73033
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Heschi Kreinick [Thu, 19 Oct 2017 22:59:41 +0000 (18:59 -0400)]
cmd/compile/internal/ssa: use reverse postorder traversal
Instead of the hand-written control flow analysis in debug info
generation, use a reverse postorder traversal, which is basically the
same thing. It should be slightly faster.
More importantly, the previous version simply gave up in the case of
non-reducible functions, and produced output that caused a later stage
to crash. It turns out that there's a non-reducible function in
compress/flate, so that wasn't a theoretical issue.
With this change, all blocks will be visited, even for non-reducible
functions.
Nils Larsgård [Tue, 24 Oct 2017 17:21:58 +0000 (19:21 +0200)]
mime/multipart: permit empty file name
Fixes #19183
Change-Id: I11502d855f5b521b03ed7a63a990cca2d0ed4083
Reviewed-on: https://go-review.googlesource.com/70931 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Theophanes [Mon, 2 Oct 2017 18:16:53 +0000 (11:16 -0700)]
database/sql: add driver.ResetSessioner and add pool support
A single database connection ususally maps to a single session.
A connection pool is logically also a session pool. Most
sessions have a way to reset the session state which is desirable
to prevent one bad query from poisoning another later query with
temp table name conflicts or other persistent session resources.
It also lets drivers provide users with better error messages from
queryies when the underlying transport or query method fails.
Internally the driver connection should now be marked as bad, but
return the actual connection. When ResetSession is called on the
connection it should return driver.ErrBadConn to remove it from
the connection pool. Previously drivers had to choose between
meaningful error messages or poisoning the connection pool.
Lastly update TestPoolExhaustOnCancel from relying on a
WAIT query fixing a flaky timeout issue exposed by this
change.
Fixes #22049
Fixes #20807
Change-Id: Idffa1a7ca9ccfe633257c4a3ae299b864f46c5b6
Reviewed-on: https://go-review.googlesource.com/67630 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Theophanes [Sun, 24 Sep 2017 02:38:32 +0000 (19:38 -0700)]
database/sql: allow drivers to only implement Context variants
Drivers shouldn't need to implement both Queryer and QueryerContext,
they should just implement QueryerContext. Same with Execer and
ExecerContext. This CL tests for QueryContext and ExecerContext
first so drivers do not need to implement Queryer and Execer
with an empty definition.
Fixes #21663
Change-Id: Ifbaa71da669f4bc60f8da8c41a04a4afed699a9f
Reviewed-on: https://go-review.googlesource.com/65733 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Sat, 21 Oct 2017 19:07:55 +0000 (12:07 -0700)]
encoding/csv: fix error documentation
We should be referring to ParseError.Err, which is the underlying error,
not ParseError.Error, which is the error method.
Change-Id: Ic3cef5ecbe1ada5fa14b9573222f29da8fc9a8d5
Reviewed-on: https://go-review.googlesource.com/72450 Reviewed-by: Tim Cooper <tim.cooper@layeh.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Mon, 23 Oct 2017 06:27:34 +0000 (17:27 +1100)]
runtime: make errno positive in netpollopen
Make netpollopen return what Windows GetLastError API returns.
It is probably copy / paste error from long time ago.
Change-Id: I28f78718c15fef3e8b5f5d11a259533d7e9c6185
Reviewed-on: https://go-review.googlesource.com/72592 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Matthew Dempsky [Fri, 20 Oct 2017 18:01:43 +0000 (11:01 -0700)]
cmd/compile: silence unnecessary unsafe error
If n.Type==nil after typechecking, then we should have already
reported a more useful error somewhere else. Just return 0 in
evalunsafe without trying to do anything else that's likely to cause
problems.
Also, further split out issue7525.go into more test files, because
cmd/compile reports at most one typechecking loop per compilation
unit.
Fixes #22351.
Change-Id: I3ebf505f72c48fcbfef5ec915606224406026597
Reviewed-on: https://go-review.googlesource.com/72251
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Robert Griesemer <gri@golang.org>
Artyom Pervukhin [Thu, 12 Oct 2017 19:03:13 +0000 (22:03 +0300)]
image/gif: avoid setting defers in the decode loop
decoder.decode() was defering close of lzw.decoders created for each
frame in a loop, thus increasing heap usage (referenced object + defered
function) until decode() returns. Memory increased proportionally to the
number of frames. Fix this by moving the sImageDescriptor case block
into its own method.
Than McIntosh [Mon, 23 Oct 2017 15:41:47 +0000 (11:41 -0400)]
cmd/compile: include non-decomposed vars for -dwarflocationlists
When enhanced DWARF location list generation is enabled (via internal
option -dwarflocationlists), variable entries were missing for "large"
(non-decomposable) locals and formals. From the debugging perspective,
this makes it appear that the variable doesn't exist, which is
probably not what we want. This change insures that a formal/local DIE
is created for these vars (with correct type, line, etc) but with a
conservative ("no info") location.
Cherry Zhang [Mon, 23 Oct 2017 13:39:13 +0000 (09:39 -0400)]
cmd/internal/obj/arm64: handle global address in LDP/STP
The addressing mode of global variable was missing, whereas the
compiler may make use of it, causing "illegal combination" error.
This CL adds support of that addressing mode.
Daniel Martí [Fri, 20 Oct 2017 10:56:31 +0000 (11:56 +0100)]
cmd/compile: clean up various bits of code
* replace a copy of IsMethod with a call of it.
* a few more switches where they simplify the code.
* prefer composite literals over "n := new(...); n.x = y; ...".
* use defers to get rid of three goto labels.
* rewrite updateHasCall into two funcs to remove gotos.
Passes toolstash-check on std cmd.
Change-Id: Icb5442a89a87319ef4b640bbc5faebf41b193ef1
Reviewed-on: https://go-review.googlesource.com/72070
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Matthew Dempsky [Fri, 20 Oct 2017 22:20:56 +0000 (15:20 -0700)]
cmd/compile: replace -l=2 with -d typecheckinl
Currently, benchmarking compile performance under -l=4 is confounded
by -l=2 enabling eager typechecking of unused inline function bodies
for debugging. This isn't logically an "inlining aggressiveness"
level, so instead move this logic under the -d umbrella flag.
Change-Id: I713f68952efbe25b6941d3ebc2f3707ccbbd6240
Reviewed-on: https://go-review.googlesource.com/72253
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Joe Tsai [Sat, 21 Oct 2017 00:04:17 +0000 (17:04 -0700)]
encoding/csv: add ParseError.RecordLine
CL 72150 fixes #22352 by reverting the problematic parts of that CL
where the line number and column number were inconsistent with each other.
This CL adds back functionality to address the issue that CL 72150
was trying to solve in the first place. That is, it reports the starting
line of the record, so that users have a frame of reference to start with
when debugging what went wrong.
In the event of gnarly CSV files with multiline quoted strings, a parse
failure likely occurs somewhere between the start of the record and
the point where the parser finally detected an error.
Since ParserError.{Line,Column} reports where the *error* occurs, we
add a RecordLine field to report where the record starts.
Also take this time to cleanup and modernize TestRead.
Fixes #19019
Fixes #22352
Change-Id: I16cebf0b81922c35f75804c7073e9cddbfd11a04
Reviewed-on: https://go-review.googlesource.com/72310 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Fri, 20 Oct 2017 01:44:00 +0000 (12:44 +1100)]
os: simplify windows Pipe
windows version of Pipe function is implemented by calling
syscall.Pipe which returns handles inheritable by client process,
and then adjusting returned handles with syscall.CloseOnExec.
Just create non-inheritable handles in the first place.
Now that we don't have a race window in the code, drop use
of syscall.ForkLock.
Change-Id: Ie325da7c2397b5995db4a5ddb0117e2ce1745187
Reviewed-on: https://go-review.googlesource.com/72010 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Tim Cooper [Thu, 12 Oct 2017 01:05:03 +0000 (22:05 -0300)]
encoding/hex: add NewEncoder, NewDecoder
NewEncoder returns an io.Writer that writes all incoming bytes as
hexadecimal characters to the underlying io.Writer. NewDecoder returns an
io.Reader that does the inverse.
Fixes #21590
Change-Id: Iebe0813faf365b42598f19a9aa41768f571dc0a8
Reviewed-on: https://go-review.googlesource.com/70210 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Fri, 20 Oct 2017 12:26:43 +0000 (05:26 -0700)]
encoding/csv: simplify and optimize Reader
The Reader implementation is slow because it operates on a rune-by-rune
basis via bufio.Reader.ReadRune. We speed this up by operating on entire
lines that we read from bufio.Reader.ReadSlice.
In order to ensure that we read the full line, we augment ReadSlice
in our Reader.readLine method to automatically expand the slice if
bufio.ErrBufferFull is every hit.
This change happens to fix #19410 because it no longer relies on
rune-by-rune parsing and only searches for the relevant delimiter rune.
In order to keep column accounting simple and consistent, this change
reverts parts of CL 52830.
This CL is an alternative to CL 36270 and builds on some of the ideas
from that change by Diogo Pinela.
Ian Lance Taylor [Thu, 19 Oct 2017 23:01:43 +0000 (16:01 -0700)]
runtime: for kqueue treat EVFILT_READ with EV_EOF as permitting a write
On systems that use kqueue, we always register descriptors for both
EVFILT_READ and EVFILT_WRITE. On at least FreeBSD and OpenBSD, when
the write end of a pipe is registered for EVFILT_READ and EVFILT_WRITE
events, and the read end of the pipe is closed, kqueue reports an
EVFILT_READ event with EV_EOF set, but does not report an EVFILT_WRITE
event. Since the write to the pipe is waiting for an EVFILT_WRITE
event, closing the read end of a pipe can cause the write end to hang
rather than attempt another write which will fail with EPIPE.
Fix this by treating EVFILT_READ with EV_EOF set as making both reads
and writes ready to proceed.
The real test for this is in CL 71770, which tests using various
timeouts with pipes.
Updates #22114
Change-Id: Ib23fbaaddbccd8eee77bdf18f27a7f0aa50e2742
Reviewed-on: https://go-review.googlesource.com/71973 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Cherry Zhang [Fri, 20 Oct 2017 14:53:48 +0000 (10:53 -0400)]
cmd/compile: skip runtime.nextFreeFast inlining test on MIPS64x
Since inlining budget calculation is fixed in CL 70151
runtime.nextFreeFast is no longer inlineable on MIPS64x because
it does not support Ctz64 as intrinsic. Skip the test.
Russ Cox [Wed, 18 Oct 2017 02:33:21 +0000 (22:33 -0400)]
cmd/go: delete ETXTBSY hack that is no longer needed
This hack existed because cmd/go used to install (write) and then run
cmd/cgo in the same invocation, and writing and then running a program
is a no-no in modern multithreaded Unix programs (see #22315).
As of CL 68338, cmd/go no longer installs any programs that it then
tries to use. It never did this for any program other than cgo, and
CL 68338 removed that special case for cgo.
Now this special case, added for #3001 long ago, can be removed too.
Change-Id: I338f1f8665e9aca823e33ef7dda9d19f665e4281
Reviewed-on: https://go-review.googlesource.com/71571
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>