The first part of this test tries to confirm that we can't create
a TempFile in a non-existent directory, but does not ensure that
the non-existent directory really does not exist. Instead, let's
create an empty temp directory, and use a non-existent subdir of
that.
cmd/go/internal/work: clean up after TestRespectGroupSticky
Use our own tempdir, to avoid having to Init (and somehow teardown)
Builder. This way we don't leave behind any temp files.
Also, don't create a hardcoded path inside a testcase.
Followup to golang/go#18878.
Fixes golang/go#19449.
Change-Id: Ieb1ebeab24ae8a74a6fa058d9c23f72b3fc1c444
Reviewed-on: https://go-review.googlesource.com/40912 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Introduce the following changes:
- 6b27048 route: drop support for go1.5
- b7fd658 route: fix typo
- 41bba8d route: add support for the manipulation of routing informaion
It was added in 2013 in CL 7064048.
All uses of it in the compiler disappeared with
(or possibly before) the SSA backend.
Several releases have gone by without it,
from which I conclude that it is now not needed.
Introduce the following changes:
- 05d3205 http2/hpack: fix memory leak in headerFieldTable lookup maps
- bce15e7 http2/hpack: speedup Encoder.searchTable
- dd2d9a6 http2/hpack: remove hpack's constant time string comparison
- 357296a all: single space after period
- 71a0359 x/net/http2: Fix various typos in doc comments.
net: delay IP protocol stack-snooping system calls
This change delays IP protocol stack-snooping system calls until the
start of connection setup for the better experience with some system
call auditing, such as seccomp on Linux. See #16789 for examples.
Also updates the documentation on favoriteAddrFamily, which is the
owner of stack-snooping system calls.
Fixes #16789.
Change-Id: I4af27bc1ed06ffb1f657b6f6381c328c1f41c66c
Reviewed-on: https://go-review.googlesource.com/40750
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Caleb Spare [Wed, 29 Mar 2017 22:04:40 +0000 (15:04 -0700)]
testing: add TB.Helper to better support test helpers
This CL implements the proposal at
https://github.com/golang/proposal/blob/master/design/4899-testing-helper.md.
It's based on Josh's CL 79890043 from a few years ago:
https://codereview.appspot.com/79890043 but makes several changes,
most notably by using the new CallersFrames API so that it works with
mid-stack inlining.
Another detail came up while I was working on this: I didn't want the
user to be able to call t.Helper from inside their TestXxx function
directly (which would mean we'd print a file:line from inside the
testing package itself), so I explicitly prevented this from working.
Fixes #4899.
Change-Id: I37493edcfb63307f950442bbaf993d1589515310
Reviewed-on: https://go-review.googlesource.com/38796
Run-TryBot: Caleb Spare <cespare@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Andreas Auernhammer [Fri, 14 Apr 2017 09:38:53 +0000 (11:38 +0200)]
vendor: update golang_org/x/crypto packages
Update the poly1305 and curve25519 packages to the current state of /x/crypto.
Updates #19967
Change-Id: Ib71534f78040f31bfd5debb06f3c4a54a77955b3
Reviewed-on: https://go-review.googlesource.com/40711 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
net: validate network in Dial{,IP} and Listen{Packet,IP} for IP networks
The argument of the first parameter for connection setup functions on
IP networks must contain a protocol name or number. This change adds
validation for arguments of IP networks to connection setup functions.
Fixes #18185.
Change-Id: I6aaedd7806e3ed1043d4b1c834024f350b99361d
Reviewed-on: https://go-review.googlesource.com/40512
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
net: don't enclose non-literal IPv6 addresses in square brackets
The net package uses various textual representations for network
identifiers and locators on the Internet protocol suite as API.
In fact, the representations are the composition of subset of multple
RFCs: RFC 3986, RFC 4007, RFC 4632, RFC 4291 and RFC 5952.
RFC 4007 describes guidelines for the use of textual representation of
IPv6 addressing/routing scope zone and doesn't prohibit the format for
implementation dependent purposes, as in, specifying a literal IPv6
address and its connected region of routing topology as application
user interface. However, a non-literal IPv6 address, for example, a
host name, with a zone enclosed in square brackets confuses us because
a zone is basically for non-global IPv6 addresses and a pair of square
brackets is used as a set of delimiters between a literal IPv6 address
and a service name or transport port number.
To mitigate such confusion, this change makes JoinHostPort not enclose
non-literal IPv6 addresses in square brackets and SplitHostPort accept
the form "host%zone:port" to recommend that anything enclosed in
square brackets should be a literal IPv6 address.
On s390x unsigned integer comparisons with immediates require the immediate
to be an unsigned 32-bit integer. The rule was checking that the immediate
was a signed 32-bit integer.
This CL also adds a test for comparisons that could be turned into compare
with immediate or equivalent instructions (depending on architecture and
optimizations applied).
Fixes #19940.
Change-Id: Ifd6aa989fd3d50e282f7d30fec9db462c28422b1
Reviewed-on: https://go-review.googlesource.com/40433
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Austin Clements [Mon, 20 Mar 2017 21:25:59 +0000 (17:25 -0400)]
runtime: free workbufs during sweeping
This extends the sweeper to free workbufs back to the heap between GC
cycles, allowing this memory to be reused for GC'd allocations or
eventually returned to the OS.
This helps for applications that have high peak heap usage relative to
their regular heap usage (for example, a high-memory initialization
phase). Workbuf memory is roughly proportional to heap size and since
we currently never free workbufs, it's proportional to *peak* heap
size. By freeing workbufs, we can release and reuse this memory for
other purposes when the heap shrinks.
This is somewhat complicated because this costs ~1–2 µs per workbuf
span, so for large heaps it's too expensive to just do synchronously
after mark termination between starting the world and dropping the
worldsema. Hence, we do it asynchronously in the sweeper. This adds a
list of "free" workbuf spans that can be returned to the heap. GC
moves all workbuf spans to this list after mark termination and the
background sweeper drains this list back to the heap. If the sweeper
doesn't finish, that's fine, since getempty can directly reuse any
remaining spans to allocate more workbufs.
Performance impact is negligible. On the x/benchmarks, this reduces
GC-bytes-from-system by 6–11%.
Austin Clements [Fri, 24 Mar 2017 16:02:12 +0000 (12:02 -0400)]
runtime: eliminate write barriers from alloc/mark bitmaps
This introduces a new type, *gcBits, to use for alloc/mark bitmap
allocations instead of *uint8. This type is marked go:notinheap, so
uses of it correctly eliminate write barriers. Since we now have a
type, this also extracts some common operations to methods both for
convenience and to avoid (*uint8) casts at most use sites.
Austin Clements [Fri, 24 Mar 2017 15:36:40 +0000 (11:36 -0400)]
runtime: rename gcBits -> gcBitsArena
This clarifies that the gcBits type is actually an arena of gcBits and
will let us introduce a new gcBits type representing a single
mark/alloc bitmap allocated from the arena.
Austin Clements [Wed, 22 Mar 2017 17:45:12 +0000 (13:45 -0400)]
runtime: don't count manually-managed spans from heap_{inuse,sys}
Currently, manually-managed spans are included in memstats.heap_inuse
and memstats.heap_sys, but when we export these stats to the user, we
subtract out how much has been allocated for stack spans from both.
This works for now because stacks are the only manually-managed spans
we have.
However, we're about to use manually-managed spans for more things
that don't necessarily have obvious stats we can use to adjust the
user-presented numbers. Prepare for this by changing the accounting so
manually-managed spans don't count toward heap_inuse or heap_sys. This
makes these fields align with the fields presented to the user and
means we don't have to track more statistics just so we can adjust
these statistics.
Austin Clements [Thu, 16 Mar 2017 18:46:53 +0000 (14:46 -0400)]
runtime: generalize {alloc,free}Stack to {alloc,free}Manual
We're going to start using manually-managed spans for GC workbufs, so
rename the allocate/free methods and pass in a pointer to the stats to
use instead of using the stack stats directly.
Austin Clements [Thu, 16 Mar 2017 18:16:31 +0000 (14:16 -0400)]
runtime: rename _MSpanStack -> _MSpanManual
We're about to generalize _MSpanStack to be used for other forms of
in-heap manual memory management in the runtime. This is an automated
rename of _MSpanStack to _MSpanManual plus some comment fix-ups.
Wei Xiao [Fri, 17 Mar 2017 03:31:09 +0000 (03:31 +0000)]
hash/crc32: optimize arm64 crc32 implementation
ARMv8 defines crc32 instruction.
Comparing to the original crc32 calculation, this patch makes use of
crc32 instructions to do crc32 calculation instead of the multiple
lookup table algorithms.
ARMv8 provides IEEE and Castagnoli polynomials for crc32 calculation
so that the perfomance of these two types of crc32 get significant
improved.
Meir Fischer [Sat, 8 Apr 2017 04:17:01 +0000 (00:17 -0400)]
net/http/fcgi: expose cgi env vars in request context
The current interface can't access all environment
variables directly or via cgi.RequestFromMap, which
only reads variables on its "white list" to be set on
the http.Request it returns. If an fcgi variable is
not on the "white list" - e.g. REMOTE_USER - the old
code has no access to its value.
This passes variables in the Request context that aren't
used to add data to the Request itself and adds a method
that parses those env vars from the Request's context.
cmd/compile: make TestAssembly resilient to output ordering
To preserve reproducible builds, the text entries
during compilation will be sorted before being printed.
TestAssembly currently assumes that function init
comes after all user-defined functions.
Remove that assumption.
Instead of looking for "TEXT" to tell you where
a function ends--which may now yield lots of
non-function-code junk--look for a line beginning
with non-whitespace.
Instead of constructing ctxt.Text in Flushplist,
which will be called concurrently,
do it in InitTextSym, which must be called serially.
This allows us to avoid a mutex for ctxt.Text,
and preserves the existing ordering of functions
for debug output.
encoding/asn1: support 31 bit identifiers with OID
The current implementation uses a max of 28 bits when decoding an
ObjectIdentifier. This change makes it so that an int64 is used to
accumulate up to 35 bits. If the resulting data would not overflow
an int32, it is used as an int. Thus up to 31 bits may be used to
represent each subidentifier of an ObjectIdentifier.
Fixes #19933
Change-Id: I95d74b64b24cdb1339ff13421055bce61c80243c
Reviewed-on: https://go-review.googlesource.com/40436 Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
os: handle relative symlinks starting with slash in Stat on windows
https://go-review.googlesource.com/c/39932/ handles relative symlinks.
But that change is incomplete.
We also have to handle relative symlinks starting with slash too.
Fixes #19937
Change-Id: I50dbccbaf270cb48a08fa57e5f450e5da18a7701
Reviewed-on: https://go-review.googlesource.com/40410 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The initialization of an ATEXT Prog's From.Sym
can race with the assemblers in a concurrent compiler.
CL 40254 contains an initial, failed attempt to
fix that race.
This CL takes a different approach: Rather than
expose an API to initialize the Prog,
expose an API to initialize the Sym.
The initialization of the Sym can then be
moved earlier in the compiler, avoiding the race.
The growth of gc.Func has negligible
performance impact; see below.
cmd/internal/obj: stop storing Text flags in From3
Prior to this CL, flags such as NOSPLIT
on ATEXT Progs were stored in From3.Offset.
Some but not all of those flags were also
duplicated into From.Sym.Attribute.
This CL migrates all of those flags into
From.Sym.Attribute and stops creating a From3.
A side-effect of this is that printing an
ATEXT Prog can no longer simply dump From3.Offset.
That's kind of good, since the raw flag value
wasn't very informative anyway, but it did
necessitate a bunch of updates to the cmd/asm tests.
The reason I'm doing this work now is that
avoiding storing flags in both From.Sym and From3.Offset
simplifies some other changes to fix the data
race first described in CL 40254.
This CL almost passes toolstash-check -all.
The only changes are in cases where the assembler
has decided that a function's flags may be altered,
e.g. to make a function with no calls in it NOSPLIT.
Prior to this CL, that information was not printed.
These patterns are the only uses of isArg and isAuto, and they all
follow a common pattern too. Extract out so that we can more easily
tweak the interface for isArg/isAuto.
cmd/dist: require _ before GOOS and GOARCH when building bootstrap
Currently, dist allows GOOS and GOARCH to appear as *any* substring in
a file name when selecting source files to go into go_bootstrap. This
was necessary prior to Go 1.4, where it needed to match names like
"windows.c", but now it's gratuitously different from go/build. This
led to a bug chase to figure out why "stubs_nonlinux.go" was not being
built on non-Linux OSes.
Change shouldbuild to require an "_" before the GOOS and GOARCH in a
file name. This is still less strict than go/build, but the behavior
is much closer.
runtime: expand inlining iteratively in CallersFrames
Currently CallersFrames expands each PC to a slice of Frames and then
iteratively returns those Frames. However, this makes it very
difficult to avoid heap allocation: either the Frames slice will be
heap allocated, or, if it uses internal scratch space for small slices
(as it currently does), the Frames object itself has to be heap
allocated.
Fix this, at least in the common case, by expanding each PC
iteratively. We introduce a new pcExpander type that's responsible for
expanding a single PC. This maintains state from one Frame to the next
in the same PC. Frames then becomes a wrapper around this responsible
for feeding it the next PC when the pcExpander runs out of frames for
the current PC.
This makes it possible to stack-allocate a Frames object, which will
make it possible to use this API for PC expansion from within the
runtime itself.
Alberto Donizetti [Tue, 7 Mar 2017 10:54:29 +0000 (11:54 +0100)]
cmd/compile: diagnose constant division by complex zero
When casting an ideal to complex{64,128}, for example during the
evaluation of
var a = complex64(0) / 1e-50
we want the compiler to report a division-by-zero error if a divisor
would be zero after the cast.
We already do this for floats; for example
var b = float32(0) / 1e-50
generates a 'division by zero' error at compile time (because
float32(1e-50) is zero, and the cast is done before performing the
division).
There's no such check in the path for complex{64,128} expressions, and
no cast is performed before the division in the evaluation of
var a = complex64(0) / 1e-50
which compiles just fine.
This patch changes the convlit1 function so that complex ideals
components (real and imag) are correctly truncated to float{32,64}
when doing an ideal -> complex{64, 128} cast.
Fixes #11674
Change-Id: Ic5f8ee3c8cfe4c3bb0621481792c96511723d151
Reviewed-on: https://go-review.googlesource.com/37891
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Move it to the x86 package, matching our handling
of deferreturn in x86 and arm.
While we're here, improve the concurrency safety
of both Plan9privates and deferreturn
by eagerly initializing them in instinit.
A recent performance improvement for PPC64.rules introduced a
regression for the case where the size of a move is <= 8 bytes
and the value used in the offset field of the instruction is not
aligned correctly for the instruction. In the cases where this happened,
the assembler was not detecting the incorrect offset and still generated
the instruction even though it was invalid.
This fix changes the PPC64.rules for the moves that are now failing
to include the correct alignment checks, along some additional testcases
for gc/ssa for the failing alignments.
I will add a fix to the assembler to detect incorrect offsets in
another CL.
This fixes #19907
Change-Id: I3d327ce0ea6afed884725b1824f9217cef2fe6bf
Reviewed-on: https://go-review.googlesource.com/40290 Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com> Reviewed-by: David Chase <drchase@google.com>
cmd/go: generate correct vendor paths with -compiler gccgo option
Curently the vendor paths are not always searched for imports if
the compiler is gccgo. This change generates the vendor paths
and adds them with -I as arguments to the gccgo compile.
Fixes #15628
Change-Id: I318accbbbd8e6af45475eda399377455a3565880
Reviewed-on: https://go-review.googlesource.com/40432
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Martí [Tue, 11 Apr 2017 19:31:41 +0000 (20:31 +0100)]
cmd/compile/internal/gc: don't panic on continue in switch
Continues outside of a loop are not allowed. Most of these possibilities
were tested in label1.go, but one was missing - a plain continue in a
switch/select but no enclosing loop.
This used to error with a "continue not in loop" in 1.8, but recently
was broken by c03e75e5. In particular, innerloop does not only account
for loops, but also for switches and selects. Swap it by bools that
track whether breaks and continues should be allowed.
While at it, improve the wording of errors for breaks that are not where
they should be. Change "loop" by "loop, switch, or select" since they
can be used in any of those.
And add tests to make sure this isn't broken again. Use a separate func
since I couldn't get the compiler to crash on f() itself, possibly due
to the recursive call on itself.
Previously, int values of #define macro are retrieved from DWARF via enums.
Currently, those values are retrieved from symbol tables.
It seems that previous code is unused.
Change-Id: Id76c54baa46d6196738ea35aebd5de99b05b9bf8
Reviewed-on: https://go-review.googlesource.com/40072 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Alessandro Arzilli [Sat, 18 Mar 2017 14:09:40 +0000 (15:09 +0100)]
cmd/link: export specialized variants of runtime._type to debug_info
The reflect package can be used to create new types at runtime, these
types will have runtime._type entries describing them but no entry in
debug_info (obviously).
A debugger that wanted to print the value of variables with such types
will have to read the runtime._type directly, however the
"specializations" of runtime._type (runtime.slicetype, runtime.maptype,
etc) are not exported to debug_info, besides runtime.interfacetype.
All those types (i.e. runtime.slicetype, runtime.maptype, etc) should
be exported to debug_info so that debuggers don't have to hard-code
their description.
Fixes #19602
Change-Id: I086d523a4421a4ed964e16bc3c2274319a98b45b
Reviewed-on: https://go-review.googlesource.com/38350 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Prevent a crash if the same type in two plugins had a recursive
definition, either by referring to a pointer to itself or a map existing
with the type as a value type (which creates a recursive definition
through the overflow bucket type).
plugin: resolve random crash when calling exported functions
open modified the plugin symbols map while ranging over it. This is
normally harmless, except that the operations performed were not
idempotent leading to function pointers being corrupted.
Alexander Menzhinsky [Mon, 10 Apr 2017 18:12:51 +0000 (13:12 -0500)]
cmd/go: add better error message when gccgo is missing
Fixes #19628
Change-Id: I19baf694c66aaca8e0d95297c97aacb40db24c47
Reviewed-on: https://go-review.googlesource.com/40250 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Daniel Theophanes [Wed, 5 Apr 2017 00:03:10 +0000 (17:03 -0700)]
database/sql: de-duplicate various methods
Form a new method pattern where *driverConn and
release functions are passed into the method.
They are named DB.execDC, DB.queryDC, DB.beginDC. This
allows more code to be de-duplicated when starting
queries.
The Stmt creation and management code are untouched.
sort.Slice was added in Go 1.8.
It's nice to use, and faster than sort.Sort,
so it'd be nice to be able to use it in the toolchain.
This CL adds obj.SortSlice, which is sort.Slice,
but with a slower fallback version for bootstrapping.
Joel Sing [Tue, 11 Apr 2017 15:10:01 +0000 (01:10 +1000)]
cmd/link,runtime/cgo: enable PT_TLS generation on OpenBSD
OpenBSD 6.0 and later have support for PT_TLS in ld.so(1). Now that OpenBSD
6.1 has been released, OpenBSD 5.9 is no longer officially supported and Go
can start generating PT_TLS for OpenBSD cgo binaries. This also allows us
to remove the workarounds in the OpenBSD cgo runtime.
This change also removes the environ and progname exports - these are now
provided directly by ld.so(1) itself.
Fixes #19932
Change-Id: I42e75ef9feb5dcd4696add5233497e3cbc48ad52
Reviewed-on: https://go-review.googlesource.com/40331 Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/internal/obj/s390x: make assembler almost concurrency-safe
CL 39922 made the arm assembler concurrency-safe.
This CL does the same, but for s390x.
The approach is similar: introduce ctxtz to hold
function-local state and thread it through
the assembler as necessary.
One race remains after this CL, similar to CL 40252.
That race is conceptually unrelated to this refactoring,
and will be addressed in a separate CL.
cmd/internal/obj/ppc64: make assembler almost concurrency-safe
CL 39922 made the arm assembler concurrency-safe.
This CL does the same, but for ppc64.
The approach is similar: introduce ctxt9 to hold
function-local state and thread it through
the assembler as necessary.
One race remains after this CL, similar to CL 40252.
That race is conceptually unrelated to this refactoring,
and will be addressed in a separate CL.
cmd/internal/obj/arm64: make assembler almost concurrency-safe
CL 39922 made the arm assembler concurrency-safe.
This CL does the same, but for arm64.
The approach is similar: introduce ctxt7 to hold
function-local state and thread it through
the assembler as necessary.
One race remains after this CL, deep in aclass,
in the check that a Prog does not take the address
of a TLS variable.
That race is conceptually unrelated to this refactoring,
and will be addressed in a separate CL.
cmd/internal/obj/mips: make assembler almost concurrency-safe
CL 39922 made the arm assembler concurrency-safe.
This CL does the same, but for mips.
The approach is similar: introduce ctxt0 to hold
function-local state and thread it through
the assembler as necessary.
One race remains after this CL, similar to CL 40252.
That race is conceptually unrelated to this refactoring,
and will be addressed in a separate CL.
Ben Shi [Mon, 27 Feb 2017 07:56:57 +0000 (07:56 +0000)]
runtime: use hardware divider to improve performance
The hardware divider is an optional component of ARMv7. This patch
detects whether it is available in runtime and use it or not.
1. The hardware divider is detected at startup and a flag is set/clear
according to a perticular bit of runtime.hwcap.
2. Each call of runtime.udiv will check this flag and decide if
use the hardware division instruction.
A rough test shows the performance improves 40-50% for ARMv7. And
the compatibility of ARMv5/v6 is not broken.
Changing mheap_.arena_used requires several steps that are currently
repeated multiple times in mheap_.sysAlloc. Consolidate these into a
single function.
In the future, this will also make it easier to add other auxiliary VM
structures.
Change-Id: Ie68837d2612e1f4ba4904acb1b6b832b15431d56
Reviewed-on: https://go-review.googlesource.com/40151
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>