net/http: deep copy Request.URL also in Request.WithContext's copy
Despite the previously known behavior of Request.WithContext
shallow copying a request, usage of the request inside server.ServeHTTP
mutates the request's URL. This CL implements deep copying of the URL.
David Chase [Mon, 20 Mar 2017 18:24:00 +0000 (14:24 -0400)]
cmd/dist: ensure android-implies-linux for file names in dist
This is one hurdle to building Go on Android; the runtime does
not build properly because *_linux.go files are excluded from
the "Building go_bootstrap" step when GOOS=android.
Jan Berktold [Thu, 23 Feb 2017 14:53:27 +0000 (15:53 +0100)]
net/http: make Transport respect non lower case Content-Encoding
The existing Transport implementation does not detect gzip encoding
when the Content-Encoding header is not lower-case. This is not
compliant with RFC2616 section 3.5 "All content-coding values are
case-insensitive." and caused issues in the wild.
Brad Fitzpatrick [Wed, 24 May 2017 00:05:04 +0000 (00:05 +0000)]
net/http: update bundled x/net/http2
This updates the bundled copy of x/net/http2 to x/net git rev a8e8f92cd6 for:
http2: remove extra goroutine stack from awaitGracefulShutdown
https://golang.org/cl/43230
http2: Discard DATA frames from the server after the response body is closed
https://golang.org/cl/43810
Fixes #20302
Fixes #18471
Fixes #20448
Change-Id: I00972836deb2fe6049f631ee44901732a641b171
Reviewed-on: https://go-review.googlesource.com/44006 Reviewed-by: Tom Bergan <tombergan@google.com>
Tom Bergan [Tue, 23 May 2017 23:03:21 +0000 (16:03 -0700)]
net/http: Add Server.RegisterOnShutdown
This will be used to allow http2 servers to register a shutdown function
so that net/http.Server.Shutdown will work when the http2 server is
configured via a manual call to http2.ConfigureServer. Currently, Shutdown
only works when the http2 server is configured automatically by the
net/http package.
Updates #20302
Updates #18471
Change-Id: Ifc2b5f3126126a106b49ea4a7e999279852b9cc9
Reviewed-on: https://go-review.googlesource.com/44003
Run-TryBot: Tom Bergan <tombergan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Austin Clements [Tue, 23 May 2017 21:54:24 +0000 (17:54 -0400)]
runtime: use pselect6 for usleep on linux/386
Commit 4dcba023c6 replaced select with pselect6 on linux/amd64 and
linux/arm, but it turns out the Android emulator uses linux/386. This
makes the equivalent change there, too.
cmd/cgo: support indirect macro expansion for string
Current code cannot handle string #define macros if those macros are
defined via other macros. This CL solve the issue.
Updates #18720
Change-Id: Ibed0773d10db3d545bb246b97e81c0d19e3af3d5
Reviewed-on: https://go-review.googlesource.com/41312 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Brad Fitzpatrick [Tue, 23 May 2017 05:22:25 +0000 (05:22 +0000)]
cmd/go: include GOARM and GO386 in computed build ID
Now:
$ GOARCH=arm GOARM=5 go install -x cmd/go
... followed by:
$ GOARCH=arm GOARM= go install -x cmd/go
... actually does work. Previously the second "go install" would reuse
the cached binaries from the GOARM=5 command and not rebuild.
(Or vice versa from GOARM= to GOARM=5)
Wade Simmons [Tue, 23 May 2017 00:31:17 +0000 (20:31 -0400)]
crypto/rand: only read necessary bytes for Int
We only need to read the number of bytes required to store the value
"max - 1" to generate a random number in the range [0, max).
Before, there was an off-by-one error where an extra byte was read from
the io.Reader for inputs like "256" (right at the boundary for a byte).
There was a similar off-by-one error in the logic for clearing bits and
thus for any input that was a power of 2, there was a 50% chance the
read would continue to be retried as the mask failed to remove a bit.
Austin Clements [Thu, 18 May 2017 18:50:12 +0000 (14:50 -0400)]
runtime: remove unused copies of special stack guards
There are two copies each of the stackPreempt/_StackPreempt and
stackFork/_StackFork constants. Remove the ones left over from C that
are no longer used.
Austin Clements [Thu, 18 May 2017 18:35:53 +0000 (14:35 -0400)]
runtime: fix stackCache=0 debug mode
Setting stackCache to 0 to disable stack caches for debugging hasn't
worked for a long time. It causes stackalloc to fall back to full span
allocation, round sub-page stacks down to 0 pages, and blow up.
Fix this debug mode so it disables the per-P caches, but continues to
use the global stack pools for small stacks, which correctly handle
sub-page stacks. While we're here, rename stackCache to stackNoCache
so it acts like the rest of the stack allocator debug modes where "0"
is the right default value.
Fixes #17291.
Change-Id: If401c41cee3448513cbd7bb2e9334a8efab257a7
Reviewed-on: https://go-review.googlesource.com/43637 Reviewed-by: Keith Randall <khr@golang.org>
Austin Clements [Thu, 18 May 2017 17:59:00 +0000 (13:59 -0400)]
runtime: fix stackFromSystem returning memory
The stackFromSystem debug mode has two problems:
1) It rounds the stack allocation to _PageSize. If the physical page
size is >8K, this can cause unmapping the memory later to either
under-unmap or over-unmap.
2) It doesn't return the rounded-up allocation size to its caller, so
when we later unmap the memory, we may pass the wrong length.
Fix these problems by rounding the size up to the physical page size
and putting that rounded-up size in the returned stack bounds.
Fixes #17289.
Change-Id: I6b854af3b06bb16e3750798397bb5e2a722ec1cb
Reviewed-on: https://go-review.googlesource.com/43636 Reviewed-by: Keith Randall <khr@golang.org>
Austin Clements [Mon, 22 May 2017 19:53:49 +0000 (15:53 -0400)]
runtime: don't corrupt arena bounds on low mmap
If mheap.sysAlloc doesn't have room in the heap arena for an
allocation, it will attempt to map more address space with sysReserve.
sysReserve is given a hint, but can return any unused address range.
Currently, mheap.sysAlloc incorrectly assumes the returned region will
never fall between arena_start and arena_used. If it does,
mheap.sysAlloc will blindly accept the new region as the new
arena_used and arena_end, causing these to decrease and make it so any
Go heap above the new arena_used is no longer considered part of the
Go heap. This assumption *used to be* safe because we had all memory
between arena_start and arena_used mapped, but when we switched to an
arena_start of 0 on 32-bit, it became no longer safe.
Most likely, we've only recently seen this bug occur because we
usually start arena_used just above the binary, which is low in the
address space. Hence, the kernel is very unlikely to give us a region
before arena_used.
Since mheap.sysAlloc is a linear allocator, there's not much we can do
to handle this well. Hence, we fix this problem by simply rejecting
the new region if it isn't after arena_end. In this case, we'll take
the fall-back path and mmap a small region at any address just for the
requested memory.
Fixes #20259.
Change-Id: Ib72e8cd621545002d595c7cade1e817cfe3e5b1e
Reviewed-on: https://go-review.googlesource.com/43870 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
Joe Richey joerichey@google.com [Mon, 22 May 2017 21:36:43 +0000 (14:36 -0700)]
crypto/rand: use blocking getrandom call on Linux when supported
By changing getRandomLinux to immediately use the getrandom() syscall
without GRND_NONBLOCK, we now only fall back to reading from
/dev/urandom on Linux if the kernel does not support the getrandom()
syscall. This means reads for crypto/rand will now block if the kernel
has insufficient entropy on Linux kernels after v3.16.
Before, if the kernel had insufficient entropy, it would fall back to
reading from /dev/urandom. This would potentially return predictable
data.
Alex Brainman [Mon, 22 May 2017 07:17:39 +0000 (17:17 +1000)]
os/exec: ignore some pipe write errors on windows
This change is windows version of CL 12152.
It also extends test to cover scenarios reported on issue #20445.
Some source files copied and renamed to make code clearer.
Volker Dobler [Wed, 22 Feb 2017 13:40:17 +0000 (14:40 +0100)]
net/http: send more cookie values in double quotes
According to RFC 6255 a cookie value may contain neither spaces " "
nor commas ",". But browsers seem to handle these pretty well and such
values are not uncommon in the wild so we do allow spaces and commas
in cookie values too. Up to now we use the double-quoted wire format
only for cookie values with leading and/or trailing spaces and commas.
Values with internal spaces/commas are sent without the optional double
quotes. This seems to be a problem for some agents.
This CL changes the behaviour for cookie values with spaces or commas:
Such values are always sent in double quotes. This should not have
any impact on existing agents and the increases of data transmitted
is negligible.
Tristan Colgate [Sat, 20 May 2017 13:50:06 +0000 (14:50 +0100)]
net/http/httputil: ReverseProxy should pass on unannounced Trailers
Trailers that are not announced in the Trailer must be passed on to
the downstream client.
Rather than iterate over each and find missing trailer values,
this re-adds all trailers to the headers if there is a disparity
between the number of announced trailers and the final number.
Daniel Theophanes [Mon, 22 May 2017 02:51:46 +0000 (19:51 -0700)]
database/sql: allow Stmt Query and Exec methods to open a new conn
Query and Exec functions on DB first attempt to get a cached
connection before requesting the connection pool to ignore
the cache and get a new connection. This change aligns Stmt to
that behavior as well.
Steven Hartland [Thu, 16 Mar 2017 12:08:35 +0000 (12:08 +0000)]
mime/multipart: Allow ReadForm to process large non-file parts
Allow the memory limit passed into ReadForm to be used as the
memory limit for processing non-file form data as well as file
form data, rather than the existing behaviour of the memory limit
only applying to the file parts and the non-file parts being
arbitrarily limited to 10MB.
This ensures backwards compatibility while still providing the
user with control over the amount of non-file data that can be
processed instead of enforcing an arbitrary 10MB limit.
Ben Shi [Wed, 17 May 2017 07:00:27 +0000 (07:00 +0000)]
cmd/internal/obj/arm: report invalid .S/.P/.W suffix in ARM instructions
Many instructions can not have a .S suffix, such as MULS, SWI, CLZ,
CMP, STREX and others. And so do .P and .W suffixes. Even wrong
assembly code is generated for some instructions with invalid
suffixes.
This patch tries to simplify .S/.W/.P checks. And a wrong assembly
test for arm is added.
Brad Fitzpatrick [Fri, 19 May 2017 19:39:20 +0000 (19:39 +0000)]
doc: remove mentions of yacc tool
It was removed in CL 27325.
Fixes #20431
Change-Id: I6842851444186e19029d040f61fdf4f87a3103a6
Reviewed-on: https://go-review.googlesource.com/43771 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Martí [Wed, 26 Apr 2017 11:04:08 +0000 (12:04 +0100)]
cmd/compile: make duplicate expr cases readable
Instead of just printing the value, print the original node to make the
error more human-friendly. Also print the value if its string form is
different than the original node, to make sure it's obvious what value
was duplicated.
This means that "case '@', '@':", which used to print:
duplicate case 64 in switch
Will now print:
duplicate case '@' (value 64) in switch
Factor this logic out into its own function to reuse it in range cases
and any other place where we might want to print a node and its value in
the future.
Also needed to split the errorcheck files because expression switch case
duplicates are now detected earlier, so they stop the compiler before it
gets to generating the AST and detecting the type switch case
duplicates.
Fixes #20112.
Change-Id: I9009b50dec0d0e705e5de9c9ccb08f1dce8a5a99
Reviewed-on: https://go-review.googlesource.com/41852
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Austin Clements [Thu, 18 May 2017 20:56:48 +0000 (16:56 -0400)]
runtime: use pselect6 for usleep on linux/amd64 and linux/arm
Android O black-lists the select system call because its libc, Bionic,
does not use this system call. Replace our use of select with pselect6
(which is allowed) on the platforms that support targeting Android.
linux/arm64 already uses pselect6 because there is no select on arm64,
so only linux/amd64 and linux/arm need changing. pselect6 has been
available since Linux 2.6.16, which is before Go's minimum
requirement.
Mikio Hara [Mon, 13 Feb 2017 21:38:10 +0000 (06:38 +0900)]
net: make {TCP,UDP,IP,Unix}Conn compliant of syscall.Conn interface
This change makes {TCP,UDP,IP,Unix}Conn types compliant of
syscall.Conn interface and adds type rawConn as an implementation of
syscall.RawConn interface.
By this change, the long-standing issues regarding unsupported socket
options and system calls can be solved partly and the broken x/net
packages due to https://go-review.googlesource.com/36799 can be
repaired.
Mikio Hara [Mon, 13 Feb 2017 20:55:41 +0000 (05:55 +0900)]
internal/poll: add RawControl, RawRead and RawWrite methods to FD
This change adds RawControl, RawRead and RawWrite methods to type FD
to make the runtime-integrated network poller work together with a
user-defined function. The methods are used via the net package from
external packages and type FD is considered as an implementation of
syscall.Conn and syscall.RawConn interfaces.
Updates #19435.
Change-Id: I4ad04b10ffddb2b54fa8d70587440960d73c0a2d
Reviewed-on: https://go-review.googlesource.com/37038
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Mikio Hara [Tue, 7 Mar 2017 20:52:07 +0000 (05:52 +0900)]
syscall: add Conn and RawConn interfaces
This change adds Conn and RawConn interfaces which can be used to
manipulate raw network connection end points typically represented as
socket descriptors.
Alex Brainman [Tue, 9 May 2017 06:50:41 +0000 (16:50 +1000)]
os: make windows Stat as fast as Lstat for files and directories
Recent CL 41834 made windows Stat work for all symlinks.
But CL 41834 also made Stat slow.
John Starks sugested
(see https://github.com/golang/go/issues/19922#issuecomment-300031421)
to use GetFileAttributesEx for files and directories instead.
This makes Stat as fast as at go1.9.
David du Colombier [Fri, 19 May 2017 00:13:50 +0000 (02:13 +0200)]
cmd/compile: skip TestScopeRanges on Plan 9
TestScopeRanges has been added in CL 40095. This
test is failing on Plan 9 because executables don't
have a DWARF symbol table.
Fixes #20418.
Change-Id: I6dd3baa636998134ccd042203c8b5c3199a4d6e1
Reviewed-on: https://go-review.googlesource.com/43670
Run-TryBot: David du Colombier <0intro@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Alessandro Arzilli [Tue, 2 May 2017 14:46:01 +0000 (16:46 +0200)]
cmd/compile: output DWARF lexical blocks for local variables
Change compiler and linker to emit DWARF lexical blocks in .debug_info
section when compiling with -N -l.
Version of debug_info is updated from DWARF v2 to DWARF v3 since
version 2 does not allow lexical blocks with discontinuous PC ranges.
Remaining open problems:
- scope information is removed from inlined functions
- variables records do not have DW_AT_start_scope attributes so a
variable will shadow other variables with the same name as soon as its
containing scope begins, even before its declaration.
A number of issues in decoder.Read and newlineFilteringReader.Read were
preventing errors from the reader supplying the encoded data from being
propagated to the caller. Fixing these issues revealed some additional
problems in which valid decoded data was not always returned to the user
when errors were actually propagated.
This commit fixes both the error propagation and the lost decoded data
problems. It also adds some new unit tests to ensure errors are handled
correctly by decoder.Read. The new unit tests increase the test coverage
of this package from 96.2% to 97.9%.
Daniel Theophanes [Thu, 23 Mar 2017 20:17:59 +0000 (13:17 -0700)]
database/sql: allow drivers to support custom arg types
Previously all arguments were passed through driver.IsValid.
This checked arguments against a few fundamental go types and
prevented others from being passed in as arguments.
The new interface driver.NamedValueChecker may be implemented
by both driver.Stmt and driver.Conn. This allows
this new interface to completely supersede the
driver.ColumnConverter interface as it can be used for
checking arguments known to a prepared statement and
arbitrary query arguments. The NamedValueChecker may be
skipped with driver.ErrSkip after all special cases are
exhausted to use the default argument converter.
In addition if driver.ErrRemoveArgument is returned
the argument will not be passed to the query at all,
useful for passing in driver specific per-query options.
Add a canonical Out argument wrapper to be passed
to OUTPUT parameters. This will unify checks that need to
be written in the NameValueChecker.
The statement number check is also moved to the argument
converter so the NamedValueChecker may remove arguments
passed to the query.
cmd/compile: search for remaining WB ops from end to beginning
The writebarrier pass processes WB ops from beginning to end,
replacing them by other values.
But it also checks whether there are more ops to process
by walking from beginning to end.
This is quadratic, so walk from end to beginning instead.
This speeds up compiling the code in issue 13554:
name old time/op new time/op delta
Pkg 11.9s ± 2% 8.3s ± 3% -29.88% (p=0.000 n=18+17)
cmd/compile: don't break up contiguous blocks in looprotate
looprotate finds loop headers and arranges for them to be placed
after the body of the loop. This eliminates a jump from the body.
However, if the loop header is a series of contiguously laid out blocks,
the rotation introduces a new jump in that series.
This CL expands the "loop header" to move to be the entire
run of contiguously laid out blocks in the same loop.
This shrinks object files a little, and actually speeds up
the compiler noticeably. Numbers below.
Fannkuch performance seems to vary a lot by machine. On my laptop:
name old time/op new time/op delta
Fannkuch11-8 2.89s ± 2% 2.85s ± 3% -1.22% (p=0.000 n=50+50)
This has a significant affect on the append benchmarks in #14758:
name old time/op new time/op delta
Foo-8 312ns ± 3% 276ns ± 2% -11.37% (p=0.000 n=30+29)
Bar-8 565ns ± 2% 456ns ± 2% -19.27% (p=0.000 n=27+28)
Alex Brainman [Tue, 2 May 2017 06:03:45 +0000 (16:03 +1000)]
cmd/link: make sure that runtime.epclntab lives in .text section
Second attempt to fix #14710.
CL 35272 already tried to fix this issue. But CL 35272 assumed
that runtime.epclntab type is STEXT, while it is actually SRODATA.
This CL uses Symbol.Sect.Seg to determine if symbol is part
of Segtext or Segdata.
Fixes #14710
Change-Id: Ic6b6f657555c87a64d2bc36cc4c07ab0591d00c4
Reviewed-on: https://go-review.googlesource.com/42390
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Austin Clements [Wed, 17 May 2017 18:27:28 +0000 (14:27 -0400)]
runtime/pprof: don't produce 0 location in count profiles
profileBuilder.locForPC returns 0 to mean "no location" because 0 is
an invalid location index. However, the code to build count profiles
doesn't check the result of locForPC, so this 0 location index ends up
in the profile's location list. This, in turn, causes problems later
when we decode the profile because it puts a nil *Location in the
sample's location slice, which can later lead to a nil pointer panic.
Fix this by making printCountProfile correctly discard the result of
locForPC if it returns 0. This makes this call match the other two
calls of locForPC.
Carl Henrik Lunde [Wed, 17 May 2017 17:37:33 +0000 (19:37 +0200)]
runtime/pprof: deflake TestGoroutineCounts
TestGoroutineCounts was flaky when running on a system under load.
This happened on three builds the last couple of days.
Fix this by running this test with a single operating system thread, so
we do not depend on the operating system scheduler. 50 000 tests ran
without failure with the new version, the old version failed 0.5% of the
time.
fuseBlockPlain was accidentally quadratic.
If you had plain blocks b1 -> b2 -> b3 -> b4,
each containing single values v1, v2, v3, and v4 respectively,
fuseBlockPlain would move v1 from b1 to b2 to b3 to b4,
then v2 from b2 to b3 to b4, etc.
There are two obvious fixes.
* Look for runs of blocks in fuseBlockPlain
and handle them in a single go.
* Fuse from end to beginning; any given value in a run
of blocks to fuse then moves only once.
The latter is much simpler, so that's what this CL does.
Somewhat surprisingly, this change does not pass toolstash-check.
The resulting set of blocks is the same,
and the values in them are the same,
but the order of values in them differ,
and that order of values (while arbitrary)
is enough to change the compiler's output.
This may be due to #20178; deadstore is the next pass after fuse.
Adding basic sorting to the beginning of deadstore
is enough to make this CL pass toolstash-check:
for _, b := range f.Blocks {
obj.SortSlice(b.Values, func(i, j int) bool { return b.Values[i].ID < b.Values[j].ID })
}
Happily, this CL appears to result in better code on average,
if only by accident. It cuts 4k off of cmd/go; go1 benchmarks
are noisy as always but don't regress (numbers below).
No impact on the standard compilebench benchmarks.
For the code in #13554, this speeds up compilation dramatically:
name old time/op new time/op delta
Pkg 53.1s ± 2% 12.8s ± 3% -75.92% (p=0.008 n=5+5)
name old user-time/op new user-time/op delta
Pkg 55.0s ± 2% 14.9s ± 3% -73.00% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
Pkg 2.04GB ± 0% 2.04GB ± 0% +0.18% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Pkg 6.21M ± 0% 6.21M ± 0% ~ (p=0.222 n=5+5)
name old object-bytes new object-bytes delta
Pkg 28.4M ± 0% 28.4M ± 0% +0.00% (p=0.008 n=5+5)
name old export-bytes new export-bytes delta
Pkg 208 ± 0% 208 ± 0% ~ (all equal)
cmd/compile: seed rand with time when race enabled
When the race detector is enabled,
the compiler randomizes the order in which functions are compiled,
in an attempt to shake out bugs.
But we never re-seed the rand source, so every execution is identical.
Fix that to get more coverage.
Hiroshi Ioka [Tue, 16 May 2017 12:52:41 +0000 (21:52 +0900)]
cmd/cgo: support large unsigned macros
Currently, cgo converts integer macros into int64 if it's possible.
As a result, some macros which satisfy
math.MaxInt64 < x <= math.MaxUint64
will lose their original values.
This CL introduces the new probe to check signs,
so we can handle signed ints and unsigned ints separately.
Fixes #20369
Change-Id: I002ba452a82514b3a87440960473676f842cc9ee
Reviewed-on: https://go-review.googlesource.com/43476 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ian Lance Taylor [Wed, 17 May 2017 01:02:56 +0000 (18:02 -0700)]
cmd/go: don't fail on missing runtime/internal/sys/zversion.go
The generated file runtime/internal/sys/zversion.go is deleted by
`go tool cmd dist clean` as part of running clean.bash. Don't treat
a missing file as a reason to stop running the go tool; just treat
is as meaning that runtime/internal/sys is stale.
No test because I don't particularly want to clobber $GOROOT.
Fixes #20385.
Change-Id: I5251a99542cc93c33f627f133d7118df56e18af1
Reviewed-on: https://go-review.googlesource.com/43559
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Ian Lance Taylor [Wed, 17 May 2017 00:41:42 +0000 (17:41 -0700)]
os: fix handling of ErrShortWrite in (*File).Write
Restore the handling of io.ErrShortWrite in (*File).Write:
if we write less than the requested amount, and there is no error from
the syscall, then return io.ErrShortWrite.
I can't figure out how to write a test for this. It would require a
non-pollable file (not a pipe) on a device that is almost but not
quite entirely full. The original code (https://golang.org/cl/36800043,
committed as part of https://golang.org/cl/36930044) does not have a test.
Fixes #20386.
Change-Id: Ied7b411e621e1eaf49f864f8db90069f276256f5
Reviewed-on: https://go-review.googlesource.com/43558
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Sean Chittenden [Sun, 14 May 2017 16:42:35 +0000 (09:42 -0700)]
runtime: mmap(2) on Solaris & Illumos can return EAGAIN.
In low memory situations mmap(2) on Illumos[2] can return EAGAIN when it
is unable to reserve the necessary space for the requested mapping. Go
was not previously handling this correctly for Illumos and would fail to
recognize it was in a low-memory situation, the result being the program
would terminate with a panic instead of running the GC.
Robert Griesemer [Mon, 15 May 2017 18:46:05 +0000 (11:46 -0700)]
go/types: fewer spurious "declared but not used" follow-on errors
Mark variables as used even when they appear within an expression
context which we can't type-check; e.g., because the expression is
erroneous, or comes from an import "C" declaration.
Fixes #20358.
Change-Id: Ib28cc78d3867c597c7a1ace54de09ada02f5b33a
Reviewed-on: https://go-review.googlesource.com/43500 Reviewed-by: Alan Donovan <adonovan@google.com>
David Chase [Mon, 15 May 2017 17:49:30 +0000 (13:49 -0400)]
cmd/compile: don't attach lines to SB, SP, similar constants
Attaching positions to SB, SP, initial mem can result in
less-good line-numbering when compiled for debugging.
This "fix" also removes source position from a zero-valued
struct (but not from its fields) and from a zero-length
array constant.
This may be a general problem for constants in entry blocks.
Fixes #20367.
Change-Id: I7e9df3341be2e2f60f127d35bb31e43cdcfce9a1
Reviewed-on: https://go-review.googlesource.com/43531
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Dmitri Shuralyov [Mon, 1 May 2017 21:28:33 +0000 (17:28 -0400)]
go/build: return partial information on Import error, for local import paths
Documentation of build.Import says:
// If the path is a local import path naming a package that can be imported
// using a standard import path, the returned package will set p.ImportPath
// to that path.
// ...
// If an error occurs, Import returns a non-nil error and a non-nil
// *Package containing partial information.
That behavior was previously untested, and broken by change in CL 33158.
Fix that by avoiding returning early on error for local import paths.
First, gather partial information, and only then check that the p.Dir
directory exists.
Add tests for this behavior.
Fixes #19769.
Fixes #20175 (duplicate of #19769).
Updates #17863.
runtime/pprof: expand inlined frames in symbolized proto profiles
Currently proto symbolization uses runtime.FuncForPC and assumes each
PC maps to a single frame. This isn't true in the presence of inlining
(even with leaf-only inlining this can get incorrect results).
Change PC symbolization to use runtime.CallersFrames to expand each PC
to all of the frames at that PC.
Austin Clements [Tue, 9 May 2017 02:31:41 +0000 (22:31 -0400)]
runtime/pprof: clean up call/return PCs in memory profiles
Proto profile conversion is inconsistent about call vs return PCs in
profile locations. The proto defines locations to be call PCs. This is
what we do when proto-izing CPU profiles, but we fail to convert the
return PCs in memory and count profile stacks to call PCs when
converting them to proto locations.
Fix this in the heap and count profile conversion functions.
TestConvertMemProfile also hard-codes this failure to convert from
return PCs to call PCs, so fix up the addresses in the synthesized
profile to be return PCs while checking that we get call PCs out of
the conversion.
Alex Brainman [Thu, 11 May 2017 01:55:59 +0000 (11:55 +1000)]
cmd/link: actually generate .debug_gdb_scripts section on windows
Adjust finddebugruntimepath to look for runtime/debug.go file
instead of runtime/runtime.go. This actually finds runtime.GOMAXPROCS
in every Go executable (including windows).
I also included "-Wl,-T,fix_debug_gdb_scripts.ld" parameter to gcc
invocation on windows to work around gcc bug (see #20183 for details).
This CL only fixes windows -buildmode=exe, buildmode=c-archive
is still broken.
Thanks to Egon Elbre and Nick Clifton for investigation.
Fixes #20183
Fixes #20218
Change-Id: I5369a4db3913226aef3d9bd6317446856b0a1c34
Reviewed-on: https://go-review.googlesource.com/43331 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Make yellow the last highlight color rather than the first.
Yellow is also the color that Chrome uses to highlight
search results, which can be confusing.
Also, when Night Shift is on on macOS,
yellow highlighting is completely invisible.
I suppose should be sleeping instead.
cmd/compile: don't update outer variables after capturevars is complete
When compiling concurrently, we walk all functions before compiling
any of them. Walking functions can cause variables to switch from
being non-addrtaken to addrtaken, e.g. to prepare for a runtime call.
Typechecking propagates addrtaken-ness of closure variables to
their outer variables, so that capturevars can decide whether to
pass the variable's value or a pointer to it.
When all functions are compiled immediately, as long as the containing
function is compiled prior to the closure, this propagation has no effect.
When compilation is deferred, though, in rare cases, this results in
a change in the addrtaken-ness of a variable in the outer function,
which in turn changes the compiler's output.
(This is rare because in a great many cases, a temporary has been
introduced, insulating the outer variable from modification.)
But concurrent compilation must generate identical results.
To fix this, track whether capturevars has run.
If it has, there is no need to update outer variables
when closure variables change.
Capturevars always runs before any functions are walked or compiled.
The remainder of the changes in this CL are to support the test.
In particular, -d=compilelater forces the compiler to walk all
functions before compiling any of them, despite being non-concurrent.
This is useful because -live is fundamentally incompatible with
concurrent compilation, but we want -c=1 to have no behavior changes.
Lars Jeppesen [Sat, 29 Apr 2017 21:25:34 +0000 (23:25 +0200)]
archive/tar: remove file type bits from mode field
When writing tar files by using the FileInfoHeader
the type bits was set in the mode field of the header
This is not correct according to the standard (GNU/Posix) and
other implementations.
Fixed #20150
Change-Id: I3be7d946a1923ad5827cf45c696546a5e287ebba
Reviewed-on: https://go-review.googlesource.com/42093 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Matt Harden [Mon, 20 Feb 2017 13:58:55 +0000 (05:58 -0800)]
net: allow Resolver to use a custom dialer
In some cases it is desirable to customize the way the DNS server is
contacted, for instance to use a specific LocalAddr. While most
operating-system level resolvers do not allow this, we have the
opportunity to do so with the Go resolver. Most of the code was
already in place to allow tests to override the dialer. This exposes
that functionality, and as a side effect eliminates the need for a
testing hook.
Austin Clements [Thu, 11 May 2017 19:28:39 +0000 (15:28 -0400)]
runtime: doubly fix "double wakeup" panic
runtime.gchelper depends on the non-atomic load of work.ndone
happening strictly before the atomic add of work.nwait. Until very
recently (commit 978af9c2db, fixing #20334), the compiler reordered
these operations. This created a race since work.ndone can change as
soon as work.nwait is equal to work.ndone. If that happened, more than
one gchelper could attempt to wake up the work.alldone note, causing a
"double wakeup" panic.
This was fixed in the compiler, but to make this code less subtle,
make the load of work.ndone atomic. This clearly forces the order of
these operations, ensuring the race doesn't happen.