The Go ecosystem provides many tools to make Go
development more productive and seamless. Document
the availability of the editor plugins and IDEs,
add an overview of feature support and screencasts.
Currently we trace mark assists even if they're satisfied entirely by
stealing. This means even if background marking is keeping up with
allocation, we'll still emit a trace event every N bytes of
allocation. The event will be a few microseconds, if that, but they're
frequent enough that, when zoomed out in the trace view, it looks like
all of the time is spent in mark assists even if almost none is.
Change this so we only emit a trace event if the assist actually has
to do assisting. This makes the traces of these events far more
useful.
Michael Munday [Mon, 17 Jul 2017 11:07:28 +0000 (07:07 -0400)]
cmd/compile: fix unaligned loads/stores to global variables on s390x
Load/store-merging and move optimizations can result in unaligned
memory accesses. This is fine so long as the load/store instruction
used does not take a relative offset. In the SSA rules this means we
must not merge (MOVDaddr (SB)) ops into loads/stores unless we can
guarantee the alignment of the target.
tSweepTerm and pauseStart are supposed to be when STW was triggered,
but right now they're captured a bit before STW. Move these down to
immediately before we trigger STW.
Currently, Windows stacks are either 128kB or 2MB depending on whether
the binary uses cgo. This is because we assume that Go system stacks
and the small amount of C code invoked by the standard library can
operate within smaller stacks, but general Windows C code assumes
larger stacks.
However, it's easy to call into arbitrary C code using the syscall
package on Windows without ever importing cgo into a binary. Such
binaries need larger system stacks even though they don't use cgo.
Fix this on 64-bit by increasing the system stack size to 2MB always.
This only costs address space, which is free enough on 64-bit to not
worry about. We keep (for now) the existing heuristic on 32-bit, where
address space comes at more of a premium.
Add Adam Kisala (individual CLA)
Add Aditya Mukerjee (individual CLA)
Add Akhil Indurti (individual CLA)
Add André Carvalho (individual CLA)
Add Andy Walker (individual CLA)
Add Awn Umar (individual CLA)
Add Bastian Ike (individual CLA)
Add Brian Downs (individual CLA)
Add Cody Oss (individual CLA)
Add Costin Chirvasuta (corporate CLA for Google Inc.)
Add Dan Ballard (individual CLA)
Add Dong-hee Na (individual CLA)
Add Dylan Waits (individual CLA)
Add Evan Hicks (individual CLA)
Add Fannie Zhang (corporate CLA for ARM Ltd.)
Add Francisco Rojas (individual CLA)
Add Gabriel Nicolas Avellaneda (individual CLA)
Add Gabríel Arthúr Pétursson (individual CLA)
Add Greg Poirier (individual CLA)
Add Iccha Sethi (individual CLA)
Add Ivan Moscoso (individual CLA)
Add Jamie Kerr (individual CLA)
Add Joe Kyo (individual CLA)
Add Joey Geiger (individual CLA)
Add John R. Lenton (individual CLA)
Add Johnny Luo (individual CLA)
Add Josh Roppo (individual CLA)
Add Kate Manson (individual CLA)
Add Leo Rudberg (corporate CLA for Google Inc.)
Add Ma Peiqi (individual CLA)
Add Martynas Budriūnas (corporate CLA for Google Inc.)
Add Maryan Hratson (individual CLA)
Add Michael Edwards (individual CLA)
Add Michael Hendricks (individual CLA)
Add Pablo Santiago Blum de Aguiar (individual CLA)
Add Pat Moroney (individual CLA)
Add Shi Han Ng (individual CLA)
Add Steven Buss (corporate CLA for Google Inc.)
Add Suzy Mueller (corporate CLA for Google Inc.)
Add Taro Aoki (individual CLA)
Add Tim Heckman (individual CLA)
Add Tony Walker (individual CLA)
Add Yasha Bubnov (individual CLA)
Updates #12042
Change-Id: Iee063dd6c5a39de16907acfb5af87e81a05ab417
Reviewed-on: https://go-review.googlesource.com/49351 Reviewed-by: Andrew Bonventre <andybons@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: Ia3e351169a4ebe6db5e5f37b668f23dc8c992c78
Reviewed-on: https://go-review.googlesource.com/48877
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
encoding/json: fix indentation in Decode stream example
The existing example for Decoder.Decode (Stream) had excessive
indentation in the godoc interface for the const jsonStream,
making it hard to read. This fixes the indentation in the
example_test.go to improve the readability in godoc.
Ivan Moscoso [Sat, 15 Jul 2017 17:53:48 +0000 (11:53 -0600)]
regexp: clarify example for 'FindString'
Clarifying that FindString only provides left-most match
Change-Id: Ic6ecec12cca759fd4b3565ef5901a110843ffd56
Reviewed-on: https://go-review.googlesource.com/48609 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Pablo Santiago Blum de Aguiar [Sat, 15 Jul 2017 18:13:45 +0000 (12:13 -0600)]
strings: add example for IndexByte
Change-Id: Ib6a59735381ce744553f1ac96eeb65a194c8da10
Reviewed-on: https://go-review.googlesource.com/48860 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I69d1359d8868d4c5b173e4d831e38cea7dfeb713
Reviewed-on: https://go-review.googlesource.com/48859 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
While there's an example for SectionReader.Seek, if someone is
seeking documentation specifically about Seeker.Seek, they may
not immediately find the SectionReader example. Offset and whence
may not be entirely intuitive to new developers either, so include
examples of both positive/negative offsets and SeekStart/SeekEnd.
Change-Id: If581bd4e9d9b4421e2ae20582c596fccb73d9aed
Reviewed-on: https://go-review.googlesource.com/48866 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Sat, 15 Jul 2017 13:33:24 +0000 (06:33 -0700)]
cmd/dist: don't run moved_goroot test on android, iOS, Plan 9
Fails on iOS because CC_FOR_TARGET points to clangwrap.sh in the
original GOROOT. We could fix that but it doesn't seem worth it.
Fails on Android with "exec format error". I'm not sure why but I
doubt it is interesting.
Fails on Plan 9 because the original GOROOT is being preserved in some
unknown way. This is issue #21016.
Updates #21016
Change-Id: I4e7115d734fc7bf21e5a2ba18fb6ad0bfa31c735
Reviewed-on: https://go-review.googlesource.com/48650
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: David du Colombier <0intro@gmail.com> Reviewed-by: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ian Lance Taylor [Wed, 28 Jun 2017 19:47:08 +0000 (12:47 -0700)]
reflect: make StructOf panic for methods that don't work
When StructOf is used with an anonymous field that has methods, and
that anonymous field is not the first field, the methods we generate
are incorrect because they do not offset to the field as required.
If we encounter that case, panic rather than doing the wrong thing.
Fixes #20824
Updates #15924
Change-Id: I3b0901ddbc6d58af5f7e84660b5e3085a431035d
Reviewed-on: https://go-review.googlesource.com/47035
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
net/http: clarify Handler panic behavior across HTTP versions
Updates #18997
Change-Id: Ib1961a4c26b42f99b98b255beb7e2a74b632e0c1
Reviewed-on: https://go-review.googlesource.com/48551 Reviewed-by: Joe Shaw <joe@joeshaw.org> Reviewed-by: Tom Bergan <tombergan@google.com>
Nathaniel Caza [Tue, 14 Feb 2017 03:00:06 +0000 (21:00 -0600)]
crypto/x509: load all trusted certs on darwin (nocgo)
The current implementation ignores certificates that exist
in the login and System keychains.
This change adds the missing System and login keychain
files to the `/usr/bin/security` command in
`execSecurityRoots`. If the current user cannot be
obtained, the login keychain is ignored.
Samuel Tan [Wed, 5 Jul 2017 23:28:41 +0000 (16:28 -0700)]
html/template: use the same escaper across multiple template executions
The escaper contains information about which templates have already been
visited and escaped. This information is necessary to prevent templates
that have already been escaped from being over-escaped. However, since we
currently create a new escaper each time we execute a template, this
information does not persist across multiple template executions.
Fix this by saving an escaper in each template name space which is shared by
all templates in that name space.
While there, fix error message formatting for an escaping unit test.
Fixes #20842
Change-Id: Ie392c3e7ce0e0a9947bdf56c99e926e7c7db76e4
Reviewed-on: https://go-review.googlesource.com/47256 Reviewed-by: Mike Samuel <mikesamuel@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
David Chase [Wed, 14 Jun 2017 21:28:28 +0000 (17:28 -0400)]
cmd/compile: fix phi-function updates for preemptible loops
Previous code failed to account for particular control flow
involving nested loops when updating phi function inputs.
Fix involves:
1) remove incorrect shortcut
2) generate a "better" order for children in dominator tree
3) note inner-loop updates and check before applying
outer-loop updates.
Ian Lance Taylor [Thu, 13 Jul 2017 22:07:07 +0000 (15:07 -0700)]
internal/poll: don't wait for unpollable files
If we get an EAGAIN error on an unpollable file, don't try to wait for
it to be ready; just return EAGAIN.
It's possible that we should instead ensure that when Stdin is a pipe
in non-blocking mode, we wait for data to appear. For now take the
conservative approach of doing what we did in previous releases.
Based on https://golang.org/cl/47555 by Totoro W.
Fixes #20915
Change-Id: Icc9e97a5a877b0a3583ec056c35412d1afab62d1
Reviewed-on: https://go-review.googlesource.com/48490
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Ian Lance Taylor [Thu, 13 Jul 2017 15:10:43 +0000 (08:10 -0700)]
testing: roll back CL 44352 (show in-progress tests upon SIGINT)
CL 44352 changed the behavior of SIGINT, which can break tests that
themselves use SIGINT. I think we can only implement this if the
testing package has a way to know whether the code under test is using
SIGINT, but os/signal does not provide an API for that. Roll back for
1.9 and think about this again for 1.10.
Updates #19397
Change-Id: I021c314db2b9d0a80d0088b120a6ade685459990
Reviewed-on: https://go-review.googlesource.com/48370
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
doc: note the StripPrefix change in Go 1.9 release notes
Fixes #20948
Change-Id: I222bf101a5c1bdc5cbb0970949070c4b58b9b83b
Reviewed-on: https://go-review.googlesource.com/48190 Reviewed-by: Ian Lance Taylor <iant@golang.org>
SysV semaphore undo lists should be shared by threads, just like
several other resources listed in cloneFlags. Currently we don't do
this, but it probably doesn't affect anything because 1) probably
nobody uses SysV semaphores from Go and 2) Go-created threads never
exit until the process does. Beyond being the right thing to do,
user-level QEMU requires this flag because it depends on glibc to
create new threads and glibc uses this flag.
doc: soften the NetBSD warning now that a unverified fix is being released
To-be-released NetBSD 7.1.1 reportedly fixes the kernel panic that was
affecting our builders and is being released because of Go's warning.
So, soften our warning.
7.1.1 might work, but I can't get a builder up and running to verify
yet as it appears that Anita either doesn't support it yet, or the
NetBSD CDN doesn't have the files yet.
Change-Id: Ifaafc566879a6afdf1174e545ad10e240da427e8
Reviewed-on: https://go-review.googlesource.com/47970 Reviewed-by: Ian Lance Taylor <iant@golang.org>
TestStackGrowth is currently a parallel test. However, it depends on a
20 second timeout, which is already dubious in a parallel test, and
became really problematic on slow builders when runtime.GC switched to
triggering concurrent GC instead of STW GC. Before that change, the
test spent much of its time in STW GC, so it wasn't *really* parallel.
After that change, it was competing with all of the other parallel
tests and GC likely started taking ~4 times longer. On most builders
the whole test runs in well under a second, but on the slow builders
that was enough to push it over the 20 second timeout.
Fix this by making the test serial.
Updates #19381 (probably fixes it, but we'll have to wait and see).
cmd/compile: fix lexical scope of escaped variables
When a local variable is moved to the heap the declaration position
should be preserved so that later on we can assign it to the correct
DW_TAG_lexical_block.
Michael Pratt [Sat, 8 Jul 2017 23:47:16 +0000 (16:47 -0700)]
cmd/link: skip R_ADDR relocs in .rela.plt for internal PIE
ld.addpltsym adds an R_X86_64_JMP_SLOT dynamic relocation to .rela.plt
and uses Addaddrplus to reference the GOT in Elf64_Rela.r_offset.
Addaddrplus results in an R_ADDR relocation, which here we transform
into an R_X86_64_64 dynamic relocation. This is wrong for several
reasons:
1. .rela.plt is not a writable, relro section. It is mapped read-only,
causing the dynamic linker to segfault when it tried to handle the
relocation. This was the immediate cause of internal PIE cgo
crashes.
2. Relocations targetting other reloc sections are, as far as I can
tell, undefined behavior in the ELF spec and are unlikely to be a
good idea.
3. Even if the relocation did work, it isn't what we want. The
relocation, if successfully handled, would have put an absolute
address as the JMP_SLOT offset, but it should be the offset from the
beginning of the binary, just like any other relocation. What we want
is a statically resolved R_ADDR relocation, just as is used below for
the R_X86_64_64 relocation.
Skipping the .rela.plt allows reloc() to handle these R_ADDR
relocations.
With this CL, internal PIE cgo binaries work.
Updates #18968
Change-Id: Ie74e6fe249e88150baa0e340b1cb128cf7f28673
Reviewed-on: https://go-review.googlesource.com/47837 Reviewed-by: Ian Lance Taylor <iant@golang.org>
On a slow or distracted machine, 0.1s is sometimes
not long enough for a non-blocking function call to complete.
This causes rare test flakes.
They can be easily reproduced by reducing the wait time to (say) 100ns.
For non-blocking functions, increase the window from 100ms to 10s.
Using different windows for block and non-blocking functions,
allows us to reduce the time for blocking functions.
The risk here is false negatives, but that risk is low;
this test is run repeatedly on many fast machines,
for which 10ms is ample time.
This reduces the time required to run the test by a factor of 10,
from ~1s to ~100ms.
runtime: use next timer to decide whether to relax
Currently, sysmon waits 60 ms during idle before relaxing. This is
primarily to avoid reducing the precision of short-duration timers. Of
course, if there are no short-duration timers, this wastes 60 ms
running the timer at high resolution.
Improve this by instead inspecting the time until the next timer fires
and relaxing the timer resolution immediately if the next timer won't
fire for a while.
Currently, sysmon relaxes the Windows timer resolution as soon as the
Go process becomes idle. However, if it's going idle because of a
short sleep (< 15.6 ms), this can turn that short sleep into a long
sleep (15.6 ms).
To address this, wait for 60 ms of idleness before relaxing the timer
resolution. It would be better to check the time until the next wakeup
and relax immediately if it makes sense, but there's currently no
interaction between sysmon and the timer subsystem, so adding this
simple delay is a much simpler and safer change for late in the
release cycle.
R11 is callee-save in the C ABI, but the temporary register in the Go
ABI. Currently it's being clobbered by runtime.addmoduledata, which
has to follow the C ABI. The observed effect of this was that
dl_open_worker was returning to a bad PC because after it failed to
restore its SP because it was using R11 as a frame pointer.
A lot of code that uses runtime.Callers makes assumptions about the
result that are not true today under gccgo and will not be true in the
future in gc. This adds a section to the release notes discussing how
to correctly use runtime.Callers.
Gustav Westling [Thu, 6 Jul 2017 18:30:26 +0000 (20:30 +0200)]
encoding/base32: make NoPadding Encoding's DecodedLen return exact size
CL 47341 added support for decoding non-padded messages. But DecodedLen
still returned a multiple of 5 for messages without a padding, even
though it is possible to calculate the len exactly when using NoPadding.
This change makes DecodedLen return the exact number of bytes that
will be written. A change to the decoding logic is also made so that it
can handle this case.
DecodedLen now has the same behaviour as DecodedLen in encoding/base64.
runtime: prevent descheduling while holding rwmutex read lock
Currently only the rwmutex write lock prevents descheduling. The read
lock does not. This leads to the following situation:
1. A reader acquires the lock and gets descheduled.
2. GOMAXPROCS writers attempt to acquire the lock (or at least one
writer does, followed by readers). This blocks all of the Ps.
3. There is no 3. The descheduled reader never gets to run again
because there are no Ps, so it never releases the lock and the system
deadlocks.
Fix this by preventing descheduling while holding the read lock. This
requires also rewriting TestParallelRWMutexReaders to always create
enough GOMAXPROCS and to use non-blocking operations for
synchronization.
Fixes #20903.
Change-Id: Ibd460663a7e5a555be5490e13b2eaaa295fac39f
Reviewed-on: https://go-review.googlesource.com/47632
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
net: don't return IPv4 unspecified addr for Resolve*Addr of [::] or [::]:n
ResolveTCPAddr, ResolveUDPAddr, and ResolveIPAddr return at most one
address. When given a name like "golang.org" to resolve that might
have more than 1 address, the net package has historically preferred
IPv4 addresses, with the assumption that many users don't yet have
IPv6 connectivity and randomly selecting between an IPv4 address and
an IPv6 address at runtime wouldn't be a good experience for IPv4-only
users.
In CL 45088 (78cf0e56) I modified the resolution of the
unspecified/empty address to internally resolve to both IPv6 "::" and
0.0.0.0 to fix issue #18806.
That code has 3 other callers I hadn't considered, though: the
Resolve*Addr functions. Since they preferred IPv4, any Resolve*Addr of
"[::]:port" or "::" (for ResolveIPAddr) would internally resolve both
"::" and 0.0.0.0 and then prefer 0.0.0.0, even though the user was
looking up an IPv6 literal.
Add tests and fix it, not by undoing the fix to #18806 but by
selecting the preference function for Resolve*Addr more explicitly: we
still prefer IPv4, but if the address being looked up was an IPv6
literal, prefer IPv6.
-timeout t
If a test runs longer than t, panic.
The default is 10 minutes (10m).
CL 45816 was supposed to be about clarifying test vs test binary,
and it did add the clarification of referring to "duration d",
but it also introduced incorrect text about timeout 0.
The new text in this CL preserves the good change and
eliminates the incorrect one:
-timeout d
If a test binary runs longer than duration d, panic.
The default is 10 minutes (10m).
Emmanuel Odeke [Wed, 21 Jun 2017 06:35:18 +0000 (00:35 -0600)]
reflect: match MakeMapWithSize docs about initial capacity with spec
Following the spec clarification in CL 40393, copy that text
to reflect docs to state that the initial capacity of MakeMapWithSize
is a hint/approximate.
Austin Clements [Fri, 30 Jun 2017 20:51:07 +0000 (16:51 -0400)]
runtime: clean up mheap.allocLarge
mheap.allocLarge just calls bestFitTreap and is the only caller of
bestFitTreap. Flatten these into a single function. Also fix their
comments: allocLarge claims to return exactly npages but can in fact
return a larger span, and h.freelarge is not in fact indexed by span
start address.
Joe Tsai [Fri, 30 Jun 2017 21:10:48 +0000 (14:10 -0700)]
net/http/httptest: allow creation of Server manually
The Server struct has exported fields, which allows users to manually
create a Server object without using using NewServer or NewTLSServer
and directly call Start or StartTLS on their object.
In order to ensure that manual creation of Server works, the
NewUnstartedServer function should not initialize Server in any way
that the user was not able to do themselves. For example, the setting
of a unexported filed, client, is not something a user can do.
Thus, rather than setting the client field in NewUnstartedServer,
we lazily initialize it when Start or StartTLS is called.
Otherwise, the Server logic can nil panic later when it assumes that this
field has been initialized.
Joe Tsai [Fri, 30 Jun 2017 22:44:25 +0000 (15:44 -0700)]
runtime: allow calling Func.Name on nil pointer
The Func type has allowed calling the Func.Name method on a nil pointer
since Go1.2, where it returned an empty string. A regression caused by
CL/37331 caused this behavior to change. This breaks code that lazily
does runtime.FuncForPC(myPtr).Name() without first checking that myPtr
is actually non-nil.
Shawn Walker-Salas [Fri, 30 Jun 2017 17:57:04 +0000 (10:57 -0700)]
net: fix sendfile for Solaris
If a retryable error such as EAGAIN/EINTR is encountered during a call
to sendfile(), we should not assume that a partial write occurred.
Instead, just like any other platform, we should always try again even
if 0 bytes were written.