David Chase [Wed, 3 Jun 2020 18:21:18 +0000 (14:21 -0400)]
runtime: make runtime-gdb.py tolerant of creatively-named gdb versions
"Fedora" and "Red Hat" are not numbers, it turns out.
Don't rely on version numbers, instead use a regexp to
handle variation across the 2 patterns thus far observed
for gdb-generated Go type names.
Cherry Zhang [Tue, 2 Jun 2020 21:45:57 +0000 (17:45 -0400)]
cmd/internal/goobj2: add referenced symbol names to object file
Currently, for symbols defined in other packages and referenced
by index, we don't record its name in the object file, as the
linker doesn't need the name, only the index. As a consequence,
tools like objdump and nm also don't know the referenced symbol
names and cannot dump it properly.
This CL adds referenced symbol names to the object file. So the
object file is self-contained. And tools can retrieve referenced
symbol names properly.
Tools now should work as good for new object files as for old
object files.
Fixes #38875.
Change-Id: I16c685c1fd83273ab1faef474e19acf4af46396f
Reviewed-on: https://go-review.googlesource.com/c/go/+/236168 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Cherry Zhang [Tue, 2 Jun 2020 18:52:16 +0000 (14:52 -0400)]
Revert "cmd/internal/goobj: add index to symbol name for indexed symbols"
This reverts CL 229246.
For new indexed object files, in CL 229246 we added symbol index
to tools (nm, objdump) output. This affects external tools that
parse those outputs. And the added index doesn't look very nice.
In this release we take it out. For future releases we may
introduce a flag to tools (nm, objdump) and optionally dump the
symbol index.
For refererenced (not defined) indexed symbols, currently the
symbol is still referenced only by index, not by name. The next
CL will make the object file self-contained, so tools can dump
the symbol names properly (as before).
For #38875.
Change-Id: I07375e85a8e826e15c82fa452d11f0eaf8535a00
Reviewed-on: https://go-review.googlesource.com/c/go/+/236167 Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org>
Than McIntosh [Tue, 2 Jun 2020 21:15:35 +0000 (17:15 -0400)]
doc/go1.15: add release notes for plugin
Add a blurb to the release notes mentioning that the
linker now supports DWARF generation for -buildmode=plugin,
and that plugin builds work now for freebsd/amd64.
David Chase [Tue, 2 Jun 2020 21:53:47 +0000 (17:53 -0400)]
runtime: repair gdb printing fix for 7.12, 8.{1,2,3}.1, 9.2
Hand-verified for listed gdb versions. Gdb (apparently)
changed the way it names certain Go types, and this change
broke the pretty-printer-activating code in runtime-gdb.py
runtime-gdb_test.go now checks channel, map, string, and slice
printing unconditionally (i.e., no opt-out for old versions).
Updates #39368.
Change-Id: I98d72e1291c66bd40d970990e1a377ff2ed0c5d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/236164
Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Austin Clements [Wed, 3 Jun 2020 01:51:39 +0000 (21:51 -0400)]
doc/go1.15: runtime release notes
Change-Id: Ie37e993e840df2c063dee98fa3f6eca8e8713ca3
Reviewed-on: https://go-review.googlesource.com/c/go/+/236177 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
fanzha02 [Wed, 3 Jun 2020 02:33:03 +0000 (10:33 +0800)]
cmd/internal/obj/arm64: fix typos in document
The current document mismatches Go syntax loads a signed-byte
instruction "MOVB" with GNU syntax loads an 64bit double-word
instruction "ldr". This is just a typo in the document, the
assembler has the correct encoding. This patch fix this error.
Than McIntosh [Fri, 29 May 2020 20:01:08 +0000 (16:01 -0400)]
cmd/{compile,link}: fix problem with DWARF end_sequence ops
During DWARF line table emission in the linker, prior to issuing a
DW_LNE_end_sequence op to mark the end of the line table for a
compilation unit, advance the PC to produce an address beyond the last
text address in the unit (this is required by the DWARF standard).
Because of the way that GDB interprets end-sequence ops, we were
effectively losing the last row in the line table for each unit, which
degraded the debugging experience.
This problem has been around for a while, but has surfaced recently
due to changes in line table generation. Prior to Go 1.14, the DWARF
line table was emitted entirely in the linker, and a single monolithic
line table was created for each Go package (including functions from
assembly). In 1.14 we moved to having the compiler emit line table
fragments for each function, and having the linker stitch together the
fragments. As part of this change we moved to a model in which each
"go tool compile/asm" output has its own DWARF line table instance,
meaning that there are many more "end sequence" ops, which made the
problem more visible.
Fixes #38192.
Change-Id: Ic29e2f6e0ac952360c81fcba5268ad70b2b44184
Reviewed-on: https://go-review.googlesource.com/c/go/+/235739
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org>
Ian Lance Taylor [Mon, 4 May 2020 23:19:16 +0000 (16:19 -0700)]
runtime: steal timers from running P's
Previously we did not steal timers from running P's, because that P
should be responsible for running its own timers. However, if the P
is running a CPU-bound G, this can cause measurable delays in running
ready timers. Also, in CL 214185 we avoided taking the timer lock of a P
with no ready timers, which reduces the chances of timer lock contention.
So, if we can't find any ready timers on sleeping P's, try stealing
them from running P's.
Fixes #38860
Change-Id: I0bf1d5dc56258838bdacccbf89493524e23d7fed
Reviewed-on: https://go-review.googlesource.com/c/go/+/232199
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Andrew G. Morgan [Tue, 2 Jun 2020 20:13:54 +0000 (13:13 -0700)]
cmd/go: add -Wl,-wrap,symbol to linker flag whitelist
This is needed for cgo support for libpsx.
Fixes: #39361
Change-Id: I500f5614ea4b82b085322af1f1ffeb1f55270a05
Reviewed-on: https://go-review.googlesource.com/c/go/+/236139
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jean de Klerk [Mon, 1 Jun 2020 18:19:05 +0000 (12:19 -0600)]
testing: capture testname on --- PASS and --- FAIL lines
This fixes an issue raised at https://github.com/golang/go/issues/38458#issuecomment-635617670
in which --- PASS and --- FAIL lines would not trigger --- CONT lines
of other tests.
David Golden [Thu, 23 Apr 2020 01:16:49 +0000 (21:16 -0400)]
time: note that formats may parse invalid strings
The existing documentation for time format constants doesn't mention
that they may parse technically-invalid strings, such as single-digit
hours when a two-digit hour is required by a specification. This commit
adds a short warning note to that effect.
Fixes #37616
Change-Id: I6e5e12bd42dc368f8ca542b4c0527a2b7d30acaf
Reviewed-on: https://go-review.googlesource.com/c/go/+/229460
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Roland Shoemaker [Wed, 13 May 2020 01:49:34 +0000 (18:49 -0700)]
crypto/x509: document support for leading periods in DNS constraints
This change adds a comment to the Verify documentation that indicates
that you can use URI and email style name constraints with a leading
period for DNS names (and explains what they do). This behavior is
not standards compliant, but matches the community application of
RFC 5280, so it makes sense to document it.
Dan Scales [Tue, 2 Jun 2020 15:50:54 +0000 (08:50 -0700)]
runtime: add three new partial orders for lock ranking
Two are associated with the new sysmon rank: lockRankSysmon -> lockRankFin
(https://build.golang.org/log/07b0b8ee6ec9421d83699a1d850d9938390fb996)
and one I encountered during testing, lockRankSysmon -> lockRankWbufSpans
Richard Miller [Tue, 2 Jun 2020 09:34:09 +0000 (10:34 +0100)]
internal/poll: add mutex to prevent SetDeadline race in Plan 9
There are data races on fd.[rw]aio and fd.[rw]timedout when Read/Write
is called on a polled fd concurrently with SetDeadline (see #38769).
Adding a mutex around accesses to each pair (read and write) prevents
the race, which was causing deadlocks in net/http tests on the builders.
Updates #38769.
Change-Id: I31719b3c9a664e81a775cda583cff31c0da946c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/235820
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David du Colombier <0intro@gmail.com>
Cherry Zhang [Mon, 1 Jun 2020 22:06:45 +0000 (18:06 -0400)]
misc/cgo/testplugin: fix typo in comment
Change-Id: I7d1a5f6936505dff8f765541b5102dcbcd6ae835
Reviewed-on: https://go-review.googlesource.com/c/go/+/235924 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Elias Naur [Mon, 1 Jun 2020 09:47:23 +0000 (11:47 +0200)]
doc: document the new Cgo EGLConfig special case
Change-Id: I7ae5eaa974b85eac421a0b1f79cb734a0fe44e72
Reviewed-on: https://go-review.googlesource.com/c/go/+/235818 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Elias Naur [Mon, 1 Jun 2020 09:58:54 +0000 (11:58 +0200)]
doc: document new Android default linker
Change-Id: I3557f6726afe325db79b2c972d107b3bcc103b8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/235819 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Fri, 29 May 2020 05:59:09 +0000 (01:59 -0400)]
cmd/link: flush file mapping before unmapping
Call FlushViewOfFile before unmapping the output file, for extra
safety. The documentation says the function does not wait for
the data to be written to disk, so it should be cheap.
Fixes #38440.
Change-Id: I05352f15d9305e6e7086a002f61802f74036b710
Reviewed-on: https://go-review.googlesource.com/c/go/+/235639 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Jay Conrod [Thu, 14 May 2020 19:03:26 +0000 (15:03 -0400)]
cmd/go/internal/modload: document mvsReqs.Max
The version "" denotes the main module, which has no version. The
mvs.Reqs interface documentation hints this is allowed, but it's not
obvious from the implementation in modload.mvsReqs.Max.
Also, replace a related TODO with a comment in mvs.Downgrade.
Fixes #39042
Change-Id: I11e10908c9b3d8c2283eaa5c04bd8e1b936851fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/234003
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Daniel Nephin [Mon, 1 Jun 2020 16:11:34 +0000 (16:11 +0000)]
cmd/test2json: attribute output to the correct test
When printing regular test output check the indentation of the output, and use
the report stack to find the appropriate test name for that output.
This change includes a whitespace change to some golden test files. The
indentation of tests was changed in CL 113177
from tabs to spaces. The golden files have been updated to match the new
output format. The tabs in the golden files cause problems because the indentation check
looks for 4 spaces.
Richard Musiol [Sat, 25 Apr 2020 16:53:53 +0000 (18:53 +0200)]
runtime: fix race condition between timer and event handler
This change fixes a race condition between beforeIdle waking up the
innermost event handler and a timer causing a different goroutine to
wake up at the exact same moment. This messes up the wasm event handling
and leads to memory corruption. The solution is to make beforeIdle
return the goroutine that must run next and have findrunnable pick
this goroutine without considering timers again.
Michael Pratt [Fri, 29 May 2020 20:01:17 +0000 (16:01 -0400)]
runtime: no SIGWINCH to pgrp while GDB is running
When run with stdin == /dev/null and stdout/stderr == pipe (i.e., as
os/exec.Command.CombinedOutput), GDB suffers from a bug
(https://sourceware.org/bugzilla/show_bug.cgi?id=26056) that causes
SIGSEGV when sent a SIGWINCH signal.
Package runtime tests TestEINTR and TestSignalDuringExec both send
SIGWINCH signals to the entire process group, thus including GDB if one
of the GDB tests is running in parallel.
TestEINTR only intends its signals for the current process, so it is
changed to do so. TestSignalDuringExec, really does want its signals to
go to children. However, it does not call t.Parallel(), so it won't run
at the same time as GDB tests.
This is a simple fix, but GDB is vulnerable, so we must be careful not
to add new parallel tests that send SIGWINCH to the entire process
group.
Fixes #39021
Change-Id: I803606fb000f08c65c1b10ec554d4ef6819e5dd5
Reviewed-on: https://go-review.googlesource.com/c/go/+/235557
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Jay Conrod [Fri, 8 May 2020 15:57:44 +0000 (11:57 -0400)]
go/build: move build constraint docs to 'go help buildconstraint'
CL 228017 added a new help page 'go help buildconstraint' which
summarized the information on build constraints in the go/build
documentation. The summary was almost as long as the go/build
documentation, since there's very little that can be left out.
This CL moves the original go/build documentation to
'go help buildconstraint' to eliminate redundnancy. The text
describing enabled tags is slightly different (targeting command-line
users more than go/build users), but the rest of the documentation is
unchanged.
Fixes #37018
Change-Id: Ic0ed4c6fdae2395dd58852e1600c701247c9c4cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/232981
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Jay Conrod [Wed, 6 May 2020 16:59:18 +0000 (12:59 -0400)]
cmd/go: group 'go get' update messages together near the end of output
In module mode, 'go get' prints a message for each version query it
resolves. This change groups those messages together near the end of
the output so they aren't mixed with other module "finding" and
"downloading" messages. They'll still be printed before build-related
messages.
Fixes #37982
Change-Id: I107a9f2b2f839e896399df906e20d6fc77f280c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/232578
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Than McIntosh [Sun, 17 May 2020 18:36:55 +0000 (14:36 -0400)]
cmd/compile: add test case for issue 38068
New test case for issue 38068, which deals with build reproducibility:
do a pair of compilations, the first with the concurrent back end
turned on, and the second with -c=1, then check to make sure we get
the same output (using a test case that triggers late inlining into
wrapper methods).
Updates #38068.
Change-Id: I4afaf78898706a66985f09d18f6f6f29876c9017
Reviewed-on: https://go-review.googlesource.com/c/go/+/234417 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Xiangdong Ji [Wed, 6 May 2020 09:54:40 +0000 (09:54 +0000)]
cmd/compile: fix incorrect rewriting to if condition
Some ARM64 rewriting rules convert 'comparing to zero' conditions of if
statements to a simplified version utilizing CMN and CMP instructions to
branch over condition flags, in order to save one Add or Sub caculation.
Such optimizations lead to wrong branching in case an overflow/underflow
occurs when executing CMN or CMP.
Fix the issue by introducing new block opcodes that don't honor the
overflow/underflow flag, in the following categories:
Block-Op Meaning ARM condition codes
1. LTnoov less than MI
2. GEnoov greater than or equal PL
3. LEnoov less than or equal MI || EQ
4. GTnoov greater than NEQ & PL
The backend generates two consecutive branch instructions for 'LEnoov'
and 'GTnoov' to model their expected behavior. A slight change to 'gc'
and amd64/386 backends is made to unify the code generation.
Add a test 'TestCondRewrite' as justification, it covers 32 incorrect rules
identified on arm64, more might be needed on other arches, like 32-bit arm.
Add two benchmarks profiling the aforementioned category 1&2 and category
3&4 separetely, we expect the first two categories will show performance
improvement and the second will not result in visible regression compared with
the non-optimized version.
This change also updates TestFormats to support using %#x.
Examples exhibiting where does the issue come from:
1: 'if x + 3 < 0' might be converted to:
before:
CMN $3, R0
BGE <else branch> // wrong branch is taken if 'x+3' overflows
after:
CMN $3, R0
BPL <else branch>
2: 'if y - 3 > 0' might be converted to:
before:
CMP $3, R0
BLE <else branch> // wrong branch is taken if 'y-3' underflows
after:
CMP $3, R0
BMI <else branch>
BEQ <else branch>
Benchmark data from different kinds of arm64 servers, 'old' is the non-optimized
version (not the parent commit), generally the optimization version outperforms.
S1:
name old time/op new time/op delta
CondRewrite/SoloJump 13.6ns ± 0% 12.9ns ± 0% -5.15% (p=0.000 n=10+10)
CondRewrite/CombJump 13.8ns ± 1% 12.9ns ± 0% -6.32% (p=0.000 n=10+10)
S2:
name old time/op new time/op delta
CondRewrite/SoloJump 11.6ns ± 0% 10.9ns ± 0% -6.03% (p=0.000 n=10+10)
CondRewrite/CombJump 11.4ns ± 0% 10.8ns ± 1% -5.53% (p=0.000 n=10+10)
S3:
name old time/op new time/op delta
CondRewrite/SoloJump 7.36ns ± 0% 7.50ns ± 0% +1.79% (p=0.000 n=9+10)
CondRewrite/CombJump 7.35ns ± 0% 7.75ns ± 0% +5.51% (p=0.000 n=8+9)
S4:
name old time/op new time/op delta
CondRewrite/SoloJump-224 11.5ns ± 1% 10.9ns ± 0% -4.97% (p=0.000 n=10+10)
CondRewrite/CombJump-224 11.9ns ± 0% 11.5ns ± 0% -2.95% (p=0.000 n=10+10)
S5:
name old time/op new time/op delta
CondRewrite/SoloJump 10.0ns ± 0% 10.0ns ± 0% -0.45% (p=0.000 n=9+10)
CondRewrite/CombJump 9.93ns ± 0% 9.77ns ± 0% -1.53% (p=0.000 n=10+9)
Volker Dobler [Tue, 26 May 2020 14:55:28 +0000 (16:55 +0200)]
net/http: clarify that AddCookie only sanitizes the Cookie being added
AddCookie properly encodes a cookie and appends it to the Cookie header
field but does not modify or sanitize what the Cookie header field
contains already. If a user manualy sets the Cookie header field to
something not conforming to RFC 6265 then a cookie added via AddCookie
might not be retrievable.
Daniel Martí [Tue, 24 Sep 2019 17:14:10 +0000 (18:14 +0100)]
encoding/xml: only initialize nil struct fields when decoding
fieldInfo.value used to initialize nil anonymous struct fields if they
were encountered. This behavior is wanted when decoding, but not when
encoding. When encoding, the value should never be modified, and these
nil fields should be skipped entirely.
To fix the bug, add a bool argument to the function which tells the
code whether we are encoding or decoding.
Finally, add a couple of tests to cover the edge cases pointed out in
the original issue.
Daniel Martí [Wed, 20 May 2020 17:03:31 +0000 (17:03 +0000)]
Revert "encoding/json: reuse values when decoding map elements"
This reverts golang.org/cl/179337.
Reason for revert: broke a few too many reasonably valid Go programs.
The previous behavior was perhaps less consistent, but the docs were
never very clear about when the decoder merges with existing values,
versus replacing existing values altogether.
Fixes #39149.
Change-Id: I1c1d857709b8398969fe421aa962f6b62f91763a
Reviewed-on: https://go-review.googlesource.com/c/go/+/234559
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
Bryan C. Mills [Tue, 26 May 2020 20:11:22 +0000 (16:11 -0400)]
os: in Symlink, stat the correct target path for drive-relative targets on Windows
Previously, when the target (“old”) path passed to os.Symlink was a
“root-relative” Windows path,¹ we would erroneously prepend
destination (“new”) path when determining which path to Stat,
resulting in an invalid path which was then masked by the lack of
error propagation for the Stat call (#39183).
If the link target is a directory (rather than a file), that would
result in the symlink being created without the
SYMBOLIC_LINK_FLAG_DIRECTORY flag, which then fails in os.Open.
Cherry Zhang [Wed, 27 May 2020 15:37:00 +0000 (11:37 -0400)]
cmd/compile: always use StackMapDontCare as register map index when reg map is not used
When go115ReduceLiveness is true (so we don't emit actual
register maps), use StackMapDontCare consistently for the
register map index, so RegMapValid is always false.
This fixes a compiler crash when doing -live=2 debug print.
Russ Cox [Fri, 22 May 2020 00:46:05 +0000 (20:46 -0400)]
net/http: handle body rewind in HTTP/2 connection loss better
In certain cases the HTTP/2 stack needs to resend a request.
It obtains a fresh body to send by calling req.GetBody.
This call was missing from the path where the HTTP/2
round tripper returns ErrSkipAltProtocol, meaning fall back
to HTTP/1.1. The result was that the HTTP/1.1 fallback
request was sent with no body at all.
This CL changes that code path to rewind the body before
falling back to HTTP/1.1. But rewinding the body is easier
said than done. Some requests have no GetBody function,
meaning the body can't be rewound. If we need to rewind and
can't, that's an error. But if we didn't read anything, we don't
need to rewind. So we have to track whether we read anything,
with a new ReadCloser wrapper. That in turn requires adding
to the couple places that unwrap Body values to look at the
underlying implementation.
This CL adds the new rewinding code in the main retry loop
as well.
The new rewindBody function also takes care of closing the
old body before abandoning it. That was missing in the old
rewind code.
Thanks to Aleksandr Razumov for CL 210123
and to Jun Chen for CL 234358, both of which informed
this CL.
Fixes #32441.
Change-Id: Id183758526c087c6b179ab73cf3b61ed23a2a46a
Reviewed-on: https://go-review.googlesource.com/c/go/+/234894
Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Cherry Zhang [Wed, 27 May 2020 00:02:43 +0000 (20:02 -0400)]
cmd/link: actually close the output file
When the output file is mmap'd, OutBuf.Close currently munmap the
file but doesn't actually close the file descriptor. This CL
makes it actually close the FD.
Jay Conrod [Thu, 21 May 2020 14:59:29 +0000 (10:59 -0400)]
cmd/go: report error for empty GOPROXY list
If GOPROXY is "", we set it to the default value,
"https://proxy.golang.org,direct". However, if GOPROXY is a non-empty
string that doesn't contain any URLs or keywords, we treat it as
either "off" or "noproxy", which can lead to some strange errors.
This change reports an error for this kind of GOPROXY value.
For #39180
Change-Id: If2e6e39d6f74c708e5ec8f90e9d4880e0e91894f
Reviewed-on: https://go-review.googlesource.com/c/go/+/234857
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Katie Hockman [Tue, 26 May 2020 19:36:58 +0000 (15:36 -0400)]
crypto/x509: allow setting MaxPathLen to -1 without IsCA
This fixes a bug in CL 228777 which disallowed
a MaxPathLen of -1 without IsCA, even though the
x509.Certificate documentation indicates that
MaxPathLen of -1 is considered "unset".
Paul D. Weber [Tue, 26 May 2020 07:00:30 +0000 (07:00 +0000)]
cmd/link/internal/ld/lib.go: use lld on Android
Set linker explicitly to lld because the default does not work on NDK
versions r19c, r20, r20b and r21. NDK 18b (or earlier) based builds
will need to specify -fuse-ld=gold.
Fixes #38838
Change-Id: Ib75f71fb9896b843910f41bd12aa1e36868fa9b3
GitHub-Last-Rev: eeaa171604b59d8ad3d86944ebf21ee758e92f95
GitHub-Pull-Request: golang/go#39217
Reviewed-on: https://go-review.googlesource.com/c/go/+/235017 Reviewed-by: Elias Naur <mail@eliasnaur.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Joel Sing [Tue, 19 May 2020 16:13:57 +0000 (02:13 +1000)]
runtime, syscall: correct openbsd/arm and openbsd/arm64 syscalls for OpenBSD 6.7
Add two no op instructions following svc on openbsd/arm64 and swi on openbsd/arm.
All except some of the most recent arm64 processors have a speculative execution
flaw that occurs across a syscall boundary, which cannot be mitigated in the
kernel. In order to protect against this leak a speculation barrier needs to be
placed after an svc or swi instruction.
In order to avoid the performance impact of these instructions, the OpenBSD 6.7
kernel returns execution two instructions past the svc or swi call. For now two
hardware no ops are added, which allows syscalls to work with both 6.6 and 6.7.
These should be replaced with real speculation barriers once OpenBSD 6.8 is
released.
Updates #36435
Change-Id: I06153cb0998199242cca8761450e53599c3e7de4
Reviewed-on: https://go-review.googlesource.com/c/go/+/234381
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Elias Naur [Mon, 25 May 2020 13:27:05 +0000 (15:27 +0200)]
cmd/link/internal/ld: consider alternative linkers in linkerFlagSupported
CL 235017 is about to change the default Android linker to lld. lld doesn't
support the --compress-debug-sections=zlib-gnu flag, but linkerFlagSupported
doesn't take any alternative linkers specified with -fuse-ld into account.
Updates #38838
Change-Id: I5f7422c06d40dedde2e4b070fc48398e8f822190
Reviewed-on: https://go-review.googlesource.com/c/go/+/235157
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Tobias Klauser [Sat, 23 May 2020 22:09:20 +0000 (00:09 +0200)]
cmd/go: fix parallel chatty tests on solaris-amd64 builder
The parallel chatty tests added in CL 229085 fail on the
solaris-amd64-oraclerel builder, because a +NN:NN offset time zone is
used. Allow for the `+` character in the corresponding regex to fix
these tests. Also move the '-' to the end of the character class, so it
is not interpreted as the range 9-T.
Change-Id: Iec9ae82ba45d2490176f274f0dc6812666eae718
Reviewed-on: https://go-review.googlesource.com/c/go/+/234978
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
matsuyoshi [Sat, 23 May 2020 07:23:04 +0000 (16:23 +0900)]
os: use same link in UserCacheDir/UserConfigDir doc
Change-Id: I94c385243c37589f56aadaa30336b400adf31308
Reviewed-on: https://go-review.googlesource.com/c/go/+/234959 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Roland Shoemaker [Tue, 12 May 2020 18:06:42 +0000 (11:06 -0700)]
encoding/asn1: document what Unmarshal returns in rest
Specifically, this change documents the behavior of Unmarshal when a
SEQUENCE contains trailing elements.
For context Unmarshal treats trailing elements of a SEQUENCE that do not
have matching struct fields as valid, as this is how ASN.1 structures
are typically extended. This can be somewhat confusing as you might
expect those elements to be appended to rest, but rest is really only
for trailing data unrelated to the structure being parsed (i.e. if you
append a second sequence to b, it would be returned in rest).
Russ Cox [Fri, 22 May 2020 00:53:45 +0000 (20:53 -0400)]
time: simplify Duration.String example
The existing example is needlessly complex.
You have to know that t.Sub returns a Duration
and also have to mentally subtract the two times
to understand what duration should be printed.
Rewrite to focus on just the Duration.String operation.
Austin Clements [Thu, 14 May 2020 20:55:39 +0000 (16:55 -0400)]
runtime: detect and report zombie slots during sweeping
A zombie slot is a slot that is marked, but isn't allocated. This can
indicate a bug in the GC, or a bad use of unsafe.Pointer. Currently,
the sweeper has best-effort detection for zombie slots: if there are
more marked slots than allocated slots, then there must have been a
zombie slot. However, this is imprecise since it only compares totals
and it reports almost no information that may be helpful to debug the
issue.
Add a precise check that compares the mark and allocation bitmaps and
reports detailed information if it detects a zombie slot.
No appreciable effect on performance as measured by the sweet
benchmarks:
Bryan C. Mills [Thu, 21 May 2020 18:16:50 +0000 (14:16 -0400)]
all: use a hermetic "go" tool in standard-library tests
The go/build package uses the "go" tool from the user's environment,
but its tests should not assume that that tool is in any particular
state, let alone appropriate for running the test.
Instead, explicitly use testenv.GoTool, adding it to $PATH in a
TestMain when necessary.
Fixes #39199
Fixes #39198
Change-Id: I56618a55ced473e75dd96eeb3a8f7084e2e64d02
Reviewed-on: https://go-review.googlesource.com/c/go/+/234880
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
Michael Pratt [Thu, 7 May 2020 22:13:21 +0000 (18:13 -0400)]
runtime: disable preemption in startTemplateThread
When a locked M wants to start a new M, it hands off to the template
thread to actually call clone and start the thread. The template thread
is lazily created the first time a thread is locked (or if cgo is in
use).
stoplockedm will release the P (_Pidle), then call handoffp to give the
P to another M. In the case of a pending STW, one of two things can
happen:
1. handoffp starts an M, which does acquirep followed by schedule, which
will finally enter _Pgcstop.
2. handoffp immediately enters _Pgcstop. This only occurs if the P has
no local work, GC work, and no spinning M is required.
If handoffp starts an M, and must create a new M to do so, then newm
will simply queue the M on newmHandoff for the template thread to do the
clone.
When a stop-the-world is required, stopTheWorldWithSema will start the
stop and then wait for all Ps to enter _Pgcstop. If the template thread
is not fully created because startTemplateThread gets stopped, then
another stoplockedm may queue an M that will never get created, and the
handoff P will never leave _Pidle. Thus stopTheWorldWithSema will wait
forever.
A sequence to trigger this hang when STW occurs can be visualized with
two threads:
Note that the P in T2 is stuck sitting in _Pidle. Since the template
thread isn't running, the new M will not be started complete the
transition to _Pgcstop.
To resolve this, we disable preemption around the assignment of
haveTemplateThread and the creation of the template thread in order to
guarantee that if handTemplateThread is set then the template thread
will eventually exist, in the presence of stops.
Jean de Klerk [Mon, 4 May 2020 20:06:34 +0000 (14:06 -0600)]
testing: reformat test chatty output
In #24929, we decided to stream chatty test output. It looks like,
foo_test.go:138: TestFoo/sub-1: hello from subtest 1
foo_test.go:138: TestFoo/sub-2: hello from subtest 2
In this CL, we refactor the output to be grouped by === CONT lines, preserving
the old test-file-before-log-line behavior:
=== CONT TestFoo/sub-1
foo_test.go:138 hello from subtest 1
=== CONT TestFoo/sub-2
foo_test.go:138 hello from subtest 2
This should remove a layer of verbosity from tests, and make it easier to group
together related lines. It also returns to a more familiar format (the
pre-streaming format), whilst still preserving the streaming feature.
Michael Anthony Knyszek [Tue, 19 May 2020 16:33:17 +0000 (16:33 +0000)]
runtime: synchronize StartTrace and StopTrace with sysmon
Currently sysmon is not stopped when the world is stopped, which is
in general a difficult thing to do. The result of this is that when
tracing starts and the value of trace.enabled changes, it's possible
for sysmon to fail to emit an event when it really should. This leads to
traces which the execution trace parser deems inconsistent.
Fix this by putting all of sysmon's work behind a new lock sysmonlock.
StartTrace and StopTrace both acquire this lock after stopping the world
but before performing any work in order to ensure sysmon sees the
required state change in tracing. This change is expected to slow down
StartTrace and StopTrace, but will help ensure consistent traces are
generated.
Updates #29707.
Fixes #38794.
Change-Id: I64c58e7c3fd173cd5281ffc208d6db24ff6c0284
Reviewed-on: https://go-review.googlesource.com/c/go/+/234617
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Cherry Zhang [Mon, 18 May 2020 19:23:46 +0000 (15:23 -0400)]
cmd/link: fix size calculation for file space preallocation on darwin
On darwin, we preallocate file storage space with fcntl
F_ALLOCATEALL in F_PEOFPOSMODE mode. This is specified as
allocating from the physical end of the file. So the size we give
it should be the increment, instead of the total size.
Than McIntosh [Fri, 15 May 2020 16:19:07 +0000 (12:19 -0400)]
cmd/compile: delay inlinable method compilation for -c=1
When the concurrent back end is not enabled, it is possible to have a
scenario where: we compile a specific inlinable non-pointer-receiver
method T.M, then at some point later on in the compilation we visit a
type that triggers generation of a pointer-receiver wrapper (*T).M,
which then results in an inline of T.M into (*T).M. This introduces
subtle differences in the DWARF as compared with when the concurrent
back end is enabled (in the concurrent case, by the time we run the
SSA back end on T.M is is marked as being inlined, whereas in the
non-current case it is not marked inlined).
As a fix, at the point where we would normally compile a given
function in the xtop list right away, if the function is a method AND
is inlinable AND hasn't been inlined, then delay its compilation until
compileFunctions (so as to make sure that when we do compile it, all
possible inlining has been complete). In addition, make sure that
the abstract function symbol for the inlined function gets recorded
correctly.
Fixes #38068.
Change-Id: I57410ab5658bd4ee5b4b80750518e9b20fd6ba52
Reviewed-on: https://go-review.googlesource.com/c/go/+/234178
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Jay Conrod [Wed, 20 May 2020 21:54:55 +0000 (17:54 -0400)]
cmd/go: rank errUseProxy lower when handling proxy errors
modfetch.TryProxies ranks errors returned by GOPROXY entries by
usefulness. It returns the error of the highest rank from the last
proxy. Errors from "direct" and "noproxy" are most useful, followed by
errors other than ErrNotExist, followed by ErrNotExist.
This change ranks errUseProxy with ErrNotExist even though it's
reported by "noproxy". There is almost always a more useful message
than "path does not match GOPRIVATE/GONOPROXY".
Fixes #39180
Change-Id: Ifa5b96462d7bf411e6d2d951888465c839d42471
Reviewed-on: https://go-review.googlesource.com/c/go/+/234687
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Cherry Zhang [Mon, 18 May 2020 18:14:11 +0000 (14:14 -0400)]
runtime: add a barrier after a new span is allocated
When copying a stack, we
1. allocate a new stack,
2. adjust pointers pointing to the old stack to pointing to the
new stack.
If the GC is running on another thread concurrently, on a machine
with weak memory model, the GC could observe the adjusted pointer
(e.g. through gp._defer which could be a special heap-to-stack
pointer), but not observe the publish of the new stack span. In
this case, the GC will see the adjusted pointer pointing to an
unallocated span, and throw. Fixing this by adding a publication
barrier between the allocation of the span and adjusting pointers.
One testcase for this is TestDeferHeapAndStack in long mode. It
fails reliably on linux-mips64le-mengzhuo builder without the fix,
and passes reliably after the fix.
Ian Lance Taylor [Wed, 20 May 2020 03:23:58 +0000 (20:23 -0700)]
syscall: preserve Windows file permissions for O_CREAT|O_TRUNC
On Windows, calling syscall.Open(file, O_CREAT|O_TRUNC, 0) for a file
that already exists would change the file to be read-only.
That is not how the Unix syscall.Open behaves, so avoid it on
Windows by calling CreateFile twice if necessary.
Fixes #38225
Change-Id: I70097fca8863df427cc8a97b9376a9ffc69c6318
Reviewed-on: https://go-review.googlesource.com/c/go/+/234534
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Yasuhiro Matsumoto [Thu, 14 May 2020 09:17:49 +0000 (18:17 +0900)]
cmd/go: use temporary file for output of gcc command on Windows
On Windows, some of gcc command (like msys2 native) output NUL as a file.
Fixes #36000
Change-Id: I02c632fa2d710a011d79f24d5beee4bc57ad994e
Reviewed-on: https://go-review.googlesource.com/c/go/+/233977
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Chase [Fri, 1 May 2020 21:30:33 +0000 (17:30 -0400)]
cmd/go: remove GOAMD64 environment variable
This removes the GOAMD64 environment variable and its documentation.
The value is instead supplied by a compiled-in constant.
Note that function alignment is also dependent on the value of
the (removed) flag; it is 32 for aligned jumps, 16 if not.
When the flag-dependent logic is removed, it will be 32.
Bryan C. Mills [Tue, 19 May 2020 05:29:11 +0000 (01:29 -0400)]
runtime: allocate fewer bytes during TestEINTR
This will hopefully address the occasional "runtime: out of memory"
failures observed on the openbsd-arm-jsing builder:
https://build.golang.org/log/c296d866e5d99ba401b18c1a2ff3e4d480e5238c
Also make the "spin" and "winch" loops concurrent instead of
sequential to cut down the test's running time.
Finally, change Block to coordinate by closing stdin instead of
sending SIGINT. The SIGINT handler wasn't necessarily registered by
the time the signal was sent.
Updates #20400
Updates #39043
Change-Id: Ie12fc75b87e33847dc25a12edb4126db27492da6
Reviewed-on: https://go-review.googlesource.com/c/go/+/234538
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Mon, 18 May 2020 16:06:11 +0000 (16:06 +0000)]
runtime: don't use (addrRange).subtract in removeGreaterEqual
Currently in (*addrRanges).removeGreaterEqual we use
(addrRange).subtract with a range from specified address to "infinity"
which is supposed to be maxOffAddr. However, maxOffAddr is necessarily
an inclusive bound on the address space, because on many platforms an
exclusive bound would overflow back to 0.
On some platforms like mips and mipsle, the address space is smaller
than what's representable in a pointer, so if there's a range which hits
the top of the address space (such as in the pageAlloc tests), the limit
doesn't overflow, but maxOffAddr is inclusive, so any attempt to prune
this range with (*addrRange).removeGreaterEqual causes a failure, since
the range passed to subtract is contained within the address range which
touches the top of the address space.
Another problem with using subtract here is that addr and
maxOffAddr.addr() may not be in the same segment which could cause
makeAddrRange to panic. While this unlikely to happen, on some platforms
such as Solaris it is possible.
Fix these issues by not using subtract at all. Create a specific
implementation of (addrRange).removeGreaterEqual which side-steps all of
this by not having to worry about the top of the address space at all.
Michael Hudson-Doyle [Wed, 20 May 2020 02:39:13 +0000 (14:39 +1200)]
cmd/go: accept smart quotes when checking for missing gold in TestNoteReading
Fixes #39157
Change-Id: Ia983f5e66698519cd19c1565cfb80e86d08fdfc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/234380 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Tue, 19 May 2020 18:17:05 +0000 (14:17 -0400)]
crypto/x509: save the temp dir in TestReadUniqueDirectoryEntries
In CL 231958, TempDir was changed to create a new temp directory on
each allocation, on the theory that it is easy to save in a variable
for callers that want the same directory repeatedly. Apply that
transformation here.
Katie Hockman [Mon, 18 May 2020 20:49:04 +0000 (16:49 -0400)]
crypto/tls: remove version check when unmarshaling sessionState
This was causing issues when fuzzing with
TestMarshalUnmarshal since the test would
occassionally set the version to VersionTLS13,
which would fail when unmarshaling. The check
doesn't add much in practice, and there is no
harm in removing it to de-flake the test.
Jay Conrod [Thu, 14 May 2020 18:52:17 +0000 (14:52 -0400)]
cmd: update golang.org/x/mod to v0.3.0 (same commit)
v0.3.0 is a tag on 859b3ef565e2, the version that was already being
used. This change is a no-op, except for letting us use a release
version instead of a pseudo-version.
For #36905
Change-Id: I70b8ce2a3f1451f5602c469501362d7a6a673b12
Reviewed-on: https://go-review.googlesource.com/c/go/+/234002
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Richard Miller [Mon, 18 May 2020 08:34:17 +0000 (09:34 +0100)]
runtime: don't enable notes (=signals) too early in Plan 9
The Plan 9 runtime startup was enabling notes (like Unix signals)
before the gsignal stack was allocated. This left a small window
of time where an interrupt (eg by the parent killing a subprocess
quickly after exec) would cause a null pointer dereference in
sigtramp. This would leave the interrupted process suspended in
'broken' state instead of exiting. We've observed this on the
builders, where it can make a test time out waiting for the broken
process to terminate.
Updates #38772
Change-Id: I54584069fd3109595f06c78724c1f6419e028aab
Reviewed-on: https://go-review.googlesource.com/c/go/+/234397
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David du Colombier <0intro@gmail.com>
Cherry Zhang [Thu, 14 May 2020 23:22:59 +0000 (19:22 -0400)]
cmd/link: detect trampoline of deferreturn call
The runtime needs to find the PC of the deferreturn call in a few
places. So for functions that have defer, we record the PC of
deferreturn call in its funcdata.
For very large binaries, the deferreturn call could be made
through a trampoline. The current code of finding deferreturn PC
fails in this case. This CL handles the trampoline as well.
Fixes #39049.
Change-Id: I929be54d6ae436f5294013793217dc2a35f080d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/234105
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jeremy Faller <jeremy@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Cherry Zhang [Thu, 14 May 2020 16:16:10 +0000 (12:16 -0400)]
cmd/link: fix SCONST symbol handling on darwin
Don't include SCONST symbols in the symbol table when
NotInSymbolTable is set. This is what the old code (genasmsym)
does.
In fact, SCONST symbol is only emitted by the field tracking
code, and is always NotInSymbolTable. So we should just not
include them at all, or not generate SCONST symbols at all. But
at this late stage I'll just restore the old behavior.
Michael Anthony Knyszek [Tue, 12 May 2020 16:08:50 +0000 (16:08 +0000)]
runtime: make maxOffAddr reflect the actual address space upper bound
Currently maxOffAddr is defined in terms of the whole 64-bit address
space, assuming that it's all supported, by using ^uintptr(0) as the
maximal address in the offset space. In reality, the maximal address in
the offset space is (1<<heapAddrBits)-1 because we don't have more than
that actually available to us on a given platform.
On most platforms this is fine, because arenaBaseOffset is just
connecting two segments of address space, but on AIX we use it as an
actual offset for the starting address of the available address space,
which is limited. This means using ^uintptr(0) as the maximal address in
the offset address space causes wrap-around, especially when we just
want to represent a range approximately like [addr, infinity), which
today we do by using maxOffAddr.
To fix this, we define maxOffAddr more appropriately, in terms of
(1<<heapAddrBits)-1.
This change also redefines arenaBaseOffset to not be the negation of the
virtual address corresponding to address zero in the virtual address
space, but instead directly as the virtual address corresponding to
zero. This matches the existing documentation more closely and makes the
logic around arenaBaseOffset decidedly simpler, especially when trying
to reason about its use on AIX.
Fixes #38966.
Change-Id: I1336e5036a39de846f64cc2d253e8536dee57611
Reviewed-on: https://go-review.googlesource.com/c/go/+/233497
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>