Bryan C. Mills [Tue, 7 Jan 2020 17:03:28 +0000 (12:03 -0500)]
net/http: avoid writing to Transport.ProxyConnectHeader
Previously, we accidentally wrote the Proxy-Authorization header for
the initial CONNECT request to the shared ProxyConnectHeader map when
it was non-nil.
Michael Matloob [Thu, 2 Jan 2020 21:42:43 +0000 (16:42 -0500)]
cmd/go: convert TestTestRegexps to the script framework
It's hard to convert this one exactly because I don't think
we can guarantee that the grep command exists to filter stdout,
so I've tried to replicate the intent of the test.
Part of converting all tests to script framework to improve
test parallelism.
Updates #36320
Updates #17751
Change-Id: Ib593799ef7634ce12efb3ff357eb34475e2ea321
Reviewed-on: https://go-review.googlesource.com/c/go/+/213130
Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Joel Sing [Mon, 6 Jan 2020 17:31:23 +0000 (04:31 +1100)]
runtime: correct setNonblock on linux/arm64
The current code uses EOR (exclusive OR), which will result in the O_NONBLOCK
flag being toggled rather than being set. Other implementations use OR, hence
this is likely a bug.
Change-Id: I5dafa9c572452070bd37789c8a731ad6d04a86cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/212766
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dan Scales [Tue, 19 Nov 2019 21:58:28 +0000 (13:58 -0800)]
runtime: avoid potential deadlock when tracing memory code
In reclaimChunk, the runtime is calling traceGCSweepDone() while holding the mheap
lock. traceGCSweepDone() can call traceEvent() and traceFlush(). These functions
not only can get various trace locks, but they may also do memory allocations
(runtime.newobject) that may end up getting the mheap lock. So, there may be
either a self-deadlock or a possible deadlock between multiple threads.
It seems better to release the mheap lock before calling traceGCSweepDone(). It is
fine to release the lock, since the operations to get the index of the chunk of
work to do are atomic. We already release the lock to call sweep, so there is no
new behavior for any of the callers of reclaimChunk.
With this change, mheap is a leaf lock (no other lock is ever acquired while it
is held).
Testing: besides normal all.bash, also ran all.bash with --long enabled, since
it does longer tests of runtime/trace.
Brad Fitzpatrick [Mon, 6 Jan 2020 16:30:48 +0000 (16:30 +0000)]
os: document that File.Seek works on directories, but not portably
Updates #36019
Change-Id: I9fea2c3c5138e2233290979e4724f6e7b91da652
Reviewed-on: https://go-review.googlesource.com/c/go/+/213439 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Fri, 27 Dec 2019 17:02:50 +0000 (12:02 -0500)]
runtime: test memmove writes pointers atomically
In the previous CL we ensures that memmove writes pointers
atomically, so the concurrent GC won't observe a partially
updated pointer. This CL adds a test.
Michael Matloob [Thu, 26 Dec 2019 21:06:49 +0000 (16:06 -0500)]
cmd/go: convert some testcases to the script framework
This change converts TestFatalInBenchmarkCauseNonZeroExitStatus,
TestBenchmarkLabels and TestWildCardsDoNotLookInUselessDirectories
to the script framework so they can run in parallel. These tests
used the testdata directory so they didn't call tg.parallel, because
they couldn't be sure the testdata directory wasn't being
modified while they were being run.
The tests are converted to the script framework instead of being modified
because the script framework tests are easier to read and are self-contained.
Also remove the directory src/cmd/go/testdata/src/badc. The testcase
that used it, "TestDisallowedCSourceFiles" was deleted in
golang.org/cl/19613.
Updates #17751
Change-Id: I0b9b417ae1a9b148067a3e5f8531229f3414f104
Reviewed-on: https://go-review.googlesource.com/c/go/+/212620
Run-TryBot: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Michael Matloob [Thu, 26 Dec 2019 20:03:12 +0000 (15:03 -0500)]
cmd/go: parallellize more test cases
Sprinkle in some more calls to tg.Parallel in the cmd/go
tests. These are the easy cases that don't operate in the
testdata directory.
I think the best thing to do for those tests that do operate
in testdata is to use the script tests. They're easier to
read and write, and jump into t.Parallel immediately.
Updates #17751
Change-Id: If8aeb8129cfdc0a9aa91f5a540f179790077fce5
Reviewed-on: https://go-review.googlesource.com/c/go/+/212618
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Joel Sing [Sun, 5 Jan 2020 14:52:19 +0000 (01:52 +1100)]
syscall: resync various riscv64 structs with x/sys/unix
For some reason the existing versions have additional padding - manually resync these
structs with those in x/sys/unix, also bringing in pollFd which does not currently
exist for linux/riscv64.
Cross checked with the output from 'go tool cgo -godefs' on a linux/riscv64 machine.
Joel Sing [Sun, 5 Jan 2020 14:48:43 +0000 (01:48 +1100)]
syscall: provide Renameat as a wrapper around renameat2 on riscv64
linux/riscv64 does not provide a renameat syscall and there is no SYS_RENAMEAT
defined in the syscall package. Provide Renameat as a wrapper around renameat2
instead.
Joel Sing [Sun, 5 Jan 2020 13:18:56 +0000 (00:18 +1100)]
syscall: fix riscv64 syscall assembly
RISCV has no instruction to store an immediate value to memory, or to
subtract an immediate value. An immediate needs to be moved to a register
first, or in the case of $0, the zero value register (ZERO or R0) can be
used instead. Restore the original riscv-go code so that it compiles.
Joel Sing [Fri, 30 Aug 2019 15:02:04 +0000 (01:02 +1000)]
cmd/link: skip symbol references when looking for missing symbols
ErrorUnresolved attempts to find the missing symbol in another ABI,
in order to provide more friendly error messages. However, in doing so
it checks the same ABI and can find the symbol reference for the symbol
that it is currently reporting the unresolved error for. Avoid this by
ignoring SXREF symbols, which is the same behaviour used when linking
is performed.
Fixes #33979
Change-Id: I3cb2477b2ad4baa7c2007323b983eb29404b0aac
Reviewed-on: https://go-review.googlesource.com/c/go/+/192597
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
Joel Sing [Tue, 31 Dec 2019 14:44:54 +0000 (01:44 +1100)]
cmd/internal/obj/riscv: correct CALL reloc symbol handling
Move the CALL reloc symbol back to p.From.Sym, rather than p.To.Sym - while
p.To.Sym is the correct final location, the call to lowerJALR flips the p.To
and p.From addresses (and proves that this form of obj.Prog rewriting is not
a good idea).
Joel Sing [Tue, 31 Dec 2019 14:28:22 +0000 (01:28 +1100)]
cmd/internal/obj/riscv: handle MOV rewrites in separate pass
Handle the rewriting of MOV pseudo-instructions in a separate pass. This allows
AMOV to be more readily used by other code, including AGETCALLERPC and ARET
implementations.
Cherry Zhang [Mon, 30 Dec 2019 16:26:17 +0000 (11:26 -0500)]
runtime: don't use R11 in nanotime1/walltime1 on ARM
R11 a.k.a. REGTMP is the temp register used by the assembler. It
may be clobbered if the assembler needs to synthesize
instructions. In particular, in nanotime1/walltime1, the load of
global variable runtime.iscgo clobbers it. So, avoid using R11
to hold a long-lived value.
Fixes #36309.
Change-Id: Iec2ab9d664532cad8fbf58da17f580e64a744f62
Reviewed-on: https://go-review.googlesource.com/c/go/+/212641 Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Andrew G. Morgan <agm@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Daniel Martí [Sun, 13 Oct 2019 12:18:03 +0000 (13:18 +0100)]
doc: document atexit_sleep_ms flag in race_detector
It was pointed out to me that the thread sanitizer library has many more
flags than the ones documented in this doc page. In particular, I was
able to make use of GORACE=atexit_sleep_ms=10 to reduce the amount that
'go test -race' will sleep before finishing.
I'm sure that this flag will be useful to others in the future, so add
it here. This is still not a complete list, but we can simply add useful
flags that work when we think they deserve a mention.
Michael Matloob [Mon, 30 Dec 2019 21:56:00 +0000 (16:56 -0500)]
cmd/go: convert TestRunInternal to the script test framework
This conversion is a bit weird, because the original test runs in the cmd/go
directory, while the script test runs in the GOPATH directory. So even though
it's not necessary for the new test, it changes dircectory to $WORK, so that
its error message regexp can have four components like the original, just
changing the old gopath directory 'testdata' the new one 'gopath'.
Part of converting all tests to script framework to improve test parallelism.
Updates #36320
Updates #17751
Change-Id: Ie5b029c43dc22167278d3104b37c0b57c61326be
Reviewed-on: https://go-review.googlesource.com/c/go/+/212814
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Michael Matloob [Mon, 30 Dec 2019 22:10:23 +0000 (17:10 -0500)]
cmd/go: convert TestCoveragePattern to the script framework
This test already runs in parallel, but still convert it to the
script framework so we can delete the testdata/src directory and
remove any ambiguity about which tests can run in parallel.
Updates #36320
Change-Id: I6470979bd8bad0631dc6ead0d4eb9c83878356e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/212815
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
The error that's tested in this test is also tested in list_test_err.txt
which uses go list -test -deps. Because both commands are just loading
packages, the difference is not meaningful.
Updates #36320
Updates #17751
Change-Id: Ie712a77d64e8985dd908a1afb515ed3ecc0a9985
Reviewed-on: https://go-review.googlesource.com/c/go/+/212937
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Michael Matloob [Mon, 30 Dec 2019 23:49:27 +0000 (18:49 -0500)]
cmd/go: convert TestPluginNonMain to script framework
TestPluginNonMain was broken before this change! (It provided the
wrong directory for testdep/p2: testdata/testdep/p2 instead of
testdata/src/testdep/p2).
Change-Id: Ib815f119bae1d758b500cd8ad82c016cb630d71e
Reviewed-on: https://go-review.googlesource.com/c/go/+/212938
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Michael Matloob [Mon, 30 Dec 2019 22:51:16 +0000 (17:51 -0500)]
cmd/go: convert tests using testdata/src/syntaxerror to scripts
This includes TestMatchesNoTestsDoesNotOverrideBuildFailure
and TestErrorMessageForSyntaxErrorInTestGoFileSaysFAIL.
Convert the tests that use the testdata/src/syntaxerror directory
to the script framework.
Part of converting all tests to script framework to improve
test parallelism.
Updates #36320
Updates #17751
Change-Id: I2b2b616e8c124996ae8c8e5b737f15bb493ec588
Reviewed-on: https://go-review.googlesource.com/c/go/+/212816
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
There should be only one copy, otherwise it may lead to confusion,
or changes made in the wrong place. This CL removes the old copies
from this repo since they're no longer used.
Updates #29206
Change-Id: I41adfb2c34ed3d870fb7a671f48ccc8f90863feb
Reviewed-on: https://go-review.googlesource.com/c/go/+/213157
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Austin Clements [Thu, 2 Jan 2020 20:23:26 +0000 (15:23 -0500)]
doc/go1.14: mention sync.Mutex changes
Change-Id: Icd92d115e5d7f00b2100598baf2522ebebcdb223
Reviewed-on: https://go-review.googlesource.com/c/go/+/213125 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Fri, 27 Dec 2019 17:02:00 +0000 (12:02 -0500)]
runtime: ensure memmove write pointer atomically on ARM64
If a pointer write is not atomic, if the GC is running
concurrently, it may observe a partially updated pointer, which
may point to unallocated or already dead memory. Most pointer
writes, like the store instructions generated by the compiler,
are already atomic. But we still need to be careful in places
like memmove. In memmove, we don't know which bits are pointers
(or too expensive to query), so we ensure that all aligned
pointer-sized units are written atomically.
Rhys Hiltner [Wed, 18 Dec 2019 00:40:46 +0000 (16:40 -0800)]
runtime: emit trace event in direct semaphore handoff
When a goroutine yields the remainder of its time to another goroutine
during direct semaphore handoff (as in an Unlock of a sync.Mutex in
starvation mode), it needs to signal that change to the execution
tracer. The discussion in CL 200577 didn't reach consensus on how best
to describe that, but pointed out that "traceEvGoSched / goroutine calls
Gosched" could be confusing.
Emit a "traceEvGoPreempt / goroutine is preempted" event in this case,
to allow the execution tracer to find a consistent event ordering
without being both specific and inaccurate about why the active
goroutine has changed.
Dmitri Shuralyov [Wed, 1 Jan 2020 05:00:00 +0000 (00:00 -0500)]
doc: 2020 is the Year of the Gopher
Starting with 2014 (golang.org/cl/46660043), we have enjoyed 6
consecutive years of the gopher. Now, the slice¹ of gophers is
ready to make its way into the next decade, as 2020 is the new
Year of the Gopher.
Tobias Klauser [Fri, 20 Dec 2019 22:34:23 +0000 (23:34 +0100)]
internal/poll: use correct fcntl implementations
Use the libc fcntl (via syscall.fcntl) on aix and solaris like it is
already done for darwin.
For the syscall-based fcntl implementation use FcntlSyscall from
internal/syscall/unix in order to get fcntl64 on 32-bit Linux
systems.
On aix, fcntl with F_DUPFD_CLOEXEC is not supported. Thus, defined
F_DUPFD_CLOEXEC = 0 in the syscall package and check its value before
calling fcntl(fd, syscall.F_DUPFD_CLOEXEC, 0).
On js/wasm, fcntl is not supported thus let its implementation return
ENOSYS directly.
Updates #36211
Change-Id: I96a2ea79e5c4eed2fefd94d0aefd72c940825682
Reviewed-on: https://go-review.googlesource.com/c/go/+/212278
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I25fbd63f10ea9892589ad44cc45761926aff0648
Reviewed-on: https://go-review.googlesource.com/c/go/+/212841 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Thu, 26 Dec 2019 19:49:39 +0000 (19:49 +0000)]
runtime: check whether scavAddr is in inUse on scavengeOne fast path
This change makes it so that we check whether scavAddr is actually
mapped before trying to look at the summary for the fast path, since we
may segfault if that that part of the summary is not mapped in.
Previously this wasn't a problem because we would conservatively map
all memory for the summaries between the lowest mapped heap address and
the highest one.
Michael Anthony Knyszek [Thu, 19 Dec 2019 17:44:58 +0000 (17:44 +0000)]
runtime: disable pageAlloc tests on OpenBSD in short mode
This change disables pageAlloc tests on OpenBSD in short mode because
pageAlloc holds relatively large virtual memory reservations and we make
two during the pageAlloc tests. The runtime may also be carrying one
such reservation making the virtual memory requirement for testing the
Go runtime three times as much as just running a Go binary.
This causes problems for folks who just want to build and test Go
(all.bash) on OpenBSD but either don't have machines with at least 4ish
GiB of RAM (per-process virtual memory limits are capped at some
constant factor times the amount of physical memory) or their
per-process virtual memory limits are low for other reasons.
Fixes #36210.
Change-Id: I8d89cfde448d4cd2fefff4ad6ffed90de63dd527
Reviewed-on: https://go-review.googlesource.com/c/go/+/212177
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Ariel Mashraki [Tue, 24 Dec 2019 14:28:16 +0000 (16:28 +0200)]
doc/go1.14: change hash/maphash package link
Package name and documentation link were wrong.
Change-Id: I274906afc3cf7a3d88e3da76549cd6ab008fd0c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/212538 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Fri, 20 Dec 2019 21:05:24 +0000 (13:05 -0800)]
internal/syscall/unix: use fcntl64 on 32-bit GNU/Linux systems
Patch up runtime testing to use the libc fcntl function on Darwin,
which is what we should be doing anyhow. This is similar to how
we handle fcntl on AIX and Solaris.
Fixes #36211
Change-Id: I47ad87e11df043ce21496a0d59523dad28960f76
Reviewed-on: https://go-review.googlesource.com/c/go/+/212299
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Ian Lance Taylor [Fri, 13 Dec 2019 01:00:08 +0000 (17:00 -0800)]
misc/cgo/test: tweak to pass with GCC 10
The test for issue 8945 was marked to only run on gccgo, but there was
no reason for that. It broke for gccgo using GCC 10, because GCC 10
defaults to -fno-common. Make the test run on gc, and split it into
test.go and testx.go to make it work with GCC 10.
The test for issue 9026 used two identical structs which GCC 10 turns
into the same type. The point of the test is not that the structs are
identical, but that they are handled in a particular order. So make
them different.
Updates #8945
Updates #9026
Change-Id: I000fb02f88f346cfbbe5dbefedd944a2c64e8d8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/211217
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Dan Scales [Thu, 19 Dec 2019 01:27:27 +0000 (17:27 -0800)]
runtime: make sure BP is saved in nanotime1/walltime1, else frame pointer may not be preserved
nanotime1 and walltime1 do not preserve BP on linux amd64. Previously, this
did not cause a problem, because nanotime/walltime do preserve the BP. But now
with mid-stack inlining, nanotime/walltime are usually inlined, so BP is not
preserved. So, the BP is now wrong in any function after a call to
nanotime()/walltime() on amd64. That means the frame pointer on the stack can
be wrong for any further function call made after the nanotime() call (notably
runtime.main and various GC functions). [386 doesn't use framepointer.]
Fix is to set a frame size of 8 for nanotime1 and walltime1, which means the
standard prolog/epilog that saves/restore BP in the stack frame is added.
I noticed this while investigating issue 16638 (use frame pointers for
runtime.Callers). This change would needed for progress on that issue (which
doesn't have a high priority). Verified that this fix works/is useful for issue
16638.
Change-Id: I19e19ef2c1a517d737a34928baae034f2eb0b2c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/212079
Run-TryBot: Dan Scales <danscales@google.com> Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Bryan C. Mills [Fri, 20 Dec 2019 04:17:55 +0000 (23:17 -0500)]
cmd/go/internal/modfetch/codehost: replace a dubious call to semver.Max
The documentation for RecentTag indicates that it returns an actual
tag, not a canonicalized prefix+version blob equivalent to a tag,
so the canonicalization due to semver.Max seems like a bug here.
Fortunately, RecentTag is not currently ever actually used as a tag,
so the removal of metadata does not result in a user-facing bug.
Nonetheless, it may be a subtle source of confusion for maintainers
in the future.
Updates #32700
Change-Id: I525423c1c0c7ec7c36c09e53b180034474f74e5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/212202
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Bryan C. Mills [Thu, 19 Dec 2019 20:46:06 +0000 (15:46 -0500)]
cmd/go: relax validation for replacements for gopkg.in paths
The 'go' command normally requires the 'go.mod' files for replacement
modules to have a major version compatible with the module they are
replacing.
However, prior to CL 206761, the 'go' command erroneously allowed
unversioned paths (which imply major version 0 or 1) to replace
'gopkg.in' paths with any major-version suffix.
An analysis of proxy.golang.org suggests that these replacements,
while uncommon, are not unheard-of. Rather than breaking the modules
that rely on them, we will continue to allow the erroneous replacement
paths for this particular pairing.
Updates #34254
Change-Id: Icb4e745981803edaa96060f17a8720a058219ab1
Reviewed-on: https://go-review.googlesource.com/c/go/+/212105
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Jay Conrod [Thu, 19 Dec 2019 18:00:28 +0000 (13:00 -0500)]
cmd/go: in 'go list -m', print effective go.mod file
When the -modfile flag is in use (either explicitly or from GOFLAGS),
'go list -m' will now print the effective go.mod file for the main
module in the GoMod field in -f or -json output.
Fixes #36220
Updates #34506
Change-Id: I89c2ee40f20e07854bb37c6e4e13eeea0cce7b0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/212100
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Daniel Martí [Wed, 18 Dec 2019 10:05:59 +0000 (10:05 +0000)]
text/template: indirect interfaces before slicing
The recently added slice function used indirectInterface, but then
forgot to actually call reflect.Value.Slice on its result. Calling the
Slice method on the original Value without indirectInterface would
result in a panic, if our slice was indeed behind an interface.
Fix that, and add test cases for all three built-in functions that work
with slices.
Fixes #36199.
Change-Id: I9a18f4f604a3b29967eefeb573f8960000936b88
Reviewed-on: https://go-review.googlesource.com/c/go/+/211877
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Cherry Zhang [Wed, 18 Dec 2019 20:19:05 +0000 (15:19 -0500)]
runtime: use innermost frame's func name for async preemption check
We don't asynchronously preempt if we are in the runtime. We do
this by checking the function name. However, it failed to take
inlining into account. If a runtime function gets inlined into
a non-runtime function, it can be preempted, and bad things can
happen. One instance of this is dounlockOSThread inlined into
UnlockOSThread which is in turn inlined into a non-runtime
function.
Fix this by using the innermost frame's function name.
Oliver Powell [Sat, 5 Oct 2019 11:05:56 +0000 (13:05 +0200)]
text/template: remove an incorrect comment and clarify docs for AddParseTree
The comment "If the name is the name of this template, overwrite this template."
is incorrect and should be "is not" instead. This comment is no longer
required once the docs are updated to mention this behaviour instead.
Fixes #34695
Change-Id: I773495b2194d7bb7619b13c1a28cbc76e8f69aac
Reviewed-on: https://go-review.googlesource.com/c/go/+/199139 Reviewed-by: Rob Pike <r@golang.org>
Dmitri Shuralyov [Fri, 13 Dec 2019 16:23:09 +0000 (11:23 -0500)]
doc: remove Go Security Policy page (moved to x/website)
This page has moved to the x/website repo in CL 211300 (commit
golang/website@3c8b7f99cadaa000e642595d0fabcd9ac496f335).
Remove the old copy in this repo since it's no longer used.
Jason A. Donenfeld [Thu, 21 Nov 2019 15:16:56 +0000 (16:16 +0100)]
runtime: do not use PowerRegisterSuspendResumeNotification on systems with "program time" timer
Systems where PowerRegisterSuspendResumeNotification returns ERROR_
FILE_NOT_FOUND are also systems where nanotime() is on "program time"
rather than "real time". The chain for this is:
Opening \\.\PdcPort fails with STATUS_OBJECT_NAME_NOT_FOUND when pdc.sys
hasn't been initialized. Pdc.sys also provides the various hooks for
sleep resumption events, which means if it's not loaded, then our "real
time" timer is actually on "program time". Finally STATUS_OBJECT_NAME_
NOT_FOUND is passed through RtlNtStatusToDosError, which returns ERROR_
FILE_NOT_FOUND. Therefore, in the case where the function returns ERROR_
FILE_NOT_FOUND, we don't mind, since the timer we're using will
correspond fine with the lack of sleep resumption notifications. This
applies, for example, to Docker users.
Fixes #35447
Fixes #35482
Change-Id: I9e1ce5bbc54b9da55ff7a3918b5da28112647eee
Reviewed-on: https://go-review.googlesource.com/c/go/+/208317 Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Matloob [Fri, 13 Dec 2019 21:53:28 +0000 (16:53 -0500)]
go/doc: ignore example functions with arguments
An Example function with arguments is not a valid example to be
run with go test. Don't return those functions from Examples. This
means that some functions that were previously showing up in
Examples will no longer show up. But those functions were not being
tested properly so the fact that they were showing up is misleading.
This fixes an issue where a confusing compiler error was showing
up when running go test on a file with an invalid example. While
that issue could have been fixed by returning an error, this is
more consistent with the behavior of go/doc.Examples, and the tests
checker in vet will catch this issue.
Fixes #35284
Change-Id: I2101a7d19f38522ef9c2e50967f9cfb30d28c730
Reviewed-on: https://go-review.googlesource.com/c/go/+/211357
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Alberto Donizetti [Fri, 13 Dec 2019 11:07:06 +0000 (12:07 +0100)]
spec: reword claim about the grammar being regular
Since the word "regular" has a precise meaning in the context of
formal languages, the Introduction sentence claiming that Go's grammar
is "compact and regular" may mislead readers.
Reword it using Rob's suggestion.
Fixes #36037
Change-Id: I00c1a5714bdab8878d9a77b36d67dae67d63da0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/211277 Reviewed-by: Robert Griesemer <gri@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Dan Scales [Wed, 11 Dec 2019 01:27:26 +0000 (17:27 -0800)]
runtime: force segv for nil defer function to be in deferreturn()
If the defer function pointer is nil, force the seg fault to happen in deferreturn
rather than in jmpdefer. jmpdefer is used fairly infrequently now because most
functions have open-coded defers.
The open-coded defer implementation calls gentraceback() with a callback when
looking for the first open-coded defer frame. gentraceback() throws an error if it
is called with a callback on an LR architecture and jmpdefer is on the stack,
because the stack trace can be incorrect in that case - see issue #8153. So, we
want to make sure that we don't have a seg fault in jmpdefer.
Fixes #36050
Change-Id: Ie25e6f015d8eb170b40248dedeb26a37b7f9b38d
Reviewed-on: https://go-review.googlesource.com/c/go/+/210978 Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Bryan C. Mills [Wed, 11 Dec 2019 17:14:31 +0000 (12:14 -0500)]
cmd/go: restore default vet analyzers for targets in GOROOT
This fixes a regression introduced in CL 209498,
found while investigating #32471.
Also fix $WORK replacement in cmd/go/internal/work.(*Builder).Showcmd
when b.WorkDir includes a backslash and appears in a quoted string.
That fix is needed in order to write a precise test that passes under Windows,
since Windows directories nearly always include backslashes.
Updates #35837
Change-Id: I5fddc5435d5d283a3e598989209d873b59b0a39c
Reviewed-on: https://go-review.googlesource.com/c/go/+/210937
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
Michael Anthony Knyszek [Mon, 18 Nov 2019 19:23:39 +0000 (19:23 +0000)]
runtime: use inUse ranges to map in summary memory only as needed
Prior to this change, if the heap was very discontiguous (such as in
TestArenaCollision) it's possible we could map a large amount of memory
as R/W and commit it. We would use only the start and end to track what
should be mapped, and we would extend that mapping as needed to
accomodate a potentially fragmented address space.
After this change, we only map exactly the part of the summary arrays
that we need by using the inUse ranges from the previous change. This
reduces the GCSys footprint of TestArenaCollision from 300 MiB to 18
MiB.
Because summaries are no longer mapped contiguously, this means the
scavenger can no longer iterate directly. This change also updates the
scavenger to borrow ranges out of inUse and iterate over only the
parts of the heap which are actually currently in use. This is both an
optimization and necessary for correctness.
Cherry Zhang [Wed, 11 Dec 2019 16:44:08 +0000 (11:44 -0500)]
test: add a test for gccgo compiler bug of missing type descriptor
The gccgo compiler did not generate type descriptor for a pointer
to a type alias defined in another package, causing linking error.
The fix is CL 210787. This CL adds a test.
Updates #36085.
Change-Id: I3237c7fedb4d92fb2dc610ee2b88087f96dc2a1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/210858
Run-TryBot: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Fri, 15 Nov 2019 23:30:30 +0000 (23:30 +0000)]
runtime: track ranges of address space which are owned by the heap
This change adds a new inUse field to the allocator which tracks ranges
of addresses that are owned by the heap. It is updated on each heap
growth.
These ranges are tracked in an array which is kept sorted. In practice
this array shouldn't exceed its initial allocation except in rare cases
and thus should be small (ideally exactly 1 element in size).
In a hypothetical worst-case scenario wherein we have a 1 TiB heap and 4
MiB arenas (note that the address ranges will never be at a smaller
granularity than an arena, since arenas are always allocated
contiguously), inUse would use at most 4 MiB of memory if the heap
mappings were completely discontiguous (highly unlikely) with an
additional 2 MiB leaked from previous allocations. Furthermore, the
copies that are done to keep the inUse array sorted will copy at most 4
MiB of memory in such a scenario, which, assuming a conservative copying
rate of 5 GiB/s, amounts to about 800µs.
However, note that in practice:
1) Most 64-bit platforms have 64 MiB arenas.
2) The copies should incur little-to-no page faults, meaning a copy rate
closer to 25-50 GiB/s is expected.
3) Go heaps are almost always mostly contiguous.
Bryan C. Mills [Wed, 11 Dec 2019 14:35:31 +0000 (09:35 -0500)]
net/http: use cancellation instead of a timeout in TestTransportProxyHTTPSConnectTimeout
The use of a timeout in this test caused it to be flaky: if the
timeout occurred before the connection was attempted, then the Accept
call on the Listener could hang indefinitely, and its goroutine would
not exit until that Listener was closed. That caused the test to fail.
A longer timeout would make the test less flaky, but it would become
even slower and would still be sensitive to timing.
Instead, replace the timeout with an explicit Context cancellation
after the CONNECT request has been read. That not only ensures that
the cancellation occurs at the appropriate point, but also makes the
test much faster: a test run with -count=1000 now executes in less
than 2s on my machine, whereas before it took upwards of 50s.
Andrew Stormont [Wed, 11 Dec 2019 00:31:44 +0000 (00:31 +0000)]
runtime: syscall_forkx on Solaris can return error on success
The syscall_forkx function returns the value of errno even on success. This can be a problem when using cgo where an atfork handler might be registered; if the atfork handler does something which causes errno to be set the caller of syscall_forkx can be misled into thinking the fork has failed. This causes the various exec functions in the runtime package to hang.
Change-Id: Ia1842179226078a0cbbea33d541aa1187dc47f68
GitHub-Last-Rev: 4dc4db75c82a826da9a50c323b7e3ddfe46ed6c0
GitHub-Pull-Request: golang/go#36076
Reviewed-on: https://go-review.googlesource.com/c/go/+/210742 Reviewed-by: Ian Lance Taylor <iant@golang.org>