Hana Kim [Wed, 29 Jan 2020 22:09:28 +0000 (17:09 -0500)]
doc/go1.14.html: describe the runtime/pprof change
golang.org/cl/204636
golang.org/cl/205097
Updates #36874
Change-Id: I773868fd027e9cc2187f0a738900f0fcb7711635
Reviewed-on: https://go-review.googlesource.com/c/go/+/216877 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Wed, 29 Jan 2020 14:14:50 +0000 (09:14 -0500)]
go/build: update TestImportDirNotExist to accept more detailed error strings
In CL 203820, we switched go/build to use the caller's working
directory for the main module (rather than srcDir), so that go/build
resolution now respects the requirements and replacements of the main
module. When the passed-in srcDir is empty, as of that CL we use "go
list" instead of falling back to in-process (GOPATH-mode) path lookup.
Unfortunately, that broke go/build.TestImportDirNotExist when
GO111MODULE=on: the test was looking for the specific error message
produced by the in-process lookup.
This change relaxes the test to accept the error message produced by
"go list" when srcDir is empty.
Joel Sing [Wed, 29 Jan 2020 09:14:18 +0000 (20:14 +1100)]
cmd/internal/obj/riscv,cmd/link: shorten the riscv64 call sequence
Now that the other dependent offset has been identified, we can remove the
unnecessary ADDI instruction from the riscv64 call sequence (reducing it
to AUIPC+JALR, rather than the previous AUIPC+ADDI+JALR).
Joel Sing [Wed, 29 Jan 2020 09:06:54 +0000 (20:06 +1100)]
cmd/compile,cmd/link: fix and re-enable open-coded defers on riscv64
The R_CALLRISCV relocation marker is on the JALR instruction, however the actual
relocation is currently two instructions previous for the AUIPC+ADDI sequence.
Adjust the platform dependent offset accordingly and re-enable open-coded defers.
Fixes #36786.
Change-Id: I71597c193c447930fbe94ce44b7355e89ae877bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/216797
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Daniel Martí [Wed, 22 Jan 2020 12:20:02 +0000 (12:20 +0000)]
doc: add the change to json.Compact in the 1.14 changelog
Fixes #36690.
Change-Id: Id4234ab9467270d51f0411375b71ece7f41269b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/215817 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Brad Fitzpatrick [Tue, 28 Jan 2020 21:40:08 +0000 (21:40 +0000)]
doc/go1.14: document RISC-V support
Fixes #36708
Updates #27532
Change-Id: I9e3bb92d15825e2c4a505e1aea41b2897f18e0ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/216757 Reviewed-by: Joel Sing <joel@sing.id.au> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 28 Jan 2020 00:35:28 +0000 (16:35 -0800)]
net/http: don't treat an alternate protocol as a known round tripper
As of CL 175857, the client code checks for known round tripper
implementations, and uses simpler cancellation code when it finds one.
However, this code was not considering the case of a request that uses
a user-defined protocol, where the user-defined protocol was
registered with the transport to use a different round tripper.
The effect was that round trippers that worked with earlier
releases would not see the expected cancellation semantics with tip.
Fixes #36820
Change-Id: I60e75b5d0badcfb9fde9d73a966ba1d3f7aa42b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/216618
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Brad Fitzpatrick [Tue, 28 Jan 2020 22:57:18 +0000 (22:57 +0000)]
cmd/dist: remove riscv64 from set of incomplete ports
Fixes #27532
Fixes #36853
Updates #28944
Change-Id: I4d0f212deb361c941ce7e5999e237a951c89a296
Reviewed-on: https://go-review.googlesource.com/c/go/+/216758 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joel Sing <joel@sing.id.au>
Michael Anthony Knyszek [Tue, 28 Jan 2020 19:59:19 +0000 (19:59 +0000)]
runtime: ensure that searchAddr always refers to inUse memory
This change formalizes an assumption made by the page allocator, which
is that (*pageAlloc).searchAddr should never refer to memory that is not
represented by (*pageAlloc).inUse. The portion of address space covered
by (*pageAlloc).inUse reflects the parts of the summary arrays which are
guaranteed to mapped, and so looking at any summary which is not
reflected there may cause a segfault.
In fact, this can happen today. This change thus also removes a
micro-optimization which is the only case which may cause
(*pageAlloc).searchAddr to point outside of any region covered by
(*pageAlloc).inUse, and adds a test verifying that the current segfault
can no longer occur.
Dmitri Shuralyov [Tue, 28 Jan 2020 18:20:57 +0000 (13:20 -0500)]
src/go.mod: import x/crypto/cryptobyte security fix for 32-bit archs
cryptobyte: fix panic due to malformed ASN.1 inputs on 32-bit archs
When int is 32 bits wide (on 32-bit architectures like 386 and arm), an
overflow could occur, causing a panic, due to malformed ASN.1 being
passed to any of the ASN1 methods of String.
Tested on linux/386 and darwin/amd64.
This fixes CVE-2020-7919 and was found thanks to the Project Wycheproof
test vectors.
x/crypto/cryptobyte is used in crypto/x509 for parsing certificates.
Malformed certificates might cause a panic during parsing on 32-bit
architectures (like arm and 386).
Joel Sing [Wed, 22 Jan 2020 16:38:31 +0000 (03:38 +1100)]
cmd/internal/obj/riscv: use signed immediates for U-instructions
On RISCV64, the U-instructions (AUIPC and LUI) take 20 bits, append 12 bits
of zeros and sign extend to 64-bits. As such, the 20 bit immediate value is
signed not unsigned.
Duplicate the register definitions and names to avoid importing the
cmd/internal/obj/riscv64 package. This makes it possible to build compiler rules
with a stable Go tool chain.
Cherry Zhang [Fri, 24 Jan 2020 19:11:04 +0000 (14:11 -0500)]
cmd/compile: on PPC64, fold offset into some loads/stores only when offset is 4-aligned
On PPC64, MOVWload, MOVDload, and MOVDstore are assembled to a
"DS from" instruction which requiers the offset is a multiple of
4. Only fold offset to such instructions if it is a multiple of 4.
Fixes #36723.
"GOARCH=ppc64 GOOS=linux go build -gcflags=all=-d=ssa/check/on std cmd"
passes now.
Change-Id: I67f2a6ac02f0d33d470f68ff54936c289a4c765b
Reviewed-on: https://go-review.googlesource.com/c/go/+/216379 Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Bryan C. Mills [Fri, 24 Jan 2020 15:26:24 +0000 (10:26 -0500)]
doc/go1.14: note that module support is ready for production use
The public proxy and checksum database launched with Go 1.13 have been
running smoothly, pkg.go.dev is serving module-aware documentation,
and in 1.14 we have improved the vendoring workflow and finished
Subversion support to reach parity with GOPATH mode for users of those
features, updated documentation (including the “How to Write Go Code”
intro document) and published blog posts describing common modes of
usage, and improved the migration path for existing v2+ modules by
making version resolution less aggressive about "+incompatible" major
versions.
We (always) have more fit-and-finish work to do, but at this point we
believe that module mode will provide a better user experience than
GOPATH mode for most users, including in production use.
Reason for revert: This change may be causing latency problems
for applications which call ReadMemStats, because it may cause
all goroutines to stop until the GC completes.
https://golang.org/cl/215157 fixes this problem, but it's too
late in the cycle to land that.
Carlos Amedee [Thu, 23 Jan 2020 21:05:29 +0000 (16:05 -0500)]
cmd/link: ensure cgo cflags do not leak into dwarf tests
Running the dwarf tests with CGO_CFLAGS set
with certain values would cause the test to fail. all.bash
would fail when CGO_CFLAGS was set to '-mmacosx-version-min=10.10'
because the --macosx-version-min flag is incompatible with some dwarf
tests. The change guards against using an unintended flag in the unit test.
Updates #35459
Change-Id: Idc9b354aba44fdab424cb0081a4b3ea7a6d0f8e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/216177
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jay Conrod [Fri, 24 Jan 2020 16:25:52 +0000 (08:25 -0800)]
cmd/go: fix cgo test when min macOS version is set
Regression tests for #24161 use a macro to conditionally compile some
stub definitions. The macro tests that the minimum macOS version is
less than 10.12.
We get duplicate definitions when building this test with
CGO_CFLAGS=-mmacosx-version-min=10.x where 10.x < 10.12. With this
change, we use a different macro, __MAC_OS_X_VERSION_MAX_ALLOWED__,
which tests the SDK version instead of the minimum macOS version. This
checks whether these definitions are present in headers.
After this change, 'go tool dist test cgo_test' should pass with
CGO_FLAGS=-mmacosx-version-min=10.10.
Updates #35459
Change-Id: I88d63601c94b0369c73c38d216a2d41ba7d4e579
Reviewed-on: https://go-review.googlesource.com/c/go/+/216243
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Filippo Valsorda [Tue, 21 Jan 2020 19:45:15 +0000 (14:45 -0500)]
crypto/x509: mitigate CVE-2020-0601 verification bypass on Windows
An attacker can trick the Windows system verifier to use a poisoned set
of elliptic curve parameters for a trusted root, allowing it to generate
spoofed signatures. When this happens, the returned chain will present
the unmodified original root, so the actual signatures won't verify (as
they are invalid for the correct parameters). Simply double check them
as a safety measure and mitigation.
Windows users should still install the system security patch ASAP.
Carlos Amedee [Wed, 22 Jan 2020 20:30:52 +0000 (15:30 -0500)]
cmd/link: ensure cgo cflags do not leak into tvOS test
Running the 'TestBuildForTvOS' test with CGO_CFLAGS set
with certain values would cause the test to fail. all.bash
would fail when CGO_CFLAGS was set to '-mmacosx-version-min=10.10'
because the --macosx-version-min flag is incompatible with tvOS.
The change guards against using an unintended flag in the unit test.
Updates #35459
Change-Id: Ifc43f3ebfb23d37aabeaac2ea9efae5b877991bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/215957
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Thu, 23 Jan 2020 17:51:26 +0000 (12:51 -0500)]
cmd/go: add a control case to the mod_vendor_trimpath test
In reviewing CL 215940, it took me a while to find the control
condition for the test, which was located in build_cache_trimpath.txt.
We could consolidate the two tests, but since they check for
regressions of separate issues (with separate root-causes), I think it
makes sense to keep them separate.
However, I would like the control condition to be present in the same
source file, so that we'll be more likely to update both cases if the
behavior of one of them is changed.
Updates #36566
Change-Id: Ic588f1dfb7977dd78d1d5ef61b9841e22bad82e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/216018
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Daniel Martí [Tue, 21 Jan 2020 10:21:17 +0000 (10:21 +0000)]
cmd/go: make Script/test_regexps less flaky under load
With the command below, I was able to reproduce failures within the
first 50 or so runs:
go test -c -o test && stress -p 32 ./test -test.run Script/test_regexp
When printing the full failure output, we'd see:
BenchmarkX
BenchmarkX: x_test.go:13: LOG: X running N=1
BenchmarkX/Y
BenchmarkX/Y: x_test.go:15: LOG: Y running N=1
BenchmarkX/Y: x_test.go:15: LOG: Y running N=100
BenchmarkX/Y: x_test.go:15: LOG: Y running N=10000
BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000
BenchmarkX/Y: x_test.go:15: LOG: Y running N=100000000
BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000000
BenchmarkX/Y 1000000000 0.000050 ns/op
BenchmarkX/Y: x_test.go:15: LOG: Y running N=1
BenchmarkX/Y: x_test.go:15: LOG: Y running N=30
BenchmarkX/Y: x_test.go:15: LOG: Y running N=1207
BenchmarkX/Y: x_test.go:15: LOG: Y running N=120700
BenchmarkX/Y: x_test.go:15: LOG: Y running N=12070000
BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000000
BenchmarkX/Y 1000000000 0.000715 ns/op
In other words, the N values aren't required to be exact. It seems like
they are cut short if the machine is under stress. That's the exact
scenario we reproduce above, since I used -p=32 on my laptop with only 4
real CPU cores.
First, don't require each line to be present. Instead, use patterns
that span multiple lines, so that we can just match the first and last
N= lines.
Second, don't require the last N= lines to be exact; simply require
them to have a reasonably large number of digits.
Fixes #36664.
Change-Id: I7a9818f1a07099fa6482a26da2ac5cbea0f8ab30
Reviewed-on: https://go-review.googlesource.com/c/go/+/215578
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Jay Conrod [Wed, 22 Jan 2020 23:46:11 +0000 (15:46 -0800)]
cmd/go: unify trimpath logic for -mod=vendor, -mod=mod
If a package has a module with a version, the package's directory is
replaced with the module path and version, followed by the package's
path within the module.
This is a follow up to CL 214945. We no longer check whether the
module has a directory (with -mod=vendor, it does not).
Updates #36566
Change-Id: I5bc952b13bc7b4659f58ee555bd6c6a087eb7792
Reviewed-on: https://go-review.googlesource.com/c/go/+/215940
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Joel Sing [Wed, 22 Jan 2020 16:31:49 +0000 (03:31 +1100)]
cmd/internal/obj/riscv: restore the ADDI instruction in jalrToSym
While this instruction is not needed for the relocation (the lower immediate
can be patched directly into the JALR instruction), other code currently
depends on the jump sequence being 12 bytes (or three instructions) long.
Put the ADDI instruction back until these can be found and fixed.
Reason for revert: the assumption that a T-E of "gzip" implies
"chunked" seems incorrect. The RFC does state that one "MUST apply
chunked as the final transfer coding" but that should be interpreted to
mean that a "chunked" encoding must be listed as the last one, not that
one should be assumed to be there if not. This is confirmed by the
alternative option to chunking on the server side being to "terminate
the message by closing the connection".
The issue seems confirmed by the fact that the code in the body of
#29162 fails with the following error:
net/http: HTTP/1.x transport connection broken: http: failed to gunzip body: unexpected EOF
This late in the cycle, revert rather than fix, also because we don't
apparently have tests for the correct behavior.
Austin Clements [Mon, 6 Jan 2020 16:10:26 +0000 (11:10 -0500)]
runtime: document special memmove requirements
Unlike C's memmove, Go's memmove must be careful to do indivisible
writes of pointer values because it may be racing with the garbage
collector reading the heap.
We've had various bugs related to this over the years (#36101, #13160,
#12552). Indeed, memmove is a great target for optimization and it's
easy to forget the special requirements of Go's memmove.
The CL documents these (currently unwritten!) requirements. We're also
adding a test that should hopefully keep everyone honest going
forward, though it's hard to be sure we're hitting all cases of
memmove.
Ian Lance Taylor [Wed, 22 Jan 2020 00:50:56 +0000 (16:50 -0800)]
runtime: don't skip checkTimers if we would clear deleted timers
The timers code used to have a problem: if code started and stopped a
lot of timers, as would happen with, for example, lots of calls to
context.WithTimeout, then it would steadily use memory holding timers
that had stopped but not been removed from the timer heap.
That problem was fixed by CL 214299, which would remove all deleted
timers whenever they got to be more than 1/4 of the total number of
timers on the heap.
The timers code had a different problem: if there were some idle P's,
the running P's would have lock contention trying to steal their timers.
That problem was fixed by CL 214185, which only acquired the timer lock
if the next timer was ready to run or there were some timers to adjust.
Unfortunately, CL 214185 partially undid 214299, in that we could now
accumulate an increasing number of deleted timers while there were no
timers ready to run. This CL restores the 214299 behavior, by checking
whether there are lots of deleted timers without acquiring the lock.
This is a performance issue to consider for the 1.14 release.
Change-Id: I13c980efdcc2a46eb84882750c39e3f7c5b2e7c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/215722
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Jay Conrod [Tue, 21 Jan 2020 22:50:36 +0000 (14:50 -0800)]
cmd/go: add -d flag to mod_get_test
'go get all' was run in this test without -d. This caused some std
packages to be reinstalled if the test is run in a slightly different
configuration than make.bash was run. run.bash would fail in some
situations because of this. Nothing in the cmd/go tests should modify
installed std or cmd packages.
Updates #35459
Change-Id: Idd259a27d55502923b7fc54f361a77f0ac11eea2
Reviewed-on: https://go-review.googlesource.com/c/go/+/215721
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Joel Sing [Thu, 16 Jan 2020 15:51:40 +0000 (02:51 +1100)]
cmd/internal/obj/riscv: correctly split immediates for FLW/FLD/FSW/FSD
The FLW/FLD/FSW/FSD instructions can have immediates that exceed 12-bits and
therefore cannot be encoded in the RISCV instruction. Handle these as we do
for other load/store instructions. Also add test coverage for all load/store
instructions with large immediates.
Fixes compilation issue reported by Carlos Eduardo de Paula.
Updates #27532
Change-Id: Ifa62f19493b3acaba5a90ac31d2df209a3afea81
Reviewed-on: https://go-review.googlesource.com/c/go/+/215037 Reviewed-by: Carlos Eduardo de Paula <me@carlosedp.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Kévin Dunglas [Tue, 21 Jan 2020 21:48:41 +0000 (21:48 +0000)]
net/http/httputil: add docs about X-Forwarded-For in ReverseProxy
ReverseProxy automatically sets the X-Forwarded-For header, if the request
already contains a X-Forwarded-For header, the value of the client IP is
appended to the existing header value.
This behavior isn't documented anywhere, and can lead to IP spoofing
security issues is the client is untrusted (the most common situation).
This PR documents this behavior.
For future versions, I proposed #36678 that implements a more secure
default behavior and adds support for other forwarded headers.
Carlos Eduardo Seo [Fri, 17 Jan 2020 20:59:59 +0000 (17:59 -0300)]
runtime: fix wrong offset when calling ppc64x nanotime syscall
There is a wrong offset when getting the results of a clock_gettime
syscall. Although the syscall will never be called in native ppc64x,
QEMU doesn't implement VDSO, so it will return wrong values.
The point of *net.OpError is to add details to an underlying lower
level error. It makes no sense to have an OpError without an Err and
a nil *OpError.Err will cause *OpError.Error() method to panic.
Fixes #33007
Change-Id: If4fb2501e02dad110a095b73e18c47312ffa6015
Reviewed-on: https://go-review.googlesource.com/c/go/+/187677 Reviewed-by: Rob Pike <r@golang.org>
Roger Peppe [Fri, 17 Jan 2020 09:06:13 +0000 (09:06 +0000)]
testing: clarify that Cleanup is run after subtests complete.
It's good to be explicit, as it's not necessarily obvious (and indeed
the behavior has changed recently with https://go-review.googlesource.com/c/go/+/214822)
without an associated doc comment change).
Change-Id: I99d6398bf15b404b1b1b196e712e926e363251e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/215217 Reviewed-by: Paul Jolly <paul@myitcv.org.uk> Reviewed-by: Ian Lance Taylor <iant@golang.org>
David Chase [Fri, 17 Jan 2020 17:03:55 +0000 (12:03 -0500)]
cmd/compile: change the "bogus line" to be 1
The previous value was "too bogus" and caused objdump to crash.
Updated infinite loop test results (only run if -args -f) in ssa/debug_test.go
Probably also fixes #36621 but that bug needs more info to tell for certain.
Fixes #36570
Change-Id: I51144641d25d559308a98d726d87806bd340cc5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/215297
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Jay Conrod [Wed, 15 Jan 2020 21:29:46 +0000 (16:29 -0500)]
cmd/go: trim paths from vendored packages with -trimpath
In CL 199821, we stopped setting the module directory for vendored
packages when -mod=vendor is used. This broke -trimpath, since we
replace the module directory with a string derived from the module
path and version. A comment in CL 202977 makes it clear that the
module directory should not be set.
With this change, -trimpath falls back to replacing the package
directory with the package path if the module directory is not set. We
also fall back to replacing the package directory if the module
version is not set to avoid adding a meaningless @ only for the main
module.
As a consequence of this change, file names in vendored packages will
not have module versions, so file names will be a little different
between -mod=mod and -mod=vendor.
Fixes #36566
Change-Id: I0e9cd76d36a2028a49d0b6697ea9a9b3140d7ff3
Reviewed-on: https://go-review.googlesource.com/c/go/+/214945
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Ian Lance Taylor [Mon, 13 Jan 2020 20:17:26 +0000 (12:17 -0800)]
runtime: keep P's first timer when in new atomically accessed field
This reduces lock contention when only a few P's are running and
checking for whether they need to run timers on the sleeping P's.
Without this change the running P's would get lock contention
while looking at the sleeping P's timers. With this change a single
atomic load suffices to determine whether there are any ready timers.
Change-Id: Ie843782bd56df49867a01ecf19c47498ec827452
Reviewed-on: https://go-review.googlesource.com/c/go/+/214185
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com>
Michael Anthony Knyszek [Mon, 13 Jan 2020 17:18:51 +0000 (17:18 +0000)]
runtime: better approximate total cost of scavenging
Currently, the scavenger is paced according to how long it takes to
scavenge one runtime page's worth of memory. However, this pacing
doesn't take into account the additional cost of actually using a
scavenged page. This operation, "sysUsed," is a counterpart to the
scavenging operation "sysUnused." On most systems this operation is a
no-op, but on some systems like Darwin and Windows we actually make a
syscall. Even on systems where it's a no-op, the cost is implicit: a
more expensive page fault when re-using the page.
On Darwin in particular the cost of "sysUnused" is fairly close to the
cost of "sysUsed", which skews the pacing to be too fast. A lot of
soon-to-be-allocated memory ends up scavenged, resulting in many more
expensive "sysUsed" operations, ultimately slowing down the application.
The way to fix this problem is to include the future cost of "sysUsed"
on a page in the scavenging cost. However, measuring the "sysUsed" cost
directly (like we do with "sysUnused") on most systems is infeasible
because we would have to measure the cost of the first access.
Instead, this change applies a multiplicative constant to the measured
scavenging time which is based on a per-system ratio of "sysUnused" to
"sysUsed" costs in the worst case (on systems where it's a no-op, we
measure the cost of the first access). This ultimately slows down the
scavenger to a more reasonable pace, limiting its impact on performance
but still retaining the memory footprint improvements from the previous
release.
Ian Lance Taylor [Mon, 13 Jan 2020 23:34:53 +0000 (23:34 +0000)]
Revert "os: handle long path in RemoveAll for windows"
This reverts CL 214437.
Does not fix the issue, and the test was wrong so it did not detect that it did not fix the issue.
Updates #36375
Change-Id: I6a4112035a1e90f4fdafed6fdf4ec9dfc718b571
Reviewed-on: https://go-review.googlesource.com/c/go/+/214601 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Rob Pike [Thu, 9 Jan 2020 05:22:31 +0000 (16:22 +1100)]
time: add a sentence about distant times to time.Time.Unix
Since Durations only span 290 years, they are not good for
manipulating very remote times. I bounced off this problem recently
while doing some astronomical calculations and it took me a while to
realize I could get a 64-bit seconds value from time.Time.Unix and
subtract two of them to evaluate the interval.
I thought it worth adding a sentence to make this clear. It didn't
occur to me for quite a while that "Unix time" spans a huge range in
the Go library.
Robert Griesemer [Mon, 13 Jan 2020 21:01:28 +0000 (13:01 -0800)]
spec: uniformly format empty interfaces as "interface{}"
Fixes #36526.
Change-Id: Ic51a287579f139422cc1a7b2fb82d6732114b031
Reviewed-on: https://go-review.googlesource.com/c/go/+/214597 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Constantin Konstantinidis [Sat, 11 Jan 2020 18:53:20 +0000 (19:53 +0100)]
os: handle long path in RemoveAll for windows
Fixes #36375
Change-Id: I407a1db23868880b83e73bc136d274659483fb69
Reviewed-on: https://go-review.googlesource.com/c/go/+/214437
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Thu, 9 Jan 2020 22:58:18 +0000 (14:58 -0800)]
cmd/compile: print recursive types correctly
Change the type printer to take a map of types that we're currently
printing. When we happen upon a type that we're already in the middle
of printing, print a reference to it instead.
A reference to another type is built using the offset of the first
byte of that type's string representation in the result. To facilitate
that computation (and it's probably more efficient, regardless), we
print the type to a buffer as we go, and build the string at the end.
It would be nice to use string.Builder instead of bytes.Buffer, but
string.Builder wasn't around in Go 1.4, and we'd like to bootstrap
from that version.
Fixes #29312
Change-Id: I49d788c1fa20f770df7b2bae3b9979d990d54803
Reviewed-on: https://go-review.googlesource.com/c/go/+/214239 Reviewed-by: Robert Griesemer <gri@golang.org>
Ian Lance Taylor [Fri, 10 Jan 2020 18:10:32 +0000 (10:10 -0800)]
runtime: don't skip timer when adjustTimers sees a modified timer
When adjustTimers sees a timerModifiedEarlier or timerModifiedLater,
it removes it from the heap, leaving a new timer at that position
in the heap. We were accidentally skipping that new timer in our loop.
In some unlikely cases this could cause adjustTimers to look at more
timers than necessary.
Change-Id: Ic71e54c175ab7d86a7fa46f1497aca71ed1c43cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/214338
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Jay Conrod [Thu, 9 Jan 2020 21:22:01 +0000 (16:22 -0500)]
cmd/go: remove references to 1.13 in 'go help modules'
In "Module support" section, there were two mentions of "Go 1.13",
assuming that's the latest version. Rather than update these to 1.14,
this CL changes those to "The go command".
Also, a minor change in wording for finding go.mod files.
Change-Id: Id194be9405b540f221464814e71c361a22cc0f55
Reviewed-on: https://go-review.googlesource.com/c/go/+/214140
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Dmitri Shuralyov [Wed, 8 Jan 2020 18:50:30 +0000 (13:50 -0500)]
CONTRIBUTORS: first round of updates for Go 1.14
This update was automatically generated using the updatecontrib command:
cd gotip
go run golang.org/x/build/cmd/updatecontrib
With minor manual changes based on publicly available information
to canonicalize letter case and formatting for a few names.
Actions taken (relative to CONTRIBUTORS at origin/master):
Added Aaron Beitch <aaronb@arista.com>
Added Adam Shelton <aashelt90@gmail.com>
Added Aditya Harindar <aditya.harindar@gmail.com>
Added Alec Benzer <alec.benzer@gmail.com>
Added Alex Buchanan <buchanae@gmail.com>
Added Alex Gaynor <alex@alloy.us>
Added Alex Harford <alex.harford@saucelabs.com>
Added Alex Zhirov <azhirov@google.com>
Added Alexandr Mayorskiy <a.mayorskiy@corp.mail.ru>
Added An Xiao <hac@zju.edu.cn>
Added Andrei Enshin <b1os@bk.ru>
Added Andrew Medvedev <andrew.y.medvedev@gmail.com>
Added Andrew Stormont <astormont@racktopsystems.com>
Added Antonio Huete Jimenez <tuxillo@quantumachine.net>
Added Arda Güçlü <ardaguclu@gmail.com>
Added Artem Alekseev <artem.alekseev@intel.com>
Added Ayan George <ayan@ayan.net>
Added Barnaby Keene <accounts@southcla.ws>
Added Ben Toews <mastahyeti@gmail.com>
Added Benjamin Wuethrich <benjamin.wuethrich@gmail.com>
Added Brandon Philips <brandon@ifup.org>
Added Carlos Amedee <carlos@golang.org>
Added Changkun Ou <hi@changkun.us>
Added Chauncy Cullitan <chauncyc@google.com>
Added Clint J. Edwards <clint.j.edwards@gmail.com>
Added Daisuke Suzuki <daisuzu@gmail.com>
Added Dan Scales <danscales@google.com>
Added David Bond <davidsbond93@gmail.com>
Added Denis Isaev <idenx@yandex.com>
Added Domas Tamašauskas <puerdomus@gmail.com>
Added Duco van Amstel <duco.vanamstel@gmail.com>
Added Eduardo Villaseñor <evillasrmx@gmail.com>
Added Eric Biggers <ebiggers@google.com>
Added Eric Brown <browne@vmware.com>
Added Eric Rutherford <erutherford@gmail.com>
Added Gabriel Rosenhouse <rosenhouse@gmail.com>
Added George Hartzell <hartzell@alerce.com>
Added Gert Cuykens <gert.cuykens@gmail.com>
Added Ghazni Nattarshah <ghazni.nattarshah@gmail.com>
Added GitHub User DQNEO (188741) <dqneoo@gmail.com>
Added GitHub User Matts966 (28551465) <Matts966@users.noreply.github.com>
Added GitHub User ajz01 (4744634) <ajzdenek@gmail.com>
Added GitHub User andig (184815) <cpuidle@gmx.de>
Added GitHub User jinmiaoluo (39730824) <jinmiaoluo@icloud.com>
Added GitHub User maltalex (10195391) <code@bit48.net>
Added GitHub User po3rin (29445112) <abctail30@gmail.com>
Added GitHub User pokutuna (57545) <popopopopokutuna@gmail.com>
Added GitHub User ramenjuniti (32011829) <ramenjuniti@gmail.com>
Added GitHub User skanehira (7888591) <sho19921005@gmail.com>
Added GitHub User witchard (4994659) <witchard@hotmail.co.uk>
Added GitHub User zikaeroh (48577114) <zikaeroh@gmail.com>
Added Günther Noack <gnoack@google.com>
Added Hasit Bhatt <hasit.p.bhatt@gmail.com>
Added Howard Zhang <howard.zhang@arm.com>
Added Huan Du <i@huandu.me>
Added Ignacio Hagopian <jsign.uy@gmail.com>
Added Ilya Sinelnikov <sidhmangh@gmail.com>
Added Irbe Krumina <irbekrm@gmail.com>
Added Isfan Azhabil <isfan.azhabil@tokopedia.com>
Added Ivan Trubach <mr.trubach@icloud.com>
Added Jaap Aarts <jaap.aarts1@gmail.com>
Added Jeremy Faller <jeremy@golang.org>
Added Johan Jansson <johan.jansson@iki.fi>
Added John Papandriopoulos <jpap.code@gmail.com>
Added Jorge L. Fatta <jorge.fatta@auth0.com>
Added Josa Gesell <josa@gesell.me>
Added Julian Tibble <julian.tibble@gmail.com>
Added Jun Zhang <jim.zoumo@gmail.com>
Added Jędrzej Szczepaniak <jbszczepaniak@gmail.com>
Added Kalman Bekesi <kalmanb@google.com>
Added Katharine Berry <ktbry@google.com>
Added Keisuke Kishimoto <keisuke.kishimoto@gmail.com>
Added Kevan Swanberg <kevswanberg@gmail.com>
Added Koki Tomoshige <tomocy.dev@gmail.com>
Added Liz Rice <liz@lizrice.com>
Added Lorenz Brun <lorenz@brun.one>
Added Luke Young <bored-engineer@users.noreply.github.com>
Added Marc Sanmiquel <marcsanmiquel@gmail.com>
Added Marko Kungla <marko.kungla@gmail.com>
Added Mikhail Fesenko <proggga@gmail.com>
Added Mohit Verma <vmohit.93@gmail.com>
Added Moritz Fain <moritz@fain.io>
Added Nathan Dias <nathan.dias@orijtech.com>
Added Nikita Vanyasin <nikita.vanyasin@gmail.com>
Added Nuno Cruces <ncruces@users.noreply.github.com>
Added Oliver Powell <oliverpowell84@gmail.com>
Added Panos Georgiadis <pgeorgiadis@suse.de>
Added Pantelis Sampaziotis <psampaz@gmail.com>
Added Paulo Gomes <paulo.gomes.uk@gmail.com>
Added Prashant Agrawal <prashant.a.vjti@gmail.com>
Added Ricardo Seriani <ricardo.seriani@gmail.com>
Added Rob Findley <rfindley@google.com>
Added Robin Zhong <robin@robinzhong.co>
Added Rohan Challa <rohan@golang.org>
Added Roman Kollár <roman.kollar.0@gmail.com>
Added Ruixin Bao <ruixin.bao@ibm.com>
Added Sardorbek Pulatov <sardorbek.pulatov@outlook.com>
Added Scott Ragan <ragansa@fb.com>
Added Sean Liao <seankhliao@gmail.com>
Added Sebastian Chlopecki <sebsebmc@gmail.com>
Added Sebastian Kinne <skinne@google.com>
Added Sergei Lemeshkin <sergeilem@gmail.com>
Added Serhat Giydiren <serhatgiydiren@gmail.com>
Added Shivashis Padhi <shivashispadhi@gmail.com>
Added Simarpreet Singh <simar@linux.com>
Added Simon Ferquel <simon.ferquel@docker.com>
Added Simon Rozman <simon@rozman.si>
Added Sjoerd Siebinga <sjoerd.siebinga@gmail.com>
Added Spencer Kocot <spencerkocot@gmail.com>
Added Srinidhi Kaushik <shrinidhi.kaushik@gmail.com>
Added Sven Taute <sven.taute@gmail.com>
Added Tao Qingyun <qingyunha@gmail.com>
Added Tianji Wu <the729@gmail.com>
Added Tomas Dabasinskas <tomas@dabasinskas.net>
Added Ville Skyttä <ville.skytta@iki.fi>
Added Vitaly Zdanevich <zdanevich.vitaly@ya.ru>
Added Vladimir Evgrafov <evgrafov.vladimir@gmail.com>
Added Vojtech Bocek <vbocek@gmail.com>
Added Wang Xuerui <git@xen0n.name>
Added William Poussier <william.poussier@gmail.com>
Added Xiangdong Ji <xiangdong.ji@arm.com>
Added Yuichi Nishiwaki <yuichi.nishiwaki@gmail.com>
Added Ziheng Liu <lzhfromustc@gmail.com>
Used GitHub User DQNEO (188741) form for DQNEO <dqneoo@gmail.com> https://github.com/golang/go/commit/f07059d949 [go]
Used GitHub User Matts966 (28551465) form for Matts966 <Matts966@users.noreply.github.com> https://github.com/golang/tools/commit/d89860af [tools]
Used GitHub User ajz01 (4744634) form for ajz01 <ajzdenek@gmail.com> https://github.com/golang/go/commit/57ad6ef15d [go]
Used GitHub User andig (184815) form for andig <cpuidle@gmx.de> https://github.com/golang/go/commit/cf630586ca [go sys]
Used GitHub User jinmiaoluo (39730824) form for jinmiaoluo <jinmiaoluo@icloud.com> https://github.com/golang/go/commit/c7e73ef60a [go]
Used GitHub User maltalex (10195391) form for maltalex <code@bit48.net> https://github.com/golang/sys/commit/c709ea0 [sys]
Used GitHub User po3rin (29445112) form for po3rin <abctail30@gmail.com> https://github.com/golang/go/commit/bf865823ba [go]
Used GitHub User pokutuna (57545) form for pokutuna <popopopopokutuna@gmail.com> https://github.com/golang/go/commit/46e0d724b3 [go]
Used GitHub User ramenjuniti (32011829) form for ramenjuniti <ramenjuniti@gmail.com> https://github.com/golang/go/commit/f9dd99cae3 [go]
Used GitHub User skanehira (7888591) form for skanehira <sho19921005@gmail.com> https://github.com/golang/go/commit/87805c92fd [go]
Used GitHub User utkarsh-extc (53217283) form for utkarsh-extc <53217283+utkarsh-extc@users.noreply.github.com> https://github.com/golang/sys/commit/51ab0e2 [sys]
Used GitHub User witchard (4994659) form for witchard <witchard@hotmail.co.uk> https://github.com/golang/go/commit/42db1da8e9 [go]
Used GitHub User zikaeroh (48577114) form for zikaeroh <zikaeroh@gmail.com> https://github.com/golang/tools/commit/e84277c2 [tools]
Used GitHub name "Adam Shelton" for Adam <aashelt90@gmail.com> https://github.com/golang/exp/commit/c286b88 [exp]
Used GitHub name "Andrew Bonventre" for Andrew <andybons@golang.org> https://github.com/golang/go/commit/8bbfc51d9a [arch blog build crypto debug exp gddo go image lint mobile net oauth2 perf playground proposal.git review sync sys talks term text time tools tour website]
Used GitHub name "David Bond" for davidsbond <davidsbond93@gmail.com> https://github.com/golang/go/commit/b421b85841 [go]
Used GitHub name "Eduardo Villaseñor" for galaxy-designer <evillasrmx@gmail.com> https://github.com/golang/tools/commit/db047d72 [tools]
Used GitHub name "George Hartzell" for hartzell <hartzell@alerce.com> https://github.com/golang/tools/commit/5eefd052 [tools]
Used GitHub name "Ignacio Hagopian" for jsign <jsign.uy@gmail.com> https://github.com/golang/go/commit/4d4ddd862d [go]
Used GitHub name "Ivan Markin" for nogoegst <nogoegst@users.noreply.github.com> https://github.com/golang/go/commit/a1addf15df [go]
Used GitHub name "Jun Zhang" for zoumo <jim.zoumo@gmail.com> https://github.com/golang/tools/commit/81ca6dc7 [tools]
Used GitHub name "Keiji Yoshida" for yosssi <yoshida.keiji.84@gmail.com> https://github.com/golang/lint/commit/ac6833c [lint]
Used GitHub name "Koki Tomoshige" for tomocy <tomocy.dev@gmail.com> https://github.com/golang/go/commit/2f04903fec [go]
Used GitHub name "Michalis Kargakis" for kargakis <mkargaki@redhat.com> https://github.com/golang/go/commit/e243d242d7 [go]
Used GitHub name "Nikita Vanyasin" for nikita-vanyasin <nikita.vanyasin@gmail.com> https://github.com/golang/go/commit/c3e8a20a65 [go]
Used GitHub name "Roberto Clapis" for Roberto <empijei@users.noreply.github.com> https://github.com/golang/go/commit/963776e689 [go]
Used GitHub name "Robin Eklind" for mewmew <rnd0x00@gmail.com> https://github.com/golang/go/commit/b8620afb8d [arch blog go proposal.git]
Used GitHub name "Sergei Lemeshkin" for sergeilem <sergeilem@gmail.com> https://github.com/golang/go/commit/a3a1bdff79 [go]
Used GitHub name "Vladimir Evgrafov" for vovapi <evgrafov.vladimir@gmail.com> https://github.com/golang/go/commit/207a0b7933 [go]
Michael Anthony Knyszek [Fri, 27 Dec 2019 16:48:23 +0000 (16:48 +0000)]
runtime: add scavtrace debug flag and remove scavenge info from gctrace
Currently, scavenging information is printed if the gctrace debug
variable is >0. Scavenging information is also printed naively, for
every page scavenged, resulting in a lot of noise when the typical
expectation for GC trace is one line per GC.
This change adds a new GODEBUG flag called scavtrace which prints
scavenge information roughly once per GC cycle and removes any scavenge
information from gctrace. The exception is debug.FreeOSMemory, which may
force an additional line to be printed.