This patch enables VSLI, VUADDW(2), VUSRA and FMOVQ SIMD instructions
required by the issue #40725. And the GNU syntax of 'FMOVQ' is 128-bit
ldr/str(immediate, simd&fp).
David Chase [Tue, 13 Oct 2020 23:24:04 +0000 (19:24 -0400)]
cmd/compile: delay expansion of OpArg until expand_calls
As it says, delay expanpsion of OpArg to the expand_calls phase,
to enable (eventually) interprocedural SSA optimizations, and
(sooner) change to a register ABI.
Includes a round of cleanup to function names and comments,
largely to match the expanded scope of the functions.
This CL removes the per-function dependence on GOSSAHASH,
but the go116lateCallExpansion kill switch remains (and was
tested locally to ensure it worked).
Two functions in expand_calls.go that performed overlapping
things were combined into a single function that is called
twice.
Fixes #42236.
For #40724.
Change-Id: Icbb78947eaa39f17f2c1210d5c2caef20abd6571
Reviewed-on: https://go-review.googlesource.com/c/go/+/262117
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Nigel Tao [Fri, 23 Oct 2020 01:11:35 +0000 (12:11 +1100)]
strconv: remove extfloat.go atof code path
Prior to this commit, strconv.ParseFloat (known in C as atof) takes the
first of four algorithms to succeed: atof64exact, eiselLemire64,
extFloat, fallback. The Eisel-Lemire implementation is a recent addition
but, now that it exists, the extFloat implementation (based on the
algorithm used by https://github.com/google/double-conversion) is
largely redundant. This Go program:
func parseOneMillionFloats(bitSize int, normallyDistributed bool) {
rng := rand.New(rand.NewSource(1))
for i := 0; i < 1_000_000; {
x := 0.0
if normallyDistributed {
x = rng.NormFloat64()
} else if bitSize == 32 {
x = float64(math.Float32frombits(rng.Uint32()))
} else {
x = math.Float64frombits(
uint64(rng.Uint32())<<32 | uint64(rng.Uint32()))
}
if math.IsInf(x, 0) {
continue
}
s := strconv.FormatFloat(x, 'g', -1, bitSize)
strconv.ParseFloat(s, bitSize)
i++
}
}
triggers the four algorithms by these percentages:
This commit removes the extfloat.go atof code (but keeps the extfloat.go
ftoa code for now), reducing the number of atof algorithms from 4 to 3.
The benchmarks (below) show some regressions but these are arguably
largely artificial situations.
Atof*RandomBits generates uniformly distributed uint32/uint64 values and
reinterprets the bits as float32/float64 values. The change in headline
numbers (arithmetic means) are primarily due to relatively large changes
for relatively rare cases.
fanzha02 [Fri, 16 Oct 2020 10:35:30 +0000 (18:35 +0800)]
cmd/asm: refactor some encoding functions for load/store with immediate offset on arm64
Some of the current functions for encoding load/store with
immediate offset instructions, like opstr12(), opstr9(),
opldr12(), opldr9() and opldrpp(), etc., they have the same
code, so this patch refactors them and merges them into two
functions opstr() and opldr().
Rémy Oudompheng [Sun, 25 Oct 2020 10:52:29 +0000 (11:52 +0100)]
cmd/compile: use magic multiply for unsigned values less than 1<<16 on 32-bit architectures
This is done by decomposing the number to be divided in 32-bit
components and using the 32-bit magic multiply. For the lowering to be
effective the constant must fit in 16 bits.
On ARM the expression n / 5 compiles to 25 instructions.
Bryan C. Mills [Wed, 28 Oct 2020 14:44:24 +0000 (10:44 -0400)]
cmd/go: use internal/testenv instead of computing canRun and skipExternal ad-hoc
Fixes #42223
Change-Id: Icf9bb61d48f0a6c7fd6f74e80e333a4837aa52ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/265781
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alberto Donizetti [Wed, 28 Oct 2020 09:10:55 +0000 (10:10 +0100)]
cmd/compile: rename mergeSymTyped to mergeSym
Also make canMergeSym take Syms instead of interface{}
Change-Id: I4926a1fc586aa90e198249d67e5b520404b40869
Reviewed-on: https://go-review.googlesource.com/c/go/+/265817
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Alberto Donizetti [Tue, 27 Oct 2020 10:30:14 +0000 (11:30 +0100)]
cmd/compile: delete log2, switch to log64
rewrite.go has two identical functions log2 and log64; the former has
been there for a while, while the latter was added together with
log{8,16,32} for use in typed rules.
This change deletes log2 and switches to using log64 everywhere.
Change-Id: I759b878814e4c115a5fa470274f22477738d69ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/265457
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
Joel Sing [Tue, 25 Aug 2020 07:53:08 +0000 (17:53 +1000)]
runtime: add defs for openbsd/mips64
Update #40995
Change-Id: I6963ead1a7c4520092361cce80edb17010e7f436
Reviewed-on: https://go-review.googlesource.com/c/go/+/250579
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Michael Pratt [Mon, 5 Oct 2020 22:12:35 +0000 (18:12 -0400)]
runtime: try to elide timer stealing if P has no timers
Following golang.org/cl/259578, findrunnable still must touch every
other P in checkTimers in order to look for timers to steal. This scales
poorly with GOMAXPROCS and potentially performs poorly by pulling remote
Ps into cache.
Add timerpMask, a bitmask that tracks whether each P may have any timers
on its timer heap.
Ideally we would update this field on any timer add / remove to always
keep it up to date. Unfortunately, updating a shared global structure is
antithetical to sharding timers by P, and doing so approximately doubles
the cost of addtimer / deltimer in microbenchmarks.
Instead we only (potentially) clear the mask when the P goes idle. This
covers the best case of avoiding looking at a P _at all_ when it is idle
and has no timers. See the comment on updateTimerPMask for more details
on the trade-off. Future CLs may be able to expand cases we can avoid
looking at the timers.
Note that the addition of idlepMask to p.init is a no-op. The zero value
of the mask is the correct init value so it is not necessary, but it is
included for clarity.
Benchmark results from WakeupParallel/syscall/pair/race/1ms (see
golang.org/cl/228577). Note that these are on top of golang.org/cl/259578:
Change-Id: I4239cd89f21ad16dfbbef58d81981da48acd0605
Reviewed-on: https://go-review.googlesource.com/c/go/+/264477
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Trust: Michael Pratt <mpratt@google.com>
Antonio Huete Jimenez [Wed, 28 Oct 2020 11:44:26 +0000 (11:44 +0000)]
os: do not use procfs for os.Executable in dragonfly
procfs(5) is not always mounted in DragonFly BSD, for example during
the binary package build with synth. os.Executable() consumers
will then fail, we've spotted this when trying to build tinygo:
[...]
copying source files
./build/tinygo build-builtins -target=armv6m-none-eabi [...]
panic: could not get executable path: readlink /proc/curproc/file:
no such file or directory
Tzu-Chiao Yeh [Mon, 24 Aug 2020 14:04:17 +0000 (22:04 +0800)]
database/sql: fix tx stmt deadlock when rollback
Tx acquires tx.closemu W-lock and then acquires stmt.closemu.W-lock
to fully close the transaction and associated prepared statement.
Stmt query and execution run in reverse ways - acquires
stmt.closemu.R-lock and then acquires tx.closemu.R-lock to grab tx
connection, which may cause deadlock.
Prevent the lock is held around tx.closePrepared to ensure no
deadlock happens.
Fixes #40985
Change-Id: If53909822b87bce11861a6e3035ecb9476d2cd17
Reviewed-on: https://go-review.googlesource.com/c/go/+/250178
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Tobias Klauser [Wed, 28 Oct 2020 12:44:53 +0000 (13:44 +0100)]
cmd/go/internal/modfetch: drop gopkg.in/russross/blackfriday.v2 from TestCodeRepoVersions
Follow-up for CL 265819.
Given the -pre tag added recently, a new stable version is likely
tagged soon. This would break TestCodeRepoVersions on the longtest
builders again. Since the other test cases in codeRepoVersionsTests
already provide enough coverage, drop gopkg.in/russross/blackfriday.v2
to avoid breaking TestCodeRepoVersions once the release happens.
Updates #28856
Change-Id: If86a637b5e47f59faf9048fc1cbbae6e8f1dcc53
Reviewed-on: https://go-review.googlesource.com/c/go/+/265917
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Sysmon can actually get the RW lock execLock while holding the sysmon
lock (if no M is available), so there is an edge from lockRankSysmon to
lockRankRwmutexR. The stack trace is sysmon() [gets sched.sysmonlock] ->
startm() -> newm() -> newm1() -> execLock.runlock() [gets
execLock.rLock]
Change-Id: I9658659ba3899afb5219114d66b989abd50540db
Reviewed-on: https://go-review.googlesource.com/c/go/+/265721
Trust: Dan Scales <danscales@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Cherry Zhang [Sat, 24 Oct 2020 17:14:36 +0000 (13:14 -0400)]
runtime: set up TLS without cgo on darwin/arm64
Currently, on darwin/arm64 we set up TLS using cgo. TLS is not
set for pure Go programs. As we use libc for syscalls on darwin,
we need to save the G register before the libc call. Otherwise it
is not signal-safe, as a signal may land during the execution of
a libc function, where the G register may be clobbered.
This CL initializes TLS in Go, by calling the pthread functions
directly without cgo. This makes it possible to save the G
register to TLS in pure Go programs (done in a later CL).
Inspired by Elias's CL 209197. Write the logic in Go instead of
assembly.
Updates #38485, #35853.
Change-Id: I257ba2a411ad387b2f4d50d10129d37fec7a226e
Reviewed-on: https://go-review.googlesource.com/c/go/+/265118
Trust: Cherry Zhang <cherryyz@google.com>
Trust: Elias Naur <mail@eliasnaur.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Use a standard "not-equal" label that we can jump to when we
detect that the arguments are not equal. This prevents the
recombination that was noticed in #39428.
Fixes #39428
Change-Id: Ib7c6b3539f4f6046043fd7148f940fcdcab70427
Reviewed-on: https://go-review.googlesource.com/c/go/+/255317
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Ian Lance Taylor [Wed, 28 Oct 2020 04:05:13 +0000 (21:05 -0700)]
runtime: move TestNeedmDeadlock to crash_cgo_test.go
It requires cgo. Also, skip the test on windows and plan9.
For #42207
Change-Id: I8522773f93bc3f9826506a41a08b86a083262e31
Reviewed-on: https://go-review.googlesource.com/c/go/+/265778
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Meng Zhuo [Tue, 27 Oct 2020 06:18:13 +0000 (14:18 +0800)]
cmd/link: remove all constants of elf
Use debug/elf instead.
Related:
CL 252478
CL 265317
Change-Id: If63b0458d9a6e825b40616bfb7a5a2c2e32402b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/265318
Trust: Meng Zhuo <mzh@golangcn.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Joel Sing <joel@sing.id.au> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 27 Oct 2020 23:09:40 +0000 (16:09 -0700)]
runtime: block signals in needm before allocating M
Otherwise, if a signal occurs just after we allocated the M,
we can deadlock if the signal handler needs to allocate an M
itself.
Fixes #42207
Change-Id: I76f44547f419e8b1c14cbf49bf602c6e645d8c14
Reviewed-on: https://go-review.googlesource.com/c/go/+/265759
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
George Tsilias [Thu, 4 Jun 2020 20:11:56 +0000 (23:11 +0300)]
runtime: handle signal 34 for musl setgid
It has been observed that setgid hangs when using cgo with musl.
This fix ensures that signal 34 gets handled in an appropriate way,
like signal 33 when using glibc.
Fixes #39343
Change-Id: I89565663e2c361f62cbccfe80aaedf290bd58d57
Reviewed-on: https://go-review.googlesource.com/c/go/+/236518
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Christopher Hlubek [Mon, 26 Oct 2020 12:44:44 +0000 (13:44 +0100)]
time: fix LoadLocationFromTZData with slim tzdata
The extend information of a time zone file with last transition < now
could result in a wrong cached zone because it used the zone of the
last transition.
This could lead to wrong zones in systems with slim zoneinfo.
Fixes #42216
Change-Id: I7c57c35b5cfa58482ac7925b5d86618c52f5444d
Reviewed-on: https://go-review.googlesource.com/c/go/+/264939
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Wed, 30 Sep 2020 00:01:33 +0000 (17:01 -0700)]
runtime: don't always adjust timers
Some programs have a lot of timers that they adjust both forward and
backward in time. This can cause a large number of timerModifiedEarlier
timers. In practice these timers are used for I/O deadlines and are
rarely reached. The effect is that the runtime spends a lot of time
in adjusttimers making sure that there are no timerModifiedEarlier
timers, but the effort is wasted because none of the adjusted timers
are near the top of the timer heap anyhow.
Avoid much of this extra work by keeping track of the earliest known
timerModifiedEarlier timer. This lets us skip adjusttimers if we know
that none of the timers will be ready to run anyhow. We will still
eventually run it, when we reach the deadline of the earliest known
timerModifiedEarlier, although in practice that timer has likely
been removed. When we do run adjusttimers, we will reset all of the
timerModifiedEarlier timers, and clear our notion of when we need
to run adjusttimers again.
This effect should be to significantly reduce the number of times we
walk through the timer list in adjusttimers.
Fixes #41699
Change-Id: I38eb2be611fb34e3017bb33d0a9ed40d75fb414f
Reviewed-on: https://go-review.googlesource.com/c/go/+/258303
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Keith Randall [Tue, 27 Oct 2020 21:15:00 +0000 (14:15 -0700)]
cmd/compile: print pointers to go:notinheap types without converting to unsafe.Pointer
Pretty minor concern, but after auditing the compiler/runtime for
conversions from pointers to go:notinheap types to unsafe.Pointer,
this is the only remaining one I found.
Update #42076
Change-Id: I81d5b893c9ada2fc19a51c2559262f2e9ff71c35
Reviewed-on: https://go-review.googlesource.com/c/go/+/265757
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Keith Randall [Thu, 22 Oct 2020 23:37:19 +0000 (16:37 -0700)]
cmd/compile, runtime: store pointers to go:notinheap types indirectly
pointers to go:notinheap types should be treated as scalars. That
means they shouldn't be stored directly in interfaces, or directly
in reflect.Value.ptr.
Also be sure to use uintpr to compare such pointers in reflect.DeepEqual.
Fixes #42076
Change-Id: I53735f6d434e9c3108d4940bd1bae14c61ef2a74
Reviewed-on: https://go-review.googlesource.com/c/go/+/264480
Trust: Keith Randall <khr@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Thu, 22 Oct 2020 20:11:16 +0000 (13:11 -0700)]
cmd/compile: fix storeType to handle pointers to go:notinheap types
storeType splits compound stores up into a scalar parts and a pointer parts.
The scalar part happens unconditionally, and the pointer part happens
under the guard of a write barrier check.
Types which are declared as pointers, but are represented as scalars because
they might have "bad" values, were not handled correctly here. They ended
up not getting stored in either set.
Fixes #42032
Change-Id: I46f6600075c0c370e640b807066247237f93c7ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/264300
Trust: Keith Randall <khr@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 27 Oct 2020 19:59:54 +0000 (12:59 -0700)]
go/internal/gccgoimporter: support notinheap annotation
The gofrontend has started emitting a notinheap annotation for types
marked go:notinheap.
For #41761
Change-Id: Ic8f7ffc32dbfe98ec09b3d835957f1be8e6c1208
Reviewed-on: https://go-review.googlesource.com/c/go/+/265702
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Alberto Donizetti [Tue, 27 Oct 2020 10:03:21 +0000 (11:03 +0100)]
cmd/compile: delete isPowerOfTwo, switch to isPowerOfTwo64
rewrite.go has two identical functions isPowerOfTwo and
isPowerOfTwo64; the former has been there for a while, while the
latter was added together with isPowerOfTwo{8,16,32} for use in typed
rules.
This change deletes isPowerOfTwo and switch to using isPowerOfTwo64
everywhere.
Change-Id: If26c94565d2393fac6f0ba117ee7ee2fc915f7cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/265417
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Alberto Donizetti [Tue, 27 Oct 2020 08:38:52 +0000 (09:38 +0100)]
cmd/compile: clean up ValAndOff funcs after untyped aux removal
Changes:
- makeValAndOff is deleted in favour of MakeValAndOff{32,64}
- canAdd is renamed to canAdd64 to uniform with existing canAdd32
- addOffset{32,64} is simplified by directly using MakeValAndOff{32,64}
- ValAndOff.Int64 is removed
Change-Id: Ic01db7fa31ddfe0aaaf1d1d77af823d48a7bee84
Reviewed-on: https://go-review.googlesource.com/c/go/+/265357
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Alberto Donizetti [Sat, 24 Oct 2020 15:27:52 +0000 (17:27 +0200)]
cmd/compile: remove support for untyped ssa rules
This change removes support in rulegen for untyped -> ssa rules.
Change-Id: I202018e191fc74f027243351bc8cf96145f2482c
Reviewed-on: https://go-review.googlesource.com/c/go/+/264679
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Dan Peterson [Thu, 22 Oct 2020 22:25:56 +0000 (19:25 -0300)]
net/http: use exponential backoff for polling in Server.Shutdown
Instead of always polling 500ms, start with an interval of 1ms and
exponentially back off to at most 500ms. 10% jitter is added to each
interval.
This makes Shutdown more responsive when connections and listeners
close quickly.
Also removes the need for the polling interval to be changed in tests
since if tests' connections and listeners close quickly Shutdown will
also return quickly.
Paul E. Murphy [Fri, 23 Oct 2020 17:12:34 +0000 (12:12 -0500)]
cmd/compile: combine more 32 bit shift and mask operations on ppc64
Combine (AND m (SRWconst x)) or (SRWconst (AND m x)) when mask m is
and the shift value produce constant which can be encoded into an
RLWINM instruction.
Combine (CLRLSLDI (SRWconst x)) if the combining of the underling rotate
masks produces a constant which can be encoded into RLWINM.
Likewise for (SLDconst (SRWconst x)) and (CLRLSDI (RLWINM x)).
Combine rotate word + and operations which can be encoded as a single
RLWINM/RLWNM instruction.
The most notable performance improvements arise from the crypto
benchmarks below (GOARCH=power8 on a ppc64le/linux):
Chris Hines [Fri, 1 May 2020 21:04:36 +0000 (17:04 -0400)]
runtime: reduce timer latency
Change the scheduler to treat expired timers with the same approach it
uses to steal runnable G's.
Previously the scheduler ignored timers on P's not marked for
preemption. That had the downside that any G's waiting on those expired
timers starved until the G running on their P completed or was
preempted. That could take as long as 20ms if sysmon was in a 10ms
wake up cycle.
In addition, a spinning P that ignored an expired timer and found no
other work would stop despite there being available work, missing the
opportunity for greater parallelism.
With this change the scheduler no longer ignores timers on
non-preemptable P's or relies on sysmon as a backstop to start threads
when timers expire. Instead it wakes an idle P, if needed, when
creating a new timer because it cannot predict if the current P will
have a scheduling opportunity before the new timer expires. The P it
wakes will determine how long to sleep and block on the netpoller for
the required time, potentially stealing the new timer when it wakes.
This change also eliminates a race between a spinning P transitioning
to idle concurrently with timer creation using the same pattern used
for submission of new goroutines in the same window.
Benchmark analysis:
CL 232199, which was included in Go 1.15 improved timer latency over Go
1.14 by allowing P's to steal timers from P's not marked for preemption.
The benchmarks added in this CL measure that improvement in the
ParallelTimerLatency benchmark seen below. However, Go 1.15 still relies
on sysmon to notice expired timers in some situations and sysmon can
sleep for up to 10ms before waking to check timers. This CL fixes that
shortcoming with modest regression on other benchmarks.
xd [Wed, 14 Oct 2020 18:02:49 +0000 (11:02 -0700)]
cmd/dist: fix build failure of misc/cgo/test on arm64
misc/cgo/test fails in 'dist test' on arm64 if the C compiler is of GCC-9.4 or
above and its 'outline atomics' feature is enabled, since the internal linking
hasn't yet supported "__attribute__((constructor))" and also mis-handles hidden
visibility.
This change addresses the problem by skipping the internal linking cases of
misc/cgo/test on linux/arm64. It fixes 'dist test' failure only, user is expected to
pass a GCC option '-mno-outline-atomics' via CGO_CFLAGS if running into the same
problem when building cgo programs using internal linking.
Ayzat Sadykov [Sat, 24 Oct 2020 22:11:51 +0000 (22:11 +0000)]
database/sql: fix comment on DB.stop()
Previously, 2 goroutines were created in OpenDB and a comment in the DB.close() field indicated that they were canceled. Later, session Resetter () was removed, but the comment remained the same. This commit just fixes this message
Joel Sing [Sat, 24 Oct 2020 14:34:17 +0000 (01:34 +1100)]
cmd/compile: eliminate unnecessary sign/zero extension for riscv64
Add additional rules to eliminate unnecessary sign/zero extension for riscv64.
Also where possible, replace an extension following a load with a different typed
load. This removes almost another 8,000 instructions from the go binary.
Of particular note, change Eq16/Eq8/Neq16/Neq8 to zero extend each value before
subtraction, rather than zero extending after subtraction. While this appears to
double the number of zero extensions, it often lets us completely eliminate them
as the load can already be performed in a properly typed manner.
As an example, prior to this change runtime.memequal16 was:
Change-Id: I16321feb18381241cab121c0097a126104c56c2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/264659
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Jason A. Donenfeld [Thu, 5 Dec 2019 17:48:21 +0000 (18:48 +0100)]
crypto/rand: generate random numbers using RtlGenRandom on Windows
CryptGenRandom appears to be unfavorable these days, whereas the classic
RtlGenRandom is still going strong.
This commit also moves the warnBlocked function into rand_unix, rather
than rand, because it's now only used on unix.
Fixes #33542
Change-Id: I5c02a5917572f54079d627972401efb6e1ce4057
Reviewed-on: https://go-review.googlesource.com/c/go/+/210057
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org>
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Joel Sing [Sat, 24 Oct 2020 13:32:23 +0000 (00:32 +1100)]
cmd/compile: use MOV pseudo-instructions for sign/zero extension
Rather than handling sign and zero extension via rules, defer to the assembler
and use MOV pseudo-instructions. The instruction can also be omitted where the
type and size is already correct. This change results in more than 6,000
instructions being removed from the go binary (in part due to omitted
instructions, in part due to MOVBU having a more efficient implementation in
the assembler than what is used in the current ZeroExt8to{16,32,64} rules).
This will also allow for further rewriting to remove redundant sign/zero
extension.
Change-Id: I05e42fd9f09f40a69948be7de772cce8946c8744
Reviewed-on: https://go-review.googlesource.com/c/go/+/264658
Trust: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Bryan C. Mills [Tue, 13 Oct 2020 14:58:13 +0000 (10:58 -0400)]
cmd/go/internal/modload: embed PackageOpts in loaderParams
Instead of duplicating PackageOpts fields in the loaderParams struct,
embed the PackageOpts directly. Many of the fields are duplicated, and
further fields that would also be duplicated will be added in
subsequent changes.
For #36460
Change-Id: I3b0770d162e901d23ec1643183eb07c413d51e0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/263138
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Joel Sing [Tue, 19 May 2020 08:55:10 +0000 (18:55 +1000)]
cmd/link,cmd/internal/obj/riscv: add TLS support for linux/riscv64
Add support for Thread Local Storage (TLS) for linux/riscv64 with external
linking, using the initial-exec model.
Update #36641
Change-Id: I3106ef9a29cde73215830b00deff43dbec1c76e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/263478
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Michael Anthony Knyszek [Tue, 14 Jul 2020 21:45:16 +0000 (21:45 +0000)]
runtime: implement addrRanges.findSucc with a binary search
This change modifies addrRanges.findSucc to more efficiently find the
successor range in an addrRanges by using a binary search to narrow down
large addrRanges and iterate over no more than 8 addrRanges.
This change makes the runtime more robust against systems that may
aggressively randomize the address space mappings it gives the runtime
(e.g. Fuchsia).
For #40191.
Change-Id: If529df2abd2edb1b1496d8690ddd284ecd7138c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/242679
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Thu, 17 Sep 2020 21:19:28 +0000 (21:19 +0000)]
runtime: implement dumpmemstats in terms of readmemstats_m
Since MemStats is now populated directly and some values are derived,
avoid duplicating the logic by instead populating the heap dump directly
from MemStats (external version) instead of memstats (runtime internal
version).
Change-Id: I0bec96bfa02d2ffd1b56475779c124a760e64238
Reviewed-on: https://go-review.googlesource.com/c/go/+/255817
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Fri, 7 Aug 2020 16:37:29 +0000 (16:37 +0000)]
runtime,runtime/metrics: export goroutine count as a metric
For #37112.
Change-Id: I994dfe848605b95ef6aec24f53869e929247e987
Reviewed-on: https://go-review.googlesource.com/c/go/+/247049
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Thu, 6 Aug 2020 21:59:13 +0000 (21:59 +0000)]
runtime,runtime/metrics: add metric for distribution of GC pauses
For #37112.
Change-Id: Ibb0425c9c582ae3da3b2662d5bbe830d7df9079c
Reviewed-on: https://go-review.googlesource.com/c/go/+/247047
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Thu, 6 Aug 2020 20:36:49 +0000 (20:36 +0000)]
runtime: add timeHistogram type
This change adds a concurrent HDR time histogram to the runtime with
tests. It also adds a function to generate boundaries for use by the
metrics package.
For #37112.
Change-Id: Ifbef8ddce8e3a965a0dcd58ccd4915c282ae2098
Reviewed-on: https://go-review.googlesource.com/c/go/+/247046
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Thu, 6 Aug 2020 19:04:46 +0000 (19:04 +0000)]
runtime,runtime/metrics: add object size distribution metrics
This change adds metrics for the distribution of objects allocated and
freed by size, mirroring MemStats' BySize field.
For #37112.
Change-Id: Ibaf1812da93598b37265ec97abc6669c1a5efcbf
Reviewed-on: https://go-review.googlesource.com/c/go/+/247045
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Austin Clements [Mon, 26 Oct 2020 18:32:13 +0000 (14:32 -0400)]
cmd/go,cmd/compile,sync: remove special import case in cmd/go
CL 253748 introduced a special case in cmd/go to allow sync to import
runtime/internal/atomic. Besides introducing unnecessary complexity
into cmd/go, this breaks other packages (like gopls) that understand
how imports work, but don't understand this special case.
Fix this by using the more standard linkname-based approach to pull
the necessary functions from runtime/internal/atomic into sync. Since
these are compiler intrinsics, we also have to tell the compiler that
the linknamed symbols are intrinsics to get this optimization in sync.
Fixes #42196.
Change-Id: I1f91498c255c91583950886a89c3c9adc39a32f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/265124
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Paul Murphy <murp@ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Michael Anthony Knyszek [Thu, 6 Aug 2020 16:47:58 +0000 (16:47 +0000)]
runtime,runtime/metrics: add heap goal and GC cycle metrics
This change adds three new metrics: the heap goal, GC cycle count, and
forced GC count. These metrics are identical to their MemStats
counterparts.
For #37112.
Change-Id: I5a5e8dd550c0d646e5dcdbdf38274895e27cdd88
Reviewed-on: https://go-review.googlesource.com/c/go/+/247044
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Change-Id: Idd3dd5c84215ddd1ab05c2e76e848aa0a4d40fb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/247043
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Wed, 5 Aug 2020 23:10:46 +0000 (23:10 +0000)]
runtime: add readMetrics latency benchmark
This change adds a new benchmark to the runtime tests for measuring the
latency of the new metrics implementation, based on the
ReadMemStats latency benchmark. readMetrics will have more metrics added
to it in the future, and this benchmark will serve as a way to measure
the cost of adding additional metrics.
Change-Id: Ib05e3ed4afa49a70863fc0c418eab35b72263e24
Reviewed-on: https://go-review.googlesource.com/c/go/+/247042
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Wed, 1 Jul 2020 16:02:42 +0000 (16:02 +0000)]
runtime,runtime/metrics: add memory metrics
This change adds support for a variety of runtime memory metrics and
contains the base implementation of Read for the runtime/metrics
package, which lives in the runtime.
It also adds testing infrastructure for the metrics package, and a bunch
of format and documentation tests.
For #37112.
Change-Id: I16a2c4781eeeb2de0abcb045c15105f1210e2d8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/247041
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Tue, 4 Aug 2020 17:29:03 +0000 (17:29 +0000)]
runtime: move malloc stats into consistentHeapStats
This change moves the mcache-local malloc stats into the
consistentHeapStats structure so the malloc stats can be managed
consistently with the memory stats. The one exception here is
tinyAllocs for which moving that into the global stats would incur
several atomic writes on the fast path. Microbenchmarks for just one CPU
core have shown a 50% loss in throughput. Since tiny allocation counnt
isn't exposed anyway and is always blindly added to both allocs and
frees, let that stay inconsistent and flush the tiny allocation count
every so often.
Change-Id: I2a4b75f209c0e659b9c0db081a3287bf227c10ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/247039
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Mon, 3 Aug 2020 20:35:40 +0000 (20:35 +0000)]
runtime: replace some memstats with consistent stats
This change replaces stacks_inuse, gcWorkBufInUse and
gcProgPtrScalarBitsInUse with their corresponding consistent stats. It
also adds checks to make sure the rest of the sharded stats line up with
existing stats in updatememstats.
Change-Id: I17d0bd181aedb5c55e09c8dff18cef5b2a3a14e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/247038
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Mon, 3 Aug 2020 20:11:04 +0000 (20:11 +0000)]
runtime: add consistent heap statistics
This change adds a global set of heap statistics which are similar
to existing memory statistics. The purpose of these new statistics
is to be able to read them and get a consistent result without stopping
the world. The goal is to eventually replace as many of the existing
memstats statistics with the sharded ones as possible.
The consistent memory statistics use a tailor-made synchronization
mechanism to allow writers (allocators) to proceed with minimal
synchronization by using a sequence counter and a global generation
counter to determine which set of statistics to update. Readers
increment the global generation counter to effectively grab a snapshot
of the statistics, and then iterate over all Ps using the sequence
counter to ensure that they may safely read the snapshotted statistics.
To keep statistics fresh, the reader also has a responsibility to merge
sets of statistics.
These consistent statistics are computed, but otherwise unused for now.
Upcoming changes will integrate them with the rest of the codebase and
will begin to phase out existing statistics.
Change-Id: I637a11f2439e2049d7dccb8650c5d82500733ca5
Reviewed-on: https://go-review.googlesource.com/c/go/+/247037
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Tue, 14 Apr 2020 21:06:26 +0000 (21:06 +0000)]
runtime/metrics: add package interface
This change creates the runtime/metrics package and adds the initial
interface as laid out in the design document.
For #37112.
Change-Id: I202dcee08ab008dd63bf96f7a4162f5b5f813637
Reviewed-on: https://go-review.googlesource.com/c/go/+/247040
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Mon, 3 Aug 2020 20:08:25 +0000 (20:08 +0000)]
runtime: add helper for getting an mcache in allocation contexts
This change adds a function getMCache which returns the current P's
mcache if it's available, and otherwise tries to get mcache0 if we're
bootstrapping. This function will come in handy as we need to replicate
this behavior in multiple places in future changes.
Change-Id: I536073d6f6dc6c6390269e613ead9f8bcb6e7f98
Reviewed-on: https://go-review.googlesource.com/c/go/+/246976
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Mon, 3 Aug 2020 19:31:23 +0000 (19:31 +0000)]
runtime: remove memstats.heap_alloc
memstats.heap_alloc is 100% a duplicate and unnecessary copy of
memstats.alloc which exists because MemStats used to be populated from
memstats via a memmove.
Change-Id: I995489f61be39786e573b8494a8ab6d4ea8bed9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/246975
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Mon, 3 Aug 2020 19:27:59 +0000 (19:27 +0000)]
runtime: remove memstats.heap_idle
This statistic is updated in many places but for MemStats may be
computed from existing statistics. Specifically by definition
heap_idle = heap_sys - heap_inuse since heap_sys is all memory allocated
from the OS for use in the heap minus memory used for non-heap purposes.
heap_idle is almost the same (since it explicitly includes memory that
*could* be used for non-heap purposes) but also doesn't include memory
that's actually used to hold heap objects.
Although it has some utility as a sanity check, it complicates
accounting and we want fewer, orthogonal statistics for upcoming metrics
changes, so just drop it.
Change-Id: I40af54a38e335f43249f6e218f35088bfd4380d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/246974
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Mon, 3 Aug 2020 19:23:30 +0000 (19:23 +0000)]
runtime: break down memstats.gc_sys
This change breaks apart gc_sys into three distinct pieces. Two of those
pieces are pieces which come from heap_sys since they're allocated from
the page heap. The rest comes from memory mapped from e.g.
persistentalloc which better fits the purpose of a sysMemStat. Also,
rename gc_sys to gcMiscSys.
Change-Id: I098789170052511e7b31edbcdc9a53e5c24573f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/246973
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Fri, 31 Jul 2020 21:32:26 +0000 (21:32 +0000)]
runtime: copy in MemStats fields explicitly
Currently MemStats is populated via an unsafe memmove from memstats, but
this places unnecessary structural restrictions on memstats, is annoying
to reason about, and tightly couples the two. Instead, just populate the
fields of MemStats explicitly.
Change-Id: I96f6a64326b1a91d4084e7b30169a4bbe6a331f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/246972
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Wed, 29 Jul 2020 20:25:05 +0000 (20:25 +0000)]
runtime: delineate which memstats are system stats with a type
This change modifies the type of several mstats fields to be a new type:
sysMemStat. This type has the same structure as the fields used to have.
The purpose of this change is to make it very clear which stats may be
used in various functions for accounting (usually the platform-specific
sys* functions, but there are others). Currently there's an implicit
understanding that the *uint64 value passed to these functions is some
kind of statistic whose value is atomically managed. This understanding
isn't inherently problematic, but we're about to change how some stats
(which currently use mSysStatInc and mSysStatDec) work, so we want to
make it very clear what the various requirements are around "sysStat".
This change also removes mSysStatInc and mSysStatDec in favor of a
method on sysMemStat. Note that those two functions were originally
written the way they were because atomic 64-bit adds required a valid G
on ARM, but this hasn't been the case for a very long time (since
golang.org/cl/14204, but even before then it wasn't clear if mutexes
required a valid G anymore). Today we implement 64-bit adds on ARM with
a spinlock table.
Change-Id: I4e9b37cf14afc2ae20cf736e874eb0064af086d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/246971
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Wed, 29 Jul 2020 19:00:37 +0000 (19:00 +0000)]
runtime: make the span allocation purpose more explicit
This change modifies mheap's span allocation API to have each caller
declare a purpose, defined as a new enum called spanAllocType.
The purpose behind this change is two-fold:
1. Tight control over who gets to allocate heap memory is, generally
speaking, a good thing. Every codepath that allocates heap memory
places additional implicit restrictions on the allocator. A notable
example of a restriction is work bufs coming from heap memory: write
barriers are not allowed in allocation paths because then we could
have a situation where the allocator calls into the allocator.
2. Memory statistic updating is explicit. Instead of passing an opaque
pointer for statistic updating, which places restrictions on how that
statistic may be updated, we use the spanAllocType to determine which
statistic to update and how.
We also take this opportunity to group all the statistic updating code
together, which should make the accounting code a little easier to
follow.
Change-Id: Ic0b0898959ba2a776f67122f0e36c9d7d60e3085
Reviewed-on: https://go-review.googlesource.com/c/go/+/246970
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Fri, 24 Jul 2020 19:58:31 +0000 (19:58 +0000)]
runtime: rename mcache fields to match Go style
This change renames a bunch of malloc statistics stored in the mcache
that are all named with the "local_" prefix. It also renames largeAlloc
to allocLarge to prevent a naming conflict, and next_sample because it
would be the last mcache field with the old C naming style.
Change-Id: I29695cb83b397a435ede7e9ad5c3c9be72767ea3
Reviewed-on: https://go-review.googlesource.com/c/go/+/246969
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Thu, 23 Jul 2020 22:36:58 +0000 (22:36 +0000)]
runtime: flush local_scan directly and more often
Now that local_scan is the last mcache-based statistic that is flushed
by purgecachedstats, and heap_scan and gcController.revise may be
interacted with concurrently, we don't need to flush heap_scan at
arbitrary locations where the heap is locked, and we don't need
purgecachedstats and cachestats anymore. Instead, we can flush
local_scan at the same time we update heap_live in refill, so the two
updates may share the same revise call.
Clean up unused functions, remove code that would cause the heap to get
locked in the allocSpan when it didn't need to (other than to flush
local_scan), and flush local_scan explicitly in a few important places.
Notably we need to flush local_scan whenever we flush the other stats,
but it doesn't need to be donated anywhere, so have releaseAll do the
flushing. Also, we need to flush local_scan before we set heap_scan at
the end of a GC, which was previously handled by cachestats. Just do so
explicitly -- it's not much code and it becomes a lot more clear why we
need to do so.
Change-Id: I35ac081784df7744d515479896a41d530653692d
Reviewed-on: https://go-review.googlesource.com/c/go/+/246968
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Thu, 23 Jul 2020 22:16:46 +0000 (22:16 +0000)]
runtime: don't flush local_tinyallocs
This change makes local_tinyallocs work like the rest of the malloc
stats and doesn't flush local_tinyallocs, instead making that the
source-of-truth.
Change-Id: I3e6cb5f1b3d086e432ce7d456895511a48e3617a
Reviewed-on: https://go-review.googlesource.com/c/go/+/246967
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Thu, 23 Jul 2020 22:07:44 +0000 (22:07 +0000)]
runtime: remove mcentral.nmalloc and add mcache.local_nsmallalloc
This change removes mcentral.nmalloc and adds mcache.local_nsmallalloc
which fulfills the same role but may be accessed non-atomically. It also
moves responsibility for updating heap_live and local_nsmallalloc into
mcache functions.
As a result of this change, mcache is now the sole source-of-truth for
malloc stats. It is also solely responsible for updating heap_live and
performing the various operations required as a result of updating
heap_live. The overall improvement here is in code organization:
previously malloc stats were fairly scattered, and now they have one
single home, and nearly all the required manipulations exist in a single
file.
Change-Id: I7e93fa297c1debf17e3f2a0d68aeed28a9c6af00
Reviewed-on: https://go-review.googlesource.com/c/go/+/246966
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Thu, 23 Jul 2020 21:10:29 +0000 (21:10 +0000)]
runtime: make nlargealloc and largealloc mcache fields
This change makes nlargealloc and largealloc into mcache fields just
like nlargefree and largefree. These local fields become the new
source-of-truth. This change also moves the accounting for these fields
out of allocSpan (which is an inappropriate place for it -- this
accounting generally happens much closer to the point of allocation) and
into largeAlloc. This move is partially possible now that we can call
gcController.revise at that point.
Furthermore, this change moves largeAlloc into mcache.go and makes it a
method of mcache. While there's a little bit of a mismatch here because
largeAlloc barely interacts with the mcache, it helps solidify the
mcache as the first allocation layer and provides a clear place to
aggregate and manage statistics.
Change-Id: I37b5e648710733bb4c04430b71e96700e438587a
Reviewed-on: https://go-review.googlesource.com/c/go/+/246965
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>