Michael Anthony Knyszek [Mon, 13 Nov 2023 21:06:40 +0000 (21:06 +0000)]
internal/trace: add task analysis for v2 traces
For v1 traces, cmd/trace contains code for analyzing tasks separately
from the goroutine analysis code present in internal/trace. As I started
to look into porting that code to v2 traces, I noticed that it wouldn't
be too hard to just generalize the existing v2 goroutine summary code to
generate exactly the same information.
This change does exactly that.
For #60773.
For #63960.
Change-Id: I0cdd9bf9ba11fb292a9ffc37dbf18c2a6a2483b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/542076
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Fri, 17 Nov 2023 17:56:27 +0000 (17:56 +0000)]
internal/trace/v2: redefine NoTask and add BackgroundTask
The v2 trace parser currently handles task inheritance and region task
association incorrectly. It assumes that a TaskID of 0 means that there
is no task. However, this is only true for task events. A TaskID of 0
means that a region gets assigned to the "background task." The parser
currently has no concept of a "background task."
Fix this by defining the background task as task ID 0 and redefining
NoTask to ^uint64(0). This aligns the TaskID values more closely with
other IDs in the parser and also enables disambiguating these two cases.
For #60773.
For #63960.
Change-Id: I09c8217b33b87c8f8f8ea3b0203ed83fd3b61e11
Reviewed-on: https://go-review.googlesource.com/c/go/+/543019 Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Mon, 13 Nov 2023 18:57:06 +0000 (18:57 +0000)]
cmd/trace/v2: add support for pprof endpoints
This change adds support for the pprof endpoints to cmd/trace/v2.
In the process, I realized we need to pass the goroutine summaries to
more places, and previous CLs had already done the goroutine analysis
during cmd/trace startup. This change thus refactors the goroutine
analysis API once again to operate in a streaming manner, and to run
at the same time as the initial trace parsing. Now we can include it in
the parsedTrace type and pass that around as the de-facto global trace
context.
Note: for simplicity, this change redefines "syscall" profiles to
capture *all* syscalls, not just syscalls that block. IIUC, this choice
was partly the result of a limitation in the previous trace format that
syscalls don't all have complete durations and many short syscalls are
treated as instant. To this end, this change modifies the text on the
main trace webpage to reflect this change.
For #60773.
For #63960.
Change-Id: I601d9250ab0849a0bfaef233fd9b1e81aca9a22a
Reviewed-on: https://go-review.googlesource.com/c/go/+/541999
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Sun, 12 Nov 2023 23:27:31 +0000 (23:27 +0000)]
internal/trace/traceviewer: make the mmu handler more self-contained
The last change made the MMU rendering code common and introduced a new
API, but it was kind of messy. Part of the problem was that some of the
Javascript in the template for the main page referred to specific
endpoints on the server.
Fix this by having the Javascript access the same endpoint but with a
different query variable. Now the Javascript code doesn't depend on
specific endpoints, just on query variables for the current endpoint.
For #60773.
For #63960.
Change-Id: I1c559d9859c3a0d62e2094c9d4ab117890b63b31
Reviewed-on: https://go-review.googlesource.com/c/go/+/541259
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Sun, 12 Nov 2023 23:10:04 +0000 (23:10 +0000)]
cmd/trace: common up the mmu page and add it to cmd/trace/v2
This change moves the MMU HTTP handlers and functionality into the
traceviewer package, since unlike the goroutine pages the vast majority
of that functionality is identical between v1 and v2. This change
involves some refactoring so that callers can plug in their own mutator
utilization computation functions (which is the only point of difference
between v1 and v2). The new interface isn't especially nice, but part of
the problem is the MMU handlers depend on specific endpoints to exist. A
follow-up CL will clean this up a bit.
Like the previous CL did for goroutine analysis, modify the v2 mutator
utilization API to accept a slice of trace events. Again, we might as
well reuse what was already parsed and will be needed for other
purposes. It also simplifies the API slightly.
For #60773.
For #63960.
Change-Id: I6c21ec8d1bf7e95eff5363d0e0005c9217fa00e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/541258
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Sun, 12 Nov 2023 21:48:31 +0000 (21:48 +0000)]
cmd/trace/v2: add goroutine analysis pages
This is a complete fork and most of a rewrite of the goroutine analysis
pages for v2 traces. It fixes an issue with the old page where GC time
didn't really make any sense, generalizes the page and breaks things
down further, and adds clarifying text.
This change also modifies the SummarizeGoroutines API to not stream the
trace. This is unfortunate, but we're already reading and holding the
entire trace in memory for the trace viewer. We can revisit this
decision in the future. Also, we want to do this now because the
GoroutineSummary holds on to pointers to events, and these events will
be used by the user region and user task analyses. While tracev2 events
are values and they should be equivalent no matter how many times we
parse a trace, this lets us reference the event in the slice directly.
For #60773.
For #63960.
Fixes #62443.
Change-Id: I1c5ab68141869378843f4f2826686038e4533090
Reviewed-on: https://go-review.googlesource.com/c/go/+/541257 Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Tue, 21 Nov 2023 17:33:20 +0000 (17:33 +0000)]
internal/trace: use the correct stack for goroutine naming in v2 traces
Currently goroutine names are determined (for v2 traces) by
internal/tracev/2.Event.Stack, but this is wrong in general. For
example, if we end up seeing a transition from GoNotExist->GoRunnable
(goroutine creation) then we're taking the stack from the creator, not
the created goroutine (which is what we're naming at that point).
Use the StateTransition.Stack instead. This is always the correct one to
use because we're always naming the goroutine that the state transition
is for.
Change-Id: I3fc7c8e4f85dfee3802d666c0c091b6953c7d6cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/544317
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Rhys Hiltner [Tue, 21 Nov 2023 16:03:54 +0000 (16:03 +0000)]
runtime: profile contended lock calls
Add runtime-internal locks to the mutex contention profile.
Store up to one call stack responsible for lock contention on the M,
until it's safe to contribute its value to the mprof table. Try to use
that limited local storage space for a relatively large source of
contention, and attribute any contention in stacks we're not able to
store to a sentinel _LostContendedLock function.
Avoid ballooning lock contention while manipulating the mprof table by
attributing to that sentinel function any lock contention experienced
while reporting lock contention.
Guard collecting real call stacks with GODEBUG=profileruntimelocks=1,
since the available data has mixed semantics; we can easily capture an
M's own wait time, but we'd prefer for the profile entry of each
critical section to describe how long it made the other Ms wait. It's
too late in the Go 1.22 cycle to make the required changes to
futex-based locks. When not enabled, attribute the time to the sentinel
function instead.
Fixes #57071
This is a roll-forward of https://go.dev/cl/528657, which was reverted
in https://go.dev/cl/543660
Reason for revert: de-flakes tests (reduces dependence on fine-grained
timers, correctly identifies contention on big-endian futex locks,
attempts to measure contention in the semaphore implementation but only
uses that secondary measurement to finish the test early, skips tests on
single-processor systems)
Change-Id: I31389f24283d85e46ad9ba8d4f514cb9add8dfb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/544195 Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Rhys Hiltner <rhys@justin.tv>
Run-TryBot: Rhys Hiltner <rhys@justin.tv>
Michael Anthony Knyszek [Tue, 21 Nov 2023 17:23:43 +0000 (17:23 +0000)]
internal/trace/v2: forward Event.Stack to StateTransition.Stack
Currently StateTransition.Stack is only set for the GoCreate case,
because there are two stacks and we need to distinguish them. But the
docs for StateTransition.Stack say that that stack always references the
resource that is transitioning. There are quite a few cases where
Event.Stack is actually the appropriate stack to for
StateTransition.Stack, but in these cases it's left empty, and the
caller just needs to understand which one to look at. This isn't great.
Forward Event.Stack to StateTransition.Stack whenever Event.Stack also
refers to the resource experiencing the state transition.
Change-Id: Ie43fc6036f2712c7982174d5739d95765312dfcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/544316
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Tue, 14 Nov 2023 17:03:24 +0000 (17:03 +0000)]
cmd/trace: add almost full support for v2 traces in the trace viewer
This change refactors the cmd/trace package and adds most of the support
for v2 traces.
The following features of note are missing in this CL and will be
implemented in follow-up CLs:
- The focustask filter for the trace viewer
- The taskid filter for the trace viewer
- The goid filter for the trace viewer
- Pprof profiles
- The MMU graph
- The goroutine analysis pages
- The task analysis pages
- The region analysis pages
This CL makes one notable change to the trace CLI: it makes the -d flag
accept an integer to set the debug mode. For old traces -d != 0 works
just like -d. For new traces -d=1 means the high-level events and -d=2
means the low-level events.
Thanks to Felix Geisendörfer (felix.geisendoerfer@datadoghq.com) for
doing a lot of work on this CL; I picked this up from him and got a
massive headstart as a result.
For #60773.
For #63960.
Change-Id: I3626e22473227c5980134a85f1bb6a845f567b1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/542218 Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Kir Kolyshkin [Mon, 20 Nov 2023 21:16:13 +0000 (13:16 -0800)]
os: remove useless if from Wait on unix
Back in the day, Wait used to accept options argument.
CL 4962042 fixed the issue of setting process.done flag when WNOHANG
option was used.
Later, CL 5688046 removed options argument from Wait, but did not remove
pid1 != 0 check which was meant to be used with WNOHANG only.
Remove the check, which is useless and also confusing.
Change-Id: I73b9ef4a0dbe35466e659ca58b896d515ba86d02
Reviewed-on: https://go-review.googlesource.com/c/go/+/543736
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
order.go ensures expressions that are passed to the runtime by address
are in fact addressable. However, in the case of local variables, if the
variable hasn't already been marked as addrtaken, then taking its
address here will effectively prevent the variable from being converted
to SSA form.
Instead, it's better to just copy the variable into a new temporary,
which we can pass by address instead. This ensures the original variable
can still be converted to SSA form.
Fixes #63332.
Change-Id: I182376d98d419df8bf07c400d84c344c9b82c0fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/541715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Kir Kolyshkin [Thu, 16 Nov 2023 08:42:04 +0000 (00:42 -0800)]
syscall: fix getting pidfd when using CLONE_NEWUSER
While working on CL 528798, I found out that sys.PidFD field (added
in CL 520266) is not filled in when CLONE_NEWUSER is used.
This happens because the code assumed that the parent and the child
run in the same memory space. This assumption is right only when
CLONE_VM is used for clone syscall, and the code only sets CLONE_VM
when CLONE_NEWUSER is not used.
Fix this, and add a test case (which fails before the fix).
Updates #51246.
Change-Id: I805203c1369cadd63d769568b132a9ffd92cc184
Reviewed-on: https://go-review.googlesource.com/c/go/+/542698
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Than McIntosh [Tue, 7 Nov 2023 15:28:56 +0000 (10:28 -0500)]
cmd/compile/internal/inline: revise -m=2 status messages
This patch revises the compiler's "-m=2" status messages related to
inlining. The "can inline" remarks will continue to use the same
format, but the remarks when a specific call site is inlined will be
changed to refer to the score used; before we had
runtime/traceback.go:1131:28: inlining call to gotraceback
runtime/traceback.go:1183:25: inlining call to readgstatus
and with GOEXPERIMENT=newinliner the new messages will be:
runtime/traceback.go:1131:28: inlining call to gotraceback with score 62
runtime/traceback.go:1183:25: inlining call to readgstatus with score 9
Change-Id: Ia86cf5351d29eda64a5426ca0a2a2ec0c2900d81
Reviewed-on: https://go-review.googlesource.com/c/go/+/540775 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
WANG Xuerui [Tue, 13 Dec 2022 08:30:45 +0000 (16:30 +0800)]
cmd/dist, cmd/link, internal, runtime: implement buildmode=plugin for linux/loong64
According to review, buildmode=shared has unfixed shortcomings and
should be considered legacy at this time. So only buildmode=plugin is
going to be added for loong64 which is a relatively new platform.
Change-Id: Iac0b9f57e4ee01755458e180bb24d1b2a146fdf0
Reviewed-on: https://go-review.googlesource.com/c/go/+/480878
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: WANG Xuerui <git@xen0n.name> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: abner chenc <chenguoqi@loongson.cn> Reviewed-by: Meidan Li <limeidan@loongson.cn>
Carlo Alberto Ferraris [Sun, 2 Apr 2023 14:54:35 +0000 (23:54 +0900)]
sync: do not unnecessarily keep alive functions wrapped by Once(Func|Value|Values)
The function passed to OnceFunc/OnceValue/OnceValues may transitively
keep more allocations alive. As the passed function is guaranteed to be
called at most once, it is safe to drop it after the first call is
complete. This avoids keeping the passed function (and anything it
transitively references) alive until the returned function is GCed.
Than McIntosh [Fri, 3 Nov 2023 18:04:57 +0000 (14:04 -0400)]
cmd/compile/internal/inline: minor compile time improvements in func flags
Some small changes to help reduce compile time for function flags
computation. The current implementation of panic path detection adds
an entry to a map for every node in the function, which is wasteful
(and shows up in cpu profiles). Switch to only adding entries where
they are useful. This is especially important for functions with large
map literals and other constructs with many non-statement nodes.
Change-Id: I9cfb2cd1cbf480f21298e6102aa99e2d77219f3d
Reviewed-on: https://go-review.googlesource.com/c/go/+/539696 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Than McIntosh [Mon, 30 Oct 2023 16:55:42 +0000 (12:55 -0400)]
cmd/compile/internal/inline: fix bug in param flags heuristics
Fix a bug in the code that analyzes how function parameters are used,
which wasn't properly handling certain types of closures. The code
in question has support for deriving flags for a given parameter based
on whether it is passed to a call. Example:
When analyzing "bar", we can derive the "FeedsIndirectCall" flag for
parameter "f1" by virtue of the fact that it is passed directly to
"bar", and bar's corresponding parameter "f2" has the flag set. For
a more complex example such as
func foo(f1 func(int)) func() int {
return func(q int) int { <<-- HERE
bar(99, f1)
return q
}
}
func bar(x int, f2 func()) {
f2(x)
}
The heuristics code would panic when examining the closure marked above
due to the fact that the call to "bar" was passing an ir.Name with class
PPARAM, but no such param was present in the enclosing function.
Change-Id: I30436ce716b51bfb03e42e7abe76a4514e6b9285
Reviewed-on: https://go-review.googlesource.com/c/go/+/539320
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Bryan C. Mills [Wed, 28 Jun 2023 19:04:24 +0000 (15:04 -0400)]
testing: use subprocesses in TestTBHelper and TestTBHelperParallel
These tests are checking the output of test functions that call the
Helper methods. However, they were reaching into package internals
instead of running those test functions as actual tests.
That not only produced significant differences in formatting (such as
indentation for subtests), but also caused test flags such as
"-failfast" passed for the overall test run to interfere with the
output formatting.
Now, we run the test functions as real tests in a subprocess,
so that we get the real output and formatting of those tests.
This makes the tests not only more realistic, but also less
sensitive to otherwise-irrelevant implementation details
(such as the names and signatures of unexported types and
functions in the testing package).
Fixes #61016.
Change-Id: I646fbbd7cfeb00382054677f726c05fc9d35d0dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/506955
Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Than McIntosh [Fri, 27 Oct 2023 12:58:16 +0000 (08:58 -0400)]
cmd/compile/internal/inline: rework use of ir.StaticValue
When running the code to compute function properties that feed
inlining heuristics, the existing heuristics implementation makes
fairly extensive use of ir.StaticValue and ir.Reassigned to sharpen
the analysis. These calls turn out to cause a significant compile time
increase, due to the fact that each call can potentially walk every
node in the IR for the function. To help with this problem, switch the
heuristics code over to using the new "batch mode" reassignment helper
added in the previous CL.
Change-Id: Ib15a62416134386e34b7cfa1130a4b413a37b225
Reviewed-on: https://go-review.googlesource.com/c/go/+/537977
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
qiulaidongfeng [Mon, 20 Nov 2023 23:08:53 +0000 (23:08 +0000)]
os: avoid TestFileChdir fail when GOROOT is a symbolic link
If GOROOT is a symbolic link,
the paths obtained from the
first and second Getwd of TestFileChdir are different,
and this CL fixes the test failure in this situation.
Add a new helper type 'ReassignOracle', useful for doing "batch mode"
reassignment analysis, e.g. deciding whether a given ir.Name or (chain
of connected names) has a single definition and is never reassigned.
The intended usage model is for clients to create/initialize a
ReassignOracle for a given function, then make a series of queries
using it (with the understanding that changing/mutating the func body
IR can invalidate the info cached in the oracle). This oracle is
intended to provide the same sort of analysis that ir.StaticValue and
ir.Reassigned carry out, but at a much reduced cost in compile
time.
Notes:
- the new helper isn't actually used for anything useful in this
patch; it will be hooked into the inline heuristics code as part of
a subsequent CL.
- this is probably not an ideal long-term solution; it would be better
to switch to a scheme based a flag on ir.Name, as opposed to a
side table.
Change-Id: I283e748e440a9f595df495f6aa48ee9c498702d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/539319 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Cherry Mui [Mon, 20 Nov 2023 21:33:29 +0000 (16:33 -0500)]
runtime: disable crash stack on Windows
Apparently, on Windows, throwing an exception on a non-system-
allocated crash stack causes EXCEPTION_STACK_OVERFLOW and hangs
the process (see issue #63938). Disable crash stack for now, which
gets us back the the behavior of Go 1.21.
Fixes #63938.
Change-Id: I4c090315b93b484e756b242f0de7a9e02f199261
Reviewed-on: https://go-review.googlesource.com/c/go/+/543996 Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Anton Telyshev [Sun, 29 Oct 2023 21:06:44 +0000 (23:06 +0200)]
cmd/vet: add lost checks in doc
Change-Id: Iacbcb582e263149fede734822cba2df4b8162968
Reviewed-on: https://go-review.googlesource.com/c/go/+/544015 Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Bryan C. Mills [Mon, 20 Nov 2023 20:20:58 +0000 (15:20 -0500)]
cmd/go: don't warn about GOROOT equal to GOPATH when both are the empty string
As of Go 1.19, runtime.GOROOT() reports the empty string if the binary
was built with -trimpath. cmd/go/internal/cfg uses the path of the go
command executable to reverse-engineer the correct GOROOT setting,
but that means that cmd/go's "GOPATH set to GOROOT" warning needs to
use cfg.GOROOT instead of runtime.GOROOT().
In addition, if we fail to find the GOROOT then there is no point in
complaining about GOPATH also being empty: the missing GOROOT will stop
everything right away anyway, so there is no point confusing the user
with an additional warning about GOPATH.
Updates #51461.
Updates #18678.
Updates #3207.
Change-Id: Id7d0f4dc2f229c202dfda4e6e8af5dea909bb16f
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/543955 Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Mauri de Souza Meneguzzo [Sun, 19 Nov 2023 20:21:22 +0000 (20:21 +0000)]
runtime: add crash stack support for wasm
Currently if morestack on g0 happens the wasm runtime prints
"RuntimeError: memory access out of bounds", which is quite misleading.
By switching to a crash stack we can get better stacktraces
for the error.
There is no way to automate tests for this feature on wasm, since
TestG0StackOverflow relies on spawning a subprocess which is not
supported by the wasm port.
The way I got this tested manually is to comment everything in
TestG0StackOverflow, leaving just runtime.G0StackOverflow().
Then it is a matter of invoking the test:
GOOS=js GOARCH=wasm go test runtime -v -run=TestG0StackOverflow
Change-Id: If450f3ee5209bb32efc1abd0a34b1cc4a29d0c46
GitHub-Last-Rev: 0d7c396e4cfeadc1188cae2b55661af10c8189e7
GitHub-Pull-Request: golang/go#63956
Reviewed-on: https://go-review.googlesource.com/c/go/+/539995
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Matthew Dempsky [Mon, 20 Nov 2023 16:19:49 +0000 (08:19 -0800)]
cmd/compile/internal/devirtualize: use CallExpr.GoDefer for PGO
CL 543657 dedup'd the go/defer statement recognition between the
inliner and static devirtualizer. This CL extends that for PGO-based
devirtualization too.
Change-Id: I998753132af1ef17329676f4e17515f16e0acb03
Reviewed-on: https://go-review.googlesource.com/c/go/+/543775
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Matthew Dempsky [Sun, 19 Nov 2023 04:16:47 +0000 (20:16 -0800)]
cmd/compile: interleave devirtualization and inlining
This CL interleaves devirtualization and inlining, so that
devirtualized calls can be inlined.
Fixes #52193.
Change-Id: I681e7c55bdb90ebf6df315d334e7a58f05110d9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/528321
Auto-Submit: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Bypass: Matthew Dempsky <mdempsky@google.com>
Ludi Rehak [Sun, 30 Jul 2023 01:25:42 +0000 (18:25 -0700)]
all: add floating point option for ARM targets
This change introduces new options to set the floating point
mode on ARM targets. The GOARM version number can optionally be
followed by ',hardfloat' or ',softfloat' to select whether to
use hardware instructions or software emulation for floating
point computations, respectively. For example,
GOARM=7,softfloat.
Previously, software floating point support was limited to
GOARM=5. With these options, software floating point is now
extended to all ARM versions, including GOARM=6 and 7. This
change also extends hardware floating point to GOARM=5.
GOARM=5 defaults to softfloat and GOARM=6 and 7 default to
hardfloat.
For #61588
Change-Id: I23dc86fbd0733b262004a2ed001e1032cf371e94
Reviewed-on: https://go-review.googlesource.com/c/go/+/514907
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Andrei Matei [Mon, 5 Jun 2023 23:14:01 +0000 (19:14 -0400)]
ssa: clarify semantics of ID-to-PC translation function
Make it clear that the `block` argument is not always used, and stop
passing it in a case where it didn't really make sense - at the end of a
function.
Change-Id: I2fa86bed6ceb53a1b1cbfda5c3392e7e9da9579d
Reviewed-on: https://go-review.googlesource.com/c/go/+/502115
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: David Chase <drchase@google.com> Reviewed-by: David Chase <drchase@google.com>
Jes Cok [Wed, 15 Nov 2023 13:08:23 +0000 (13:08 +0000)]
syscall: support O_SYNC flag for os.OpenFile on windows
os.OpenFile on windows did not use the O_SYNC flag. This meant
that even if the user set O_SYNC, os.OpenFile would ignore it.
This change adds a new flag FILE_FLAG_WRITE_THROUGH, which is
the equivalent of O_SYNC flag on Linux and is documented in
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea
Matthew Dempsky [Mon, 20 Nov 2023 05:00:17 +0000 (21:00 -0800)]
cmd/compile/internal/inline: allow inlining of checkptr arguments
The early return here is meant to suppress inlining of the function
call itself. However, it also suppresses recursing to visit the call
arguments, which are safe to inline.
Change-Id: I75887574c00931cb622277d04a822bc84c29bfa2
Reviewed-on: https://go-review.googlesource.com/c/go/+/543658
Auto-Submit: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Matthew Dempsky [Mon, 20 Nov 2023 04:18:50 +0000 (20:18 -0800)]
cmd/compile/internal/ir: add CallExpr.GoDefer
The devirtualizer and inliner both want to recognize call expressions
that are part of a go or defer statement. This CL refactors them to
use a single CallExpr.GoDefer flag, which gets set during
normalization of go/defer statements during typecheck.
While here, drop some OCALLMETH assertions. Typecheck has been
responsible for desugaring them into OCALLFUNC for a while now, and
ssagen will check this again for us later anyway.
Change-Id: I3fc370f4417431aae97239313da6fe523f512a2e
Reviewed-on: https://go-review.googlesource.com/c/go/+/543657 Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Mauri de Souza Meneguzzo [Sat, 18 Nov 2023 17:52:22 +0000 (17:52 +0000)]
runtime/internal/atomic: add generic implementation for And/Or
Without having all the architectures implementing the And/Or operators
merged I can't proceed with the public sync/atomic apis. This CL adds a
generic implementation that should work for all the remaining arches,
while waiting for the native assembly implementations in CL 531835,
CL 531678, CL 531895.
I regret the oversight of not pushing this earlier.
For #61395
Change-Id: Ib2d67f359fe324b4743eb79e9c8e52e8f6f5476c
GitHub-Last-Rev: d350927ba1c51d1f708be2f2904f826fdb79b8cd
GitHub-Pull-Request: golang/go#64214
Reviewed-on: https://go-review.googlesource.com/c/go/+/543175 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Russ Cox [Sun, 6 Aug 2023 03:18:31 +0000 (23:18 -0400)]
math/rand/v2: add ChaCha8
ChaCha8 provides a cryptographically strong generator
alongside PCG, so that people who want stronger randomness
have access to that. On systems with 128-bit vector math
assembly (amd64 and arm64), ChaCha8 runs at about the same
speed as PCG (25% slower on amd64, 2% faster on arm64).
Obviously all the claimed benchmark variation other than the
new ChaCha8 benchmark is a lie.
Johan Brandhorst-Satzkorn [Fri, 10 Nov 2023 06:20:27 +0000 (22:20 -0800)]
misc/wasm: support new wasmtime CLI
Wasmtime 14.0.0 introduced new CLI flags and removed the existing
flags, in particular the --max-wasm-stack flag we were using to avoid
errors in some tests.
This introduces a regular expression based switch that uses the old
flags for wasmtime versions < 14 and the new flags otherwise.
Fixes #63718
Change-Id: I44673e7d9f8729065757abdbf8c41e8a61897d6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/541219 Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Joel Sing [Sun, 12 Nov 2023 12:01:34 +0000 (23:01 +1100)]
runtime: add support for crash stack on ppc64/ppc64le
Change-Id: I8d1011509c4f0f529e97055280606603747a2e1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/541775 Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Paul Murphy <murp@ibm.com> Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
Michael Anthony Knyszek [Fri, 17 Nov 2023 16:45:45 +0000 (16:45 +0000)]
runtime: use span.elemsize for accounting in mallocgc
Currently the final size computed for an object in mallocgc excludes the
allocation header. This is correct in a number of cases, but definitely
wrong for memory profiling because the "free" side accounts for the full
allocation slot.
This change makes an explicit distinction between the parts of mallocgc
that care about the full allocation slot size ("the GC's accounting")
and those that don't (pointer+len should always be valid). It then
applies the appropriate size to the different forms of accounting in
mallocgc.
For #64153.
Change-Id: I481b34b2bb9ff923b59e8408ab2b8fb9025ba944
Reviewed-on: https://go-review.googlesource.com/c/go/+/542735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Rhys Hiltner [Tue, 12 Sep 2023 22:44:48 +0000 (15:44 -0700)]
runtime: profile contended lock calls
Add runtime-internal locks to the mutex contention profile.
Store up to one call stack responsible for lock contention on the M,
until it's safe to contribute its value to the mprof table. Try to use
that limited local storage space for a relatively large source of
contention, and attribute any contention in stacks we're not able to
store to a sentinel _LostContendedLock function.
Avoid ballooning lock contention while manipulating the mprof table by
attributing to that sentinel function any lock contention experienced
while reporting lock contention.
Guard collecting real call stacks with GODEBUG=profileruntimelocks=1,
since the available data has mixed semantics; we can easily capture an
M's own wait time, but we'd prefer for the profile entry of each
critical section to describe how long it made the other Ms wait. It's
too late in the Go 1.22 cycle to make the required changes to
futex-based locks. When not enabled, attribute the time to the sentinel
function instead.
Fixes #57071
Change-Id: I3eee0ccbfc20f333b56f20d8725dfd7f3a526b41
Reviewed-on: https://go-review.googlesource.com/c/go/+/528657
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Rhys Hiltner <rhys@justin.tv> Reviewed-by: Than McIntosh <thanm@google.com>
Michael Anthony Knyszek [Thu, 16 Nov 2023 17:42:25 +0000 (17:42 +0000)]
runtime: put allocation headers back at the start the object
A persistent performance regression was discovered on
perf.golang.org/dashboard and this was narrowed down to the switch to
footers. Using allocation headers instead resolves the issue.
The benchmark results for allocation footers weren't realistic, because
they were performed on a machine with enough L3 cache that it completely
hid the additional cache miss introduced by allocation footers.
This means that in some corner cases the Go runtime may no longer
allocate 16-byte aligned memory. Note however that this property was
*mostly* incidental and never guaranteed in any documentation.
Allocation headers were tested widely within Google and no issues were
found, so we're fairly confident that this will not affect very many
users.
Nonetheless, by Hyrum's Law some code might depend on it. A follow-up
change will add a GODEBUG flag that ensures 16 byte alignment at the
potential cost of some additional memory use. Users experiencing both a
performance regression and an alignment issue can also disable the
GOEXPERIMENT at build time.
Bryan C. Mills [Thu, 16 Nov 2023 22:03:06 +0000 (17:03 -0500)]
cmd/go: use a nil *Origin to represent "uncheckable"
Previously, we used the presence of individual origin fields
to decide whether an Origin could be checked for staleness,
with a nil Origin representing “use whatever you have”.
However, that turns out to be fairly bug-prone: if we forget
to populate an Origin somewhere, we end up with an incomplete
check instead of a non-reusable origin (see #61415, #61423).
As of CL 543155, the reusability check for a given query
now depends on what is needed by the query more than what
is populated in the origin. With that in place, we can simplify
the handling of the Origin struct by using a nil pointer
to represent inconsistent or unavailable origin data, and
otherwise always reporting whatever origin information we have
regardless of whether we expect it to be reused.
Updates #61415.
Updates #61423.
Change-Id: I97c51063d6c2afa394a05bf304a80c72c08f82cf
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/543216
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Bryan C. Mills [Thu, 16 Nov 2023 19:11:18 +0000 (14:11 -0500)]
cmd/go: check that expected Origin fields are present to reuse module info
When 'go list' or 'go mod download' uses a proxy to resolve a version
query like "@latest", it may have origin metadata about the resolved
version but not about the inputs that would be needed to resolve the
same query without using the proxy.
We shouldn't just redact the incomplete information, because it might
be useful independent of the -reuse flag. Instead, we examine the
query to decide which origin information it ought to need, and avoid
reusing it if that information isn't included.
Paul E. Murphy [Mon, 26 Sep 2022 18:58:37 +0000 (13:58 -0500)]
cmd/internal/obj/ppc64: cleanup and remove usage of getimpliedreg
getimpliedreg was used to set a default register in cases where
one was implied but not set by the assembler or compiler.
In most cases with constant values, R0 is implied, and is the value
0 by architectural design. In those cases, R0 is always used, so
treat 0 and REG_R0 as interchangeable in those encodings.
Similarly, the pseudo-register SP or FP is used to in place of the
stack pointer, always R1 on PPC64. Unconditionally set this during
classification of NAME_AUTO and NAME_PARAM as it may be 0.
The case where REGSB might be returned from getimpliedreg is never
used. REGSB is aliased to R2, but in practice it is either R0 or R2
depending on buildmode. See symbolAccess in asm9.go for an example.
Change-Id: I7283e66d5351f56a7fe04cee38714910eaa73cb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/434775 Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com> Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Tobias Klauser [Tue, 14 Nov 2023 21:39:13 +0000 (22:39 +0100)]
net/netip: optimize AddrPort.String for IPv6 addresses
Inline the IPv6-case of joinHostPort and avoid the check for a colon in
the address. The address is already known to be an IPv6 address. Also
use iota.Uitoa to convert the uin16 port.
aimuz [Wed, 15 Nov 2023 15:21:23 +0000 (15:21 +0000)]
internal/zstd: fix seek offset bounds check in skipFrame
This change enhances the zstd Reader's skipFrame function to validate
the new offset when skipping frames in a seekable stream, preventing
invalid offsets that could occur previously.
A set of "bad" test strings has been added to fuzz_test.go to extend
the robustness checks against potential decompression panics.
Additionally, a new test named TestReaderBad is introduced in
zstd_test.go to verify proper error handling with corrupted input
strings.
The BenchmarkLarge function has also been refactored for clarity,
removing unnecessary timer stops and resets.
Updates #63824
Change-Id: Iccd248756ad6348afa1395c7799350d07402868a
GitHub-Last-Rev: 63055b91e9413491fe8039ea42d55b823c89ec15
GitHub-Pull-Request: golang/go#64056
Reviewed-on: https://go-review.googlesource.com/c/go/+/541220 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Klaus Post <klauspost@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Cuong Manh Le [Thu, 16 Nov 2023 15:19:37 +0000 (22:19 +0700)]
cmd/compile: use internal/buildcfg for checking newinliner enable
internal/goexperiment reports what GOEXPERIMENT the compiler itself was
compiled with, not what experiment to use for the object code that the
compiler is compiling.
Fixes #64189
Change-Id: I892d78611f8c76376032fd7459e755380afafac6
Reviewed-on: https://go-review.googlesource.com/c/go/+/542995
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Bryan C. Mills [Wed, 15 Nov 2023 18:02:11 +0000 (13:02 -0500)]
cmd/go: propagate origin information for inexact module queries
Module queries for "@latest" and inexact constraints (like "@v1.3")
may consult information about tags and/or branches before finally
returning either a result or an error.
To correctly invalidate the origin information for the -reuse flag,
the reported Origin needs to reflect all of those inputs.
Fixes #61415.
Change-Id: I054acbef7d218a92a3bbb44517326385e458d907
Reviewed-on: https://go-review.googlesource.com/c/go/+/542717
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Rhys Hiltner [Thu, 16 Nov 2023 17:44:21 +0000 (09:44 -0800)]
runtime: disable automatic GC for STW metric tests
A follow-up to https://go.dev/cl/534161 -- calls to runtime/trace.Start
and Stop synchronize with the GC, waiting for any in-progress mark phase
to complete. Disable automatic GCs to quiet the system, so we can
observe only the test's intentional pauses.
Change-Id: I6f8106c42528f9bda9afec1c151119783bbc78dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/543075
Run-TryBot: Rhys Hiltner <rhys@justin.tv>
TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Rhys Hiltner <rhys@justin.tv> Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Cherry Mui [Mon, 13 Nov 2023 17:31:31 +0000 (12:31 -0500)]
runtime: print g pointer in crash stack dump
When debugging a runtime crash with a stack trace, sometimes we
have the g pointer in some places (e.g. as an argument of a
traceback function), but the g's goid in some other places (the
stack trace of that goroutine), which are usually not easy to
match up. This CL makes it print the g pointer. This is only
printed in crash mode, so it doesn't change the usual user stack
trace.
Change-Id: I19140855bf020a327ab0619b665ec1d1c70cca8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/541996 Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Pratt [Thu, 16 Nov 2023 18:00:55 +0000 (13:00 -0500)]
cmd/compile: allow disable of PGO function value devirtualization with flag
Extend the pgodevirtualize debug flag to distinguish interface and
function devirtualization. Setting 1 keeps interface devirtualization
enabled but disables function value devirtualization.
For #64209.
Change-Id: I33aa7eb95ca0bdb215256d8c7cc8f9dac53ae30e
Reviewed-on: https://go-review.googlesource.com/c/go/+/543115 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>