Tamir Duberstein [Mon, 13 May 2019 18:20:53 +0000 (14:20 -0400)]
runtime: resolve latent TODOs
These were added in https://go-review.googlesource.com/1224; according
to austin@google.com these annotations are not valuable - resolving by
removing the TODOs.
Jay Conrod [Thu, 9 May 2019 19:42:55 +0000 (15:42 -0400)]
cmd/go: make 'go get' match patterns against packages, not modules
This is a follow-up to CL 174099, fixing an important TODO.
The 'go help modget' documentation will be clarified in anotehr CL,
pending further discussion.
When invoked without -m, 'go get' will no longer match arguments
containing "..." against module paths. If a module's path matches a
pattern but no packages within the module match the pattern, the
module should not be upgraded. For example, if
golang.org/x/tools/playground and golang.org/x/tools are separate
modules, and only golang.org/x/tools is in the build list,
'go get golang.org/x/tools/playground/...' should add
golang.org/x/tools/playground to the build list and leave
golang.org/x/tools alone.
Updates #26902
Change-Id: I2bd18c7950db1aa7bd8527210c1baf2c7d174375
Reviewed-on: https://go-review.googlesource.com/c/go/+/176578
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
index/suffixarray: index 3-10X faster in half the memory
This CL changes the index/suffixarray construction algorithm from QSufSort to SAIS.
For an N-byte input, QSufSort runs in O(N log N) time and requires
an N-int temporary work space in addition to the N-int output.
In contrast, SAIS runs in O(N) time and, for essentially all real inputs,
is able to use the N-int output buffer as its temporary work space.
(In pathological cases, SAIS must allocate a temporary work space of
at most N/2 ints. There exist more complex variants that guarantee
to avoid the work space in all cases, but they hardly seem worth the cost
given how rare these pathological cases are.)
The SAIS code therefore uses 50% of the memory across the board.
It also runs 3-10X faster on real input text.
This CL also adds more extensive algorithmic tests, including an
exhaustive test over small inputs to catch corner case problems.
Bryan C. Mills [Fri, 10 May 2019 19:26:56 +0000 (15:26 -0400)]
cmd/go/internal/modfetch: fix GOSUMDB test failures
Use cfg.GOSUMDB consistently instead of re-resolving it from the environment.
Set cfg.GOSUMDB to 'off' explicitly in coderepo_test, since it may
include modules that cannot be fetched using a released version of the
'go' command.
Fixes #31964
Change-Id: I17cae9e0c6aa1168ba534e6da4e3652800ac81e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/176538
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
LE Manh Cuong [Fri, 10 May 2019 08:32:24 +0000 (15:32 +0700)]
errors: remove useless condition checking in Is
golang.org/cl/175260 fixed Is panics if target is uncomparable. It did
add an useless condition checking whether target is comparable. Just
remove that condition.
Change-Id: I0a317056479638d209b0a0cbc7010c153558c087
Reviewed-on: https://go-review.googlesource.com/c/go/+/176497 Reviewed-by: Joan Lopez de la Franca Beltran <joanjan14@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Joel Sing [Fri, 10 May 2019 16:21:22 +0000 (02:21 +1000)]
cmd/link: add support for R_AARCH64_LDST128_ABS_LO12_NC relocations
These are encountered when compiling with -linkmode=internal on openbsd/arm64.
Fixes #31940
Change-Id: I851e6a7da0a3fb3e23b4fa2ed8dce3051c680f11
Reviewed-on: https://go-review.googlesource.com/c/go/+/176697
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Tamir Duberstein [Sat, 11 May 2019 18:08:52 +0000 (14:08 -0400)]
net: comment duplicate constant
Change-Id: If5a4d8eff4e51d72fb9dc1d5db2bfe674ec5753b
Reviewed-on: https://go-review.googlesource.com/c/go/+/176717 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Fri, 10 May 2019 19:08:17 +0000 (15:08 -0400)]
test: use a real use function in nilptr2.go
Adjust the dummy use function to a real use. As suggested by the
println calls in the test, nilptr2.go supposes to check that a
used nil pointer dereference panics. This use function is not
real enough so an optimized compiler such as gccgo could
eliminate the call.
The spec requires that even a dummy use would cause a panic.
Unfortunately, due to #31151 this is not true for gccgo at -O1 or
above.
This is silly; there are no non-rewrite lines in the file.
Furthermore, the rulelog file tends to be gigantic
for any non-trivial compilation (measured in gigabytes).
Caleb Spare [Wed, 8 May 2019 20:38:14 +0000 (13:38 -0700)]
cmd/go: move automatic testing.Init call into generated test code
In CL 173722, we moved the flag registration in the testing package into
an Init function. In order to avoid needing changes to user code, we
called Init automatically as part of testing.MainStart.
However, that isn't early enough if flag.Parse is called before the
tests run, as part of package initialization.
Fix this by injecting a bit of code to call testing.Init into test
packages. This runs before any other initialization code in the user's
test package, so testing.Init will be called before any user code can
call flag.Parse.
Jay Conrod [Fri, 10 May 2019 15:49:08 +0000 (11:49 -0400)]
cmd/go: force -coverpkg main packages to be built as libraries
This fixes TestScript/cover_pkgall_multiple_mains, which started
failing after CL 174657.
When compiling main packages with coverage instrumentation
(e.g., for -coverpkg all), we now pass -p with the full import path
instead of '-p main'. This avoids link errors
'duplicate symbol main.main (types 1 and 1)'.
Russ Cox [Fri, 10 May 2019 15:13:06 +0000 (11:13 -0400)]
runtime: fix windows-amd64-2012 build
I forgot that in Go assembly, x+16(SP) is not the same as 16(SP).
The former is the virtual stack pointer (one word below FP on x86)
while the latter is the actual stack pointer.
Bryan C. Mills [Fri, 10 May 2019 15:10:50 +0000 (11:10 -0400)]
cmd/go/testdata/script: remove skips and clarify comments in mod_*_upgrade_pseudo tests
These tests were added in CL 174206.
They required a 'git' binary and network access in an earlier draft,
but now use the test-local module proxy instead, so no longer need to
be skipped when those resources are not present.
Updates #30634
Change-Id: I5f36c6c776209a89bc45d133847df5052b55da59
Reviewed-on: https://go-review.googlesource.com/c/go/+/176537
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox [Thu, 9 May 2019 22:34:29 +0000 (18:34 -0400)]
runtime: fix s390x build
The new prototypes of duffzero and duffcopy must be
accompanied by functions. Otherwise buildmode=shared
(in particular, misc/cgo/testshared) has missing symbols.
The right fix, of course, is to implement these on s390x.
Brad Fitzpatrick [Thu, 9 May 2019 21:04:39 +0000 (21:04 +0000)]
os/user: make Current return better error w/o cgo & complete environment
Fixes #31949
Change-Id: Ib96a43e4c56a00c5ba04e4d213255a063058ae08
Reviewed-on: https://go-review.googlesource.com/c/go/+/176337 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Russ Cox [Wed, 8 May 2019 18:56:48 +0000 (14:56 -0400)]
runtime: fix vet complaints for solaris/amd64, illumos/amd64
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for these GOOS/GOARCHes,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
Russ Cox [Wed, 8 May 2019 17:59:48 +0000 (13:59 -0400)]
runtime: fix vet complaints for linux/arm64, linux/mips*, linux/ppc64*, linux/s390x
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for these GOOS/GOARCHes,
except for an unresolved complaint on mips/mipsle that is a bug in vet,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
Russ Cox [Wed, 8 May 2019 17:16:05 +0000 (13:16 -0400)]
runtime: fix vet complaints for linux/amd64
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "GOOS=linux GOARCH=amd64 go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
Michael Anthony Knyszek [Thu, 9 May 2019 19:55:39 +0000 (19:55 +0000)]
runtime: fix js/wasm lock implementation
Trybots started failing on js/wasm after golang.org/cl/175797 landed,
but it seemed completely unrelated. It would fail very consistently on
the heapsampling.go test.
Digging deeper it was very difficult to ascertain what was going wrong,
but clearly m.locks for some m was non-zero when calling into the
scheduler.
The failure comes from the fact that lock calls into gosched, but it's
unclear how exactly we got there in the first place; there should be no
preemption in this single-threaded context.
Furthermore, lock shouldn't be calling gosched_m at all because in a
single-threaded context, the thread shouldn't be preempted until it
actually unlocks.
But, digging further it turns out the implementation in lock_js.go never
incremented or decremented m.locks. This is definitely wrong because
many parts of the runtime depend on that value being set correctly.
So, this change removes the loop which calls into gosched_m (which
should be unnecessary) and increments and decrements m.locks
appropriately. This appears to fix the aforementioned failure.
Jay Conrod [Fri, 3 May 2019 22:06:41 +0000 (18:06 -0400)]
all: document vendoring in the standard library
Added documentation that explains special cases for vendored packages
in the standard library and provides instructions for updating vendor
directories.
Fixes #31806
Change-Id: Ib697ed18eae28023ab0bfb9f4d250992c393571d
Reviewed-on: https://go-review.googlesource.com/c/go/+/174999 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Michael Anthony Knyszek [Tue, 7 May 2019 19:04:41 +0000 (19:04 +0000)]
runtime: split spans when scavenging if it's more than we need
This change makes it so that during scavenging we split spans when the
span we have next for scavenging is larger than the amount of work we
have left to do.
The purpose of this change is to improve the worst-case behavior of the
scavenger: currently, if the scavenger only has a little bit of work to
do but sees a very large free span, it will scavenge the whole thing,
spending a lot of time to get way ahead of the scavenge pacing for no
reason.
With this change the scavenger should follow the pacing more closely,
but may still over-scavenge by up to a physical huge page since the
splitting behavior avoids breaking up huge pages in free spans.
This change is also the culmination of the scavenging improvements, so
we also include benchmark results for this series (starting from
"runtime: merge all treaps into one implementation" until this patch).
This patch stack results in average and peak RSS reductions (up to 11%
and 7% respectively) for some benchmarks, with mostly minimal
performance degredation (3-4% for some benchmarks, ~0% geomean). Each of
these benchmarks was executed with GODEBUG=madvdontneed=1 on Linux; the
performance degredation is even smaller when MADV_FREE may be used, but
the impact on RSS is much harder to measure. Applications that generally
maintain a steady heap size for the most part show no change in
application performance.
These benchmarks are taken from an experimental benchmarking suite
representing a variety of open-source Go packages, the raw results may
be found here:
Michael Anthony Knyszek [Wed, 17 Oct 2018 23:29:42 +0000 (23:29 +0000)]
runtime: add background scavenger
This change adds a background scavenging goroutine whose pacing is
determined when the heap goal changes. The scavenger is paced to use
at most 1% of the mutator's time for most systems. Furthermore, the
scavenger's pacing is computed based on the estimated number of
scavengable huge pages to take advantage of optimizations provided by
the OS.
The purpose of this scavenger is to deal with a shrinking heap: if the
heap goal is falling over time, the scavenger should kick in and start
returning free pages from the heap to the OS.
Also, now that we have a pacing system, the credit system used by
scavengeLocked has become redundant. Replace it with a mechanism which
only scavenges on the allocation path if it makes sense to do so with
respect to the new pacing system.
Michael Anthony Knyszek [Thu, 18 Oct 2018 20:09:03 +0000 (20:09 +0000)]
runtime: remove periodic scavenging
This change removes the periodic scavenger which goes over every span
in the heap and scavenges it if it hasn't been used for 5 minutes. It
should no longer be necessary if we have background scavenging
(follow-up).
Than McIntosh [Tue, 30 Apr 2019 18:45:35 +0000 (14:45 -0400)]
cmd/compile: emit DWARF call_line attrs with data4 form on iOS
When targeting iOS, change the format (DWARF "form") of the call line
attribute for inlined subroutine DIEs, to work around an apparent bug
in /usr/bin/symbols on Darwin.
[Just for posterity: there is nothing wrong with using DW_FORM_udata
for the call_line attribute form; this is perfectly legal DWARF (and
is in fact recommended by the standard relative to data4, which is
less descriptive and often takes more space). The form switch is there
just to make the Apple tools happy.]
Updates #31459.
Change-Id: Iaf362788a8c6684eea4cde8956c0661b694cecc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/174538
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Russ Cox [Thu, 9 May 2019 12:57:20 +0000 (08:57 -0400)]
cmd/vet/all: update whitelist for vet fixes
The vetall builder runs vet straight out of golang.org/x/tools,
so submiting CL 176097 in that repo will break the builder
by making all these whitelist entries stale.
Submiting this CL will fix it, by removing them.
The addition of the gcWriteBarrier declaration in runtime/stubs.go
is necessary because the diagnostic is no longer emitted on arm,
so it must be removed from all.txt. Adding it to runtime is better
than adding it to every-other-goarch.txt.
Than McIntosh [Wed, 8 May 2019 19:44:49 +0000 (15:44 -0400)]
cmd/link: fix link time regression in object file reading
In CL 173938, the linker's object file reader was switched over to
selectively create strings backed with read-only mmap'd memory.
In the process a call to r.rd.Offset() was added to readSymName(),
which greatly increased the number of system calls (Offset does a
seek system call).
This patch changes the object file reader so that all reads are done
directly from the mmap'd data if it is present, and adds logic to keep
track of the offset within the rodata consumed so far. Doing this gets
rid of the calls to r.rd.Offset() and the corresponding seek system
calls.
Also as part of this change, hoist the calls to objabi.PathToPrefix
up into the initial setup code for object reading, and store the
result in the reader (since objabi.PathToPrefix was also coming up
as hot in the profile).
cmd/go: diagnose go.mod and vendor out of sync in std and cmd
The most common failure mode of the current std/cmd setup is
going to be people running "go get m@latest" and then not running
"go mod vendor" and being confused about getting the old m.
Diagnose and report what to do.
Also, having done the check, when in the standard library,
switch the go command to -mod=vendor mode.
This avoids some network accesses I saw when running
'go clean -modcache' before doing some work in cmd.
Brad Fitzpatrick [Wed, 8 May 2019 21:28:14 +0000 (21:28 +0000)]
cmd/go: set the "generate" build tag in go generate, per design doc
And use it in two internal windows packages, so they don't show up in
"go list std" or binary releases on non-Windows platforms.
Fixes #31920
Change-Id: Iaa292b6015c9d7310dd677c9e296006440ba5e27
Reviewed-on: https://go-review.googlesource.com/c/go/+/175983 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Austin Clements [Wed, 8 May 2019 20:42:53 +0000 (16:42 -0400)]
cmd/link: write memprofile in legacy format for compilebench
compilebench depends on the legacy heap profile format to read the
allocation stats of build tools. We're adding a benchmark for the
linker to compilebench, so we need the linker to emit the heap profile
in the legacy format.
This is the linker equivalent of CL 35484, which did this for the
compiler.
Marwan Sulaiman [Fri, 8 Mar 2019 07:33:59 +0000 (02:33 -0500)]
cmd/go: disallow go.sum updates in -mod=readonly
When running go build with the flag -mod=readonly, it fails the build if
go.sum files requires updating. This ensures that CI/CD systems get a
complete go.sum file so that they'd never hit a notary,
assuming the CI/CD system passes the above flag.
I am not familiar with the entire codebase but I assume goSum.dirty
will always be true if go.sum has any missing lines.
Fixes #30667
Change-Id: I767d3b594055d8c10048f4c68e6687c94bb0545c
Reviewed-on: https://go-review.googlesource.com/c/go/+/166237 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Alan Donovan [Thu, 25 Oct 2018 20:44:44 +0000 (16:44 -0400)]
go/types: add CheckExpr function to type-check an expression
In IDE-like applications (and also in tests), there is often a need to
type-check an expression as if it appeared at a particular position in
the source code of an already typed-checked package.
Eval was added to address this need, but did so only partially,
stopping short of exposing a type-annotated expression tree. This
makes it impossible to resolve an expression such as new(T).x and
discover what Object x refers to. CheckExpr exposes that generality.
Eval is now implemented in terms of CheckExpr.
This change includes a test that demonstrates the object resolution
functionality just described.
Dmitry Vyukov [Tue, 7 May 2019 07:41:46 +0000 (09:41 +0200)]
regexp: clarify docs re Submatch result
Currently we say that a negative index means no match,
but we don't say how "no match" is expressed when 'Index'
is not present. Say how it is expressed.
Change-Id: I82b6c9038557ac49852ac03642afc0bc545bb4a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/175677 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Russ Cox [Tue, 7 May 2019 16:17:05 +0000 (12:17 -0400)]
doc: update /doc/asm compiler output example
The compiler output shown in the doc is now quite old
(most of the changes happened in Go 1.5).
Update it to be more like what users will actually see.
Also explain how to get literal machine code again.
Cherry Zhang [Wed, 8 May 2019 03:34:13 +0000 (23:34 -0400)]
reflect: add a test for Calling a Method of a direct interface type
Gccgo's implementation of direct interface types has bugs that
causes reflect Call of method from Type.Method fail. CL 175837
and CL 175798 fix the bug. This CL adds a test.
Stephan Zuercher [Mon, 6 May 2019 17:55:45 +0000 (10:55 -0700)]
image/png: fix palette extension to handle 255 color images
The PNG decode path attempts to handle paletted images that refer to
non-existent palette indicies. This PR fixes a corner case were images
that have exactly 255 palette colors and an IDAT chunk that references
palette index 255 produce an invalid image such that invoking At on
the pixel(s) in question causes an index out of range panic.
erifan01 [Thu, 28 Mar 2019 10:53:42 +0000 (10:53 +0000)]
math/big: fix the bug in assembly implementation of shlVU on arm64
For the case where the addresses of parameter z and x of the function
shlVU overlap and the address of z is greater than x, x (input value)
can be polluted during the calculation when the high words of x are
overlapped with the low words of z (output value).
Robert Griesemer [Mon, 6 May 2019 23:26:15 +0000 (16:26 -0700)]
cmd/compile: avoid compiler crash for recursive interface type
This change is a simple work-around to avoid a compiler crash
and provide a reasonable error message. A future change should
fix the root cause for this problem.
Fixes #23823.
Change-Id: Ifc80d9f4d35e063c378e54d5cd8d1cf4c0d2ec6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/175518 Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/cgo: fix unexpected semicolon in rewritten line
Followup to CL 157961 and CL 158457.
Finish the list of operators and punctuation
that disable semicolon insertion at end-of-line
The reported case was "(" but "." was also missing.
Fixes #31017.
Change-Id: I0c06443f38dc8250c62e3aadd104abfa0e3be074
Reviewed-on: https://go-review.googlesource.com/c/go/+/174524
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Filippo Valsorda [Thu, 2 May 2019 21:31:56 +0000 (17:31 -0400)]
crypto/ed25519: promote from golang.org/x/crypto/ed25519
The crypto/tls and crypto/x509 APIs leak PublicKey and PrivateKey types,
so in order to add support for Ed25519 certificates we need the ed25519
package in the stdlib.
It's also a primitive that's reasonable to use directly in applications,
as it is a modern, safe and fast signing algorithm, for which there
aren't higher level APIs. (The nacl/sign API is limiting in that it
repeats the message.)
A few docs changes will come in a follow-up, and a CL will land on
golang.org/x/crypto/ed25519 to make it a type alias wrapper on Go 1.13+.
Brad Fitzpatrick [Sun, 5 May 2019 02:25:42 +0000 (02:25 +0000)]
cmd/dist: allow builders to control granularity of test/ directory sharding
Even with 10 shards on builders, it still takes about ~2.5 minutes per
shard (and getting slower all the time as the test/ directory grows).
I'm currently experimenting with massively sharding out testing on
Cloud Run (each dist test & normal TestFoo func all running in
parallel), and in such a setup, 2.5 minutes is an eternity. I'd like
to increase that dist test's sharding from 10 to more like 1,000.
Updates golang/go#31834
Change-Id: I8b02989727793b5b5b2013d67e1eb01ef4786e28
Reviewed-on: https://go-review.googlesource.com/c/go/+/175297 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Mon, 6 May 2019 19:58:54 +0000 (12:58 -0700)]
reflect: special-case panic message when returning Value{} from MakeFunc function
Before this CL we used to panic with "nil pointer dereference" because
the value we're calling assignTo on is the zero Value. Provide a better
error message.
Michael Anthony Knyszek [Thu, 18 Apr 2019 15:42:58 +0000 (15:42 +0000)]
runtime: ensure free and unscavenged spans may be backed by huge pages
This change adds a new sysHugePage function to provide the equivalent of
Linux's madvise(MADV_HUGEPAGE) support to the runtime. It then uses
sysHugePage to mark a newly-coalesced free span as backable by huge
pages to make the freeHugePages approximation a bit more accurate.
The problem being solved here is that if a large free span is composed
of many small spans which were coalesced together, then there's a chance
that they have had madvise(MADV_NOHUGEPAGE) called on them at some point,
which makes freeHugePages less accurate.
Michael Anthony Knyszek [Thu, 2 May 2019 07:05:21 +0000 (07:05 +0000)]
runtime: split spans during allocation without treap removal
Now that the treap is first-fit, we can make a nice optimization.
Mainly, since we know that span splitting doesn't modify the relative
position of a span in a treap, we can actually modify a span in-place
on the treap. The only caveat is that we need to update the relevant
metadata.
To enable this optimization, this change introduces a mutate method on
the iterator which takes a callback that is passed the iterator's span.
The method records some properties of the span before it calls into the
callback and then uses those records to see what changed and update
treap metadata appropriately.
Michael Anthony Knyszek [Mon, 29 Apr 2019 20:58:39 +0000 (20:58 +0000)]
runtime: scavenge huge spans first
This change adds two new treap iteration types: one for large
unscavenged spans (contain at least one huge page) and one for small
unscavenged spans. This allows us to scavenge the huge spans first by
first iterating over the large ones, then the small ones.
Also, since we now depend on physHugePageSize being a power of two,
ensure that that's the case when it's retrieved from the OS.
Michael Anthony Knyszek [Wed, 1 May 2019 18:02:14 +0000 (18:02 +0000)]
runtime: make treap iteration more efficient
This change introduces a treapIterFilter type which represents the
power set of states described by a treapIterType.
This change then adds a treapIterFilter field to each treap node
indicating the types of spans that live in that subtree. The field is
maintained via the same mechanism used to maintain maxPages. This allows
pred, succ, start, and end to be judicious about which subtrees it will
visit, ensuring that iteration avoids traversing irrelevant territory.
Without this change, repeated scavenging attempts can end up being N^2
as the scavenger walks over what it already scavenged before finding new
spans available for scavenging.
Finally, this change also only scavenges a span once it is removed from
the treap. There was always an invariant that spans owned by the treap
may not be mutated in-place, but with this change violating that
invariant can cause issues with scavenging.
Michael Anthony Knyszek [Mon, 29 Apr 2019 20:42:27 +0000 (20:42 +0000)]
runtime: merge all treaps into one implementation
This change modifies the treap implementation to support holding all
spans in a single treap, instead of keeping them all in separate treaps.
This improves ergonomics for nearly all treap-related callsites.
With that said, iteration is now more expensive, but it never occurs on
the fast path, only on scavenging-related paths.
This change opens up the opportunity for further optimizations, such as
splitting spans without treap removal (taking treap removal off the span
allocator's critical path) as well as improvements to treap iteration
(building linked lists for each iteration type and managing them on
insert/removal, since those operations should be less frequent).
Cherry Zhang [Wed, 1 May 2019 00:47:48 +0000 (20:47 -0400)]
cmd/internal/obj: write package path at compile time if possible
Currently, when the compiler emits a symbol name in the object
file, it uses "". for the package path of the package being
compiled. This is then expanded in the linker to the actual
package path.
With CL 173938, it does not need an allocation if the symbol name
does not need expansion. In many cases, the compiler actually
knows the package path (through the -p flag), so we could just
write it out in compile time, without fixing it up in the linker.
This reduces allocations in the linker.
In case that the package path is not known (compiler's -p flag is
missing, or the object file is generated by the assembler), the
linker still does the expansion.
This reduces ~100MB allocations (~10% inuse_space) in linking
k8s.io/kubernetes/cmd/kube-apiserver on Linux/AMD64.
Also makes the linker a little faster: linking cmd/go on
Linux/AMD64:
Real 1.13 ± 1% 1.11 ± 1% -2.13% (p=0.000 n=10+10)
User 1.17 ± 3% 1.14 ± 5% -3.14% (p=0.003 n=10+10)
Sys 0.34 ±15% 0.34 ±15% ~ (p=0.986 n=10+10)
The caveat is that the object files get slightly bigger. On
Linux/AMD64, runtime.a gets 2.1% bigger, cmd/compile/internal/ssa
(which has a longer import path) gets 2.8% bigger.
This reveals that when building an unnamed plugin (e.g.
go build -buildmode=plugin x.go), the go command passes different
package paths to the compiler and to the linker. Before this CL
there seems nothing obviously broken, but given that the compiler
already emits the package's import path in various places (e.g.
debug info), I guess it is possible that this leads to some
unexpected behavior. Now that the compiler writes the package
path in more places, this disagreement actually leads to
unresolved symbols. Adjust the go command to use the same package
path for both compiling and linking.
David Chase [Thu, 2 May 2019 17:36:10 +0000 (13:36 -0400)]
cmd/compile: make numberlines line mismatch check ignore columns
This does not repair #31786, and in fact also unfixes the revert
of CL 174617. We were just getting lucky when it looked like
it was working. And unfortunately for the bug, there does not
appear to be any particular problems with the line numbers;
if anything they're a couple of extras, i.e., stepping might
repeat, rather than skip. Delve works fine either way.
Updates #31786.
Change-Id: I5c2fdc2a0265bb99773b3a85492a3db557dffee4
Reviewed-on: https://go-review.googlesource.com/c/go/+/174948
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>