Ian Gudger [Fri, 17 Apr 2020 11:04:23 +0000 (04:04 -0700)]
net: add (*Resolver).LookupIP
Previously, looking up only IPv4 or IPv6 addresses was only possible
with DefaultResolver via ResolveIPAddr. Add this functionality to the
Resolver type with a new method, LookupIP. This largely brings Resolver
functionally to parity with the global functions. The name LookupIP is
used over ResolveIPAddr to be consistent with the other Resolver
methods.
There are two main benefits to (*Resolver).LookupIP over
(*Resolver).LookupHost. First is an ergonomic benefit. Wanting a
specific family of address is common enough to justify a method, evident
by the existence of ResolveIPAddr. Second, this opens the possibility of
not performing unnecessary DNS requests when only a specific family of
addresses are needed. This optimization is left to follow up work.
Daniel Theophanes [Tue, 28 Apr 2020 14:31:12 +0000 (07:31 -0700)]
database/sql: document Connect and Close may need a timeout
Opening a connection with Connect should still create a derived
context with a timeout because some clients will not use a timeout
and the connection pool may open a connection asynchronously.
Likewise, if a connection close makes a network operation it should
provide some type of sane timeout for the operation.
Michael Anthony Knyszek [Tue, 28 Apr 2020 15:14:56 +0000 (15:14 +0000)]
runtime: fix block leak due to race in span set
The span set data structure may leak blocks due to a race in the logic
to check whether it's safe to free a block. The simplest example of this
race is between two poppers:
1. Popper A claims slot spanSetEntries-2.
2. Popper B claims slot spanSetEntries-1.
3. Popper A gets descheduled before it subtracts from block.used.
4. Popper B subtracts from block.used, sees that claimed
spanSetEntries-1, but also that block.used != 0, so it returns.
5. Popper A comes back and subtracts from block.used, but it didn't
claim spanSetEntries-1 so it also returns.
The spine is left with a stale block pointer and the block later gets
overwritten by pushes, never to be re-used again.
The problem here is that we designate the claimer of slot
spanSetEntries-1 to be the one who frees the block, but that may not be
the thread that actually does the last subtraction from block.used.
Fixing this problem is tricky, and the fundamental problem there is that
block.used is not stable: it may be observed to be zero, but that
doesn't necessarily mean you're the last popper!
Do something simpler: keep a counter of how many pops have happened to a
given block instead of block.used. This counter monotonically increases
when a pop is _completely done_. Because this counter is monotonically
increasing, and only increases when a popper is done, then we know for
sure whichever popper is the last to increase it (i.e. its value is
spanSetBlockEntries) is also the last popper in the block. Because the
race described above still exists, the last popper may not be the one
which claimed the last slot in the block, but we know for certain nobody
else is popping from that block anymore so we can safely free it.
Finally, because pops serialize with pushes to the same slot, we need
not worry about concurrent pushers at all.
Updates #37487.
Change-Id: I6697219372774c8ca7d8ee6895eaa230a64ce9e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/230497
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Anthony Knyszek [Tue, 28 Apr 2020 16:40:38 +0000 (16:40 +0000)]
runtime: flush mcaches to mcentral before reading memstats
Currently mcaches are flushed to mcentral after a bunch of memstats have
already been read. This is not safe (in the sense that it doesn't ensure
consisent memstats) since memstats may in general change when mcentral
data structures are manipulated.
Note that prior to the new mcentral implementation this was not a
problem because mcentral operations happened to never modify certain
memstats. As of the new mcentral implementation, we might for example
persistentalloc when uncaching a span, which would change memstats. This
can cause a skew between the value of sys (which currently is calculated
before mcaches are flushed) and the value of gc_sys and other_sys.
Fix this by moving mcache flushing to the very top of updatememstats.
Also leave a comment explaining that this must be done first, in
general, because mcentrals make no guarantee that they will not
influence memstats (and doing so would be unnecessarily restrictive).
This is a second attempt at CL 230024, with
cmd/go/testdata/script/mod_retention.txt updated to perform a
version-independent comparison on the 'go' version added to a go.mod
file that lacks one.
Fixes #38708
Change-Id: I15dcd83b51ed5ec57946b419bcbaec41e85a46f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/230382
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Ruixin Bao [Mon, 27 Apr 2020 23:20:45 +0000 (16:20 -0700)]
cmd/compile: adopt strong aux typing for some s390x rules
Convert some optimizations rules to strongly-typed versions. So far, I
have only converted the rules that need no additional changes (i.e: only
need to change '->' to "=>").
This CL covers the rules from line 478 - line 800 in S390X.rules file.
Some compare and branch rules also fall in this range, but they were
already done previously in another CL.
Passes toolstash-check.
Change-Id: I9167c5f1a32f4fd6c29bacc13fff95e83b0533e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/230338 Reviewed-by: Michael Munday <mike.munday@ibm.com>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
go/types: improve errors for method calls with pointer receivers
The compiler has better error messages for methods called without a
pointer receiver when one is expected. This change is similar to
CL 229801, but for method calls.
Also, added better error messages for functions called with the wrong
capitalization. I left the third TODO in this switch statement almost
as-is because I'm not sure that the extra complexity is worth it -
I adjusted the error to look like the one the compiler reports.
os, internal/poll, internal/syscall/unix: use copy_file_range on Linux
Linux 4.5 introduced (and Linux 5.3 refined) the copy_file_range
system call, which allows file systems the opportunity to implement
copy acceleration techniques. This commit adds support for
copy_file_range(2) to the os package.
Introduce a new ReadFrom method on *os.File, which makes *os.File
implement the io.ReaderFrom interface. If dst and src are both files,
this enables io.Copy(dst, src) to call dst.ReadFrom(src), which, in
turn, will call copy_file_range(2) if possible. If copy_file_range(2)
is not supported by the host kernel, or if either of dst or src
refers to a non-regular file, ReadFrom falls back to the regular
io.Copy code path.
Add internal/poll.CopyFileRange, which acquires locks on the
appropriate poll.FDs and performs the actual work, as well as
internal/syscall/unix.CopyFileRange, which wraps the copy_file_range
system call itself at the lowest level.
Rework file layout in internal/syscall/unix to accomodate the
additional system call numbers needed for copy_file_range.
Merge these definitions with the ones used by getrandom(2) into
sysnum_linux_$GOARCH.go files.
A note on additional optimizations: if dst and src both refer to pipes
in the invocation dst.ReadFrom(src), we could, in theory, use the
existing splice(2) code in package internal/poll to splice directly
from src to dst. Attempting this runs into trouble with the poller,
however. If we call splice(src, dst) and see EAGAIN, we cannot know
if it came from src not being ready for reading or dst not being
ready for writing. The write end of src and the read end of dst are
not under our control, so we cannot reliably use the poller to wait
for readiness. Therefore, it seems infeasible to use the new ReadFrom
method to splice between pipes directly. In conclusion, for now, the
only optimization enabled by the new ReadFrom method on *os.File is
the copy_file_range optimization.
Fixes #36817.
Change-Id: I696372639fa0cdf704e3f65414f7321fc7d30adb
Reviewed-on: https://go-review.googlesource.com/c/go/+/229101
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
IntSize is an untyped constant that does not need explicit conversion.
Annotating IntSize as an int and running github.com/mdempsky/unconvert
reveals these two cases.
Michael Anthony Knyszek [Wed, 22 Apr 2020 21:36:11 +0000 (21:36 +0000)]
runtime: ensure allocToCache updates searchAddr in a valid way
Currently allocToCache assumes it can move the search address past the
block it allocated the cache from, which violates the property that
searchAddr should always point to mapped memory (i.e. memory represented
by pageAlloc.inUse).
This bug was already fixed once for pageAlloc.alloc in the Go 1.14
release via CL 216697, but that changed failed to take into account
allocToCache.
Fixes #38605.
Change-Id: Id08180aa10d19dc0f9f551a1d9e327a295560dff
Reviewed-on: https://go-review.googlesource.com/c/go/+/229577
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Ruixin(Peter) Bao [Mon, 27 Apr 2020 19:23:37 +0000 (15:23 -0400)]
hash/crc32: simplify hasVX checking on s390x
Originally, we use an assembly function that returns a boolean result to
tell whether the machine has vector facility or not. It is now no longer
needed when we can directly use cpu.S390X.HasVX variable. This CL
also removes the last occurence of hasVectorFacility function on s390x.
Change-Id: Id20cb746c21eacac5e13344b362e2d87adfe4317
Reviewed-on: https://go-review.googlesource.com/c/go/+/230337 Reviewed-by: Michael Munday <mike.munday@ibm.com>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ruixin(Peter) Bao [Mon, 27 Apr 2020 13:51:01 +0000 (09:51 -0400)]
math/big: simplify hasVX checking on s390x
Originally, we use an assembly function that returns a boolean result to
tell whether the machine has vector facility or not. It is now no longer
needed when we can directly use cpu.S390X.HasVX variable.
Change-Id: Ic1dae851982532bcfd9a9453416c112347f21d87
Reviewed-on: https://go-review.googlesource.com/c/go/+/230318 Reviewed-by: Michael Munday <mike.munday@ibm.com>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ruixin Bao [Mon, 27 Apr 2020 19:02:52 +0000 (12:02 -0700)]
math: simplify hasVX checking on s390x
Originally, we use an assembly function that returns a boolean result to
tell whether the machine has vector facility or not. It is now no longer
needed when we can directly use cpu.S390X.HasVX variable.
Change-Id: Ic3ffeb9e63238ef41406d97cdc42502145ddb454
Reviewed-on: https://go-review.googlesource.com/c/go/+/230319 Reviewed-by: Michael Munday <mike.munday@ibm.com>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ruixin(Peter) Bao [Mon, 21 Oct 2019 17:18:31 +0000 (13:18 -0400)]
crypto/ed25519: implement ed25519 on s390x using KDSA instruction
This CL allows the usage of KDSA instruction when it is available. The
instruction is designed to be resistant to side channel attacks and
offers performance improvement for ed25519.
Benchmarks:
name old time/op new time/op delta
Signing-8 120µs ±20% 62µs ±12% -48.40% (p=0.000 n=10+10)
Verification-8 325µs ±17% 69µs ±10% -78.80% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
Signing-8 448B ± 0% 0B -100.00% (p=0.000 n=10+10)
Verification-8 288B ± 0% 0B -100.00% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Signing-8 5.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
Verification-8 2.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
Change-Id: I0330ce83d807370b419ce638bc2cae4cb3c250dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/202578
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Munday <mike.munday@ibm.com>
Dan Scales [Mon, 13 Apr 2020 15:37:03 +0000 (08:37 -0700)]
runtime: added several new lock-rank partial order edges
Several new ones came from my testing (long, repeated runs) and one (assistQueue ->
spine) came from the staticlockranking builder (filed as issue 38441).
Fixes #38441
Change-Id: I4268da0d8b8cc51251eba6bd936110c8ab4c4e61
Reviewed-on: https://go-review.googlesource.com/c/go/+/229480
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Fri, 24 Apr 2020 20:06:57 +0000 (20:06 +0000)]
runtime: bound small object sweeping to 100 spans when allocating
Currently, the small object sweeper will sweep until it finds a free
slot or there are no more spans of that size class to sweep. In dense
heaps, this can cause sweeping for a given size class to take
unbounded time, and gets worse with larger heaps.
This CL limits the small object sweeper to try at most 100 spans
before giving up and allocating a fresh span. Since it's already shown
that 100 spans are completely full at that point, the space overhead
of this fresh span is at most 1%.
This CL is based on an experimental CL by Austin Clements (CL 187817)
and is updated to be part of the mcentral implementation, gated by
go115NewMCentralImpl.
Michael Anthony Knyszek [Thu, 20 Feb 2020 20:58:45 +0000 (20:58 +0000)]
runtime: add new mcentral implementation
Currently mcentral is implemented as a couple of linked lists of spans
protected by a lock. Unfortunately this design leads to significant lock
contention.
The span ownership model is also confusing and complicated. In-use spans
jump between being owned by multiple sources, generally some combination
of a gcSweepBuf, a concurrent sweeper, an mcentral or an mcache.
So first to address contention, this change replaces those linked lists
with gcSweepBufs which have an atomic fast path. Then, we change up the
ownership model: a span may be simultaneously owned only by an mcentral
and the page reclaimer. Otherwise, an mcentral (which now consists of
sweep bufs), a sweeper, or an mcache are the sole owners of a span at
any given time. This dramatically simplifies reasoning about span
ownership in the runtime.
As a result of this new ownership model, sweeping is now driven by
walking over the mcentrals rather than having its own global list of
spans. Because we no longer have a global list and we traditionally
haven't used the mcentrals for large object spans, we no longer have
anywhere to put large objects. So, this change also makes it so that we
keep large object spans in the appropriate mcentral lists.
In terms of the static lock ranking, we add the spanSet spine locks in
pretty much the same place as the mcentral locks, since they have the
potential to be manipulated both on the allocation and sweep paths, like
the mcentral locks.
This new implementation is turned on by default via a feature flag
called go115NewMCentralImpl.
Benchmark results for 1 KiB allocation throughput (5 runs each):
Michael Anthony Knyszek [Thu, 20 Feb 2020 21:28:02 +0000 (21:28 +0000)]
runtime: implement the spanSet data structure
This change implements the spanSet data structure which is based off of
the gcSweepBuf data structure. While the general idea is the same (one
has two of these which one switches between every GC cycle; one to push
to and one to pop from), there are some key differences.
Firstly, we never have a need to iterate over this data structure so
delete numBlocks and block. Secondly, we want to be able to pop from the
front of the structure concurrently with pushes to the back. As a result
we need to maintain both a head and a tail and this change introduces an
atomic headTail structure similar to the one used by sync.Pool. It also
implements popfirst in a similar way.
As a result of this headTail, we need to be able to explicitly reset the
length, head, and tail when it goes empty at the end of sweep
termination, so add a reset method.
Michael Anthony Knyszek [Thu, 13 Feb 2020 21:13:40 +0000 (21:13 +0000)]
runtime: manage a pool of spanSetBlocks and free them eagerly
This change adds a global pool of spanSetBlocks to the spanSet data
structure and adds support for eagerly freeing these blocks back to the
pool if the block goes empty.
This change prepares us to use this data structure in more places in the
runtime by allowing reuse of spanSetBlock.
Michael Anthony Knyszek [Thu, 20 Feb 2020 21:22:03 +0000 (21:22 +0000)]
runtime: add spanSet data structure
This change copies the gcSweepBuf data structure into a new file and
renames it spanSet. It will serve as the basis for a heavily modified
version of the gcSweepBuf data structure for the new mcentral
implementation.
We move it into a separate file now for two reasons:
1. We will need both implementations as they will coexist simultaneously
for a time.
2. By creating it now in a new change it'll make future changes which
modify it easier to review (rather than introducing the new file then).
Joel Sing [Mon, 30 Mar 2020 15:00:50 +0000 (02:00 +1100)]
cmd/compile: implement multi-control branches for riscv64
Implement multi-control branches for riscv64, switching to using the BNEZ
pseudo-instruction when rewriting conditionals. This will allow for further
branch optimisations to later be performed via rewrites.
Change-Id: I7f2c69f3c77494b403f26058c6bc8432d8070ad0
Reviewed-on: https://go-review.googlesource.com/c/go/+/226399 Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Joel Sing <joel@sing.id.au>
cmd/compile: eliminate some array equality alg loops
type T [3]string
Prior to this change, we generated this equality alg for T:
func eqT(p, q *T) (r bool) {
for i := range *p {
if len(p[i]) == len(q[i]) {
} else {
return
}
}
for j := range *p {
if runtime.memeq(p[j].ptr, q[j].ptr, len(p[j])) {
} else {
return
}
}
return true
}
That first loop can be profitably eliminated;
it's cheaper to spell out 3 length equality checks.
We now generate:
func eqT(p, q *T) (r bool) {
if len(p[0]) == len(q[0]) &&
len(p[1]) == len(q[1]) &&
len(p[2]) == len(q[2]) {
} else {
return
}
for i := 0; i < len(p); i++ {
if runtime.memeq(p[j].ptr, q[j].ptr, len(p[j])) {
} else {
return
}
}
return true
}
We now also eliminate loops for small float arrays as well,
and for any array of size 1.
These cutoffs were selected to minimize code size on amd64
at this moment, for lack of a more compelling methodology.
Any smallish number would do.
The switch from range loops to plain for loops allowed me
to use a temp instead of a named var, which eliminated
a pointless argument to checkAll.
The code to construct them is also a bit clearer, in my opinion.
cmd/compile: improve equality algs for arrays of strings
type T [8]string
Prior to this change, we generated this equality algorithm for T:
func eqT(p, q *T) (r bool) {
for i := range *p {
if p[i] == q[i] {
} else {
return
}
}
return true
}
This change splits this into two loops, so that we can do the
cheap (length) half early and only then do the expensive (contents) half.
We now generate:
func eqT(p, q *T) (r bool) {
for i := range *p {
if len(p[i]) == len(q[i]) {
} else {
return
}
}
for j := range *p {
if runtime.memeq(p[j].ptr, q[j].ptr, len(p[j])) {
} else {
return
}
}
return true
}
The generated code is typically ~17% larger because it contains
two loops instead of one. In the future, we might want to unroll
the first loop when the array is small.
cmd/compile: improve equality algs for arrays of interfaces
type T [8]interface{}
Prior to this change, we generated this equality algorithm for T:
func eqT(p, q *T) bool {
for i := range *p {
if p[i] != q[i] {
return false
}
}
return true
}
This change splits this into two loops, so that we can do the
cheap (type) half early and only then do the expensive (data) half.
We now generate:
func eqT(p, q *T) (r bool) {
for i := range *p {
if p[i].type == q[i].type {
} else {
return
}
}
for j := range *p {
if runtime.efaceeq(p[j].type, p[j].data, q[j].data) {
} else {
return
}
}
return true
}
The use of a named return value and a bare return is to work
around some typechecking problems that stymied me.
The structure of using equals and else (instead of not equals and then)
was for implementation convenience and clarity. As a bonus,
it generates slightly shorter code on AMD64, because zeroing a register
to return is cheaper than writing $1 to it.
The generated code is typically ~17% larger because it contains
two loops instead of one. In the future, we might want to unroll
the first loop when the array is small.
This change splits the two halves of the interface equality,
so that we can do the cheap (type) half early and the expensive
(data) half late. We now generate:
This change splits the two halves of the string equality,
so that we can do the cheap (length) half early and the expensive
(contents) half late. We now generate:
Daniel Martí [Sun, 26 Apr 2020 20:44:36 +0000 (21:44 +0100)]
cmd/compile: remove If type in rulegen
We only generate if statements via CondBreak, which is nice as the
control flow is simple and easy to work with. It seems like the If type
was added but never used, so remove it to avoid confusion.
We had a TODO about replacing CondBreak with If instead. I gave that a
try, but it doesn't seem worth the effort. The code gets more complex
and we don't really win anything in return.
While at it, don't use op strings as format strings in exprf. This
doesn't cause any issue at the moment, but it's best to be explicit
about the operator not containing any formatting verbs.
Change-Id: Ib59ad72d3628bf91594efc609e222232ad1e8748
Reviewed-on: https://go-review.googlesource.com/c/go/+/230257 Reviewed-by: Keith Randall <khr@golang.org>
Michael Munday [Thu, 23 Apr 2020 06:33:14 +0000 (23:33 -0700)]
cmd/compile: adopt strong aux typing for some s390x rules
Apply strong aux typing to lowering rules that do not require
modification beyond substituting -> for =>. Other lowering rules
and all the optimization rules will follow. I'm breaking it up
to allow toolstash-check to pass on the big CLs.
Passes toolstash-check -all.
Change-Id: I6f1340058a8eb5a1390411e59fcbea9d7f777e58
Reviewed-on: https://go-review.googlesource.com/c/go/+/229400
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Daniel Martí [Sat, 18 Apr 2020 16:34:00 +0000 (17:34 +0100)]
cmd/compile: minor rulegen simplifications
The commuteDepth variable is no longer necessary; remove it.
Else branches after a log.Fatal call are unnecessary.
Also make the unbalanced return an integer, so we can differentiate
positive from negative cases. We only want to continue a rule with the
following lines if this balance is positive, for example.
While at it, make the balance loop stop when it goes negative, to not
let ")(" seem balanced.
Tyson Andre [Sat, 25 Apr 2020 21:39:19 +0000 (21:39 +0000)]
net/http/httputil: fix typo in unit test name
Everywhere else is using "cancellation"
The reasoning is mentioned in 170060
> Though there is variation in the spelling of canceled,
> cancellation is always spelled with a double l.
>
> Reference: https://www.grammarly.com/blog/canceled-vs-cancelled/
Tyson Andre [Sat, 25 Apr 2020 20:53:42 +0000 (20:53 +0000)]
math/cmplx: fix typo in code comment
Everywhere else is using "cancellation" as of 2019
The reasoning is mentioned in 170060.
> Though there is variation in the spelling of canceled,
> cancellation is always spelled with a double l.
>
> Reference: https://www.grammarly.com/blog/canceled-vs-cancelled/
Ian Lance Taylor [Fri, 17 Apr 2020 22:42:12 +0000 (15:42 -0700)]
os, net: define and use os.ErrDeadlineExceeded
If an I/O operation fails because a deadline was exceeded,
return os.ErrDeadlineExceeded. We used to return poll.ErrTimeout,
an internal error, and told users to check the Timeout method.
However, there are other errors with a Timeout method that returns true,
notably syscall.ETIMEDOUT which is returned for a keep-alive timeout.
Checking errors.Is(err, os.ErrDeadlineExceeded) should permit code
to reliably tell why it failed.
This change does not affect the handling of net.Dialer.Deadline,
nor does it change the handling of net.DialContext when the context
deadline is exceeded. Those cases continue to return an error
reported as "i/o timeout" for which Timeout is true, but that error
is not os.ErrDeadlineExceeded.
Fixes #31449
Change-Id: I0323f42e944324c6f2578f00c3ac90c24fe81177
Reviewed-on: https://go-review.googlesource.com/c/go/+/228645
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org>
cmd/compile: optimize Move with all-zero ro sym src to Zero
We set up static symbols during walk that
we later make copies of to initialize local variables.
It is difficult to ascertain at that time exactly
when copying a symbol is profitable vs locally
initializing an autotmp.
During SSA, we are much better placed to optimize.
This change recognizes when we are copying from a
global readonly all-zero symbol and replaces it with
direct zeroing.
This often allows the all-zero symbol to be
deadcode eliminated at link time.
This is not ideal--it makes for large object files,
and longer link times--but it is the cleanest fix I could find.
This makes the final binary for the program in #38554
shrink from >500mb to ~2.2mb.
Ian Lance Taylor [Wed, 22 Apr 2020 23:03:30 +0000 (16:03 -0700)]
runtime: sleep in TestSegv program to let signal be delivered
Since we're sleeping rather than waiting for the goroutines,
let the goroutines run forever.
Fixes #38595
Change-Id: I4cd611fd7565f6e8d91e50c9273d91c514825314
Reviewed-on: https://go-review.googlesource.com/c/go/+/229484
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Hana (Hyang-Ah) Kim [Fri, 24 Apr 2020 20:57:07 +0000 (16:57 -0400)]
doc/go1.15: include changes in net/http/pprof and runtime/pprof
net/http/pprof: delta profile support
runtime/pprof: profile labels plumbing for goroutine profiles
Change-Id: I92e750dc894c8c6b3c3ba10f7be58bb541d3c289
Reviewed-on: https://go-review.googlesource.com/c/go/+/230023 Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/compile: add more non-ID comparisons to schedule
These comparisons are fairly arbitrary,
but they should be more stable in the face
of other compiler changes than value ID.
This reduces the number of value ID
comparisons in schedule while running
make.bash from 542,442 to 99,703.
There are lots of changes to generated code
from this change, but they appear to
be overall neutral.
It is possible to further reduce the
number of comparisons in schedule;
I have changes locally that reduce the
number to about 25,000 during make.bash.
However, the changes are increasingly
complex and arcane, and reduce in much less
code churn. Given that the goal is stability,
that suggests that this is a reasonable
place to stop, at least for now.
cmd/compile: add Value.Uses comparison during scheduling
Falling back to comparing Value.ID during scheduling
is undesirable: Not only are we simply hoping for a good
outcome, but the decision we make will be easily perturbed
by other compiler changes, leading to random fluctuations.
This change adds another decision point to the scheduler
by scheduling Values with many uses earlier.
Values with fewer uses are less likely to be spilled for
other reasons, so we should issue them as late as possible
in the hope of avoiding a spill.
This reduces the number of Value ID comparisons
in schedule while running make.bash
from 1,000,844 to 542,442.
As you would expect, this changes a lot of functions,
but the overall trend is positive:
go/types: improve error message for pointer receiver errors
The compiler produces high quality error messages when an interface is
implemented by *T, rather than T. This change improves the analogous
error messages in go/types, from "missing method X" to "missing method
X (X has pointer receiver)".
I am open to improving this message further - I didn't copy the compiler
error message exactly because, at one of the call sites of
(*check).missingMethod, we no longer have access to the name of the
interface.
Keep track of all expressions encountered while
generating a rewrite result, and re-use them whenever possible.
Named expressions may still be used for clarity when desired.
Ruixin(Peter) Bao [Tue, 24 Mar 2020 15:51:22 +0000 (11:51 -0400)]
math/big: rewrite subVW to use fast path on s390x
This CL replaces the original subVW implementation with a implementation
that uses a similar idea as CL 164968.
When we know the borrow bit is zero, we can copy the rest of words as
they will not be updated. Also, since we are copying vector of a words,
a faster implementation of copy is written in this CL to copy a word or
multiple words at a time.
Dmitri Shuralyov [Wed, 22 Apr 2020 23:31:13 +0000 (19:31 -0400)]
doc: remove The Go Project page (moved to x/website)
This page has moved to the x/website repo in CL 229482 (commit
golang/website@70f4ee8c7e31e1c90ae44d835ff6214dc0496205).
Remove the old copy in this repo since it's no longer used.
For #29206.
Change-Id: Ief093ed8c5dfec43e06d473e4282275f61da74a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/229485 Reviewed-by: Alexander Rakoczy <alex@golang.org>
In the dev.link branch we continued developing the new object
file format support and the linker improvements described in
https://golang.org/s/better-linker . Since the last merge, more
progress has been made to improve the new linker.
Ruixin(Peter) Bao [Tue, 3 Dec 2019 21:06:18 +0000 (16:06 -0500)]
math/big: rewrite addVW to use fast path on s390x
Rewrite addVW to use a fast path and remove the original
vector and non vector implementation of addVW in assembly. This CL uses
a similar idea as CL 164968, where we copy the rest of words when we
know carry bit is zero.
In addition, since we are copying vector of words, a faster
implementation of copy is written in this CL to copy a word or multiple
words at a time.
crypto/x509: add x509omitbundledroots build tag to not embed roots
On darwin/arm64, the copy of the system roots takes 256 KiB of disk
and 560 KiB of memory after parsing them (which is retained forever in
a package global by x509/root.go). In constrained environments like
iOS NetworkExtensions where total disk+RAM is capped at 15 MiB, these
certs take 5.3% of the total allowed memory.
It turns out you can get down from 816 KiB to 110 KiB by instead
storing compressed x509 certs in the binary and lazily inflating just
the needed certs at runtime as a function of the certs presented to
you by the server, then building a custom root CertPool in the
crypto/tls.Config.VerifyPeerCertificate hook.
This then saves 706 KiB.
Arguably that should be the default Go behavior, but involves
cooperation between x509 and tls, and adds a dependency to
compress/gzip. Also, it may not be the right trade-off for everybody,
as it involves burning more CPU on new TLS connections. Most iOS apps
don't run in a NetworkExtension context limiting them to 15 MiB.
The build tag is chosen to match the existing "nethttpomithttp2".
cmd/compile: fix misalignment in sources column of generated ssa.html
Fix regression where line numbers in the sources column of generated
ssa.html output became misaligned with the source code. This was due
to some new margins applied to certain h2 elements during the work
to combine identical columns.
David Chase [Thu, 23 Apr 2020 23:52:31 +0000 (19:52 -0400)]
cmd/internal/obj: add IsAsm flag
This allows more exciting changes to compiler-generated assembly
language that might not be correct for tricky hand-crafted
assembly (e.g., nop padding breaking tables of call or branch
instructions).
Matthew Dempsky [Wed, 22 Apr 2020 02:48:02 +0000 (19:48 -0700)]
cmd/compile: use fixVariadicCall in escape analysis
This CL uses fixVariadicCall before escape analyzing function calls.
This has a number of benefits, though also some minor obstacles:
Most notably, it allows us to remove ODDDARG along with the logic
involved in setting it up, manipulating EscHoles, and later copying
its escape analysis flags to the actual slice argument. Instead, we
uniformly handle all variadic calls the same way. (E.g., issue31573.go
is updated because now f() and f(nil...) are handled identically.)
It also allows us to simplify handling of builtins and generic
function calls. Previously handling of calls was hairy enough to
require multiple dispatches on n.Op, whereas now the logic is uniform
enough that we can easily handle it with a single dispatch.
The downside is handling //go:uintptrescapes is now somewhat clumsy.
(It used to be clumsy, but it still is, too.) The proper fix here is
probably to stop using escape analysis tags for //go:uintptrescapes
and unsafe-uintptr, and have an earlier pass responsible for them.
Finally, note that while we now call fixVariadicCall in Escape, we
still have to call it in Order, because we don't (yet) run Escape on
all compiler-generated functions. In particular, the generated "init"
function for initializing package-level variables can contain calls to
variadic functions and isn't escape analyzed.
Passes toolstash-check -race.
Change-Id: I4cdb92a393ac487910aeee58a5cb8c1500eef881
Reviewed-on: https://go-review.googlesource.com/c/go/+/229759
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Matthew Dempsky [Wed, 30 Nov 2016 01:27:15 +0000 (17:27 -0800)]
go/types: add UsesCgo config to support _cgo_gotypes.go
This CL adds a UsesCgo config setting to go/types to specify that the
_cgo_gotypes.go file generated by cmd/cgo has been provided as a
source file. The type checker then internally resolves C.bar qualified
identifiers to _Cfoo_bar as appropriate.
It also adds support to srcimporter to automatically run cgo.
Unfortunately, this functionality is not compatible with overriding
OpenFile, because cmd/cgo and gcc will directly open files.
Updates #16623.
Updates #35721.
Change-Id: I1e1965fe41b765b7a9da3431f2a86cc16025dee2
Reviewed-on: https://go-review.googlesource.com/c/go/+/33677
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
cmd/asm,cmd/internal/obj/ppc64: update instructions and tests
This change adds some instructions that were missing from the
ppc64 assembler, mostly power9 but a few others from earlier.
Tests in cmd/asm for ppc64 were updated: ppc64.s includes the
new instructions, and ppc64enc.s now includes not only the
new instructions but most ppc64 opcodes to provide a more
complete test of the ppc64 assembler.
The ppc64 instruction set is used for linux/ppc64le,
linux/ppc64, and aix/ppc64.