[dev.link] cmd/oldlink: decouple from goobj2 package
The new object file support in the old linker should not be used.
This is a minimal change that removes stuff from the old linker's
loader package, so that it decouples from the goobj2 package,
allowing the latter to evolve.
Keep the change local in the loader package, so most of the old
linker doesn't need to change. At this point I don't think we
want to make significant changes to the old linker.
Change-Id: I078c4cbb35dc4627c4b82f512a4aceec9b594925
Reviewed-on: https://go-review.googlesource.com/c/go/+/226800 Reviewed-by: Jeremy Faller <jeremy@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
[dev.link] cmd/internal/goobj2: change StringRef encoding for better locality
Previously, StringRef is encoded as an offset pointing to
{ len, [len]byte }. This CL changes it to { len, offset }, where
offset points the bytes.
With the new format, reading a string header is just reading two
adjacent uint32s, without accessing the string table. This should
improve locality of object file reading.
Change-Id: Iec30708f9d9adb2f0242db6c4767c0f8e730f4df
Reviewed-on: https://go-review.googlesource.com/c/go/+/226797
Run-TryBot: Cherry Zhang <cherryyz@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Cherry Zhang [Fri, 27 Mar 2020 20:32:22 +0000 (16:32 -0400)]
[dev.link] cmd/link: set attributes atomically
Now concurrent relocsym may access symbols attributes
concurrently, causing data race when using the race detector. I
think it is still safe as we read/write on different bits, and
not write the same symbol's attributes from multiple goroutines,
so it will always reads the right value regardless whether the
write happens before or after, as long as the memory model is not
so insane.
Use atomic accesses to appease the race detector. It doesn't seem
to cost much, at least on x86.
Jeremy Faller [Tue, 17 Mar 2020 14:24:40 +0000 (10:24 -0400)]
[dev.link] cmd/link: parallelize asmb on amd64
Introduces a parallel OutBuf implementation, and POC on amd64. Due to
some of the weird behaviors I saw on MacOS (SIGBUS while calling msync),
I will wait for feedback to port to other architectures.
On my mac, sped up Asmb by ~78% for cmd/compile (below). Will likely
have an appreciable speedup on kubelet benchmark.
Than McIntosh [Mon, 30 Mar 2020 14:06:54 +0000 (10:06 -0400)]
[dev.link] cmd/link: move the wavefront past addexport()
Reorganize the linker phase ordering so that addexport() runs before
loadlibfull. In previous CLs addexport() was changed to use loader
APIs but then copy back its work into sym.Symbol, so this change
removes the copying/shim code in question.
Than McIntosh [Mon, 30 Mar 2020 14:05:23 +0000 (10:05 -0400)]
[dev.link] cmd/link: run setArchSyms earlier
Add some new loader.Sym equivalents to the archSyms struct so that we
can run setArchSyms earlier in the pipeline, and add a "mode" to
setArchSyms to control whether it should create loader.Sym symbols or
their *sym.Symbol equivalents.
These change needed for a subsequent patch in which addexport() is run
earlier as well
Andy Pan [Fri, 27 Mar 2020 03:21:17 +0000 (03:21 +0000)]
runtime: converge duplicate calls to netpollBreak into one
There might be some concurrent (maybe not concurrent, just sequential but in a short time window) and duplicate calls to `netpollBreak`, trying to wake up a net-poller. If one has called `netpollBreak` and that waking event hasn't been received by epollwait/kevent/..., then the subsequent calls of `netpollBreak` ought to be ignored or in other words, these calls should be converged into one.
Lynn Boger [Thu, 26 Mar 2020 20:01:40 +0000 (16:01 -0400)]
cmd/compile: add rules to eliminate unnecessary signed shifts
This change to the rules removes some unnecessary signed shifts
that appear in the math/rand functions. Existing rules did not
cover some of the signed cases.
A little improvement seen in math/rand due to removing 1 of 2
instructions generated for Int31n, which is inlined quite a bit.
Bryan C. Mills [Fri, 27 Mar 2020 03:10:32 +0000 (23:10 -0400)]
cmd/go/internal/work: disallow testgo binary from installing to GOROOT
Installing to GOROOT makes tests non-parallelizable, since each test
depends on the installed contents of GOROOT already being up-to-date
and may reasonably assume that those contents do not change over the
course of the test.
Fixes #37573
Updates #30316
Change-Id: I2afe95ad11347bee3bb7c2d77a657db6d691cf05
Reviewed-on: https://go-review.googlesource.com/c/go/+/225897 Reviewed-by: Michael Matloob <matloob@golang.org>
Bryan C. Mills [Tue, 17 Mar 2020 14:22:39 +0000 (10:22 -0400)]
run.{bash,bat,rc}: use ../bin/go instead of the go binary in $PATH
https://golang.org/doc/contribute.html#quick_test currently suggests
running 'make.bash' and 'run.bash' separately, but 'run.bash'
potentially uses a 'go' command resolved from the wrong GOROOT,
which in turn sets the wrong GOROOT for further commands.
Bryan C. Mills [Thu, 26 Mar 2020 16:00:07 +0000 (12:00 -0400)]
cmd/dist: skip API check on plan9 builders
The plan9-arm builder has a very slow filesystem and frequently times
out on this test. The api check verifies the API for all supported
GOOS/GOARCH/CGO_ENABLED combination anyway, so if we skip it on one
builder (or even most builders) there should be no loss of coverage.
Updates #37951
Change-Id: I86a93df2ec60a6af6d942e3954eef09ce67bb39e
Reviewed-on: https://go-review.googlesource.com/c/go/+/225662 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Cherry Zhang [Thu, 26 Mar 2020 18:43:15 +0000 (14:43 -0400)]
[dev.link] cmd/link: migrate symbol's Value and Align when converting to sym.Symbols
Currently, in LoadFull we migrate a symbol's Value to sym.Symbol
only for external symbols. And symbol's Align is not migrated at
all. As we move LoadFull forward, there are already places where
we set symbol's Value and Align (e.g. in doelf). Migrate them
correctly.
Currently I think we only set them on external symbols, but as
we move forward I think we'll need to set them on Go symbols as
well.
Jay Conrod [Thu, 12 Mar 2020 22:56:40 +0000 (18:56 -0400)]
cmd/go: add support for GOPROXY fallback on unexpected errors
URLs in GOPROXY may now be separated with commas (,) or pipes (|). If
a request to a proxy fails with any error (including connection errors
and timeouts) and the proxy URL is followed by a pipe, the go command
will try the request with the next proxy in the list. If the proxy is
followed by a comma, the go command will only try the next proxy if
the error a 404 or 410 HTTP response.
The go command will determine how to connect to the checksum database
using the same logic. Before accessing the checksum database, the go
command sends a request to <proxyURL>/sumdb/<sumdb-name>/supported.
If a proxy responds with 404 or 410, or if any other error occurs and
the proxy URL in GOPROXY is followed by a pipe, the go command will
try the request with the next proxy. If all proxies respond with 404
or 410 or are configured to fall back on errors, the go command will
connect to the checksum database directly.
This CL does not change the default value or meaning of GOPROXY.
Fixes #37367
Change-Id: If53152ec1c3282c67d4909818b666af58884fb2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/223257
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Bryan C. Mills [Thu, 26 Mar 2020 02:17:04 +0000 (22:17 -0400)]
cmd/go/internal/base: rename EnvForDir to AppendPWD
EnvForDir does not immediately evoke “append”, and thus may not prompt
the reader to consider the possibility of aliasing bugs (as in
issue #38077). To make this behavior more obvious at the call site, rename
cmd/go/internal/base.EnvForDir to AppendPWD and swap the order of
arguments to a conventional “append” function (similar to those in the
strconv package).
For #38077
Change-Id: I16f09aa0fa8a269d51f0511eb402a44e2759eb94
Reviewed-on: https://go-review.googlesource.com/c/go/+/225578 Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Bryan C. Mills [Thu, 26 Mar 2020 02:24:44 +0000 (22:24 -0400)]
cmd/go: do not append to the global cfg.OrigEnv slice
Appending to a global slice is only safe if its length is already
equal to its capacity. That property is not guaranteed for slices in
general, and empirically does not hold for this one.
This is a minimal fix to make it easier to backport.
A more robust cleanup of the base.EnvForDir function will be sent in a
subsequent CL.
Fixes #38077
Updates #37940
Change-Id: I731d5bbd0e516642c2cf43e713eeea15402604e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/225577
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Michael Anthony Knyszek [Wed, 18 Mar 2020 15:09:40 +0000 (15:09 +0000)]
runtime: ensure minTriggerRatio never exceeds maxTriggerRatio
Currently, the capping logic for the GC trigger ratio is such that if
gcpercent is low, we may end up setting the trigger ratio far too high,
breaking the promise of SetGCPercent and GOGC has a trade-off knob (we
won't start a GC early enough, and we will use more memory).
This change modifies the capping logic for the trigger ratio by scaling
the minTriggerRatio with gcpercent the same way we scale
maxTriggerRatio.
This makes all modern public keys in the standard library implement a
common interface (below) that can be used by applications for better
type safety and allows for checking that public (and private keys via
Public()) are equivalent.
interface {
Equal(crypto.PublicKey) bool
}
Equality for ECDSA keys is complicated, we take a strict interpretation
that works for all secure applications (the ones not using the
unfortunate non-constant time CurveParams implementation) and fails
closed otherwise.
Tests in separate files to make them x_tests and avoid an import loop
with crypto/x509.
Re-landing of CL 223754. Dropped the test that was assuming named curves
are not implemented by CurveParams, because it's not true for all
curves, and anyway is not a property we need to test. There is still a
test to check that different curves make keys not Equal.
Cherry Zhang [Wed, 25 Mar 2020 16:26:32 +0000 (12:26 -0400)]
[dev.link] cmd/internal/goobj2: bump up version number
As we now have -go115newobj flag, it is better to use go115 in
the object file as well. And it already diverges from the go114
"new" object file format.
Cherry Zhang [Wed, 25 Mar 2020 21:10:16 +0000 (17:10 -0400)]
[dev.link] cmd/link: define _etext, etc. in the linker on Solaris
On Solaris, in the runtime it defines the external name of
runtime.etext as _etext (runtime/os3_solaris.go:13). In CL 224939
we changed to put external names in the ELF symbol table more
consistently. In this case it will contain _etext but not
runtime.etext.
To be conservative, this CL defines both runtime.etext and _text
in the linker.
Xiangdong Ji [Thu, 5 Dec 2019 03:22:34 +0000 (03:22 +0000)]
runtime: fix threshold calculation of TestPhysicalMemoryUtilization
Variable 'procs' used to calculate the threshold of overuse in
TestPhysicalMemoryUtilization should be updated if GOMAXPROCS
gets changed, otherwise the threshold could be a large number,
making the test meaningless.
Change-Id: I876cbf11457529f56bae77af1e35f4538a721f95
Reviewed-on: https://go-review.googlesource.com/c/go/+/210297
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Rob Pike [Wed, 25 Mar 2020 22:32:27 +0000 (09:32 +1100)]
scan: for style, adjust code for bad scan read counts
Make the code more consistent with the rest of the file.
Should have caught this in review of CL 225357.
Change-Id: I12824cb436539c31604684e043ebb7587cc92471
Reviewed-on: https://go-review.googlesource.com/c/go/+/225557 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Wed, 25 Mar 2020 16:41:20 +0000 (16:41 +0000)]
runtime: prevent preemption while timer is in timerModifying
Currently if a goroutine is preempted while owning a timer in the
timerModifying state, it could self-deadlock. When the goroutine is
preempted and calls into the scheduler, it could call checkTimers. If
checkTimers encounters the timerModifying timer and calls runtimer on
it, then runtimer will spin, waiting for that timer to leave the
timerModifying state, which it never will.
So far we got lucky that for the most part that there were no preemption
points while timerModifying is happening, however CL 221077 seems to
have introduced one, leading to sporadic self-deadlocks.
This change disables preemption explicitly while a goroutines holds a
timer in timerModifying. Since only checkTimers (and thus runtimer) is
called from the scheduler, this is sufficient to prevent
preemption-based self-deadlocks.
Fixes #38070.
Updates #37894.
Change-Id: Idbfac310889c92773023733ff7e2ff87e9896f0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/225497
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Fri, 20 Mar 2020 19:15:35 +0000 (15:15 -0400)]
cmd/api: make NewWatcher populate its own package and import metadata
This partially undoes the optimizations of CL 177597, but makes up
some of the difference by caching the package list and import metadata
and making the initial calls concurrently, including in TestMain.
That reduces the critical path from two sequential 'go list'
invocations to just one (run many times concurrently), and eliminates
the need for assumptions about the consistency of the 'std' dependency
graph across platforms (and hard-coded special cases for packages that
violate those assumptions).
In the process, this simplifies and fixes TestBenchmark (which has
been silently broken since CL 164623).
This increases 'time go tool dist test api' on my workstation from
0m8.4s / 0m13.8s / 0m1.7s to 0m10.5s / 0m23.1s / 0m5.1s,
compared to 0m12.4s / 0m23.2s / 0m4.7s before CL 177597.
(That is, this change retains about half of the wall-time speedup, but
almost none of the user-time speedup.)
Tested manually using 'go test -race -bench=. cmd/api'.
Fixes #37951
Change-Id: Icd537e035e725e1ee7c41d97da5c6651233b927e
Reviewed-on: https://go-review.googlesource.com/c/go/+/224619
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
fanzha02 [Wed, 25 Mar 2020 03:11:24 +0000 (11:11 +0800)]
cmd/link/internal/arm64: increase the function alignment to 16
On arm64, a function's address is 16 bytes aligned, and
the assembler aligns the size of function symbol to 16 bytes,
so to keep the consistent, this patch changes the function
alignment in the linker to 16 bytes.
cmd/internal/obj/arm64: add support of PCALIGN directive
Recently, we get requirements of instructions and functions alignment
from the gVisor project. To fit the alignment requirement of interrupt
table, they require an instruction's address to be aligned 128 bytes
and a function's entry address to be aligned 2K bytes. Thus we add
support for PCALIGN directive first. Below is a discussion about this
topic. https://groups.google.com/forum/m/#!topic/golang-dev/RPj90l5x86I
Functions in Go are aligned to 16 bytes on arm64, thus now we only
support 8 and 16 bytes alignment.
This patch adds support for PCALIGN directive. This directive can be
used within Go asm to align instruction by padding NOOP directives.
This patch also adds a test to verify the correnctness of the PCALIGN
directive. The test is contributed by Fannie Zhang <Fannie.Zhang@arm.com>.
Bryan C. Mills [Tue, 24 Mar 2020 20:18:02 +0000 (16:18 -0400)]
test: make runindir tests pass regardless of whether module mode is in use
The "runindir" tests used "go run", but relied on relative imports
(which are not supported by "go run" in module mode). Instead, such
tests must use fully-qualified imports, which require either a go.mod
file (in module mode) or that the package be in an appropriate
subdirectory of GOPATH/src (in GOPATH mode).
To set up such a directory, we use yet another copy of the same
overlayDir function currently found in the misc subdirectory of this
repository.
Fixes #33912
Updates #30228
Change-Id: If3d7ea2f7942ba496d98aaaf24a90bcdcf4df9f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/225205 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Than McIntosh [Tue, 24 Mar 2020 13:05:19 +0000 (09:05 -0400)]
[dev.link] cmd/link: convert addexport to loader interfaces
Convert the linker's "addexport" function to use loader interfaces
for symbol manipulation instead of *sym.Symbol.
At the moment "addexport" happens after loadlibfull (there are other
chunks of functionality in the way that haven't been converted), so
this implementation contains temporary shim code to copy back the
contents of updated loader.Sym's into the corresponding sym.Symbol.
Than McIntosh [Fri, 20 Mar 2020 12:31:38 +0000 (08:31 -0400)]
[dev.link] cmd/link: move setupdynexp before loadlibfull
Rewrite setupdynexp to work with loader.Sym, and move the call to it
before the body of loadlibfull. After loadlibfull is complete,
construct the old *sym.Symbol version of dynexp, since not all all
clients that access this list are converted to the loader APIs.
Change-Id: I347d24958e2f3e2332fbe33f2eb6ec25cc126bdb
Reviewed-on: https://go-review.googlesource.com/c/go/+/224382 Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Than McIntosh [Thu, 19 Mar 2020 15:04:37 +0000 (11:04 -0400)]
[dev.link] cmd/link: minor tweaks to PropagateLoaderChangesToSymbols
Update PropagateLoaderChangesToSymbols so that it no longer requires
a sym.Symbols pointer. The intent is to generalize it a little to
allow it to be used in more than just linker Dwarf generation.
Than McIntosh [Thu, 19 Mar 2020 12:17:54 +0000 (08:17 -0400)]
[dev.link] cmd/link: relocating Errorf() to ErrorReporter
Add an Errorf method to ErrorReporter. The hope is that we can
consolidate error handling/reporting in this helper, and eventually
do away with Link.Errorf and the global Errorf function (which
can be removed once we've eliminated enough uses of *sym.Symbol).
Change-Id: Ie1147020b8409b9c57acfd71c942b287b214afca
Reviewed-on: https://go-review.googlesource.com/c/go/+/224380 Reviewed-by: Jeremy Faller <jeremy@golang.org>
Joel Sing [Mon, 9 Mar 2020 16:31:22 +0000 (03:31 +1100)]
cmd/compile: fold constants into immediate instructions on riscv64
Where possible, fold constants into versions of instructions that take
an immediate. This avoids the need to allocate a register and load the
immediate into it.
Change-Id: If911ca41235e218490679aed2ce5f48bf807a2b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/222639
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Lynn Boger [Tue, 17 Mar 2020 13:24:47 +0000 (09:24 -0400)]
cmd/internal/obj/ppc64: fix PCALIGN on ppc64le
This fixes a potential issue with the previous implementation
of PCALIGN on ppc64. Previously PCALIGN was processed inside of
asmout and indicated the padding size by setting the value in
the optab, changing it back after the alignment instructions
were added. Now PCALIGN is processed outside of asmout, and optab
is not changed.
Ian Lance Taylor [Tue, 24 Mar 2020 04:11:01 +0000 (21:11 -0700)]
internal/poll: assume we have CancelIoEX on Windows
As of the Go 1.11 release we require at least Windows 7, so CancelIoEx
is always available. This lets us simplify the code to not require
dedicated threads to handle I/O requests.
Fixes #37956
Change-Id: If1dc4ac4acb61c43e4f2a9f26f225869050262a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/225060
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Dan Scales [Fri, 20 Mar 2020 16:31:20 +0000 (09:31 -0700)]
runtime: fix code so defer record is not added to g0 defer list during panic
newdefer() actually adds the new defer to the current g's defer chain. That
happens even if we are on the system stack, in which case the g will be the g0
stack. For open-coded defers, we call newdefer() (only during panic processing)
while on the system stack, so the new defer is unintentionally added to the
g0._defer defer list. The code later correctly adds the defer to the user g's
defer list.
The g0._defer list is never used. However, that pointer on the g0._defer list can
keep a defer struct alive that is intended to be garbage-collected (smaller defers
use a defer pool, but larger-sized defer records are just GC'ed). freedefer() does
not zero out pointers when it intends that a defer become garbage-collected. So,
we can have the pointers in a defer that is held alive by g0._defer become invalid
(in particular d.link). This is the cause of the bad pointer bug in this issue
The fix is to change newdefer (only used in two places) to not add the new defer
to the gp._defer list. We just do it after the call with the correct gp pointer.
(As mentioned above, this code was already there after the newdefer in
addOneOpenDeferFrame.) That ensures that defers will be correctly
garbage-collected and eliminate the bad pointer.
This fix definitely fixes the original repro. I added a test and tried hard to
reproduce the bug (based on the original repro code), but awasn't actually able to
cause the bug. However, the test is still an interesting mix of heap-allocated,
stack-allocated, and open-coded defers.
Fixes #37688
Change-Id: I1a481b9d9e9b9ba4e8726ef718a1f4512a2d6faf
Reviewed-on: https://go-review.googlesource.com/c/go/+/224581
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Ian Lance Taylor [Tue, 24 Mar 2020 03:42:29 +0000 (20:42 -0700)]
runtime: always use GetQueuedCompletionStatusEx on Windows
We used to fall back to GetQueuedCompletionStatus if
GetQueuedCompletionStatus was not available, but as of Go 1.11 we
require Windows 7 or later, so GetQueuedCompletionStatusEx is always
available.
Fixes #37957
Change-Id: I7d8d49a92ab7b1f5afdc54a442f696aaf4a5168e
Reviewed-on: https://go-review.googlesource.com/c/go/+/225059
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Cherry Zhang [Sat, 21 Mar 2020 20:06:57 +0000 (16:06 -0400)]
[dev.link] cmd/link: use Extname in doxcoff
On AIX, when external linking, we need to change the function
names to start with a dot and make function descriptors with the
names without the dot. Currently this is done through symbol
renaming, which is not friendly for switching to the loader.
In this CL we use symbol's external name for this. This allows us
to get rid of symbol renaming.
Cherry Zhang [Sat, 21 Mar 2020 17:50:46 +0000 (13:50 -0400)]
[dev.link] cmd/link: convert mangleTypeSym to new style
Use symbol's Extname, instead of symbol renaming, for the mangled
names.
The old symbol Rename has an interesting logic of "merging"
symbols, when a symbol is renamed to the name of an existing
symbol. It turns out that this is needed for linking against
shared libraries, where the Go object has a reference to a symbol
with the original name, and the shared libary provides a symbol
under the mangled name. Implement this logic with the loader.
Jeremy Faller [Mon, 9 Mar 2020 15:08:04 +0000 (11:08 -0400)]
[dev.link] cmd/link: remove global datap, moved to *Link
This change moves datap from global space into the link context. Rather
than having it exist in context, we could have it returned from dodata,
and pass it as a parameter, but it is used in awkward places in the
Arch functions. Easiest for now is just keeping it in the context, until
we more formally move it to slices of loader.Sym.
nao20010128nao [Sat, 21 Mar 2020 06:52:58 +0000 (06:52 +0000)]
syscall/js: make wasm_exec.js compatible with Webpack
In Webpack, require("fs") will always be empty. This behavior throws an error: "fs.writeSync is not function". It happens when you did "fmt.Println".
This PR avoids such problem and use polyfill in wasm_exec.js on Webpack.
Dan Scales [Wed, 11 Mar 2020 19:08:03 +0000 (12:08 -0700)]
cmd/go: define a build tag for any GOEXPERIMENT which is enabled.
For each experiment that has been enabled in the toolchain, define a build tag
with the same name (but prefixed by "goexperiment.") which can be used for
compiling alternative files for the experiment. This allows changes for the
experiment, like extra struct fields in the runtime, without affecting the base
non-experiment code at all.
I use this capability in my CL for static lock ranking
(https://go-review.googlesource.com/c/go/+/207619), so that static lock ranking
can be fully enabled as a GOEXPERIMENT, but there is no overhead in the runtime
when the experiment is not enabled.
I added a test in cmd/go/testdata/scripts to make sure the build tags are being
defined properly. In order to implement the test, I needed to provide environment
variable GOEXPSTRING to the test scripts (with its value set from
objabi.Expstring(), so that it can determine the experiments baked into the
toolchain.
I filed https://github.com/golang/go/issues/37937 to make a builder with
GOEXPERIMENT set to 'staticlockranking'. This builder will ensure another variant
of GOEXPERIMENT is being tested regularly for this change, as well as checking
static lock ranking in the runtime.
Change-Id: Ieb4b86107238febd105558c1e639d30cfe57ab5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/222925
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
This makes all modern public keys in the standard library implement a
common interface (below) that can be used by applications for better
type safety and allows for checking that public (and private keys via
Public()) are equivalent.
interface {
Equal(crypto.PublicKey) bool
}
Equality for ECDSA keys is complicated, we take a strict interpretation
that works for all secure applications (the ones not using the
unfortunate non-constant time CurveParams implementation) and fails
closed otherwise.
Tests in separate files to make them x_tests and avoid an import loop
with crypto/x509.
Giovanni Bajo [Mon, 23 Mar 2020 08:21:44 +0000 (09:21 +0100)]
cmd/compile: fold LEAQ with constant scale into LEA
Discovered this after rebasing CL196679 (use poset bounds in prove).
Some tests fail with that CL applied:
codegen/smallintiface.go:11: linux/amd64/: opcode not found: "^LEAQ\\truntime.staticuint64s\\+8\\(SB\\)"
codegen/smallintiface.go:16: linux/amd64/: opcode not found: "^LEAQ\\truntime.staticuint64s\\+2024\\(SB\\)"
codegen/smallintiface.go:21: linux/amd64/: opcode not found: "^LEAQ\\truntime.staticuint64s\\+24\\(SB\\)"
The only difference in prove SSA dumps is that a single Lsh64x64
op with constant shift (<< 3) is marked as bounded. This triggers
a different rule matching sequence in lower, which manages to generate
worse code for the above testcases.
This CL fixes the above test after CL196679 is applied. Right now,
these rules never trigger (this CL passes toolstash -cmp), so I can't
write a test.
Change-Id: I353f1c79c1875cac1da82cd8afa1e05e42684f1c
Reviewed-on: https://go-review.googlesource.com/c/go/+/224877
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Roland Shoemaker [Fri, 20 Mar 2020 20:28:43 +0000 (20:28 +0000)]
crypto/x509: add RevocationList and CreateRevocationList
The existing Certificate.CreateCRL method generates non-conformant CRLs and
as such cannot be used for implementations that require standards
compliance. This change implements a new top level method, CreateCRL, which
generates compliant CRLs, and offers an extensible API if any
extensions/fields need to be supported in the future.
Here is an example Issuer/CRL generated using this change:
-----BEGIN CERTIFICATE-----
MIIBNjCB3aADAgECAgEWMAoGCCqGSM49BAMCMBIxEDAOBgNVBAMTB3Rlc3Rpbmcw
IhgPMDAwMTAxMDEwMDAwMDBaGA8wMDAxMDEwMTAwMDAwMFowEjEQMA4GA1UEAxMH
dGVzdGluZzBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABLHrudbSM36sn1VBrmm/
OfQTyEsI4tIUV1VmneOKHL9ENBGCiec4GhQm2SGnDT/sZy2bB3c3yozh/roS6cZJ
UZqjIDAeMA4GA1UdDwEB/wQEAwIBAjAMBgNVHQ4EBQQDAQIDMAoGCCqGSM49BAMC
A0gAMEUCIQCoAYN6CGZPgd5Sw5a1rd5VexciT5MCxTfXj+ZfJNfoiAIgQVCTB8AE
Nm2xset7+HOgtQYlKNw/rGd8cFcv5Y9aUzo=
-----END CERTIFICATE-----
-----BEGIN X509 CRL-----
MIHWMH0CAQEwCgYIKoZIzj0EAwIwEjEQMA4GA1UEAxMHdGVzdGluZxgPMDAwMTAx
MDIwMDAwMDBaGA8wMDAxMDEwMzAwMDAwMFowFjAUAgECGA8wMDAxMDEwMTAxMDAw
MFqgHjAcMA4GA1UdIwQHMAWAAwECAzAKBgNVHRQEAwIBBTAKBggqhkjOPQQDAgNJ
ADBGAiEAjqfj/IG4ys5WkjrbTNpDbr+saHGO/NujLJotlLL9KzgCIQDm8VZPzj0f
NYEQgAW4nsiUzlvEUCoHMw0141VCZXv67A==
-----END X509 CRL-----
Michael Anthony Knyszek [Mon, 27 Jan 2020 20:41:38 +0000 (20:41 +0000)]
runtime: use OnesCount64 to count allocated objects in a span
This change modifies the implementation of (*mspan).countAlloc by
using OnesCount64 (which on many systems is intrinsified). It does so by
using an unsafe pointer cast, but in this case we don't care about
endianness because we're just counting bits set.
This change means we no longer need the popcnt table which was redundant
in the runtime anyway. We can also simplify the logic here significantly
by observing that mark bits allocations are always 8-byte aligned, so we
don't need to handle any edge-cases due to the fact that OnesCount64
operates on 64 bits at a time: all irrelevant bits will be zero.
Overall, this implementation is significantly faster than the old one on
amd64, and should be similarly faster (or better!) on other systems
which support the intrinsic. On systems which do not, it should be
roughly the same performance because OnesCount64 is implemented using a
table in the general case.
Michael Anthony Knyszek [Wed, 18 Mar 2020 18:46:04 +0000 (18:46 +0000)]
runtime: add countAlloc benchmark
This change adds a small microbenchmark for (*mspan).countAlloc, which
we're about to replace. Admittedly this isn't a critical piece of code,
but the benchmark was useful in understanding the performance change.
Michael Anthony Knyszek [Fri, 20 Mar 2020 19:08:13 +0000 (19:08 +0000)]
cmd/compile: panic if trying to alias an intrinsic with no definitions
Currently if we try to alias an intrinsic which hasn't been defined for
any architecture (such as by accidentally creating the alias before the
intrinsic is created with addF), then we'll just silently not apply any
intrinsics to those aliases.
Catch this particular case by panicking in alias if we try to apply the
alias and it did nothing.
Change-Id: I98e75fc3f7206b08fc9267cedb8db3e109ec4f5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/224637
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Michael Anthony Knyszek [Fri, 20 Mar 2020 15:26:46 +0000 (15:26 +0000)]
cmd/compile: declare runtime bit func aliases after math/bits intrinsics
Currently runtime/internal/sys bit-manipulation functions are aliased to
math/bits functions, which are intrinsified. Unfortunately these aliases
are declared before the intrinsified versions are generated, resulting
in the generic version of the code being copied over.
This change moves the aliases for bit operations in runtime/internal/sys
after the addF calls to generate those intrinsics in SSA, so that the
intrinsified SSA representation of those functions actually get copied
over.
This should improve the overall performance of the runtime (especially
the page allocator) since these bit operations will actually be
intrinsified now.
Than McIntosh [Thu, 19 Mar 2020 16:55:43 +0000 (12:55 -0400)]
[dev.link] cmd/link: set "local" attribute on symbols during preload
Set the "local" attribute on object file loader symbols during preload,
as opposed to the current strategy of only propagating the local attr
when we convert to sym.Symbols.
Change-Id: I18965e2fdaa9a0b9315d95466d17d9f41aa40a45
Reviewed-on: https://go-review.googlesource.com/c/go/+/224379
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jeremy Faller <jeremy@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Cherry Zhang [Fri, 20 Mar 2020 21:02:47 +0000 (17:02 -0400)]
[dev.link] cmd/link: invoke oldlink if old object format is chosen
Now we can choose the old object file format by setting
-gcflags=all=-go115newobj=false -asmflags=all=-go115newobj=false -ldflags=all=-go115newobj=false
Tested that setting all three to default false and it still works.
Change-Id: I9514b62a676916cc383b8afa389489fe7b8fa2bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/224625 Reviewed-by: Than McIntosh <thanm@google.com>