Ian Lance Taylor [Fri, 26 Apr 2019 13:56:58 +0000 (06:56 -0700)]
runtime: test for cgo build tag in TestGdbPythonCgo
Testing whether cgo is enabled in go/build is not the same as testing
whether the go tool supports cgo. They differ, for example, when using
GOARCH=386 on an amd64 system, as for a cross-build cgo is disabled by default.
Change-Id: Ib59106c92a3131b73ac6a91c0f7658a1769acf73
Reviewed-on: https://go-review.googlesource.com/c/go/+/174098
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
AIX doesn't allow to mmap an address range which is already mmap.
Therefore, once the region has been allocated, it must munmap before
being able to play with it.
LE Manh Cuong [Wed, 30 Jan 2019 05:38:15 +0000 (12:38 +0700)]
time: fix misleading error with the leading zero format
When the leading zero format is used, we currently don't handle the
month and year properly.
For the month, we were reporting an out of range error when getnum
returns zero of its own, as it also returns the month 0. That's
confusing, so only check the range when getnum returns a nil error.
For the year, we don't restore the value when parsing error occurs. For
example, with the incorrect input "111-01", "01" will be used to report
an error. So restore the value when an error occurs fix the problem.
Fixes #29918
Fixes #29916
Change-Id: I3145f8c46813a0457766b7c302482e6b56f94ed6
Reviewed-on: https://go-review.googlesource.com/c/go/+/160338
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Expand "go version" from printing just the version of the
go command itself to being able to print the version of any
go binary (when listed on the command line).
This is a simplified version of rsc.io/goversion, which will
now be deprecated in favor of "go version".
(Preparatory changes to runtime, cmd/go, and cmd/link
made this version information easier to find, allowing a
much simpler implementation than in rsc.io/goversion.)
It is useful to be able to dig the Go version out of a binary,
even a stripped binary. rsc.io/goversion does this for x86
binaries by disassembling the binary to find runtime/proc.go's
startup-time reference to runtime.buildVersion's address.
That approach is quite fragile: the implementation doesn't
work for non-x86 and must be updated as the generated
code changes.
rsc.io/goversion finds the module version string by looking
for random 16-byte framing around the actual string.
This is less fragile but fairly kludgy and requires scanning
the entire data segment.
cmd/buildid finds the build ID by looking for an ELF note
or else falling back to scanning the beginning of the text
segment for a magic string. This has proved quite reliable
and doesn't require scanning much of the binary.
This CL makes it possible to find the Go and module versions
using a scan more like the build ID scan: a symbol early in
the writable data segment starts with a magic number and
then has pointers to the two string variables.
Than McIntosh [Thu, 25 Apr 2019 14:35:47 +0000 (10:35 -0400)]
cmd/link: use read-only mmap to back selected symbol name strings
When reading symbol names from an object file, if a name does not
need fixup (conversion of "". to package path), then generate
a string whose backing store is in read-only memory (from an mmap
of the object file), avoiding the need for an allocation. This
yields a modest reduction in total linker heap use.
runtime: switch to P 0 before destroying current P
Ps are strictly numbered from 0 to GOMAXPROCS-1, so if procresize
happens to be running on a P that's being destroyed, it moves itself
to P 0.
However, currently procresize destroys the unused Ps *before* moving
itself to P 0. This means it may briefly run on a destroyed P. This is
basically harmless, but has at least one very confusing consequence:
since destroying a P has write barriers, it may enqueue pointers to a
destroyed write barrier buffer. As far as I can tell, there are no
negative consequences of this, but this seems really fragile.
This CL swaps the order of things, so now procresize moves itself to P
0 if necessary before destroying Ps. This ensures it always has a
valid P.
This is part of refactoring for #10958 and #24543, but is a good
cleanup regardless.
cmd/link: avoid writing to read-only memory in addstrdata
When the linker's -X flag is used, it will overwrite the symbol's
content (sym.P) in addstrdata. The symbol's content may be in
read-only memory, in which case overwriting it will fault. Do
copy-on-write to fix this.
Following discussion on golang.org/issue/26334, this CL changes
the GOPROXY environment setting to be a list of proxies, tried in
sequence. The first successful or non-404/410 error is taken as
authoritative. Otherwise the next proxy is tried, and so on.
As in earlier releases, GOPROXY=direct means "connect directly",
but now it can appear in a longer list as well.
This will let companies run a proxy holding only their private modules
and let users set GOPROXY=thatproxy,publicproxy or GOPROXY=thatproxy,direct
to fall back to an alternate mechanism for fetching public modules.
- rename PasswordRedacted to Redacted
- move URL into Response in redacted form, remove from Get result list
- add Response.Err to construct non-200 errors
(otherwise GetBytes is not just a wrapper)
- make 404/410 errors satisfy Is(err, os.ErrNotExist)
David du Colombier [Thu, 25 Apr 2019 07:01:11 +0000 (09:01 +0200)]
net: fix lookupHost to return DNSError on Plan 9
CL 168597 added IsNotFound field to DNSError.
However, this change broke TestLookupNonLDH on Plan 9
because LookupHost is expected to return a DNSError,
while on Plan 9, it returned an error string.
This change fixes the implementation of lookupHost on
Plan 9 to return a DNSError instead of an error string.
Fixes #31672.
Change-Id: Ia805c8965af63ddee7ccfdebb9462a5502b0269d
Reviewed-on: https://go-review.googlesource.com/c/go/+/173857
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Brian Kessler [Wed, 24 Apr 2019 04:04:38 +0000 (22:04 -0600)]
cmd/compile: add signed divisibility by power of 2 rules
For powers of two (c=1<<k), the divisibility check x%c == 0 can be made
just by checking the trailing zeroes via a mask x&(c-1) == 0 even for signed
integers. This avoids division fix-ups when just divisibility check is needed.
To apply this rule, we match on the fixed-up version of the division. This is
neccessary because the mod and division rewrite rules are already applied
during the initial opt pass.
The speed up on amd64 due to elimination of unneccessary fix-up code is ~55%:
unicode/utf8: remove some bounds checks from DecodeRune
The compiler couldn't quite see that reading p[2] and p[3] was safe.
This change provides a few hints to help it.
First, make sz an int throughout, rather than just when checking the input length.
Second, use <= instead of == in later comparisons.
name old time/op new time/op delta
DecodeASCIIRune-8 2.62ns ± 3% 2.60ns ± 5% ~ (p=0.126 n=18+19)
DecodeJapaneseRune-8 4.46ns ±10% 4.01ns ± 5% -10.00% (p=0.000 n=19+20)
We were using hex literals and had the binary literal in a comment.
When I was working with this code, I always referred to the comment.
That's an indicator that we should just use the binary literal directly.
Than McIntosh [Wed, 24 Apr 2019 12:27:04 +0000 (08:27 -0400)]
test: new test for issue 31637
This pair of packages caused a crash in gollvm, due to a glitch in the
way the front end handles empty/non-name parameters for functions that
are inline candidates.
Updates #31637.
Change-Id: I571c0658a00974dd36025e571638c0c836a3cdfd
Reviewed-on: https://go-review.googlesource.com/c/go/+/173617
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Than McIntosh [Tue, 23 Apr 2019 16:38:36 +0000 (12:38 -0400)]
go/internal/gccgoimporter: revise previous anonymous field fix.
Revise the fix for #31540 to take into account the possibility that we
may see an alias to a name that has already been installed into
package scope. This scenario is not possible to reproduce with the
current importer unit tests; changes to the harness to enable this
scenario will be added in a later CL.
Updates #31540.
Change-Id: Ie155d5e0b998604177a78471cba2413f57d40229
Reviewed-on: https://go-review.googlesource.com/c/go/+/173440 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joel Sing [Wed, 24 Apr 2019 13:04:23 +0000 (23:04 +1000)]
cmd/dist: handle arm64 as a machine hardware name
OpenBSD/arm64 reports itself as arm64 from `uname -m` - this currently
matches as gohostarch=arm, rather than gohostarch=arm64. Correct this
by matching on both aarch64 and arm64 (the alternative would be to use
`uname -p`, however that's likely to cause upset elsewhere).
Marcel van Lohuizen [Wed, 27 Mar 2019 12:27:01 +0000 (13:27 +0100)]
unicode: update table using new generator in x/text
The changes in Unicode 11 exposes a bug in maketables.go.
We update the Unicode 10 tables using a new generator
to minimize the changes upgrading to Unicode 11.
This change switches over the generation from core to that in
x/text. To properly update the tables one needs to run the generate
in x/text anyway, so this makes that official.
The RangeTable generator in x/text also generates slightly compacter
tables.
"go build -trimpath" trims the recorded file paths in the
resulting packages and executables to avoid recording
the names of any local directories. Instead, the files appear
to be stored in directories named either "go/src/..." (for the
standard library) or named after the module or package
in which the files appear.
Marcel van Lohuizen [Wed, 27 Mar 2019 16:52:25 +0000 (17:52 +0100)]
unicode: remove script test
The script test requires a manual update on each new
Unicode release, which interupts the automated flow.
The test is removed in favor of one that fits within the
automated scripts.
See https://go-review.googlesource.com/c/text/+/169638.
This CL affects the low-level -trimpath flag provided
by both cmd/asm and cmd/compile. Previously, the flag
took the name of a single directory that would be trimmed
from recorded paths in the resulting object file.
This CL makes the flag take a semicolon-separated list of paths.
Further, each path can now end in an optional "=>replacement"
to specify what to replace that leading path prefix with,
instead of only dropping it.
A followup CL will add a mode to cmd/go that uses this
richer -trimpath to build binaries that do not contain any
local path names.
This avoids bounds checks in the calling code.
The nominal increased size of the array in the binary
is compensated for by the decreased size of the functions that call it.
The benchmark changes are a bit scattered, but overall positive.
The cmd/go/internal/web package was forked in order to support direct
HTTPS fetches from widely-used hosting providers,¹ but direct fetches
were subsequently dropped in CL 107657. The forked web2 package, with
its GitHub-specific diagnostics and .netrc support, remained in use
for module proxy support, but was not used for the initial '?go-get=1'
path resolution, so the .netrc file was only used to fetch from
already-resolved module protocol servers.
This CL moves the .netrc support into its own (new) package,
cmd/go/internal/auth, and consolidates the web and web2 packages back
into just web. As a result, fetches via the web package now support
.netrc, and fetches that previously used web2 now enforce the same
security policies as web (such as prohibiting HTTPS-to-HTTP
redirects).
David Chase [Tue, 2 Apr 2019 21:26:49 +0000 (17:26 -0400)]
cmd/link: revert/revise CL 98075 because LLDB is very picky now
This was originally
Revert "cmd/link: fix up debug_range for dsymutil (revert CL 72371)"
which has the effect of no longer using Base Address Selection
Entries in DWARF. However, the build-time costs of that are
about 2%, so instead the hacky fixup that generated technically
incorrect DWARF was removed from the linker, and the choice
is instead made in the compiler, dependent on platform, but
also under control of a flag so that we can report this bug
against LLDB/dsymutil/dwarfdump (really, the LLVM dwarf
libraries).
This however does not solve #31188; debugging still fails,
but dwarfdump no longer complains. There are at least two
LLDB bugs involved, and this change will at allow us
to report them without them being rejected because our
now-obsolete workaround for the first bug creates
not-quite-DWARF.
Brian Kessler [Mon, 18 Mar 2019 05:11:00 +0000 (23:11 -0600)]
cmd/compile: add signed divisibility by power of 2 rules
For powers of two (c=1<<k), the divisibility check x%c == 0 can be made
just by checking the trailing zeroes via a mask x&(c-1)==0 even for signed
integers. This avoids division fixups when just divisibility check is needed.
To apply this rule the generic divisibility rule for A%B = A-(A/B*B) is disabled
on the "opt" pass, but this does not affect generated code as this rule is applied
later.
The speed up on amd64 due to elimination of unneccessary fixup code is ~55%:
Keith Randall [Mon, 1 Apr 2019 19:22:22 +0000 (12:22 -0700)]
runtime: randomize package initialization order in race mode
This is one small step to force people to not depend on the order of
initialization of packages which are not explicitly ordered by import
directives. Similar to randomizing map iteration order, this makes
sure people aren't depending on the behavior of the current release,
so that we can change the order in future releases without breaking
everyone.
Maybe one day we can randomize always, but for now we do it just in
race mode. (We would need to measure the impact on startup time before
we enabled it always.)
Keith Randall [Tue, 23 Apr 2019 01:21:37 +0000 (18:21 -0700)]
cmd/compile: always mark atColumn1 results as statements
In 31618, we end up comparing the is-stmt-ness of positions
to repurpose real instructions as inline marks. If the is-stmt-ness
doesn't match, we end up not being able to remove the inline mark.
Always use statement-full positions to do the matching, so we
always find a match if there is one.
Also always use positions that are statements for inline marks.
Fixes #31618
Change-Id: Idaf39bdb32fa45238d5cd52973cadf4504f947d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/173324
Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com>
Michael Munday [Mon, 18 Sep 2017 15:41:13 +0000 (16:41 +0100)]
math: consolidate assembly stub implementations
Where assembly functions are just jumps to the Go implementation
put them into a stubs_<arch>.s file. This reduces the number of
files considerably and makes it easier to see what is really
implemented in assembly.
I've also run the stubs files through asmfmt to format them in
a more consistent way.
Eventually we should replace these 'stub' assembly files with
a pure Go implementation now that we have mid-stack inlining
(see #31362).
Nikhil Benesch [Sat, 2 Feb 2019 17:35:44 +0000 (12:35 -0500)]
cmd/go: include AR env var in gccgo build IDs
The gccgo toolchain uses the archiver specified by the AR environment
variable, or `ar` by default. Teach the build ID to take the value of
this environment variable into account, since different archivers can
produce different results.
cmd/go: move runtime/debug.modinfo to runtime.modinfo
It is easier to ensure that the symbol is always present
if we move it to package runtime. Avoids init-time work.
Also moves it next to buildVersion, the other similar symbol.
Shubham Sharma [Thu, 21 Mar 2019 15:40:12 +0000 (21:10 +0530)]
net: add IsNotFound field to DNSError
This adds the ability to determine if a lookup error was
due to a non-existent hostname. Previously users needed
to do string matching on the DNSError.Err value.
Daniel Martí [Mon, 22 Apr 2019 16:36:43 +0000 (23:36 +0700)]
encoding/json: index names for the struct decoder
In the common case, structs have a handful of fields and most inputs
match struct field names exactly.
The previous code would do a linear search over the fields, stopping at
the first exact match, and otherwise using the first case insensitive
match.
This is unfortunate, because it means that for the common case, we'd do
a linear search with bytes.Equal. Even for structs with only two or
three fields, that is pretty wasteful.
Worse even, up until the exact match was found via the linear search,
all previous fields would run their equalFold functions, which aren't
cheap even in the simple case.
Instead, cache a map along with the field list that indexes the fields
by their name. This way, a case sensitive field search doesn't involve a
linear search, nor does it involve any equalFold func calls.
This patch should also slightly speed up cases where there's a case
insensitive match but not a case sensitive one, as then we'd avoid
calling bytes.Equal on all the fields. Though that's not a common case,
and there are no benchmarks for it.
name old time/op new time/op delta
CodeDecoder-8 11.0ms ± 0% 10.6ms ± 1% -4.42% (p=0.000 n=9+10)
name old speed new speed delta
CodeDecoder-8 176MB/s ± 0% 184MB/s ± 1% +4.62% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
CodeDecoder-8 2.28MB ± 0% 2.28MB ± 0% ~ (p=0.725 n=10+10)
name old allocs/op new allocs/op delta
CodeDecoder-8 76.9k ± 0% 76.9k ± 0% ~ (all equal)
cmd/go: add env -w and env -u to set and unset default env vars
Setting environment variables for go command configuration
is too difficult and system-specific. This CL adds go env -w,
to change the default settings more easily, in a portable way.
It also adds go env -u, to unset those changes.
See https://golang.org/design/30411-env for details.
cmd/go/internal/generate: stop premature variable substitution in commands
go:generate commands passed no arguments are currently subject
to premature variable substitution due to mistakenly assuming append
guarantees a copy. The change fixes this by forcing a slice copy at
each invocation of a command.
The previous code assumed that append would always generate a
copy of its inputs. However, append wouldn't create a copy if there was
no need to increase capacity and it would just return the original
input slice. This resulted in premature variable substitutions in
the "master word list" of generate commands, thus yielding incorrect
results across multiple invocations of the same command when the
body contained substitutions e.g. environment variables, moreover
these can change during the lifetime of go:generate processing a
file.
Note that this behavior would not manifest itself if any arguments were
passed to the command, because append would make a copy of the slice
as it needed to increase its capacity. The "hacky" work-around was to
always pass at least one argument to any command, even if the
command ignores it. e.g.,
//go:generate MyNoArgsCmd ' '
This CL fixes that issue and removes the need for the hack mentioned
above.
Fixes #31608
Change-Id: I782ac2234bd7035a37f61c101ee4aee38ed8d29f
GitHub-Last-Rev: 796d3430191f183c123c450a60b4a7987cc85e20
GitHub-Pull-Request: golang/go#31527
Reviewed-on: https://go-review.googlesource.com/c/go/+/172580
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Keith Randall [Wed, 10 Apr 2019 23:44:46 +0000 (16:44 -0700)]
cmd/compile: use correct package name for stack object symbol
Stack object generation code was always using the local package name
for its symbol. Normally that doesn't matter, as we usually only
compile functions in the local package. But for wrappers, the compiler
generates functions which live in other packages. When there are two
other packages with identical functions to wrap, the same name appears
twice, and the compiler goes boom.
Fixes #31252
Change-Id: I7026eebabe562cb159b8b6046cf656afd336ba25
Reviewed-on: https://go-review.googlesource.com/c/go/+/171464
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
It will use the full names that appear in netbsd's /usr/include/sys/syscall.h names.
This adds some compat-goo (sys_sigprocmask->SYS_sigprocmask14), which might not be pretty, but the information about whether the compat version is used is probably important, as Go will keep using interfaces even after they are considered compatibility, which has caused problems in the past.
also, the same names appear in ktrace (with the numbers).
This works well enough to run some code natively on arm64, but not well enough for more complicated code. I've been suggested to start a pull request anyway.
Bryan C. Mills [Thu, 28 Mar 2019 17:18:37 +0000 (13:18 -0400)]
cmd/go: only add a 'go' directive on 'go mod tidy' or when a conversion occurs
If the go.mod file exists and is empty, we initialize it from any of
various formats supported by legacy dependency-management tools.
We also initialize the 'go' directive at that point: we know that the
go.mod file is incomplete, because it does not reflect the information
in the legacy configuration file, and since we know that the go.mod
file is incomplete, we should complete it with as much information as
we have — including the version of the language currently in use.
However, if there is no legacy configuration file present, then we
cannot infer that the go.mod file is incomplete: it may correctly
specify a module without external dependencies. In that case, we
should not initialize the 'go' directive either: the user will not be
expecting unnecessary edits to the go.mod file, and we generally do
not make unnecessary-but-helpful edits unless 'go mod tidy' is invoked
explicitly.
Fixes #30790
Fixes #31100
Change-Id: I05a7872bce54a917c10d910cd9a616cab52e2730
Reviewed-on: https://go-review.googlesource.com/c/go/+/169877
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Matthew Dempsky [Fri, 19 Apr 2019 19:20:56 +0000 (12:20 -0700)]
cmd/compile: fix ICE from go/defer call to variadic function
The special case logic for go/defer arguments in Escape.call was
scattered around a bit and was somewhat inconsistently handled across
different types of function calls and parameters. This CL pulls the
logic out into a separate callStmt method that's used uniformly for
all kinds of function calls and arguments.
David Chase [Tue, 16 Apr 2019 01:27:04 +0000 (21:27 -0400)]
cmd/compile: shortcut intrinsic inlining AFTER getcallerXX check
A check in inl.go to prevent inlining of functions calling
either getcallerpc or getcallersp does not work when these
functions are intrinsics. Swap checks to fix.
Includes test.
No bug, this was discovered in the course of a ridiculous
experiment with inlining.
Change-Id: Ie1392523bb89882d586678f2674e1a4eadc5e431
Reviewed-on: https://go-review.googlesource.com/c/go/+/172217
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Than McIntosh [Thu, 18 Apr 2019 13:45:40 +0000 (09:45 -0400)]
go/internal/gccgoimporter: improve alias handling for anonymous fields
The code in the parser that deals with anonymous structure fields
records the fact that a field is anonymous, then tries to install a proxy
name for the field based on the name of the type used to declare
the field. If that type was an alias, the current recipe for determining
the proxy name was not working properly; enhance the code to recover
and report the alias name used.
Fixes #31540.
Change-Id: I9b7369ed558a288b56d85170c6f1144daf5228eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/172603 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Than McIntosh [Fri, 19 Apr 2019 18:50:57 +0000 (14:50 -0400)]
cmd/link: adjust whitelist for -strictdups checking for plan9
Add a couple of additional entries to the white list used to screen
out errors for builtin functions; these correspond to cases
that appear to come up only on the plan9 builder.
Updates #31503.
Change-Id: I48ab942ab2894240efe651ec7b7eace7aa5cb45e
Reviewed-on: https://go-review.googlesource.com/c/go/+/172986 Reviewed-by: David du Colombier <0intro@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
os: disable the use of netpoll on directories as well on *BSDs
Follow up CL 156379.
Updates #19093
Change-Id: I5ea3177fc5911d3af71cbb32584249e419e9d4a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/172937
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/go/internal/modload: fix boundary conditions in matchPackages
This makes the boundary logic of matchPackages consistent with
modload.dirInModule.
Previously, matchPackages always stopped at go.mod file, even within
the vendor tree. However, we do not guarantee that the vendor tree is
free of such files in general.
matchPackages also issued needless stat operations for modules in the
module cach, which we already know to be free of nested modules. On
systems with slow filesystems (such as macOS), those extra calls could
potentially slow package matching considerably.
Change-Id: I71979ab752e1d3971b370b37085d30502690413b
Reviewed-on: https://go-review.googlesource.com/c/go/+/172985
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
This resurrects CL 121198, except that this time we map read-only.
In case that we need to apply relocations to the symbol's
content that is backed by read-only memory, we do our own copy-
on-write. This can happen if we failed to mmap the output file,
or we build for Wasm.
Memory profile for building k8s.io/kubernetes/cmd/kube-apiserver
on Linux/AMD64:
Old (before this sequence of CLs):
inuse_space 1598.75MB total
669.87MB 41.90% 41.90% 669.87MB 41.90% cmd/link/internal/objfile.(*objReader).readSlices
New:
inuse_space 1280.45MB total
441.18MB 34.46% 34.46% 441.18MB 34.46% cmd/link/internal/objfile.(*objReader).readSlices
Move the phase of applying relocations later, after the sections
and segments are written to the mmap'd output region. Then apply
relocations directly in the output region, instead of the input.
So the input slices we read in don't need to be modified.
This is in preparation for mmap'ing input files read-only.
cmd/link: apply DWARF relocations while doing compression
We are preparing for applying relocations to the output buffer.
However, for DWARF compression, relocations need to be applied
before compression, but we don't have an output buffer at that
time. We also cannot delay DWARF compression to when we mmap the
output file, because we need the size of the DWARF sections to
compute the file size.
Instead of applying all the relocations together, we apply
relocations in DWARF sections one symbol at a time, right before
it is writing out for compression. As the symbol content may be
in read-only memory (in the future), we use a temporary buffer
for applying the relocations, and immediately write it out.
If compression is not used, relocations are still applied all
together.
This is in preparation for mmap'ing input files read-only.
Apply R_DWARFFILEREF relocations later, along with other
relocations, so that we don't modify symbols' contents before
they are written to the output buffer.
This is in preparation for mmap'ing input files read-only.
Use mmap for writing most of the output file content,
specifically, the sections and segments. After layout, we
already know the sizes and file offsets for the sections and
segments. So we can just write the bytes by copying to a mmap'd
backing store.
The writing of the output file is split into two parts. The first
part writes the sections and segments to the mmap'd region. The
second part writes some extra content, for which we don't know
the size, so we use direct file IO.
This is in preparation for mmap'ing input files read-only.
Pass "set print thread-events off" to gdb to suppress thread
event prints, like "[New Thread 0xe7b83b40 (LWP 18609)]". We
don't check them, and the extra output may confuse our other
checks, in particular, checkCleanBacktrace.
Ian Lance Taylor [Fri, 19 Apr 2019 16:48:36 +0000 (09:48 -0700)]
bootstrap.bash: make source writable before cleaning
Otherwise the "git clean" command fails with errors like
rm: cannot remove '/home/iant/go-linux-ppc64-bootstrap/pkg/mod/golang.org/x/text@v0.0.0-20170915032832-14c0d48ead0c/encoding/simplifiedchinese/all.go': Permission denied
Change-Id: Iecfb1fed6d59819d7fdceb9e391a2b3f81ea620c
Reviewed-on: https://go-review.googlesource.com/c/go/+/172998
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ian Lance Taylor [Fri, 19 Apr 2019 16:50:01 +0000 (09:50 -0700)]
cmd/link: require cgo support for TestSectionsWithSameName
The test doesn't really require cgo, but it does require that we know
the right flags to use to run the C compiler, and that is not
necessarily correct if we don't support cgo.
Fixes #31565
Change-Id: I04dc8db26697caa470e91ad712376aa621cf765d
Reviewed-on: https://go-review.googlesource.com/c/go/+/172981
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
runtime, cmd/compile: re-order PCDATA and FUNCDATA indices
The pclntab encoding supports writing only some PCDATA and FUNCDATA values.
However, the encoding is dense: The max index in use determines the space used.
We should thus choose a numbering in which frequently used indices are smaller.
This change re-orders the PCDATA and FUNCDATA indices using that principle,
using a quick and dirty instrumentation to measure index frequency.
The phi tighten pass moves rematerializable phi args
to the immediate predecessor of the phis.
This reduces value lifetimes for regalloc.
However, the critical edge removal pass can introduce
new blocks, which can change what a block's
immediate precedessor is. This can result in tightened
phi args being spilled unnecessarily.
This change moves the phi tighten pass after the
critical edge pass, when the block structure is stable.
cmd/go/internal/modcmd: skip files with the "ignore" constraint in 'go mod vendor'
'go mod vendor' already drops test files and testdata directories, so
users should not expect the vendored module to include unnecessary
files in general.
Files tagged "ignore" are typically only used to refresh or regenerate
source files within the module to be vendored, so users of that module
do not need them.