Cuong Manh Le [Wed, 6 May 2020 17:35:28 +0000 (00:35 +0700)]
cmd/compile: another fix initializing blank fields in struct literal
CL 230121 fixed the bug that struct literal blank fields type array/struct
can not be initialized. But it still misses some cases when an expression
causes "candiscard(value)" return false. When these happen, we recursively
call fixedlit with "var_" set to "_", and hit the bug again.
To fix it, just making splitnode return "nblank" whenever "var_" is "nblank".
Fixes #38905
Change-Id: I281941b388acbd551a4d8ca1a235477f8d26fb6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/232617
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
kakulisen [Thu, 14 May 2020 08:20:58 +0000 (16:20 +0800)]
math: simplify the code
Simplifying some code without compromising performance.
My CPU is Intel Xeon Gold 6161, 2.20GHz, 64-bit operating system.
The memory is 8GB. This is my test environment, I hope to help you judge.
Benchmark:
name old time/op new time/op delta
Log1p-4 21.8ns ± 5% 21.8ns ± 4% ~ (p=0.973 n=20+20)
Change-Id: Icd8f96f1325b00007602d114300b92d4c57de409
Reviewed-on: https://go-review.googlesource.com/c/go/+/233940 Reviewed-by: Robert Griesemer <gri@golang.org>
sync: fix goroutine leak for when TestMutexFairness times out
If the timeout triggers before writing to the done channel, the
goroutine will be blocked waiting for a corresponding read that’s
no longer existent, thus a goroutine leak. This change fixes that by
using a buffered channel instead.
Keith Randall [Sat, 8 Aug 2020 14:58:04 +0000 (07:58 -0700)]
cmd/compile: fix live variable computation for deferreturn
Taking the live variable set from the last return point is problematic.
See #40629 for details, but there may not be a return point, or it may
be before the final defer.
Additionally, keeping track of the last call as a *Value doesn't quite
work. If it is dead-code eliminated, the storage for the Value is reused
for some other random instruction. Its live variable information,
if it is available at all, is wrong.
Instead, just mark all the open-defer argument slots as live
throughout the function. (They are already zero-initialized.)
Fixes #40629
Change-Id: Ie456c7db3082d0de57eaa5234a0f32525a1cce13
Reviewed-on: https://go-review.googlesource.com/c/go/+/247522
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dan Scales <danscales@google.com>
modload.Init initialized the build cache with the intent of providing
a better error message in Go 1.12, when the build cache became
mandatory (in module mode, packages aren't installed outside the build
cache). Unfortunately, this didn't provide a more descriptive error
(the cache calls base.Fatalf with its own message), and it caused
errors for commands that don't use the cache (like 'go mod edit').
This CL removes the cache initialization from modload.Init. The
builder will initialize it when it's needed.
For #39882
Change-Id: Ibc01ae4e59358dcd08a07ffc97bf556514d0366f
Reviewed-on: https://go-review.googlesource.com/c/go/+/240548
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Jay Conrod [Thu, 25 Jun 2020 21:50:38 +0000 (17:50 -0400)]
cmd/go: migrate to module.MatchPrefixPatterns
In CL 239797, str.GlobsMatchPath was copied to golang.org/x/mod/module
as MatchPrefixPatterns. This CL updates x/mod, switches calls to use
the new function, and deletes the old function.
For #38725
Change-Id: I7241032228b574aa539426a92d2f5aad9ee001e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/240061
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Jay Conrod [Mon, 8 Jun 2020 22:06:11 +0000 (18:06 -0400)]
cmd/go: don't save sums for modules loaded for import resolution
modfetch.WriteGoSum now accepts a map[module.Version]bool parameter.
This is used to prevent some new sums from being saved to go.sum when
they would be removed by the next 'go mod tidy'. Previusly, sums were
saved for modules looked up during import resolution.
A new function, modload.TrimGoSum, is also introduced, which marks
sums for deletion. 'go mod tidy' now uses this. The new logic
distinguishes between go.mod sums and content sums, which lets 'go mod
tidy' delete sums for modules in the build graph but not the build
list.
Fixes #31580
Fixes #36260
Fixes #33008
Change-Id: I06c4125704a8bbc9969de05265967ec1d2e6d3e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/237017
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Jay Conrod [Thu, 4 Jun 2020 15:41:43 +0000 (11:41 -0400)]
cmd/go: don't initialize Builder in envcmd.MkEnv
The Builder isn't needed by MkEnv, and Builder.Init doesn't have side
effects that change the environment. Builder.Init does currently call
CheckGOOSARCHPair, but that's being moved out in CL 234658.
Builder.Init creates the temporary work directory used by the
builder. For the builder created in MkEnv, this directory is never
used. Creating this directory can cause unnecessary errors for
commands that don't use a builder like 'go clean' and 'go list'.
Fixes #38395
Updates #24398
Change-Id: Ib93ae55afdf958000470657f4c4ff5bd92700e46
Reviewed-on: https://go-review.googlesource.com/c/go/+/236563
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Tim Möhlmann [Thu, 13 Aug 2020 08:56:31 +0000 (11:56 +0300)]
database/sql: make Rows.Scan properly wrap underlying errors
The prior implementation used the format verb %v which unfortunately
improperly wrapped any underlying scanner errors, and we couldn't use
errors.Is nor errors.As. This change fixes that by using the %w verb.
Added a unit to ensure that both error sub string matching works, but
also that errors.Is works as expected.
Michael Munday [Mon, 10 Aug 2020 15:01:21 +0000 (08:01 -0700)]
cmd/internal/obj: fix inline marker issue on s390x
The optimization that replaces inline markers with pre-existing
instructions assumes that 'Prog' values produced by the compiler are
still reachable after the assembler has run. This was not true on
s390x where the assembler was removing NOP instructions from the
linked list of 'Prog' values. This led to broken inlining data
which in turn caused an infinite loop in the runtime traceback code.
Fix this by stopping the s390x assembler backend removing NOP
values. It does not make any difference to the output of the
assembler because NOP instructions are 0 bytes long anyway.
Fixes #40473.
Change-Id: Ib4fabadd1de8adb80421f75950ee9aad2111147a
Reviewed-on: https://go-review.googlesource.com/c/go/+/247697
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Lynn Boger [Tue, 11 Aug 2020 16:04:25 +0000 (12:04 -0400)]
cmd/internal/obj/ppc64: don't remove NOP in assembler
Previously, the assembler removed NOPs from the Prog list in
obj9.go. NOPs shouldn't be removed if they were added as
an inline mark, as described in the issue below.
Fixes #40689
Once the NOPs were left in the Prog list, some instructions
were flagged as invalid because they had an operand which was
not represented in optab. In order to preserve the previous
assembler behavior, entries were added to optab for those
operand cases. They were not flagged as errors before because
the NOP instructions were removed before the code to check the
valid opcode/operand combinations.
Change-Id: Iae5145f94459027cf458e914d7c5d6089807ccf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/247842
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Paul Murphy <murp@ibm.com> Reviewed-by: Michael Munday <mike.munday@ibm.com> Reviewed-by: Keith Randall <khr@golang.org>
Minor cleanup: remove the symbol attribute AttrSeenGlobal, since it is
redundant with the existing attribute AttrOnList (no need to have what
amounts to a separate flag for checking the same property).
Change-Id: Ia269b64de37c2bb4a2314bbecf3d2091c6d57424
Reviewed-on: https://go-review.googlesource.com/c/go/+/239477
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Than McIntosh [Wed, 12 Aug 2020 16:59:55 +0000 (12:59 -0400)]
doc: add a release notes blurb on 1.16 linker improvements
Add a draft version of a blurb on improvements to the linker. This
will need to be finalized later in the release since there are still
some additional changes to be made to the linker in 1.16.
Go 1.14 included a (rather awful) workaround for a Linux kernel bug
that corrupted vector registers on x86 CPUs during signal delivery
(https://bugzilla.kernel.org/show_bug.cgi?id=205663). This bug was
introduced in Linux 5.2 and fixed in 5.3.15, 5.4.2 and all 5.5 and
later kernels. The fix was also back-ported by major distros. This
workaround was necessary, but had unfortunate downsides, including
causing Go programs to exceed the mlock ulimit in many configurations
(#37436).
We're reasonably confident that by the Go 1.16 release, the number of
systems running affected kernels will be vanishingly small. Hence,
this CL removes this workaround.
This effectively reverts CLs 209597 (version parser), 209899 (mlock
top of signal stack), 210299 (better failure message), 223121 (soft
mlock failure handling), and 244059 (special-case patched Ubuntu
kernels). The one thing we keep is the osArchInit function. It's empty
everywhere now, but is a reasonable hook to have.
Updates #35326, #35777 (the original register corruption bugs).
Updates #40184 (request to revert in 1.15).
Fixes #35979.
Change-Id: Ie213270837095576f1f3ef46bf3de187dc486c50
Reviewed-on: https://go-review.googlesource.com/c/go/+/246200
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Wed, 12 Aug 2020 17:30:43 +0000 (13:30 -0400)]
doc/go1.15: clarify external linking can still be used for building PIE
In Go 1.15 we switched the default linking mode for PIE on
Linux/AMD64 and Linux/ARM64 to internal linking. Clarify that
the previous behavior (external linking) can still be used with
a flag.
Fixes #40719.
Change-Id: Ib7042622bc91e1b1aa31f520990d03b5eb6c56bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/248199 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cherry Zhang [Wed, 12 Aug 2020 21:37:18 +0000 (17:37 -0400)]
cmd: merge branch 'dev.link' into master
In the dev.link branch we have continued developing the new object
file format support and the linker improvements described in
https://golang.org/s/better-linker . Since the last merge (May 1st
2020), more progress has been made to improve the new linker, with
improvements on both linker speed and memory usage.
Michael Matloob [Wed, 17 Jun 2020 19:50:14 +0000 (15:50 -0400)]
cmd/go: propagate context into PackagesForBuild and Do for tracing
This change propagates context into PackagesForErrors and Do for
the purpose of tracing, and calls trace.StartSpan on PackagesForErrors
and Do, so that the trace now shows the broad outline of where
the "Loading" and "Execution" phases are in the build.
Updates #38714
Change-Id: Ib9a7cf7030210f68f76663d1c8a7461e0a226611
Reviewed-on: https://go-review.googlesource.com/c/go/+/238541
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Michael Matloob [Fri, 19 Jun 2020 21:59:16 +0000 (17:59 -0400)]
cmd/trace: move viewer data structs into cmd/internal/traceviewer
The ViewerEvent, ViewerData and ViewerFrame structs are moved into
cmd/internal/traceviewer, and renamed Event, Data, and Frame.
The structs are the same, except for the following: A definition
for the JSON "bp" field that's defined in the trace format, but
missing in the structs has been added. Also, the Tid and Pid fields
on Event have been renamed TID and PID to better match Go style.
Finally, the footer field on ViewerData, which hasn't been used
for a while, has been removed.
This CL is in preparation for the usage of these structs by cmd/go's
tracing functionality.
Updates #38714
Change-Id: I345f23617b96d4629b876ae717f89d56a67e05a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/239098
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
Dmitri Shuralyov [Wed, 12 Aug 2020 14:34:15 +0000 (10:34 -0400)]
doc/go1.16: start draft release notes
This template is based on CL 220278 and previous ones like it.
Include Compiler and Linker sections proactively, they can be
removed if they don't end up being needed for Go 1.16.
Use two spaces of indentation for TODOs to set a better precedent
for the final text that will take its place.
'relnote -html' does not report any changes at this time.
For #40700.
Change-Id: I096b0ce0d33aaaa6fae9c91c0d2dfb89b9c5e94c
Reviewed-on: https://go-review.googlesource.com/c/go/+/248198 Reviewed-by: Carlos Amedee <carlos@golang.org>
Dmitri Shuralyov [Tue, 11 Aug 2020 22:37:44 +0000 (18:37 -0400)]
internal/goversion: update Version to 1.16
This is the start of the Go 1.16 development cycle, so update the
Version value accordingly. It represents the Go 1.x version that
will soon open up for development (and eventually become released).
Historically, we used to bump this at an arbitrary time throughout
the development cycle, but it's better to be more predictable about
updating it. The start of a development cycle should be the most
appropriate time: it clearly marks the boundary between 1.15 and
1.16 development, and doing it early can help catch issues in other
tooling. See issue #38704 for more background.
There is no longer a need to update the list of Go versions in
src/go/build/doc.go because it does not exist as of CL 232981.
For #40705.
Updates #38704.
Updates #37018.
Change-Id: Id8ee733b5e79c53b6cd03509c6560614d8743833
Reviewed-on: https://go-review.googlesource.com/c/go/+/248038 Reviewed-by: Carlos Amedee <carlos@golang.org>
Cherry Zhang [Wed, 5 Aug 2020 00:25:10 +0000 (20:25 -0400)]
runtime: make nanotime1 reentrant
Currently, nanotime1 (and walltime1) is not reentrant, in that it
sets m.vdsoSP at entry and clears it at exit. If a signal lands
in between, and nanotime1 is called from the signal handler, it
will clear m.vdsoSP while we are still in nanotime1. If (in the
unlikely event) it is signaled again, m.vdsoSP will be wrong,
which may cause the stack unwinding code to crash.
This CL makes it reentrant, by saving/restoring the previous
vdsoPC and vdsoSP, instead of setting it to 0 at exit.
TODO: have some way to test?
Change-Id: I9ee53b251f1d8a5a489c71d4b4c0df1dee70c3e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/246763
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jeremy Faller [Wed, 29 Jul 2020 15:06:02 +0000 (11:06 -0400)]
[dev.link] use per package filenames to build pclntab
In order to prevent renumbering of filenames in pclntab generation, use
the per-package file list (previously only used for DWARF generation) as
file-indices. This is the largest step to eliminate renumbering of
filenames in pclntab.
Note, this is probably not the final state of the file table within the
object file. In this form, the linker loads all filenames for all
objects. I'll move to storing the filenames as regular string
symbols,and defaulting all string symbols to using the larger hash value
to make generation of pcln simplest, and most memory friendly.
Than McIntosh [Wed, 8 Jul 2020 15:02:51 +0000 (11:02 -0400)]
[dev.link] cmd/link: add PPC64 section splitting test
Add a new PPC64-only linker test that does a build with the
-debugppc64textsize debugging option (selecting a lower the threshold
for text section splitting) to verify that no bugs have been
introduced in the linker code that manages this process.
Change-Id: Iea3f16a04c894d528eab2cb52f1ec1d75a2770cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/241499
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jeremy Faller <jeremy@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Than McIntosh [Tue, 7 Jul 2020 16:54:46 +0000 (12:54 -0400)]
[dev.link] cmd/link: add PPC64 debugging option to encourage text section splits
Add a new debugging command line option (-debugppc64textsize=N) that
forces the start of a new text section after ".text" hits N bytes as
opposed to the architected limit of 2^26. This is intended to enable
testing of the linker code paths that handle multiple .text sections
on PPC64 without resorting to building giant applications.
Updates #20492.
Change-Id: I74ab7fd1e412e9124de5bd0d8d248c5e73225ae3
Reviewed-on: https://go-review.googlesource.com/c/go/+/241073
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jeremy Faller <jeremy@golang.org>
Than McIntosh [Wed, 8 Jul 2020 14:43:42 +0000 (10:43 -0400)]
[dev.link] cmd/link: fix ppc64-specific bug in genelfsym
The code in the the linker's genelfsym() routine was not properly
including runtime.text.%d marker symbols that are emitted on PPC64
when a very large text section is split into chunks. This bug was
introduced in CL 233338 when portions of asmb2() were converted
from sym.Symbol to loader.Sym usage.
Change-Id: Idfed944c41e1805f78f35be67bcdd18bdefd7819
Reviewed-on: https://go-review.googlesource.com/c/go/+/241498
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org>
Katie Hockman [Tue, 4 Aug 2020 15:45:32 +0000 (11:45 -0400)]
encoding/binary: read at most MaxVarintLen64 bytes in ReadUvarint
This CL ensures that ReadUvarint consumes only a limited
amount of input (instead of an unbounded amount).
On some inputs, ReadUvarint could read an arbitrary number
of bytes before deciding to return an overflow error.
After this CL, ReadUvarint returns that same overflow
error sooner, after reading at most MaxVarintLen64 bytes.
Fix authored by Robert Griesemer and Filippo Valsorda.
Thanks to Diederik Loerakker, Jonny Rhea, Raúl Kripalani,
and Preston Van Loon for reporting this.
Michael Matloob [Tue, 4 Aug 2020 17:24:37 +0000 (13:24 -0400)]
cmd/go: fix error stacks when there are scanner errors
After golang.org/cl/228784 setLoadPackageDataError tries to decide whether an
error is caused by an imported package or an importing package by examining the
error itself to decide. Ideally, the errors themselves would belong to a
specific interface or some other property to make it unambiguous that they
were import errors. Since they don't, setLoadPackageDataError just checked
for nogoerrors and classified all other errors as import errors. But
it missed scanner errors which are also "caused" by the imported
package.
Fixes #40544
Change-Id: I39159bfdc286bee73697decd07b8aa9451f2db06
Reviewed-on: https://go-review.googlesource.com/c/go/+/246717
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Cherry Zhang [Mon, 3 Aug 2020 21:59:41 +0000 (17:59 -0400)]
[dev.link] cmd/link: fix pclntab symbol handling on AIX
On AIX, container symbols are handled in a weird way (unlike
other platforms): the outer symbol needs to have size (but still
no data), and the inner symbols must not be in the symbol table
(otherwise it overlaps with the outer symbol, which the system
linker doesn't like).
As of CL 241598, pclntab becomes a container symbol. We need to
follow the rule above for AIX.
[dev.link] cmd/compile, cmd/link: generate itablink at link time
Currently, at compile time, for each itab symbol, we create an
"itablink" symbol which holds solely the address of the itab
symbol. At link time, all the itablink symbols are grouped
together to form the itablinks slice.
This CL removes the itablink symbols, and directly generate the
itablinks slice in the linker. This removes a number of symbols,
which are dupOK and generally have long names. And also removes
a special handling of itablink symbols in the deadcode pass which
iterates through all symbols.
Change-Id: I475c3c8899e9fbeec9abc7647b1e4a69aa5c3c5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/245901 Reviewed-by: Jeremy Faller <jeremy@golang.org>
[dev.link] cmd/compile, cmd/link: make itab symbols content-addressable
Extend the content-addressable symbol mechanism to itab symbols.
Itab symbols require global uniqueness (as at run time we compare
pointers), so it needs to be reliably deduplicated. Currently the
content hash depends on symbol name expansion, so we can only do
this when all Go packages are built with know package paths. Fall
back to checking names if any Go package is built with unknown
package path.
Change-Id: Icf5e8873755050c20e5fc6549f6de1c883254c89
Reviewed-on: https://go-review.googlesource.com/c/go/+/245719 Reviewed-by: Jeremy Faller <jeremy@golang.org>
[dev.link] cmd/compile, cmd/link: reference type symbol of defined type by index
The type descriptor symbol of a defined (named) type (and pointer
to it) is defined only in the package that defines the type. It
is not dupOK, unlike other type descriptors. So it can be
referenced by index. Currently it is referenced by name for
cross-package references, because the index is not exported and
so not known to the referencing package.
This CL passes the index through the export data, so the symbol
can be referenced by index, and does not need to be looked up by
name. This also makes such symbol references consistent: it is
referenced by index within the defining package and also cross-
package, which makes it easier for content hashing (in later CLs).
One complication is that we need to set flags on referenced
symbols (specifically, the UsedInIface flag). Before, they are
non-package refs, which naturally carry flags in the object file.
For indexed refs, we currently don't put their flags in the
object file. Introduce a new block for this.
Change-Id: I8126f8e318ac4e6609eb2ac136201fd6c264c256
Reviewed-on: https://go-review.googlesource.com/c/go/+/245718 Reviewed-by: Jeremy Faller <jeremy@golang.org>
Jeremy Faller [Tue, 21 Jul 2020 19:53:30 +0000 (15:53 -0400)]
[dev.link] move FuncID creation into the compiler/assembler
Leaving creation of the funcID till the linker requires the linker to
load the function and file names into memory. Moving these into the
compiler/assembler prevents this.
This work is a step towards moving all func metadata into the compiler.
Keith Randall [Sat, 1 Aug 2020 19:55:47 +0000 (12:55 -0700)]
runtime/race: fix ppc64le build
The .syso test also fails for ppc64le. Not sure why. For now, just
disable the test for that architecture. The test really only needs to
run on a single builder of any arch.
Change-Id: I346cdc01ada09d43c4c504fbc30be806f59d5422
Reviewed-on: https://go-review.googlesource.com/c/go/+/246358
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Fri, 31 Jul 2020 20:56:18 +0000 (13:56 -0700)]
runtime/race: rebuild some .syso files to remove getauxval dependency
We can't depend on getauxval because it only exists in glibc >= 2.16.
Tsan has been updated to avoid that dependency
(https://reviews.llvm.org/D84859). This CL rebuilds the affected
.syso files, and adds a test to make sure we don't regress.
[dev.link] cmd/link: make symbol attribute setting more reliable
For dupOK symbols, their attributes should be OR'd. Most of the
attributes are expected to be set consistently across multiple
definitions, but UsedInIface must be OR'd, and for alignment we
need to pick the largest one. Currently the attributes are not
always OR'd, depending on addSym returning true or false. This
doesn't cause any real problem, but it would be a problem if we
make type descriptor symbols content-addressable.
This CL removes the second result of addSym, and lets preloadSyms
always set the attributes. Also removes the alignment handling on
addSym, handles it in preloadSyms only.
Change-Id: I06b3f0adb733f6681956ea9ef54736baa86ae7bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/245720 Reviewed-by: Jeremy Faller <jeremy@golang.org>
Notably, _Ctype_union_U was declared as a defined type, but uses were
being rewritten into uses of the underlying type, which matched how
_Cfunc_f was declared.
After CL 230037, cgo started consistently rewriting "C.foo" type
expressions as "_Ctype_foo", which caused it to start emitting:
type _Ctype_enum_E uint32
type _Ctype_union_U [8]byte
Of course, this fails to type-check because _Ctype_enum_E and
_Ctype_union_U are defined types.
This CL changes cgo to emit:
type _Ctype_enum_E = uint32
type _Ctype_union_U = [8]byte
// f unchanged since CL 230037
// _Cfunc_f still unchanged
It would probably be better to fix this in (*typeConv).loadType so
that cgo generated code uses the _Ctype_foo aliases too. But as it
wouldn't have any effect on actual compilation, it's not worth the
risk of touching it at this point in the release cycle.
Updates #39537.
Fixes #40494.
Change-Id: I88269660b40aeda80a9a9433777601a781b48ac0
Reviewed-on: https://go-review.googlesource.com/c/go/+/246057 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Anthony Knyszek [Mon, 13 Jul 2020 19:51:50 +0000 (19:51 +0000)]
runtime: validate candidate searchAddr in pageAlloc.find
Currently pageAlloc.find attempts to find a better estimate for the
first free page in the heap, even if the space its looking for isn't
necessarily going to be the first free page in the heap (e.g. if npages
>= 2). However, in doing so it has the potential to return a searchAddr
candidate that doesn't actually correspond to mapped memory, but this
candidate might still be adopted. As a result, pageAlloc.alloc's fast
path may look at unmapped summary memory and segfault. This case is rare
on most operating systems since the heap is kept fairly contiguous, so
the chance that the candidate searchAddr discovered is unmapped is
fairly low. Even so, this is totally possible and outside the user's
control when it happens (in fact, it's likely to happen consistently for
a given user on a given system).
Fix this problem by ensuring that our candidate always points to mapped
memory. We do this by looking at mheap's arenas structure first. If it
turns out our candidate doesn't correspond to mapped memory, then we
look at inUse to round up the searchAddr to the next mapped address.
While we're here, clean up some documentation related to searchAddr.
Fixes #40191.
Change-Id: I759efec78987e4a8fde466ae45aabbaa3d9d4214
Reviewed-on: https://go-review.googlesource.com/c/go/+/242680
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Jeremy Faller [Tue, 28 Jul 2020 17:49:13 +0000 (13:49 -0400)]
[dev.link] ensure package path is set when TEXT symbols are created
We're reworking pclntab generation in the linker, and with that we're
moving FuncID generation in to the compiler. Determining the FuncID is
done by a lookup on the package.function name; therefore, we need the
package whenever we make the TEXT symbols.
Jeremy Faller [Thu, 16 Jul 2020 20:18:49 +0000 (16:18 -0400)]
[dev.link] create runtime.funcnametab
Move the function names out of runtime.pclntab_old, creating
runtime.funcnametab. There is an unfortunate artifact in this change in
that calculating the funcID still requires loading the name. Future work
will likely pull this out and put it into the object file Funcs.
Jeremy Faller [Wed, 24 Jun 2020 18:30:16 +0000 (14:30 -0400)]
[dev.link] cmd/link: add runtime.pcheader
As of July 2020, a fair amount of the new linker's live memory, and
runtime is spent generating pclntab. In an effort to streamline that
code, this change starts breaking up the generation of runtime.pclntab
into smaller chunks that can run later in a link. These changes are
described in an (as yet not widely distributed) document that lays out
an improved format. Largely the work consists of breaking up
runtime.pclntab into smaller pieces, stopping much of the data
rewriting, and getting runtime.pclntab into a form where we can reason
about its size and look to shrink it. This change is the first part of
that work -- just pulling out the header, and demonstrating where a
majority of that work will be.
[dev.link] cmd/internal/obj: trim trailing zeros for content hashing
The symbol's data in the object file (sym.P) may already not
contain trailing zeros (e,g, for [10]int{1}), but sometimes it
does (e.g. for [10]int{1,0}). The linker can already handle this
case. We just always trim the trailing zeros for content hashing,
so it can deduplicate [10]int{1} and [10]int{1,0}.
Note: in theory we could just trim the zeros in the symbol data
as well. But currently the linker depends on reading symbol data
for certain symbols (e.g. type symbol decoding), and trimming
will complicates things in the linker.
Cholerae Hu [Fri, 24 Jul 2020 03:00:36 +0000 (11:00 +0800)]
cmd/compile: don't addLocalInductiveFacts if there is no direct edge from if block to phi block
Currently in addLocalInductiveFacts, we only check whether
direct edge from if block to phi block exists. If not, the
following logic will treat the phi block as the first successor,
which is wrong.
This patch makes prove pass more conservative, so we disable
some cases in test/prove.go. We will do some optimization in
the following CL and enable these cases then.
Fixes #40367.
Change-Id: I27cf0248f3a82312a6f7dabe11c79a1a34cf5412
Reviewed-on: https://go-review.googlesource.com/c/go/+/244579 Reviewed-by: Zach Jones <zachj1@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We have Reloc and Reloc2. Reloc2 is the better approach and most
code uses Reloc2. There are still uses of Reloc. This CL migrates
them to Reloc2, and removes Reloc.
Change-Id: Id5f6a6019e1e044add682d05e70ebb1548ec58d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/245577 Reviewed-by: Jeremy Faller <jeremy@golang.org>
[dev.link] cmd/link: refactor ExtReloc data structures
We used to generate all external relocations in memory, then emit
the relocation records at a later pass. The data structures were
chosen so that it takes as little memory as possible. Now we just
stream out external relocations, and ExtReloc is just a local
variable. Change the data structure to avoid repeated read of
some fields. Also get rid of ExtRelocView, as it is no longer
necessary.
This reverts CL 202578 and CL 230677 which added an optimization
to use KDSA when available on s390x.
Inconsistencies have been found between the two implementations
in their handling of certain edge cases. Since the Go 1.15 release
is extremely soon it seems prudent to remove this optimization
for now and revisit it in a future release.
Fixes #40475.
Change-Id: Ifb2ed9b9e573784df57383671f1c29d8abae90d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/245497
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ruixin(Peter) Bao <ruixin.bao@ibm.com> Reviewed-by: Filippo Valsorda <filippo@golang.org>
Michael Pratt [Mon, 27 Jul 2020 19:04:17 +0000 (15:04 -0400)]
runtime: ensure startm new M is consistently visible to checkdead
If no M is available, startm first grabs an idle P, then drops
sched.lock and calls newm to start a new M to run than P.
Unfortunately, that leaves a window in which a G (e.g., returning from a
syscall) may find no idle P, add to the global runq, and then in stopm
discover that there are no running M's, a condition that should be
impossible with runnable G's.
To avoid this condition, we pre-allocate the new M ID in startm before
dropping sched.lock. This ensures that checkdead will see the M as
running, and since that new M must eventually run the scheduler, it will
handle any pending work as necessary.
Outside of startm, most other calls to newm/allocm don't have a P at
all. The only exception is startTheWorldWithSema, which always has an M
if there is 1 P (i.e., the currently running M), and if there is >1 P
the findrunnable spinning dance ensures the problem never occurs.
This has been tested with strategically placed sleeps in the runtime to
help induce the correct race ordering, but the timing on this is too
narrow for a test that can be checked in.
cmd/link: don't mark shared library symbols reachable unconditionally
During the transitioning period, we mark symbols from Go shared
libraries reachable unconditionally. That might be useful when
there was still a large portion of the linker using sym.Symbols,
and only reachable symbols were converted to sym.Symbols. Marking
them reachable brings them to the dynamic symbol table, even if
they are not needed, increased the binary size unexpectedly.
That time has passed. Now we largely operate on loader symbols,
and it is not needed to mark them reachable anymore.
OutData was used for a symbol to point to its data in the output
buffer, in order to apply relocations. Now we fold relocation
application to Asmb next to symbol data writing. We can just pass
the output data as a local variable.
Linking cmd/compile,
name old time/op new time/op delta
Asmb_GC 19.0ms ±10% 16.6ms ± 9% -12.50% (p=0.032 n=5+5)
name old alloc/op new alloc/op delta
Asmb_GC 3.78MB ± 0% 0.14MB ± 1% -96.41% (p=0.008 n=5+5)
name old live-B new live-B delta
Asmb_GC 27.5M ± 0% 23.9M ± 0% -13.24% (p=0.008 n=5+5)
Change-Id: Id870a10dce2a0a7447a05029c6d0ab39b47d0a12
Reviewed-on: https://go-review.googlesource.com/c/go/+/244017 Reviewed-by: Jeremy Faller <jeremy@golang.org>