This commit is a copy of filepath.WalkDir adapted to use fs.FS
instead of the native OS file system. It is the last implementation
piece of the io/fs proposal.
The original io/fs proposal was to adopt filepath.Walk, but we
have since introduced the more efficient filepath.WalkDir (#42027),
so this CL adopts that more efficient option instead.
(The changes in path/filepath bring the two copies more in line
with each other. The main change is unembedding the field
in statDirEntry, so that the fs.DirEntry passed to the WalkDirFunc
for the root of the tree does not have any extra methods.)
For #41190.
Change-Id: I9359dfcc110338c0ec64535f22cafb38d0b613a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/243916
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Russ Cox [Thu, 29 Oct 2020 16:20:53 +0000 (12:20 -0400)]
path/filepath: add WalkDir
WalkDir is like Walk but can use ReadDir to read directories,
instead of Readdirnames + Lstat on every entry,
which is usually a significant performance improvement.
(The Lstat can still happen if the walk function calls d.Info.)
Fixes #42027.
[Replay of CL 266240 after it was reverted due to accidentally
enabling on Windows a test that does not work on Windows.
The original code only ran the test on os.Getuid() > 0.
The rolled-back CL skipped the test on os.Getuid() == 0.
But on Windows, os.Getuid(), it turns out, always returns -1.
So what looked like a test for root was also excluding Windows.
This CL revises the test to skip Windows explicitly.]
Change-Id: I9b3661013d6449b7486532445d934ae91e5393ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/267887
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Michael Munday [Mon, 11 May 2020 16:44:48 +0000 (09:44 -0700)]
cmd/compile: optimize shift pairs and masks on s390x
Optimize combinations of left and right shifts by a constant value
into a 'rotate then insert selected bits [into zero]' instruction.
Use the same instruction for contiguous masks since it has some
benefits over 'and immediate' (not restricted to 32-bits, does not
overwrite source register).
To keep the complexity of this change under control I've only
implemented 64 bit operations for now.
There are a lot more optimizations that can be done with this
instruction family. However, since their function overlaps with other
instructions we need to be somewhat careful not to break existing
optimization rules by creating optimization dead ends. This is
particularly true of the load/store merging rules which contain lots
of zero extensions and shifts.
This CL does interfere with the store merging rules when an operand
is shifted left before it is stored:
binary.BigEndian.PutUint64(b, x << 1)
This is unfortunate but it's not critical and somewhat complex so
I plan to fix that in a follow up CL.
Change-Id: I7591cce41e06867ba10a745daae9333513062746
Reviewed-on: https://go-review.googlesource.com/c/go/+/233317
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Trust: Michael Munday <mike.munday@ibm.com>
Joel Sing [Thu, 5 Nov 2020 05:41:32 +0000 (16:41 +1100)]
cmd/dist: remove openbsd/mips64 from incomplete ports
Remove openbsd/mips64 from incomplete ports lists - all of the necessary code
has landed and we want to run tests so we can see/deal with remaining failures.
Update #40995
Change-Id: I5d4f89af82ff3abe57570a9a8abf889498093d32
Reviewed-on: https://go-review.googlesource.com/c/go/+/267606
Trust: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Jonathan Swinney [Wed, 4 Nov 2020 16:18:23 +0000 (16:18 +0000)]
cmd/compile: improve atomic swap intrinsics on arm64
ARMv8.1 has added new instructions for atomic memory operations. This
change builds on the previous change which added support for atomic add, 0a7ac93c27c9ade79fe0f66ae0bb81484c241ae5, to include similar support for
atomic-compare-and-swap, atomic-swap, atomic-or, and atomic-and
intrinsics. Since the new instructions are not guaranteed to be present,
we guard their usages with a branch on a CPU feature.
Peformance on an ARMv8.1 machine:
name old time/op new time/op delta
CompareAndSwap-16 37.9ns ±16% 24.1ns ± 4% -36.44% (p=0.000 n=10+9)
CompareAndSwap64-16 38.6ns ±15% 24.1ns ± 3% -37.47% (p=0.000 n=10+10)
name old time/op new time/op delta
Swap-16 46.9ns ±32% 12.5ns ± 6% -73.40% (p=0.000 n=10+10)
Swap64-16 53.4ns ± 1% 12.5ns ± 6% -76.56% (p=0.000 n=10+10)
Than McIntosh [Thu, 5 Nov 2020 19:19:47 +0000 (14:19 -0500)]
cmd/link: report error if builtin referenced but not defined
When the compiler refers to a runtime builtin, it emits an indexed
symbol reference in the object file via predetermined/preassigned ID
within the PkgIdxBuiltin pseudo-package. At link time when the loader
encounters these references, it redirects them to the corresponding
defined symbol in the runtime package. This redirection process
currently assumes that if a runtime builtin is referenced, we'll
always have a definition for it. This assumption holds in most cases,
however for the builtins "runtime.racefuncenter" and
"runtime.racefuncexit", we'll only see definitions if the runtime
package we're linking against was built with "-race".
In the bug in question, build passes "-gcflags=-race" during
compilation of the main package, but doesn't pass "-race" directly to
'go build', and as a result the final link combines a
race-instrumented main with a non-race runtime; this results in R_CALL
relocations with zero-valued target symbols, resulting in a panic
during stack checking.
This patch changes the loader's resolve method to detect situations
where we're asking for builtin "runtime.X", but the runtime package
read in doesn't contain a definition for X.
Fixes #42396.
Change-Id: Iafd38bd3b0f7f462868d120ccd4d7d1b88b27436
Reviewed-on: https://go-review.googlesource.com/c/go/+/267881
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Rob Findley [Tue, 27 Oct 2020 02:44:54 +0000 (22:44 -0400)]
go/types: add unexported start and end positions to type checker errors
Tools often need to associate errors not with a single position, but
with a span of source code. For example, gopls currently estimates
diagnostic spans using heuristics to expand the positions reported by
the type checker to surrounding source code. Unfortunately this is often
inaccurate.
This CL lays the groundwork to solve this within go/types by adding a
start and end position to type checker errors. This is an experimental
API, both because we are uncertain of the ideal representation for these
spans and because their initial positioning is naive. In most cases this
CL simply expands errors to the surrounding ast.Node being typechecked,
if available. This might not be the best error span to present to the
user. For these reasons the API is unexported -- gopls can read these
positions using reflection, allowing us to gain experience and improve
them during the next development cycle.
For golang/go#42290
Change-Id: I39a04d70ea2bb2134b4d4c937f32b2ddb4456430
Reviewed-on: https://go-review.googlesource.com/c/go/+/265250
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Michael Matloob [Tue, 3 Nov 2020 18:12:19 +0000 (13:12 -0500)]
cmd/go: account for flags when parsing regexps in TestScript
Test script expects the regexp argument for stdout, stderr, and cmp
to be the first argument after the command, but that might not be the
case if the -q or -count flags are provided. Treat the first argument
after a flag as a regexp instead.
For #39958
Change-Id: I369926109ec10cca8b2c3baca27e7a3f7baf364b
Reviewed-on: https://go-review.googlesource.com/c/go/+/267877
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Michael Pratt [Mon, 2 Nov 2020 21:34:51 +0000 (16:34 -0500)]
runtime: disable preemption in startm
startm contains a critical section from when it takes ownership of a P
(either on function entry or call to pidleput) until it wakes the M
receiving the P. If preempted in this critical section, the owned P is
left in limbo. If preempted for a GC stop, there will be nothing to stop
the owned P and STW will wait forever.
golang.org/cl/232298 introduced the first call to startm that is not on
the system stack (via a wakep call), introducing the possibility of
preemption. Disable preemption in startm to ensure this remains
non-preemptible.
Since we're not always on the system stack anymore, we also need to be
careful in allocm.
Updates #42237
Change-Id: Icb95eef9eb262121856485316098331beea045da
Reviewed-on: https://go-review.googlesource.com/c/go/+/267257 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Trust: Michael Pratt <mpratt@google.com>
cmd/go/internal/modget: resolve paths at the requested versions
Previously, we resolved each argument to 'go get' to a package path or
module path based on what was in the build list at existing versions,
even if the argument specified a different version explicitly. That
resulted in bugs like #37438, in which we variously resolved the wrong
version or guessed the wrong argument type for what is unambiguously a
package argument at the requested version.
We were also using a two-step upgrade/downgrade algorithm, which could
not only upgrade more that is strictly necessary, but could also
unintentionally upgrade *above* the requested versions during the
downgrade step.
This change instead uses an iterative approach, with an explicit
disambiguation step for the (rare) cases where an argument could match
the same package path in multiple modules. We use a hook in the
package loader to halt package loading as soon as an incorrect version
is found — preventing over-resolving — and verify that the result
after applying downgrades successfully obtained the requested versions
of all modules.
Making 'go get' be correct and usable is especially important now that
we are defaulting to read-only mode (#40728), for which we are
recommending 'go get' more heavily.
While I'm in here refactoring, I'm also reworking the API boundary
between the modget and modload packages. Previously, the modget
package edited the build list directly, and the modload package
accepted the edited build list without validation. For lazy loading
(#36460), the modload package will need to maintain additional
metadata about the requirement graph, so it needs tighter control over
the changes to the build list.
As of this change, modget no longer invokes MVS directly, but instead
goes through the modload package. The resulting API gives clearer
reasons in case of updates, which we can use to emit more useful
errors.
Fixes #37438
Updates #36460
Updates #40728
Change-Id: I596f0020f3795870dec258147e6fc26a3292c93a
Reviewed-on: https://go-review.googlesource.com/c/go/+/263267
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
Bryan C. Mills [Sat, 31 Oct 2020 01:00:41 +0000 (21:00 -0400)]
cmd/go/internal/modload: fix (*mvsReqs).Max when the second argument is the empty string
As far as I can tell, this bug had gone unnoticed because everything
that uses Max so far happened to only ever present the empty string as
the first argument.
For #37438
Change-Id: Ie8c42313157d2c2c17e4058c53b5bb026b95a1c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/266860
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Bryan C. Mills [Fri, 30 Oct 2020 16:24:51 +0000 (12:24 -0400)]
cmd/go/internal/modload: return a module-only result from QueryPattern
This allows a single QueryPattern call to resolve a path that could be
either a package or a module. It is important to be able to make a
single QueryPattern call — rather than a QueryPattern followed by a
Query for the specific module path — to provide appropriate fallback
behavior: if the proxy returns package results but does not contain a
module result, we don't want to fall back to the next proxy to look
for the (probably-nonexistent) module.
For #37438
Change-Id: I419b8bb3ab4565f443bb5cee9a8b206f453b9801
Reviewed-on: https://go-review.googlesource.com/c/go/+/266657
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Rob Findley [Tue, 3 Nov 2020 22:54:24 +0000 (17:54 -0500)]
go/types: simplify error messages for untyped value assignability
CL 242083 corrected an inaccurate error message related to the
assignability of untyped constant values. Previously the error message
was of the form "cannot convert ... to ...", which is misleading when
there is no explicit conversion in the syntax. The new error message
corrected this to "cannot use ... as ... in ...", but also appended an
inner error message that can be quite verbose. For example:
cannot use "123" (untyped string constant) as int value in assignment:
cannot convert "123" (untyped string constant) to int"
This might be more accurate, but is a regression in readability. Correct
this by only including the inner error message in the rare cases where
it is helpful: if the constant value overflows or is truncated.
For golang/go#22070
Change-Id: I8b8ee6ef713f64facc319894be09398b0b5ea500
Reviewed-on: https://go-review.googlesource.com/c/go/+/267717
Run-TryBot: Robert Findley <rfindley@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Martin Möhrmann [Thu, 5 Nov 2020 04:59:34 +0000 (05:59 +0100)]
internal/cpu: fix and cleanup ARM64 cpu feature fields and options
Remove all cpu features from the ARM64 struct that are not initialized
to reduce cache lines used and to avoid those features being
accidentially used without actual detection if they are present.
Add missing option to mask the CPUID feature.
Change-Id: I94bf90c0655de1af2218ac72117ac6c52adfc289
Reviewed-on: https://go-review.googlesource.com/c/go/+/267658
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Martin Möhrmann <moehrmann@google.com>
Emmanuel T Odeke [Tue, 26 May 2020 21:14:08 +0000 (14:14 -0700)]
net/mail: avoid ParseDate confusion if day name includes "T"
Fixes the check for RFC 5322 "obsolete time zone" to ensure
that we correctly extract the entire date from the "T" of the
implied time zone.
Obsolete Time zones come in the form:
* GMT
* PST
* MDT
etc, as per Section 4.3 of RFC 5322,
https://tools.ietf.org/html/rfc5322#section-4.3.
The prior check from CL 117596 erronenously used strings.Index
which selects the first "T", and that meant that dates containing
days "Tue" or "Thu" could not be parsed.
We also now deal with "T" in the CFWS "Comment Folding White Space".
Thus we'll now accept dates:
* Thu, 20 Nov 1997 09:55:06 MDT
* Thu, 20 Nov 1997 09:55:06 MDT (MDT)
* Fri, 21 Nov 1997 09:55:06 MDT (This comment)
* Fri, 21 Nov 1997 09:55:06 MDT (MDT
Fixes #39260
Change-Id: I6d59d99bc4f05a82582c826b5c5a080a25fd999b
Reviewed-on: https://go-review.googlesource.com/c/go/+/235200
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Johan Knutzen [Wed, 21 Oct 2020 18:45:03 +0000 (18:45 +0000)]
syscall: expose bInheritHandles of CreateProcess
Certain use cases require this parameter to be false. This includes
spawning a child process in a different windows session than session 0.
Docs regarding the behavior of this parameter to CreateProcess:
https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessa
Fixes #42098
Change-Id: If998f57d6f2962824aacbee75e1b508b255ab293
GitHub-Last-Rev: 584eb13e36a3ef7e0cd959295e92fb129f21d1f8
GitHub-Pull-Request: golang/go#41957
Reviewed-on: https://go-review.googlesource.com/c/go/+/261917
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
Russ Cox [Thu, 29 Oct 2020 14:57:38 +0000 (10:57 -0400)]
cmd/go: add GOVCS setting to control version control usage
The go command runs commands like git and hg to download modules.
In the past, we have had problems with security bugs in version
control systems becoming security bugs in “go get”.
The original modules draft design removed use of these commands
entirely, saying:
> We want to move away from invoking version control tools such as bzr,
> fossil, git, hg, and svn to download source code. These fragment the
> ecosystem: packages developed using Bazaar or Fossil, for example, are
> effectively unavailable to users who cannot or choose not to install
> these tools. The version control tools have also been a source of
> exciting security problems. It would be good to move them outside the
> security perimeter.
The removal of these commands was not possible in the end: being able
to fetch directly from Git repos is too important, especially for
closed source. But the security exposure has not gone away.
We remain vulnerable to problems in VCS systems, especially the less
scrutinized ones.
This change adds a GOVCS setting to let users control which version
control systems are allowed by default.
It also changes the default allowed version control systems to git and hg
for public code and any version control system for private code
(import path or module path matched by the GOPRIVATE setting).
See the changes in alldocs.go for detailed documentation.
See #41730 for proposal and discussion.
Fixes #41730.
Change-Id: I1999ddf7445b36a7572965be5897c7a1ff7f4265
Reviewed-on: https://go-review.googlesource.com/c/go/+/266420
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Russ Cox [Thu, 29 Oct 2020 16:20:53 +0000 (12:20 -0400)]
path/filepath: add WalkDir
WalkDir is like Walk but can use ReadDir to read directories,
instead of Readdirnames + Lstat on every entry,
which is usually a significant performance improvement.
(The Lstat can still happen if the walk function calls d.Info.)
Fixes #42027.
Change-Id: Ie11024b23be2656e320d41fd81ff0d8810aa729e
Reviewed-on: https://go-review.googlesource.com/c/go/+/266240
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Paul E. Murphy [Tue, 3 Nov 2020 22:59:25 +0000 (16:59 -0600)]
cmd/asm: fix rlwnm reg,reg,const,reg encoding on ppc64
The wrong value for the first reg parameter was selected.
Likewise the wrong opcode was selected. This should match
rlwnm (rrr type), not rlwinm (irr type).
Similarly, fix the optab matching rules so clrlslwi does
not match reg,reg,const,reg arguments. This is not a valid
operand combination for clrlslwi.
Russ Cox [Wed, 4 Nov 2020 11:55:59 +0000 (06:55 -0500)]
os: avoid nil returns from Readdirnames, Readdir, ReadDir
The refactoring of this code while adding ReadDir stopped
pre-allocating a 100-entry slice for the results.
That seemed like a good idea in general, since many
directories have nowhere near 100 entries, but it had the
side effect of returning a nil slice for an empty directory.
Some “golden” tests that are too sensitive about nil vs not
inside Google broke because Readdirnames(-1) was now
returning nil instead of []string{} on an empty directory.
It seems likely there are other such tests in the wild, and
it doesn't seem worth breaking them.
This commit restores the non-nil-ness of the old result,
without restoring the excessive preallocation.
Fixes #42367.
Change-Id: I2be72030ac703346e859a97c2d4e456fadfce9b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/267637
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Curtis La Graff [Tue, 3 Nov 2020 20:26:17 +0000 (20:26 +0000)]
cmd/go/internal/modfetch/codehost: add support for new fossil info hash prefix
A recent update of the Fossil SCM application changes
the line prefix when the fossil info command is used.
Instead of the revision hash starting with "uuid:", it has been
changed to "hash:".
To support older and new versions, fossilParseStat will
now check for either version of the prefix when attempting
to find the line containing the hash of the desired revision.
Fixes #42323
Change-Id: I6eff49f9989b37b295322a8569e222a1fd02f6e3
GitHub-Last-Rev: f4e6652307732fd3213684f13e42d17528271d88
GitHub-Pull-Request: golang/go#42324
Reviewed-on: https://go-review.googlesource.com/c/go/+/267080
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Bryan C. Mills <bcmills@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Joel Sing [Tue, 3 Nov 2020 13:29:39 +0000 (00:29 +1100)]
cmd/dist: enable additional cgo tests on openbsd architectures
OpenBSD gained __thread support quite some time ago.
Change-Id: I7de0a5c0c4de1a7ce59e48ac939fc2daf56be8f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/267318
Trust: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
Jonathan Swinney [Mon, 2 Nov 2020 16:36:10 +0000 (16:36 +0000)]
cmd/asm: rename arm64 instructions LDANDx to LDCLRx
The LDANDx instructions were misleading because they correspond to the
mnemonic LDCLRx as defined in the Arm Architecture Reference Manual for
Armv8. This changes the assembler to use the same mnemonic as the GNU
assembler and the manual.
Joel Sing [Sun, 1 Nov 2020 16:58:08 +0000 (03:58 +1100)]
runtime: allow physical page aligned stacks to be allocated
Add a physPageAlignedStack boolean which if set, results in over allocation
by a physical page, the allocation being rounded to physical page alignment
and the unused memory surrounding the allocation being freed again.
OpenBSD/octeon has 16KB physical pages and requires stacks to be physical page
aligned in order for them to be remapped as MAP_STACK. This change allows Go
to work on this platform.
Based on a suggestion from mknyszek in issue #41008.
Updates #40995
Fixes #41008
Change-Id: Ia5d652292b515916db473043b41f6030094461d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/266919
Trust: Joel Sing <joel@sing.id.au> Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Ben Hoyt [Tue, 3 Nov 2020 21:13:42 +0000 (10:13 +1300)]
strconv: revert ParseFloat/ParseComplex error on incorrect bitSize
This is a partial revert of https://go-review.googlesource.com/c/go/+/248219
because we found that a non-trivial amount of code erroneously calls
ParseFloat(s, 10) or even ParseFloat(s, 0) and expects it to work --
before that change was merged, ParseFloat accepted a bitSize of
anything other than 32 or 64 to mean 64 (and ParseComplex was similar).
So revert that behavior to avoid breaking people's code, and add tests
for this.
I may add a vet check to flag ParseFloat(s, not_32_or_64) in a later
change.
See #42297 for more details.
Change-Id: I4bc0156bd74f67a39d5561b6e5fde3f2d20bd622
Reviewed-on: https://go-review.googlesource.com/c/go/+/267319
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Tue, 3 Nov 2020 05:31:06 +0000 (21:31 -0800)]
cmd/go: in cgoflags, permit -DX1, prohibit -Wp,-D,opt
Restrict -D and -U to ASCII C identifiers, but do permit trailing digits.
When using -Wp, prohibit commas in -D values.
Change-Id: Ibfc4dfdd6e6c258e131448e7682610c44eee9492
Reviewed-on: https://go-review.googlesource.com/c/go/+/267277
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Joel Sing [Tue, 19 May 2020 08:56:01 +0000 (18:56 +1000)]
cmd/dist,cmd/go,runtime: add support for cgo on linux/riscv64
Fixes #36641
Change-Id: I51868d83ce341d78d33b221d184c5a5110c60d14
Reviewed-on: https://go-review.googlesource.com/c/go/+/263598
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Change-Id: Ia14659a9c44f9e1504eb88b5693932b9dd4bb286
Reviewed-on: https://go-review.googlesource.com/c/go/+/252939
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
kemalelmizan [Thu, 27 Aug 2020 23:39:43 +0000 (06:39 +0700)]
cmd/doc: adding validation before adding comment marker
Previous fix in issue #20929 for adding comment marker does
not check whether string field have // prefix or not.
This commit ensures string field does not contain // before adding
prefix to the line. Test also included in this commit.
Fixes #40992
Change-Id: Ibc5e8ef147eeb2ed732fb9e19815c8b21fcfb2ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/251237
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Michael Anthony Knyszek [Mon, 2 Nov 2020 19:03:16 +0000 (19:03 +0000)]
runtime: decouple consistent stats from mcache and allow P-less update
This change modifies the consistent stats implementation to keep the
per-P sequence counter on each P instead of each mcache. A valid mcache
is not available everywhere that we want to call e.g. allocSpan, as per
issue #42339. By decoupling these two, we can add a mechanism to allow
contexts without a P to update stats consistently.
In this CL, we achieve that with a mutex. In practice, it will be very
rare for an M to update these stats without a P. Furthermore, the stats
reader also only needs to hold the mutex across the update to "gen"
since once that changes, writers are free to continue updating the new
stats generation. Contention could thus only arise between writers
without a P, and as mentioned earlier, those should be rare.
A nice side-effect of this change is that the consistent stats acquire
and release API becomes simpler.
Fixes #42339.
Change-Id: Ied74ab256f69abd54b550394c8ad7c4c40a5fe34
Reviewed-on: https://go-review.googlesource.com/c/go/+/267158
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Mon, 2 Nov 2020 16:58:38 +0000 (16:58 +0000)]
runtime: make getMCache inlineable
This change moves the responsibility of throwing if an mcache is not
available to the caller, because the inlining cost of throw is set very
high in the compiler. Even if it was reduced down to the cost of a usual
function call, it would still be too expensive, so just move it out.
This choice also makes sense in the context of #42339 since we're going
to have to handle the case where we don't have an mcache to update stats
in a few contexts anyhow.
Also, add getMCache to the list of functions that should be inlined to
prevent future regressions.
getMCache is called on the allocation fast path and because its not
inlined actually causes a significant regression (~10%) in some
microbenchmarks.
Fixes #42305.
Change-Id: I64ac5e4f26b730bd4435ea1069a4a50f55411ced
Reviewed-on: https://go-review.googlesource.com/c/go/+/267157
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Michael Matloob [Mon, 2 Nov 2020 19:19:12 +0000 (19:19 +0000)]
cmd/go: support cgo files in overlays
This is a roll-forward of golang.org/cl/262618, which was reverted in
golang.org/cl/267037. The only differences between this CL and the
original are the three calls to fflush from the C files in
build_overlay.txt, to guarantee that the string we're expecting is
actually written out.
This requires rewriting the paths of the files passed to the cgo tool
toolchain to use the overlaid paths instead of the disk paths of
files. Because the directories of the overlaid paths don't exist in
general, the cgo tool have been updated to run in base.Cwd instead of
the package directory.
For #39958
Change-Id: If7e5e057c62c0c22ddb724f9fe650902fc5f4832
Reviewed-on: https://go-review.googlesource.com/c/go/+/267197 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Michael Matloob <matloob@golang.org>
Austin Clements [Sun, 1 Nov 2020 22:48:42 +0000 (17:48 -0500)]
runtime: default to MADV_DONTNEED on Linux
In Go 1.12, we changed the runtime to use MADV_FREE when available on
Linux (falling back to MADV_DONTNEED) in CL 135395 to address issue
#23687. While MADV_FREE is somewhat faster than MADV_DONTNEED, it
doesn't affect many of the statistics that MADV_DONTNEED does until
the memory is actually reclaimed under OS memory pressure. This
generally leads to poor user experience, like confusing stats in top
and other monitoring tools; and bad integration with management
systems that respond to memory usage.
We've seen numerous issues about this user experience, including
#41818, #39295, #37585, #33376, and #30904, many questions on Go
mailing lists, and requests for mechanisms to change this behavior at
run-time, such as #40870. There are also issues that may be a result
of this, but root-causing it can be difficult, such as #41444 and
#39174. And there's some evidence it may even be incompatible with
Android's process management in #37569.
This CL changes the default to prefer MADV_DONTNEED over MADV_FREE, to
favor user-friendliness and minimal surprise over performance. I think
it's become clear that Linux's implementation of MADV_FREE ultimately
doesn't meet our needs. We've also made many improvements to the
scavenger since Go 1.12. In particular, it is now far more prompt and
it is self-paced, so it will simply trickle memory back to the system
a little more slowly with this change. This can still be overridden by
setting GODEBUG=madvdontneed=0.
Fixes #42330 (meta-issue).
Fixes #41818, #39295, #37585, #33376, #30904 (many of which were
already closed as "working as intended").
Bryan C. Mills [Mon, 2 Nov 2020 14:12:28 +0000 (09:12 -0500)]
all: update dependency on golang.org/x/sys and regenerate Windows syscalls
Steps run:
$ cd $(go env GOROOT)/src
$ go get -d golang.org/x/sys
$ go mod tidy
$ go mod vendor
$ go generate syscall/... internal/syscall/...
$ cd cmd
$ go get -d golang.org/x/sys
$ go mod tidy
$ go mod vendor
$ cd ..
$ git add .
This change subsumes CL 260860.
For #36905
Change-Id: I7c677c6aa1ad61b9cbd8cf9ed208ed5a30f29c87
Reviewed-on: https://go-review.googlesource.com/c/go/+/267103
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Bryan C. Mills [Mon, 12 Oct 2020 14:51:34 +0000 (10:51 -0400)]
syscall: switch go:generate directives back to mksyscall_windows.go
Adjust mksyscall_windows.go to activate module mode and set
-mod=readonly, and to suppress its own deprecation warning when run
from within GOROOT/src.
We can't vendor the mkwinsyscall tool in to the std module directly,
because std-vendored dependencies (unlike the dependencies of all
other modules) turn into actual, distinct packages in 'std' when
viewed from outside the 'std' module. We don't want to introduce a
binary in the 'std' meta-pattern, but we also don't particularly want
to add more special-cases to the 'go' command right now when we have
an existing wrapper program that can do the job.
I also regenerated the affected packages to ensure that they are
consistent with the current version of mksyscall, which produced some
declaration-order changes in
internal/syscall/windows/zsyscall_windows.go.
Fixes #41916
Updates #25922
Change-Id: If6e6f8ba3dd372a7ecd6820ee6c0ca38d55f0f35
Reviewed-on: https://go-review.googlesource.com/c/go/+/261499
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Jonathan Swinney [Fri, 30 Oct 2020 18:46:23 +0000 (18:46 +0000)]
runtime: improve memmove performance on arm64
Replace the memmove implementation for moves of 17 bytes or larger
with an implementation from ARM optimized software. The moves of 16
bytes or fewer are unchanged, but the registers used are updated to
match the rest of the implementation.
This implementation makes use of new optimizations:
- software pipelined loop for large (>128 byte) moves
- medium size moves (17..128 bytes) have a new implementation
- address realignment when src or dst is unaligned
- preference for aligned src (loads) or dst (stores) depending on CPU
To support preference for aligned loads or aligned stores, a new CPU
flag is added. This flag indicates that the detected micro
architecture performs better with aligned loads. Some tested CPUs did
not exhibit a significant difference and are left with the default
behavior of realigning based on the destination address (stores).
Cherry Zhang [Sun, 1 Nov 2020 22:05:32 +0000 (17:05 -0500)]
cmd/compile: remove racefuncenterfp when it is not needed
We already remove racefuncenter and racefuncexit if they are not
needed (i.e. the function doesn't have any other race calls).
racefuncenterfp is like racefuncenter but used on LR machines.
Remove unnecessary racefuncenterfp as well.
Change-Id: I65edb00e19c6d9ab55a204cbbb93e9fb710559f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/267099
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Change-Id: Iec1e236ba793f24394442d04eb846f8a73ab2e68
Reviewed-on: https://go-review.googlesource.com/c/go/+/267037
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com>
Emmanuel T Odeke [Fri, 29 May 2020 09:17:38 +0000 (02:17 -0700)]
cmd/vet: bring in pass to catch invalid uses of testing.T in goroutines
Add "go/analysis/passes/testinggoroutine" from x/tools and vendor its source in.
This pass will catch misuses of:
* testing.T.Fail*
* testing.T.Fatal*
* testing.T.Skip*
inside goroutines explicitly started by the go keyword.
The pass was implemented in CL 212920.
While here, found 2 misuses in:
* database/sql/sql_test.go
* runtime/syscall_windows_test.go
and fixed them in CL 235527.
Fixes #5746
Change-Id: I1740ad3f1d677bb5d78dc5d8d66bac6ec287a2b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/235677
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Matthew Dempsky [Fri, 30 Oct 2020 19:58:28 +0000 (12:58 -0700)]
cmd/compile: fix recognition of unnamed return variables
In golang.org/cl/266199, I reused the existing code in inlining that
recognizes anonymous variables. However, it turns out that code
mistakenly recognizes anonymous return parameters as named when
inlining a function from the same package.
The issue is funcargs (which is only used for functions parsed from
source) synthesizes ~r names for anonymous return parameters, but
funcargs2 (which is only used for functions imported from export data)
does not.
This CL fixes the behavior so that anonymous return parameters are
handled identically whether a function is inlined within the same
package or across packages. It also adds a proper cross-package test
case demonstrating #33160 is fixed in both cases.
Change-Id: Iaa39a23f5666979a1f5ca6d09fc8c398e55b784c
Reviewed-on: https://go-review.googlesource.com/c/go/+/266719
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: David Chase <drchase@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Colin Arnott [Thu, 16 Jul 2020 17:42:47 +0000 (17:42 +0000)]
os: export errFinished as ErrProcessDone
(*Process).Signal returns an error sentinel, previously errFinished,
when (*Process).done or syscall.ESRCH. Callers would like the ability to
test for this state, so the value has been exported as ErrProcessDone.
Fixes #39444
Change-Id: I510e7647cc032af290180de5149f35ab7b09a526
Reviewed-on: https://go-review.googlesource.com/c/go/+/242998
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Joel Sing [Mon, 26 Oct 2020 07:35:40 +0000 (18:35 +1100)]
cmd/compile: avoid unnecessary sign/zero extension for consts on riscv64
Sign extension for consts is unnecessary and zero extension for consts can be avoided
via casts. This removes over 16,000 instructions from the Go binary, in part because it
allows for better zero const absorbtion in blocks - for example,
`(BEQ (MOVBU (MOVBconst [0])) cond yes no)` now becomes `(BEQZ cond yes no)` when
this change is combined with existing rules.
Change-Id: I27e791bfa84869639db653af6119f6e10369ba3d
Reviewed-on: https://go-review.googlesource.com/c/go/+/265041
Trust: Joel Sing <joel@sing.id.au> Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Michael Matloob [Thu, 15 Oct 2020 15:50:31 +0000 (11:50 -0400)]
cmd/go: support cgo files in overlays
This requires rewriting the paths of the files passed to the cgo tool
toolchain to use the overlaid paths instead of the disk paths of
files. Because the directories of the overlaid paths don't exist in
general, the cgo tool have been updated to run in base.Cwd instead of
the package directory.
For #39958
Change-Id: I8986de889f56ecc2e64fa69f5f6f29fa907408f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/262618
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Michael Matloob [Tue, 27 Oct 2020 21:51:58 +0000 (17:51 -0400)]
cmd/cgo: add -trimpath flag allowing paths to be rewritten in outputs
cmd/cgo now has a -trimpath flag that behaves the same as the
-trimpath flag to cmd/compile. This will be used to correct paths
to cgo files that are overlaid.
The code that processes trimpath in internal/objapi has been slightly
refactored because it's currently only accessible via AbsFile, which
does some additional processing to the path names. Now an
ApplyRewrites function is exported that just applies the trimpath
rewrites.
Also remove unused srcfile argument to cmd/cgo.(*Package).godefs.
For #39958
Change-Id: I497d48d0bc2fe1f6ab2b5835cbe79f15b839ee59
Reviewed-on: https://go-review.googlesource.com/c/go/+/266358
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Dan Scales [Thu, 29 Oct 2020 22:31:16 +0000 (15:31 -0700)]
cmd/compile: fmt improvements for AST nodes and some comments on AST nodes
Changed fmt.go to print out some extra information for various kinds of
Nodes. This includes some extra (small) info in the %j (jconv) output,
and some missing sections (such as Dcls and the body of a closure) in
nodedump().
Also, added some extra doc comments for a few Node types in syntax.go
Change-Id: I2ec7184e2abe0d5fbe3fb5a2506da7c7b06f2fb1
Reviewed-on: https://go-review.googlesource.com/c/go/+/266437
Run-TryBot: Dan Scales <danscales@google.com>
Trust: Dan Scales <danscales@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Michael Anthony Knyszek [Mon, 20 Jul 2020 18:19:56 +0000 (18:19 +0000)]
runtime: release worldsema with a direct G handoff
Currently worldsema is not released with direct handoff, so the
semaphore is an unfair synchronization mechanism. If, for example,
ReadMemStats is called in a loop, it can continuously stomp on attempts
by the GC to stop the world.
Note that it's specifically possible for ReadMemStats to delay a STW to
end GC since ReadMemStats is able to STW during a GC since #19112 was
fixed.
While this particular case is unlikely and the right answer in most
applications is to simply not call such an expensive operation in a
loop, this pattern is used often in tests.
Fixes #40459.
Change-Id: Ia4a54f0fd956ea145a319f9f06c4cd37dd52fd8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/243977
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: David Chase <drchase@google.com>
Michael Pratt [Fri, 30 Oct 2020 19:22:52 +0000 (15:22 -0400)]
runtime: tighten systemstack in lock assertions
We use systemstack on the locking path to avoid stack splits which could
cause locks to be recorded out of order (see comment on lockWithRank).
This concern is irrelevant on lock assertions, where we simply need to
see if a lock is held and don't care if another is taken in the
meantime. Thus we can simply drop these unless we actually need to
crash.
Updates #40677
Change-Id: I85d730913a59867753ee1ed0386f8c5efda5c432
Reviewed-on: https://go-review.googlesource.com/c/go/+/266718
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Trust: Michael Pratt <mpratt@google.com>
Michael Pratt [Thu, 29 Oct 2020 20:03:57 +0000 (16:03 -0400)]
runtime: elide timer re-check if P has no timers
In golang.org/cl/264477, I missed this new block after rebasing past
golang.org/cl/232298. These fields must be zero if there are no timers.
Updates #28808
Updates #18237
Change-Id: I2d9e1cbf326497c833daa26b11aed9a1e12c2270
Reviewed-on: https://go-review.googlesource.com/c/go/+/266367
Run-TryBot: Michael Pratt <mpratt@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Michael Pratt [Fri, 21 Aug 2020 15:59:55 +0000 (11:59 -0400)]
runtime: add heap lock assertions
Some functions that required holding the heap lock _or_ world stop have
been simplified to simply requiring the heap lock. This is conceptually
simpler and taking the heap lock during world stop is guaranteed to not
contend. This was only done on functions already called on the
systemstack to avoid too many extra systemstack calls in GC.
Updates #40677
Change-Id: I15aa1dadcdd1a81aac3d2a9ecad6e7d0377befdc
Reviewed-on: https://go-review.googlesource.com/c/go/+/250262
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Trust: Michael Pratt <mpratt@google.com>
Michele Di Pede [Fri, 30 Oct 2020 10:55:18 +0000 (11:55 +0100)]
cmd/compile: code cleanup
Change-Id: Ibf68e663f29a5cb3b64a7d923c005c16da647769
Reviewed-on: https://go-review.googlesource.com/c/go/+/266537 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Matthew Dempsky [Fri, 30 Oct 2020 17:36:31 +0000 (10:36 -0700)]
cmd/compile: fix reassignVisitor
reassignVisitor was short-circuiting on assignment statements after
checking the LHS, but there might be further assignment statements
nested within the RHS expressions.
Fixes #42284.
Change-Id: I175eef87513b973ed5ebe6a6527adb9766dde6cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/266618
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Cherry Zhang [Thu, 29 Oct 2020 15:27:17 +0000 (11:27 -0400)]
runtime, cmd: support race detector on darwin/arm64
https://reviews.llvm.org/D90435 is the counterpart in LLVM TSAN.
race_linux_arm64.syso is built with LLVM commit 00da38ce2d36c07f12c287dc515d37bb7bc410e9 on a macOS/ARM64 machine.
(It is not built on a builder with golang.org/x/build/cmd/racebuild
as we don't have darwin/arm64 builder for now.)
Updates #38485.
Change-Id: I391efdacd9480197e308370bfccd05777deb4aee
Reviewed-on: https://go-review.googlesource.com/c/go/+/266373
Trust: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Bryan C. Mills [Thu, 29 Oct 2020 20:36:50 +0000 (16:36 -0400)]
cmd/go/internal/modload: handle NotExist errors in (*mvsReqs).Previous
Previous is used during downgrading. If the module proxy does not
advertise any versions (for example, because it contains only
pseudo-versions), then Previous should return "none" instead of a
non-nil error.
For #37438
Change-Id: I4edfec19cfeb3ffe50df4979f99a01321c442509
Reviewed-on: https://go-review.googlesource.com/c/go/+/266370
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Bryan C. Mills [Thu, 29 Oct 2020 20:06:19 +0000 (16:06 -0400)]
cmd/go: make TestScript/mod_get_patchmod self-contained
I find it pretty difficult to reason about test-dependency modules
when they aren't in the same file as the rest of the test.
Now that 'go get' supports replacements (CL 258220 and CL 266018),
we can localize tests that need 'go get' but don't specifically depend
on module proxy semantics.
For #36460
For #37438
Change-Id: Ib37a6c170f251435399dfc23e60d96681a81eadc
Reviewed-on: https://go-review.googlesource.com/c/go/+/266369
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Than McIntosh [Fri, 16 Oct 2020 13:22:31 +0000 (09:22 -0400)]
cmd/link: emit include directories in DWARF line table prologue
This patch changes the way the linker emits the DWARF line table
prologue, specifically the file table. Previously files were left
unmodified, and the directory table was empty. For each compilation
unit we now scan the unit file table and build up a common set of
directories, emit them into the directory table, and then emit file
entries that refer to the dirs. This provides a modest binary size
savings.
[where the value following the section name above is the section size
in hex, so roughly a 30% decrease in this case.]
The actual savings will depend on the length of the pathnames
involved, so it's hard to really pin down how much savings we'll see
here. In addition, emitting the files this way reduces the
"compressibility" of the line table, so there could even be cases
where we don't win at all.
Updates #6853, #19784, #36495.
Change-Id: I298d8561da5ed3ebc9d38aa772874851baa2f4f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/263017
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
Trust: Than McIntosh <thanm@google.com>
Ian Lance Taylor [Thu, 29 Oct 2020 23:23:27 +0000 (16:23 -0700)]
runtime: simplify nobarrierWakeTime
Also use the simplified nobarrierWakeTime in findrunnable, as it no
longer needs the current time.
Change-Id: I77b125d6a184dde0aeb517fc068164c274f0a046
Reviewed-on: https://go-review.googlesource.com/c/go/+/266304
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Than McIntosh [Wed, 14 Oct 2020 12:36:11 +0000 (08:36 -0400)]
reflect,runtime: use internal ABI for selected ASM routines, attempt 2
[This is a roll-forward of CL 262319, with a fix for some Darwin test
failures].
Change the definitions of selected runtime assembly routines
from ABI0 (the default) to ABIInternal. The ABIInternal def is
intended to indicate that these functions don't follow the existing Go
runtime ABI. In addition, convert the assembly reference to
runtime.main (from runtime.mainPC) to ABIInternal. Finally, for
functions such as "runtime.duffzero" that are called directly from
generated code, make sure that the compiler looks up the correct
ABI version.
This is intended to support the register abi work, however these
changes should not have any issues even when GOEXPERIMENT=regabi is
not in effect.
Updates #27539, #40724.
Change-Id: Idf507f1c06176073563845239e1a54dad51a9ea9
Reviewed-on: https://go-review.googlesource.com/c/go/+/266638
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Alberto Donizetti [Thu, 29 Oct 2020 11:03:18 +0000 (12:03 +0100)]
cmd/compile: replace int32(a.Off()) calls with a.Off32()
Change-Id: Ib833954e8400a63624359a81f5196ca9080199a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266081
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Rob Findley [Fri, 30 Oct 2020 15:35:48 +0000 (11:35 -0400)]
go/types: reorganize error codes into categories
In CL 264179, some reorganization of error codes was deferred in order
to minimize diffs between patch-sets.
This CL reorganizes the error codes as discussed. It is a pure
reordering, with no material changes other than the changing of internal
const values.
For #42290
Change-Id: I0e9b421a92e96b19e53039652f8de898c5255290
Reviewed-on: https://go-review.googlesource.com/c/go/+/266637
Run-TryBot: Robert Findley <rfindley@google.com>
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Robert Griesemer [Wed, 7 Oct 2020 18:33:11 +0000 (11:33 -0700)]
spec: split shift examples into groups for 32- and 64-bit ints
In the current (pre-CL) version of the spec, the 2nd last shift
example appears to be using the array declared in the last example.
On a 32-bit platform, that array would have length 0, which would
lead to a panic in the 2nd last example. Also, if this code were
inside a function, it wouldn't compile (array declared after use).
Use an explicitly declared array for that specific shift example.
Also, split out all cases that produce different results for 32-
vs 64-bit ints.
Fixes #41835.
Change-Id: Ie45114224509e4999197226f91f7f6f934449abb
Reviewed-on: https://go-review.googlesource.com/c/go/+/260398
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Michael Pratt [Tue, 13 Oct 2020 22:11:26 +0000 (18:11 -0400)]
runtime: simplify gcBgMarkWorker preemption
gcBgMarkWorker G's are primarily scheduled by findRunnableGCWorker, but
that no longer needs to be strictly enforced. Temporary preemption to a
runq is fine when the P is not in use.
We still releasem in gopark in the normal case for efficiency: if
gcDrain stops because gp.preempt is set, then gopark would always
preempt. That is fine, but inefficient, since it will reschedule simply
to park again. Thus, we keep releasem in unlockf to skip this extra
cycle.
Change-Id: I6d1a42e3ca41b76227142a6b5bfb376c9213e3c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/262349
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Trust: Michael Pratt <mpratt@google.com>
Michael Pratt [Tue, 13 Oct 2020 16:39:13 +0000 (12:39 -0400)]
runtime: manage gcBgMarkWorkers with a global pool
Background mark workers perform per-P marking work. Currently each
worker is assigned a P at creation time. The worker "attaches" to the P
via p.gcBgMarkWorker, making itself (usually) available to
findRunnableGCWorker for scheduling GC work.
While running gcMarkDone, the worker "detaches" from the P (by clearing
p.gcBgMarkWorker), since it may park for other reasons and should not be
scheduled by findRunnableGCWorker.
Unfortunately, this design is complex and difficult to reason about. We
simplify things by changing the design to eliminate the hard P
attachment. Rather than workers always performing work from the same P,
workers perform work for whichever P they find themselves on. On park,
the workers are placed in a pool of free workers, which each P's
findRunnableGCWorker can use to run a worker for its P.
Now if a worker parks in gcMarkDone, a P may simply use another worker
from the pool to complete its own work.
The P's GC worker mode is used to communicate the mode to run to the
selected worker. It is also used to emit the appropriate worker
EvGoStart tracepoint. This is a slight change, as this G may be
preempted (e.g., in gcMarkDone). When it is rescheduled, the trace
viewer will show it as a normal goroutine again. It is currently a bit
difficult to connect to the original worker tracepoint, as the viewer
does not display the goid for the original worker (though the data is in
the trace file).
Change-Id: Id7bd3a364dc18a4d2b1c99c4dc4810fae1293c1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/262348
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Trust: Michael Pratt <mpratt@google.com>
Rob Findley [Mon, 19 Oct 2020 15:51:17 +0000 (11:51 -0400)]
go/types: add internal error codes
Tools using go/types sometimes need to implement special handling for
certain errors produced by the type-checker. They can offer suggested
fixes, expand the error position to surrounding syntax, highlight
related syntax (for example in the case of a declaration cycle), link to
additional documentation, group errors by category, or correlate errors
with signals produced by other static analysis tools.
All these require a notion of error identity. Tools need to be able to
reliably determine the nature of an error without re-implementing type
checking logic or parsing error messages. This CL is a first-pass at
adding such an identifier to types.Error: a (for the moment unexported)
field containing one of many declared errorCode constants.
A wide variety of error code constants are defined, and assigned to type
checker errors according to their 'functional equivalence', meaning that
they should be ideally be stable under refactoring.
With few exceptions, each error code is documented with an example that
produces it. This is enforced by tests.
When error codes are exported they will represent quite a large API
surface. For this reason, as well as the likelihood that error codes
will change at the start, both the code field and the codes themselves
are initially unexported. gopls will read these fields using reflection
during this experimental phase. Others can obviously do the same,
provided they accept the lack of forward compatibility.
For #42290
Change-Id: I15e3c2bffd2046c20297b1857057d421f633098a
Reviewed-on: https://go-review.googlesource.com/c/go/+/264179
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
Cherry Zhang [Thu, 29 Oct 2020 19:50:53 +0000 (15:50 -0400)]
runtime: allocate at desired address when race detector is on
Currently, on all supported platforms, the race detector (LLVM
TSAN) expects the Go heap is at 0xc000000000 - 0xe000000000.
Move the raceenabled condition first, so we always allocate
there.
This means on Linux/ARM64 when race detector is on we will
allocate to 0xc000000000 - 0xe000000000, instead of 0x4000000000.
The old address is meant for 39-bit VMA. But the race detector
only supports 48-bit VMA anyway. So this is fine.
Change-Id: I51ac8eff68297b37c8c651a93145cc94f83a939d
Reviewed-on: https://go-review.googlesource.com/c/go/+/266372
Trust: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Matthew Dempsky [Thu, 29 Oct 2020 20:30:54 +0000 (13:30 -0700)]
cmd/compile: fix devirtualization of promoted interface methods
A method selector expression can pick out a method or promoted method
(represented by ODOTMETH), but it can also pick out an interface
method from an embedded interface-typed field (represented by
ODOTINTER).
In the case that we're picking out an interface method, we're not able
to fully devirtualize the method call. However, we're still able to
improve escape analysis somewhat. E.g., the included test case
demonstrates that we can optimize "i.M()" to "i.(T).I.M()", which
means the T literal can be stack allocated instead of heap allocated.
Fixes #42279.
Change-Id: Ifa21d19011e2f008d84f9624b7055b4676b6d188
Reviewed-on: https://go-review.googlesource.com/c/go/+/266300
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Robert Griesemer [Fri, 30 Oct 2020 00:09:16 +0000 (17:09 -0700)]
strings: complete Reader doc string
Follow-up on https://golang.org/cl/247523.
Change-Id: I9e91a6d77271e640d84851f2e2a4c6d2150a2b43
Reviewed-on: https://go-review.googlesource.com/c/go/+/266438
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ben Hoyt [Fri, 14 Aug 2020 10:58:49 +0000 (22:58 +1200)]
strconv: fix incorrect bit size in ParseComplex; add tests
In ParseComplex, the "size" passed to parseFloatPrefix should be 64 for
complex128, not 128. It still works because of how parseFloatPrefix
is forgiving about the size if it's not 32, but worth fixing anyway.
Make ParseComplex and ParseFloat return a bit size error for anything
other than 128 or 64 (for ParseComplex), or 64 or 32 (for ParseFloat).
Add "InvalidBitSize" tests for these cases.
Add tests for ParseComplex with bitSize==64: this is done in a similar
way to how the ParseFloat 32-bit tests work, re-using the tests for the
larger bit size.
Add tests for FormatComplex -- there were none before.
Fixes #40706
Change-Id: I16ddd546e5237207cc3b8c2181dd708eca42b04f
Reviewed-on: https://go-review.googlesource.com/c/go/+/248219
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Minux Ma <minux@golang.org>
Trust: Robert Griesemer <gri@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Norman B. Lancaster [Thu, 1 Oct 2020 19:14:04 +0000 (19:14 +0000)]
strings: complete documentation of strings.Reader
There is no documentation on a number of methods of the strings.Reader
struct, so this change adds documentation referring to the relevant
io.* interfaces implemented. This is consistent with pre-existing
documentation in this struct.
Fixes #40381
Change-Id: I3dec65ecafca5b79d85d30a676d297e5ee9ab47e
GitHub-Last-Rev: f42429946a2b90b9fbfbd1ea5943f0c50e0439b5
GitHub-Pull-Request: golang/go#40654
Reviewed-on: https://go-review.googlesource.com/c/go/+/247523
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>