Jay Conrod [Fri, 13 Sep 2019 17:58:58 +0000 (13:58 -0400)]
cmd/go: don't include package dir in cache key when -trimpath is set
The '-trimpath' flag tells 'go build' to trim any paths from the
output files that are tied to the current workspace or toolchain. When
this flag is set, we do not need to include the package directory in
the text hashed to construct the action ID for each package.
Fixes #33772
Change-Id: I20b902d2f58019709b15864ca79aa0d9255ae707
Reviewed-on: https://go-review.googlesource.com/c/go/+/195318
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Matthew Dempsky [Thu, 12 Sep 2019 17:18:03 +0000 (10:18 -0700)]
cmd/compile: trim function name prefix from escape diagnostics
This information is redundant with the position information already
provided. Also, no other -m diagnostics print out function name.
While here, report parameter leak diagnostics against the parameter
declaration position rather than the function, and use Warnl for
"moved to heap" messages.
Test cases updated programmatically by removing the first word from
every "no match for" error emitted by run.go:
go run run.go |& \
sed -E -n 's/^(.*):(.*): no match for `([^ ]* (.*))` in:$/\1!\2!\3!\4/p' | \
while IFS='!' read -r fn line before after; do
before=$(echo "$before" | sed 's/[.[\*^$()+?{|]/\\&/g')
after=$(echo "$after" | sed -E 's/(\&|\\)/\\&/g')
fn=$(find . -name "${fn}" | head -1)
sed -i -E -e "${line}s/\"${before}\"/\"${after}\"/" "${fn}"
done
Ian Lance Taylor [Thu, 12 Sep 2019 00:57:58 +0000 (17:57 -0700)]
os/signal: split up sleeps waiting for signal
Try to deflake TestNohup.
The kernel will deliver a signal as a thread returns from a syscall.
If the only active thread is sleeping, and the system is busy,
the kernel may not get around to waking up a thread to catch the signal.
Try splitting up the sleep, to give the kernel another change to deliver.
I don't know if this will help, but it seems worth a try.
Fixes #33174
Change-Id: I34b3240af706501ab8538cb25c4846d1d30d7691
Reviewed-on: https://go-review.googlesource.com/c/go/+/194879 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Lynn Boger [Tue, 25 Jun 2019 17:15:59 +0000 (13:15 -0400)]
doc: update ppc64 section for asm.html
Update the section in asm.html related to PPC64. Remove the line
that says it is in an experimental state, add a link to the
new doc.go file that has all the detail for the Go assembler for
PPC64.
Change-Id: I45d9891669e01d94e2721be576d572e02cd9d2db
Reviewed-on: https://go-review.googlesource.com/c/go/+/183840
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Lucas Bremgartner [Fri, 13 Sep 2019 19:46:50 +0000 (19:46 +0000)]
encoding/json: make Number with the ,string option marshal with quotes
Add quotes when marshaling a json.Number with the string option
set via a struct tag. This ensures that the resulting json
can be unmarshaled into the source struct without error.
Fixes #34268
Change-Id: Ide167d9dec77019554870b5957b37dc258119d81
GitHub-Last-Rev: dde81b71208be01c253bb87dbb6f81ac6e0785be
GitHub-Pull-Request: golang/go#34269
Reviewed-on: https://go-review.googlesource.com/c/go/+/195043 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Cherry Zhang [Wed, 28 Aug 2019 18:42:06 +0000 (14:42 -0400)]
test/codegen: document -all_codegen option in README
It is useful to know about the -all_codegen option for running
codegen tests for all platforms. I was puzzling that some codegen
test was not failing on my local machine or on trybot, until I
found this option.
Change-Id: I062cf4d73f6a6c9ebc2258195779d2dab21bc36d
Reviewed-on: https://go-review.googlesource.com/c/go/+/192101 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The wasm_exec.js wrapper tries to set up the argv and envp following
the UNIX conventions, but doesn't get it quite right, which can cause
runtime.goenv to crash if you get unlucky.
The main problem was that the envp array wasn't terminated with a nil
pointer, so the runtime didn't know when to stop reading the array.
This CL adds that nil pointer to the end of the envp array.
The other problem was harmless, but confusing. In the UNIX convention,
the argv array consists of argc pointers followed by a nil pointer,
followed by the envp array. However, wasm_exec.js put the environment
variable count between the two pointer arrays rather than a nil
pointer. The runtime never looks at this slot, so it didn't matter,
but the break from convention left Cherry and I trying to debug why it
*wasn't* losing any environment variables before we realized that that
layouts happened to be close enough to work. This CL switches to the
UNIX convention of simply terminating the argv array with a nil
pointer.
Tim Cooper [Mon, 15 Jul 2019 22:54:23 +0000 (17:54 -0500)]
log: add Lmsgprefix flag
The Lmsgprefix flag moves the logger's prefix from the
beginning of the line to after the log header. For example,
a logger with the prefix "LOG " and LstdFlags would output:
LOG 2009/11/10 23:00:00 entry text
Adding the Lmsgprefix flag would output:
2009/11/10 23:00:00 LOG entry text
Fixes #32062
Change-Id: I9f7c9739abeb53c424112aaeed33444eeefdfbbc
Reviewed-on: https://go-review.googlesource.com/c/go/+/186182
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Jay Conrod [Fri, 13 Sep 2019 15:10:36 +0000 (11:10 -0400)]
cmd/go: fix link error for -coverpkg in GOPATH mode
If a generated test main package transitively depends on a main
package, the main package will now always be rebuilt as a library and
will not be compiled with '-p main'.
This expands the fix for #30907, which only applied to packages with
the BuildInfo set (main packages built in module mode). Linking
multiple packages with BuildInfo caused link errors, but it appears
these errors apply to some symbols in GOPATH mode.
Fixes #34114
Change-Id: Ic1e53437942269a950dd7e45d163707922c92edd
Reviewed-on: https://go-review.googlesource.com/c/go/+/195279
Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Joel Sing [Thu, 12 Sep 2019 18:33:06 +0000 (04:33 +1000)]
cmd/link: simplify determineLinkMode
Simplify determineLinkMode by calling mustLinkExternal upfront,
then doing a first pass for LinkModeAuto, followed by a second pass
that determines if the link mode is valid.
Change-Id: I9d7668107c159f8fe330b8c05fee035bbe9875fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/195078 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ruixin Bao [Sun, 8 Sep 2019 22:50:24 +0000 (18:50 -0400)]
cmd/compile: add math/bits.Mul64 intrinsic on s390x
This change adds an intrinsic for Mul64 on s390x. To achieve that,
a new assembly instruction, MLGR, is introduced in s390x/asmz.go. This assembly
instruction directly uses an existing instruction on Z and supports multiplication
of two 64 bit unsigned integer and stores the result in two separate registers.
In this case, we require the multiplcand to be stored in register R3 and
the output result (the high and low 64 bit of the product) to be stored in
R2 and R3 respectively.
Joel Sing [Sat, 7 Sep 2019 15:56:26 +0000 (01:56 +1000)]
cmd/asm,cmd/internal/obj: initial support for riscv64 assembler
Provide the initial framework for the riscv64 assembler. For now this
only supports raw WORD instructions, but at least allows for basic
testing. Additional functionality will be added in separate changes.
This change updates the status of gccgo 8 and gccgo 9, which are now
released.
It also replaces every instance of two-spaces with one space in text
paragraphs, which is the preferred style in Go documentation.
Fixes #34167
Change-Id: I94a4d85c06281f2623d39a68db7b6c95b5867999
Reviewed-on: https://go-review.googlesource.com/c/go/+/193842 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ruixin Bao [Tue, 27 Aug 2019 19:22:28 +0000 (15:22 -0400)]
cmd/internal/obj/s390x: use 12 bit load and store instruction when possible on s390x
Originally, we default to use load and store instruction with 20 bit displacement.
However, that is not necessary. Some instructions have a displacement smaller
than 12 bit. This CL allows the usage of 12 bit load and store instruction when
that happens.
This change also reduces the size of .text section in go binary by 19 KB.
Some tests are also added to verify the functionality of the change.
Change-Id: I13edea06ca653d4b9ffeaefe8d010bc2f065c2ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/194857 Reviewed-by: Michael Munday <mike.munday@ibm.com>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/go/internal/modload: add an Unwrap method on ImportMissingError
Jay suggested this in CL 189780, and it seems semantically correct.
As far as I can tell this has no impact one way or the other right
now, but might prevent confusion (or at least give us more experience
with error handling!) in future changes.
Updates #30748
Updates #28459
Updates #30322
Change-Id: I5d7e9a08ea141628ed6a8fd03c62d0d3c2edf2bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/194817
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Bryan C. Mills [Mon, 5 Aug 2019 16:02:55 +0000 (12:02 -0400)]
cmd/go/internal/web: include snippets of plain-text server responses in error detail
For the server response to be displayed, the response must be served
as type text/plain with charset us-ascii or utf-8, and must consist of
only graphic characters and whitespace.
We truncate the server response at the first blank line or after 8
lines or a fixed number of characters, and tab-indent (if multiple
lines) to ensure that the response is offset from ordinary go command
output.
Fixes #30748
Change-Id: I0bc1d734737e456e3251aee2252463b6355e8c97
Reviewed-on: https://go-review.googlesource.com/c/go/+/189783
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Bryan C. Mills [Fri, 9 Aug 2019 21:31:23 +0000 (17:31 -0400)]
cmd/go/internal/module: in VersionError, do not wrap an existing ModuleError
VersionError wraps the given error in a ModuleError struct.
If the given error is already a ModuleError for the same path and
version, we now return it directly instead of wrapping.
This makes it safer to call VersionError if we don't know whether
a given error is already wrapped.
Updates #30748
Change-Id: I41b23f6c3ead0ec382e848696da51f478da1ad35
Reviewed-on: https://go-review.googlesource.com/c/go/+/189781
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Daniel Martí [Tue, 27 Aug 2019 20:47:37 +0000 (22:47 +0200)]
cmd/compile: stop using go/types in rulegen
Using go/types to get rid of all unused variables in CL 189798 was a
neat idea, but it was pretty expensive. go/types is a full typechecker,
which does a lot more work than we actually need. Moreover, we had to
run it multiple times, to catch variables that became unused after
removing existing unused variables.
Instead, write our own little detector for unused imports and variables.
It doesn't use ast.Walk, as we need to know what fields we're
inspecting. For example, in "foo := bar", "foo" is declared, and "bar"
is used, yet they both appear as simple *ast.Ident cases under ast.Walk.
The code is documented to explain how unused variables are detected in a
single syntax tree pass. Since this happens after we've generated a
complete go/ast.File, we don't need to worry about our own simplified
node types.
The generated code is the same, but rulegen is much faster and uses less
memory at its peak, so it should scale better with time.
With 'benchcmd Rulegen go run *.go' on perflock, we get:
name old time/op new time/op delta
Rulegen 4.00s ± 0% 3.41s ± 1% -14.70% (p=0.008 n=5+5)
name old user-time/op new user-time/op delta
Rulegen 14.1s ± 1% 10.6s ± 1% -24.62% (p=0.008 n=5+5)
name old sys-time/op new sys-time/op delta
Rulegen 318ms ±26% 263ms ± 9% ~ (p=0.056 n=5+5)
name old peak-RSS-bytes new peak-RSS-bytes delta
Rulegen 231MB ± 4% 181MB ± 3% -21.69% (p=0.008 n=5+5)
Change-Id: I8387d52818f6131357868ad348dac8c96d926191
Reviewed-on: https://go-review.googlesource.com/c/go/+/191782
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Nigel Tao [Fri, 23 Aug 2019 06:28:23 +0000 (16:28 +1000)]
compress/lzw: fix comment re high-code invariant
The listed invariant, while technically true, was misleading, and the
invariant can be tightened. We never actually get to (d.hi ==
d.overflow), due to the "d.hi--" line in the decoder.decode method.
This is a comment-only commit, changing the comment to match the code.
A follow-up commit could restore the comment, changing the code to match
the original intented invariant. But the first step is to have the
comment and the code say the same thing.
Change-Id: Ifc9f78d5060454fc107af9be298026bf3043d400
Reviewed-on: https://go-review.googlesource.com/c/go/+/191358 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
go/types: fix iota undefined after ConstDecl inside function in ConstSpec
When reaching const declaration, Checker override context iota to use
correct iota value, but does not restore the old value when exit, and
always set context iota to nil. It ends up with undefined iota after
const declaration.
To fix it, preserve the original iota value and restore it after const
declaration.
Fixes #34228
Change-Id: I42d5efb55a57e5ddc369bb72d31f1f039c92361c
Reviewed-on: https://go-review.googlesource.com/c/go/+/194737
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Wed, 11 Sep 2019 03:06:51 +0000 (20:06 -0700)]
cmd/compile: move duplicate type-case checking into typecheck
Part of the general trend of moving yyerror calls out of walk and into
typecheck.
Notably, this requires splitting test/typeswitch2.go into two files,
because now some of the errors are reported during typecheck and
others are still reported during walk; and if there were any errors
during typecheck, then cmd/compile exits without invoking walk.
Passes toolstash-check.
Change-Id: I05ee0c00b99af659ee1eef098d342d0d736cf31e
Reviewed-on: https://go-review.googlesource.com/c/go/+/194659 Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Tue, 10 Sep 2019 17:36:47 +0000 (10:36 -0700)]
cmd/compile: separate type and expression switch typechecking
While superficially type and expression switch handling seem similar
and that it would be worthwhile to unify typechecking them, it turns
out they're actually different enough that separately handling them is
fewer lines of code and easier to understand as well.
Passes toolstash-check.
Change-Id: I357d6912dd580639b6001bccdb2e227ed83c6fe9
Reviewed-on: https://go-review.googlesource.com/c/go/+/194566
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
cmd/gofmt: don't turn nil slices into empty slices during rewriting
The go/ast package uses and guarantees nil slices for optional
elements that weren't present in the parsed source code, such as the
list of return values of a function. Packages using go/ast rely on
this attribute and check for nils explicitly.
One such package is go/printer. In the presence of empty slices
instead of nil slices, it generates invalid code, such as "case :"
instead of "default:". The issues that this CL fixes are all
manifestations of that problem, each for a different syntactic
element.
Fixes #33103
Fixes #33104
Fixes #33105
Change-Id: I219f95a7da820eaf697a4ee227d458ab6e4a80bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/187917 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Jay Conrod [Tue, 10 Sep 2019 20:06:12 +0000 (16:06 -0400)]
cmd/go: strip trailing slash from versioned arguments
'go get' accepts arguments of the form path@version, and it passes
them through search.CleanPatterns before querying proxies. With this
change, CleanPatterns preserves text after '@' and will strip trailing
slashes from the patn.
Previously, we did not strip trailing slashes when a version was
present, which caused proxy base URL validation to fail. Module paths
that end with ".go" (for example, github.com/nats-io/nats.go) use
trailing slashes to prevent 'go build' and other commands from
interpreting packages as source file names, so this caused unnecessary
problems for them.
Updates #32483
Change-Id: Id3730c52089e52f1cac446617c20132a3021a808
Reviewed-on: https://go-review.googlesource.com/c/go/+/194600
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Aofei Sheng [Wed, 21 Aug 2019 21:23:41 +0000 (05:23 +0800)]
cmd/go/internal/modconv: use modules to examine instead of using only direct source control entries
Since modules now support parsing multiple forms of versions (including
commit hash and source control tag), I think modconv.ConvertLegacyConfig
no longer needs modfetch.ImportRepoRev. So I suggest that we use modules
to convert legacy config instead of using VCS directly. By doing this,
we can make the module proxy participate in the conversion process and
benefit from it (such as speeding up "go mod init" or breaking through
the firewall).
And since modconv.ConvertLegacyConfig is the only caller of
modfetch.ImportRepoRev, I think modfetch.ImportRepoRev can be removed.
Fixes #33767
Change-Id: Ic79b14fa805ed297ca1735a8498cfed2a5ddeec2
Reviewed-on: https://go-review.googlesource.com/c/go/+/191218
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Joel Sing [Sun, 8 Sep 2019 17:17:43 +0000 (03:17 +1000)]
cmd/internal/obj/riscv: fix up instruction groupings
Some of the instructions were incorrectly grouped - untangle this and
separate the RV64I instructions, which are under separate sections of
the RISC-V specification.
Bryan C. Mills [Mon, 5 Aug 2019 18:39:48 +0000 (14:39 -0400)]
cmd/go/internal/get: propagate parse errors in parseMetaGoImports
The signature of parseMetaGoImports implies that it can return an error,
but it has not done so since CL 119675. Restore the missing error check,
and remove the named return-values to avoid reintroducing this bug in the
future.
Updates #30748
Updates #21291
Change-Id: Iab19ade5b1c23c282f3c385a55ed277465526515
Reviewed-on: https://go-review.googlesource.com/c/go/+/189778
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
fanzha02 [Thu, 8 Aug 2019 01:21:46 +0000 (01:21 +0000)]
cmd/go/internal/work: use pie link mode when using MSAN on arm64
Currently, when running the "CC=clang go run -msan misc/cgo/
testsanitizers/testdata/msan.go" command on arm64, it will
report an error and the error is reported by llvm/compiler-rt/
lib/msan and it is "Make sure to compile with -fPIE and to link
with -pie".
This CL fixes this issue, using PIE link mode when using MSAN
on arm64.
This CL also updates the related document and go build help message.
Fixes #33712
Change-Id: I0cc9d95f3fa264d6c042c27a40ccbb82826922fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/190482
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/go/internal/modfetch/codehost: treat nonexistent repositories as “not found”
If a go-import directive refers to a nonexistent repository, today we
treat that as an error fetching a module that actually exists.
That makes the HTTP server responsible for determining which
repositories do or do not exist, which may in general depend on
the user's separately-stored credentials, and imposes significant
complexity on such a server, which can otherwise be very simple.
Instead, check the repository URL and/or error message to try to
determine whether the repository exists at all. If the repo does not
exist, treat its absence as a “not found” error — as if the server had
not returned it in the first place.
Updates #34094
Change-Id: I142619ff43b96d0de428cdd0b01cca828c9ba234
Reviewed-on: https://go-review.googlesource.com/c/go/+/194561 Reviewed-by: Jay Conrod <jayconrod@google.com>
Lucas Bremgartner [Tue, 10 Sep 2019 18:52:02 +0000 (18:52 +0000)]
encoding/json: fix and optimize marshal for quoted string
Since Go 1.2 every string can be marshaled to JSON without error even if it
contains invalid UTF-8 byte sequences. Therefore there is no need to use
Marshal again for the only reason of enclosing the string in double quotes.
Not using Marshal here also removes the error check as there has not been a
way for Marshal to fail anyway.
name old time/op new time/op delta
Issue34127-4 360ns ± 3% 200ns ± 3% -44.56% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
Issue34127-4 56.0B ± 0% 40.0B ± 0% -28.57% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Issue34127-4 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.008 n=5+5)
Fixes #34154
Change-Id: Ib60dc11980f9b20d8bef2982de7168943d632263
GitHub-Last-Rev: 9b0ac1d4c5318b6bf9ed7930320f2bd755f9939c
GitHub-Pull-Request: golang/go#34127
Reviewed-on: https://go-review.googlesource.com/c/go/+/193604 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ruixin(Peter) Bao [Thu, 22 Aug 2019 13:53:54 +0000 (09:53 -0400)]
cmd/compile/internal/s390x: replace 4-byte NOP with a 2-byte NOP on s390x
Added a new instruction, NOPH, with the encoding [0x0700](i.e: bcr 0, 0) and
replace the current 4-byte nop that was encoded using the WORD instruction.
This reduces the size of .text section in go binary by around 17KB and make
generated code easier to read.
Change-Id: I6a756df39e93c4415ea6d038ba4af001b8ccb286
Reviewed-on: https://go-review.googlesource.com/c/go/+/194344 Reviewed-by: Michael Munday <mike.munday@ibm.com>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Howard Zhang [Tue, 6 Aug 2019 09:42:37 +0000 (02:42 -0700)]
syscall: implement rawVforkSyscall for linux/arm64
This allows the use of CLONE_VFORK and CLONE_VM for fork/exec, preventing
"fork/exec ...: cannot allocate memory" failures from occuring when attempting
to execute commands from a Go process that has a large memory footprint.
Additionally, this should reduce the latency of fork/exec on linux/arm64.
With CLONE_VM the child process shares the same memory with the parent
process. On its own this would lead to conflicting use of the same
memory, so CLONE_VFORK is used to suspend the parent process until the
child releases the memory when switching to the new program binary
via the exec syscall. When the parent process continues to run, one
has to consider the changes to memory that the child process did,
namely the return address of the syscall function needs to be restored
from a register.
exec.Command() callers can start in a faster manner, as child process who
do exec commands job can be cloned faster via vfork than via fork on arm64.
The same problem was addressed on linux/amd64 via issue #5838.
Updates #31936
Contributed by Howard Zhang <howard.zhang@arm.com> and Bin Lu <bin.lu@arm.com>
As discussed in #32912, a crash occurs when go runtime calls a VDSO function (say
__vdso_clock_gettime) and a signal arrives to that thread.
Since VDSO functions temporarily destroy the G register (R10),
Go functions asynchronously executed in that thread (i.e. Go's signal
handler) can try to load data from the destroyed G, which causes
segmentation fault.
To fix the issue a guard is inserted in front of sigtrampgo, so that the control escapes from
signal handlers without touching G in case the signal occurred in the VDSO context.
The test case included in the patch is take from discussion in a relevant thread on github:
https://github.com/golang/go/issues/32912#issuecomment-517874531.
This patch not only fixes the issue on AArch64 but also that on 32bit ARM.
Fixes #32912
Change-Id: I657472e54b7aa3c617fabc5019ce63aa4105624a
GitHub-Last-Rev: 28ce42c4a02a060f08c1b0dd1c9a392123fd2ee9
GitHub-Pull-Request: golang/go#34030
Reviewed-on: https://go-review.googlesource.com/c/go/+/192937
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Sven Taute [Mon, 9 Sep 2019 21:56:46 +0000 (21:56 +0000)]
encoding/base32: increase performance and code reuse
Add benchmarks for the Encode/Decode functions operating on []byte and increase decoding performance by removing the calls to strings.Map/bytes.Map and reusing the newline filtering code that is used by NewDecoder.
Cut allocations in half for DecodeString.
Comparison using the new benchmarks:
name old time/op new time/op delta
Encode 16.7µs ± 1% 17.0µs ± 2% +2.25% (p=0.000 n=9+9)
EncodeToString 21.1µs ± 1% 20.9µs ± 1% -0.96% (p=0.000 n=10+10)
Decode 141µs ± 1% 54µs ± 1% -61.51% (p=0.000 n=10+10)
DecodeString 81.4µs ± 0% 54.7µs ± 1% -32.79% (p=0.000 n=9+10)
The root cause is that DeepEqual fails to cache interface values
in visited map, which is used to detect cycle. DeepEqual calls
CanAddr to check whether a value should be cached or not. However,
all values referenced by interface don't have flagAddr thus all these
values are not cached.
THe fix is to remove CanAddr calls and use underlying ptr in value
directly. As ptr is only read-only in DeepEqual for caching, it's
safe to do so. We don't use UnsafeAddr this time, because this method
panics when CanAddr returns false.
Fixes #33907
Change-Id: I2aa88cc060a2c2192b1d34c129c0aad4bd5597e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/191940
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Marko Kungla [Sat, 7 Sep 2019 09:07:08 +0000 (12:07 +0300)]
reflect: enhance docs for IsZero and IsValid
Make it clear that IsValid checks that we have valid
reflect.Value and not the value of `v`
fixes #34152
Change-Id: Ib3d359eeb3a82bf733b9ed17c777fc4c143bc29c
Reviewed-on: https://go-review.googlesource.com/c/go/+/193841 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Matthew Dempsky [Wed, 4 Sep 2019 22:16:25 +0000 (15:16 -0700)]
cmd/compile: better integrate parameter tagging with escape.go
This CL moves parameter tagging to before escape analysis is complete,
so we still have access to EscLocation. This will be useful once
EscLocation starts tracking higher-fidelity escape details.
Notably, this CL stops using n.Esc to record parameter escape analysis
details. Now escape analysis only ever sets n.Esc to EscNone or
EscHeap. (It still defaults to EscUnknown, and is set to EscNever in
some places though.)
Than McIntosh [Tue, 10 Sep 2019 18:42:50 +0000 (14:42 -0400)]
go/internal/gccgoimporter: remove guard on some assertions
Remove unnecessary conditional guard for a couple of assertions in the
type parser's update() method (inspired by comment from Robert). No
change in functionality.
Change-Id: I706a54569e75c6960768247889b7dec3f267dde9
Reviewed-on: https://go-review.googlesource.com/c/go/+/194565
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Ian Lance Taylor [Tue, 10 Sep 2019 02:22:46 +0000 (19:22 -0700)]
go/internal/gccgoimporter: support embedded field in pointer loop
If an embedded field refers to a type via a pointer, the parser needs
to know the name of the embedded field. It is possible that the
pointer type is not yet resolved. This CL fixes the parser to handle
that case by setting the pointer element type to the unresolved named
type while the pointer is being resolved.
Fixes #34182
Change-Id: I48435e0404362a85effd7463685c502290fa3c57
Reviewed-on: https://go-review.googlesource.com/c/go/+/194440
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Ian Lance Taylor [Tue, 10 Sep 2019 02:54:41 +0000 (19:54 -0700)]
cmd/go: for gccgo, look for tool build ID before hashing entire file
Also fix the key used to store the ID.
This is a significant speedup in cmd/go run time when using an
unreleased toolchain. For example, the TestGoBuildTestOnly cmd/go test
goes from 15 seconds to 1 second.
Change-Id: Ibfd697d55084db059c6b563f70f71f635e935391
Reviewed-on: https://go-review.googlesource.com/c/go/+/194441
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
This CL gets rid of the MOVDreg and MOVDnop SSA operations on
s390x. They were originally inserted to help avoid situations
where a sign/zero extension was elided but a spill invalidated
the optimization. It's not really clear we need to do this though
(amd64 doesn't have these ops for example) so long as we are
careful when removing sign/zero extensions. Also, the MOVDreg
technique doesn't work if the register is spilled before the
MOVDreg op (I haven't seen that in practice).
Removing these ops reduces the complexity of the rules and also
allows us to unblock optimizations. For example, the compiler can
now merge the loads in binary.{Big,Little}Endian.PutUint16 which
it wasn't able to do before. This CL reduces the size of the .text
section in the go tool by about 4.7KB (0.09%).
Romain Baugue [Wed, 1 May 2019 12:52:57 +0000 (14:52 +0200)]
encoding/json: don't indirect pointers when decoding null
The indirect method checked the type of the child when indirecting a
pointer. If the current value is a pointer and we are decoding null, we
can skip this entirely and return early, avoiding the whole descent.
Fixes #31776
Change-Id: Ib8b2a2357572c41f56fceac59b5a858980f3f65e
Reviewed-on: https://go-review.googlesource.com/c/go/+/174699
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Robert Griesemer [Fri, 30 Aug 2019 00:34:17 +0000 (17:34 -0700)]
cmd/compile/internal/scanner: report at most one lexical error per number literal
Leave reporting of multiple errors for strings alone for now;
we probably want to see all incorrect escape sequences in
runes/strings independent of other errors.
Fixes #33961.
Change-Id: Id722e95f802687963eec647d1d1841bd6ed17d35
Reviewed-on: https://go-review.googlesource.com/c/go/+/192499
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Matthew Dempsky [Fri, 6 Sep 2019 23:05:36 +0000 (16:05 -0700)]
cmd/compile: report more precise errors about untyped constants
Previously, we used a single "untyped number" type for all untyped
numeric constants. This led to vague error messages like "string(1.0)"
reporting that "1 (type untyped number)" can't be converted to string,
even though "string(1)" is valid.
This CL makes cmd/compile more like go/types by utilizing
types.Ideal{int,rune,float,complex} instead of types.Types[TIDEAL],
and keeping n.Type in sync with n.Val().Ctype() during constant
folding.
Thanks to K Heller for looking into this issue, and for the included
test case.
Fixes #21979.
Change-Id: Ibfea88c05704bc3c0a502a455d018a375589754d
Reviewed-on: https://go-review.googlesource.com/c/go/+/194019 Reviewed-by: Robert Griesemer <gri@golang.org>
Martin Möhrmann [Sat, 24 Aug 2019 08:38:23 +0000 (10:38 +0200)]
compile: prefer an AND instead of SHR+SHL instructions
On modern 64bit CPUs a SHR, SHL or AND instruction take 1 cycle to execute.
A pair of shifts that operate on the same register will take 2 cycles
and needs to wait for the input register value to be available.
Large constants used to mask the high bits of a register with an AND
instruction can not be encoded as an immediate in the AND instruction
on amd64 and therefore need to be loaded into a register with a MOV
instruction.
However that MOV instruction is not dependent on the output register and
on many CPUs does not compete with the AND or shift instructions for
execution ports.
Using a pair of shifts to mask high bits instead of an AND to mask high
bits of a register has a shorter encoding and uses one less general
purpose register but is slower due to taking one clock cycle longer
if there is no register pressure that would make the AND variant need to
generate a spill.
For example the instructions emitted for (x & 1 << 63) before this CL are: 48c1ea3f SHRQ $0x3f, DX 48c1e23f SHLQ $0x3f, DX
after this CL the instructions are the same as GCC and LLVM use: 48b80000000000000080 MOVQ $0x8000000000000000, AX
4821d0 ANDQ DX, AX
Some platforms such as arm64 already have SSA optimization rules to fuse
two shift instructions back into an AND.
Removing the general rule to rewrite AND to SHR+SHL speeds up this benchmark:
var GlobalU uint
func BenchmarkAndHighBits(b *testing.B) {
x := uint(0)
for i := 0; i < b.N; i++ {
x &= 1 << 63
}
GlobalU = x
}
amd64/darwin on Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz:
name old time/op new time/op delta
AndHighBits-4 0.61ns ± 6% 0.42ns ± 6% -31.42% (p=0.000 n=25+25):
Updates #33826
Updates #32781
Change-Id: I862d3587446410c447b9a7265196b57f85358633
Reviewed-on: https://go-review.googlesource.com/c/go/+/191780
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Joel Sing [Thu, 5 Sep 2019 17:13:11 +0000 (03:13 +1000)]
cmd/internal/obj: instructions and registers for RISC-V
Start implementing an assembler for RISC-V - this provides register
definitions and instruction mnemonics as defined in the RISC-V
Instruction Set Manual, along with instruction encoding.
The instruction encoding is generated by the parse_opcodes script with
the "opcodes" and "opcodes-pseudo" files from (`make inst.go`):
https://github.com/riscv/riscv-opcodes
This is based on the riscv-go port:
https://github.com/riscv/riscv-go
Contributors to the riscv-go port are:
Amol Bhave <ammubhave@gmail.com>
Benjamin Barenblat <bbaren@google.com>
Josh Bleecher Snyder <josharian@gmail.com>
Michael Pratt <michael@pratt.im>
Michael Yenik <myenik@google.com>
Ronald G. Minnich <rminnich@gmail.com>
Stefan O'Rear <sorear2@gmail.com>
Emmanuel T Odeke [Fri, 30 Aug 2019 06:07:43 +0000 (23:07 -0700)]
net: handle >=2GiB files with sendfile on Windows
CL 187037 applied a fix to handle the case where
files larger than 2GiB were not being sendfile-d,
in one shot, rejecting any files whose size was
larger than the 2GiB.
This CL allows files that are larger than limit
by SendFile-ing in chunks of upto 2GiB per chunk.
The test has been excluded as testing with 3GB
requires creating a local file, flushing it
and then doing sendfile which takes a while
and could cause flakes on computers without capacity,
but the test can be retroactively accessed at:
https://go-review.googlesource.com/c/go/+/192518/8/src/net/sendfile_windows_test.go
This CL detangles the hairy mess that was convlit+defaultlit. In
particular, it makes the following changes:
1. convlit1 now follows the standard typecheck behavior of setting
"n.Type = nil" if there's an error. Notably, this means for a lot of
test cases, we now avoid reporting useless follow-on error messages.
For example, after reporting that "1 << s + 1.0" has an invalid shift,
we no longer also report that it can't be assigned to string.
2. Previously, assignconvfn had some extra logic for trying to
suppress errors from convlit/defaultlit so that it could provide its
own errors with better context information. Instead, this extra
context information is now passed down into convlit1 directly.
3. Relatedly, this CL also removes redundant calls to defaultlit prior
to assignconv. As a consequence, when an expression doesn't make sense
for a particular assignment (e.g., assigning an untyped string to an
integer), the error messages now say "untyped string" instead of just
"string". This is more consistent with go/types behavior.
4. defaultlit2 is now smarter about only trying to convert pairs of
untyped constants when it's likely to succeed. This allows us to
report better error messages for things like 3+"x"; instead of "cannot
convert 3 to string" we now report "mismatched types untyped number
and untyped string".
Passes toolstash-check.
Change-Id: I26822a02dc35855bd0ac774907b1cf5737e91882
Reviewed-on: https://go-review.googlesource.com/c/go/+/187657
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Matthew Dempsky [Thu, 5 Sep 2019 00:45:50 +0000 (17:45 -0700)]
cmd/compile: use CTNIL for pointer-typed OLITERALs
We used to be more aggressive about constant folding in the frontend,
handling expressions that the Go spec does not consider constant;
e.g., "(*int)(unsafe.Pointer(uintptr(200)))". However, that led to a
lot of subtle Go spec conformance issues, so we've since abandoned
that effort (CL 151320), leaving SSA to handle these cases instead.
As such, the only time we now end up with pointer-typed OLITERALs is
when "nil" is implicitly converted to a pointer-typed variable.
Instead of representing these OLITERALs with an CTINT of 0, we can
just use CTNIL.
Saves a few bytes of memory and lines of code.
Change-Id: Ibc5c756b992fdc89c3bdaf4fda3aa352e8e2b101
Reviewed-on: https://go-review.googlesource.com/c/go/+/193437
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Mihai Borobocea [Fri, 6 Sep 2019 17:30:51 +0000 (20:30 +0300)]
text/template: refer to sorted map keys as "ordered" not "comparable" in docs
Consistent with the spec's definition of "ordered" and "comparable".
Fixes #34147
Change-Id: Id13186df5343588d80eaebfeb23092596a846d51
Reviewed-on: https://go-review.googlesource.com/c/go/+/193840 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Rob Pike <r@golang.org>
K. "pestophagous" Heller [Wed, 21 Aug 2019 06:13:25 +0000 (23:13 -0700)]
cmd/compile: improve errors for invalid conversions of consts
Follow-up to Change-Id: If6e52c59eab438599d641ecf6f110ebafca740a9
This addresses the remaining tech debt on issue 21979.
The aforementioned previous CL silenced one of two mostly redundant
compiler errors. However, the silenced error was the more expressive
error. This CL now imbues the surviving error with the same level
of expressiveness as the old semi-redundant error.
Fixes #21979
Change-Id: I3273d48c88bbab073fabe53421d801df621ce321
Reviewed-on: https://go-review.googlesource.com/c/go/+/191079
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Ian Lance Taylor [Thu, 5 Sep 2019 21:35:28 +0000 (14:35 -0700)]
misc/cgo/test: use __atomic intrinsics instead of __sync
GCC has supported the __atomic intrinsics since 4.7, and clang
supports them as well. They are better than the __sync intrinsics in
that they specify a memory model and, more importantly for our purposes,
they are reliably implemented either in the compiler or in libatomic.
Change-Id: I5e0036ea3300f65c28b1c3d1f3b93fb61c1cd646
Reviewed-on: https://go-review.googlesource.com/c/go/+/193603
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
Than McIntosh [Wed, 4 Sep 2019 15:25:06 +0000 (11:25 -0400)]
cmd/link: memoize/cache whether plugin.Open symbol available
Perform a single lookup of "plugin.Open" at the point where we set the
loaded flag for the context, then cache whether the result is nil, so
that we can consult this cached value later on (instead of having to
look up the symbol each time). This helps speed up the DynLinkingGo()
context method, which is called from within some very hot loops in the
linker (when linking 'hyperkube' from kubernetes, reduces total calls
to "sym.(*Symbols).ROLookup" from 6.5M to 4.3M)