LE Manh Cuong [Sun, 25 Aug 2019 16:29:32 +0000 (23:29 +0700)]
cmd/compile: make typecheck set n.Type.Nod when returning OTYPE
typecheck only set n.Type.Nod for declared type, and leave it nil for
anonymous types, type alias. It leads to compiler crashes, because
n.Type.Nod is nil at the time dowidth was called.
Fixing it by set n.Type.Nod right after n.Type initialization if n.Op is
OTYPE.
When embedding interface cycles involve in type alias, it also helps
pointing the error message to the position of the type alias
declaration, instead of position of embedding interface.
Fixes #31872
Change-Id: Ia18391e987036a91f42ba0c08b5506f52d07f683
Reviewed-on: https://go-review.googlesource.com/c/go/+/191540
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Than McIntosh [Mon, 26 Aug 2019 13:18:03 +0000 (09:18 -0400)]
test: new testcase for gollvm bug
Testcase for a gollvm bug (assert in Llvm_backend::materializeComposite).
Updates golang/go#33020.
Change-Id: Icdf5b4b2b6eb55a5b48a31a61c41215b1ae4cf01
Reviewed-on: https://go-review.googlesource.com/c/go/+/191743 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jason A. Donenfeld [Sat, 24 Aug 2019 15:13:33 +0000 (17:13 +0200)]
runtime: remove slow time compatibility hacks for wine
A few years ago, Wine-specific detection was added as an ugly hack to
work around shortcomings in the emulation layer. Probably it's best to
not special case this emulator versus that emulator versus the real
deal, but there were two arguments presented in the hack's favor:
1. Wine is useful and developers will appreciate being able to debug
stuff with it.
2. The existing KUSER_SHARED_DATA technique for gathering time is
undocumented, and we shouldn't be relying on it anyway, since
Microsoft might remove it without notice.
As it turns out, neither one of these are, at the time of writing, true.
(1) has been handled for some time by Wine with the introduction of the
commit entitled "ntdll: Create thread to update user_shared_data time
values when necessary". And (2) is in fact documented:
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntddk/ns-ntddk-kuser_shared_data
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/-kuser
It's in use so widely by both third-party software (such as games, a
massive market segment) and by Microsoft binaries that removing it from
the operating system will basically never happen.
So with both issues taken care of, this commit simply gets rid of the
old hack.
Change-Id: I80093f50e0d10d53648128d0f9dd76b1b92a119e
Reviewed-on: https://go-review.googlesource.com/c/go/+/191759
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Elias Naur [Wed, 31 Jul 2019 12:33:57 +0000 (14:33 +0200)]
runtime: don't forward SIGPIPE on macOS
macOS and iOS deliver SIGPIPE signals to the main thread and not
the thread that raised it by writing to a closed socket or pipe.
SIGPIPE signals can be suppressed for sockets with the SO_NOSIGPIPE
option, but there is no similar option for pipes. We have no other
choice but to never forward SIGPIPE on macOS.
Fixes #33384
Change-Id: Ice3de75b121f00006ee11c26d560e619536460be
Reviewed-on: https://go-review.googlesource.com/c/go/+/188297
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Prashant Agrawal [Sat, 18 May 2019 00:55:20 +0000 (17:55 -0700)]
debug/pe: enable parsing of variable length optional header in PE file
The debug/pe package assumes there are always 16 entries in
DataDirectory in OptionalHeader32/64
ref pe.go:
...
NumberOfRvaAndSizes uint32
DataDirectory [16]DataDirectory
}
...
But that is not always the case, there could be less no of
entries (PE signed linux kernel for example):
$ sudo pev /boot/vmlinuz-4.15.0-47-generic
....
Data-dictionary entries: 6
....
In such case, the parsing gives incorrect results.
This changes aims to fix that by:
1. Determining type of optional header by looking at header
magic instead of size
2. Parsing optional header in 2 steps:
a. Fixed part
b. Variable data directories part
Testing:
1. Fixed existing test cases to reflect the change
2. Added new file (modified linux kernel image)
which has smaller number of data directories
Fixes #32126
Change-Id: Iee56ecc4369a0e75a4be805e7cb8555c7d81ae2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/177959
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Tianon Gravi [Wed, 14 Aug 2019 15:19:55 +0000 (15:19 +0000)]
runtime: treat CTRL_CLOSE_EVENT, CTRL_LOGOFF_EVENT, CTRL_SHUTDOWN_EVENT as SIGTERM on Windows
This matches the existing behavior of treating CTRL_C_EVENT, CTRL_BREAK_EVENT as a synthesized SIGINT event.
See https://docs.microsoft.com/en-us/windows/console/handlerroutine for a good documentation source upstream to confirm these values.
As for the usage of these events, the "Timeouts" section of that upstream documentation is important to note, especially the limited window in which to do any cleanup before the program will be forcibly killed (defaults typically 5s, but as low as 500ms, and in many cases configurable system-wide).
These events are especially relevant for Windows containers, where these events (particularly `CTRL_SHUTDOWN_EVENT`) are one of the only ways containers can "gracefully" shut down (https://github.com/moby/moby/issues/25982#issuecomment-466804071).
This was verified by making a simple `main()` which implements the same code as in `ExampleNotify_allSignals` but in a `for` loop, building a `main.exe`, running that in a container, then doing `docker kill -sTERM` on said container. The program prints `Got signal: SIGTERM`, then exits after the aforementioned timeout, as expected. Behavior before this patch is that the program gets no notification (and thus no output) but still exits after the timeout.
Fixes #7479
Change-Id: I2af79421cd484a0fbb9467bb7ddb5f0e8bc3610e
GitHub-Last-Rev: 9e05d631b542393f5ebb0eb3747157c8bd0de635
GitHub-Pull-Request: golang/go#33311
Reviewed-on: https://go-review.googlesource.com/c/go/+/187739
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Dmitri Shuralyov [Wed, 28 Aug 2019 17:34:16 +0000 (13:34 -0400)]
CONTRIBUTORS: second round of updates for Go 1.13
This update has been automatically generated using the updatecontrib
command:
cd gotip
go run golang.org/x/build/cmd/updatecontrib
With minor manual changes based on publicly available information
to canonicalize letter case for a few names.
Actions taken (relative to CONTRIBUTORS at origin/master):
Added Albert Teoh <albert.teoh@gmail.com>
Added Allen Li <ayatane@google.com>
Added Anderson Queiroz <contato@andersonq.eti.br>
Added Andrew Todd <andrew.todd@wework.com>
Added Artem Kolin <artemkaxboy@gmail.com>
Added Bharath Thiruveedula <tbharath91@gmail.com>
Added Christian Muehlhaeuser <muesli@gmail.com>
Added Darren McCleary <darren.rmc@gmail.com>
Added David Finkel <david.finkel@gmail.com>
Added Eddie Scholtz <escholtz@google.com>
Added GitHub User tatsumack (4510569) <tatsu.mack@gmail.com>
Added GitHub User utkarsh-extc (53217283) <53217283+utkarsh-extc@users.noreply.github.com>
Added GitHub User yuanhh (1298735) <yuan415030@gmail.com>
Added Illya Yalovyy <yalovoy@gmail.com>
Added James Eady <jmeady@google.com>
Added Jan Steinke <jan.steinke@gmail.com>
Added Javier Revillas <jrevillas@massivedynamic.io>
Added Jordi Martin <jordimartin@gmail.com>
Added Jorge Araya <jorgejavieran@yahoo.com.mx>
Added Kelly Heller <pestophagous@gmail.com>
Added Kevin Gillette <extemporalgenome@gmail.com>
Added Mark Glines <mark@glines.org>
Added Nao Yonashiro <owan.orisano@gmail.com>
Added Pascal Dierich <pascal@pascaldierich.com>
Added Pure White <wu.purewhite@gmail.com>
Added Sam Arnold <sarnold64@bloomberg.net>
Added Sander van Harmelen <sander@vanharmelen.nl>
Added Sergei Zagurskii <gvozdoder@gmail.com>
Added Shivani Singhal <shivani.singhal2804@gmail.com>
Added Stefan Baebler <sbaebler@outbrain.com>
Added Tianon Gravi <admwiggin@gmail.com>
Added Toshihiro Shiino <shiino.toshihiro@gmail.com>
Added Wagner Riffel <wgrriffel@gmail.com>
Used GitHub User tatsumack (4510569) form for tatsumack <tatsu.mack@gmail.com> https://github.com/golang/website/commit/7eeb7ef [website]
Used GitHub User utkarsh-extc (53217283) form for utkarsh-extc <53217283+utkarsh-extc@users.noreply.github.com> https://github.com/golang/sys/commit/51ab0e2 [sys]
Used GitHub User yuanhh (1298735) form for yuanhh <yuan415030@gmail.com> https://github.com/golang/crypto/commit/60c769a [crypto]
Used GitHub name "Akhil Indurti" for smasher164 <aindurti@gmail.com> https://github.com/golang/go/commit/5ca44dc403 [build exp go sys]
Used GitHub name "Artem Kolin" for artemkaxboy <artemkaxboy@gmail.com> https://github.com/golang/go/commit/e881604d1c [go]
Used GitHub name "Ivan Markin" for nogoegst <nogoegst@users.noreply.github.com> https://github.com/golang/go/commit/a1addf15df [go]
Used GitHub name "Keiji Yoshida" for yosssi <yoshida.keiji.84@gmail.com> https://github.com/golang/lint/commit/ac6833c [lint]
Used GitHub name "Michalis Kargakis" for kargakis <mkargaki@redhat.com> https://github.com/golang/go/commit/e243d242d7 [go]
Used GitHub name "Roberto Clapis" for Roberto <empijei@users.noreply.github.com> https://github.com/golang/go/commit/963776e689 [go]
Used GitHub name "Robin Eklind" for mewmew <rnd0x00@gmail.com> https://github.com/golang/go/commit/b8620afb8d [blog go proposal.git]
Updates #12042
Change-Id: I1b21a18138849c537048558dd4324a823ba12a14
Reviewed-on: https://go-review.googlesource.com/c/go/+/192099 Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
LE Manh Cuong [Fri, 24 May 2019 18:27:40 +0000 (01:27 +0700)]
cmd/compile: ensure interface-to-concrete comparison panics when it should
In interface-to-concrete comparisons, we are short circuiting on the interface
value's dynamic type before evaluating the concrete expression for side effects,
causing concrete expression won't panic at runtime, while it should.
To fix it, evaluating the RHS of comparison before we do the short-circuit.
We also want to prioritize panics in the LHS over the RHS, so evaluating
the LHS too.
Fixes #32187
Change-Id: I15b58a523491b7fd1856b8fdb9ba0cba5d11ebb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/178817
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Matthew Dempsky [Thu, 25 Jul 2019 19:40:02 +0000 (12:40 -0700)]
cmd/compile: remove -newescape flag
Drops support for old escape analysis pass. Subsequent, separate CL
will remove dead code.
While here, fix a minor error in fmt.go: it was still looking for
esc.go's NodeEscState in n.Opt() rather than escape.go's EscLocation.
But this only affected debug diagnostics printed during escape
analysis itself.
Change-Id: I62512e1b31c75ba0577550a5fd7824abc3159ed5
Reviewed-on: https://go-review.googlesource.com/c/go/+/187597 Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Jordi Martin [Wed, 7 Aug 2019 11:14:38 +0000 (11:14 +0000)]
io: add error check on pipe close functions to avoid error overwriting
The current implementation allows multiple calls `Close` and `CloseWithError` in every side of the pipe, as a result, the original error can be overwritten.
This CL fixes this behavior adding an error existence check on `atomicError` type
and keeping the first error still available.
Fixes #24283
Change-Id: Iefe8f758aeb775309424365f8177511062514150
GitHub-Last-Rev: b559540d7af3a0dad423816b695525ac2d6bd864
GitHub-Pull-Request: golang/go#33239
Reviewed-on: https://go-review.googlesource.com/c/go/+/187197 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Mostyn Bramley-Moore [Tue, 27 Aug 2019 22:26:38 +0000 (22:26 +0000)]
crypto/ecdsa: improve documentation readability
Include references in the package-level comment block, expand
the obscure IRO acronym, and add a reference for "the standard
(cryptographic) assumptions".
Tobias Klauser [Wed, 8 May 2019 10:12:49 +0000 (12:12 +0200)]
syscall: move Renameat to syscall_linux_$GOARCH.go
On linux/riscv64, the renameat syscall no longer exists and has been
superseded by renameat2. Thus we'll have to use Renameat2 to implement
Renameat on linux/riscv64 for #27532. Prepare for this by moving the
Renameat definition to the GOARCH specific files.
Follow CL 157899 which did the same for golang.org/x/sys/unix
Cherry Zhang [Thu, 22 Aug 2019 22:04:57 +0000 (18:04 -0400)]
cmd/compile: don't mark stack object symbol DUPOK
Stack object symbol is named as <FunctionName>.stkobj. If the
function itself is not DUPOK, its stack object symbol should only
be defined in the package where the function is defined,
therefore no duplicates.
If in the future we change the stack object symbol to
content-hash naming, as other gcdata symbols, we can mark it as
DUPOK.
Stefan Baebler [Wed, 28 Aug 2019 12:10:16 +0000 (12:10 +0000)]
net/url: improve url parsing error messages by quoting
Current implementation doesn't always make it obvious what the exact
problem with the URL is, so this makes it clearer by consistently quoting
the invalid URL, as is the norm in other parsing implementations, eg.:
strconv.Atoi(" 123") returns an error: parsing " 123": invalid syntax
Updates #29261
Change-Id: Icc6bff8b4a4584677c0f769992823e6e1e0d397d
GitHub-Last-Rev: 648b9d93fe149ec90f3aeca73019158a344de03e
GitHub-Pull-Request: golang/go#29384
Reviewed-on: https://go-review.googlesource.com/c/go/+/185117 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Martin Möhrmann [Sat, 25 May 2019 12:23:34 +0000 (14:23 +0200)]
net/url: use strings.IndexByte instead of strings.Index in split function
Production profiling shows ~15% of url.Parse time being spend in the overhead
of calling strings.IndexByte through strings.Index instead of calling
strings.IndexByte directly.
name old time/op new time/op delta
Split 15.5ns ± 2% 10.7ns ± 3% -30.98% (p=0.000 n=20+19)
Change-Id: Ie25dd4afa93539a1335a91ab2a4a367f97bd3df0
Reviewed-on: https://go-review.googlesource.com/c/go/+/178877
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Cuong Manh Le [Thu, 22 Aug 2019 17:15:40 +0000 (00:15 +0700)]
cmd/compile: remove gc.eqtype in comments
golang.org/cl/143180 changed gc.eqtype to types.Identical, but gc.eqtype
is still mentioned in some comments. Remove them and update comments to
mention new functions instead.
Change-Id: I6c5eece5221f524556ee55db80de0e4bdfaf166e
Reviewed-on: https://go-review.googlesource.com/c/go/+/191357 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Agniva De Sarker [Sat, 22 Jun 2019 07:37:57 +0000 (13:07 +0530)]
runtime,syscall/js: reuse wasm memory DataView
Currently, every call to mem() incurs a new DataView object. This was necessary
because the wasm linear memory could grow at any time.
Now, whenever the memory grows, we make a call to the front-end. This allows us to
reuse the existing DataView object and create a new one only when the memory actually grows.
This gives us a boost in performance during DOM operations, while incurring an extra
trip to front-end when memory grows. However, since the GrowMemory calls are meant to decrease
over the runtime of an application, this is a good tradeoff in the long run.
The benchmarks have been tested inside a browser (Google Chrome 75.0.3770.90 (Official Build) (64-bit)).
It is hard to get stable nos. for DOM operations since the jumps make the timing very unreliable.
But overall, it shows a clear gain.
name old time/op new time/op delta
DOM 135µs ±26% 84µs ±10% -37.22% (p=0.000 n=10+9)
This CL optimizes math.bits.TrailingZeros16 on 386 with
a pair of BSFL and ORL instrcutions.
The case TrailingZeros16-4 of the benchmark test in
math/bits shows big improvement.
name old time/op new time/op delta
TrailingZeros16-4 1.55ns ± 1% 0.87ns ± 1% -43.87% (p=0.000 n=50+49)
Change-Id: Ia899975b0e46f45dcd20223b713ed632bc32740b
Reviewed-on: https://go-review.googlesource.com/c/go/+/189277
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Ben Shi [Tue, 20 Aug 2019 09:03:41 +0000 (09:03 +0000)]
cmd/compile: generate Select on WASM
This CL performs the branchelim optimization on WASM with its
select instruction. And the total size of pkg/js_wasm decreased
about 80KB by this optimization.
Change-Id: I868eb146120a1cac5c4609c8e9ddb07e4da8a1d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/190957
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Richard Musiol <neelance@gmail.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
cmd/internal/obj/arm64: add support for most system registers
This patch supports the EL0 and EL1 system registers used in MRS/MSR
instructions. This patch refactors the assembler code, allowing the
assembler to read system register information from the automatically
generated sysRegEnc.go file and move existing declared system registers
to the sysRegEnc.go file.
This patch adds 431 system registers, it is worth noting that the number
of special registers is initialized to less than 1024 in the list7.go file.
This CL also adds some test cases to test the newly added system registers.
The test cases are contributed by Dianhong Xu <Dianhong.Xu@arm.com>
Change-Id: Ic09a937eaaeefe82bd08b5dd726808f8ff6cebf6
Reviewed-on: https://go-review.googlesource.com/c/go/+/189577 Reviewed-by: Ben Shi <powerman1st@163.com>
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Filippo Valsorda [Tue, 27 Aug 2019 21:27:45 +0000 (17:27 -0400)]
crypto/tls: remove SSLv3 support
SSLv3 has been irreparably broken since the POODLE attack 5 years ago
and RFC 7568 (f.k.a. draft-ietf-tls-sslv3-diediedie) prohibits its use
in no uncertain terms.
As announced in the Go 1.13 release notes, remove support for it
entirely in Go 1.14.
While working on #30645, I noticed that many instances
in which the walkinrange optimization could apply
were not even being considered.
This was because of extraneous blocks in the CFG,
of the type that shortcircuit normally removes.
The change improves the shortcircuit pass to handle
most of those cases. (There are a few that can only be
reasonably detected later in compilation, after other
optimizations have been run, but not enough to be worth chasing.)
Notable changes:
* Instead of calculating live-across-blocks values, use v.Uses == 1.
This is cheaper and more straightforward.
v.Uses did not exist when this pass was initially written.
* Incorporate a fusePlain and loop until stable.
This is necessary to find many of the instances.
* Allow Copy and Not wrappers around Phi values.
This significantly increases effectiveness.
* Allow removal of all preds, creating a dead block.
The previous pass stopped unnecessarily at one pred.
* Use phielimValue during cleanup instead of manually
setting the op to OpCopy.
The result is marginally faster compilation and smaller code.
Bryan C. Mills [Tue, 27 Aug 2019 18:18:48 +0000 (14:18 -0400)]
cmd/go/internal/get: remove '--' separator from 'git ls-remote' command
'git ls-remote' started recognizing the '--' separator at some point
after 2.7.4, but git defaults to version 2.7.4 on Ubuntu 16.04 LTS,
which remains supported by Ubuntu until April 2021.
We added '--' tokens to most VCS commands as a defensive measure in
CL 181237, but it isn't strictly necessary here because the 'scheme'
argument to our template is chosen from a predefined list: we can
safely drop it to retain compatibility.
Fixes #33836
Updates #26746
Change-Id: Ibb53366b95f8029b587e0b7646a439330d759ac7
Reviewed-on: https://go-review.googlesource.com/c/go/+/191978
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Matthew Dempsky [Mon, 5 Aug 2019 20:12:01 +0000 (13:12 -0700)]
cmd/compile: fix "previous" position info for duplicate switch cases
Because the Node AST represents references to declared objects (e.g.,
variables, packages, types, constants) by directly pointing to the
referred object, we don't have use-position info for these objects.
For switch statements with duplicate cases, we report back where the
first duplicate value appeared. However, due to the AST
representation, if the value was a declared constant, we mistakenly
reported the constant declaration position as the previous case
position.
This CL reports back against the 'case' keyword's position instead, if
there's no more precise information available to us.
It also refactors code to emit the same "previous at" error message
for duplicate values in map literals.
Bryan C. Mills [Mon, 26 Aug 2019 20:44:36 +0000 (16:44 -0400)]
net/http: fix wantConnQueue memory leaks in Transport
I'm trying to keep the code changes minimal for backporting to Go 1.13,
so it is still possible for a handful of entries to leak,
but the leaks are now O(1) instead of O(N) in the steady state.
Longer-term, I think it would be a good idea to coalesce idleMu with
connsPerHostMu and clear entries out of both queues as soon as their
goroutines are done waiting.
The decoder called this function to check numbers being decoded into a
json.Number. However, these can't be quoted as strings, so the tokenizer
has already verified they are valid JSON numbers.
Verified this by adding a test with such an input. As expected, it
produces a syntax error, not the fmt.Errorf - that line could never
execute.
Since the only remaining non-test caller of isvalidnumber is in
encode.go, move the function there.
This change should slightly reduce the amount of work when decoding into
json.Number, though that isn't very common nor part of any current
benchmarks.
Daniel Martí [Sat, 10 Aug 2019 17:27:45 +0000 (19:27 +0200)]
cmd/compile: teach rulegen to remove unused decls
First, add cpu and memory profiling flags, as these are useful to see
where rulegen is spending its time. It now takes many seconds to run on
a recent laptop, so we have to keep an eye on what it's doing.
Second, stop writing '_ = var' lines to keep imports and variables used
at all times. Now that rulegen removes all such unused names, they're
unnecessary.
To perform the removal, lean on go/types to first detect what names are
unused. We can configure it to give us all the type-checking errors in a
file, so we can collect all "declared but not used" errors in a single
pass.
We then use astutil.Apply to remove the relevant nodes based on the line
information from each unused error. This allows us to apply the changes
without having to do extra parser+printer roundtrips to plaintext, which
are far too expensive.
We need to do multiple such passes, as removing an unused variable
declaration might then make another declaration unused. Two passes are
enough to clean every file at the moment, so add a limit of three passes
for now to avoid eating cpu uncontrollably by accident.
The resulting performance of the changes above is a ~30% loss across the
table, since go/types is fairly expensive. The numbers were obtained
with 'benchcmd Rulegen go run *.go', which involves compiling rulegen
itself, but that seems reflective of how the program is used.
name old time/op new time/op delta
Rulegen 5.61s ± 0% 7.36s ± 0% +31.17% (p=0.016 n=5+4)
name old user-time/op new user-time/op delta
Rulegen 7.20s ± 1% 9.92s ± 1% +37.76% (p=0.016 n=5+4)
name old sys-time/op new sys-time/op delta
Rulegen 135ms ±19% 169ms ±17% +25.66% (p=0.032 n=5+5)
name old peak-RSS-bytes new peak-RSS-bytes delta
Rulegen 71.0MB ± 2% 85.6MB ± 2% +20.56% (p=0.008 n=5+5)
We can live with a bit more resource usage, but the time/op getting
close to 10s isn't good. To win that back, introduce concurrency in
main.go. This further increases resource usage a bit, but the real time
on this quad-core laptop is greatly reduced. The final benchstat is as
follows:
name old time/op new time/op delta
Rulegen 5.61s ± 0% 3.97s ± 1% -29.26% (p=0.008 n=5+5)
name old user-time/op new user-time/op delta
Rulegen 7.20s ± 1% 13.91s ± 1% +93.09% (p=0.008 n=5+5)
name old sys-time/op new sys-time/op delta
Rulegen 135ms ±19% 269ms ± 9% +99.17% (p=0.008 n=5+5)
name old peak-RSS-bytes new peak-RSS-bytes delta
Rulegen 71.0MB ± 2% 226.3MB ± 1% +218.72% (p=0.008 n=5+5)
It might be possible to reduce the cpu or memory usage in the future,
such as configuring go/types to do less work, or taking shortcuts to
avoid having to run it many times. For now, ~2x cpu and ~4x memory usage
seems like a fair trade for a faster and better rulegen.
Finally, we can remove the old code that tried to remove some unused
variables in a hacky and unmaintainable way.
Change-Id: Iff9e83e3f253babf5a1bd48cc993033b8550cee6
Reviewed-on: https://go-review.googlesource.com/c/go/+/189798
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Andrew Gerrand [Fri, 31 May 2019 12:00:42 +0000 (22:00 +1000)]
archive/zip: remove unused special case
This removes a special case that was added to fix issue #10956, but that
was never actually effective. The code in the test case still fails to
read, so perhaps the zip64 support added in CL 6463050 inadvertently
caught this particular case.
It's possible that the original theorized bug still exists, but I'm not
convinced it was ever fixed.
Daniel Martí [Thu, 11 Jul 2019 12:27:19 +0000 (21:27 +0900)]
encoding/json: fix the broken "overwriting of data" tests
Because TestUnmarshal actually allocates a new value to decode into
using ptr's pointer type, any existing data is thrown away. This was
harmless in alomst all of the test cases, minus the "overwriting of
data" ones added in 2015 in CL 12209.
I spotted that nothing covered decoding a JSON array with few elements
into a slice which already had many elements. I initially assumed that
the code was buggy or that some code could be removed, when in fact
there simply wasn't any code covering the edge case.
Move those two tests to TestPrefilled, which already served a very
similar purpose. Remove the map case, as TestPrefilled already has
plenty of prefilled map cases. Moreover, we no longer reset an entire
map when decoding, as per the godoc:
To unmarshal a JSON object into a map, Unmarshal first
establishes a map to use. If the map is nil, Unmarshal allocates
a new map. Otherwise Unmarshal reuses the existing map, keeping
existing entries.
Finally, to ensure that ptr is used correctly in the future, make
TestUnmarshal error if it's anything other than a pointer to a zero
value. That is, the only correct use should be new(type). Don't rename
the ptr field, as that would be extremely noisy and cause unwanted merge
conflicts.
Daniel Martí [Thu, 16 May 2019 10:26:40 +0000 (11:26 +0100)]
cmd/compile: initial rulegen rewrite
rulegen.go produces plaintext Go code directly, which was fine for a
while. However, that's started being a bottleneck for making code
generation more complex, as we can only generate code directly one line
at a time.
Some workarounds were used, like multiple layers of buffers to generate
chunks of code, to then use strings.Contains to see whether variables
need to be defined or not. However, that's error-prone, verbose, and
difficult to work with.
A better approach is to generate an intermediate syntax tree in memory,
which we can inspect and modify easily. For example, we could run a
number of "passes" on the syntax tree before writing to disk, such as
removing unused variables, simplifying logic, or moving declarations
closer to their uses.
This is the first step in that direction, without changing any of the
generated code. We didn't use go/ast directly, as it's too complex for
our needs. In particular, we only need a few kinds of simple statements,
but we do want to support arbitrary expressions. As such, define a
simple set of statement structs, and add thin layers for printer.Fprint
and ast.Inspect.
A nice side effect of this change, besides removing some buffers and
string handling, is that we can now avoid passing so many parameters
around. And, while we add over a hundred lines of code, the tricky
pieces of code are now a bit simpler to follow.
While at it, apply some cleanups, such as replacing isVariable with
token.IsIdentifier, and consistently using log.Fatalf.
Follow-up CLs will start improving the generated code, also simplifying
the rulegen code itself. I've added some TODOs for the low-hanging fruit
that I intend to work on right after.
Updates #30810.
Change-Id: Ic371c192b29c85dfc4a001be7fbcbeec85facc9d
Reviewed-on: https://go-review.googlesource.com/c/go/+/177539
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Javier Revillas [Sun, 25 Aug 2019 16:05:41 +0000 (16:05 +0000)]
net/http: fix a typo in comments
HTTP is an initialism, not an acronym, where you pronounce each letter as a
word. It's "an H", not "a H".
Running `find src/net/http -type f | xargs grep -n 'an HTTP' | wc -l` shows
that the "an HTTP" form is used 67 times across the `net/http` package.
Furthermore, `find src/net/http -type f | xargs grep -n 'a HTTP' | wc -l`
yields only 4 results.
Change-Id: I219c292a9e2c9bf7a009dbfe82ea8b15874685e9
GitHub-Last-Rev: 6ebd095023af47444b6b0fc5b6d7b26d85f4c7b7
GitHub-Pull-Request: golang/go#33810
Reviewed-on: https://go-review.googlesource.com/c/go/+/191700 Reviewed-by: Toshihiro Shiino <shiino.toshihiro@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/compile: handle sign/zero extensions in prove, via update method
Array accesses with index types smaller than the machine word size may
involve a sign or zero extension of the index value before bounds
checking. Currently, this defeats prove because the facts about the
original index value don't flow through the sign/zero extension.
This CL fixes this by looking back through value-preserving sign/zero
extensions when adding facts via Update and, where appropriate, applying
the same facts using the pre-extension value. This fix is enhanced by
also looking back through value-preserving extensions within
ft.isNonNegative to infer whether the extended value is known to be
non-negative. Without this additional isNonNegative enhancement, this
logic is rendered significantly less effective by the limitation
discussed in the next paragraph.
In Update, the application of facts to pre-extension values is limited
to cases where the domain of the new fact is consistent with the type of
the pre-extension value. There may be cases where this cross-domain
passing of facts is valid, but distinguishing them from the invalid
cases is difficult for me to reason about and to implement.
Assessing which cases to allow requires details about the context and
inferences behind the fact being applied which are not available
within Update. Additional difficulty arises from the fact that the SSA
does not curently differentiate extensions added by the compiler for
indexing operations, extensions added by the compiler for implicit
conversions, or explicit extensions from the source.
Examples of some cases that would need to be filtered correctly for
cross-domain facts:
(1) A uint8 is zero-extended to int for indexing (a value-preserving
zeroExt). When, if ever, can signed domain facts learned about the int be
applied to the uint8?
(2) An int8 is sign-extended to int16 (value-preserving) for an equality
comparison. Equality comparison facts are currently always learned in both
the signed and unsigned domains. When, if ever, can the unsigned facts
learned about the int16, from the int16 != int16 comparison, be applied
to the original int8?
This is an alternative to CL 122695 and CL 174309. Compared to CL 122695,
this CL differs in that the facts added about the pre-extension value will
pass through the Update method, where additional inferences are processed
(e.g. fence-post implications, see #29964). CL 174309 is limited to bounds
checks, so is narrower in application, and makes the code harder to read.
Fixes #26292.
Fixes #29964.
Fixes #15074
Removes 238 bounds checks from std/cmd.
Change-Id: I1f87c32ee672bfb8be397b27eab7a4c2f304893f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174704
Run-TryBot: Zach Jones <zachj1@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Giovanni Bajo <rasky@develer.com>
Sergei Zagurskii [Wed, 22 May 2019 14:00:34 +0000 (17:00 +0300)]
reflect: optimize directlyAssignable to avoid rtype.Name call
directlyAssignable invoked rtype.Name() just to compare its result
to empty string. We really only need to check whether rtype has
name. It can be done much cheaper, by checking tflagNamed.
Benchmark: https://play.golang.org/p/V2BzESPuf2w
name old time/op new time/op delta
DirectlyAssignable-12 32.7ns ± 6% 6.6ns ± 6% -79.80% (p=0.008 n=5+5)
Jason A. Donenfeld [Mon, 26 Aug 2019 13:15:34 +0000 (07:15 -0600)]
ld: fix up header copy and paste error
Some constants were added above the initial copyright blurb, and then
later a new copyright blurb was added on top of that. So we wound up
with two header sections, one of which contained a useful comment that
became obscured. This commit fixes up that mistake.
Change-Id: I8b9b8c34495cdceae959e151e8ccdee3137f6ca4
Reviewed-on: https://go-review.googlesource.com/c/go/+/191841
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Robert Griesemer [Fri, 23 Aug 2019 04:27:33 +0000 (21:27 -0700)]
go/types: process each segment of delayed actions in FIFO order
The stack of delayed actions is grown by pushing a new action
on top with Checker.later. Checker.processDelayed processes
all actions above a top watermark and then resets the stack
to top.
Until now, pushed actions above the watermark were processed
in LIFO order. This change processes them in FIFO order, which
seems more natural (if an action A was delayed before an action
B, A should be processed before B for that stack segment).
(With this change, Checker.later could be used instead of
Checker.atEnd to postpone interface method type comparison
and then the specific example in issue #33656 does type-check.
However, in general we want interface method type comparisons
to run after all interfaces are completed. With Checker.later
we may still end up mixing interface completions and interface
method type comparisons in ways leading to other errors for
sufficiently convoluted code.)
Also, move Checker.processDelayed from resolver.go to check.go.
Change-Id: Id31254605e6944c490eab410553fff907630cc64
Reviewed-on: https://go-review.googlesource.com/c/go/+/191458 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Robert Griesemer [Fri, 23 Aug 2019 00:03:30 +0000 (17:03 -0700)]
go/types: postpone interface method type comparison to the end
Introduce a new list of final actions that is executed at the
end of type checking and use it to collect method type comparisons
and also map key checks.
Fixes #33656.
Change-Id: Ia77a35a45a9d7eaa7fc3e9e19f41f32dcd6ef9d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/191418 Reviewed-by: Alan Donovan <adonovan@google.com>
When creating a new interface via the exported API calls, a shared
empty and completed Interface value is returned if there are no
methods or embedded interfaces. This is a minor optimization and
matches the internal behavior when creating empty interfaces.
Since calling Interface.Complete is idempotent, and since there
are no other legitimate ways to create Interface values externally
but via NewInterface/NewInterfaceType calls, and completed Interfaces
are considered "immutable", this change is not expected to affect
clients. The only observable behavior that changed is the string
value for empty interfaces created via the above API calls; those
empty interfaces now don't show "incomplete" anymore even before
Interface.Complete is called. Except in special test cases, this
behavior is unlikely to affect clients.
Change-Id: Idf7f2cd112241c5b81a43b4544bbe3f2e003d8d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/191417 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Robert Griesemer [Thu, 22 Aug 2019 00:13:45 +0000 (17:13 -0700)]
go/types: allow embedding overlapping interfaces
Quietly drop duplicate methods from embedded interfaces
if they have an identical signature to existing methods.
Instead of adjusting the prior syntax-based only method set
computation where methods don't have signature information
(and thus where de-duplication according to the new rules
would have been somewhat tricky to get right), this change
completely rewrites interface method set computation, taking
a page from the cmd/compiler's implementation. In a first
pass, when type-checking interfaces, explicit methods and
embedded interfaces are collected, but the interfaces are
not "expanded", that is the final method set computation
is done lazily, either when needed for method lookup, or
at the end of type-checking.
While this is a substantial rewrite, it allows us to get
rid of the separate (duplicate and delicate) syntactical
method set computation and generally simplifies checking
of interface types significantly. A few (esoteric) test
cases now have slightly different error messages but all
tests that are accepted by cmd/compile are also accepted
by go/types.
(This is a replacement for golang.org/cl/190258.)
Updates #6977.
Change-Id: Ic8b9321374ab4f617498d97c12871b69d1119735
Reviewed-on: https://go-review.googlesource.com/c/go/+/191257 Reviewed-by: Matthew Dempsky <mdempsky@google.com>