Keith Randall [Tue, 30 Apr 2019 21:03:07 +0000 (14:03 -0700)]
cmd/compile: fix line numbers for index panics
In the statement x = a[i], the index panic should appear to come from
the line number of the '['. Previous to this CL we sometimes used the
line number of the '=' instead.
Emmanuel T Odeke [Mon, 11 Mar 2019 22:52:20 +0000 (15:52 -0700)]
net/http: make Server return 501 for unsupported transfer-encodings
Ensures that our HTTP/1.X Server properly responds
with a 501 Unimplemented as mandated by the spec at
RFC 7230 Section 3.3.1, which says:
A server that receives a request message with a
transfer coding it does not understand SHOULD
respond with 501 (Unimplemented).
Elias Naur [Tue, 30 Apr 2019 18:00:23 +0000 (20:00 +0200)]
cmd/dist: detect GOHOSTARCH on iOS
cmd/dist defaults to GOHOSTARCH=amd64 on darwin because no other
darwin host could build Go. With the upcoming self-hosted iOS
builders, GOHOSTARCH=arm64 is also possible.
Elias Naur [Tue, 30 Apr 2019 17:57:46 +0000 (19:57 +0200)]
cmd/dist: set the default external linker on platforms without gcc
The go tool already sets -extld to the appropriate compiler. This
CL changes cmd/dist to do the same, to fix bootstrapping on platforms
that only have clang (Android and iOS).
all: refer to map elements as elements instead of values
The spec carefully and consistently uses "key" and "element"
as map terminology. The implementation, not so much.
This change attempts to make the implementation consistently
hew to the spec's terminology. Beyond consistency, this has
the advantage of avoid some confusion and naming collisions,
since v and value are very generic and commonly used terms.
I believe that I found all everything, but there are a lot of
non-obvious places for these to hide, and grepping for them is hard.
Hopefully this change changes enough of them that we will start using
elem going forward. Any remaining hidden cases can be removed ad hoc
as they are discovered.
The only externally-facing part of this change is in package reflect,
where there is a minor doc change and a function parameter name change.
all: add new GOOS=illumos, split out of GOOS=solaris
Like GOOS=android which implies the "linux" build tag, GOOS=illumos
implies the "solaris" build tag. This lets the existing ecosystem of
packages still work on illumos, but still permits packages to start
differentiating between solaris and illumos.
cmd/go/internal/modfetch: fix concurrent read/write race in modfetch
On Windows systems, the failure rate for cmd/go's TestScript/mod_concurrent
is somewhere around 3-10% without this change. With the change, I have yet
to see a failure.
Richard Musiol [Fri, 26 Apr 2019 23:16:10 +0000 (01:16 +0200)]
cmd/internal/obj/wasm: cache SP in a local
We use Wasm global variables extensively for simulating
registers, especially SP. V8 does not handle global variables
efficiently.
This CL reduces global variable accesses by caching the global SP
in a local variable in each function. The local cache is set on
function entry and updated after each call (where the stack could
have moved). Within a function, the SP access will use the local
variable.
Supersedes https://golang.org/cl/173979.
Running on Chrome Version 73.0.3683.103 on darwin/amd64:
Richard Musiol [Sun, 28 Apr 2019 10:16:44 +0000 (12:16 +0200)]
runtime: do not use heap arena hints on wasm
The address space of WebAssembly's linear memory is contiguous, so
requesting specific addresses is not supported. Do not use heap arena
hints so we do not have unused memory ranges.
This fixes go1 benchmarks on wasm which ran out of memory since
https://golang.org/cl/170950.
Ordering the initialization this way limits the lifetime of the
temporaries involved. In particular, for large maps the number of
simultaneously live temporaries goes from ~2*len(m) to ~2. This change
makes the compiler (regalloc, mostly) a lot happier. The compiler runs
faster and uses a lot less memory.
For #26546, changes compile time of a big map from 8 sec to 0.5 sec.
Fixes #26552
Update #26546
Change-Id: Ib7d202dead3feaf493a464779fd9611c63fcc25f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174417
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Joel Sing [Mon, 29 Apr 2019 02:24:30 +0000 (12:24 +1000)]
cmd,runtime: enable cgo for openbsd/arm64
Updates #31656.
Change-Id: Ide6f829282fcdf20c67998b766a201a6a92c3035
Reviewed-on: https://go-review.googlesource.com/c/go/+/174132
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Fri, 26 Apr 2019 05:39:56 +0000 (22:39 -0700)]
runtime: account for callbacks in checkdead on Windows
When a callback runs on a different thread in Windows, as in the
runtime package test TestCallbackInAnotherThread, it will use the
extra M. That can cause the test in checkdead to fail incorrectly.
Check whether there actually is an extra M before expecting it.
I think this is a general problem unrelated to timers. I think the test
was passing previously because the timer goroutine was using an M.
But I haven't proved that. This change seems correct, and it avoids
the test failure when using the new timers on Windows.
Updates #27707
Change-Id: Ieb31c04ff0354d6fae7e173b59bcfadb8b0464cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174037 Reviewed-by: Keith Randall <khr@golang.org>
Change-Id: Ib102ae95acfd89fc3c9942a4ec82c74362f62045
Reviewed-on: https://go-review.googlesource.com/c/go/+/174299
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime: make mmap return 0 instead of -1 on aix/ppc64
Most of the platforms are returning 0 instead of -1 when mmap syscalls
is failing. This patch corrects it for AIX in order to fix
TestMmapErrorSign and to improve AIX compatibility.
Change-Id: I1dad88d0e69163ad55c504b2b4a997892fd876cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174297
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
testing: delay flag registration; move to an Init function
Any code that imports the testing package forces the testing flags to be
defined, even in non-test binaries. People work around this today by
defining a copy of the testing.TB interface just to avoid importing
testing.
Fix this by moving flag registration into a new function, testing.Init.
Delay calling Init until the testing binary begins to run, in
testing.MainStart.
Init is exported for cases where users need the testing flags to be
defined outside of a "go test" context. In particular, this may be
needed where testing.Benchmark is called outside of a test.
syscall: don't return EINVAL on zero Chmod mode on Windows
Fixes #20858
Change-Id: I45c397795426aaa276b20f5cbeb80270c95b920c
Reviewed-on: https://go-review.googlesource.com/c/go/+/174320 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
os/exec: always set SYSTEMROOT on Windows if not listed in Cmd.Env
Fixes #25210
Change-Id: If27b61776154dae9b9b67bf4e4f5faa785d98105
Reviewed-on: https://go-review.googlesource.com/c/go/+/174318 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joel Sing [Sun, 28 Apr 2019 07:59:05 +0000 (17:59 +1000)]
runtime: initialise cpu.HWCap on openbsd/arm64
OpenBSD does not provide auxv, however we still need to initialise cpu.HWCap.
For now initialise it to the bare minimum, until some form of CPU capability
detection is implemented or becomes available - see issue #31746.
Richard Musiol [Thu, 25 Apr 2019 08:55:49 +0000 (10:55 +0200)]
misc/wasm: fix command line arguments containing multi-byte characters
Command line arguments containing multi-byte characters were causing
go_js_wasm_exec to crash (RangeError: Source is too large), because
their byte length was not handled correctly. This change fixes the bug.
BigMikes [Sun, 30 Dec 2018 14:18:58 +0000 (15:18 +0100)]
net: correct docs of KeepAlive field in Dialer type
KeepAlive field used to report the wording "keep-alive period"
which may be misleading. This field does not represent the whole
TCP keepalive time, that is the inactivity period upon which one
endpoint starts probing the other end. But it acctually specifies
the keepalive interval, that is the time between two keepalive
probes.
Dmitri Shuralyov [Mon, 29 Apr 2019 04:23:16 +0000 (00:23 -0400)]
net/http: remove "number:" from Response.Status string
The behavior of Value.String method on non-string JavaScript types has
changed after CL 169757.
Update the implementation of Transport.RoundTrip method to construct the
Response.Status string without relying on result.Get("status").String(),
since that now returns strings like "<number: 200>" instead of "200".
Daniel Martí [Sun, 28 Apr 2019 16:03:35 +0000 (23:03 +0700)]
all: remove a few unused parameters
I recently modified tabwriter to reduce the number of defers due to
flush calls. However, I forgot to notice that the new function
flushNoDefers can no longer return an error, due to the lack of the
defer.
In crypto/tls, hashForServerKeyExchange never returned a non-nil error,
so simplify the code.
Finally, in go/types and net we can find a few trivially unused
parameters, so remove them.
Adds a sample Fuzz test function to package encoding/json following the
guidelines defined in #31309, based on
https://github.com/dvyukov/go-fuzz-corpus/blob/master/json/json.go
This CL adds support for consulting the Go checksum database
when downloading a module that is not already listed in go.sum.
The overall system is described at golang.org/design/25530-sumdb,
and this CL implements the functionality described specifically in
golang.org/design/25530-sumdb#command-client.
Although the eventual plan is to set GOPROXY and GOSUMDB to
default to a Google-run proxy serving the public Go ecosystem,
this CL leaves them off by default.
Alessandro Arzilli [Tue, 5 Feb 2019 07:49:49 +0000 (08:49 +0100)]
runtime: whitelist debugCall32..debugCall65536 in debugCallCheck
Whitelists functions debugCall32 through debugCall65536 in
runtime.debugCallCheck so that any instruction inside those functions
is considered a safe point.
This is useful for implementing nested function calls.
For example when evaluating:
f(g(x))
The debugger should:
1. initiate the call to 'f' until the entry point of 'f',
2. complete the call to 'g(x)'
3. copy the return value of 'g(x)' in the arguments of 'f'
4. complete the call to 'f'
Similarly for:
f().amethod()
The debugger should initiate the call to '.amethod()', then initiate
and complete the call to f(), copy the return value to the arguments
of '.amethod()' and finish its call.
However in this example, unlike the other example, it may be
impossible to determine the entry point of '.amethod()' until after
'f()' is evaluated, which means that the call to 'f()' needs to be
initiated while stopped inside a debugCall... function.
Jason A. Donenfeld [Sat, 27 Apr 2019 09:45:11 +0000 (11:45 +0200)]
syscall: allow setting security attributes on processes
This allows creating processes that can only be debugged/accessed by
certain tokens, according to a particular security descriptor. We
already had everything ready for this but just neglected to pass through
the value from the user-accessible SysProcAttr.
Change-Id: I4a3fcc9f5078aa0058b26c103355c984093ae03f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174197
Run-TryBot: Jason Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Alex Brainman [Sat, 27 Apr 2019 01:02:31 +0000 (11:02 +1000)]
cmd/go/internal/renameio: use ERROR_ACCESS_DENIED to check for errors
CL 172418 added code to check for "Access is denied" error.
But "Access is denied" error will be spelled differently on
non-English version of Windows.
Elias Naur [Sat, 27 Apr 2019 10:15:42 +0000 (12:15 +0200)]
cmd/link/internal/ld,syscall: replace getfsstat64 with getfsstat
getfsstat64 is deprecated but not yet caught by the App Store checks.
Use the supported getfsstat$INODE64 form instead to ensure forward
compatibility.
Change-Id: I0d97e8a8b254debb3de1cfcb3778dbed3702c249
Reviewed-on: https://go-review.googlesource.com/c/go/+/174200
Run-TryBot: Elias Naur <mail@eliasnaur.com> Reviewed-by: Keith Randall <khr@golang.org>
Elias Naur [Sat, 27 Apr 2019 09:36:40 +0000 (11:36 +0200)]
cmd/link/internal/ld,syscall: drop $INODE64 suffixes on simulators
Some libc functions are suffixed with "$INODE64" on macOS.
Unfortunately, the iOS simulator doesn't have the suffixes, so we can't
use GOARCH to distinguish the two platform.
Add linker support for adding the suffix, using the macho platform
to determine whether it is needed.
While here, add the correct suffix for fdopendir on 386. It's
"$INODE64$UNIX2003", believe it or not. Without the suffix,
Brian Kessler [Sun, 10 Mar 2019 04:58:16 +0000 (21:58 -0700)]
cmd/compile: add unsigned divisibility rules
"Division by invariant integers using multiplication" paper
by Granlund and Montgomery contains a method for directly computing
divisibility (x%c == 0 for c constant) by means of the modular inverse.
The method is further elaborated in "Hacker's Delight" by Warren Section 10-17
This general rule can compute divisibilty by one multiplication and a compare
for odd divisors and an additional rotate for even divisors.
To apply the divisibility rule, we must take into account
the rules to rewrite x%c = x-((x/c)*c) and (x/c) for c constant on the first
optimization pass "opt". This complicates the matching as we want to match
only in the cases where the result of (x/c) is not also available.
So, we must match on the expanded form of (x/c) in the expression x == c*(x/c)
in the "late opt" pass after common subexpresion elimination.
Note, that if there is an intermediate opt pass introduced in the future we
could simplify these rules by delaying the magic division rewrite to "late opt"
and matching directly on (x/c) in the intermediate opt pass.
Additional rules to lower the generic RotateLeft* ops were also applied.
On amd64, the divisibility check is 25-50% faster.
Daniel Theophanes [Fri, 26 Apr 2019 18:46:26 +0000 (11:46 -0700)]
database/sql: add NullInt32
It is common for database integers to be represented as int32
internally. Although NullInt64 is already defined,
this should remove some type casts and make working with those eaiser.
For linking large binaries, the slice of Relocs consumes a large
amount of memory. We can reduce this memory consumption by
shrinking the size of the Reloc struct. This CL moves the fields
used only in external linking or only on PPC64 and S390X to a
lazily initialized side struct.
Linking k8s.io/kubernetes/cmd/kube-apiserver on Linux/AMD64,
before:
inuse_space 1240.25MB total
438.11MB 35.32% 35.32% 438.11MB 35.32% cmd/link/internal/objfile.(*objReader).readSlices
after:
inuse_space 1123.39MB total
306.85MB 27.31% 55.03% 306.85MB 27.31% cmd/link/internal/objfile.(*objReader).readSlices
Under GOGC=5 (to simulate system under memory pressure), the max
RSS reduces from ~2.05G to ~1.83G. Even with external linking the
max RSS doesn't increase.
Daniel Theophanes [Fri, 26 Apr 2019 18:09:51 +0000 (11:09 -0700)]
database/sql: check if src is nil before converting to string
A nil src (NULL database value) will result in a "nil" string,
which will never parse correctly in a ParseInt or similar
numeric conversion. The resulting error is confusing. Check
for a nil src prior to converting the value to string
if the resulting string will be parsed after that.
cmd/go: add internal/note, internal/sumweb, internal/tlog from golang.org/x/exp/sumdb
Copied and updated import paths.
Eventually we will probably publish
these packages somewhere in golang.org/x
(as non-internal packages) and then we will
be able to vendor them properly.
For now, copy.
sumweb.globsMatchPath moved to str.GlobsMatchPath.
Change-Id: I4585e6dc5daa423e4ca9669195d41e58e7c8c275
Reviewed-on: https://go-review.googlesource.com/c/go/+/173950 Reviewed-by: Jay Conrod <jayconrod@google.com>
Ian Lance Taylor [Fri, 26 Apr 2019 13:56:58 +0000 (06:56 -0700)]
runtime: test for cgo build tag in TestGdbPythonCgo
Testing whether cgo is enabled in go/build is not the same as testing
whether the go tool supports cgo. They differ, for example, when using
GOARCH=386 on an amd64 system, as for a cross-build cgo is disabled by default.
Change-Id: Ib59106c92a3131b73ac6a91c0f7658a1769acf73
Reviewed-on: https://go-review.googlesource.com/c/go/+/174098
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
AIX doesn't allow to mmap an address range which is already mmap.
Therefore, once the region has been allocated, it must munmap before
being able to play with it.
LE Manh Cuong [Wed, 30 Jan 2019 05:38:15 +0000 (12:38 +0700)]
time: fix misleading error with the leading zero format
When the leading zero format is used, we currently don't handle the
month and year properly.
For the month, we were reporting an out of range error when getnum
returns zero of its own, as it also returns the month 0. That's
confusing, so only check the range when getnum returns a nil error.
For the year, we don't restore the value when parsing error occurs. For
example, with the incorrect input "111-01", "01" will be used to report
an error. So restore the value when an error occurs fix the problem.
Fixes #29918
Fixes #29916
Change-Id: I3145f8c46813a0457766b7c302482e6b56f94ed6
Reviewed-on: https://go-review.googlesource.com/c/go/+/160338
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Expand "go version" from printing just the version of the
go command itself to being able to print the version of any
go binary (when listed on the command line).
This is a simplified version of rsc.io/goversion, which will
now be deprecated in favor of "go version".
(Preparatory changes to runtime, cmd/go, and cmd/link
made this version information easier to find, allowing a
much simpler implementation than in rsc.io/goversion.)
It is useful to be able to dig the Go version out of a binary,
even a stripped binary. rsc.io/goversion does this for x86
binaries by disassembling the binary to find runtime/proc.go's
startup-time reference to runtime.buildVersion's address.
That approach is quite fragile: the implementation doesn't
work for non-x86 and must be updated as the generated
code changes.
rsc.io/goversion finds the module version string by looking
for random 16-byte framing around the actual string.
This is less fragile but fairly kludgy and requires scanning
the entire data segment.
cmd/buildid finds the build ID by looking for an ELF note
or else falling back to scanning the beginning of the text
segment for a magic string. This has proved quite reliable
and doesn't require scanning much of the binary.
This CL makes it possible to find the Go and module versions
using a scan more like the build ID scan: a symbol early in
the writable data segment starts with a magic number and
then has pointers to the two string variables.
Than McIntosh [Thu, 25 Apr 2019 14:35:47 +0000 (10:35 -0400)]
cmd/link: use read-only mmap to back selected symbol name strings
When reading symbol names from an object file, if a name does not
need fixup (conversion of "". to package path), then generate
a string whose backing store is in read-only memory (from an mmap
of the object file), avoiding the need for an allocation. This
yields a modest reduction in total linker heap use.
runtime: switch to P 0 before destroying current P
Ps are strictly numbered from 0 to GOMAXPROCS-1, so if procresize
happens to be running on a P that's being destroyed, it moves itself
to P 0.
However, currently procresize destroys the unused Ps *before* moving
itself to P 0. This means it may briefly run on a destroyed P. This is
basically harmless, but has at least one very confusing consequence:
since destroying a P has write barriers, it may enqueue pointers to a
destroyed write barrier buffer. As far as I can tell, there are no
negative consequences of this, but this seems really fragile.
This CL swaps the order of things, so now procresize moves itself to P
0 if necessary before destroying Ps. This ensures it always has a
valid P.
This is part of refactoring for #10958 and #24543, but is a good
cleanup regardless.
cmd/link: avoid writing to read-only memory in addstrdata
When the linker's -X flag is used, it will overwrite the symbol's
content (sym.P) in addstrdata. The symbol's content may be in
read-only memory, in which case overwriting it will fault. Do
copy-on-write to fix this.
Following discussion on golang.org/issue/26334, this CL changes
the GOPROXY environment setting to be a list of proxies, tried in
sequence. The first successful or non-404/410 error is taken as
authoritative. Otherwise the next proxy is tried, and so on.
As in earlier releases, GOPROXY=direct means "connect directly",
but now it can appear in a longer list as well.
This will let companies run a proxy holding only their private modules
and let users set GOPROXY=thatproxy,publicproxy or GOPROXY=thatproxy,direct
to fall back to an alternate mechanism for fetching public modules.
- rename PasswordRedacted to Redacted
- move URL into Response in redacted form, remove from Get result list
- add Response.Err to construct non-200 errors
(otherwise GetBytes is not just a wrapper)
- make 404/410 errors satisfy Is(err, os.ErrNotExist)
David du Colombier [Thu, 25 Apr 2019 07:01:11 +0000 (09:01 +0200)]
net: fix lookupHost to return DNSError on Plan 9
CL 168597 added IsNotFound field to DNSError.
However, this change broke TestLookupNonLDH on Plan 9
because LookupHost is expected to return a DNSError,
while on Plan 9, it returned an error string.
This change fixes the implementation of lookupHost on
Plan 9 to return a DNSError instead of an error string.
Fixes #31672.
Change-Id: Ia805c8965af63ddee7ccfdebb9462a5502b0269d
Reviewed-on: https://go-review.googlesource.com/c/go/+/173857
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Brian Kessler [Wed, 24 Apr 2019 04:04:38 +0000 (22:04 -0600)]
cmd/compile: add signed divisibility by power of 2 rules
For powers of two (c=1<<k), the divisibility check x%c == 0 can be made
just by checking the trailing zeroes via a mask x&(c-1) == 0 even for signed
integers. This avoids division fix-ups when just divisibility check is needed.
To apply this rule, we match on the fixed-up version of the division. This is
neccessary because the mod and division rewrite rules are already applied
during the initial opt pass.
The speed up on amd64 due to elimination of unneccessary fix-up code is ~55%:
unicode/utf8: remove some bounds checks from DecodeRune
The compiler couldn't quite see that reading p[2] and p[3] was safe.
This change provides a few hints to help it.
First, make sz an int throughout, rather than just when checking the input length.
Second, use <= instead of == in later comparisons.
name old time/op new time/op delta
DecodeASCIIRune-8 2.62ns ± 3% 2.60ns ± 5% ~ (p=0.126 n=18+19)
DecodeJapaneseRune-8 4.46ns ±10% 4.01ns ± 5% -10.00% (p=0.000 n=19+20)
We were using hex literals and had the binary literal in a comment.
When I was working with this code, I always referred to the comment.
That's an indicator that we should just use the binary literal directly.
Than McIntosh [Wed, 24 Apr 2019 12:27:04 +0000 (08:27 -0400)]
test: new test for issue 31637
This pair of packages caused a crash in gollvm, due to a glitch in the
way the front end handles empty/non-name parameters for functions that
are inline candidates.
Updates #31637.
Change-Id: I571c0658a00974dd36025e571638c0c836a3cdfd
Reviewed-on: https://go-review.googlesource.com/c/go/+/173617
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Than McIntosh [Tue, 23 Apr 2019 16:38:36 +0000 (12:38 -0400)]
go/internal/gccgoimporter: revise previous anonymous field fix.
Revise the fix for #31540 to take into account the possibility that we
may see an alias to a name that has already been installed into
package scope. This scenario is not possible to reproduce with the
current importer unit tests; changes to the harness to enable this
scenario will be added in a later CL.
Updates #31540.
Change-Id: Ie155d5e0b998604177a78471cba2413f57d40229
Reviewed-on: https://go-review.googlesource.com/c/go/+/173440 Reviewed-by: Ian Lance Taylor <iant@golang.org>