Daniel Martí [Fri, 7 Jun 2019 09:39:21 +0000 (10:39 +0100)]
os: change UserConfigDir on Darwin to ~/Library/Application Support
The old code used ~/Library/Preferences, which is documented by
Apple as:
This directory contains app-specific preference files. You
should not create files in this directory yourself. Instead, use
the NSUserDefaults class or CFPreferences API to get and set
preference values for your app.
It looks like we missed everything after the first sentence; it's
definitely not the right choice for files that Go programs and users
should be touching directly.
Instead, use ~/Library/Application Support, which is documented as:
Use this directory to store all app data files except those
associated with the user’s documents. For example, you might use
this directory to store app-created data files, configuration
files, templates, or other fixed or modifiable resources that
are managed by the app. An app might use this directory to store
a modifiable copy of resources contained initially in the app’s
bundle. A game might use this directory to store new levels
purchased by the user and downloaded from a server.
This seems in line with what UserConfigDir is for, so use it.
The documentation quotes above are obtained from the surprisingly long
link below:
Keith Randall [Fri, 7 Jun 2019 17:22:53 +0000 (13:22 -0400)]
cmd/link: fix deferreturn detector
The logic for detecting deferreturn calls is wrong.
We used to look for a relocation whose symbol is runtime.deferreturn
and has an offset of 0. But on some architectures, the relocation
offset is not zero. These include arm (the offset is 0xebfffffe) and
s390x (the offset is 6).
This ends up setting the deferreturn offset at 0, so we end up using
the entry point live map instead of the deferreturn live map in a
frame which defers and then segfaults.
Instead, use the IsDirectCall helper to find calls.
Brad Fitzpatrick [Fri, 7 Jun 2019 17:56:24 +0000 (17:56 +0000)]
net/http: support BaseContext & ConnContext for http2 Server
This is the net/http half of #32476. This supplies the method needed
by the other half in x/net/http2 in the already-submitted CL 181259,
which this CL also bundles in h2_bundle.go.
Thanks to Tom Thorogood (@tmthrgd) for the bug report and test.
Fixes #32476
Updates #30694
Change-Id: I79d2a280e486fbf75d116f6695fd3abb61278765
Reviewed-on: https://go-review.googlesource.com/c/go/+/181260
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Austin Clements [Thu, 6 Jun 2019 21:27:14 +0000 (17:27 -0400)]
cmd/dist,cmd/compile: remove -allabis mode
dist passes the -allabis flag to the compiler to avoid having to
recreate the cross-package ABI logic from cmd/go. However, we removed
that logic from cmd/go in CL 179863 and replaced it with a different
mechanism that doesn't depend on the build system. Hence, passing
-allabis in dist is no longer necessary.
This CL removes -allabis from dist and, since that was the only use of
it, removes support for it from the compiler as well.
Matt Layher [Fri, 7 Jun 2019 17:07:32 +0000 (13:07 -0400)]
syscall: refer to Proc.Call documentation for LazyProc.Call on Windows
The documentation comment was duplicated for each of these methods, and
the LazyProc.Call documentation incorrectly mentioned that Call accepts
only 15 arguments, but it actually accepts 18 now.
To prevent further documentation drift, refer the reader to the
documentation for Proc.Call instead of duplicating it for LazyProc.Call.
In addition, note that LazyProc's Addr, Call, and Find methods each
trigger a procedure lookup.
Constantin Konstantinidis [Sat, 1 Jun 2019 16:14:04 +0000 (18:14 +0200)]
cmd/go: fix error messages for go mod download in GOPATH mode
Checks if modules are enabled in GOPATH mode.
Error message returned when no version is provided. Relevant tests
updated. Test for GO111MODULE=off added.
Fixes #27783
Change-Id: I12cdaced5fa38a9c49c0ecfed4c479eb86ed061f
Reviewed-on: https://go-review.googlesource.com/c/go/+/179998
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Daniel Martí [Fri, 31 May 2019 09:46:00 +0000 (10:46 +0100)]
cmd/go: recommend -d when -m is used
In #32038, it was decided to remove get's -m, since one former use case
is removed, and the other can be done via -d, as pointed by Russ.
However, a user getting this short error might not realise that they can
switch to -d to skip building packages. Add a short mention to point
them in the right direction.
It's important to note "packages", because -m was a flag that acted on
modules, while -d acts on packages. Simply replacing -m with -d might
not be enough in some cases because of that distinction.
Change-Id: I0947b25c4223bdad3cd0e535848527da8db8a16d
Reviewed-on: https://go-review.googlesource.com/c/go/+/179361 Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Daniel Martí [Tue, 16 Apr 2019 13:13:47 +0000 (22:13 +0900)]
text/template: clarify the safety of Template.New
In particular, the returned template isn't independent from the parent.
For example, it can't be parsed concurrently with other children
templates. Only methods which are explicitly safe for concurrent use,
like Execute, may be used concurrently.
Fixes #30281.
Change-Id: Idc84bf4199c035316cdb83b950fd4a8f2a71cd0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/172297
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
Austin Clements [Thu, 6 Jun 2019 21:23:19 +0000 (17:23 -0400)]
runtime/internal/atomic: remove erroneous ABI wrappers
CL 179862 introduced go:linkname directives to create ABI wrappers for
Store and Store64 on s390x, but a concurrent change (CL 180439)
replaced the Go definitions of these functions with assembly
definitions. This resulted in conflicting definitions for the ABI0
symbols, which led to a bootstrap linking failure.
Fix this by removing the now-incorrect go:linkname directives for
Store and Store64. This should fix the linux-s390x builders.
Updates #31230.
Change-Id: I8de8c03c23412fc217d428c0018cc56eb2f9996f
Reviewed-on: https://go-review.googlesource.com/c/go/+/181078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Russ Cox [Wed, 5 Jun 2019 23:59:58 +0000 (19:59 -0400)]
net: remove non-cgo macOS resolver code
The built-in Go resolver works significantly better.
In particular, the use of res_search does not support
CNAME or PTR queries and may not even be thread-safe.
This CL is essentially a revert of CL 166297 plus fixes,
including CL 180842.
See CL 180842 for additional notes about problems
with this approach.
Russ Cox [Wed, 5 Jun 2019 16:35:29 +0000 (12:35 -0400)]
net: fix non-cgo macOS resolver code
This code was added in April in CL 166297, for #12524.
This CL fixes the following problems in the code:
- The test for failure in the assembly stubs checked for
64-bit -1 instead of 32-bit -1 to decide to fetch errno.
- These C routines (res_init and res_search) don't set errno anyway,
so the Go code using errno to decide success is incorrect.
(The routines set h_errno, which is a racy global variable
that can't safely be consulted, storing values in a different
error space.)
- The Go call passed res_search a non-NUL-terminated name.
- The C res_search rejects calls asking for TypeALL as opposed to
more specific answers like TypeA/TypeAAAA/TypeCNAME,
breaking cgoLookupHost in all cases and cgoLookupIP
except with IP-version-specific networks.
- The DNS response packet was parsed twice, once with msg.Unpack
(discarded), and once with the lower-level dnsmessage.Parser.
The Parser loop was missing a call to p.SkipAllQuestions, with the
result that no DNS response packet would ever parse successfully.
- The parsing of the DNS response answers, if reached, behaved as if
that the AResource and AAAAResource record contained textual
IP addresses, while in fact they contain binary ones. The calls to
parseIPv4 and parseIPv6 therefore would always returns nil,
so that no useful result would be returned from the resolver.
With these fixes, cgoLookupIP can correctly resolve google.com
and return both the A and AAAA addresses.
Even after fixing all these things, TestGoLookupIP still fails,
because it is testing that in non-cgo builds the cgo stubs
correctly report "I can't handle the lookup", and as written the
code intentionally violates that expectation.
This CL adds new direct tests of the pseudo-cgo routines.
The direct IP address lookups succeed, but the CNAME query
causes res_search to hang, and the PTR query fails unconditionally
(a trivial C program confirms these behaviors are due to res_search itself).
Traditionally, res_search is only intended for single-threaded use.
It is unclear whether this one is safe for use from multiple goroutines.
If you run net.test under lldb, that causes syslog messages to be
printed to standard error suggesting double-free bugs:
2019-06-05 19:52:43.505246-0400 net.test[6256:6831076] dnssd_clientstub DNSServiceRefDeallocate called with invalid DNSServiceRef 0x5c000f0 FFFFFFFFDDDDDDDD
2019-06-05 19:52:43.505274-0400 net.test[6256:6831076] dnssd_clientstub DNSServiceRefDeallocate called with invalid DNSServiceRef 0x5c000f0 FFFFFFFFDDDDDDDD
2019-06-05 19:52:43.505303-0400 net.test[6256:6831076] dnssd_clientstub DNSServiceRefDeallocate called with invalid DNSServiceRef 0x5c000f0 FFFFFFFFDDDDDDDD
2019-06-05 19:52:43.505329-0400 net.test[6256:6831076] dnssd_clientstub DNSServiceRefDeallocate called with invalid DNSServiceRef 0x5c000f0 FFFFFFFFDDDDDDDD
This res_search is from libsystem_info; a normal C program would
get res_search (#defined to res_9_search) from libresolv instead.
It is unclear what the relation between the two is.
Issue #12524 was about supporting the /etc/resolver directory tree,
but only libresolv contains code for that; libsystem_info does not.
So this code probably does not enable use of /etc/resolver.
In short:
- Before this CL, the code clearly had never run successfully.
- The code appears not to improve upon the usual non-cgo fallback.
- The code carries with it no tests of improved behavior.
- The code breaks existing tests.
- Calling res_search does not work for PTR/CNAME queries,
so the code breaks existing behavior, even after this CL.
- It's unclear whether res_search is safe to call from multiple threads.
- It's unclear whether res_search is used by any other macOS programs.
Given this, it probably makes sense to delete this code rather
than rejigger the test. This CL fixes the code first, so that there
is a working copy to bring back later if we find out that it really
is necessary.
Russ Cox [Wed, 5 Jun 2019 18:51:57 +0000 (14:51 -0400)]
runtime: document, fix libc error checks on macOS
It matters whether we are calling a function that would
return a 32-bit or 64-bit -1 on error. A few sites were wrong
and this key detail was omitted from syscall/syscallX docs.
This removes the special case for finding assembly references to Go
symbols in runtime and runtime/internal/atomic. These are no longer
necessary because we've now marked all symbols in these packages that
must be accessible from assembly in other packages.
Austin Clements [Fri, 31 May 2019 20:38:56 +0000 (16:38 -0400)]
runtime: mark all Go symbols called from assembly in other packages
This marks all Go symbols called from assembly in other packages with
"go:linkname" directives to ensure they get ABI wrappers.
Now that we have this go:linkname convention, this also removes the
abi0Syms definition in the runtime, which was used to give morestackc
an ABI0 wrapper. Instead, we now just mark morestackc with a
go:linkname directive.
This was tested with buildall.bash in the default configuration, with
-race, and with -gcflags=all=-d=ssa/intrinsics/off. Since I couldn't
test cgo on non-Linux configurations, I manually grepped for runtime
symbols in runtime/cgo.
Austin Clements [Fri, 31 May 2019 19:45:06 +0000 (15:45 -0400)]
cmd/compile: make the second argument to go:linkname optional
The //go:linkname directive can be used to make a symbol accessible to
another package (when it wouldn't normally be). Sometimes you want to
do this without actually changing the symbol's object file symbol
name; for example, in gccgo this makes unexported symbols non-static,
and in gc this provides ABI0 wrappers for Go symbols so they can be
called from assembly in other packages. Currently, this results in
stutter like
//go:linkname entersyscall runtime.entersyscall
This CL makes the second argument to go:linkname optional for the case
where the intent is simply to expose the symbol rather than to rename
it in the object file.
Austin Clements [Fri, 31 May 2019 19:32:22 +0000 (15:32 -0400)]
cmd/compile: generate ABI wrappers for //go:linkname'd symbols
Calling a Go symbol from assembly in another package currently results
in a link failure because the Go symbol is defined as ABIInternal, but
the assembly call is from ABI0. In general this is okay because you
shouldn't do this anyway, but there are special cases where this is
necessary, especially between the runtime and packages closely tied to
the runtime in std.
Currently, we address this for runtime symbols with a hack in cmd/go
that knows to scan related packages when building the symabis file for
the runtime and runtime/internal/atomic. However, in addition to being
a messy solution in the first place, this hack causes races in cmd/go
that are difficult to work around.
We considered creating dummy references from assembly in the runtime
to these symbols, just to make sure they get ABI0 wrappers. However,
there are a fairly large number of these symbols on some platforms,
and it can vary significantly depending on build flags (e.g., race
mode), so even this solution is fairly unpalatable.
This CL addresses this by providing a way to mark symbols in Go code
that should be made available to assembly in other packages. Rather
than introduce a new pragma, we lightly expand the meaning of
"//go:linkname", since that pragma already generally indicates that
you're making the symbol available in a way it wasn't before. This
also dovetails nicely with the behavior of go:linkname in gccgo, which
makes unexported symbols available to other packages.
Follow-up CLs will make use of this and then remove the hack from
cmd/go.
Michael Munday [Tue, 4 Jun 2019 18:17:41 +0000 (19:17 +0100)]
cmd/compile, runtime: make atomic loads/stores sequentially consistent on s390x
The z/Architecture does not guarantee that a load following a store
will not be reordered with that store, unless they access the same
address. Therefore if we want to ensure the sequential consistency
of atomic loads and stores we need to perform serialization
operations after atomic stores.
We do not need to serialize in the runtime when using StoreRel[ease]
and LoadAcq[uire]. The z/Architecture already provides sufficient
ordering guarantees for these operations.
David Chase [Wed, 5 Jun 2019 18:53:28 +0000 (14:53 -0400)]
cmd/compile: add -smallframes gc flag for GC latency diagnosis
Shrinks the size of things that can be stack allocated from
10M to 128k for declared variables and from 64k to 16k for
implicit allocations (new(T), &T{}, etc).
Usage: "go build -gcflags -smallframes hello.go"
An earlier GOEXPERIMENT version of this caused only one
problem, when a gc-should-detect-oversize-stack test no
longer had an oversized stack to detect. The change was
converted to a flag to make it easier to access (for
diagnosing "long" GC-related single-thread pauses) and to
remove interference with the test.
Includes test to verify behavior.
Updates #27732.
Change-Id: I1255d484331e77185e07c78389a8b594041204c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/180817
Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Meng Zhuo [Thu, 6 Jun 2019 10:07:13 +0000 (18:07 +0800)]
syscall: skip test if unprivileged_userns_clone sysctl is missing
The original test (CL 166460) didn't check the existence of
/proc/sys/kernel/unprivileged_userns_clone and continue the test
if the file doesn't exist.
Russ Cox [Wed, 5 Jun 2019 16:31:42 +0000 (12:31 -0400)]
runtime: use default system stack size, not 64 kB, on non-cgo macOS
At least one libc call we make
(res_search, which calls _mdns_query and then mdns_item_call)
pushes a 64 kB stack frame onto the stack.
Then it faults on the guard page.
Use the default system stack size, under the assumption
that the C code being called is compatible with that stack size.
For #31705.
Change-Id: I1b0bfc2e54043c49f0709255988ef920ce30ee82
Reviewed-on: https://go-review.googlesource.com/c/go/+/180779
Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Myasoedov [Tue, 4 Jun 2019 18:07:57 +0000 (20:07 +0200)]
doc: clarify safety of multiple and concurent os.(*File).Close() calls
Fixes #32427
Change-Id: I4b863bd3836067dcc2eb3a9c3a7169656763d003
Reviewed-on: https://go-review.googlesource.com/c/go/+/180438 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Bryan C. Mills [Tue, 4 Jun 2019 20:50:59 +0000 (16:50 -0400)]
cmd/go: replace uses of ioutil.ReadFile with renameio.ReadFile
Windows does not have atomic renames; instead, it produces one of a
handful of errors in case a read races with a rename.
CL 180219 added a utility function that retries those errors in most
cases; this change updates the locations that use renameio for writes
to also use the new renameio.ReadFile function for reads.
It remains possible for a renameio.ReadFile to fail with a spurious
ERROR_FILE_NOT_FOUND, but with retries in place for the other errors
(and practical limits on write concurrency) such failures are unlikely
in practice.
Fixes #32188
Change-Id: I78c81051cc871325c1e3229e696b921b0fcd865a
Reviewed-on: https://go-review.googlesource.com/c/go/+/180517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Ian Lance Taylor [Tue, 4 Jun 2019 01:04:58 +0000 (18:04 -0700)]
cmd/cgo: roll back "use C exact-width integer types to represent Go types"
Roll back CL 159258 and CL 168337. Those changes broke existing
code. I can't see any way to keep existing code working while also
producing good error messages for types like C.ulong (such as the ones
already tested for in misc/cgo/errors).
This is not an exact roll back because parts of the code have changed
since those CLs.
Updates #29878
Fixes #31093
Change-Id: I56fe76c167ff0ab381ed273b9ca4b952402e1434
Reviewed-on: https://go-review.googlesource.com/c/go/+/180357
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Dmitri Shuralyov [Tue, 4 Jun 2019 21:21:33 +0000 (21:21 +0000)]
net/http: roll back "clean the path of the stripped URL by StripPrefix"
Roll back CL 161738. That fix changed StripPrefix behavior in the
general case, not just in the situation where where stripping the
prefix from path resulted in the empty string, causing issue #31622.
That kind of change to StripPrefix behavior is not backwards compatible,
and there can be a smaller, more targeted fix for the original issue.
Tyler Bui-Palsulich [Tue, 28 May 2019 19:50:01 +0000 (15:50 -0400)]
cmd/go: document GO{NO}SUMDB in 'go help environment' and 'go help modules'
GOSUMDB and GONOSUMDB are described in detail by
'go help module-auth'. So, this change mentions the two
variables and says to see 'go help module-auth'.
This also adds GONOPROXY to 'go help environment'.
Agniva De Sarker [Tue, 4 Jun 2019 03:56:35 +0000 (09:26 +0530)]
doc: fix missing anchor links in contribute.html
Some <h3>s were missing ids due to which the anchor links
weren't getting generated.
Fixes #32415
Change-Id: Ica21425c1a7c49735231c1de96b6c77dd594ce64
Reviewed-on: https://go-review.googlesource.com/c/go/+/180397 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Andrew Gerrand [Fri, 31 May 2019 10:33:01 +0000 (20:33 +1000)]
errors: fix package example
The example in example_test.go requires that the whole file be
displayed; the addition of ExampleAs meant that only the body of the
package example function was shown, rather than the surrounding context.
This change moves ExampleAs to the file wrap_test.go file, restoring the
package example to its former glory.
Update #31716
Change-Id: Id0ea77bc06023b239a63c1d6a7c8b3c1dae91ce9
Reviewed-on: https://go-review.googlesource.com/c/go/+/179737 Reviewed-by: Marcel van Lohuizen <mpvl@golang.org> Reviewed-by: Jean de Klerk <deklerk@google.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Keith Randall [Mon, 3 Jun 2019 17:03:44 +0000 (10:03 -0700)]
runtime: get map of args of unstarted goroutines like we do for defers
Normally, reflect.makeFuncStub records the context value at a known
point in the stack frame, so that the runtime can get the argument map
for reflect.makeFuncStub from that known location.
This doesn't work for defers or goroutines that haven't started yet,
because they haven't allocated a frame or run an instruction yet. The
argument map must be extracted from the context value. We already do
this for defers (the non-nil ctxt arg to getArgInfo), we just need to
do it for unstarted goroutines as well.
When we traceback a goroutine, remember the context value from
g.sched. Use it for the first frame we find.
(We never need it for deeper frames, because we normally don't stop at
the start of reflect.makeFuncStub, as it is nosplit. With this CL we
could allow makeFuncStub to no longer be nosplit.)
Andrew Gerrand [Fri, 31 May 2019 06:58:44 +0000 (16:58 +1000)]
cmd/cover: fix counting of blocks split by goto statements
When adding coverage counters to a block, the block's statement list is
mutated. CL 77150 removed the part where the mutated list is assigned
back to its parent node; this was confusing ast.Walk, which would then
lose its place and stop walking the current block, dropping counters in
the process.
This change has addCounters make a copy of the list before mutating
it, so that the original list doesn't change under Walk's feet.
Fix #32200
Change-Id: Ia3b67d8cee860ceb7caf8748cb7a80ff9c6276e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/179581 Reviewed-by: Rob Pike <r@golang.org>
Jay Conrod [Fri, 31 May 2019 18:14:00 +0000 (14:14 -0400)]
cmd/go: ignore build tags when 'go get' modifies build list
In module mode, 'go get' should not consider build constraints when
loading packages in order to modify the module graph. With this
change, 'go get' considers all build tags to be true except for
"ignore" and malformed build constraint expressions.
When 'go get' builds packages, it still applies build constraints for
the target platform.
Fixes #32345
Change-Id: I6dceae6f10a5185870537de730b36292271ad124
Reviewed-on: https://go-review.googlesource.com/c/go/+/179898 Reviewed-by: Bryan C. Mills <bcmills@google.com>
Keith Randall [Tue, 28 May 2019 21:59:23 +0000 (14:59 -0700)]
cmd/compile: don't move nil checks across a VarDef
We need to make sure that there's no possible faulting
instruction between a VarDef and that variable being
fully initialized. If there was, then anything scanning
the stack during the handling of that fault will see
a live but uninitialized variable on the stack.
If we have:
NilCheck p
VarDef x
x = *p
We can't rewrite that to
VarDef x
NilCheck p
x = *p
Particularly, even though *p faults on p==nil, we still
have to do the explicit nil check before the VarDef.
Cherry Zhang [Fri, 31 May 2019 21:11:50 +0000 (17:11 -0400)]
cmd/compile: make sure build works when intrinsics are disabled
Some runtime functions, like getcallerpc/sp, don't have Go or
assembly implementations and have to be intrinsified. Make sure
they are, even if intrinsics are disabled.
This makes "go build -gcflags=all=-d=ssa/intrinsics/off hello.go"
work.
Bryan C. Mills [Fri, 31 May 2019 19:24:49 +0000 (15:24 -0400)]
cmd/go/internal/modfetch: use the resolved version to search for tags in (*codeRepo).convert
Previously, we used the passed-in statVers as the basis for tag search,
but it is not always valid.
Instead, use info.Name, which (by precondition) must be valid.
Updates #32161
Updates #27171
Change-Id: Iaecb5043bdf2fefd26fbe3f8e3714b07d22f580f
Reviewed-on: https://go-review.googlesource.com/c/go/+/179857
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Than McIntosh [Fri, 31 May 2019 16:05:07 +0000 (12:05 -0400)]
cmd/link: revise test case to work on pre-10.14 macos
Rework this recently introduced test case to insure that it works with
older versions of the OS. It was using a new framework library not
available on pre-10.14 to trigger the weak symbol reference; switch to
using a new symbol from an existing library. Tested on MacOS 10.14 and
10.11.
Updates #32233.
Change-Id: I1fe2a9255fca46cb7cdf33ff7fed67bba86fdc22
Reviewed-on: https://go-review.googlesource.com/c/go/+/179837
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Andrew Gerrand [Fri, 31 May 2019 10:59:35 +0000 (20:59 +1000)]
strconv: document handling of NaN and ±Inf
In addition to the example that was added in 203b80ab, mention these
special cases in the doc comment. This change also adjusts the example
to include "+Inf", as it was not otherwise mentioned that the plus
symbol may be present.
Brad Fitzpatrick [Wed, 29 May 2019 21:49:20 +0000 (21:49 +0000)]
net/http: prevent Transport from spamming stderr on server 408 reply
HTTP 408 responses now exist and are seen in the wild (e.g. from
Google's GFE), so make Go's HTTP client not spam about them when seen.
They're normal (now).
Greg Thelen [Sat, 25 May 2019 18:44:44 +0000 (11:44 -0700)]
syscall: use Ctty before fd shuffle
On unix if exec.Command() is given both ExtraFiles and Ctty, and the
Ctty file descriptor overlaps the range of FDs intended for the child,
then cmd.Start() the ioctl(fd,TIOCSCTTY) call fails with an
"inappropriate ioctl for device" error.
When child file descriptors overlap the new child's ctty the ctty will
be closed in the fd shuffle before the TIOCSCTTY. Thus TIOCSCTTY is
used on one of the ExtraFiles rather than the intended Ctty file. Thus
the error.
exec.Command() callers can workaround this by ensuring the Ctty fd is
larger than any ExtraFiles destined for the child.
Fix this by doing the ctty ioctl before the fd shuffle.
Test for this issue by modifying TestTerminalSignal to use more
ExtraFiles. The test fails on linux and freebsd without this change's
syscall/*.go changes. Other platforms (e.g. darwin, aix, solaris) have
the same fd shuffle logic, so the same fix is applied to them. However,
I was only able to test on linux (32 and 64 bit) and freebsd (64 bit).
Manual runs of the test in https://golang.org/issue/29458 start passing
with this patch:
Before:
% /tmp/src/go/bin/go run t
successfully ran child process with ParentExtraFileFdNum=5, ChildExtraFileFd=6, ParentPtyFd=7
panic: failed to run child process with ParentExtraFileFdNum=10, ChildExtraFileFd=11, ParentPtyFd=11: fork/exec /bin/true: inappropriate ioctl for device
After:
% /tmp/src/go/bin/go run t
successfully ran child process with ParentExtraFileFdNum=5, ChildExtraFileFd=6, ParentPtyFd=7
successfully ran child process with ParentExtraFileFdNum=10, ChildExtraFileFd=11, ParentPtyFd=11
Fixes #29458
Change-Id: I99513de7b6073c7eb855f1eeb4d1f9dc0454ef8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/178919
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Johan Brandhorst [Tue, 28 May 2019 20:51:54 +0000 (21:51 +0100)]
net/http: enable WASM fetch where supported
The existing check was introduced to allow tests to pass
on WASM without an environment where the fetch RoundTripper
could run. However, the check now prohibits the use of the
Fetch RoundTripper in all WASM tests, even where the
RoundTripper could run. The new change should only disable
the RoundTripper when used in an environment without fetch.
Matthew Dempsky [Thu, 28 Mar 2019 21:35:49 +0000 (14:35 -0700)]
cmd/compile: fix package initialization ordering
This CL rewrites cmd/compile's package-level initialization ordering
algorithm to be compliant with the Go spec. See documentation in
initorder.go for details.
Incidentally, this CL also improves fidelity of initialization loop
diagnostics by including referenced functions in the emitted output
like go/types does.
Fixes #22326.
Change-Id: I7c9ac47ff563df4d4f700cf6195387a0f372cc7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/170062
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Jason A. Donenfeld [Sun, 26 May 2019 13:28:05 +0000 (15:28 +0200)]
cmd/link: do not generate NT 4 compatibility binaries
Incredibly, the subsystem version numbers in the PE header influence how
win32k handles various syscalls. The first time a win32k syscall is
invoked and the kernel upgrades the thread object to a tagTHREADINFO
with all of the lovely undocumented UI members and such, it sets the
dwExpWinVer member (offset 624 in Windows 10 build 1809) to the result
of RtlGetExpWinVer(PsGetProcessSectionBaseAddress(proc)).
RtlGetExpWinVer, also undocumented, then calls into the undocumented
RtlImageNtHeader function, which returns a fortunately documented
IMAGE_NT_HEADERS structure. It uses the subsystem members in there to
set the dwExpWinVer member of our newly minted tagTHREADINFO object.
Later, functions like SendInput consult this to vary their behaviors and
return values. In fact, littered through out win32k are checks like `if
(gsti->dwExpWinVer >= 0x501) { ... }`.
I don't think Go ever supported NT 4.0. These days the minimum version
is Windows 7, which is 6.1. So, let's set the version numbers in the PE
header at that, which should give us the behavior that MSDN advertises
for various functions, as opposed to bizarre archeological remnants.
Interestingly, I suspect that most people never noticed the brokenness,
because most people trying to do serious Win32 UI stuff wind up linking
in cgo, if not for actually using C, then just to have a larger system
stack so that the stack doesn't get corrupted by various UI functions.
When MingW is used, the PE header gets a later version. But recently
there's been a bug report of some people trying to do more modest UI
manipulation using SendInput in a setting where this cgo hack probably
isn't required, so they ran into the weird historical compatibility
stuff.
Fixes #31685
Change-Id: I54461ce820f6e9df349e37be5ecc5a44c04a3e26
Reviewed-on: https://go-review.googlesource.com/c/go/+/178977
Run-TryBot: Jason Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Brad Fitzpatrick [Tue, 28 May 2019 17:45:22 +0000 (17:45 +0000)]
net/http: quiet some log spam in tests
One of these tests creates a bunch of connections concurrently, then
discovers it doesn't need them all, which then makes the server log
that the client went away midway through the TLS handshake. Perhaps
the server should recognize that as a case not worthy of logging
about, but this is a safer way to eliminate the stderr spam during go
test for now.
The other test's client gives up on its connection and closes it,
similarly confusing the server.
cmd/compile: process blocks containing only dead values in fuseIf
The code in #29218 resulted in an If block containing only its control.
That block was then converted by fuseIf into a plain block;
as a result, that control value was dead.
However, the control value was still present in b.Values.
This prevented further fusing of that block.
This change beefs up the check in fuseIf to allow fusing
blocks that contain only dead values (if any).
In the case of #29218, this enables enough extra
fusing that the control value could be eliminated,
allowing all values in turn to be eliminated.
This change also fuses 34 new blocks during make.bash.
It is not clear that this fixes every variant of #29218,
but it is a reasonable standalone change.
And code like #29218 is rare and fundamentally buggy,
so we can handle new instances if/when they actually occur.
Rob Pike [Fri, 17 May 2019 03:43:51 +0000 (13:43 +1000)]
cmd/doc: always print package clause except for commands
There was an implicit heuristic before about when to print the
package clause or omit it, but it was undocumented and confusing.
Get rid of it and print it always unless asking for the package
docs for a command, which are more of a usage message than a
programming question. This simplifies the processing.
There are several paths to the output, so to put the fix in one
place we place a wrapper before the output buffer than adds the
clause when Write is first called.
The tests don't verify this behavior, but they didn't before either.
Unsure what the right approach is but this will do for now.
Ian Lance Taylor [Wed, 1 May 2019 04:06:01 +0000 (21:06 -0700)]
runtime: remove VDSO fallback test and benchmarks
These tests assume that it is OK to switch between time implementations,
but the clock_gettime call uses CLOCK_MONOTONIC and the fallback call,
gettimeofday, uses CLOCK_REALTIME. Disabling the clock_gettime call means
that calls to nanotime will start returning very different values.
This breaks the new timer code, which assumes that nanotime will return
a consistently increasing value.
This test is not very useful in any case as it doesn't check the results.
Removing this file also removes BenchmarkTimeNow, which is a duplicate
of BenchmarkNow in the time package.
Russ Cox [Fri, 24 May 2019 12:55:30 +0000 (08:55 -0400)]
cmd/go: respect default proxy setting, add direct fallback
Getenv("GOPROXY") says what the environment variable is
(including looking in the go env file), but it doesn't include
the default setting. This code needs to use cfg.GOPROXY
to get the actual default. Fix and test that.
Also, we forgot to include the fallback to direct for when
the proxy serves a 404. Add and test that too.
Also add HTTP fetch information to -x build flag output.
(It does not belong in the -v output, despite the GOPATH go get
command doing this.)
Change-Id: Ieab7ef13cda3e1ad041dbe04921af206e2232c9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/178720
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Bryan C. Mills [Thu, 16 May 2019 13:21:49 +0000 (09:21 -0400)]
cmd/go: when resolving packages, try all module paths before falling back to the next proxy
Since we're mucking with error-propagation in modload.Query* anyway,
simplify the classification logic. Ensure that “module not found”
errors are reported as such in various places, since non-“not found”
errors terminate the module search.
Following the previous CL, this removes more global variables on
Wasm.
PC_B is used mostly for intra-function jumps, and for a function
telling its callee where to start or resume. This usage can be
served by a parameter. The top level loop (wasm_pc_f_loop) uses
PC_B for resuming a function. This value is either set by gogo,
or loaded from the Go stack at function return. Instead of
loading PC_B at each function return, we could make gogo stores
PC_B at the same stack location, and let the top level loop do
the load. This way, we don't need to use global PC_B to
communicate with the top level loop, and we can replace global
PC_B with a parameter.
PC_F is similar. It is even more so in that the only reader is
the top level loop. Let the top level loop read it from the stack,
and we can get rid of PC_F entirely.
PC_F and PC_B are used less entensively as SP, so this CL has
smaller performance gain.
Michael Anthony Knyszek [Fri, 17 May 2019 14:48:04 +0000 (14:48 +0000)]
runtime: ensure mheap lock stack growth invariant is maintained
Currently there's an invariant in the runtime wherein the heap lock
can only be acquired on the system stack, otherwise a self-deadlock
could occur if the stack grows while the lock is held.
This invariant is upheld and documented in a number of situations (e.g.
allocManual, freeManual) but there are other places where the invariant
is either not maintained at all which risks self-deadlock (e.g.
setGCPercent, gcResetMarkState, allocmcache) or is maintained but
undocumented (e.g. gcSweep, readGCStats_m).
This change adds go:systemstack to any function that acquires the heap
lock or adds a systemstack(func() { ... }) around the critical section,
where appropriate. It also documents the invariant on (*mheap).lock
directly and updates repetitive documentation to refer to that comment.
Martin Sucha [Thu, 23 May 2019 18:34:17 +0000 (20:34 +0200)]
strings: clarify example of ContainsAny
I have seen code that literally copied the example like this:
if strings.ContainsAny(s, "1 & 2 & 3") {
The developer apparently thought that this is the way to
specify multiple characters and I noticed this pattern
being used in the example. Let's update the example so
that it's clear how multiple Unicode code points should
be specified.
Richard Musiol [Wed, 15 May 2019 23:03:10 +0000 (01:03 +0200)]
syscall/js: replace TypedArrayOf with CopyBytesToGo/CopyBytesToJS
The typed arrays returned by TypedArrayOf were backed by WebAssembly
memory. They became invalid each time we grow the WebAssembly memory.
This made them very error prone and hard to use correctly.
This change removes TypedArrayOf completely and instead introduces
CopyBytesToGo and CopyBytesToJS for copying bytes between a byte
slice and an Uint8Array. This breaking change is still allowed for
the syscall/js package.
bill_ofarrell [Thu, 16 May 2019 16:45:52 +0000 (12:45 -0400)]
crypto/ecdsa: implement ecdsa on s390x for P256/P384/P521 using KDSA instruction
Utilize KDSA when available. This guarantees constant time operation on all three curves mentioned,
and is faster than conventional assembly. The IBM Z model(s) that support KDSA as used in this CL
are not yet publicly available, and so we are unable to release performance data at this time.
Change-Id: I85360dcf90fe42d2bf32afe3f638e282de10a518
Reviewed-on: https://go-review.googlesource.com/c/go/+/174437
Run-TryBot: Michael Munday <mike.munday@ibm.com> Reviewed-by: Michael Munday <mike.munday@ibm.com>
Brad Fitzpatrick [Thu, 23 May 2019 14:01:16 +0000 (14:01 +0000)]
SECURITY.md: add security file
This is now recognized and recommended by GitHub.
Fixes #32201
Change-Id: Iafb5ef1b2bee5f021a711b0b758aaf6a74758c5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/178697 Reviewed-by: Ian Lance Taylor <iant@golang.org>