sort.Slice was added in Go 1.8.
It's nice to use, and faster than sort.Sort,
so it'd be nice to be able to use it in the toolchain.
This CL adds obj.SortSlice, which is sort.Slice,
but with a slower fallback version for bootstrapping.
Joel Sing [Tue, 11 Apr 2017 15:10:01 +0000 (01:10 +1000)]
cmd/link,runtime/cgo: enable PT_TLS generation on OpenBSD
OpenBSD 6.0 and later have support for PT_TLS in ld.so(1). Now that OpenBSD
6.1 has been released, OpenBSD 5.9 is no longer officially supported and Go
can start generating PT_TLS for OpenBSD cgo binaries. This also allows us
to remove the workarounds in the OpenBSD cgo runtime.
This change also removes the environ and progname exports - these are now
provided directly by ld.so(1) itself.
Fixes #19932
Change-Id: I42e75ef9feb5dcd4696add5233497e3cbc48ad52
Reviewed-on: https://go-review.googlesource.com/40331 Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/internal/obj/s390x: make assembler almost concurrency-safe
CL 39922 made the arm assembler concurrency-safe.
This CL does the same, but for s390x.
The approach is similar: introduce ctxtz to hold
function-local state and thread it through
the assembler as necessary.
One race remains after this CL, similar to CL 40252.
That race is conceptually unrelated to this refactoring,
and will be addressed in a separate CL.
cmd/internal/obj/ppc64: make assembler almost concurrency-safe
CL 39922 made the arm assembler concurrency-safe.
This CL does the same, but for ppc64.
The approach is similar: introduce ctxt9 to hold
function-local state and thread it through
the assembler as necessary.
One race remains after this CL, similar to CL 40252.
That race is conceptually unrelated to this refactoring,
and will be addressed in a separate CL.
cmd/internal/obj/arm64: make assembler almost concurrency-safe
CL 39922 made the arm assembler concurrency-safe.
This CL does the same, but for arm64.
The approach is similar: introduce ctxt7 to hold
function-local state and thread it through
the assembler as necessary.
One race remains after this CL, deep in aclass,
in the check that a Prog does not take the address
of a TLS variable.
That race is conceptually unrelated to this refactoring,
and will be addressed in a separate CL.
cmd/internal/obj/mips: make assembler almost concurrency-safe
CL 39922 made the arm assembler concurrency-safe.
This CL does the same, but for mips.
The approach is similar: introduce ctxt0 to hold
function-local state and thread it through
the assembler as necessary.
One race remains after this CL, similar to CL 40252.
That race is conceptually unrelated to this refactoring,
and will be addressed in a separate CL.
Ben Shi [Mon, 27 Feb 2017 07:56:57 +0000 (07:56 +0000)]
runtime: use hardware divider to improve performance
The hardware divider is an optional component of ARMv7. This patch
detects whether it is available in runtime and use it or not.
1. The hardware divider is detected at startup and a flag is set/clear
according to a perticular bit of runtime.hwcap.
2. Each call of runtime.udiv will check this flag and decide if
use the hardware division instruction.
A rough test shows the performance improves 40-50% for ARMv7. And
the compatibility of ARMv5/v6 is not broken.
Changing mheap_.arena_used requires several steps that are currently
repeated multiple times in mheap_.sysAlloc. Consolidate these into a
single function.
In the future, this will also make it easier to add other auxiliary VM
structures.
Change-Id: Ie68837d2612e1f4ba4904acb1b6b832b15431d56
Reviewed-on: https://go-review.googlesource.com/40151
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
George Gkirtsou [Sun, 9 Apr 2017 20:12:39 +0000 (21:12 +0100)]
os: more descriptive error for File.ReadAt and File.WriteAt with negative offset.
The existing implementation does not provide a useful error message
if a negative offset is passed in File.ReadAt or File.WriteAt. This
change is to return descriptive errors. An error of type *PathError
is returned to keep it consistent with rest of the code.
There is no need to add an exported error variable since it's used only
in one file.
Kevin Burke [Mon, 10 Apr 2017 17:22:26 +0000 (10:22 -0700)]
crypto/tls: recommend P256 elliptic curve
Users (like myself) may be tempted to think the higher-numbered curve
is somehow better or more secure, but P256 is currently the best
ECDSA implementation, due to its better support in TLS clients, and a
constant time implementation.
For example, sites that present a certificate signed with P521
currently fail to load in Chrome stable, and the error on the Go side
says simply "remote error: tls: illegal parameter".
cmd/internal/obj/arm: make assembler concurrency-safe
Move global state from obj.Link
to a new function-local state struct arm.ctxt5.
This ends up being cleaner than threading
all the state through as parameters; there's a lot of it.
While we're here, move newprog from a parameter to ctxt5.
We reserve the variable name c for ctxt5,
so a few local variables named c have been renamed.
Instead of lazily initializing deferreturn
and Sym_div and friends, initialize them up front.
Samuel Tan [Wed, 5 Apr 2017 01:26:21 +0000 (18:26 -0700)]
html/template: panic if predefined escapers are found in pipelines during rewriting
Report an error if a predefined escaper (i.e. "html", "urlquery", or "js")
is found in a pipeline that will be rewritten by the contextual auto-escaper,
instead of trying to merge the escaper-inserted escaping directives
with these predefined escapers. This merging behavior is a source
of several security and correctness bugs (eee #19336, #19345, #19352,
and #19353.)
This merging logic was originally intended to ease migration of text/template
templates with user-defined escapers to html/template. Now that
migration is no longer an issue, this logic can be safely removed.
NOTE: this is a backward-incompatible change that fixes known security
bugs (see linked issues for more details). It will explicitly break users
that attempt to execute templates with pipelines containing predefined
escapers.
runtime: say where the compiler knows about var writeBarrier
The runtime.writeBarrier variable tries to be helpful by telling you
that the compiler also knows about this variable, which you could
probably guess, but doesn't say how the compiler knows about it. In
fact, the compiler has a complete copy in builtin/runtime.go that
needs to be kept in sync. Say so.
Nick Kubala [Fri, 7 Apr 2017 21:23:49 +0000 (14:23 -0700)]
os/exec: Update Cmd.Run() docs to reflect correct error return types
Change-Id: I3fe92d74ff259abdf5d1fd28cdc822db88aae191
Reviewed-on: https://go-review.googlesource.com/39993 Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is a re-roll of CL 39710,
which broke deterministic builds.
typenamesym is called from three places:
typename, ngotype, and Type.Symbol.
Only in typename do we actually need a Node.
ngotype and Type.Symbol require only a Sym.
And writing the newly created Node to
Sym.Def is unsafe in a concurrent backend.
Rather than use a mutex protect to Sym.Def,
make typenamesym not touch Sym.Def.
The assignment to Sym.Def was serving a second purpose,
namely to prevent duplicate entries on signatlist.
Preserve that functionality by switching signatlist to a map.
This in turn requires that we sort signatlist
when exporting it, to preserve reproducibility.
We sort using exactly the same mechanism
that the export code (dtypesym) uses.
Failure to do that led to non-deterministic builds (#19872).
Since we've already calculated the Type's export name,
we could pass it to dtypesym, sparing it a bit of work.
That can be done as a future optimization.
cmd/compile: allow composite literal structs with _ fields
Given code such as
type T struct {
_ string
}
func f() {
var x = T{"space"}
// ...
}
the compiler rewrote the 'var x' line as
var x T
x._ = "space"
The compiler then rejected the assignment to
a blank field, thus rejecting valid code.
It also failed to catch a number of invalid assignments.
And there were insufficient checks for validity
when emitting static data, leading to ICEs.
To fix, check earlier for explicit blanks field names,
explicitly handle legit blanks in sinit,
and don't try to emit static data for nodes
for which typechecking has failed.
This dowidth currently happens during AST to SSA conversion.
As such, it is a concurrency pinch point.
It's a bit silly, but do it here in walk instead.
This appears (fingers crossed) to be the last
unresolved dowidth concurrency problem.
Dave Cheney [Fri, 7 Apr 2017 10:07:54 +0000 (20:07 +1000)]
cmd/compile/internal/gc: remove queuemethod
queuemethod was unused. As queuemethod is unused, nothing appends to the
methodqueue global. As methodqueue is always nil or empty, there are no
live callers of domethod, so it can be removed.
Change-Id: Ic7427ac4621bbf403947815e3988c3a1113487f2
Reviewed-on: https://go-review.googlesource.com/39931
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Matthew Dempsky [Fri, 7 Apr 2017 20:47:10 +0000 (13:47 -0700)]
cmd/compile/internal/gc: cleanup mkinlcall
I had too many failed attempts trying to remove iterFields that I
decided to overhaul this function. Much simpler and easier to
understand now (at least IMO).
Passes toolstash-check -all.
Change-Id: I41d00642a969698df3f4689e41a386346b966638
Reviewed-on: https://go-review.googlesource.com/39856
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Hiroshi Ioka [Thu, 19 Jan 2017 23:34:18 +0000 (08:34 +0900)]
cmd/cgo: support floating point #define macros
Current code doesn't support floating point #define macros.
This CL compiles floats to a object file and retrive values from it.
That approach is the same work as we've already done for integers.
Updates #18720
Change-Id: I88b7ab174d0f73bda975cf90c5aeb797961fe034
Reviewed-on: https://go-review.googlesource.com/35511 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
There are some LSyms that are lazily initialized,
and which cannot be made eagerly initialized,
such as elements of a constant pool.
To avoid needing a mutex to protect the internals of
those LSyms, this CL introduces LookupInit,
which allows an LSym to be initialized only once.
By itself this is not fully concurrency-safe,
but Ctxt.Hash will need mutex protection anyway,
and that will be enough to support one-time LSym initialization.
Alessandro Arzilli [Tue, 7 Mar 2017 17:59:14 +0000 (18:59 +0100)]
cmd/compile: output DWARF lexical blocks for local variables
Change compiler and linker to emit DWARF lexical blocks in debug_info.
Version of debug_info is updated from DWARF v.2 to DWARF v.3 since version 2
does not allow lexical blocks with discontinuous ranges.
Second attempt at https://go-review.googlesource.com/#/c/29591/
Remaining open problems:
- scope information is removed from inlined functions
- variables in debug_info do not have DW_AT_start_scope attributes so a
variable will shadow other variables with the same name as soon as its
containing scope begins, before its declaration.
Updates golang/go#12899, golang/go#6913
Change-Id: I0e260a45b564d14a87b88974eb16c5387cb410a5
Reviewed-on: https://go-review.googlesource.com/36879
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Ilya Tocar [Tue, 28 Mar 2017 20:30:31 +0000 (15:30 -0500)]
cmd/compile/internal/gc: improve comparison with constant strings
Currently we expand comparison with small constant strings into len check
and a sequence of byte comparisons. Generate 16/32/64-bit comparisons,
instead of bytewise on 386 and amd64. Also increase limits on what is
considered small constant string.
Shaves ~30kb (0.5%) from go executable.
This also updates test/prove.go to keep test case valid.
Robert Griesemer [Wed, 5 Apr 2017 00:54:02 +0000 (17:54 -0700)]
cmd/compile: factor out Pkg, Sym, and Type into package types
- created new package cmd/compile/internal/types
- moved Pkg, Sym, Type to new package
- to break cycles, for now we need the (ugly) types/utils.go
file which contains a handful of functions that must be installed
early by the gc frontend
- to break cycles, for now we need two functions to convert between
*gc.Node and *types.Node (the latter is a dummy type)
- adjusted the gc's code to use the new package and the conversion
functions as needed
- made several Pkg, Sym, and Type methods functions as needed
- renamed constructors typ, typPtr, typArray, etc. to types.New,
types.NewPtr, types.NewArray, etc.
Passes toolstash-check -all.
Change-Id: I8adfa5e85c731645d0a7fd2030375ed6ebf54b72
Reviewed-on: https://go-review.googlesource.com/39855 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
typenamesym is called from three places:
typename, ngotype, and Type.Symbol.
Only in typename do we actually need a Node.
ngotype and Type.Symbol require only a Sym.
And writing the newly created Node to
Sym.Def is unsafe in a concurrent backend.
Rather than use a mutex protect to Sym.Def,
make typenamesym not touch Sym.Def.
The assignment to Sym.Def was serving a second purpose,
namely to prevent duplicate entries on signatlist.
Preserve that functionality by switching signatlist to a map.
This in turn requires that we sort signatlist
when exporting it, to preserve reproducibility.
We'd like to use Type.cmp for sorting,
but that causes infinite recursion at the moment;
see #19869.
For now, use Type.LongString as the sort key,
which is a complete description of the type.
Type.LongString is relatively expensive,
but we calculate it only once per type,
and signatlist is generally fairly small,
so the performance impact is minimal.
When a constant is both MOVCON (can fit into a MOV instruction)
and BITCON (can fit into a logical instruction), the assembler
chooses to use the MOVCON encoding, which is actually longer for
logical instructions. We add MBCON rules explicitly to make sure
it uses the BITCON encoding.
cmd/compile: use ANDconst to mask out leading/trailing bits on ARM64
For an AND that masks out leading or trailing bits, generic rules
rewrite it to a pair of shifts. On ARM64, the mask actually can
fit into an AND instruction. So we rewrite it back to AND.
Jeff Wendling [Thu, 6 Apr 2017 15:45:30 +0000 (15:45 +0000)]
cmd/dist: disable plugin test on linux-arm with GOARM=5
Plugin support is patchy at the moment, so disable the test for
now until the test can be fixed. This way, we can get builders
for ARMv5 running for the rest of the code.
The existing bulk/cached Prog allocator, Ctxt.NewProg, is not concurrency-safe.
This CL moves Prog allocation to its clients, the compiler and the assembler.
The assembler is so fast and generates so few Progs that it does not need
optimization of Prog allocation. I could not generate measureable changes.
And even if I could, the assembly is a miniscule portion of build times.
The compiler already has a natural place to manage Prog allocation;
this CL migrates the Prog cache there.
It will be made concurrency-safe in a later CL by
partitioning the Prog cache into chunks and assigning each chunk
to a different goroutine to manage.
This CL does cause a performance degradation when the compiler
is invoked with the -S flag (to dump assembly).
However, such usage is rare and almost always done manually.
The one instance I know of in a test is TestAssembly
in cmd/compile/internal/gc, and I did not detect
a measurable performance impact there.
Passes toolstash-check -all.
Minor compiler performance impact.
cmd/compile: teach assemblers to accept a Prog allocator
The existing bulk Prog allocator is not concurrency-safe.
To allow for concurrency-safe bulk allocation of Progs,
I want to move Prog allocation and caching upstream,
to the clients of cmd/internal/obj.
This is a preliminary enabling refactoring.
After this CL, instead of calling Ctxt.NewProg
throughout the assemblers, we thread through
a newprog function that returns a new Prog.
That function is set up to be Ctxt.NewProg,
so there are no real changes in this CL;
this CL only establishes the plumbing.
Austin Clements [Thu, 16 Mar 2017 18:55:10 +0000 (14:55 -0400)]
runtime: initialize more fields of stack spans
Stack spans don't internally use many of the fields of the mspan,
which means things like the size class and element size get left over
from whatever last used the mspan. This can lead to confusing crashes
and debugging.
Zero these fields or initialize them to something reasonable. This
also lets us simplify some code that currently has to distinguish
between heap and stack spans.
Commit 44ed88a5a7 moved printing of the "sweep done" gcpacertrace
message so that it is printed when the final sweeper finishes.
However, by this point some other thread has often already observed
that there are no more spans to sweep and zeroed sweepPagesPerByte.
Avoid printing a 0 sweep ratio in the trace when this race happens by
getting the value of the sweep ratio upon entry to sweepone and
printing that.
The comments in this package state that users should be
migrating code that uses the syscall package to its
corresponding package in x/sys. However, the syscall.Signal
and syscall.Errno types and the syscall.SysProcAttr struct is
not defined in the x/sys package and still need to be referenced
from within syscall. This adds a change to the comments to
clarify that the migration will need to continue to use some
references to syscall for now.
Fixes #19560
Change-Id: I8abb96b93bea90070ce461da16dc7bcf7b4b29c1
Reviewed-on: https://go-review.googlesource.com/39450 Reviewed-by: Rob Pike <r@golang.org>
Alex Brainman [Mon, 13 Feb 2017 03:52:19 +0000 (14:52 +1100)]
path/filepath: add test to walk symlink
For #17540.
Change-Id: Ie01f39797526934fa553f4279cbde6c7cbf14154
Reviewed-on: https://go-review.googlesource.com/36854 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is a holdover from the days when we did not
have full SSA coverage and compiled things optimistically,
and catching the panic obscures useful information.
Filip Gruszczyński [Sat, 18 Mar 2017 03:10:38 +0000 (20:10 -0700)]
reflect: add MakeMapWithSize for creating maps with size hint
Providing size hint when creating a map allows avoiding re-allocating
underlying data structure if we know how many elements are going to
be inserted. This can be used for example during decoding maps in
gob.
Fixes #19599
Change-Id: I108035fec29391215d2261a73eaed1310b46bab1
Reviewed-on: https://go-review.googlesource.com/38335 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Martin Möhrmann [Mon, 6 Mar 2017 08:34:39 +0000 (09:34 +0100)]
strings: speed up Fields
- use a string lookup to detect if a single byte is a space character
- determine the exact number of fields for ASCII and
a possibly underestimated number of fields for non ASCII strings
by doing a separate byte for byte scan of the input string
before collecting the fields in an extra pass
- provide a fast path for ASCII only strings when collecting the fields
- avoid utf8.DecodeRuneInString and unicode.IsSpace for ASCII characters
Used golang.org/cl/33108 from Joe Tsai as starting point.
Keith Randall [Fri, 17 Mar 2017 04:33:03 +0000 (21:33 -0700)]
cmd/compile: intrinsics for math/bits.OnesCount
Popcount instructions on amd64 are not guaranteed to be
present, so we must guard their call. Rewrite rules can't
generate control flow at the moment, so the intrinsifier
needs to generate that code.
When a map is small, it's not worth putting
the contents in an array and then looping over the array.
Just generate code instead.
This makes smaller binaries.
It might also be better for cache lines.
It also can avoids adding control flow in the middle
of the init function, which can be very large.
Eliminating this source of extra blocks
makes phi insertion easier for temp-heavy init functions.
This reduces the time required for compiler to
panic while compiling the code in #19751
from 15 minutes to 45 seconds.
The cutoff of 25 was chosen fairly unscientifically
by looking at the size of cmd/go.
There are probably more sophisticated mechanisms available.
For example, the smaller the key/value sizes, the better
generated code will be vs a table.
Nevertheless this is simple and seems like a good start.
Subsequent rules can then assume if there is a constant arg to Eq64,
it will be the first one. This fix kinda works, but it is fragile and
only works when we remember to include the required extra rules.
The fundamental problem is that the rule matcher doesn't
know anything about commuting ops. This CL fixes that fact.
We already have information about which ops commute. (The register
allocator takes advantage of commutivity.) The rule generator now
automatically generates multiple rules for a single source rule when
there are commutative ops in the rule. We can now drop all of our
almost-duplicate source-level rules and the canonicalization rules.
I have some CLs in progress that will be a lot less verbose when
the rule generator handles commutivity for me.
I had to reorganize the load-combining rules a bit. The 8-way OR rules
generated 128 different reorderings, which was causing the generator
to put too much code in the rewrite*.go files (the big ones were going
from 25K lines to 132K lines). Instead I reorganized the rules to
combine pairs of loads at a time. The generated rule files are now
actually a bit (5%) smaller.
Make.bash times are ~unchanged.
Compiler benchmarks are not observably different. Probably because
we don't spend much compiler time in rule matching anyway.
I've also done a pass over all of our ops adding commutative markings
for ops which hadn't had them previously.
Fixes #18292
Change-Id: Ic1c0e43fbf579539f459971625f69690c9ab8805
Reviewed-on: https://go-review.googlesource.com/38801
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>