Cleanup CL https://golang.org/cl/41691 broke the plan9 build by removing
a use of a package but not removing the package import.
Trybots don't check that. I filed #20119 for that.
Change-Id: Ia030e6924665dfb871ca964455b899d51b0200c2
Reviewed-on: https://go-review.googlesource.com/41752 Reviewed-by: David du Colombier <0intro@gmail.com>
cmd/go, cmd/dist: temporarily disable race and PIE internal link tests on Alpine
In an effort to at least understand the complete set of things not
working on Alpine Linux, I've been trying to get the build passing
again, even with tests disabled.
The race detector is broken on Alpine. That is #14481 (and #9918).
So disable those tests for now.
Also, internal linking with PIE doesn't work on Alpine yet.
That is #18243. So disable that test for now.
With this CL, all.bash almost passes. There's some cgo test failing
still, but there's no bug yet, so that can be a separate CL.
Change-Id: I3ffbb0e787ed54cb82f298b6bd5bf3ccfbc82622
Reviewed-on: https://go-review.googlesource.com/41678
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/compile: improve efficiency of binary export position encoding
Use -64 instead of 0 as the magic "new file"
line delta, since it is much less common.
Use a new path encoding that breaks up paths
into /-separated components, allowing
reuse of the component strings, and making
many re-used paths a single byte to encode.
isifacemethod accessed thisT without checking if it was initialized,
opening the possibility for a bug during type checking. Give better
name, move it to package types, and provide accessor instead.
Michael Munday [Tue, 11 Apr 2017 18:30:18 +0000 (14:30 -0400)]
cmd: fix the order that s390x operands are printed in
The assembler reordered the operands of some instructions to put the
first operand into From3. Unfortunately this meant that when the
instructions were printed the operands were in a different order than
the assembler would expect as input. For example, 'MVC $8, (R1), (R2)'
would be printed as 'MVC (R1), $8, (R2)'.
Originally this was done to ensure that From contained the source
memory operand. The current compiler no longer requires this and so
this CL simply makes all instructions use the standard order for
operands: From, Reg, From3 and finally To.
Ian Lance Taylor [Tue, 25 Apr 2017 04:49:26 +0000 (21:49 -0700)]
os: fix race between file I/O and Close
Now that the os package uses internal/poll on Unix and Windows systems,
it can rely on internal/poll reference counting to ensure that the
file descriptor is not closed until all I/O is complete.
That was already working. This CL completes the job by not trying to
modify the Sysfd field when it might still be used by the I/O routines.
Ben Shi [Thu, 20 Apr 2017 09:08:52 +0000 (09:08 +0000)]
cmd/internal/obj/arm: use new form of MOVW introduced in ARMv7
As discussion in issue #18293, "MOVW $Imm-16, Reg" was introduced in
ARMv7. It directly encoded the 16-bit immediate into the instruction
instead of put it in the constant pool.
This patch makes the arm assembler choose this form of MOVW if available.
Besides 4 bytes are saved in the constant pool, the go1 benchmark test
also shows a slight improvement.
runtime: ignore TestCgoPprofPIE test failures on Alpine
Updates #19938
Updates #18243
Change-Id: Ib6e704c0a5d596bdfaa6493902d2528bec55bf16
Reviewed-on: https://go-review.googlesource.com/41628 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Implemented low-level time system for windows on hardware (software),
which does not support memory mapped _KSYSTEM_TIME page update.
In particular this problem exists on Wine where _KSYSTEM_TIME
only contains time at the start, and is never modified.
On start we try to detect Wine and if it's so we fallback to
GetSystemTimeAsFileTime() for current time and a monotonic
timer based on QueryPerformanceCounter family of syscalls:
https://msdn.microsoft.com/en-us/library/windows/desktop/dn553408(v=vs.85).aspx
Fixes #18537
Change-Id: I269d22467ed9b0afb62056974d23e731b80c83ed
Reviewed-on: https://go-review.googlesource.com/35710 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/go: fix TestCgoConsistentResults when using clang instead of gcc
As Ian said at:
https://github.com/golang/go/issues/19964#issuecomment-296347750
> the -fdebug-prefix-map option is being applied to the debug info but
> not to the initial .file pseudo-op.
>
> My only current thought for how to fix this is that instead of
> compiling $WORK/a/b/foo.c, we should change the command to (cd
> $WORK/a/b && clang -g -c foo.c). We'll still want
> -fdebug-prefix-map, I think, but that should fix the .file
> pseudo-op.
This CL does that.
Fixes #19964
Change-Id: I442b1201cab9e0448fc520ab243ad364d59cd7c3
Reviewed-on: https://go-review.googlesource.com/41629
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ronald G. Minnich [Mon, 24 Apr 2017 23:09:24 +0000 (16:09 -0700)]
syscall: fix ordering of Unshare and chroot on Linux
When unshare specifies a new namespace, the syscall
package changes / to make namespace changes private.
If a chroot is specified, the unshare must be done first.
If the chroot is done first then the unshare will
not specify the correct /.
A new test is included which test combining chroot
and CLONE_NEWNS; it fails without the patch and works with
it.
Fixes #20103
Change-Id: I86022803c784bd418a30383321f3d64103d95c62
Reviewed-on: https://go-review.googlesource.com/41626 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Josselin Costanzi [Sat, 7 Jan 2017 13:12:57 +0000 (14:12 +0100)]
encoding/base64: Optimize DecodeString
Optimize DecodeString for the common case where most of the input isn't
a newline or a padding character.
Also add some testcases found when fuzzing this implementation against
upstream.
Change Decode benchmark to run with different input sizes.
Andrew Benton [Tue, 7 Mar 2017 22:00:54 +0000 (14:00 -0800)]
encoding/asn1: add NullBytes and NullRawValue for working with ASN.1 NULL
There were a number of places in crypto/x509 that used hardcoded
representations of the ASN.1 NULL type, in both byte slice and
RawValue struct forms. This change adds two new exported vars to
the asn1 package for working with ASN.1 NULL in both its forms, and
converts all usages from the x509 package.
In addition, tests were added to exercise Marshal and Unmarshal on
both vars.
See #19446 for discussion.
Change-Id: I63dbd0835841ccbc810bd6ec794360a84e933f1e
Reviewed-on: https://go-review.googlesource.com/38660
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Adam Langley <agl@golang.org>
We can also remove the internal/load import as a result.
Found with honnef.co/go/tools/cmd/unused.
Change-Id: Ie70c5713e7a6f238158804acec07807c14f8e092
Reviewed-on: https://go-review.googlesource.com/41473 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Keith Randall [Tue, 21 Mar 2017 21:51:38 +0000 (14:51 -0700)]
cmd/compile: rotate loops so conditional branch is at the end
Old loops look like this:
loop:
CMPQ ...
JGE exit
...
JMP loop
exit:
New loops look like this:
JMP entry
loop:
...
entry:
CMPQ ...
JLT loop
This removes one instruction (the unconditional jump) from
the inner loop.
Kinda surprisingly, it matters.
This is a bit different than the peeling that the old obj
library did in that we don't duplicate the loop exit test.
We just jump to the test. I'm not sure if it is better or
worse to do that (peeling gets rid of the JMP but means more
code duplication), but this CL is certainly a much simpler
compiler change, so I'll try this way first.
The obj library used to do peeling before
CL https://go-review.googlesource.com/c/36205 turned it off.
Fixes #15837 (remove obj instruction reordering)
The reordering is already removed, this CL implements the only
part of that reordering that we'd like to keep.
Fixes #14758 (append loop)
name old time/op new time/op delta
Foo-12 817ns ± 4% 538ns ± 0% -34.08% (p=0.000 n=10+9)
Bar-12 850ns ±11% 570ns ±13% -32.88% (p=0.000 n=10+10)
Update #19595 (BLAS slowdown)
name old time/op new time/op delta
DgemvMedMedNoTransIncN-12 13.2µs ± 9% 10.2µs ± 1% -22.26% (p=0.000 n=9+9)
Fixes #19633 (append loop)
name old time/op new time/op delta
Foo-12 810ns ± 1% 540ns ± 0% -33.30% (p=0.000 n=8+9)
Update #18977 (Fannkuch11 regression)
name old time/op new time/op delta
Fannkuch11-8 2.80s ± 0% 3.01s ± 0% +7.47% (p=0.000 n=9+10)
This one makes no sense. There's strictly 1 less instruction in the
inner loop (17 instead of 18). They are exactly the same instructions
except for the JMP that has been elided.
Daniel Theophanes [Thu, 13 Apr 2017 23:04:40 +0000 (16:04 -0700)]
database/sql: allow using a single connection from the database
Databases have the following concepts: Statement, Batch, and Session.
A statement is often a single line like:
SELECT Amount from Account where ID = 50;
A batch is one or more statements submitted together for the query
to process. It may be a DELETE, INSERT, two UPDATES and a SELECT in
a single query text.
A session is usually represented by a single database connection.
This often is an issue when dealing with scopes in databases.
Temporary tables and variables can have batch, session, or global
scope depending on the syntax, database, and use.
Furthermore, some databases (sybase and derivatives in perticular)
that prevent certain statements from being in the same batch
and may necessitate being in the same session.
By allowing users to extract a Conn from the database they can manage
session on their own without hacking around it by making connection
pools of single connections (a real workaround presented in issue).
It is tempting to just use a transaction, but this isn't always
desirable or an option if running an interactive session or
alter script set that itself starts transactions.
math: remove asm version of sincos everywhere, except 386
We have dedicated asm implementation of sincos only on 386 and amd64,
on everything else we are just jumping to generic version.
However amd64 version is actually slower than generic one:
Alberto Donizetti [Sat, 22 Apr 2017 13:28:58 +0000 (15:28 +0200)]
cmd/compile: more error position tests for the typechecker
This change adds line position tests for several yyerror calls in the
typechecker that are currently not tested in any way.
Untested yyerror calls were found by replacing them with
yerrorl(src.NoXPos, ...)
(thus destroying position information in the error), and then running
the test suite. No failures means no test coverage for the relevant
yyerror call.
For #19683
Change-Id: Iedb3d2f02141b332e9bfa76dbf5ae930ad2fddc3
Reviewed-on: https://go-review.googlesource.com/41477
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Current implementation use EPIPE as an error for Seek on pipes.
According to http://pubs.opengroup.org/onlinepubs/009695399/functions/lseek.html,
it should use ESPIPE instead.
Fixes #20066
Change-Id: I24c3b95be946bc19a287d6b10f447b034a9a1283
Reviewed-on: https://go-review.googlesource.com/41311 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I9d399db8ac26ad44adeace3bf1e5b11cbfe3e0d3
Reviewed-on: https://go-review.googlesource.com/41313 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/go/internal/work: more TestRespectSetgidDir fixes
Hopefully the last refactoring of TestRespectGroupSticky:
* Properly tested (+simplified) FreeBSD fix
* Tested on Darwin (10.12.4)
* Rename to TestRespectSetgidDir (I believe this is the accepted
terminology)
Fixes golang/go#19596.
Change-Id: I8d689ac3e245846cb3f1338ea13e35be512ccb9c
Reviewed-on: https://go-review.googlesource.com/41430 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
- Add new BranchStmt.Target field: It's the destination for break,
continue, or goto statements.
- When parsing with CheckBranches enabled, set the BranchStmt.Target
field. We get the information practically for free from the branch
checker, so keep it for further use.
- Fix a couple of comments.
- This could use a test, but the new Target field is currently not
used, and writing a test is tedious w/o a general tree visitor.
Do it later. For now, visually verified output from syntax dump.
Change-Id: Id691d89efab514ad885e19ac9759506106579520
Reviewed-on: https://go-review.googlesource.com/40988 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
ARM's udiv function is nosplit and it shouldn't be preemptied
(passing args in registers). It is in some sense like DUFFCOPY,
which we don't mark as safepoint.
cmd/compile: provide a way to auto-discover -d debug keys
Currently one needs to refer to the sources to have a list of accepted
debug keys. We can copy what 'ssa/help' does and introspect the list of
debug keys to print a more detailed help:
$ go tool compile -d help
usage: -d arg[,arg]* and arg is <key>[=<value>]
<key> is one of:
append print information about append compilation
closure print information about closure compilation
disablenil disable nil checks
dclstack run internal dclstack check
gcprog print dump of GC programs
nil print information about nil checks
panic do not hide any compiler panic
slice print information about slice compilation
typeassert print information about type assertion inlining
wb print information about write barriers
export print export data
pctab print named pc-value table
ssa/help print help about SSA debugging
Keith Randall [Thu, 9 Jun 2016 05:02:08 +0000 (22:02 -0700)]
cmd/compile: experiment which clobbers all dead pointer fields
The experiment "clobberdead" clobbers all pointer fields that the
compiler thinks are dead, just before and after every safepoint.
Useful for debugging the generation of live pointer bitmaps.
Helped find the following issues:
Update #15936
Update #16026
Update #16095
Update #18860
Change-Id: Id1d12f86845e3d93bae903d968b1eac61fc461f9
Reviewed-on: https://go-review.googlesource.com/23924
Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
runtime/debug: increase threshold on TestSetGCPercent
Currently TestSetGCPercent checks that NextGC is within 10 MB of the
expected value. For some reason it's much noisier on some of the
builders. To get these passing again, raise the threshold to 20 MB.
Robert Griesemer [Fri, 21 Apr 2017 18:11:15 +0000 (11:11 -0700)]
cmd/compile/internal/types: remove unused lineno arguments for PushDcl/MarkDcl
More steps towards simpler symbol handling:
- Pushdcl's incoming pos argument, saved in a newly pushed *Sym, was always
immediately overwritten by the Lastlineno value of the saved *Sym.
- Markdcl's incoming pos argument, saved in the stack mark *Sym, was not
restored when the stack mark was popped.
- Popdcl always maintained the most recent Lastlineno for a *Sym given
by package and name, making it unnecessary to save Lastlineno in the
first place. Removed Lastlineno from the set of fields that need saving,
and simplified Popdcl.
Change-Id: Ie93da1fbd780dcafc2703044e781c0c6298df569
Reviewed-on: https://go-review.googlesource.com/41390
Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Currently SetGCPercent forces a GC in order to recompute GC pacing.
Since we can now recompute pacing on the fly using gcSetTriggerRatio,
change SetGCPercent (really runtime.setGCPercent) to go through
gcSetTriggerRatio and not trigger a GC.
This changes gcSetTriggerRatio so it can be called even during
concurrent mark or sweep. In this case, it will adjust the pacing of
the current phase, accounting for progress that has already been made.
To make this work for concurrent sweep, this introduces a "basis" for
the pagesSwept count, much like the basis we just introduced for
heap_live. This lets gcSetTriggerRatio shift the basis to the current
heap_live and pagesSwept and compute a slope from there to completion.
This avoids creating a discontinuity where, if the ratio has
increased, there has to be a flurry of sweep activity to catch up.
Instead, this creates a continuous, piece-wise linear function as
adjustments are made.
runtime: drive proportional sweep directly off heap_live
Currently, proportional sweep maintains its own count of how many
bytes have been allocated since the beginning of the sweep cycle so it
can compute how many pages need to be swept for a given allocation.
However, this requires a somewhat complex reimbursement scheme since
proportional sweep must be done before a span is allocated, but we
don't know how many bytes to charge until we've allocated a span. This
means that the allocated byte count used by proportional sweep can go
up and down, which has led to underflow bugs in the past (#18043) and
is going to interfere with adjusting sweep pacing on-the-fly (for #19076).
This approach also means we're maintaining a statistic that is very
closely related to heap_live, but has a different 0 value. This is
particularly confusing because the sweep ratio is computed based on
heap_live, so you have to understand that these two statistics are
very closely related.
Replace all of this and compute the sweep debt directly from the
current value of heap_live. To make this work, we simply save the
value of heap_live when the sweep ratio is computed to use as a
"basis" for later computing the sweep debt.
This eliminates the need for reimbursement as well as the code for
maintaining the sweeper's version of the live heap size.
For #19076.
Coincidentally fixes #18043, since this eliminates sweep reimbursement
entirely.
runtime: consolidate all trigger-derived computations
Currently, the computations that derive controls from the GC trigger
are spread across several parts of the mark termination code.
Consolidate computing the absolute trigger, the heap goal, and sweep
pacing into a single function called at the end of mark termination.
Unlike the code being consolidated, this has to be more careful about
negative gcpercent. Many of the consolidated code paths simply didn't
execute if GC was off.
This is a step toward being able to change the GC trigger ratio in the
middle of concurrent sweeping and marking. For this commit, we try to
stick close to the original structure of the code that's being
consolidated, so it doesn't yet support mid-cycle adjustments.
Austin Clements [Fri, 31 Mar 2017 21:09:41 +0000 (17:09 -0400)]
runtime: rationalize triggerRatio
gcController.triggerRatio is the only field in gcController that
persists across cycles. As global mutable state, the places where it
written and read are spread out, making it difficult to see that
updates and downstream calculations are done correctly.
Improve this situation by doing two things:
1) Move triggerRatio to memstats so it lives with the other
trigger-related fields and makes gcController entirely transient
state.
2) Commit the new trigger ratio during mark termination when we
compute other next-cycle controls, including the absolute trigger.
This forces us to explicitly thread the new trigger ratio from
gcController.endCycle to mark termination, so we're not just pulling
it out of global state.
runtime: consistently use atomic loads for heap_live
heap_live is updated atomically without locking, so we should also use
atomic loads to read it. Fix the reads of heap_live that happen
outside of STW to be atomic.
Change-Id: I0f8c695146b39cff72ca2374f861f3e9f72b0f77
Reviewed-on: https://go-review.googlesource.com/41314 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The inliner's ishairy passes a budget and a reason down through the
walk. Lift these into a visitor object and turn ishairy and its
helpers into methods.
David Lazar [Thu, 20 Apr 2017 20:19:55 +0000 (16:19 -0400)]
testing: use function names to identify helpers
Previously, helpers were identified by entry PC, but this breaks if the
helper is inlined (as in notHelperCallingHelper). Instead, identify
helpers by function name (with package path). Now TestTBHelper and
TestTBHelperParallel pass with -l=4.
To keep the code unified, this change makes it so that the runner
is also identified by function name instead of entry PC.
David Lazar [Wed, 19 Apr 2017 16:57:52 +0000 (12:57 -0400)]
cmd/compile: don't inline functions that call runtime.getcaller{pc,sp}
runtime.getcaller{pc,sp} expect their argument to be a pointer to the
caller's first function argument. This assumption breaks when the caller
is inlined. For example, with -l=4, calls to runtime.entersyscall (which
calls getcallerpc) are inlined and that breaks multiple cgo tests.
This change modifies the compiler to refuse to inline functions that
call runtime.getcaller{pc,sp}. Alternatively, we could mark these
functions //go:noinline but that limits optimization opportunities if
the calls to getcaller{pc,sp} are eliminated as dead code.
Previously TestCgoPprofPIE, TestCgoPprof, and TestCgoCallbackGC failed
with -l=4. Now all of the runtime tests pass with -l=4.
Change-Id: I258bca9025e20fc451e673a18f862b5da1e07ae7
Reviewed-on: https://go-review.googlesource.com/40998
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Austin Clements <austin@google.com>
On 32-bit architectures (or if we fail to map a 64-bit-style arena),
we try to map the heap arena just above the end of the process image.
While we can accept any address, using lower addresses is preferable
because lower addresses cause us to map less of the heap bitmap.
However, if a program is linked against C code that has global
constructors, those constructors may call brk/sbrk to allocate memory
(e.g., many C malloc implementations do this for small allocations).
The brk also starts just above the process image, so this may adjust
the brk past the beginning of where we want to put the heap arena. In
this case, the kernel will pick a different address for the arena and
it will usually be very high (at least, as these things go in a 32-bit
address space).
Fix this by consulting the current value of the brk and using this in
addition to the end of the process image to compute the initial arena
placement.
This is implemented only on Linux currently, since we have no evidence
that it's an issue on any other OSes.
Fixes #19831.
Change-Id: Id64b45d08d8c91e4f50d92d0339146250b04f2f8
Reviewed-on: https://go-review.googlesource.com/39810
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/go/internal/work: fix TestRespectGroupSticky on FreeBSD
FreeBSD doesn't allow non-root users to enable the SetGID bit on
files or directories in /tmp, however it does allow this in
subdirectories, so create the test directory one level deeper.
Followup to golang/go#19596.
Change-Id: I30e71c6d6a156badc863e8068df10ef6ed817e26
Reviewed-on: https://go-review.googlesource.com/41216 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Keith Randall [Wed, 19 Apr 2017 18:19:53 +0000 (11:19 -0700)]
cmd/compile: zero ambiguously live variables at VARKILLs
At VARKILLs, zero a variable if it is ambiguously live.
After the VARKILL anything this variable references
might be collected. If it were to become live again later,
the GC will see references to already-collected objects.
We don't know a variable is ambiguously live until very
late in compilation (after lowering, register allocation, ...),
so it is hard to generate the code in an arch-independent way.
We also have to be careful not to clobber any registers.
Fortunately, this almost never happens so performance is ~irrelevant.
There are only 2 instances where this triggers in the stdlib.
Fixes #20029
Change-Id: Ia9585a91d7b823fad4a9d141d954464cc7af31f4
Reviewed-on: https://go-review.googlesource.com/41076
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>