Implemented low-level time system for windows on hardware (software),
which does not support memory mapped _KSYSTEM_TIME page update.
In particular this problem exists on Wine where _KSYSTEM_TIME
only contains time at the start, and is never modified.
On start we try to detect Wine and if it's so we fallback to
GetSystemTimeAsFileTime() for current time and a monotonic
timer based on QueryPerformanceCounter family of syscalls:
https://msdn.microsoft.com/en-us/library/windows/desktop/dn553408(v=vs.85).aspx
Fixes #18537
Change-Id: I269d22467ed9b0afb62056974d23e731b80c83ed
Reviewed-on: https://go-review.googlesource.com/35710 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/go: fix TestCgoConsistentResults when using clang instead of gcc
As Ian said at:
https://github.com/golang/go/issues/19964#issuecomment-296347750
> the -fdebug-prefix-map option is being applied to the debug info but
> not to the initial .file pseudo-op.
>
> My only current thought for how to fix this is that instead of
> compiling $WORK/a/b/foo.c, we should change the command to (cd
> $WORK/a/b && clang -g -c foo.c). We'll still want
> -fdebug-prefix-map, I think, but that should fix the .file
> pseudo-op.
This CL does that.
Fixes #19964
Change-Id: I442b1201cab9e0448fc520ab243ad364d59cd7c3
Reviewed-on: https://go-review.googlesource.com/41629
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ronald G. Minnich [Mon, 24 Apr 2017 23:09:24 +0000 (16:09 -0700)]
syscall: fix ordering of Unshare and chroot on Linux
When unshare specifies a new namespace, the syscall
package changes / to make namespace changes private.
If a chroot is specified, the unshare must be done first.
If the chroot is done first then the unshare will
not specify the correct /.
A new test is included which test combining chroot
and CLONE_NEWNS; it fails without the patch and works with
it.
Fixes #20103
Change-Id: I86022803c784bd418a30383321f3d64103d95c62
Reviewed-on: https://go-review.googlesource.com/41626 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Josselin Costanzi [Sat, 7 Jan 2017 13:12:57 +0000 (14:12 +0100)]
encoding/base64: Optimize DecodeString
Optimize DecodeString for the common case where most of the input isn't
a newline or a padding character.
Also add some testcases found when fuzzing this implementation against
upstream.
Change Decode benchmark to run with different input sizes.
Andrew Benton [Tue, 7 Mar 2017 22:00:54 +0000 (14:00 -0800)]
encoding/asn1: add NullBytes and NullRawValue for working with ASN.1 NULL
There were a number of places in crypto/x509 that used hardcoded
representations of the ASN.1 NULL type, in both byte slice and
RawValue struct forms. This change adds two new exported vars to
the asn1 package for working with ASN.1 NULL in both its forms, and
converts all usages from the x509 package.
In addition, tests were added to exercise Marshal and Unmarshal on
both vars.
See #19446 for discussion.
Change-Id: I63dbd0835841ccbc810bd6ec794360a84e933f1e
Reviewed-on: https://go-review.googlesource.com/38660
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Adam Langley <agl@golang.org>
We can also remove the internal/load import as a result.
Found with honnef.co/go/tools/cmd/unused.
Change-Id: Ie70c5713e7a6f238158804acec07807c14f8e092
Reviewed-on: https://go-review.googlesource.com/41473 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Keith Randall [Tue, 21 Mar 2017 21:51:38 +0000 (14:51 -0700)]
cmd/compile: rotate loops so conditional branch is at the end
Old loops look like this:
loop:
CMPQ ...
JGE exit
...
JMP loop
exit:
New loops look like this:
JMP entry
loop:
...
entry:
CMPQ ...
JLT loop
This removes one instruction (the unconditional jump) from
the inner loop.
Kinda surprisingly, it matters.
This is a bit different than the peeling that the old obj
library did in that we don't duplicate the loop exit test.
We just jump to the test. I'm not sure if it is better or
worse to do that (peeling gets rid of the JMP but means more
code duplication), but this CL is certainly a much simpler
compiler change, so I'll try this way first.
The obj library used to do peeling before
CL https://go-review.googlesource.com/c/36205 turned it off.
Fixes #15837 (remove obj instruction reordering)
The reordering is already removed, this CL implements the only
part of that reordering that we'd like to keep.
Fixes #14758 (append loop)
name old time/op new time/op delta
Foo-12 817ns ± 4% 538ns ± 0% -34.08% (p=0.000 n=10+9)
Bar-12 850ns ±11% 570ns ±13% -32.88% (p=0.000 n=10+10)
Update #19595 (BLAS slowdown)
name old time/op new time/op delta
DgemvMedMedNoTransIncN-12 13.2µs ± 9% 10.2µs ± 1% -22.26% (p=0.000 n=9+9)
Fixes #19633 (append loop)
name old time/op new time/op delta
Foo-12 810ns ± 1% 540ns ± 0% -33.30% (p=0.000 n=8+9)
Update #18977 (Fannkuch11 regression)
name old time/op new time/op delta
Fannkuch11-8 2.80s ± 0% 3.01s ± 0% +7.47% (p=0.000 n=9+10)
This one makes no sense. There's strictly 1 less instruction in the
inner loop (17 instead of 18). They are exactly the same instructions
except for the JMP that has been elided.
Daniel Theophanes [Thu, 13 Apr 2017 23:04:40 +0000 (16:04 -0700)]
database/sql: allow using a single connection from the database
Databases have the following concepts: Statement, Batch, and Session.
A statement is often a single line like:
SELECT Amount from Account where ID = 50;
A batch is one or more statements submitted together for the query
to process. It may be a DELETE, INSERT, two UPDATES and a SELECT in
a single query text.
A session is usually represented by a single database connection.
This often is an issue when dealing with scopes in databases.
Temporary tables and variables can have batch, session, or global
scope depending on the syntax, database, and use.
Furthermore, some databases (sybase and derivatives in perticular)
that prevent certain statements from being in the same batch
and may necessitate being in the same session.
By allowing users to extract a Conn from the database they can manage
session on their own without hacking around it by making connection
pools of single connections (a real workaround presented in issue).
It is tempting to just use a transaction, but this isn't always
desirable or an option if running an interactive session or
alter script set that itself starts transactions.
math: remove asm version of sincos everywhere, except 386
We have dedicated asm implementation of sincos only on 386 and amd64,
on everything else we are just jumping to generic version.
However amd64 version is actually slower than generic one:
Alberto Donizetti [Sat, 22 Apr 2017 13:28:58 +0000 (15:28 +0200)]
cmd/compile: more error position tests for the typechecker
This change adds line position tests for several yyerror calls in the
typechecker that are currently not tested in any way.
Untested yyerror calls were found by replacing them with
yerrorl(src.NoXPos, ...)
(thus destroying position information in the error), and then running
the test suite. No failures means no test coverage for the relevant
yyerror call.
For #19683
Change-Id: Iedb3d2f02141b332e9bfa76dbf5ae930ad2fddc3
Reviewed-on: https://go-review.googlesource.com/41477
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Current implementation use EPIPE as an error for Seek on pipes.
According to http://pubs.opengroup.org/onlinepubs/009695399/functions/lseek.html,
it should use ESPIPE instead.
Fixes #20066
Change-Id: I24c3b95be946bc19a287d6b10f447b034a9a1283
Reviewed-on: https://go-review.googlesource.com/41311 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I9d399db8ac26ad44adeace3bf1e5b11cbfe3e0d3
Reviewed-on: https://go-review.googlesource.com/41313 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/go/internal/work: more TestRespectSetgidDir fixes
Hopefully the last refactoring of TestRespectGroupSticky:
* Properly tested (+simplified) FreeBSD fix
* Tested on Darwin (10.12.4)
* Rename to TestRespectSetgidDir (I believe this is the accepted
terminology)
Fixes golang/go#19596.
Change-Id: I8d689ac3e245846cb3f1338ea13e35be512ccb9c
Reviewed-on: https://go-review.googlesource.com/41430 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
- Add new BranchStmt.Target field: It's the destination for break,
continue, or goto statements.
- When parsing with CheckBranches enabled, set the BranchStmt.Target
field. We get the information practically for free from the branch
checker, so keep it for further use.
- Fix a couple of comments.
- This could use a test, but the new Target field is currently not
used, and writing a test is tedious w/o a general tree visitor.
Do it later. For now, visually verified output from syntax dump.
Change-Id: Id691d89efab514ad885e19ac9759506106579520
Reviewed-on: https://go-review.googlesource.com/40988 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
ARM's udiv function is nosplit and it shouldn't be preemptied
(passing args in registers). It is in some sense like DUFFCOPY,
which we don't mark as safepoint.
cmd/compile: provide a way to auto-discover -d debug keys
Currently one needs to refer to the sources to have a list of accepted
debug keys. We can copy what 'ssa/help' does and introspect the list of
debug keys to print a more detailed help:
$ go tool compile -d help
usage: -d arg[,arg]* and arg is <key>[=<value>]
<key> is one of:
append print information about append compilation
closure print information about closure compilation
disablenil disable nil checks
dclstack run internal dclstack check
gcprog print dump of GC programs
nil print information about nil checks
panic do not hide any compiler panic
slice print information about slice compilation
typeassert print information about type assertion inlining
wb print information about write barriers
export print export data
pctab print named pc-value table
ssa/help print help about SSA debugging
Keith Randall [Thu, 9 Jun 2016 05:02:08 +0000 (22:02 -0700)]
cmd/compile: experiment which clobbers all dead pointer fields
The experiment "clobberdead" clobbers all pointer fields that the
compiler thinks are dead, just before and after every safepoint.
Useful for debugging the generation of live pointer bitmaps.
Helped find the following issues:
Update #15936
Update #16026
Update #16095
Update #18860
Change-Id: Id1d12f86845e3d93bae903d968b1eac61fc461f9
Reviewed-on: https://go-review.googlesource.com/23924
Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
runtime/debug: increase threshold on TestSetGCPercent
Currently TestSetGCPercent checks that NextGC is within 10 MB of the
expected value. For some reason it's much noisier on some of the
builders. To get these passing again, raise the threshold to 20 MB.
Robert Griesemer [Fri, 21 Apr 2017 18:11:15 +0000 (11:11 -0700)]
cmd/compile/internal/types: remove unused lineno arguments for PushDcl/MarkDcl
More steps towards simpler symbol handling:
- Pushdcl's incoming pos argument, saved in a newly pushed *Sym, was always
immediately overwritten by the Lastlineno value of the saved *Sym.
- Markdcl's incoming pos argument, saved in the stack mark *Sym, was not
restored when the stack mark was popped.
- Popdcl always maintained the most recent Lastlineno for a *Sym given
by package and name, making it unnecessary to save Lastlineno in the
first place. Removed Lastlineno from the set of fields that need saving,
and simplified Popdcl.
Change-Id: Ie93da1fbd780dcafc2703044e781c0c6298df569
Reviewed-on: https://go-review.googlesource.com/41390
Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Currently SetGCPercent forces a GC in order to recompute GC pacing.
Since we can now recompute pacing on the fly using gcSetTriggerRatio,
change SetGCPercent (really runtime.setGCPercent) to go through
gcSetTriggerRatio and not trigger a GC.
This changes gcSetTriggerRatio so it can be called even during
concurrent mark or sweep. In this case, it will adjust the pacing of
the current phase, accounting for progress that has already been made.
To make this work for concurrent sweep, this introduces a "basis" for
the pagesSwept count, much like the basis we just introduced for
heap_live. This lets gcSetTriggerRatio shift the basis to the current
heap_live and pagesSwept and compute a slope from there to completion.
This avoids creating a discontinuity where, if the ratio has
increased, there has to be a flurry of sweep activity to catch up.
Instead, this creates a continuous, piece-wise linear function as
adjustments are made.
runtime: drive proportional sweep directly off heap_live
Currently, proportional sweep maintains its own count of how many
bytes have been allocated since the beginning of the sweep cycle so it
can compute how many pages need to be swept for a given allocation.
However, this requires a somewhat complex reimbursement scheme since
proportional sweep must be done before a span is allocated, but we
don't know how many bytes to charge until we've allocated a span. This
means that the allocated byte count used by proportional sweep can go
up and down, which has led to underflow bugs in the past (#18043) and
is going to interfere with adjusting sweep pacing on-the-fly (for #19076).
This approach also means we're maintaining a statistic that is very
closely related to heap_live, but has a different 0 value. This is
particularly confusing because the sweep ratio is computed based on
heap_live, so you have to understand that these two statistics are
very closely related.
Replace all of this and compute the sweep debt directly from the
current value of heap_live. To make this work, we simply save the
value of heap_live when the sweep ratio is computed to use as a
"basis" for later computing the sweep debt.
This eliminates the need for reimbursement as well as the code for
maintaining the sweeper's version of the live heap size.
For #19076.
Coincidentally fixes #18043, since this eliminates sweep reimbursement
entirely.
runtime: consolidate all trigger-derived computations
Currently, the computations that derive controls from the GC trigger
are spread across several parts of the mark termination code.
Consolidate computing the absolute trigger, the heap goal, and sweep
pacing into a single function called at the end of mark termination.
Unlike the code being consolidated, this has to be more careful about
negative gcpercent. Many of the consolidated code paths simply didn't
execute if GC was off.
This is a step toward being able to change the GC trigger ratio in the
middle of concurrent sweeping and marking. For this commit, we try to
stick close to the original structure of the code that's being
consolidated, so it doesn't yet support mid-cycle adjustments.
Austin Clements [Fri, 31 Mar 2017 21:09:41 +0000 (17:09 -0400)]
runtime: rationalize triggerRatio
gcController.triggerRatio is the only field in gcController that
persists across cycles. As global mutable state, the places where it
written and read are spread out, making it difficult to see that
updates and downstream calculations are done correctly.
Improve this situation by doing two things:
1) Move triggerRatio to memstats so it lives with the other
trigger-related fields and makes gcController entirely transient
state.
2) Commit the new trigger ratio during mark termination when we
compute other next-cycle controls, including the absolute trigger.
This forces us to explicitly thread the new trigger ratio from
gcController.endCycle to mark termination, so we're not just pulling
it out of global state.
runtime: consistently use atomic loads for heap_live
heap_live is updated atomically without locking, so we should also use
atomic loads to read it. Fix the reads of heap_live that happen
outside of STW to be atomic.
Change-Id: I0f8c695146b39cff72ca2374f861f3e9f72b0f77
Reviewed-on: https://go-review.googlesource.com/41314 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The inliner's ishairy passes a budget and a reason down through the
walk. Lift these into a visitor object and turn ishairy and its
helpers into methods.
David Lazar [Thu, 20 Apr 2017 20:19:55 +0000 (16:19 -0400)]
testing: use function names to identify helpers
Previously, helpers were identified by entry PC, but this breaks if the
helper is inlined (as in notHelperCallingHelper). Instead, identify
helpers by function name (with package path). Now TestTBHelper and
TestTBHelperParallel pass with -l=4.
To keep the code unified, this change makes it so that the runner
is also identified by function name instead of entry PC.
David Lazar [Wed, 19 Apr 2017 16:57:52 +0000 (12:57 -0400)]
cmd/compile: don't inline functions that call runtime.getcaller{pc,sp}
runtime.getcaller{pc,sp} expect their argument to be a pointer to the
caller's first function argument. This assumption breaks when the caller
is inlined. For example, with -l=4, calls to runtime.entersyscall (which
calls getcallerpc) are inlined and that breaks multiple cgo tests.
This change modifies the compiler to refuse to inline functions that
call runtime.getcaller{pc,sp}. Alternatively, we could mark these
functions //go:noinline but that limits optimization opportunities if
the calls to getcaller{pc,sp} are eliminated as dead code.
Previously TestCgoPprofPIE, TestCgoPprof, and TestCgoCallbackGC failed
with -l=4. Now all of the runtime tests pass with -l=4.
Change-Id: I258bca9025e20fc451e673a18f862b5da1e07ae7
Reviewed-on: https://go-review.googlesource.com/40998
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Austin Clements <austin@google.com>
On 32-bit architectures (or if we fail to map a 64-bit-style arena),
we try to map the heap arena just above the end of the process image.
While we can accept any address, using lower addresses is preferable
because lower addresses cause us to map less of the heap bitmap.
However, if a program is linked against C code that has global
constructors, those constructors may call brk/sbrk to allocate memory
(e.g., many C malloc implementations do this for small allocations).
The brk also starts just above the process image, so this may adjust
the brk past the beginning of where we want to put the heap arena. In
this case, the kernel will pick a different address for the arena and
it will usually be very high (at least, as these things go in a 32-bit
address space).
Fix this by consulting the current value of the brk and using this in
addition to the end of the process image to compute the initial arena
placement.
This is implemented only on Linux currently, since we have no evidence
that it's an issue on any other OSes.
Fixes #19831.
Change-Id: Id64b45d08d8c91e4f50d92d0339146250b04f2f8
Reviewed-on: https://go-review.googlesource.com/39810
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/go/internal/work: fix TestRespectGroupSticky on FreeBSD
FreeBSD doesn't allow non-root users to enable the SetGID bit on
files or directories in /tmp, however it does allow this in
subdirectories, so create the test directory one level deeper.
Followup to golang/go#19596.
Change-Id: I30e71c6d6a156badc863e8068df10ef6ed817e26
Reviewed-on: https://go-review.googlesource.com/41216 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Keith Randall [Wed, 19 Apr 2017 18:19:53 +0000 (11:19 -0700)]
cmd/compile: zero ambiguously live variables at VARKILLs
At VARKILLs, zero a variable if it is ambiguously live.
After the VARKILL anything this variable references
might be collected. If it were to become live again later,
the GC will see references to already-collected objects.
We don't know a variable is ambiguously live until very
late in compilation (after lowering, register allocation, ...),
so it is hard to generate the code in an arch-independent way.
We also have to be careful not to clobber any registers.
Fortunately, this almost never happens so performance is ~irrelevant.
There are only 2 instances where this triggers in the stdlib.
Fixes #20029
Change-Id: Ia9585a91d7b823fad4a9d141d954464cc7af31f4
Reviewed-on: https://go-review.googlesource.com/41076
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
Make poolLocal size multiple of 128, so it aligns to CPU cache line
on the most common architectures.
This also has the following benefits:
- It may help compiler substituting integer multiplication
by bit shift inside indexLocal.
- It shrinks poolLocal size from 176 bytes to 128 bytes on amd64,
so now it fits two cache lines (or a single cache line on certain
Intel CPUs - see https://software.intel.com/en-us/articles/optimizing-application-performance-on-intel-coret-microarchitecture-using-hardware-implemented-prefetchers).
No measurable performance changes on linux/amd64 and linux/386.
Robert Griesemer [Thu, 20 Apr 2017 19:50:41 +0000 (12:50 -0700)]
go/gcimporter: fix importing of anonymous interfaces
Imported interfaces must be completed, whether they are named or not.
The original code was collecting all types (including anonymous ones)
in the importer's typList. That list was used in the end to complete
interface types. When we introduced tracking of named types only, we
lost anonymous interfaces. Use an independent list of interface types
so the completion code is independent of which types are tracked.
Added test and factored some of the existing tests.
Fixes #20046.
Change-Id: Icd1329032aec33f96890380dd5042de3bef8cdc7
Reviewed-on: https://go-review.googlesource.com/41198 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
David Lazar [Wed, 19 Apr 2017 16:04:18 +0000 (12:04 -0400)]
runtime: make test independent of inlining
TestBreakpoint expects to see "runtime.Breakpoint()" in the stack trace.
If runtime.Breakpoint() is inlined, then the stack trace prints
"runtime.Breakpoint(...)" since the runtime does not have information
about arguments (or lack thereof) to inlined functions. This change
makes the test independent of inlining by looking for the string
"runtime.Breakpoint(". Now TestBreakpoint passes with -l=4.
TestBlockProfile matches samples against a regexp that accepts "," in
profile PCs. I suspect this was just a syntax mistake. Remove "," from
the character class.
TestBlockProfile currently requires exactly five PCs in each sample.
With more aggressive inlining there may be fewer, so change this test
to use the same pattern as TestMutexProfile, which accepts one or more
PCs. With this change, this test passes when compiled with -l=4.
CL 40876 changed ExampleFrames so that the output
was stable with and without mid-stack inlining.
However, that change lost some of the
pedagogical and copy/paste value of the example.
It was unclear why both more and i were being tracked,
and whether the 5 in i < 5 is related to len(pc),
and if so, why and how.
This CL rewrites the example with lots more comments,
and such that the core structure more closely matches
normal usage, and such that it is obvious
which lines of code should be deleted when copying.
As a bonus, it also now illustrates Frame.File.
The period recorded in CPU profiles is in nanoseconds, but was being
computed incorrectly as hz * 1000. As a result, many absolute times
displayed by pprof were incorrect.
Samuel Tan [Thu, 13 Apr 2017 17:57:04 +0000 (10:57 -0700)]
html/template: ignore case when handling type attribute in script element
Convert the parsed attribute name to lowercase before checking its value in
the HTML parser state machine. This ensures that the type attribute in
the script element is handled in a case-sensitive manner, just like all
other attribute names.