Matthew Dempsky [Mon, 23 Feb 2015 08:05:30 +0000 (17:05 +0900)]
runtime: simplify OpenBSD semaphores
OpenBSD's thrsleep system call includes an "abort" parameter, which
specifies a memory address to be tested after being registered on the
sleep channel (i.e., capable of being woken up by thrwakeup). By
passing a pointer to waitsemacount for this parameter, we avoid race
conditions without needing a lock. Instead we just need to use
atomicload, cas, and xadd to mutate the semaphore count.
Change-Id: If9f2ab7cfd682da217f9912783cadea7e72283a8
Reviewed-on: https://go-review.googlesource.com/5563 Reviewed-by: Dmitry Vyukov <dvyukov@google.com> Reviewed-by: Joel Sing <jsing@google.com>
Because mksycall.pl does not sort symbols before generating the output file
the diff includes some unavoidable code moves as Dup2 is processed latter in
the run.
Adam Langley [Wed, 25 Feb 2015 00:06:08 +0000 (16:06 -0800)]
doc/go1.5: mention the change in certificate wildcard handling.
https://go-review.googlesource.com/#/c/5691/ shouldn't affect anyone
because it reflects the RFC and behaviour of modern browsers, but it's
probably worth mentioning.
Change-Id: I8a8eedaac5226b1d8018dc56de6b2d41bd82a642
Reviewed-on: https://go-review.googlesource.com/5862 Reviewed-by: Rob Pike <r@golang.org>
Dave Cheney [Tue, 24 Feb 2015 23:20:13 +0000 (10:20 +1100)]
syscall: split implementation of Pipe/Pipe2 per GOOS
Updates #9974
This proposal moves the definition of Pipe an Pipe2 from the generic
syscall_linux.go to the GOOS specific variants. This is in preparation
for the arm64 port.
For platforms where pipe2(2) is not supported in the minimum 2.6.23 kernel,
amd64 and 386, we retain pipe(2). For all other platforms pipe(2) is removed
and Pipe forwards to pipe2(2).
Because mksycall.pl does not sort symbols before generating the output file
the diff includes some unavoidable code moves as Pipe and Pipe2 are processed
latter in the run.
Rick Hudson [Tue, 24 Feb 2015 22:49:55 +0000 (17:49 -0500)]
runtime: reset gcscanvalid and gcworkdone when GODEBUG=gctrace=2
When GODEBUG=gctrace=2 two gcs are preformed. During the first gc
the stack scan sets the g's gcscanvalid and gcworkdone flags to true
indicating that the stacks have to be scanned and do not need to
be rescanned. These need to be reset to false for the second GC so the
stacks are rescanned, otherwise if the only pointer to an object is
on the stack it will not be discovered and the object will be freed.
Typically this will include the object that was just allocated in
the mallocgc call that initiated the GC.
Robert Griesemer [Tue, 24 Feb 2015 07:51:05 +0000 (23:51 -0800)]
go/ast, go/parser: correct End() position for *ast.EmptyStmt
- added a new field ast.EmptyStmt.Implicit to indicate explicit
or implicit semicolon
- fix ast.EmptyStmt.End() accordingly
- adjusted parser and added test case
Adam Langley [Mon, 23 Feb 2015 22:51:40 +0000 (14:51 -0800)]
crypto/tls: allow larger initial records.
Some servers which misunderstood the point of the CertificateRequest
message send huge reply records. These records are large enough that
they were considered “insane” by the TLS code and rejected.
This change removes the sanity test for record lengths. Although the
maxCiphertext test still remains, just above, which (roughly) enforces
the 16KB protocol limit on record sizes:
https://tools.ietf.org/html/rfc5246#section-6.2.1
Adam Langley [Mon, 23 Feb 2015 22:27:50 +0000 (14:27 -0800)]
crypto/x509: don't reject certs with critical policy extensions.
There was a missing continue that caused certificates with critical
certificate-policy extensions to be rejected. Additionally, that code
structure in general was prone to exactly that bug so I changed it
around to hopefully be more robust in the future.
Fixes #9964.
Change-Id: I58fc6ef3a84c1bd292a35b8b700f44ef312ec1c1
Reviewed-on: https://go-review.googlesource.com/5670 Reviewed-by: Andrew Gerrand <adg@golang.org>
Dmitry Vyukov [Fri, 20 Feb 2015 08:50:56 +0000 (11:50 +0300)]
sync: add active spinning to Mutex
Currently sync.Mutex is fully cooperative. That is, once contention is discovered,
the goroutine calls into scheduler. This is suboptimal as the resource can become
free soon after (especially if critical sections are short). Server software
usually runs at ~~50% CPU utilization, that is, switching to other goroutines
is not necessary profitable.
This change adds limited active spinning to sync.Mutex if:
1. running on a multicore machine and
2. GOMAXPROCS>1 and
3. there is at least one other running P and
4. local runq is empty.
As opposed to runtime mutex we don't do passive spinning,
because there can be work on global runq on on other Ps.
Mikio Hara [Tue, 24 Feb 2015 05:13:47 +0000 (14:13 +0900)]
net: enable TestTCPReadWriteAllocs in short mode
The change 2096 removed unwanted allocations and a few noises in test
using AllocsPerRun. Now it's safe to enable this canary test on netpoll
hotpaths.
Adam Langley [Mon, 23 Feb 2015 21:28:57 +0000 (13:28 -0800)]
crypto/rsa: drop the primality check in crypto/rsa.Validate.
This check is expensive and adversely impacts startup times for some
servers with several, large RSA keys.
It was nice to have, but it's not really going to stop a targetted
attack and was never designed to – hopefully people's private keys
aren't attacker controlled!
Overall I think the feeling is that people would rather have the CPU
time back.
This change deletes the C implementations of
the Go compiler and assembler from the master branch.
The Go implementations are a bit slower right now,
due mainly to garbage generated by taking addresses
of stack variables all over the place (it was C code,
after all). That will be cleaned up (mechanically) over the
next week or so, and things will get faster.
Austin Clements [Mon, 23 Feb 2015 19:33:56 +0000 (14:33 -0500)]
runtime: eliminate unnecessary assumption in heapBitsForObject
The slow path of heapBitsForObjects somewhat subtly assumes that the
pointer will not point to the first word of the object and will round
the pointer wrong if this assumption is violated. This assumption is
safe because the fast path should always take care of this case, but
there's no benefit to making this assumption, it makes the code more
difficult to experiment with than necessary, and it's trivial to
eliminate.
The cleanups move var declarations as close to the use
as possible, splitting disjoint uses of the var into separate
variables. They also remove dead code (especially in
func sudoaddable), which helps with the var moving.
There's more cleanup to come, but this alone cuts the
time spent compiling html/template on my 2013 MacBook Pro
from 3.1 seconds to 2.3 seconds.
Change-Id: I4de499f47b1dd47a560c310bbcde6b08d425cfd6
Reviewed-on: https://go-review.googlesource.com/5637 Reviewed-by: Rob Pike <r@golang.org>
Shenghou Ma [Mon, 23 Feb 2015 08:55:54 +0000 (03:55 -0500)]
runtime/pprof: make TestBlockProfile more robust
It's using debug mode of pprof.writeBlock, so the output actually goes
through text/tabwriter. It is possible that tabwriter expands each tab
into multiple tabs in certain cases.
For example, this output has been observed on the new arm64 port: 10073805 1 @ 0x1088ec 0xd1b8c 0xd0628 0xb68c0 0x867f4
# 0x1088ec sync.(*Cond).Wait+0xfc /home/minux/go.git/src/sync/cond.go:63
# 0xd1b8c runtime/pprof_test.blockCond+0x22c /home/minux/go.git/src/runtime/pprof/pprof_test.go:454
# 0xd0628 runtime/pprof_test.TestBlockProfile+0x1b8 /home/minux/go.git/src/runtime/pprof/pprof_test.go:359
# 0xb68c0 testing.tRunner+0x140 /home/minux/go.git/src/testing/testing.go:447
Russ Cox [Mon, 23 Feb 2015 19:00:51 +0000 (14:00 -0500)]
[dev.cc] test: disable syntax error tests
These don't work with the new compiler, because the
new compiler doesn't have the custom syntax errors
that I built for the old compiler. It will, just not yet.
(Issue #9968.)
Change-Id: I658f7dab2c7f855340a501f9ae4479c097b28cd3
Reviewed-on: https://go-review.googlesource.com/5632 Reviewed-by: Rob Pike <r@golang.org>
Rick Hudson [Thu, 19 Feb 2015 23:11:24 +0000 (18:11 -0500)]
runtime: Add prefetch to allocation code
The routine mallocgc retrieves objects from freelists. Prefetch
the object that will be returned in the next call to mallocgc.
Experiments indicate that this produces a 1% improvement when using
prefetchnta and less when using prefetcht0, prefetcht1, or prefetcht2.
Benchmark numbers indicate a 1% improvement over no
prefetch, much less over prefetcht0, prefetcht1, and prefetcht2.
These numbers were for the garbage benchmark with MAXPROCS=4
no prefetch >> 5.96 / 5.77 / 5.89
prefetcht0(uintptr(v.ptr().next)) >> 5.88 / 6.17 / 5.84
prefetcht1(uintptr(v.ptr().next)) >> 5.88 / 5.89 / 5.91
prefetcht2(uintptr(v.ptr().next)) >> 5.87 / 6.47 / 5.92
prefetchnta(uintptr(v.ptr().next)) >> 5.72 / 5.84 / 5.85
Alexandre Cesaro [Tue, 23 Dec 2014 19:29:13 +0000 (20:29 +0100)]
net/mail: move RFC 2047 code to internal/mime
The code concerning quoted-printable encoding (RFC 2045) and its
variant for MIME headers (RFC 2047) is currently spread in
mime/multipart and net/mail. It is also not exported.
This commit is the second step to fix that issue. It moves the
RFC 2047 encoding and decoding functions from net/mail to
internal/mime. The exported API is unchanged.
Russ Cox [Sun, 22 Feb 2015 17:41:32 +0000 (12:41 -0500)]
[dev.cc] cmd/go: fix expansion of 'std', add 'cmd'
The wildcard 'std' is defined in documentation to be all the packages
in the Go standard library. It has also historically matched commands
in the main repo, but as we implement core commands in Go, that
becomes problematic. We need a wildcard that means just the library,
and since 'std' is already documented to have that definition, make it so.
Add a new wildcard 'cmd' for the commands in the main repo ($GOROOT).
Commands that want both can say 'std cmd' (or 'cmd std') to get the
effect of the old 'std'.
Update make.bash etc to say both std and cmd most of the time.
Exception: in race.bash, do not install race-enabled versions of
the actual commands. This avoids trying to write binaries while
using them, but more importantly it avoids enabling the race
detector and its associated memory overhead for the already
memory-hungry compilers.
Jan Kratochvil [Sat, 21 Feb 2015 16:35:01 +0000 (17:35 +0100)]
gdb: fix "gdb.error: No struct named reflect.rtype."
With a trivial Golang-built program loaded in gdb-7.8.90.20150214-7.fc23.x86_64
I get this error:
(gdb) source ./src/runtime/runtime-gdb.py
Loading Go Runtime support.
Traceback (most recent call last):
File "./src/runtime/runtime-gdb.py", line 230, in <module>
_rctp_type = gdb.lookup_type("struct reflect.rtype").pointer()
gdb.error: No struct type named reflect.rtype.
(gdb) q
No matter if this struct should or should not be in every Golang-built binary
this change should fix that with no disadvantages.
Change-Id: I0c490d3c9bbe93c65a2183b41bfbdc0c0f405bd1
Reviewed-on: https://go-review.googlesource.com/5521 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Mark Bucciarelli [Sun, 22 Feb 2015 02:14:45 +0000 (21:14 -0500)]
Call --> CallSlice in two spots. No logic change, docs only.
Change-Id: I6011e162214db2d65efc1ecdb5ec600ca8e5bfe9
Reviewed-on: https://go-review.googlesource.com/5542 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Rob Pike [Sat, 21 Feb 2015 02:15:20 +0000 (18:15 -0800)]
[dev.cc] cmd/asm: fix build
Representation in printout of MRC instruction differs between
32- and 64-bit machines. It's just a hex dump. Fix this one day,
but for now just comment out the instruction.
Change-Id: I4709390659e2e0f2d18ff6f8e762f97cdbfb4c16
Reviewed-on: https://go-review.googlesource.com/5424 Reviewed-by: Rob Pike <r@golang.org>
Rob Pike [Fri, 20 Feb 2015 18:06:44 +0000 (10:06 -0800)]
[dev.cc] cmd/asm: add end-to-end test
Add trivial golden test that verifies output matches expectation.
The input is based on the old grammar and is intended to cover
the space of the input language.
Rob Pike [Sat, 21 Feb 2015 00:02:11 +0000 (16:02 -0800)]
[dev.cc] cm/asm: fix up arm after cross-check with 5a
As with the previous round for ppc64, this CL fixes a couple of things
that 5a supported but asm did not, both simple.
1) Allow condition code on MRC instruction; this was marked as a TODO.
2) Allow R(n) notation in ARM register shifts. The code needs a rethink
but the tests we're leading toward will make the rewrite easier to test and
trust.
Dmitry Vyukov [Fri, 30 Jan 2015 10:31:43 +0000 (13:31 +0300)]
cmd/trace: add new command
Trace command allows to visualize and analyze traces.
Run as:
$ go tool trace binary trace.file
The commands opens web browser with the main page,
which contains links for trace visualization,
blocking profiler, network IO profiler and per-goroutine
traces.
Also move trace parser from runtime/pprof/trace_parser_test.go
to internal/trace/parser.go, so that it can be shared between
tests and the command.
Change-Id: Ic97ed59ad6e4c7e1dc9eca5e979701a2b4aed7cf
Reviewed-on: https://go-review.googlesource.com/3601 Reviewed-by: Andrew Gerrand <adg@golang.org>
David Crawshaw [Fri, 20 Feb 2015 18:15:56 +0000 (13:15 -0500)]
[dev.cc] runtime: print to stderr as well as android logd
Restores stack traces in the android/arm builder.
Change-Id: If637aa2ed6f8886126b77cf9cc8a0535ec7c4369
Reviewed-on: https://go-review.googlesource.com/5453 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Change-Id: I7d598e53de86586bb9702d8e9276a4d6aece2dfc
Reviewed-on: https://go-review.googlesource.com/4950 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Dmitry Vyukov [Sat, 14 Feb 2015 12:54:25 +0000 (15:54 +0300)]
runtime: adjust program counters in race detector
In most cases we pass return PC to race detector,
and race runtime subtracts one from them.
However, in manual instrumentation in runtime
we pass function start PC to race runtime.
Race runtime can't distinguish these cases
and so it does not subtract one from top PC.
This leads to bogus line numbers in some cases.
Make it consistent and always pass what looks
like a return PC, so that race runtime can
subtract one and still get PC in the same function.
Also delete two unused functions.
Update #8053
Change-Id: I4242dec5e055e460c9a8990eaca1d085ae240ed2
Reviewed-on: https://go-review.googlesource.com/4902 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Dmitry Vyukov [Tue, 17 Feb 2015 11:25:49 +0000 (14:25 +0300)]
runtime: fix cputicks on x86
See the following issue for context:
https://github.com/golang/go/issues/9729#issuecomment-74648287
In short, RDTSC can produce skewed results without preceding LFENCE/MFENCE.
Information on this matter is very scrappy in the internet.
But this is what linux kernel does (see rdtsc_barrier).
It also fixes the test program on my machine.
Update #9729
Change-Id: I3c1ffbf129fdfdd388bd5b7911b392b319248e68
Reviewed-on: https://go-review.googlesource.com/5033 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Mikio Hara [Tue, 10 Feb 2015 03:24:11 +0000 (12:24 +0900)]
net, syscall: more accurate parsers for routing messages on BSD variants
This changes fixes two issues with regard to handling routing messages
as follows:
- Misparsing on platforms (such as FreeBSD) supporting multiple
architectures in the same kernel (kern.supported_archs="amd64 i386")
- Misparsing with unimplemented messages such as route, interface
address state notifications
To fix those issues, this change implements all the required socket
address parsers, adds a processor architecture identifying function to
FreeBSD and tests.
Fixes #9707.
Fixes #8203.
Change-Id: I7ed7b4a0b6f10f54b29edc681a2f35603f2d8d45
Reviewed-on: https://go-review.googlesource.com/4330 Reviewed-by: Ian Lance Taylor <iant@golang.org>
It would allow go get and friends to transparently work as expected,
automatically rewriting https URLs to use SSH for auth. This worked both
when pushing and pulling.
In Go 1.4 this broke, now requiring the use of `go get -f` instead of `go get`
in order to fetch private repositories. This seems neither intended nor
practical, as it requires changing a lot of tooling.
So just use `git config remote.origin.url` instead of `git remote -v` as
this reflects the actual substitution intended in the `insteadOf` config
directive.
Also remove now useless parsing.
Also add a check against supported schemes to avoid errors in later
commands using this URL and expecting such a scheme.
Fixes #9697
Change-Id: I907327f83504302288f913a68f8222a5c2d673ee
Reviewed-on: https://go-review.googlesource.com/3504 Reviewed-by: Andrew Gerrand <adg@golang.org>
Rob Pike [Fri, 20 Feb 2015 00:34:51 +0000 (16:34 -0800)]
[dev.cc] cmd/asm: bring asm on ppc64 in sync with 9a
I created a .s file that covered every instruction and operand production
in 9a/a.y and made sure that 9a and asm give bit-identical results for it.
I found a few things, including one addressing mode (R1+R2) that was
not present in the source we use. Fixed those
I also found quite a few things where 9a's grammar accepts the instruction
but liblink rejects it. These need to be sorted out, and I will do that separately.
Once that's done, I'll turn my test file into a proper test.
Changes to converter:
- fatal does not return, so no fallthrough after fatal in switch
- many more function results and variables identified as bool
- simplification of negated boolean expressions
Russ Cox [Wed, 18 Feb 2015 02:52:42 +0000 (21:52 -0500)]
[dev.cc] cmd/gc: tweak default fatal in ordersafeexpr for c2go
c2go was putting a fallthrough after the fatal call.
Changed c2go to know that fatal doesn't return,
but then there is a missing return at the end of
the translated Go function.
Move code around a little to make C and Go agree.
Russ Cox [Thu, 19 Feb 2015 18:38:46 +0000 (13:38 -0500)]
runtime: reorganize memory code
Move code from malloc1.go, malloc2.go, mem.go, mgc0.go into
appropriate locations.
Factor mgc.go into mgc.go, mgcmark.go, mgcsweep.go, mstats.go.
A lot of this code was in certain files because the right place was in
a C file but it was written in Go, or vice versa. This is one step toward
making things actually well-organized again.
Austin Clements [Thu, 19 Feb 2015 02:56:12 +0000 (21:56 -0500)]
runtime: use more natural types in struct workbuf
Until recently, struct workbuf had only lfnode and uintptr fields
before the obj array to make it convenient to compute the size of the
obj array. It slowly grew more fields until this became inconvenient
enough that it was restructured to make the size computation easy.
Now the size computation doesn't care what the field types are, so
switch to more natural types.
Austin Clements [Tue, 17 Feb 2015 15:53:31 +0000 (10:53 -0500)]
runtime: switch to gcWork abstraction
This converts the garbage collector from directly manipulating work
buffers to using the new gcWork abstraction.
The previous management of work buffers was rather ad hoc. As a
result, switching to the gcWork abstraction changes many details of
work buffer management.
If greyobject fills a work buffer, it can now pull from work.partial
in addition to work.empty.
Previously, gcDrain started with a partial or empty work buffer and
fetched an empty work buffer if it filled its current buffer (in
greyobject). Now, gcDrain starts with a full work buffer and fetches
an partial or empty work buffer if it fills its current buffer (in
greyobject). The original behavior was bad because gcDrain would
immediately drop the empty work buffer returned by greyobject and
fetch a full work buffer, which greyobject was likely to immediately
overflow, fetching another empty work buffer, etc. The new behavior
isn't great at the start because greyobject is likely to immediately
overflow the full buffer, but the steady-state behavior should be more
stable. Both before and after this change, gcDrain fetches a full
work buffer if it drains its current buffer. Basically all of these
choices are bad; the right answer is to use a dual work buffer scheme.
Previously, shade always fetched a work buffer (though usually from
m.currentwbuf), even if the object was already marked. Now it only
fetches a work buffer if it actually greys an object.
Austin Clements [Thu, 12 Feb 2015 17:53:48 +0000 (12:53 -0500)]
runtime: introduce higher-level GC work abstraction
This introduces a producer/consumer abstraction for GC work pointers
that internally handles the details of filling, draining, and
shuffling work buffers.
In addition to simplifying the GC code, this should make it easy for
us to change how we use work buffers, including cleaning up how we use
the work.partial queue, reintroducing a FIFO lookahead cache, adding
prefetching, and using dual buffers to avoid flapping.
This commit doesn't change any existing code. The following commit
will switch the garbage collector from explicit workbuf manipulation
to gcWork.