Adam Langley [Fri, 21 Feb 2014 20:56:41 +0000 (15:56 -0500)]
crypto/tls: enforce that either ServerName or InsecureSkipVerify be given.
crypto/tls has two functions for creating a client connection: Dial,
which most users are expected to use, and Client, which is the
lower-level API.
Dial does what you expect: it gives you a secure connection to the host
that you specify and the majority of users of crypto/tls appear to work
fine with it.
Client gives more control but needs more care. Specifically, if it
wasn't given a server name in the tls.Config then it didn't check that
the server's certificates match any hostname - because it doesn't have
one to check against. It was assumed that users of the low-level API
call VerifyHostname on the certificate themselves if they didn't supply
a hostname.
A review of the uses of Client both within Google and in a couple of
external libraries has shown that nearly all of them got this wrong.
Thus, this change enforces that either a ServerName or
InsecureSkipVerify is given. This does not affect tls.Dial.
See discussion at https://groups.google.com/d/msg/golang-nuts/4vnt7NdLvVU/b1SJ4u0ikb0J.
Russ Cox [Thu, 20 Feb 2014 21:18:05 +0000 (16:18 -0500)]
runtime/debug: add SetPanicOnFault
SetPanicOnFault allows recovery from unexpected memory faults.
This can be useful if you are using a memory-mapped file
or probing the address space of the current program.
Russ Cox [Thu, 20 Feb 2014 20:58:47 +0000 (15:58 -0500)]
runtime: use goc2c as much as possible
Package runtime's C functions written to be called from Go
started out written in C using carefully constructed argument
lists and the FLUSH macro to write a result back to memory.
For some functions, the appropriate parameter list ended up
being architecture-dependent due to differences in alignment,
so we added 'goc2c', which takes a .goc file containing Go func
declarations but C bodies, rewrites the Go func declaration to
equivalent C declarations for the target architecture, adds the
needed FLUSH statements, and writes out an equivalent C file.
That C file is compiled as part of package runtime.
Native Client's x86-64 support introduces the most complex
alignment rules yet, breaking many functions that could until
now be portably written in C. Using goc2c for those avoids the
breakage.
Separately, Keith's work on emitting stack information from
the C compiler would require the hand-written functions
to add #pragmas specifying how many arguments are result
parameters. Using goc2c for those avoids maintaining #pragmas.
For both reasons, use goc2c for as many Go-called C functions
as possible.
This CL is a replay of the bulk of CL 15400047 and CL 15790043,
both of which were reviewed as part of the NaCl port and are
checked in to the NaCl branch. This CL is part of bringing the
NaCl code into the main tree.
No new code here, just reformatting and occasional movement
into .h files.
LGTM=r
R=dave, alex.brainman, r
CC=golang-codereviews
https://golang.org/cl/65220044
Russ Cox [Thu, 20 Feb 2014 20:50:30 +0000 (15:50 -0500)]
cmd/pack: fix match
Match used len(ar.files) == 0 to mean "match everything"
but it also deleted matched things from the list, so once you
had matched everything you asked for, match returned true
for whatever was left in the archive too.
Concretely, if you have an archive containing f1, f2, then
pack t foo.a f1
would match f1 and then, because len(ar.files) == 0 after
deleting f1 from the match list, also match f2.
Avoid the problem by recording explicitly whether match
matches everything.
cmd/ld: fix off-by-one error in DWARF .debug_line transcription
The liblink refactor changed the DWARF .debug_line flow control. The mapping was off by one pcline entry. The fix here preserves pc until it can be compared to pcline.pc.
Sample dwarfdump .debug_line output for main.main from the program in issue 7351, before liblink (correct):
Rob Pike [Thu, 20 Feb 2014 01:17:36 +0000 (17:17 -0800)]
syscall: terminate error string in exec package on Plan 9
Try to prevent messages like this:
'./pack' file does not exist����������������������������������������������������������������������������������������������������
TBR=adonovan
Rob Pike [Wed, 19 Feb 2014 23:01:50 +0000 (15:01 -0800)]
cmd/pack: dump output of command of "go env" command in test
Get more information to help understand build failure on Plan 9.
Also Windows.
(TestHello is failing because GOCHAR does not appear in output.
What does?)
Precisestack makes stack collection completely precise,
in the sense that there are no "used and not set" errors
in the collection of stack frames, no times where the collector
reads a pointer from a stack word that has not actually been
initialized with a pointer (possibly a nil pointer) in that function.
The most important part is interfaces: precisestack means
that if reading an interface value, the interface value is guaranteed
to be initialized, meaning that the type word can be relied
upon to be either nil or a valid interface type word describing
the data word.
This requires additional zeroing of certain values on the stack
on entry, which right now costs about 5% overall execution
time in all.bash. That cost will come down before Go 1.3
(issue 7345).
There are at least two known garbage collector bugs right now,
issues 7343 and 7344. The first happens even without precisestack.
The second I have only seen with precisestack, but that does not
mean that precisestack is what causes it. In fact it is very difficult
to explain by what precisestack does directly. Precisestack may
be exacerbating an existing problem. Both of those issues are
marked for Go 1.3 as well.
The reasons for enabling precisestack now are to give it more
time to soak and because the copying stack work depends on it.
Russ Cox [Wed, 19 Feb 2014 22:08:44 +0000 (17:08 -0500)]
cmd/pack: add 'c' command to create archive
When Go 1.3 is released, this will keep existing
Go 1.2 build scripts that use 'go tool pack grc' working.
For efficiency, such scripts should be changed to
use 6g -pack instead, but keeping the old behavior
available enables a more graceful transition.
Rob Pike [Wed, 19 Feb 2014 20:50:50 +0000 (15:50 -0500)]
cmd/gc: fix printf format in typecheck.c
There are probably more of these, but bound and len are 64 bits so use %lld
in message about array index out of bounds.
Fixes the 386 build.
Adam Langley [Wed, 19 Feb 2014 16:17:09 +0000 (11:17 -0500)]
crypto/tls: improve documentation for ServerName.
Users of the low-level, Client function are frequenctly missing the
fact that, unless they pass a ServerName to the TLS connection then it
cannot verify the certificates against any name.
This change makes it clear that at least one of InsecureSkipVerify and
ServerName should always be set.
Russ Cox [Wed, 19 Feb 2014 15:01:15 +0000 (10:01 -0500)]
cmd/go: skip writing dwarf debug info for ephemeral binaries
Update #6853
For an ephemeral binary - one created, run, and then deleted -
there is no need to write dwarf debug information, since the
binary will not be used with gdb. In this case, instruct the linker
not to spend time and disk space generating the debug information
by passing the -w flag to the linker.
Omitting dwarf information reduces the size of most binaries by 25%.
We may be more aggressive about this in the future.
LGTM=bradfitz, r
R=r, bradfitz
CC=golang-codereviews
https://golang.org/cl/65890043
Russ Cox [Wed, 19 Feb 2014 15:00:44 +0000 (10:00 -0500)]
cmd/ld: drop gcargs, gclocals symbols from symbol table
Update #6853
Every function now has a gcargs and gclocals symbol
holding associated garbage collection information.
Put them all in the same meta-symbol as the go.func data
and then drop individual entries from symbol table.
Removing gcargs and gclocals reduces the size of a
typical binary by 10%.
Russ Cox [Wed, 19 Feb 2014 04:41:15 +0000 (23:41 -0500)]
cmd/ld: remove Plan 9 symbol table
Update #6853
Nothing reads the Plan 9 symbol table anymore.
The last holdout was 'go tool nm', but since being rewritten in Go
it uses the standard symbol table for the binary format
(ELF, Mach-O, PE) instead.
Removing the Plan 9 symbol table saves ~15% disk space
on most binaries.
Two supporting changes included in this CL:
debug/gosym: use Go 1.2 pclntab to synthesize func-only
symbol table when there is no Plan 9 symbol table
debug/elf, debug/macho, debug/pe: ignore final EOF from ReadAt
Dominik Honnef [Wed, 19 Feb 2014 03:23:55 +0000 (22:23 -0500)]
misc/emacs: add support for ff-find-other-file
c-mode classically uses ff-find-other-file to toggle between headers
and implementation. For Go it seemingly makes sense to jump between
implementation and test.
While there's no enforced mapping of file names for tests, the mapping
in this CL seems to be very common at least throughout the standard
library, and ff-find-other-file fails gracefully when the mapping
doesn't apply.
Marcel van Lohuizen [Tue, 18 Feb 2014 19:12:59 +0000 (20:12 +0100)]
unicode: upgrade to Unicode 6.3.0
This is a relatively minor change.
This does not result in changes to go.text/unicode/norm. The go.text
packages will therefore be relatively unaffected. It does make the
way for an upgrade to CLDR 24, though.
The tests of all.bash pass, as well as the tests in go.text after
this update.
««« original CL description
cmd/gc, runtime: enable precisestack by default
Precisestack makes stack collection completely precise,
in the sense that there are no "used and not set" errors
in the collection of stack frames, no times where the collector
reads a pointer from a stack word that has not actually been
initialized with a pointer (possibly a nil pointer) in that function.
The most important part is interfaces: precisestack means
that if reading an interface value, the interface value is guaranteed
to be initialized, meaning that the type word can be relied
upon to be either nil or a valid interface type word describing
the data word.
This requires additional zeroing of certain values on the stack
on entry, which right now costs about 5% overall execution
time in all.bash. That cost will come down before Go 1.3
(issue 7345).
There are at least two known garbage collector bugs right now,
issues 7343 and 7344. The first happens even without precisestack.
The second I have only seen with precisestack, but that does not
mean that precisestack is what causes it. In fact it is very difficult
to explain by what precisestack does directly. Precisestack may
be exacerbating an existing problem. Both of those issues are
marked for Go 1.3 as well.
The reasons for enabling precisestack now are to give it more
time to soak and because the copying stack work depends on it.
Russ Cox [Tue, 18 Feb 2014 01:12:40 +0000 (20:12 -0500)]
cmd/gc, runtime: enable precisestack by default
Precisestack makes stack collection completely precise,
in the sense that there are no "used and not set" errors
in the collection of stack frames, no times where the collector
reads a pointer from a stack word that has not actually been
initialized with a pointer (possibly a nil pointer) in that function.
The most important part is interfaces: precisestack means
that if reading an interface value, the interface value is guaranteed
to be initialized, meaning that the type word can be relied
upon to be either nil or a valid interface type word describing
the data word.
This requires additional zeroing of certain values on the stack
on entry, which right now costs about 5% overall execution
time in all.bash. That cost will come down before Go 1.3
(issue 7345).
There are at least two known garbage collector bugs right now,
issues 7343 and 7344. The first happens even without precisestack.
The second I have only seen with precisestack, but that does not
mean that precisestack is what causes it. In fact it is very difficult
to explain by what precisestack does directly. Precisestack may
be exacerbating an existing problem. Both of those issues are
marked for Go 1.3 as well.
The reasons for enabling precisestack now are to give it more
time to soak and because the copying stack work depends on it.
Russ Cox [Tue, 18 Feb 2014 01:11:53 +0000 (20:11 -0500)]
runtime: clear f, arg to avoid leak in timerproc
I have seen this cause leaks where not all objects in a sync.Pool
would be reclaimed during the sync package tests.
I found it while debugging the '0 of 100 finalized' failure we are
seeing on arm, but it seems not to be the root cause for that one.
Dmitriy Vyukov [Mon, 17 Feb 2014 02:29:56 +0000 (06:29 +0400)]
testing: ease writing parallel benchmarks
Add b.RunParallel function that captures parallel benchmark boilerplate:
creates worker goroutines, joins worker goroutines, distributes work
among them in an efficient way, auto-tunes grain size.
Fixes #7090.
Russ Cox [Sun, 16 Feb 2014 01:01:15 +0000 (20:01 -0500)]
cmd/gc: record &x[0] as taking address of x, if x is an array
Not recording the address being taken was causing
the liveness analysis not to preserve x in the absence
of direct references to x, which in turn was making the
net test fail with GOGC=0.
In addition to the test, this fixes a bug wherein
GOGC=0 go test -short net
crashed if liveness analysis was in use (like at tip, not like Go 1.2).
Russ Cox [Sun, 16 Feb 2014 01:00:57 +0000 (20:00 -0500)]
cmd/gc: avoid pointer beyond array in range loop
This problem was discovered by reading the code.
I have not seen it in practice, nor do I have any ideas
on how to trigger it reliably in a test. But it's still worth
fixing.
Russ Cox [Sat, 15 Feb 2014 15:58:55 +0000 (10:58 -0500)]
cmd/gc: correct liveness for fat variables
The VARDEF placement must be before the initialization
but after any final use. If you have something like s = ... using s ...
the rhs must be evaluated, then the VARDEF, then the lhs
assigned.
There is a large comment in pgen.c on gvardef explaining
this in more detail.
This CL also includes Ian's suggestions from earlier CLs,
namely commenting the use of mode in link.h and fixing
the precedence of the ~r check in dcl.c.
This CL enables the check that if liveness analysis decides
a variable is live on entry to the function, that variable must
be a function parameter (not a result, and not a local variable).
If this check fails, it indicates a bug in the liveness analysis or
in the generated code being analyzed.
The race detector generates invalid code for append(x, y...).
The code declares a temporary t and then uses cap(t) before
initializing t. The new liveness check catches this bug and
stops the compiler from writing out the buggy code.
Consequently, this CL disables the race detector tests in
run.bash until the race detector bug can be fixed
(golang.org/issue/7334).
Except for the race detector bug, the liveness analysis check
does not detect any problems (this CL and the previous CLs
fixed all the detected problems).
The net test still fails with GOGC=0 but the rest of the tests
now pass or time out (because GOGC=0 is so slow).
Rob Pike [Sat, 15 Feb 2014 00:26:47 +0000 (16:26 -0800)]
text/template: don't panic when function call evaluates a nil pointer
Catch the error instead and return it to the user. Before this fix,
the template package panicked. Now you get:
template: bug11:1:14: executing "bug11" at <.PS>: dereference of nil pointer of type *string
Extended example at http://play.golang.org/p/uP6pCW3qKT
Adam Langley [Fri, 14 Feb 2014 22:17:19 +0000 (17:17 -0500)]
compress/bzip2: support superfluous Huffman levels.
These should never be found in a bzip2 file but it does appear that
there's a buggy encoder that is producing them. Since the official
bzip2 handles this case, this change makes the Go code do likewise.
With this change, the code produces the same output as the official
bzip2 code on the invalid example given in the bug.
Fixes #7279.
LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/64010043
David du Colombier [Fri, 14 Feb 2014 21:27:47 +0000 (22:27 +0100)]
runtime: fix "invalid address in sys call" on Plan 9
Rfork is not splitting the stack when creating a new thread,
so the parent and child are executing on the same stack.
However, if the parent returns and keeps executing before
the child can read the arguments from the parent stack,
the child will not see the right arguments. The solution
is to load the needed pieces from the parent stack into
register before INT $64.
Ian Lance Taylor [Fri, 14 Feb 2014 19:06:53 +0000 (11:06 -0800)]
runtime: if traceback sees a breakpoint, don't change the PC
Changing the PC confuses gdb, because execution does not
continue where gdb expects it. Not changing the PC has the
potential to confuse a stack dump, but when running under gdb
it seems better to confuse a stack dump than to confuse gdb.
Mikio Hara [Fri, 14 Feb 2014 15:47:28 +0000 (00:47 +0900)]
syscall: make use of include/linux when generating system constants
On Linux include/net directory is just to help porting applications
from BSDs and files under net keep less information than include/linux.
Making use of files under include/linux instead of include/net prevents
lack of information.
Dmitriy Vyukov [Fri, 14 Feb 2014 09:24:48 +0000 (13:24 +0400)]
runtime: remove misleading message during crash
The following checkdead message is false positive:
$ go test -race -c runtime
$ ./runtime.test -test.cpu=2 -test.run=TestSmhasherWindowed -test.v
=== RUN TestSmhasherWindowed-2
checkdead: find g 18 in status 1
SIGABRT: abort
PC=0x42bff1
Dmitriy Vyukov [Fri, 14 Feb 2014 09:20:41 +0000 (13:20 +0400)]
runtime: fix mem profile when both large and small objects are allocated at the same stack
Currently small and large (size>rate) objects are merged into a single entry.
But rate adjusting is required only for small objects.
As a result pprof either incorrectly adjusts large objects
or does not adjust small objects.
With this change objects of different sizes are stored in different buckets.
Russ Cox [Fri, 14 Feb 2014 05:38:24 +0000 (00:38 -0500)]
cmd/gc: correct liveness for various non-returning functions
When the liveness code doesn't know a function doesn't return
(but the generated code understands that), the liveness analysis
invents a control flow edge that is not really there, which can cause
variables to seem spuriously live. This is particularly bad when the
variables are uninitialized.
Dmitriy Vyukov [Fri, 14 Feb 2014 05:20:51 +0000 (09:20 +0400)]
runtime: fix windows cpu profiler
Currently it periodically fails with the following message.
The immediate cause is the wrong base register when obtaining g
in sys_windows_amd64/386.s.
But there are several secondary problems as well.
runtime: unknown pc 0x0 after stack split
panic: invalid memory address or nil pointer dereference
fatal error: panic during malloc
[signal 0xc0000005 code=0x0 addr=0x60 pc=0x42267a]
runtime stack:
runtime.panic(0x7914c0, 0xc862af)
c:/src/perfer/work/windows-amd64-a15f344a9efa/go/src/pkg/runtime/panic.c:217 +0x2c
runtime: unexpected return pc for runtime.externalthreadhandler called from 0x0
Russ Cox [Fri, 14 Feb 2014 04:56:53 +0000 (23:56 -0500)]
cmd/gc: correct liveness for func ending in panic
The registerization code needs the function to end in a RET,
even if that RET is actually unreachable.
The liveness code needs to avoid such unreachable RETs.
It had a special case for final RET after JMP, but no case
for final RET after UNDEF. Instead of expanding the special
cases, let fixjmp - which already knows what is and is not
reachable definitively - mark the unreachable RET so that
the liveness code can identify it.
Russ Cox [Fri, 14 Feb 2014 04:33:20 +0000 (23:33 -0500)]
cmd/gc: correct liveness for wrappers containing tail jumps
A normal RET is treated as using the return values,
but a tail jump RET does not - it is jumping to the
function that is going to fill in the return values.
If a tail jump RET is recorded as using the return values,
since nothing initializes them they will be marked as
live on entry to the function, which is clearly wrong.
Found and tested by the new code in plive.c that looks
for variables that are incorrectly live on entry.
That code is disabled for now because there are other
cases remaining to be fixed. But once it is enabled,
test/live1.go becomes a real test of this CL.
Russ Cox [Fri, 14 Feb 2014 03:45:16 +0000 (22:45 -0500)]
cmd/gc: handle variable initialization by block move in liveness
Any initialization of a variable by a block copy or block zeroing
or by multiple assignments (componentwise copying or zeroing
of a multiword variable) needs to emit a VARDEF. These cases were not.
Russ Cox [Fri, 14 Feb 2014 03:30:35 +0000 (22:30 -0500)]
cmd/5g, cmd/8g: fix build
The test added in CL 63630043 fails on 5g and 8g because they
were not emitting the VARDEF instruction when clearing a fat
value by clearing the components. 6g had the call in the right place.
Mikio Hara [Fri, 14 Feb 2014 03:20:21 +0000 (12:20 +0900)]
net: disable TestDNSThreadLimit even in non-short mode by default
TestDNSThreadLimit creates tons of DNS queries and it occasionally
causes an unintentional traffic jam and/or crash of some virtual
machine software, especially its builtin networking stuff.
We can run TestDNSThreadLimit with -dnsflood flag instead.
LGTM=dave, rsc
R=rsc, dave
CC=golang-codereviews
https://golang.org/cl/63600043
Shenghou Ma [Fri, 14 Feb 2014 02:01:33 +0000 (21:01 -0500)]
cmd/gc: rephrase the invalid indexing operation error message
Old:
prog.go:9: invalid operation: this[i] (index of type int)
New:
prog.go:9: invalid operation: this[i] (type int does not support indexing)
LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/52540043
Russ Cox [Fri, 14 Feb 2014 01:59:39 +0000 (20:59 -0500)]
cmd/gc: distinguish unnamed vs blank-named return variables better
Before, an unnamed return value turned into an ONAME node n with n->sym
named ~anon%d, and n->orig == n.
A blank-named return value turned into an ONAME node n with n->sym
named ~anon%d but n->orig == the original blank n. Code generation and
printing uses n->orig, so that this node formatted as _.
But some code does not use n->orig. In particular the liveness code does
not know about the n->orig convention and so mishandles blank identifiers.
It is possible to fix but seemed better to avoid the confusion entirely.
Now the first kind of node is named ~r%d and the second ~b%d; both have
n->orig == n, so that it doesn't matter whether code uses n or n->orig.
After this change the ->orig field is only used for other kinds of expressions,
not for ONAME nodes.
This requires distinguishing ~b from ~r names in a few places that care.
It fixes a liveness analysis bug without actually changing the liveness code.
Russ Cox [Fri, 14 Feb 2014 00:59:09 +0000 (19:59 -0500)]
cmd/gc: relax address-of escape analysis
Make the loop nesting depth of &x depend on where x is declared,
not on where the &x appears. The latter is only a conservative
estimate of the former. Being more careful can avoid some
variables escaping, and it is easier to reason about.
It would have avoided issue 7313, although that was still a bug
worth fixing.
Not much effect in the tree: one variable in the whole tree
is saved from a heap allocation (something in x509 parsing).
Markus Zimmermann [Fri, 14 Feb 2014 00:43:52 +0000 (16:43 -0800)]
container/list: mark must be an element of the list
The methods MoveAfter and MoveBefore of the container/list package did silently corrupt the interal structure of the list if a mark element is used which is not an element of the list.
Nick Craig-Wood [Fri, 14 Feb 2014 00:19:38 +0000 (16:19 -0800)]
math/big: Optimise ARM assembler
Tweak the ARM assembler to improve its performance.
* Use TEQ instead of CMP which preserves the carry flag. This means
we can avoid saving and restoring CPSR which is very slow.
* Use conditional instructions to read the value of the carry flag.
* Use 3 argument ARM instructions to save instructions
* Improve scheduling for MOVW instructions (LDR)
* Use RSB constant to save an instruction in bitLen
Results of -test.bench 'VV|VW|VU|WW|Bit' -test.benchtime 3s on Samsung
Exynos5 Chromebook.
There are a few small regressions in the benchmarks which I believe to
be noise, perhaps due to different cacheline alignment.
The changes to bitLen are apparently no faster, however less
instructions means less I-cache usage which is a win. I suspect it
will be a win on older ARM processors.
David Symonds [Thu, 13 Feb 2014 23:15:38 +0000 (10:15 +1100)]
net/http: increase panic stack trace buffer size from 4 KB to 64 KB.
4 KB is a bit too small in some situations (e.g. panic during a
template execution), and ends up with an unhelpfully-truncated trace.
64 KB should be much more likely to capture the useful information.
There's not a garbage generation issue, since this code should only
be triggered when there's something seriously wrong with the program.
David du Colombier [Thu, 13 Feb 2014 19:30:42 +0000 (20:30 +0100)]
lib9: fix Solaris build
In file included from src/lib9/utf/utfecpy.c:17:0:
src/lib9/utf/utfdef.h:28:0: error: "nil" redefined [-Werror]
In file included from src/lib9/utf/utfrrune.c:17:0:
src/lib9/utf/utfdef.h:28:0: error: "nil" redefined [-Werror]