Keith Randall [Tue, 30 Sep 2014 15:51:02 +0000 (08:51 -0700)]
runtime: fix scanning of gc work buffer
GC types were not being generated for the garbage collector
work buffer. The markfor object was being collected as a result.
This broke amd64p32 and maybe plan9 builds. Why it didn't break
every build I'm not sure...
Keith Randall [Tue, 30 Sep 2014 04:21:36 +0000 (21:21 -0700)]
runtime: initialize traceback variables earlier
Our traceback code needs to know the PC of several special
functions, including goexit, mcall, etc. Make sure that
these PCs are initialized before any traceback occurs.
net/http: make Transport.CloseIdleConnections also close pending dials
See comment 4 of https://code.google.com/p/go/issues/detail?id=8483#c4:
"So if a user creates a http.Client, issues a bunch of
requests and then wants to shutdown it and all opened connections;
what is she intended to do? The report suggests that just waiting for
all pending requests and calling CloseIdleConnections won't do, as
there can be new racing connections. Obviously she can't do what
you've done in the test, as it uses the unexported function. If this
happens periodically, it can lead to serious resource leaks (the
transport is also preserved alive). Am I missing something?"
This CL tracks the user's intention to close all idle
connections (CloseIdleConnections sets it true; and making a
new request sets it false). If a pending dial finishes and
nobody wants it, before it's retained for a future caller, the
"wantIdle" bool is checked and it's closed if the user has
called CloseIdleConnections without a later call to make a new
request.
Dmitri Shuralyov [Tue, 30 Sep 2014 00:04:48 +0000 (17:04 -0700)]
go/format, cmd/gofmt: fix issues with partial Go code with indent
Fixes #5551.
Fixes #4449.
Adds tests for both issues.
Note that the two issues occur only when formatting partial Go code
with indent.
The best way to understand the change is as follows: I took the code
of cmd/gofmt and go/format, combined it into one unified code that
does not suffer from either 4449 nor 5551, and then applied that code
to both cmd/gofmt and go/format.
As a result, there is now much more identical code between the two
packages, making future code deduplication easier (it was not possible
to do that now without adding public APIs, which I was advised not to
do at this time).
More specifically, I took the parse() of cmd/gofmt which correctly
preserves comments (issue 5551) and modified it to fix issue where
it would sometimes modify literal values (issue 4449).
I ended up removing the matchSpace() function because it no longer
needed to do some of its work (insert indent), and a part of its work
had to be done in advance (determining the indentation of first code
line), because that calculation is required for cfg.Fprint() to run.
adjustIndent is used to adjust the indent of cfg.Fprint() to compensate
for the body of wrapper func being indented by one level. This allows
to get rid of the bytes.Replace text manipulation of inner content,
which was problematic and sometimes altered raw string literals (issue
4449). This means that sometimes the value of cfg.Indent is negative,
but that works as expected.
So now the algorithm for formatting partial Go code is:
1. Determine and prepend leading space of original source.
2. Determine and prepend indentation of first code line.
3. Format and write partial Go code (with all of its leading &
trailing space trimmed).
4. Determine and append trailing space of original source.
Tom Linford [Mon, 29 Sep 2014 23:51:49 +0000 (09:51 +1000)]
x509: add root certs for android.
On android, root certificates appear to be stored in the folder
/system/etc/security/cacerts, which has many certs in several
different files. This change adds a new array of directories in
which certs can be found.
To test this, I simply tried making a request with the http
library to an HTTPS URL on an android emulator and manually
verified that it worked.
Ian Lance Taylor [Mon, 29 Sep 2014 20:32:14 +0000 (13:32 -0700)]
cmd/yacc: fix handling of tokens that don't start with letters
CL 149110043 changed yacc to no longer keep a leading space
for quoted tokens. That is OK by itself but unfortunately
yacc was relying on that leading space to notice which tokens
it should not output as const declarations.
Add a few such tokens to expr.y, although it won't make any
immediate difference as we seem to have no tests for yacc.
Robert Griesemer [Mon, 29 Sep 2014 19:44:50 +0000 (12:44 -0700)]
spec: specify variable initialization order explicitly
The existing spec rules on package initialization were
contradictory: They specified that 1) dependent variables
are initialized in dependency order, and 2) independent
variables are initialized in declaration order. This 2nd
rule cannot be satisfied in general. For instance, for
var (
c = b + 2
a = 0
b = 1
)
because of its dependency on b, c must be initialized after b,
leading to the partial order b, c. Because a is independent of
b but is declared before b, we end up with the order: a, b, c.
But a is also independent of c and is declared after c, so the
order b, c, a should also be valid in contradiction to a, b, c.
The new rules are given in form of an algorithm which outlines
initialization order explicitly.
which gives the desired mask in X3. The goal in using the
CMPSD was to avoid a conditional branch.
This code fails when called from a PortAudio callback.
In particular, the failure behavior is exactly as if the
CMPSD always chose the 'true' execution.
Notice that the comparison X0 == X3 is comparing as
floating point values the 64-bit pattern v&2 and the actual
floating point value zero. The only possible values for v&2
are 0x0000000000000000 (floating point zero)
and 0x0000000000000002 (floating point 1e-323, a denormal).
If they are both comparing equal to zero, I conclude that
in a PortAudio callback (whatever that means), the processor
is running in "denormals are zero" mode.
I confirmed this by placing the processor into that mode
and running the test case in the bug; it produces the
incorrect output reported in the bug.
In general, if a Go program changes the floating point math
modes to something other than what Go expects, the math
library is not going to work exactly as intended, so we might
be justified in not fixing this at all.
However, it seems reasonable that the client code might
have expected "denormals are zero" mode to only affect
actual processing of denormals. This code has produced
what is in effect a gratuitous denormal by being extra clever.
There is nothing about the computation being requested
that fundamentally requires a denormal.
It is also easy to do this computation in integer math instead:
mask = ((v&2)>>1)-1
Do that.
For the record, the other math tests that fail if you put the
processor in "denormals are zero" mode are the tests for
Frexp, Ilogb, Ldexp, Logb, Log2, and FloatMinMax, but all
fail processing denormal inputs. Sincos was the only function
for which that mode causes incorrect behavior on non-denormal inputs.
The existing tests check that the new assembly is correct.
There is no test for behavior in "denormals are zero" mode,
because I don't want to add assembly to change that.
Fixes #8623.
LGTM=josharian
R=golang-codereviews, josharian
CC=golang-codereviews, iant, r
https://golang.org/cl/151750043
cmd/go: always build _test.go files and link into test
go test's handling of _test.go files when the entire
package's set of files has no Test functions has varied
over the past few releases. There are a few interesting
cases (all contain no Test functions):
(1) x_test.go has syntax errors
(2) x_test.go has type errors
(3) x_test.go has runtime errors (say, a func init that panics)
In Go 1.1, tests with (1) or (2) failed; (3) passed.
In Go 1.2, tests with (1) or (2) failed; (3) passed.
In Go 1.3, tests with (1) failed; (2) or (3) passed.
After this CL, tests with (1), (2), or (3) all fail.
This is clearly a corner case, but it seems to me that
the behavior of the test should not change if you
add or remove a line like
func TestAlwaysPasses(t *testing.T) {}
That implies that the _test.go files must always
be built and always be imported into the test binary.
Doing so means that (1), (2), and (3) must all fail.
Fixes #8337.
LGTM=iant
R=golang-codereviews, iant
CC=adg, golang-codereviews, r
https://golang.org/cl/150980043
1) yacc accepts either 'x' or "x" to mean token value 0x78
2) yacc also accepts 'xyz' and "XYZ" to mean token value 0x78
Use strconv.Unquote to simplify the handling of quoted
strings and check that each has only one rune.
Although this does clean things up, it makes 'x' and "x"
treated as different internally (now they are stored as
`'x'` and `"x"`; before they were both ` x`). Grammars that
use both interchangeably will now die with an error
similar to the one from issue 7967:
yacc bug -- cannot have 2 different Ts with same value
"+" and '+'
The echoing of the quotes should make clear what is going on.
The other semantic change caused by using strconv.Unquote
is that '\"' and "\'" are no longer valid. Like in Go, they must be
spelled without the backslash: '"' and "'".
On the other hand, now yacc and Go agree about what character
and string literals mean.
The one line that you can't test easily was broken.
This manifested as a failure of a pre-existing test
in test.bash but I didn't notice it (there are a few other
long-standing failures that need to be fixed).
cmd/go: make build -a skip standard packages in Go releases
Today, 'go build -a my/pkg' and 'go install -a my/pkg'
recompile not just my/pkg and all its dependencies that
you wrote but also the standard library packages.
Recompiling the standard library is problematic on
some systems because the installed copy is not writable.
The -a behavior means that you can't use 'go install -a all'
or 'go install -a my/...' to rebuild everything after a Go
release - the rebuild stops early when it cannot overwrite
the installed standard library.
During development work, however, you do want install -a
to rebuild everything, because anything might have changed.
Resolve the conflict by making the behavior of -a depend
on whether we are using a released copy of Go or a devel copy.
In the release copies, -a no longer applies to the standard library.
In the devel copies, it still does.
This is the latest in a long line of refinements to the
"do I build this or not" logic. It is surely not the last.
cmd/go: make malformed import path message more precise
If you say 'go get -v' you get extra information when import
paths are not of the expected form.
If you say 'go get -v src/rsc.io/pdf' the message says that
src/rsc.io/pdf does not contain a hostname, which is incorrect.
The problem is that it does not begin with a hostname.
Fixes #7432.
LGTM=r
R=golang-codereviews, r
CC=bradfitz, golang-codereviews, iant
https://golang.org/cl/144650043
cmd/go: re-resolve and check vcs roots during go get -u
If you do 'go get -u rsc.io/pdf' and then rsc.io/pdf's redirect
changes to point somewhere else, after this CL a later
'go get -u rsc.io/pdf' will tell you that.
Fixes #8548.
LGTM=iant
R=golang-codereviews, iant
CC=adg, golang-codereviews, n13m3y3r, r
https://golang.org/cl/147170043
Keith Randall [Thu, 25 Sep 2014 14:59:01 +0000 (07:59 -0700)]
cgo: adjust return value location to account for stack copies.
During a cgo call, the stack can be copied. This copy invalidates
the pointer that cgo has into the return value area. To fix this
problem, pass the address of the location containing the stack
top value (which is in the G struct). For cgo functions which
return values, read the stktop before and after the cgo call to
compute the adjustment necessary to write the return value.
Rob Pike [Wed, 24 Sep 2014 21:33:30 +0000 (14:33 -0700)]
fmt: document and fix the handling of precision for strings and byte slices
Previous behavior was undocumented and inconsistent. Now it is documented
and consistent and measures the input size, since that makes more sense
when talking about %q and %x. For %s the change has no effect.
In linker, refuse to write conservative (array of pointers) as the
garbage collection type for any variable in the data/bss GC program.
In the linker, attach the Go type to an already-read C declaration
during dedup. This gives us Go types for C globals for free as long
as the cmd/dist-generated Go code contains the declaration.
(Most runtime C declarations have a corresponding Go declaration.
Both are bss declarations and so the linker dedups them.)
In cmd/dist, add a few more C files to the auto-Go-declaration list
in order to get Go type information for the C declarations into the linker.
In C compiler, mark all non-pointer-containing global declarations
and all string data as NOPTR. This allows them to exist in C files
without any corresponding Go declaration. Count C function pointers
as "non-pointer-containing", since we have no heap-allocated C functions.
In runtime, add NOPTR to the remaining pointer-containing declarations,
none of which refer to Go heap objects.
In runtime, also move os.Args and syscall.envs data into runtime-owned
variables. Otherwise, in programs that do not import os or syscall, the
runtime variables named os.Args and syscall.envs will be missing type
information.
I believe that this CL eliminates the final source of conservative GC scanning
in non-SWIG Go programs, and therefore...
Those C files would have been compiled with 6c.
It's close to impossible to use C correctly anymore,
and the C compilers are going away eventually.
Make them unavailable now.
runtime: keep g->syscallsp consistent after cgo->Go callbacks
Normally, the caller to runtime.entersyscall() must not return before
calling runtime.exitsyscall(), lest g->syscallsp become a dangling
pointer. runtime.cgocallbackg() violates this constraint. To work around
this, save g->syscallsp and g->syscallpc around cgo->Go callbacks, then
restore them after calling runtime.entersyscall(), which restores the
syscall stack frame pointer saved by cgocall. This allows the GC to
correctly trace a goroutine that is currently returning from a
Go->cgo->Go chain.
This also adds a check to proc.c that panics if g->syscallsp is clearly
invalid. It is not 100% foolproof, as it will not catch a case where the
stack was popped then pushed back beyond g->syscallsp, but it does catch
the present cgo issue and makes existing tests fail without the bugfix.
Rob Pike [Wed, 24 Sep 2014 01:24:35 +0000 (18:24 -0700)]
cmd/pack: fix c command for existing file
There were at least two bugs:
1) It would overwrite a non-archive.
2) It would truncate a non-archive and then fail.
In general the file handling was too clever to be correct.
Make it more straightforward, doing the creation
separately from archive management.
Rob Pike [Mon, 22 Sep 2014 18:46:02 +0000 (11:46 -0700)]
text/template: allow comparison functions to work between any integers
Previously, signed and unsigned integers could not be compared, but
this has problems with things like comparing 'x' with a byte in a string.
Since signed and unsigned integers have a well-defined ordering,
even though their types are different, and since we already allow
comparison regardless of the size of the integers, why not allow it
regardless of the sign?
This test is testing bad pointers. It loads the bad pointer into a pointer variable,
but before it gets a chance to dereference it, calls convT2E. That call causes a stack copy,
which exposes that live but bad pointer variable.
CL 144940043 renamed it from Sched to SchedType
to avoid a lowercasing conflict in the Go code with
the variable named sched.
We've been using just T resolve those conflicts, not Type.
The FooType pattern is already taken for the kind-specific
variants of the runtime Type structure: ChanType, MapType,
and so on. SchedType isn't a Type.