Rob Pike [Sat, 27 Jun 2015 01:07:30 +0000 (11:07 +1000)]
fmt: restore padding for %x on byte slices and strings
Also improve the documentation. A prior fix in this release
changed the properties for empty strings and slices, incorrectly.
Previous behavior is now restored and better documented.
Add lots of tests.
The behavior is that when using a string-like format (%s %q %x %X)
a byte slice is equivalent to a string, and printed as a unit. The padding
applies to the entire object. (The space and sharp flags apply
elementwise.)
Brad Fitzpatrick [Sat, 27 Jun 2015 03:51:13 +0000 (20:51 -0700)]
net/http: fix now-flaky TransportAndServerSharedBodyRace test
TestTransportAndServerSharedBodyRace got flaky after
issue #9662 was fixed by https://golang.org/cl/11412, which made
servers hang up on clients when a Handler stopped reading its body
early.
This test was affected by a race between the the two goroutines in the
test both only reading part of the request, which was an unnecessary
detail for what the test was trying to test (concurrent Read/Close
races on an *http.body)
Also remove an unused remnant from an old test from which this one was
derived. And make the test not deadlock when it fails. (which was why
the test was showing up as 2m timeouts on the dashboard)
Keith Randall [Sat, 27 Jun 2015 19:57:06 +0000 (12:57 -0700)]
cmd/link/internal/ld: exclude only real container symbols from symtab
It looks like the test for whether symbols contain subsymbols is wrong.
In particular, symbols in C libraries are mistakenly considered container
symbols.
Fix the test so only symbols which actually have a subsymbol
are excluded from the symtab. When linking cgo programs the list
of containers is small, something like:
Adam Langley [Sun, 14 Jun 2015 17:48:13 +0000 (10:48 -0700)]
encoding/asn1: be stricter by reserialising parsed times.
The time package does normalisation of times: for example day zero is
converted to the last day of the previous month and the 31st of February
is moved into March etc. This makes the ASN.1 parsing a little
worryingly lax.
This change causes the parser to reserialise parsed times to ensure that
they round-trip correctly and thus were not normalised.
Alex Brainman [Fri, 22 May 2015 00:58:57 +0000 (10:58 +1000)]
runtime: store syscall parameters in m not on stack
Stack can move during callback, so libcall struct cannot be stored on stack.
asmstdcall updates return values and errno in libcall struct parameter, but
these could be at different location when callback returns.
Store these in m, so they are not affected by GC.
Todd Neal [Thu, 18 Jun 2015 12:26:39 +0000 (07:26 -0500)]
net/http: fix race on postPendingDial test hook
The race occurs rarely, but by putting some delays and more reads/writes
of prePendingDial/postPendingDial in the handlePendingDial function I
could reproduce it.
Didier Spezia [Mon, 8 Jun 2015 20:04:14 +0000 (20:04 +0000)]
html/template: prevent panic while escaping pipelines
AFAIK, the documentation does not explicitly state whether
variables can store a callable entity or not. I believe the
current implementation in text/template assumes they cannot
though. The call builtin function is supposed to be used for
this purpose.
Template "{{0|$}}" should generate an error at runtime,
instead of a panic.
Similarly, template "{{0|(nil)}}" should not generate
a panic.
This CL aborts the sanitization process for a given pipeline
when no identifier can be derived from the selected node.
It happens with malformed pipelines.
We now have the following errors:
{{ 0 | $ }}
template: foo:1:10: executing "foo" at <$>: can't give argument to non-function $
{{ 0 | (nil) }}
template: foo:1:11: executing "foo" at <nil>: nil is not a command
Fixes #11118
Fixes #11356
Change-Id: Idae52f806849f4c9ab7aca1b4bb4b59a74723d0e
Reviewed-on: https://go-review.googlesource.com/10823 Reviewed-by: Rob Pike <r@golang.org>
c9s [Fri, 29 May 2015 17:12:35 +0000 (01:12 +0800)]
cmd/go: handle error when git remote origin doesn't exist
- Let runOutput return the error message
- When `git config ...` returns empty buffer, it means the config key is
correct, but there is no corresponding value.
- Return the correct error when the url of remote origin is not found.
- Update error message
Mikio Hara [Tue, 23 Jun 2015 12:40:33 +0000 (21:40 +0900)]
net: relax IP interface address determination on linux
Linux allows to have a peer IP address on IP interface over ethernet
link encapsulation, though it only installs a static route with the peer
address as an on-link nexthop.
Austin Clements [Fri, 26 Jun 2015 21:24:12 +0000 (17:24 -0400)]
runtime: repeat bitmap for slice of GCprog n-1 times, not n times
Currently, to write out the bitmap of a slice of a type with a GCprog,
we construct a new GCprog that executes the underlying type's GCprog
to write out the bitmap once and then repeats those bits n more times.
This results in n+1 repetitions of the bitmap, which is one more
repetition than it should be. This corrupts the bitmap of the heap
following the slice and may write past the mapped bitmap memory and
segfault.
Fix this by repeating the bitmap only n-1 more times.
Dmitry Vyukov [Sun, 8 Feb 2015 15:11:35 +0000 (18:11 +0300)]
sync: simplify WaitGroup
A comment in waitgroup.go describes the following scenario
as the reason to have dynamically created semaphores:
// G1: Add(1)
// G1: go G2()
// G1: Wait() // Context switch after Unlock() and before Semacquire().
// G2: Done() // Release semaphore: sema == 1, waiters == 0. G1 doesn't run yet.
// G3: Wait() // Finds counter == 0, waiters == 0, doesn't block.
// G3: Add(1) // Makes counter == 1, waiters == 0.
// G3: go G4()
// G3: Wait() // G1 still hasn't run, G3 finds sema == 1, unblocked! Bug.
However, the scenario is incorrect:
G3: Add(1) happens concurrently with G1: Wait(),
and so there is no reasonable behavior of the program
(G1: Wait() may or may not wait for G3: Add(1) which
can't be the intended behavior).
With this conclusion we can:
1. Remove dynamic allocation of semaphores.
2. Remove the mutex entirely and instead pack counter and waiters
into single uint64.
This makes the logic significantly simpler, both Add and Wait
do only a single atomic RMW to update the state.
Matt T. Proud [Sat, 20 Jun 2015 04:29:18 +0000 (06:29 +0200)]
testing/quick: improve function signature error.
This commit fixes a cosmetic defect whereby quick.Check reports that
the provided function returns too many values when it may, in fact,
return too few:
func f() {}
func TestFoo(t *testing.T) {
if err := quick.Check(f, nil); err != nil {
t.Fatal(err)
}
}
// yields
// $ go test -v foo_test.go
// === RUN TestFoo
// --- FAIL: TestFoo (0.00s)
// foo_test.go:76: function returns more than one value.
Marcel van Lohuizen [Sun, 21 Jun 2015 18:21:04 +0000 (20:21 +0200)]
unicode: upgrade to 8.0.0
Not sure if I'm on time for 1.5; Unicode 8 just got released.
Straighforward upgrade. Only changed maketables.go to prevent it from adding
the Cherokee upper and lower case mappings. This change causes the caseOrbit
table to NOT change. Added tests to verify that the relevant functions still
produce the correct result, even for Cherokee.
Russ Cox [Thu, 25 Jun 2015 23:27:20 +0000 (19:27 -0400)]
runtime: reduce slice growth during append to 2x
The new inlined code for append assumed that it could pass the
desired new cap to growslice, not the number of new elements.
But growslice still interpreted the argument as the number of new elements,
making it always grow by >2x (more precisely, 2x+1 rounded up
to the next malloc block size). At the time, I had intended to change
the other callers to use the new cap as well, but it's too late for that.
Instead, introduce growslice_n for the old callers and keep growslice
for the inlined (common case) caller.
Matt Layher [Sat, 13 Jun 2015 15:25:17 +0000 (11:25 -0400)]
net/http: add comment to exported ErrNoCookie and ErrNoLocation values
All other exported errors in net/http are commented. This change adds
documentation to ErrNoCookie and ErrNoLocation to explain where they are
returned, and why.
Austin Clements [Wed, 24 Jun 2015 15:43:40 +0000 (11:43 -0400)]
reflect: test repeats with large scalar tails
This adds a GC bitmap test of a type with many pointer bits and a
large scalar tail, such as the one in issue #11286. This test would
have failed prior to the fix in a8ae93f. This test is a more direct
version of the test introduced in that commit (which was distilled
from the failing test in the issue).
Austin Clements [Wed, 24 Jun 2015 15:25:28 +0000 (11:25 -0400)]
reflect: test GC bits for slices
Currently we test bitmap repetitions constructed by the compiler (for
small arrays) and repetitions performed by GC programs (large arrays
and reflect-constructed arrays), but we don't test dynamic repetitions
performed by the runtime for slice backing stores. Add tests that
parallel the array tests using slices.
Todd Neal [Wed, 17 Jun 2015 14:31:08 +0000 (09:31 -0500)]
net/http: Fix test that occasionally fails
The consecutive calls to Head would sometimes get different
connections depending on if the readLoop had finished executing
and placed its connection on the idle list or not. This change
ensures that readLoop completes before we make our second connection.
Ian Lance Taylor [Thu, 18 Jun 2015 01:28:50 +0000 (18:28 -0700)]
debug/elf: change R_ARM_REL32 to R_ARM_ABS32 in applyRelocationsARM
The original version of applyRelocationsARM was added in
http://golang.org/cl/7266. It was added to fix the ARM build, which
had been broken by http://golang.org/cl/6780.
Before CL 6780, there was no relocation processing for ARM. CL 6780
changed the code to require relocation processing for every supported
target. CL 7266 fixed the ARM build by adding a relocation processing
function, but in fact no actual processing was done. The code only
looked for REL32 relocations, but ARM debug info has no such
relocations. The test case added in CL 7266 doesn't have any either.
This didn't matter because no relocation processing was required on
ARM, at least not for GCC-generated debug info. GCC generates ABS32
relocations, but only against section symbols which have the value 0.
Therefore, the addition done by correct handling of ABS32 doesn't
change anything.
Clang, however, generates ABS32 relocations against local symbols,
some of which have non-zero values. For those, we need to handle
ABS32 relocations.
This patch corrects the CL 7266 to look for ABS32 relocations instead
of REL32 relocations. The code was already written to correctly
handle ABS32 relocations, it just mistakenly said REL32.
This is the ARM equivalent of https://golang.org/cl/96680045, which
fixed the same problem in the same way for clang on 386.
With this patch, clang-3.5 can be used to build Go on ARM GNU/Linux.
Fixes #8980.
Change-Id: I0c2d72eadfe6373bde99cd03eee40de6a582dda1
Reviewed-on: https://go-review.googlesource.com/11222 Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Adam Langley [Sun, 14 Jun 2015 23:41:31 +0000 (16:41 -0700)]
crypto/x509: don't panic when decrypting invalid PEM data.
If an encrypted PEM block contained ciphertext that was not a multiple
of the block size then the code would panic. This change tests for that
case and returns an error.
Todd Neal [Tue, 23 Jun 2015 23:59:52 +0000 (18:59 -0500)]
cmd/compile: fix half multiply issue
In walkdiv, an OMUL node was created and passed to typecheck,
before the op was changed back to OHMUL. In some instances,
the node that came back was an evaluated literal constant that
occurred with a full multiply. The end result was a literal node
with a non-shifted value and an OHMUL op. This change causes code
to be generated for the OHMUL.
Dmitry Vyukov [Wed, 24 Jun 2015 17:25:51 +0000 (19:25 +0200)]
cmd/compile: add instrumentation of OKEY
Instrument operands of OKEY.
Also instrument OSLICESTR. Previously it was not needed
because of preceeding bounds checks (which were instrumented).
But the preceeding bounds checks have disappeared.
Vincent Batts [Wed, 4 Mar 2015 17:29:16 +0000 (12:29 -0500)]
archive/tar: fix round-trip attributes
The issue was identified while
working with round trip FileInfo of the headers of hardlinks. Also,
additional test cases for hard link handling.
(review carried over from http://golang.org/cl/165860043)
Brad Fitzpatrick [Wed, 24 Jun 2015 09:53:24 +0000 (11:53 +0200)]
net/http: bound the number of bytes read seeking EOF in Handler's Body.Close
If a client sent a POST with a huge request body, calling
req.Body.Close in the handler (which is implicit at the end of a
request) would end up consuming it all.
Put a cap on that, using the same threshold used elsewhere for similar
cases.
Fixes #9662
Change-Id: I26628413aa5f623a96ef7c2609a8d03c746669e5
Reviewed-on: https://go-review.googlesource.com/11412 Reviewed-by: Andrew Gerrand <adg@golang.org>
Todd Neal [Wed, 24 Jun 2015 12:48:35 +0000 (07:48 -0500)]
fmt: handle negative width/prec when supplied as an argument
Negative width arguments now left align the way a minus-width in the
format string aligns. The minus in the format string overrides the sign
of the argument as in C.
Precision behavior is modified to include an error if the argument is
negative. This differs from a negative precision in a format string
which just terminates the format.
Additional checks for large magnitude widths and precisions are added to
make the runtime behavior (failure, but with different error messages),
more consistent between format string specified width/precision and
argument specified width/precision.
Fixes #11376
Change-Id: I8c7ed21088e9c18128a45d4c487c5ab9fafd13ef
Reviewed-on: https://go-review.googlesource.com/11405 Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
Matthew Dempsky [Tue, 23 Jun 2015 18:36:57 +0000 (11:36 -0700)]
net/http: sync Get and Head's documentation
Instead of ambiguously referring to "the Client's CheckRedirect
function" in Head, describe the default behavior like for Get as users
aren't expected to change DefaultClient.CheckRedirect.
While here, use consistent punctuation for the Get and Head Client
method documentation.
Brad Fitzpatrick [Wed, 24 Jun 2015 11:57:33 +0000 (13:57 +0200)]
net/http: don't always require certFile, keyFile in Server.ListenAndServerTLS
The ListenAndServerTLS function still requires the certFile and
keyFile, but the Server.ListenAndServerTLS method doesn't need to
require the certFile and keyFile if the Server.TLSConfig.Certificates
are already populated.
Fixes #8599
Change-Id: Id2e3433732f93e2619bfd78891f775d89f1d651e
Reviewed-on: https://go-review.googlesource.com/11413 Reviewed-by: Andrew Gerrand <adg@golang.org>
Didier Spezia [Sat, 20 Jun 2015 11:25:59 +0000 (11:25 +0000)]
go/format: fix //line corner case when formatting statements
The code formatting mechanism can be applied to partial Go code,
such as a list of statements. The statements are wrapped into a
function definition (to be parsed fine), and unwrapped after formatting.
When the statements contain //line annotations, it may fail,
because not all comments are flushed by the printer before the final '}'.
Formatting "\ta()\n//line :1" results in "\ta() }\n\n//line", which
is wrong.
Tweaked the wrapping/unwrapping code to make sure comments are flushed
before the '}'.
Fixes #11276
Change-Id: Id15c80279b0382ee9ed939cca1647f525c4929f5
Reviewed-on: https://go-review.googlesource.com/11282
Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
Dmitry Vyukov [Thu, 7 May 2015 09:33:45 +0000 (12:33 +0300)]
runtime/race: make test more robust
The test is flaky on builders lately. I don't see any issues other than
usage of very small sleeps. So increase the sleeps. Also take opportunity
to refactor the code.
On my machine this change significantly reduces failure rate with GOMAXPROCS=2.
I can't reproduce the failure with GOMAXPROCS=1.
Russ Cox [Wed, 24 Jun 2015 12:39:44 +0000 (08:39 -0400)]
net: make LookupIP("1.2.3.4") behavior consistent
To date, the behavior has depended on whether we're using cgo and
in turn what the host resolver does. Most host resolvers will "resolve"
IP addresses, but the non-cgo pure Go path has not.
This CL makes resolution of IP addresses always work, even if we're not using cgo
and even if the host resolver does not "resolve" IP addresses.
Austin Clements [Tue, 23 Jun 2015 15:35:21 +0000 (11:35 -0400)]
runtime: fix heap bitmap repeating with large scalar tails
When heapBitsSetType repeats a source bitmap with a scalar tail
(typ.ptrdata < typ.size), it lays out the tail upon reaching the end
of the source bitmap by simply increasing the number of bits claimed
to be in the incoming bit buffer. This causes later iterations to read
the appropriate number of zeros out of the bit buffer before starting
on the next repeat of the source bitmap.
Currently, however, later iterations of the loop continue to read bits
from the source bitmap *regardless of the number of bits currently in
the bit buffer*. The bit buffer can only hold 32 or 64 bits, so if the
scalar tail is large and the padding bits exceed the size of the bit
buffer, the read from the source bitmap on the next iteration will
shift the incoming bits into oblivion when it attempts to put them in
the bit buffer. When the buffer does eventually shift down to where
these bits were supposed to be, it will contain zeros. As a result,
words that should be marked as pointers on later repetitions are
marked as scalars, so the garbage collector does not trace them. If
this is the only reference to an object, it will be incorrectly freed.
Fix this by adding logic to drain the bit buffer down if it is large
instead of reading more bits from the source bitmap.
Austin Clements [Tue, 23 Jun 2015 18:04:09 +0000 (14:04 -0400)]
runtime: document memory ordering for h_spans
h_spans can be accessed concurrently without synchronization from
other threads, which means it needs the appropriate memory barriers on
weakly ordered machines. It happens to already have the necessary
memory barriers because all accesses to h_spans are currently
protected by the heap lock and the unlocks happen in exactly the
places where release barriers are needed, but it's easy to imagine
that this could change in the future. Document the fact that we're
depending on the barrier implied by the unlock.
Rob Pike [Mon, 22 Jun 2015 22:40:40 +0000 (08:40 +1000)]
cmd/asm: fix shifts again, this time for sure
There are two conditions to worry about:
1) The shift count cannot be negative. Since the evaluator uses unsigned
arithmetic throughout, this means checking that the high bit of
the shift count is always off, which is done by converting to int64
and seeing if the result is negative.
2) For right shifts, the value cannot be negative. We don't want a
high bit in the value because right shifting a value depends on the
sign, and for clarity we always want unsigned shifts.
Next step is to build some testing infrastructure for the parser.
Rob Pike [Mon, 22 Jun 2015 20:23:04 +0000 (06:23 +1000)]
cmd/asm: fix handling of negative shifts.
The change that "fixed" LSH was incorrect, and the fix for RSH was poor.
Make both use a correct, simple test: if the 64-bit value as a signed
integer is negative, it's an error.
Really fixes #11278.
Change-Id: I72cca03d7ad0d64fd649fa33a9ead2f31bd2977b
Reviewed-on: https://go-review.googlesource.com/11325 Reviewed-by: Andrew Gerrand <adg@golang.org>
Rick Hudson [Mon, 22 Jun 2015 18:05:00 +0000 (14:05 -0400)]
runtime: remove race and increase precision in pointer validation.
This CL removes the single and racy use of mheap.arena_end outside
of the bookkeeping done in mHeap_init and mHeap_Alloc.
There should be no way for heapBitsForSpan to see a pointer to
an invalid span. This CL makes the check for this more precise by
checking that the pointer is between mheap_.arena_start and
mheap_.arena_used instead of mheap_.arena_end.
Austin Clements [Mon, 22 Jun 2015 15:18:23 +0000 (11:18 -0400)]
runtime: one more Map{Bits,Spans} before arena_used update
In order to avoid a race with a concurrent write barrier or garbage
collector thread, any update to arena_used must be preceded by mapping
the corresponding heap bitmap and spans array memory. Otherwise, the
concurrent access may observe that a pointer falls within the heap
arena, but then attempt to access unmapped memory to look up its span
or heap bits.
Commit d57c889 fixed all of the places where we updated arena_used
immediately before mapping the heap bitmap and spans, but it missed
the one place where we update arena_used and depend on later code to
update it again and map the bitmap and spans. This creates a window
where the original race can still happen. This commit fixes this by
mapping the heap bitmap and spans before this arena_used update as
well. This code path is only taken when expanding the heap reservation
on 32-bit over a hole in the address space, so these extra mmap calls
should have negligible impact.
Austin Clements [Fri, 19 Jun 2015 16:29:42 +0000 (12:29 -0400)]
runtime: document relaxed access to arena_used
The unsynchronized accesses to mheap_.arena_used in the concurrent
part of the garbage collector look like a problem waiting to happen.
In fact, they are safe, but the reason is somewhat subtle and
undocumented. This commit documents this reasoning.
Russ Cox [Fri, 19 Jun 2015 21:04:56 +0000 (17:04 -0400)]
net/url: add RawPath field, a hint at the desired encoding of Path
Historically we have declined to try to provide real support for URLs
that contain %2F in the path, but they seem to be popping up more
often, especially in (arguably ill-considered) REST APIs that shoehorn
entire paths into individual path elements.
The obvious thing to do is to introduce a URL.RawPath field that
records the original encoding of Path and then consult it during
URL.String and URL.RequestURI. The problem with the obvious thing
is that it breaks backward compatibility: if someone parses a URL
into u, modifies u.Path, and calls u.String, they expect the result
to use the modified u.Path and not the original raw encoding.
Split the difference by treating u.RawPath as a hint: the observation
is that there are many valid encodings of u.Path. If u.RawPath is one
of them, use it. Otherwise compute the encoding of u.Path as before.
If a client does not use RawPath, the only change will be that String
selects a different valid encoding sometimes (the original passed
to Parse).
This ensures that, for example, HTTP requests use the exact
encoding passed to http.Get (or http.NewRequest, etc).
Also add new URL.EscapedPath method for access to the actual
escaped path. Clients should use EscapedPath instead of
reading RawPath directly.
All the old workarounds remain valid.
Fixes #5777.
Might help #9859.
Fixes #7356.
Fixes #8767.
Fixes #8292.
Fixes #8450.
Fixes #4860.
Fixes #10887.
Fixes #3659.
Fixes #8248.
Fixes #6658.
Reduces need for #2782.
Russ Cox [Fri, 19 Jun 2015 18:43:09 +0000 (14:43 -0400)]
cmd/go: fix test for issue 8181
The test was translated from shell incorrectly,
and it depended on having hg installed, which
may not be the case.
Moved repo to GitHub, updated code, and fixed
go list ... command to be expected to succeed.
Fixes test for #8181.
Change-Id: I7f3e8fb20cd16cac5ed24de6fd952003bc5e08d4
Reviewed-on: https://go-review.googlesource.com/11301 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Jeff R. Allen [Thu, 18 Jun 2015 12:28:54 +0000 (14:28 +0200)]
net/textproto: skip zero-length keys
A header of ": value" results in an empty key. Do not add
it to the headers, because RFC7230 (section 3.2) says that
field-names are tokens, which are one or more characters.
Rob Pike [Sat, 20 Jun 2015 10:28:46 +0000 (20:28 +1000)]
cmd/doc: add test for constructor, fix build
Most important: skip test on darwin/arm64 for unclear reasons.
First cut at the test missed this feature of go doc: when asking for
the docs for a type, include any function that looks like it constructs
a that type as a return value.
Change-Id: I124e7695e5d365e2b12524b541a9a4e6e0300fbc
Reviewed-on: https://go-review.googlesource.com/11295 Reviewed-by: Rob Pike <r@golang.org>
Ian Lance Taylor [Fri, 19 Jun 2015 20:48:06 +0000 (13:48 -0700)]
syscall: skip non-root user namespace test if kernel forbids
Some Linux kernels apparently have a sysctl that prohibits
nonprivileged processes from creating user namespaces. If we see a
failure for that reason, skip the test.
Rob Pike [Fri, 19 Jun 2015 22:12:10 +0000 (08:12 +1000)]
cmd/doc: fix test on nacl
nacl is really giving a hard time. avoid all external dependencies in the test.
Worked with trybots, failed in the build. No explanation, but this should fix it.
TBR=rsc
Change-Id: Icb644286dbce88f17ee3d96ad90efba34a80a92d
Reviewed-on: https://go-review.googlesource.com/11291 Reviewed-by: Rob Pike <r@golang.org>