Russ Cox [Fri, 11 Dec 2015 03:28:15 +0000 (03:28 +0000)]
Revert "fmt: check newline in the end of input"
This change can break real code. There are other newline-related bugs in this code, and fixing them will also break real code. If we're going to break real code, let's fix all the bugs together and just break things once.
Russ Cox [Fri, 4 Dec 2015 17:43:25 +0000 (12:43 -0500)]
net/url: reject space in host; do not escape < > " in host
Host names in URLs must not use %-escaping for ASCII bytes, per RFC 3986.
url.Parse has historically allowed spaces and < > " in the URL host.
In Go 1.5, URL's String method started escaping those,
but then Parse would rejects the escaped form.
This CL is an attempt at some consistency between Parse and String
as far as the accepted host characters and the encoding of host characters,
so that if Parse succeeds, then Parse -> String -> Parse also succeeds.
Allowing space seems like a mistake, so reject that in Parse.
(Similarly, reject \t, \x01, and so on, all of which were being allowed.)
Allowing < > " doesn't seem awful, so continue to do that,
and go back to the Go 1.4 behavior of not escaping them in String.
Russ Cox [Sat, 5 Dec 2015 06:15:26 +0000 (01:15 -0500)]
net: do not unlink unix socket in UnixListener created from fd
Fixes #11826.
Change-Id: Id220dd558ca8d8d78c01975087122d27757deea0
Reviewed-on: https://go-review.googlesource.com/17458 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Rahul Chaudhry [Thu, 10 Dec 2015 23:06:42 +0000 (15:06 -0800)]
runtime: fix GODEBUG=schedtrace=X delay handling.
debug.schedtrace is an int32. Convert it to int64 before
multiplying with constant 1000000. Otherwise, schedtrace
values more than 2147 result in int32 overflow causing
incorrect delays between traces.
Change-Id: I064e8d7b432c1e892a705ee1f31a2e8cdd2c3ea3
Reviewed-on: https://go-review.googlesource.com/17712 Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Russ Cox [Mon, 7 Dec 2015 14:52:31 +0000 (09:52 -0500)]
math/big: fix misuse of Unicode
ˆ (U+02C6) is a circumflex accent, not an exponentiation operator.
In the rest of the source code for this package, exponentation is
written as **, so do the same here.
Change-Id: I107b85be242ab79d152eb8a6fcf3ca2b197d7658
Reviewed-on: https://go-review.googlesource.com/17671 Reviewed-by: Robert Griesemer <gri@golang.org>
Keith Randall [Thu, 3 Dec 2015 21:20:58 +0000 (13:20 -0800)]
cmd/compile: captureless closures are constants
In particular, we can initialize globals with them at link time instead
of generating code for them in an init() function. Less code, less
startup cost.
But the real reason for this change is binary size. This change reduces
the binary size of hello world by ~4%.
The culprit is fmt.ssFree, a global variable which is a sync.Pool of
scratch scan states. It is initalized with a captureless closure as the
pool's New action. That action in turn references all the scanf code.
If you never call any of the fmt.Scanf* routines, ssFree is never used.
But before this change, ssFree is still referenced by fmt's init
function. That keeps ssFree and all the code it references in the
binary. With this change, ssFree is initialized at link time. As a
result, fmt.init never mentions ssFree. If you don't call fmt.Scanf*,
ssFree is unreferenced and it and the scanf code are not included.
This change is an easy fix for what is generally a much harder problem,
the unnecessary initializing of unused globals (and retention of code
that they reference). Ideally we should have separate init code for
each global and only include that code if the corresponding global is
live. (We'd need to make sure that the initializing code has no side
effects, except on the global being initialized.) That is a much harder
change.
Brad Fitzpatrick [Thu, 10 Dec 2015 18:24:03 +0000 (10:24 -0800)]
net/http: make NewRequest with empty method mean GET
Until recently, we always permitted an empty string to NewRequest.
Keep that property, since it broke tests within in Google when trying
out Go 1.6, and probably would've broken others too.
Rob Pike [Wed, 9 Dec 2015 22:39:26 +0000 (14:39 -0800)]
cmd/doc: search the tree in breadth-first order
This is a simple change to the command that should resolve problems like finding
vendored packages before their non-vendored siblings. By searching in breadth-first
order, we find the matching package lowest in the hierarchy, which is more likely
to be correct than the deeper one, such as a vendored package, that will be found
in a depth-first scan.
This may be sufficient to resolve the issue, and has the merit that it is very easy
to explain. I will leave the issue open for now in case my intuition is wrong.
Mikio Hara [Fri, 4 Dec 2015 10:06:01 +0000 (19:06 +0900)]
net, internal/syscall/windows: fix interface and address identification on windows
The current implementation including Go 1.5 through 1.5.2 misuses
Windows API and mishandles the returned values from GetAdapterAddresses
on Windows. This change fixes various issues related to network facility
information by readjusting interface and interface address parsers.
Also fixes fragile screen scraping test cases in net_windows_test.go.
Additional information for reviewers:
It seems like almost all the issues above have the same root cause and
it is misunderstanding of Windows API. If my interpretation of the
information on MSDN is correctly, current implementation contains the
following bugs:
- SIO_GET_INTERFACE_LIST should not be used for IPv6. The behavior of
SIO_GET_INTERFACE_LIST is different on kernels and probably it doesn't
work correctly for IPv6 on old kernels such as Windows XP w/ SP2.
Unfortunately MSDN doesn't describe the detail of
SIO_GET_INTERFACE_LIST, but information on the net suggests so.
- Fetching IP_ADAPTER_ADDRESSES structures with fixed size area may not
work when using IPv6. IPv6 generates ton of interface addresses for
various addressing scopes. We need to adjust the area appropriately.
- PhysicalAddress field of IP_ADAPTER_ADDRESSES structure may have extra
space. We cannot ignore PhysicalAddressLength field of
IP_ADAPTER_ADDRESS structure.
- Flags field of IP_ADAPTER_ADDRESSES structure doesn't represent any of
administratively and operatinal statuses. It just represents settings
for windows network adapter.
- MTU field of IP_ADAPTER_ADDRESSES structure may have a uint32(-1) on
64-bit platform. We need to convert the value to interger
appropriately.
- IfType field of IP_ADAPTER_ADDRESSES structure is not a bit field.
Bitwire operation for the field is completely wrong.
- OperStatus field of IP_ADAPTER_ADDRESSES structure is not a bit field.
Bitwire operation for the field is completely wrong.
- IPv6IfIndex field of IP_ADAPTER_ADDRESSES structure is just a
substitute for IfIndex field. We cannot prefer IPv6IfIndex to IfIndex.
- Windows XP, 2003 server and below don't set OnLinkPrefixLength field
of IP_ADAPTER_UNICAST_ADDRESS structure. We cannot rely on the field
on old kernels. We can use FirstPrefix field of IP_ADAPTER_ADDRESSES
structure and IP_ADAPTER_PREFIX structure instead.
- Length field of IP_ADAPTER_{UNICAST,ANYCAST,MULTICAST}_ADDRESS
sturecures doesn't represent an address prefix length. It just
represents a socket address length.
Change-Id: Icabdaf7bd1d41360a981d2dad0b830b02b584528
Reviewed-on: https://go-review.googlesource.com/17412 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Brad Fitzpatrick [Wed, 9 Dec 2015 22:02:46 +0000 (22:02 +0000)]
net/http: run more tests in http2 mode
Failing ones are marked skipped.
Fixes #13543 (was just a test issue)
Updates #13555 (to be fixed later)
Updates #13556 (to be fixed later)
Updates #13557 (to be fixed later)
Fixes bug in golang.org/cl/17428 (http1 now uses HTTP status 431, not 413)
This code used to be necessary because of the error messages generated
by the YACC-based parser, but they're no longer relevant under the new
recursive descent parser:
- LBRACE no longer exists, so "{ or {" can never occur.
- The parser never generates error messages about "@" or "?" now
(except in import sections, where they're actually legitimate).
- The s/LLITERAL/litbuf/ substitution is handled in p.syntax_error.
Alex Brainman [Wed, 9 Dec 2015 01:53:57 +0000 (12:53 +1100)]
path/filepath: remove code working around Join bug
EvalSymlinks code assumes that Join has a bug
(see issue #11551 for details). But issue #11551 has
been fixed. Remove the workaround so it does not
confuses us when we read code next time.
Matt T. Proud [Tue, 8 Dec 2015 12:02:17 +0000 (13:02 +0100)]
encoding/pem: make TestFuzz testing/quick safe
This adapts pem.TestFuzz to sanitize the generated Block fields,
because the encoder and wireformat do not differentiate between nil
and empty slices and maps, while reflect.DeepEqual rightfully does.
In the commit mentioned below, we adapt quick.Value in
testing/quick to generate these value states, which had heretofore
been impossible with the standard library fuzz test facility.
This commit is a piecemeal extraction from ...
https://go-review.googlesource.com/#/c/16470
..., which rsc requested to be separated from the nil slice and map
generations.
Adam Langley [Fri, 4 Dec 2015 22:17:03 +0000 (14:17 -0800)]
crypto/elliptic: resample private keys if out of range.
The orders of the curves in crypto/elliptic are all very close to a
power of two. None the less, there is a tiny bias in the private key
selection.
This change makes the distribution uniform by resampling in the case
that a private key is >= to the order of the curve. (It also switches
from using BitSize to Params().N.BitLen() because, although they're the
same value here, the latter is technically the correct thing to do.)
The private key sampling and nonce sampling in crypto/ecdsa don't have
this issue.
Robert Griesemer [Fri, 4 Dec 2015 01:28:46 +0000 (17:28 -0800)]
go/parser, go/types: report invalid else branch in if statements
- Only accept valid if statement syntax in go/parser.
- Check AST again in go/types since it may have been modified and the
AST doesn't preclude other statements in the else branch of an if
statement.
- Removed a test from gofmt which verified that old-style if statements
permitting any statement in the else branch were correctly reformatted.
It's been years since we switched to the current syntax; no need to
support this anymore.
Aleksandr Demakin [Tue, 25 Aug 2015 16:54:57 +0000 (19:54 +0300)]
cmd/go: fix bad shared lib name with buildmode=shared
Use import paths of packages to build a shared lib name.
Use arguments for meta-packages 'std', 'cmd', and 'all'.
Fixes #12236
Change-Id: If274d63301686ef34e198287eb012f9062541ea0
Reviewed-on: https://go-review.googlesource.com/13921 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Rob Pike [Thu, 3 Dec 2015 21:06:04 +0000 (13:06 -0800)]
encoding/gob: document behavior of zero-valued arrays, slices, and maps
The documentation was inconsistent. It said zero values were not sent, but
that zero-valued elements of arrays and arrays were sent. But which rule
applies if the array is all zero elements, and is therefore itself a zero value?
The answer is: the array is transmitted. In principle the other choice could
be made, but there would be considerable expense and complexity required
to implement this behavior now, not to mention worries about changes of
behavior.
Therefore we just document the situation: Arrays, slices, and maps are
always encoded. It would perhaps be nice to have sorted this out earlier,
but it was a missed opportunity.
Russ Cox [Mon, 7 Dec 2015 19:22:08 +0000 (14:22 -0500)]
runtime: best-effort detection of concurrent misuse of maps
If reports like #13062 are really concurrent misuse of maps,
we can detect that, at least some of the time, with a cheap check.
There is an extra pair of memory writes for writing to a map,
but to the same cache line as h.count, which is often being modified anyway,
and there is an extra memory read for reading from a map,
but to the same cache line as h.count, which is always being read anyway.
So the check should be basically invisible and may help reduce the
number of "mysterious runtime crash due to map misuse" reports.
Didier Spezia [Sat, 29 Aug 2015 11:30:10 +0000 (11:30 +0000)]
cmd/compile/internal/gc: fix panic in Type Stringer
The following code:
func n() {(interface{int})}
generates:
3: interface contains embedded non-interface int
3: type %!v(PANIC=runtime error: invalid memory address or nil pointer dereference) is not an expression
It is because the corresponding symbol (Sym field in Type object)
is nil, resulting in a panic in typefmt.
Just skip the symbol if it is nil, so that the error message becomes:
3: interface contains embedded non-interface int
3: type interface { int } is not an expression
Matthew Dempsky [Mon, 7 Dec 2015 10:12:12 +0000 (02:12 -0800)]
cmd/compile: base mapaccess optimizations on algtype
algtype already controls the behavior of the normal map access code
paths, so it makes sense to base the decision on which optimized paths
are applicable on it too.
Enables use of optimized paths for key types like [8]byte and struct{s
string}.
Fixes #13271.
Change-Id: I48c52d97abaa7259ad5aba9641ea996a967cd359
Reviewed-on: https://go-review.googlesource.com/17464
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
Daniel Theophanes [Sun, 6 Dec 2015 15:51:30 +0000 (07:51 -0800)]
cmd/go: always show current value for GO15VENDOREXPERIMENT
Prior behavior would show empty string when unset. In go1.5 this
would result in "off". In go1.6 this will result in "on". This
change will make empty or "0" off and "1" on for go1.5 and go1.6.
Vendor tools can then rely on this value.
Change-Id: I7e145a32e813dfde02dc262a9186c7af28db7b92
Reviewed-on: https://go-review.googlesource.com/17487 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox [Sat, 5 Dec 2015 03:51:03 +0000 (22:51 -0500)]
misc/cgo/stdio: reenable tests
The build tags are necessary to keep "go build" in that directory
building only stdio.go, but we have to arrange for test/run.go to
treat them as satisfied.
Fixes #12625.
Change-Id: Iec0cb2fdc2c9b24a4e0530be25e940aa0cc9552e
Reviewed-on: https://go-review.googlesource.com/17454
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Russ Cox [Sat, 5 Dec 2015 04:16:50 +0000 (23:16 -0500)]
runtime: document that NumCPU does not change
Fixes #11609.
Change-Id: I3cf64164fde28ebf739706728b84d8ef5b6dc90e
Reviewed-on: https://go-review.googlesource.com/17456 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Mikio Hara [Fri, 4 Dec 2015 08:54:21 +0000 (17:54 +0900)]
net: adjust Lookup API test cases
This change makes existing Lookup API test cases conform to the new
return value form that all the Lookup APIs except LookupTXT must return
a single or multiple absolute domain names.
syscall: route_freebsd switch routing socket sysctl to use NET_RT_IFLISTL
Switch IfMsghdr and IfaMsghdr to their 'l' variants, make the IfData layout
to be based on FreeBSD-11.0 (freebsdVersion >= 1100011).
Using freebsdVersion, detect the appropriate layout at runtime and decode
routing socket messages into the new IfData layout.
Didier Spezia [Sat, 10 Oct 2015 10:57:35 +0000 (10:57 +0000)]
cmd/link: clean up dwarf.go
Try to remove the most visible artefacts resulting from the
C to Go translation. It includes:
- refactoring the find function to eliminate goto and variable declarations
- removing useless variables still having a _ = xxx
- decreasing the number of upfront variable declarations
Russ Cox [Fri, 4 Dec 2015 20:46:22 +0000 (15:46 -0500)]
misc/cgo/testcshared: use fd 100 instead of fd 10 for back-channel communication
There is a report that fd 10 is already in use when run on some OS X machines.
I don't see how, and I can't reproduce the problem on my own OS X machine,
but it's easy enough to fix.
Fixes #12161.
Change-Id: I73511bdd91258ecda181d60d2829add746d1198b
Reviewed-on: https://go-review.googlesource.com/17451 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Russ Cox [Fri, 4 Dec 2015 21:10:15 +0000 (16:10 -0500)]
runtime/cgo: assume Solaris thread stack is at least 1 MB
When run with "ulimit -s unlimited", the misc/cgo/test test binary
finds a stack size of 0x3000 returned by getcontext, causing the
runtime to try to stay within those bounds and then fault when
called back in the test after 64 kB has been used by C.
I suspect that Solaris is doing something clever like reporting the
current stack size and growing the stack as faults happen.
On all the other systems, getcontext reports the maximum stack size.
And when the ulimit is not unlimited, even Solaris reports the
maximum stack size.
Work around this by assuming that any stack on Solaris must be at least 1 MB.
Fixes #12210.
Change-Id: I0a6ed0afb8a8f50aa1b2486f32b4ae470ab47dbf
Reviewed-on: https://go-review.googlesource.com/17452 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Wed, 2 Dec 2015 23:41:44 +0000 (15:41 -0800)]
archive/tar: properly parse GNU base-256 encoding
Motivation:
* Previous implementation did not detect integer overflow when
parsing a base-256 encoded field.
* Previous implementation did not treat the integer as a two's
complement value as specified by GNU.
The relevant GNU specification says:
<<<
GNU format uses two's-complement base-256 notation to store values
that do not fit into standard ustar range.
>>>
Joe Tsai [Wed, 2 Dec 2015 23:48:06 +0000 (15:48 -0800)]
archive/tar: properly format GNU base-256 encoding
Motivation:
* Previous implementation silently failed when an integer overflow
occurred. Now, we report an ErrFieldTooLong.
* Previous implementation did not encode in two's complement format and was
unable to encode negative numbers.
The relevant GNU specification says:
<<<
GNU format uses two's-complement base-256 notation to store values
that do not fit into standard ustar range.
>>>
Brad Fitzpatrick [Fri, 4 Dec 2015 01:43:10 +0000 (17:43 -0800)]
net/http: convert TestSetsRemoteAddr to use clientServerTest
This is an example of converting an old HTTP/1-only test to test
against both HTTP/1 and HTTP/2.
Please send more of these!
Also, for comparing the http.Transport's responses between HTTP/1 and
HTTP/2, see clientserver_test.go's h12Compare type and tests using
h12Compare. Sometimes that's the more appropriate option.
Russ Cox [Wed, 25 Nov 2015 16:34:41 +0000 (11:34 -0500)]
encoding/json: streamline, unexport valid Number checking
Followup to CL 12250.
For #10281.
Change-Id: If25d9cac92f10327bb355f2d11b00c625b464661
Reviewed-on: https://go-review.googlesource.com/17199 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Russ Cox [Thu, 3 Dec 2015 21:00:04 +0000 (16:00 -0500)]
net/mail: do not parse RFC 2047 tokens in quoted strings
RFC 2047 tokens like =?utf-8?B?whatever?= can only appear
unquoted, but this code was trying to decode them even when
they came out of quoted strings. Quoted strings must be left alone.
Sokolov Yura [Mon, 12 Oct 2015 13:06:38 +0000 (16:06 +0300)]
sort: improve average quicksort performance
- change way of protection from O(N^2) on duplicate values.
Previous algorithm does additional comparisons and swaps
on every split pass.
Changed algorithm does one ordinal quicksort split pass,
and if distribution is skewed, then additional pass to
separate pivot's duplicates.
Changed algorithm could be slower on very ununique slice,
but it is still protected from O(N^2).
- increase small slice size and do simple shell sort pass
to amortize worst case on small slices.
Small slice has higher probability to have skewed
distribution, so lets sort it with simpler algorithm.
Mikio Hara [Fri, 4 Dec 2015 01:58:13 +0000 (10:58 +0900)]
net/mail: gofmt
Change-Id: Ic704a2614e310bc7aa3bdee89a020c27f4292efa
Reviewed-on: https://go-review.googlesource.com/17410 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Matthew Dempsky [Thu, 3 Dec 2015 20:14:07 +0000 (12:14 -0800)]
cmd/compile: fix live variable reuse in orderstmt
The call "poptemp(t, order)" at line 906 should match up with the
assignment "t := marktemp(order)" at line 770, so use a new temporary
variable for stripping the ODCL nodes from a "case x := <-ch" node's
Ninit list.
Fixes #13469.
Passes toolstash/buildall.
Change-Id: Ia7eabd40c79cfdcb83df00b6fbd0954e0c44c5c7
Reviewed-on: https://go-review.googlesource.com/17393 Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Mohit Agarwal [Thu, 3 Dec 2015 19:11:44 +0000 (00:41 +0530)]
misc/cgo/testsanitizers: check linux major/minor versions
Fix a typo in de5b386; using `$ver` to determine linux major/minor
versions would produce those for clang, use `$linuxver` instead.
Updates #12898.
Change-Id: I2c8e84ad02749fceaa958afd65e558bb0b08dddb
Reviewed-on: https://go-review.googlesource.com/17323 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ian Lance Taylor [Thu, 3 Dec 2015 21:20:49 +0000 (13:20 -0800)]
doc: add note about cgo pointer passing rules to go1.6.txt
Change-Id: I988d1b230ce516bf2997ec0932a854323b2bab7c
Reviewed-on: https://go-review.googlesource.com/17395 Reviewed-by: Ian Lance Taylor <iant@golang.org>
This adds support for compressed ELF sections. This compression is
treated as a framing issue and hence the package APIs all
transparently decompress compressed sections. This requires some
subtlety for (*Section).Open, which returns an io.ReadSeeker: since
the decompressed data comes from an io.Reader, this commit introduces
a Reader-to-ReadSeeker adapter that is efficient for common uses of
Seek and does what it can otherwise.
Austin Clements [Wed, 2 Dec 2015 03:55:28 +0000 (22:55 -0500)]
dwbug/elf: support old-style compressed DWARF
GCC and LLVM support zlib-compressing DWARF debug sections (and
there's some evidence that this may be happening by default in some
circumstances now).
Add support for reading compressed DWARF sections. Since ELF
relocations apply to the decompressed data, decompression is done
before applying relocations. Since relcations are applied by
debug/elf, decompression must also be handled there.
Note that this is different from compressed ELF sections, which is a
more general mechanism used by very recent versions of GCC.