compress/flate: prevent panic when reinitializing huffmanDecoder with bad input
The huffmanDecoder struct appears to be intented for reuse by calling init a
second time with a second sequence of code lengths. Unfortunately, it can
currently panic if the second sequence of code lengths has a minimum value
greater than 10 due to failure to reinitialize the links table.
This change prevents the panic by resetting the huffmanDecoder struct back to
the struct's zero value at the beginning of the init method if the
huffmanDecoder is being reused (determined by checking if min has been set to a
non-zero value).
runtime: account for all sys memory in MemStats
Currently lots of sys allocations are not accounted in any of XxxSys,
including GC bitmap, spans table, GC roots blocks, GC finalizer blocks,
iface table, netpoll descriptors and more. Up to ~20% can unaccounted.
This change introduces 2 new stats: GCSys and OtherSys for GC metadata
and all other misc allocations, respectively.
Also ensures that all XxxSys indeed sum up to Sys. All sys memory allocation
functions require the stat for accounting, so that it's impossible to miss something.
Also fix updating of mcache_sys/inuse, they were not updated after deallocation.
Marco Hennings [Fri, 6 Sep 2013 20:49:38 +0000 (16:49 -0400)]
archive/tar: fix a case where USTAR-split is not working correctly.
For some long filenames the USTAR-split code does not work
correctly. It is wrongly assumed that the path would not be too long,
but it is.
The user visible result was that a filename was split, but it still
caused an error.
The cause was a wrongly calculated nlen. In addition I noticed that
at this place it is also seems necessary to check if the prefix will
fit in the 155 chars available for the prefix.
cmd/gofmt: sort more, remove some duplicate imports
* Sort imports by import path, then import name, then comment. Currently, gofmt sorts only by import path.
* If two imports have the same import path and import name, and one of them has no comment, remove the import with no comment. (See the discussion at issue 4414.)
This message was helpful for pre-Go 1 users updating to Go 1.
That time is past. Now the message is confusing because it
depends on knowing what pre-Go 1 looked like.
api: update go1.1, except and next.txt with constant values
O_SYNC changes only on linux-arm (and linux-arm-cgo), but
changes to match O_SYNC on linux-{386,amd64} and what Linux
upstream now uses. See discussion and links on
https://golang.org/cl/13261050/
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/13575045
Lucio De Re [Fri, 6 Sep 2013 06:15:44 +0000 (16:15 +1000)]
cmd/dist: Plan 9 build needs an additional include path
cmd/cc: bv.c imports libc.h twice
When using the Plan 9 compiler, the invocation
#include <../ld/textflag.h>
works for the toolchain, but not for the MACH library.
Module cmd/cc/bv.c includes libc.h and "cc.h", which in
turn also includes libc.h. In the Plan 9 context, this
causes a number of duplicate definitions.
R=golang-dev, rsc, r
CC=golang-dev
https://golang.org/cl/13303047
reflect: do not cache trivial values in DeepEqual.
DeepEqual caches addresses of compared values
each time it visits addressable values. This is
more expensive than actually comparing them in
the common case of large slices of bytes or integers.
Also add a fast path for slices with identical
underlying array.
Rob Pike [Wed, 4 Sep 2013 22:23:11 +0000 (08:23 +1000)]
html/template: export the parse.Tree for the escaped template
The underlying parse tree is visible in text/template, so it should be visible here.
Done by copying the underlying *parse.Tree up to the top level of the struct, and then making sure it's kept up to date.
Fixes #6318.
Rob Pike [Wed, 4 Sep 2013 03:42:22 +0000 (13:42 +1000)]
text/template: allow eq to take more than two arguments
Based on an old suggestion by rsc, it compares the second
and following arguments to the first.
Unfortunately the code cannot be as pretty as rsc's original
because it doesn't require identical types.
Mikio Hara [Sat, 31 Aug 2013 01:28:49 +0000 (10:28 +0900)]
net: make resolveInternetAddr return a list of addresses
This CL makes resolveInternetAddr return a list of addresses that
contain a pair of different address family IP addresses if possible,
but doesn't contain any API behavioral changes yet. A simple IP
address selection mechanism for Resolve{TCP,UDP,IP}Addr and Dial API
still prefers IPv4.
This is in preparation for TCP connection setup with fast failover on
dual IP stack node as described in RFC 6555.
Tad Glines [Fri, 30 Aug 2013 16:27:33 +0000 (09:27 -0700)]
database/sql: add SetMaxOpenConns
Update #4805
Add the ability to set an open connection limit.
Fixed case where the Conn finalCloser was being called with db.mu locked.
Added separate benchmarks for each path for Exec and Query.
Replaced slice based idle pool with list based idle pool.
Adam Langley [Fri, 30 Aug 2013 14:14:45 +0000 (10:14 -0400)]
crypto/x509: expose arbitary X.509 extensions.
This change allows people who want to parse or set odd X.509 extensions
to do so without having to add support for them all to the package.
I tried to make it so that only a single member: Extensions would be
needed. However, that would mean detecting when the caller had altered
the contents of it so that parsing and marshaling a certificate
wouldn't ignore all changes to the other members. This ended up being
messy, thus the current design where there are two members: one for
reading and another for writing.
As crypto/x509 adds support for more extensions in the future, the raw
extensions will still be in Extensions for older code that expects it
there. Also, future extensions will be overridden by any raw extensions
added to ExtraExtensions by code that was written before support was
added.
R=golang-dev, r
CC=golang-dev, jpsugar
https://golang.org/cl/12056043
Dmitriy Vyukov [Fri, 30 Aug 2013 11:46:12 +0000 (15:46 +0400)]
libbio, all cmd: consistently use BGETC/BPUTC instead of Bgetc/Bputc
Also introduce BGET2/4, BPUT2/4 as they are widely used.
Slightly improve BGETC/BPUTC implementation.
This gives ~5% CPU time improvement on go install -a -p1 std.
Before:
real user sys
0m23.561s 0m16.625s 0m5.848s
0m23.766s 0m16.624s 0m5.846s
0m23.742s 0m16.621s 0m5.868s
after:
0m22.999s 0m15.841s 0m5.889s
0m22.845s 0m15.808s 0m5.850s
0m22.889s 0m15.832s 0m5.848s
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/12745047
Nigel Tao [Fri, 30 Aug 2013 06:03:16 +0000 (16:03 +1000)]
image/color/palette: move Plan9Palette and WebSafePalette out of the
image/color package into their own package. They require some non-
trivial init-time code (interface conversions, currently 40KiB of text)
that would otherwise burden any Go program that imported image/color.
««« original CL description
database/sql: add SetMaxOpenConns
Update #4805
Add the ability to set an open connection limit.
Fixed case where the Conn finalCloser was being called with db.mu locked.
Added seperate benchmarks for each path for Exec and Query.
Replaced slice based idle pool with list based idle pool.
Tad Glines [Fri, 30 Aug 2013 00:20:39 +0000 (17:20 -0700)]
database/sql: add SetMaxOpenConns
Update #4805
Add the ability to set an open connection limit.
Fixed case where the Conn finalCloser was being called with db.mu locked.
Added seperate benchmarks for each path for Exec and Query.
Replaced slice based idle pool with list based idle pool.
Mikio Hara [Fri, 30 Aug 2013 00:09:45 +0000 (09:09 +0900)]
net: add netaddr interface
This CL adds the netaddr interface that will carry a single network
endpoint address or a short list of IP addresses to dial helper
functions in the upcoming CLs.
This is in preparation for TCP connection setup with fast failover on
dual IP stack node as described in RFC 6555.
Brad Fitzpatrick [Thu, 29 Aug 2013 23:42:13 +0000 (16:42 -0700)]
misc/pprof: work with either LWP::UserAgent or curl
Use either LWP::UserAgent or curl to make HTTP requests so it
works on Windows (most Perl distros include LWP::UserAgent),
and also on OS X (whose Perl at least sometimes doesn't
include LWP::UserAgent).
Keith Randall [Thu, 29 Aug 2013 22:53:34 +0000 (15:53 -0700)]
runtime: jump to badmcall instead of calling it.
This replaces the mcall frame with the badmcall frame instead of
leaving the mcall frame on the stack and adding the badmcall frame.
Because mcall is no longer on the stack, traceback will now report what
called mcall, which is what we would like to see in this situation.
Brad Fitzpatrick [Thu, 29 Aug 2013 21:31:10 +0000 (14:31 -0700)]
regexp/syntax: optimize EmptyOpContext
Minor. Saw this in a profile at few percent of CPU and was
curious what it was. Improves overall regexp benchmarks
anywhere from 0 to 3%, but they're a pain to run. You need to
run them in isolation for long runs to get stable numbers.
benchmark old ns/op new ns/op delta
BenchmarkEmptyOpContext 537 473 -11.92%
Adam Langley [Thu, 29 Aug 2013 21:18:59 +0000 (17:18 -0400)]
crypto/tls: support AES-GCM.
AES-GCM is the only current TLS ciphersuite that doesn't have
cryptographic weaknesses (RC4), nor major construction issues (CBC mode
ciphers) and has some deployment (i.e. not-CCM).
Brad Fitzpatrick [Thu, 29 Aug 2013 20:55:30 +0000 (13:55 -0700)]
regexp: fix a benchmark case
I noticed that this one benchmark in particular was very
noisy. Looking into it, I saw that the table was wrong
and inconsistent with the lines above and below.
Carl Shapiro [Thu, 29 Aug 2013 20:52:38 +0000 (13:52 -0700)]
runtime: check bitmap word for allocated bit in markonly
When searching for an allocated bit, flushptrbuf would search
backward in the bitmap word containing the bit of pointer
being looked-up before searching the span. This extra check
was not replicated in markonly which, instead, after not
finding an allocated bit for a pointer would directly look in
the span.
Using statistics generated from godoc, before this change span
lookups were, on average, more common than word lookups. It
was common for markonly to consult spans for one third of its
pointer lookups. With this change in place, what were
previously span lookups are overwhelmingly become by the word
lookups making the total number of span lookups a relatively
small fraction of the whole.
This change also introduces some statistics gathering about
lookups guarded by the CollectStats enum.
Keith Randall [Thu, 29 Aug 2013 19:36:59 +0000 (12:36 -0700)]
cmd/cc,runtime: change preprocessor to expand macros inside of
#pragma textflag and #pragma dataflag directives.
Update dataflag directives to use symbols instead of integer constants.
Daniel Morsing [Thu, 29 Aug 2013 14:48:44 +0000 (16:48 +0200)]
cmd/gc: make method names for function scoped types unique
Types in function scope can have methods on them if they embed another type, but we didn't make the name unique, meaning that 2 identically named types in different functions would conflict with eachother.
Rémy Oudompheng [Thu, 29 Aug 2013 08:16:09 +0000 (10:16 +0200)]
cmd/gc: fix detection of initialization loop.
The compiler computes initialization order by finding
a spanning tree between a package's global variables.
But it does so by walking both variables and functions
and stops detecting cycles between variables when they
mix with a cycle of mutually recursive functions.
Andrew Gerrand [Thu, 29 Aug 2013 04:41:44 +0000 (14:41 +1000)]
os/exec: return idempotent Closer from StdinPipe
Before this fix, it was always an error to use the Close method on the
io.WriteCloser obtained from Cmd.StdinPipe, as it would race with the
Close performed by Cmd.Wait.
Brad Fitzpatrick [Wed, 28 Aug 2013 18:16:55 +0000 (11:16 -0700)]
time: add more docs on Sleep
Merge the comment from runtime/time.goc ("at least")
and also note that negative is okay and won't crash.
I see people going out of their way to avoid passing
a negative value to Sleep.
The previous wording, though accurate, was hard to parse.
In particular, it was tempting to interpret "the method"
as referring to "the function f" instead of "Do", and
required effort to find the correct antecedent for
"this receiver".
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/13307043