"archive/tar: reuse temporary buffer in writeHeader" introduced a
change which was supposed to help lower the number of allocations from
512 bytes for every call to writeHeader. This change broke the writing
of PAX headers.
writeHeader calls writePAXHeader and writePAXHeader calls writeHeader
again. writeHeader will end up writing the PAX header twice.
example broken header:
PaxHeaders.4007/NetLock_Arany_=Class_Gold=_Ftanstvny.crt0000000000000000000000000000007112301216634021512 xustar0000000000000000
PaxHeaders.4007/NetLock_Arany_=Class_Gold=_Ftanstvny.crt0000000000000000000000000000007112301216634021512 xustar0000000000000000
example correct header:
PaxHeaders.4290/NetLock_Arany_=Class_Gold=_Ftanstvny.crt0000000000000000000000000000007112301216634021516 xustar0000000000000000 0100644000000000000000000000270412301216634007250 0ustar0000000000000000
This commit adds a dedicated buffer for pax headers to the Writer
struct. This change increases the size of the struct by 512 bytes, but
allows tar/writer to avoid allocating 512 bytes for all written
headers and it avoids allocating 512 more bytes for pax headers.
As written, the ! applies before the &1.
This would crash writing out missing pcdata tables
if we ever used non-contiguous IDs in a function.
We don't, but fix anyway.
Keith Randall [Wed, 16 Jul 2014 21:16:19 +0000 (14:16 -0700)]
runtime: convert map implementation to Go.
It's a bit slower, but not painfully so. There is still room for
improvement (saving space so we can use nosplit, and removing the
requirement for hash/eq stubs).
Rob Pike [Wed, 16 Jul 2014 18:26:50 +0000 (18:26 +0000)]
cmd/ld: use count, not upper bound, in type of array
DWARF says only one is necessary.
The count is preferable because it admits 0-length arrays.
Update debug/dwarf to handle either form.
runtime: make a test more robust
The issue is discovered during testing of a change to runtime.
Even if it is unlikely to happen, the comment can safe an hour
next person who hits it.
runtime: refactor routines for stopping, running goroutine from m
This CL adds 'dropg', which is called to drop the association
between m and its current goroutine, and it makes schedule
handle locked goroutines correctly, instead of requiring all
callers of schedule to do that.
The effect is that if you want to take over an m for, say,
garbage collection work while still allowing the current g
to run on some other m, you can do an mcall to a function
that is:
// dissociate gp
dropg();
gp->status = Gwaiting; // for ready
// put gp on run queue for others to find
runtime·ready(gp);
/* ... do other work here ... */
// done with m, let it run goroutines again
schedule();
Before this CL, the dropg() body had to be written explicitly,
and the check for lockedg before schedule had to be
written explicitly too, both of which make the code a bit
more fragile than it needs to be.
Robert Griesemer [Mon, 14 Jul 2014 22:08:09 +0000 (15:08 -0700)]
spec: permit "for range x" (no index variables)
This is a fully backward-compatible language change.
There are not a lot of cases in the std library, but
there are some. Arguably this makes the syntax a bit
more regular - any trailing index variable that is _
can be left away, and there's some analogy to type
switches where the temporary can be left away.
Implementation-wise the change should be trivial as
it can be done completely syntactically. For instance,
the respective change in go/parser is a dozen lines
(see https://golang.org/cl/112970044 ).
Fixes #6102.
LGTM=iant, r, rsc
R=r, rsc, iant, ken
CC=golang-codereviews
https://golang.org/cl/104680043
Rob Pike [Fri, 11 Jul 2014 15:16:00 +0000 (15:16 +0000)]
ld: change DWARF output for structs
The debug/dwarf package cannot parse the format generated here,
but the format can be changed so it does.
After this edit, tweaking the expression defining the offset
of a struct field, the dwarf package can parse the tables (again?).
Pietro Gagliardi [Thu, 10 Jul 2014 19:44:40 +0000 (12:44 -0700)]
debug/elf: add (*File).DynamicSymbols, ErrNoSymbols, and tests for (*File).Symbols and (*File).DynamicSymbols, and formalize symbol order.
Added a complement to (*File).Symbols for the dynamic symbol table.
Would be useful, for instance, if seraching for certain shared objects
compatible with certain libraries (for instance, LADSPA requires an
exported symbol "ladspa_descriptor").
Added a variable ErrNoSymbols that canonicalizes a return from
(*File).Symbols and (*File).DyanmicSymbols if the file has no symbols.
Added tests for both (*File).Symbols and (*File).DynamicSymbols;
there was never a test for (*File).Symbols at all. A small C program using
libelf, included in the test data, was used to produce the golden
symbols to compare against.
As part of the requirements for testing, (*File).Symbols and (*File).DynamicSymbols now document the order in which the symbol tables are returned (in the order the symbols appear in the file).
runtime: grow heap by 64K instead of 128K
When we've switched to 8K pages,
heap started to grow by 128K instead of 64K,
because it was implicitly assuming that pages are 4K.
Fix that and make the code more robust.
debug/plan9obj, cmd/addr2line: on Plan 9 use a.out header
size instead of abusing text symbol
cmd/addr2line needs to know the virtual address of the start
of the text segment (load address plus header size). For
this, it used the text symbol added by the linker. This is
wrong on amd64. Header size is 40 bytes, not 32 like on 386
and arm. Function alignment is 16 bytes causing text to be
at 0x200030.
debug/plan9obj now exports both the load address and the
header size; cmd/addr2line uses this new information and
doesn't rely on text anymore.
NetlinkRIB is currently allocating a page sized slice of bytes in a
for loop and it's also calling Getpagesize() in the same for loop.
This CL changes NetlinkRIB to preallocate the page sized slice of
bytes before reaching the for loop. This reduces memory allocations
and lowers the number of calls to Getpagesize() to 1 per NetlinkRIB
call.
This CL reduces the allocated memory from 141.5 MB down to 52 MB in
a test.
LGTM=crawshaw, dave
R=dave, dsymonds, crawshaw
CC=bradfitz, dsymonds, golang-codereviews
https://golang.org/cl/110920043
After this change, the panic is replaced by a message:
$ go build -o out ...doesntexist
warning: "...doesntexist" matched no packages
no packages to build
The motivation to return 1 exit error code is to allow -o flag
to be used to guarantee that the output binary is written to
when exit status is 0. If someone uses an import path pattern
to specify a single package and suddenly that matches no packages,
it's better to return exit code 1 instead of silently doing nothing.
This is consistent with the case when -o flag is given and multiple
packages are matched.
It's also somewhat consistent with the current behavior with the
panic, except that gave return code 2. But it's similar in
that it's also non-zero (indicating failure).
I've changed the language to be similar to output of go test
when an import path pattern matches no packages (it also has a return status of
1):
$ go test ...doesntexist
warning: "...doesntexist" matched no packages
no packages to test
runtime: fix spurious "[string too long]" error
Maxstring is not updated in the new string routines,
this makes runtime think that long strings are bogus.
Fixes #8339.
Rob Pike [Mon, 7 Jul 2014 23:07:24 +0000 (16:07 -0700)]
cmd/ld: fix off-by-one in DWARF frame tables
The code generating the .debug_frame section emits pairs of "advance PC",
"set SP offset" pseudo-instructions. Before the fix, the PC advance comes
out before the SP setting, which means the emitted offset for a block is
actually the value at the end of the block, which is incorrect for the
block itself.
The easiest way to fix this problem is to emit the SP offset before the
PC advance.
One delicate point: the last instruction to come out is now an
"advance PC", which means that if there are padding intsructions after
the final RET, they will appear to have a non-zero offset. This is odd
but harmless because there is no legal way to have a PC in that range,
or to put it another way, if you get here the SP is certainly screwed up
so getting the wrong (virtual) frame pointer is the least of your worries.
David Crawshaw [Fri, 4 Jul 2014 01:04:48 +0000 (21:04 -0400)]
runtime/cgo: replace fprintf(stderr, ...) with fatalf(...) for linux/android
Both stdout and stderr are sent to /dev/null in android
apps. Introducing fatalf allows android to implement its
own copy that sends fatal errors to __android_log_print.
LGTM=minux, dave
R=minux, dave
CC=golang-codereviews
https://golang.org/cl/108400045
There are currently several schemes for acquiring a TLS
slot to save the g register. None of them appear to work
for android. The closest are linux and darwin.
Linux uses a linker TLS relocation. This is not supported
by the android linker.
Darwin uses a fixed offset, and calls pthread_key_create
until it gets the slot it wants. As the runtime loads
late in the android process lifecycle, after an
arbitrary number of other libraries, we cannot rely on
any particular slot being available.
So we call pthread_key_create, take the first slot we are
given, and put it in runtime.tlsg, which we turn into a
regular variable in cmd/ld.
runtime: delete unnecessary confusing code
The code in GC that handles gp->gobuf.ctxt is wrong,
because it does not mark the ctxt object itself,
if just queues the ctxt object for scanning.
So the ctxt object can be collected as garbage.
However, Gobuf.ctxt is void*, so it's always marked and
scanned through G.
crypto/x509: fix format strings in test
Currently it says:
--- PASS: TestDecrypt-2 (0.11s)
pem_decrypt_test.go:17: test 0. %!s(x509.PEMCipher=1)
--- PASS: TestEncrypt-2 (0.00s)
pem_decrypt_test.go:42: test 0. %!s(x509.PEMCipher=1)
runtime: make runtime·usleep and runtime·osyield callable from cgo callback
runtime·usleep and runtime·osyield fall back to calling an
assembly wrapper for the libc functions in the absence of a m,
so they can be called in cgo callback context.
A temporary 512 bytes buffer is allocated for every call to
readHeader. This buffer isn't returned to the caller and it could
be reused to lower the number of memory allocations.
This CL improves it by using a pool and zeroing out the buffer before
putting it back into the pool.
benchmark old ns/op new ns/op delta
BenchmarkListFiles100k 545249903538832687 -1.18%
benchmark old allocs new allocs delta
BenchmarkListFiles100k 21051672005692 -4.73%
benchmark old bytes new bytes delta
BenchmarkListFiles100k 10590347254831527 -48.22%
This improvement is very important if your code has to deal with a lot
of tarballs which contain a lot of files.
Adam Langley [Wed, 2 Jul 2014 22:28:57 +0000 (15:28 -0700)]
crypto/rsa: fix out-of-bound access with short session keys.
Thanks to Cedric Staub for noting that a short session key would lead
to an out-of-bounds access when conditionally copying the too short
buffer over the random session key.
2. Add /*c2go */ comment giving effect of #define.
This is necessary for function-like #defines and
non-enum-able #defined constants.
(Not all compilers handle negative or large enums.)
3. Add extra braces in struct initializer.
(c2go does not implement the full rules.)
This is enough to let c2go typecheck the source tree.
There may be more changes once it is doing
other semantic analyses.