David du Colombier [Wed, 16 Aug 2017 18:30:26 +0000 (20:30 +0200)]
cmd/link: skip TestFieldOverlap on Plan 9
TestSizes has been added in CL 50170. This test is
failing on Plan 9 because executables don't have
a DWARF symbol table.
Fixes #21480.
Change-Id: I51079abdc18ad944617bdbcfe2dad970a0cea0f2
Reviewed-on: https://go-review.googlesource.com/56210
Run-TryBot: David du Colombier <0intro@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Bryan C. Mills [Mon, 7 Aug 2017 23:30:03 +0000 (19:30 -0400)]
runtime/cgo: defeat inlining in x_cgo_yield
We use a call to strncpy to work around a TSAN bug (wherein TSAN only
delivers asynchronous signals when the thread receiving the signal
calls a libc function). Unfortunately, GCC 7 inlines the call,
avoiding the TSAN libc trap entirely.
Per Ian's suggestion, use global variables as strncpy arguments: that
way, the compiler can't make any assumptions about the concrete values
and can't inline the call away.
fixes #21196
Change-Id: Ie95f1feaf9af1a8056f924f49c29cfc8515385d7
Reviewed-on: https://go-review.googlesource.com/55872 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: Id9a54f0c81b1b5db5b5efb12a2ad6509c4ab42b3
Reviewed-on: https://go-review.googlesource.com/55770 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Keith Randall [Thu, 20 Jul 2017 02:52:29 +0000 (19:52 -0700)]
cmd/link: fix bad dwarf for sudog<T>
The DWARF entries for type-specific sudog entries used the
channel value type instead of a pointer-to-value type for the elem field.
Fixes #21094
R=go1.10
Change-Id: I3f63a5664f42b571f729931309f2c9f6f38ab031
Reviewed-on: https://go-review.googlesource.com/50170 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ilya Tocar [Wed, 9 Aug 2017 19:50:58 +0000 (14:50 -0500)]
cmd/compile/internal/ssa: use sse to zero on amd64
Use 16-byte stores instead of 8-byte stores to zero small blocks.
Also switch to duffzero for 65+ bytes only, because for each
duffzero call we also save/restore BP, so call requires 4 instructions
and replacing it with 4 sse stores doesn't cause code-bloat.
Also switch duffzero to use leaq, instead of addq to avoid clobbering flags.
griesemer [Tue, 15 Aug 2017 12:44:26 +0000 (14:44 +0200)]
go/importer: make source importer more tolerant in presence of errors
If the source importer only encounters "soft" type checking errors
it can safely return the type-checked package because it will be
completely set up. This makes the source importer slightly more
robust in the presence of errors.
Fixes #20855.
Change-Id: I5af9ccdb30eee6bca7a0fab872f6057bde521bf3
Reviewed-on: https://go-review.googlesource.com/55730 Reviewed-by: Alan Donovan <adonovan@google.com>
Daniel Martí [Thu, 10 Aug 2017 02:17:59 +0000 (11:17 +0900)]
reflect: remove useless parameter from newName
pkgPath always received the empty string. Worse yet, it panicked if it
received anything else. This has been the case ever since newName was
introduced in early 2016.
Change-Id: I5f164305bd30c34455ef35e776c7616f303b37e4
Reviewed-on: https://go-review.googlesource.com/54331
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Crawshaw <crawshaw@golang.org>
Change-Id: I087980d30308353c4a450636122f7e87c8310090
Reviewed-on: https://go-review.googlesource.com/56090 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Brian Kessler [Sat, 29 Jul 2017 05:29:30 +0000 (22:29 -0700)]
math/big: recognize z.Mul(x, x) as squaring of x
updates #13745
Multiprecision squaring can be done in a straightforward manner
with about half the multiplications of a basic multiplication
due to the symmetry of the operands. This change implements
basic squaring for nat types and uses it for Int multiplication
when the same variable is supplied to both arguments of
z.Mul(x, x). This has some overhead to allocate a temporary
variable to hold the cross products, shift them to double and
add them to the diagonal terms. There is a speed benefit in
the intermediate range when the overhead is neglible and the
asymptotic performance of karatsuba multiplication has not been
reached.
Show a speed up of 10-25% in the range where basicSqr is optimal,
improved single word squaring and no significant difference when
the fallback to standard multiplication is used.
Change-Id: Iae2c82ca91cf890823f91e5c83bbe9a2c534b72b
Reviewed-on: https://go-review.googlesource.com/53638 Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Alberto Donizetti [Tue, 15 Aug 2017 21:49:12 +0000 (23:49 +0200)]
cmd/go: make go tool suggest 'go doc cmd/<command>'
$ gotip tool -h says:
For more about each tool command, see 'go tool command -h'.
But it's better to suggest
go doc cmd/<command>
Fixes #18313
Change-Id: I0a36d585906a5e1879e5b7927d1b6173e97cb500
Reviewed-on: https://go-review.googlesource.com/55990 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Brian Kessler [Fri, 21 Jul 2017 06:31:51 +0000 (23:31 -0700)]
math/big: speed up GCD x, y calculation
The current implementation of the extended Euclidean GCD algorithm
calculates both cosequences x and y inside the division loop. This
is unneccessary since the second Bezout coefficient can be obtained
at the end of calculation via a multiplication, subtraction and a
division. In case only one coefficient is needed, e.g. ModInverse
this calculation can be skipped entirely. This is a standard
optimization, see e.g.
"Handbook of Elliptic and Hyperelliptic Curve Cryptography"
Cohen et al pp 191
Available at:
http://cs.ucsb.edu/~koc/ccs130h/2013/EllipticHyperelliptic-CohenFrey.pdf
Updates #15833
Change-Id: I1e0d2e63567cfed97fd955048fe6373d36f22757
Reviewed-on: https://go-review.googlesource.com/50530 Reviewed-by: Robert Griesemer <gri@golang.org>
Brian Kessler [Thu, 13 Jul 2017 05:02:39 +0000 (22:02 -0700)]
math: eliminate overflow in Pow(x,y) for large y
The current implementation uses a shift and add
loop to compute the product of x's exponent xe and
the integer part of y (yi) for yi up to 1<<63.
Since xe is an 11-bit exponent, this product can be
up to 74-bits and overflow both 32 and 64-bit int.
This change checks whether the accumulated exponent
will fit in the 11-bit float exponent of the output
and breaks out of the loop early if overflow is detected.
The current handling of yi >= 1<<63 uses Exp(y * Log(x))
which incorrectly returns Nan for x<0. In addition,
for y this large, Exp(y * Log(x)) can be enumerated
to only overflow except when x == -1 since the
boundary cases computed exactly:
Alex Brainman [Mon, 5 Jun 2017 04:29:14 +0000 (14:29 +1000)]
cmd/link: delete addpesection
Change-Id: Iee9db172d28d4d372fa617907078a494e764bf12
Reviewed-on: https://go-review.googlesource.com/55260 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Mon, 5 Jun 2017 04:21:35 +0000 (14:21 +1000)]
cmd/link: use peSection everywhere
Change-Id: I4d4e8452b9b9e628f3ea8b2b727ad63ec2a1dd31
Reviewed-on: https://go-review.googlesource.com/55259 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Fri, 2 Jun 2017 06:53:22 +0000 (16:53 +1000)]
cmd/link: add peSection
Change-Id: Id3aeeaeaacf5f079fb2ddad579f2f209b7fc0e06
Reviewed-on: https://go-review.googlesource.com/55258 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Alex Brainman [Thu, 1 Jun 2017 07:33:15 +0000 (17:33 +1000)]
cmd/link: introduce and use peFile and peStringTable
Change-Id: Icd13b32d35cde474c9292227471f916a64af88eb
Reviewed-on: https://go-review.googlesource.com/55257 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Tue, 15 Aug 2017 01:38:46 +0000 (18:38 -0700)]
archive/tar: make Writer error handling consistent
The Writer logic was not consistent about when an IO error would
persist across multiple calls on Writer's methods.
Thus, to make the error handling more consistent we always check
the persistent state of the error prior to every exported method
call, and return an error if set. Otherwise, it is the responsibility
of every exported method to persist any fatal errors that may occur.
As a simplification, we can remove the close field since that
information can be represented by simply storing ErrWriteAfterClose
in the err field.
Change-Id: I8746ca36b3739803e0373253450db69b3bd12f38
Reviewed-on: https://go-review.googlesource.com/55590
Run-TryBot: Joe Tsai <joetsai@digital-static.net>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Mon, 14 Aug 2017 23:44:15 +0000 (16:44 -0700)]
archive/tar: add support for long binary strings in GNU format
The GNU tar format defines the following type flags:
TypeGNULongName = 'L' // Next file has a long name
TypeGNULongLink = 'K' // Next file symlinks to a file w/ a long name
Anytime a string exceeds the field dedicated to store it, the GNU format
permits a fake "file" to be prepended where that file entry has a Typeflag
of 'L' or 'K' and the contents of the file is a NUL-terminated string.
Contrary to previous TODO comments,
the GNU format supports arbitrary strings (without NUL) rather UTF-8 strings.
The manual says the following:
<<<
The name, linkname, magic, uname, and gname are
null-terminated character strings
>>>
<<<
All characters in header blocks are represented
by using 8-bit characters in the local variant of ASCII.
>>>
From this description, we gather the following:
* We must forbid NULs in any GNU strings
* Any 8-bit value (other than NUL) is permitted
Since the modern world has moved to UTF-8, it is really difficult to
determine what a "local variant of ASCII" means. For this reason,
we treat strings as just an arbitrary binary string (without NUL)
and leave it to the user to determine the encoding of this string.
(Practically, it seems that UTF-8 is the typical encoding used
in GNU archives seen in the wild).
The implementation of GNU tar seems to confirm this interpretation
of the manual where it permits any arbitrary binary string to exist
within these fields so long as they do not contain the NUL character.
The fact that we permit arbitrary binary in GNU strings goes
hand-in-hand with the fact that GNU also permits a "base-256" encoding
of numeric fields, which is effectively two-complement binary.
Change-Id: Ic037ec6bed306d07d1312f0058594bd9b64d9880
Reviewed-on: https://go-review.googlesource.com/55573 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In the existing implementation, if pattern is an empty string,
program calls a panic with the message which is a concatenation of
"http: invalid pattern " and pattern.
In this case, pattern is an empty, so the commit removes
this concatenation and the trailing space.
Fixes: #21102
Change-Id: I49f58b52d835311a6ac642de871eb15646e48a54
Reviewed-on: https://go-review.googlesource.com/50350 Reviewed-by: Dmitri Shuralyov <shurcool@gmail.com> Reviewed-by: Tom Bergan <tombergan@google.com>
Run-TryBot: Tom Bergan <tombergan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ian Lance Taylor [Tue, 8 Aug 2017 22:50:43 +0000 (15:50 -0700)]
testing: don't fail all tests after racy test failure
The code was adding race.Errors to t.raceErrors before checking
Failed, but Failed was using t.raceErrors+race.Errors. We don't want
to change Failed, since that would affect tests themselves, so modify
the harness to not unnecessarily change t.raceErrors.
Updates #19851
Fixes #21338
Change-Id: I7bfdf281f90e045146c92444f1370d55c45221d4
Reviewed-on: https://go-review.googlesource.com/54050 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/compile: insert ' ' and \n at beginning of walkprint
Rather than emitting spaces and newlines for println
as we walk the expression, construct it all up front.
This enables further optimizations.
This requires using printstring instead of print in
the implementation of printsp and printnl,
on pain of infinite recursion.
That's ok; it's more efficient anyway, and just as simple.
While we're here, do it for other print routines as well.
Hana Kim [Tue, 15 Aug 2017 19:22:28 +0000 (15:22 -0400)]
.gitignore: include only Go project artifiacts
This effectively reverts
https://golang.org/cl/53770
and adds a guide on what not to add in this file.
Update #21458
Change-Id: I7c740d492b70628b5d9f9e1622014995a3f6f8ec
Reviewed-on: https://go-review.googlesource.com/55871 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Daniel Morsing [Wed, 2 Aug 2017 18:01:17 +0000 (19:01 +0100)]
runtime: move selectdone into g
Writing to selectdone on the stack of another goroutine meant a
pretty subtle dance between the select code and the stack copying
code. Instead move the selectdone variable into the g struct.
Keith Randall [Tue, 30 May 2017 19:59:25 +0000 (12:59 -0700)]
cmd/compile: set itab function pointers at compile time
I noticed that we don't set an itab's function pointers at compile
time. Instead, we currently do it at executable startup.
Set the function pointers at compile time instead. This shortens
startup time. It has no effect on normal binary size. Object files
will have more relocations, but that isn't a big deal.
For PIE there are additional pointers that will need to be adjusted at
load time. There are already other pointers in an itab that need to be
adjusted, so the cache line will already be paged in. There might be
some binary size overhead to mark these pointers. The "go test -c
-buildmode=pie net/http" binary is 0.18% bigger.
Update #20505
Change-Id: I267c82489915b509ff66e512fc7319b2dd79b8f7
Reviewed-on: https://go-review.googlesource.com/44341
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Martin Möhrmann <moehrmann@google.com>
Andreas Auernhammer [Fri, 26 May 2017 09:33:49 +0000 (11:33 +0200)]
crypto/tls: don't check whether an ec point is on a curve twice
The processClientKeyExchange and processServerKeyExchange functions unmarshal an
encoded EC point and explicitly check whether the point is on the curve. The explicit
check can be omitted because elliptic.Unmarshal fails if the point is not on the curve
and the returned error would always be the same.
Fixes #20496
Change-Id: I5231a655eace79acee2737dd036a0c255ed42dbb
Reviewed-on: https://go-review.googlesource.com/44311 Reviewed-by: Adam Langley <agl@golang.org> Reviewed-by: Avelino <t@avelino.xxx>
Run-TryBot: Adam Langley <agl@golang.org>
Austin Clements [Tue, 15 Aug 2017 15:32:02 +0000 (11:32 -0400)]
runtime: fix getclosureptr doc
Change-Id: I1b42fca2107b06e6fc95728f7bf3d08d005c4cb4
Reviewed-on: https://go-review.googlesource.com/55810 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Tobias Klauser [Mon, 14 Aug 2017 07:47:58 +0000 (09:47 +0200)]
syscall: really use utimensat for UtimesNano on Solaris
golang.org/cl/55130 added utimensat for Solaris but didn't use it in
UtimesNano (despite indicating otherwise in the commit message). Fix
this by also using utimensat for UtimesNano on Solaris.
Because all versions of Solaris suppported by Go support utimensat,
there is no need for the fallback logic and utimensat can be called
unconditionally.
This issue was pointed out by Shawn Walker-Salas.
Updates #16480
Change-Id: I114338113a6da3cfcb8bca950674bdc8f5a7a9e5
Reviewed-on: https://go-review.googlesource.com/55141
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Hana (Hyang-Ah) Kim [Fri, 4 Aug 2017 21:38:46 +0000 (17:38 -0400)]
.gitignore: exclude /.idea/
Some IDEs (e.g. intellij IDE based ones) create the .idea folder
to store project specific settings. This is irrelevant to Go project
that does not assume any specific IDEs, but interferes with git.
Change-Id: I0c93d9a3f7edff095fbe0c7a53b06c92b391c970
Reviewed-on: https://go-review.googlesource.com/53770 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Agniva De Sarker [Sun, 13 Aug 2017 18:15:04 +0000 (23:45 +0530)]
encoding/hex: save allocation in DecodeString()
The destination slice does not need to be created at all. The source
slice itself can be used as the destination because the decode loop
increments by one and then the 'seen' byte is not used anymore. Therefore
the decoded byte can be stored in that index of the source slice itself.
This trick cannot be applied to EncodeString() because in that case,
the destination slice is large than the source. And for a single byte
in the source slice, two bytes in the destination slice is written.
func BenchmarkDecodeString(b *testing.B) {
for i := 0; i < b.N; i++ {
DecodeString("0123456789abcdef")
}
}
name old time/op new time/op delta
DecodeString 71.0ns ± 6% 58.0ns ± 0% -18.28% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
DecodeString 16.0B ± 0% 8.0B ± 0% -50.00% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
DecodeString 1.00 ± 0% 1.00 ± 0% ~ (all equal)
Change-Id: Id98db4e712444557a804155457a4dd8d1b8b416d
Reviewed-on: https://go-review.googlesource.com/55611 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Tobias Klauser [Tue, 15 Aug 2017 08:27:42 +0000 (10:27 +0200)]
syscall: fix definition of _AT_FDCWD on dragonfly
CL golang.org/cl/55130 messed up the definition of _AT_FDCWD on
dragonfly.
This fixes the following test failure on dragonfly/amd64:
--- FAIL: TestPackageMainTestImportsArchiveNotBinary (0.00s)
go_test.go:192: chtimes ./testdata/src/main_test/m.go: bad file descriptor
Change-Id: I4c96983769e6b02d714859dc838875c3c0f1be50
Reviewed-on: https://go-review.googlesource.com/55690
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
The stxr/stxrw/stxrb/stxrh instructions belong to STLXR-like instructions
set and they require special handling. The current code has no special
handling for those instructions.
The fix adds the special handling for those instructions.
Improve static branch prediction in arm64 wrapper prologue
by making the unusual case branch forwards. (Most other
architectures implement this optimization.)
Additionally, replace a CMP+BNE pair with a CBNZ
to save one instruction.
Hiroshi Ioka [Tue, 15 Aug 2017 11:53:49 +0000 (20:53 +0900)]
misc/cgo/testshared: call flag.Parse in TestMain
Otherwise, some test flags don't work.
Change-Id: Iacf3930d0eec28e4d690cd382adbb2ecf866a0e2
Reviewed-on: https://go-review.googlesource.com/55615 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
philhofer [Wed, 9 Aug 2017 05:01:26 +0000 (05:01 +0000)]
cmd/compile: add support for arm64 bit-test instructions
Add support for generating TBZ/TBNZ instructions.
The bit-test-and-branch pattern shows up in a number of
important places, including the runtime (gc bitmaps).
Before this change, there were 3 TB[N]?Z instructions in the Go tool,
all of which were in hand-written assembly. After this change, there
are 285. Also, the go1 benchmark binary gets about 4.5kB smaller.
Martin Möhrmann [Sun, 13 Aug 2017 18:03:02 +0000 (20:03 +0200)]
cmd/compile: generate makechan calls with int arguments
Where possible generate calls to runtime makechan with int arguments
during compile time instead of makechan with int64 arguments.
This eliminates converting arguments for calls to makechan with
int64 arguments for platforms where int64 values do not fit into
arguments of type int.
A similar optimization for makeslice was introduced in CL
golang.org/cl/27851.
Joe Tsai [Tue, 15 Aug 2017 00:16:52 +0000 (17:16 -0700)]
archive/tar: re-implement USTAR path splitting
The logic for USTAR was disabled because a previous implementation of
Writer had a wrong understanding of the differences between USTAR and GNU,
causing the prefix field is incorrectly be populated in GNU files.
Now that this issue has been fixed, we can re-enable the logic for USTAR
path splitting, which allows Writer to use the USTAR for a wider range
of possible inputs.
Updates #9683
Updates #12594
Updates #17630
Change-Id: I9fe34e5df63f99c6dd56fee3a7e7e4d6ec3995c9
Reviewed-on: https://go-review.googlesource.com/55574
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Tue, 15 Aug 2017 03:11:02 +0000 (20:11 -0700)]
archive/tar: centralize errors in common.go
Move all sentinel errors to common.go since some of them are
returned by both the reader and writer and remove errInvalidHeader
since it not used.
Also, consistently use the "tar: " prefix for errors.
Change-Id: I0afb185bbf3db80dfd9595321603924454a4c2f9
Reviewed-on: https://go-review.googlesource.com/55650 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Martin Möhrmann [Sat, 5 Aug 2017 11:44:25 +0000 (13:44 +0200)]
runtime: simplify memory capacity check in growslice
Instead of comparing if the number of elements will
not fit into memory check if the memory size of the
slices backing memory is higher then the memory limit.
Note that due to integer division from capmem > _MaxMem
it does not follow that uintptr(newcap) > maxSliceCap(et.size).
Consolidated runtime GrowSlice benchmarks by using sub-benchmarks and
added more struct sizes to show performance improvement when division
is avoided for element sizes larger than 32 bytes.
Russ Cox [Fri, 11 Aug 2017 20:22:10 +0000 (16:22 -0400)]
cmd/link: implement R_X86_64_PC64 relocations
Change-Id: I1d7bd5cff7350a4e0f78b8efc8406e79c74732d1
Reviewed-on: https://go-review.googlesource.com/55370 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Mon, 14 Aug 2017 22:39:19 +0000 (15:39 -0700)]
archive/tar: add support for atime and ctime to Writer
Both the GNU and PAX formats support atime and ctime fields.
The implementation is trivial now that we have:
* support for formatting PAX records for timestamps
* dedicated methods that only handle one format (e.g., GNU)
Fixes #17876
Change-Id: I0c604fce14a47d722098afc966399cca2037395d
Reviewed-on: https://go-review.googlesource.com/55570
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Mon, 14 Aug 2017 22:57:46 +0000 (15:57 -0700)]
archive/tar: reject bad key-value pairs for PAX records
We forbid empty keys or keys with '=' because it leads to ambiguous parsing.
Relevent PAX specification:
<<<
A keyword shall not include an <equals-sign>.
>>>
Also, we forbid the writer from encoding records with an empty value.
While, this is a valid record syntactically, the semantics of an empty
value is that previous records with that key should be deleted.
Since we have no support (and probably never will) for global PAX records,
deletion is a non-sensible operation.
<<<
If the <value> field is zero length,
it shall delete any header block field,
previously entered extended header value,
or global extended header value of the same name.
>>>
Fixes #20698
Fixes #15567
Change-Id: Ia29c5c6ef2e36cd9e6d7f6cff10e92b96a62f0d1
Reviewed-on: https://go-review.googlesource.com/55571 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Mon, 14 Aug 2017 22:24:31 +0000 (15:24 -0700)]
archive/tar: support PAX subsecond resolution times
Add support for PAX subsecond resolution times. Since the parser
supports negative timestamps, the formatter also handles negative
timestamps.
The relevant PAX specification is:
<<<
Portable file timestamps cannot be negative. If pax encounters a
file with a negative timestamp in copy or write mode, it can reject
the file, substitute a non-negative timestamp, or generate a
non-portable timestamp with a leading '-'.
>>>
<<<
All of these time records shall be formatted as a decimal
representation of the time in seconds since the Epoch.
If a <period> ( '.' ) decimal point character is present,
the digits to the right of the point shall represent the units of
a subsecond timing granularity, where the first digit is tenths of
a second and each subsequent digit is a tenth of the previous digit.
>>>
Fixes #11171
Change-Id: Ied108f3d2654390bc1b0ddd66a4081c2b83e490b
Reviewed-on: https://go-review.googlesource.com/55552
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall [Wed, 31 May 2017 18:57:07 +0000 (11:57 -0700)]
runtime: initialize itab.hash always
We weren't initializing this field for dynamically-generated itabs.
Turns out it doesn't matter, as any time we use this field we also
generate a static itab for the interface type / concrete type pair.
But we should initialize it anyway, just to be safe.
Performance on the benchmarks in CL 44339:
benchmark old ns/op new ns/op delta
BenchmarkItabFew-12 1040585 26466 -97.46%
BenchmarkItabAll-12 2288734994287696 -98.13%
Keith Randall [Wed, 31 May 2017 15:45:10 +0000 (08:45 -0700)]
runtime: new itab lookup table
Keep itabs in a growable hash table.
Use a simple open-addressable hash table, quadratic probing, power
of two sized.
Synchronization gets a bit more tricky. The common read path now
has two atomic reads, one to get the table pointer and one to read
the entry out of the table.
I set the max load factor to 75%, kind of arbitrarily. There's a
space-speed tradeoff here, and I'm not sure where we should land.
Because we use open addressing the itab.link field is no longer needed.
I'll remove it in a separate CL.
Joe Tsai [Mon, 14 Aug 2017 22:33:46 +0000 (15:33 -0700)]
archive/tar: properly handle header-only "files" in Writer
Certain special type-flags, specifically 1, 2, 3, 4, 5, 6,
do not have a data section. Thus, regardless of what the size field
says, we should not attempt to write any data for these special types.
The relevant PAX and USTAR specification says:
<<<
If the typeflag field is set to specify a file to be of type 1 (a link)
or 2 (a symbolic link), the size field shall be specified as zero.
If the typeflag field is set to specify a file of type 5 (directory),
the size field shall be interpreted as described under the definition
of that record type. No data logical records are stored for types 1, 2, or 5.
If the typeflag field is set to 3 (character special file),
4 (block special file), or 6 (FIFO), the meaning of the size field is
unspecified by this volume of POSIX.1-2008, and no data logical records shall
be stored on the medium.
Additionally, for type 6, the size field shall be ignored when reading.
If the typeflag field is set to any other value, the number of logical
records written following the header shall be (size+511)/512, ignoring
any fraction in the result of the division.
>>>
Fixes #15565
Change-Id: Id11886b723b3b13deb15221dca51c25cd778a6b5
Reviewed-on: https://go-review.googlesource.com/55553 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Joe Tsai [Mon, 14 Aug 2017 23:14:08 +0000 (16:14 -0700)]
archive/tar: roundtrip reading device numbers
Both GNU and BSD tar do not care if the devmajor and devminor values are
set on entries (like regular files) that aren't character or block devices.
While this is non-sensible, it is more consistent with the Writer to actually
read these fields always. In a vast majority of the cases these will still
be zero. In the rare situation where someone actually cares about these,
at least information was not silently lost.
Change-Id: I6e4ba01cd897a1b13c28b1837e102a4fdeb420ba
Reviewed-on: https://go-review.googlesource.com/55572 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Yuval Pavel Zholkover [Mon, 14 Aug 2017 16:05:56 +0000 (19:05 +0300)]
syscall: add missing int flag argument to utimensat
Fixes #21437
Change-Id: I55fbf5114ae1bb7f4aa1a20450e8d5309756cd5b
Reviewed-on: https://go-review.googlesource.com/55430
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Hiroshi Ioka [Sat, 12 Aug 2017 13:52:17 +0000 (22:52 +0900)]
cmd/link: improve error message
ld.SymKind and objabi.RelocType have string representations,
which is human friendly. Prefer to use it.
Change-Id: I458ee0ca5866be0db8462c36cd053561a8206c95
Reviewed-on: https://go-review.googlesource.com/55253 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The preceding cleanup made it clear that two cases
(have golden data, unreachable key) are handled identically.
Simplify the control flow to reflect that.
Simplifies the code and generates shorter machine code.
Hiroshi Ioka [Mon, 14 Aug 2017 14:02:56 +0000 (23:02 +0900)]
cmd/go: correctly quote environment variables in -x output
This fixes the -x output so that when it reports environment variables they
are correctly quoted for later execution by the shell.
Also fix -x output to use the right path to the pack tool, and note when
we are touching a file.
Fixes #21427
Change-Id: I323ef4edf9905b08bc26944b94183d8da2fa9675
Reviewed-on: https://go-review.googlesource.com/55350 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Hiroshi Ioka [Mon, 14 Aug 2017 08:26:22 +0000 (17:26 +0900)]
debug/macho: make Type implements fmt.(Go)Stringer interfaces
Fixes #21436
Change-Id: I56f43e2852696c28edbcc772a54125a9a9c32497
Reviewed-on: https://go-review.googlesource.com/55262 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Hiroshi Ioka [Sat, 12 Aug 2017 15:19:51 +0000 (00:19 +0900)]
cmd/link: correct Mach-O file flag
Only set MH_NOUNDEFS if there are no undefined symbols.
Doesn't seem to matter, but may as well do it right.
Change-Id: I6c472e000578346c28cf0e10f24f870e3a0de628
Reviewed-on: https://go-review.googlesource.com/55310 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Keith Randall [Mon, 14 Aug 2017 22:07:57 +0000 (15:07 -0700)]
cmd/link,compile: Provide size for func types
They are currently not given a size, which makes the DWARF reader
very confused. Particularly things like [4]func() get a size of -4, not 32.
Fixes #21097
Change-Id: I01e754134d82fbbe6567e3c7847a4843792a3776
Reviewed-on: https://go-review.googlesource.com/55551 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It shrinks evacuate's machine code by ~25% and its stack size by ~15%.
It also eliminates a critical branch.
Whether an entry should go to x or y is designed to be unpredictable.
As a result, half of the branch predictions for useX were wrong.
Mispredicting that branch can easily incur an expensive cache miss.
Switching to an xy array allows elimination of that branch,
which in turn reduces cache misses.
Make the calculation of k and v a bit lazier.
None of the following code cares about indirect-vs-direct k,
and it happens on all code paths, so check t.indirectkey earlier.
Simplifies the code and reduces both machine code and stack size.
Joe Tsai [Mon, 14 Aug 2017 21:59:18 +0000 (14:59 -0700)]
archive/tar: remove writeHeader and writePAXHeaderLegacy
Previous CLs (CL/54970, CL55231, and CL/55237) re-implemented tar.Writer
entirely using specialized methods (writeUSTARHeader, writePAXHeader,
and writeGNUHeader) allowing tar.Writer to entirely side-step the broken
and buggy logic in writeHeader.
Since writeHeader and writePAXHeaderLegacy is now dead-code,
we can delete them.
One minor change is that we call Writer.Flush at the start of WriteHeader.
This used to be performed by writeHeader, but doing so in WriteHeader
ensures each of the specialized methods can benefit from its effect.
Fixes #17665
Fixes #12594
Change-Id: Iff2ef8e7310d40ac5484d2f8852fc5df25201426
Reviewed-on: https://go-review.googlesource.com/55550 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Joe Tsai [Mon, 14 Aug 2017 07:03:26 +0000 (00:03 -0700)]
archive/tar: implement specialized logic for GNU format
Rather than going through writeHeader, which attempts to handle all formats,
implement writeGNUHeader, which only has an understanding of the GNU format.
Currently, the implementation is nearly identical to writeUSTARHeader, except:
* formatNumeric is used instead of formatOctal
* the GNU magic value is used
This is kept as a separate method since it makes more logical sense
when we add support for sparse files, long filenames, and atime/ctime fields,
which do not affect USTAR.
Updates #12594
Change-Id: I76efc0b39dc649efc22646dfc9867a7c165f34a8
Reviewed-on: https://go-review.googlesource.com/55237
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Martin Möhrmann <moehrmann@google.com>
Ilya Tocar [Tue, 1 Aug 2017 20:12:54 +0000 (15:12 -0500)]
encoding/base32: improve performance in common case
Unroll loop to improve perfromance back to 1.8 level.
name old time/op new time/op delta
EncodeToString-6 63.0µs ± 3% 51.7µs ± 2% -17.94% (p=0.000 n=10+10)
name old speed new speed delta
EncodeToString-6 130MB/s ± 3% 159MB/s ± 2% +21.83% (p=0.000 n=10+10)
Hiroshi Ioka [Mon, 14 Aug 2017 10:39:17 +0000 (19:39 +0900)]
debug/macho: add relocation types
Fixes #21435
Change-Id: I5f8d93a45b84a871ceea881ecb1a38a37e96006c
Reviewed-on: https://go-review.googlesource.com/55263 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Hiroshi Ioka [Sat, 5 Aug 2017 09:25:26 +0000 (18:25 +0900)]
cmd/go, cmd/link: enable buildmode=pie on darwin/amd64
Change some configurations to enable the feature. Also add the test.
This CL doesn't include internal linking support which is tentatively
disabled due to #18968. We could do that another day.
Fixes #21220
Change-Id: I601d2d78446d36332acc70be0d5b9461ac635208
Reviewed-on: https://go-review.googlesource.com/54790 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Justin Nuß [Wed, 2 Aug 2017 17:46:41 +0000 (19:46 +0200)]
encoding/csv: preserve \r\n in quoted fields
The parser mistakenly assumed it could always fold \r\n into \n, which
is not true since a \r\n inside a quoted fields has no special meaning
and should be kept as is.
Fix this by not folding \r\n to \n inside quotes fields.
Fixes #21201
Change-Id: Ifebc302e49cf63e0a027ee90f088dbc050a2b7a6
Reviewed-on: https://go-review.googlesource.com/52810 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
griesemer [Fri, 11 Aug 2017 14:51:40 +0000 (16:51 +0200)]
spec: better comment in example for type definition
The old comment for the example
type PtrMutex *Mutex
talked about the method set of the base type of PtrMutex.
It's more direct and clearer to talk about the underlying
type of PtrMutex for this specific example.
Also removed link inside pre-formatted region of text.
Fixes #20900.
Change-Id: Ie37340e53670e34ebe13e780ba8ccb1bba67795c
Reviewed-on: https://go-review.googlesource.com/55070 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Carlos Eduardo Seo [Mon, 10 Jul 2017 18:28:27 +0000 (15:28 -0300)]
runtime, internal/cpu: CPU capabilities detection for ppc64x
This change replaces the current runtime capabilities check for ppc64x with the
new internal/cpu package. It also adds support for the new POWER9 ISA and
capabilities.
Updates #15403
Change-Id: I5b64a79e782f8da3603e5529600434f602986292
Reviewed-on: https://go-review.googlesource.com/53830 Reviewed-by: Martin Möhrmann <moehrmann@google.com>
Joe Tsai [Fri, 11 Aug 2017 18:47:16 +0000 (11:47 -0700)]
archive/tar: adjust bytediff to print full context
Since test files don't exceed 10KiB, print the full context of the diff,
including bytes that are equal.
Also, fix the labels for got and want; they were backwards before.
Change-Id: Ibac022e5f988d26812c3f75b643cae8b95603fc9
Reviewed-on: https://go-review.googlesource.com/55151 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Joe Tsai [Sat, 12 Aug 2017 01:58:58 +0000 (18:58 -0700)]
archive/tar: implement specialized logic for PAX format
Rather than going through writeHeader, which attempts to handle all formats,
implement writePAXHeader, which only has an understanding of the PAX format.
In PAX, the USTAR header is filled out in a best-effort manner.
Thus, we change logic of formatString and formatOctal to try their best to
output something (possibly truncated) in the event of an error.
The new implementation of PAX headers causes several tests to fail.
An investigation into the new output reveals that the new behavior is correct,
while the tests had actually locked in incorrect behavior before.
A dump of the differences is listed below (-before, +after):
<< writer-big.tar >>
This change is due to fact that we changed the Header.Devminor to force the
tar.Writer to choose the GNU format over the PAX one.
The ability to control the output is an open issue (see #18710).
- 00000150 00 30 30 30 30 30 30 30 00 00 00 00 00 00 00 00 |.0000000........|
+ 00000150 00 ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 |................|
The previous logic tried to use the specified timestmap in the PAX headers file,
but this is problematic as this timestamp can overflow, defeating the point
of using PAX, which is intended to extend tar.
The new logic uses the zero timestamp similar to what GNU and BSD tar do.
- 00000080 30 30 30 30 32 33 32 00 31 32 33 33 32 37 37 30 |0000232.12332770|
+ 00000080 30 30 30 30 32 35 36 00 30 30 30 30 30 30 30 30 |0000256.00000000|
The previous logic populated the devminor and devmajor fields.
The new logic leaves them zeroed just like what GNU and BSD tar do.
- 00000140 00 00 00 00 00 00 00 00 00 30 30 30 30 30 30 30 |.........0000000|
- 00000150 00 30 30 30 30 30 30 30 00 00 00 00 00 00 00 00 |.0000000........|
+ 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
+ 00000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
The previous logic uses PAX headers, but fails to add a record for the size.
The new logic does properly add a record for the size.
- 00000290 31 36 67 69 67 2e 74 78 74 0a 00 00 00 00 00 00 |16gig.txt.......|
- 000002a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
+ 00000290 31 36 67 69 67 2e 74 78 74 0a 32 30 20 73 69 7a |16gig.txt.20 siz|
+ 000002a0 65 3d 31 37 31 37 39 38 36 39 31 38 34 0a 00 00 |e=17179869184...|
The previous logic encoded the size as a base-256 field,
which is only valid in GNU, but the previous PAX headers implies this should
be a PAX file. This result in a strange hybrid that is neither GNU nor PAX.
The new logic uses PAX headers to store the size.
- 00000470 37 35 30 00 30 30 30 31 37 35 30 00 80 00 00 00 |750.0001750.....|
- 00000480 00 00 00 04 00 00 00 00 31 32 33 33 32 37 37 30 |........12332770|
+ 00000470 37 35 30 00 30 30 30 31 37 35 30 00 30 30 30 30 |750.0001750.0000|
+ 00000480 30 30 30 30 30 30 30 00 31 32 33 33 32 37 37 30 |0000000.12332770|
<< ustar.issue12594.tar >>
The previous logic used the specified timestamp for the PAX headers file.
The new logic just uses the zero timestmap.
- 00000080 30 30 30 30 32 33 31 00 31 32 31 30 34 34 30 32 |0000231.12104402|
+ 00000080 30 30 30 30 32 33 31 00 30 30 30 30 30 30 30 30 |0000231.00000000|
The previous logic populated the devminor and devmajor fields.
The new logic leaves them zeroed just like what GNU and BSD tar do.
- 00000140 00 00 00 00 00 00 00 00 00 30 30 30 30 30 30 30 |.........0000000|
- 00000150 00 30 30 30 30 30 30 30 00 00 00 00 00 00 00 00 |.0000000........|
+ 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
+ 00000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
Change-Id: I33419eb1124951968e9d5a10d50027e03133c811
Reviewed-on: https://go-review.googlesource.com/55231 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>