Concurrent use of tls.Config is allowed, and may lead to
KeyLogWriter being written to concurrently. Without a mutex
to protect it, corrupted output may occur. A mutex is added
for correctness.
The mutex is made global to save size of the config struct as
KeyLogWriter is rarely enabled.
GoToolPath requires a *testing.T to handle errors.
GoTool provides a variant that returns errors
for clients without a *testing.T,
such as that found in CL 27811.
flag: use strconv instead of fmt in values' String funcs
The existing implementation of flag values with fmt package uses
more memory and works slower than the implementation with strconv
package.
Change-Id: I9e749179f66d5c50cafe98186641bcdbc546d2db
Reviewed-on: https://go-review.googlesource.com/28914 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Edward Muller [Wed, 7 Sep 2016 18:39:31 +0000 (11:39 -0700)]
go/build: add help info for unset $GOPATH
We relay this info in a few places, in a few different ways, but not
consistently everywhere. This led one of our users to start googling
and not find https://golang.org/doc/code.html#Workspaces, of which `go
help gopath` is the most equivalent.
Change-Id: I28a94375739f3aa4f200e145293ca2a5f65101e1
Reviewed-on: https://go-review.googlesource.com/28690
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Robert Griesemer [Fri, 9 Sep 2016 18:29:33 +0000 (11:29 -0700)]
cmd/compile: use regular rather than indexed format string
This enables the format test to process this file (the format
test doesn't handle indexed formats, and this is the only place
in the compiler where they occur).
Change-Id: I99743f20c463f181a589b210365f70162227d4e0
Reviewed-on: https://go-review.googlesource.com/28932
Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
David Crawshaw [Fri, 9 Sep 2016 12:13:16 +0000 (08:13 -0400)]
cmd: use obj.GOOS, obj.GOARCH, etc
As cmd/internal/obj is coordinating the definition of GOOS, GOARCH,
etc across the compiler and linker, turn its functions into globals
and use them everywhere.
Change-Id: I5db5addda3c6b6435c37fd5581c7c3d9a561f492
Reviewed-on: https://go-review.googlesource.com/28854
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Jack Lindamood [Tue, 26 Jul 2016 21:20:36 +0000 (14:20 -0700)]
context: reduce memory usage of context tree
Modifies context package to use map[]struct{} rather than map[]bool,
since the map is intended as a set object. Also adds Benchmarks to
the context package switching between different types of root nodes
and a tree with different depths.
Included below are bytes deltas between the old and new code, using
these benchmarks.
Michael Munday [Thu, 8 Sep 2016 23:27:24 +0000 (19:27 -0400)]
runtime: fix SIGILL in checkvectorfacility on s390x
STFLE does not necessarily write to all the double-words that are
requested. It is therefore necessary to clear the target memory
before calling STFLE in order to ensure that the facility list does
not contain false positives.
Dave Day [Thu, 1 Sep 2016 03:13:37 +0000 (13:13 +1000)]
net/url: handle escaped paths in ResolveReference
Currently, path resolution is done using the un-escaped version of
paths. This means that path elements like one%2ftwo%2fthree are
handled incorrectly, and optional encodings (%2d vs. -) are dropped.
This function makes escaped handling consistent with Parse: provided
escapings are honoured, and RawPath is only set if necessary.
A helper method setPath is introduced to handle the correct setting of
Path and RawPath given the encoded path.
Fixes #16947
Change-Id: I40b1215e9066e88ec868b41635066eee220fde37
Reviewed-on: https://go-review.googlesource.com/28343
Run-TryBot: Dave Day <djd@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
encoding/hex: implement examples using all exported functions
Fixes #11254.
Updates #16360.
Implements examples using all exported functions.
This CL also updates Decode documentation to
state that only hexadecimal characters are accepted
in the source slice src, but also that the length
of src must be even.
Sina Siadat [Thu, 8 Sep 2016 07:09:12 +0000 (11:39 +0430)]
net/http/httputil: remove custom hop-by-hop headers from response in ReverseProxy
Hop-by-hop headers (explicitly mentioned in RFC 2616) were already
removed from the response. This removes the custom hop-by-hop
headers listed in the "Connection" header of the response.
Kevin Burke [Sun, 26 Jun 2016 16:47:43 +0000 (09:47 -0700)]
encoding/json: Use a lookup table for safe characters
The previous check for characters inside of a JSON string that needed
to be escaped performed seven different boolean comparisons before
determining that a ASCII character did not need to be escaped. Most
characters do not need to be escaped, so this check can be done in a
more performant way.
Use the same strategy as the unicode package for precomputing a range
of characters that need to be escaped, then do a single lookup into a
character array to determine whether the character needs escaping.
I also took the data set reported in #5683 (browser
telemetry data from Mozilla), added named structs for
the data set, and turned it into a proper benchmark:
https://github.com/kevinburke/jsonbench/blob/master/go/bench_test.go
The results from that test are similarly encouraging. On a 64-bit
Mac:
Martin Möhrmann [Tue, 6 Sep 2016 08:38:16 +0000 (10:38 +0200)]
runtime: remove maxstring
Before this CL the runtime prevented printing of overlong strings with the print
function when the length of the string was determined to be corrupted.
Corruption was checked by comparing the string size against the limit
which was stored in maxstring.
However maxstring was not updated everywhere were go strings were created
e.g. for string constants during compile time. Thereby the check for maximum
string length prevented the printing of some valid strings.
The protection maxstring provided did not warrant the bookkeeping
and global synchronization needed to keep maxstring updated to the
correct limit everywhere.
io/ioutil: return better error when TempDir called with non-extant dir
Fixes #14196
Change-Id: Ife7950289ac6adbcfc4d0f2fce31f20bc2657858
Reviewed-on: https://go-review.googlesource.com/28772 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Matthew Dempsky [Tue, 6 Sep 2016 21:26:56 +0000 (14:26 -0700)]
cmd/compile: remove unnecessary FuncType cloning
Since FuncTypes are represented as structs rather than linking the
parameter lists together, we no longer need to worry about duplicating
the parameter lists.
Sina Siadat [Sun, 4 Sep 2016 07:50:14 +0000 (12:20 +0430)]
net/http/httputil: copy header map if necessary in ReverseProxy
We were already making a copy of the map before removing
hop-by-hop headers. This commit does the same for proxied
headers mentioned in the "Connection" header.
A test is added to ensure request headers are not modified.
Joe Tsai [Fri, 2 Sep 2016 08:14:57 +0000 (01:14 -0700)]
net: remove parsing of negative decimals in IPv4 literal
https://golang.org/cl/27206 fixed the dtoi function such that
it now properly parses negative number. Ironically, this causes
several other functions that depended on dtoi to now (incorrectly)
parse negative numbers.
For example, ParseCIDR("-1.0.0.0/32") used to be rejected prior to the
above CL, but is now accepted even though it is an invalid CIDR notation.
This CL fixes that regression.
We fix this by removing the signed parsing logic entirely from dtoi.
It was introduced relatively recently in https://golang.org/cl/12447
to fix a bug where an invalid port was improperly being parsed as OK.
It seems to me that the fix in that CL to the port handling logic was
sufficient such that a change to dtoi was unnecessary.
Updates #16350
Change-Id: I414bb1aa27d0a226ebd4b05a09cb40d784691b43
Reviewed-on: https://go-review.googlesource.com/28414 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
Robert Griesemer [Fri, 2 Sep 2016 22:12:49 +0000 (15:12 -0700)]
cmd/compile: add format verification test
TestFormats finds potential (Printf, etc.) format strings.
If they are used in a call, the format verbs are verified
based on the matching argument type against a precomputed
table of valid formats (formatMapping, below). The table
can be used to automatically rewrite format strings with
the -u flag.
Run as: go test -run Formats [-u]
A formatMapping based on the existing formats is printed
when the test is run in verbose mode (-v flag). The table
needs to be updated whenever a new (type, format) combination
is found and the format verb is not 'v' (as in "%v").
Known bugs:
- indexed format strings ("%[2]s", etc.) are not suported
(the test will fail)
- format strings that are not simple string literals cannot
be updated automatically
(the test will fail with respective warnings)
This also fixes a bug, that caused a string of length 16 to use
two 8-byte comparisons instead of one 16-byte. And adds a test for
cases when partial_match fails.
Robert Griesemer [Wed, 7 Sep 2016 00:26:15 +0000 (17:26 -0700)]
go/constant: document that Value.String and ExactString return quoted strings
This has always been the case but it was not obvious from the documentation.
The reason for the quoting is that String() may return an abbreviated string,
starting with double-quote (") but ending in ... (w/o a quote). The missing
quote indicates the abbreviation (in contrast to a string ending in ...").
constant.StringVal can be used to obtain the unquoted string of a String Value.
Change-Id: Id0ba45b6ff62b3e024386ba8d907d6b3a4fcb6d7
Reviewed-on: https://go-review.googlesource.com/28576 Reviewed-by: Alan Donovan <adonovan@google.com>
Revert of cmd/compile: ignore contentEscapes for marking nodes as escaping
Reason for revert: broke the build due to cherrypick;
relies on an unsubmitted parent CL.
Original issue's description:
> cmd/compile: ignore contentEscapes for marking nodes as escaping
>
> We can still stack allocate and VarKill nodes which don't
> escape but their content does.
>
> Fixes #16996
>
> Change-Id: If8aa0fcf2c327b4cb880a3d5af8d213289e6f6bf
> Reviewed-on: https://go-review.googlesource.com/28575
> Run-TryBot: Keith Randall <khr@golang.org>
> TryBot-Result: Gobot Gobot <gobot@golang.org>
> Reviewed-by: David Chase <drchase@google.com>
>
syscall: make Getpagesize return page size from runtime
syscall.Getpagesize currently returns hard-coded page sizes on all
architectures (some of which are probably always wrong, and some of
which are definitely not always right). The runtime now has this
information, queried from the OS during runtime init, so make
syscall.Getpagesize return the page size that the runtime knows.
Now that the runtime fetches the true physical page size from the OS,
make the physical page size used by heap growth a variable instead of
a constant. This isn't used in any performance-critical paths, so it
shouldn't be an issue.
sys.PhysPageSize is also renamed to sys.DefaultPhysPageSize to make it
clear that it's not necessarily the true page size. There are no uses
of this constant any more, but we'll keep it around for now.
Currently the physical page size assumed by the runtime is hard-coded.
On Linux the runtime at least fetches the OS page size during init and
sanity checks against the hard-coded value, but they may still differ.
On other OSes we wouldn't even notice.
Add support on all OSes to fetch the actual OS physical page size
during runtime init and lift the sanity check of PhysPageSize from the
Linux init code to general malloc init. Currently this is the only use
of the retrieved page size, but we'll add more shortly.
Currently we assume the physical page size on ARM is 4kB. While this
is usually true, the architecture also supports 16kB and 64kB physical
pages, and Linux (and possibly other OSes) can be configured to use
these larger page sizes.
With Go 1.6, such a configuration could potentially run, but generally
resulted in memory corruption or random panics. With current master,
this configuration will cause the runtime to panic during init on
Linux when it checks the true physical page size (and will still cause
corruption or panics on other OSes).
However, the assumed physical page size only has to be a multiple of
the true physical page size, the scavenger can now deal with large
physical page sizes, and the rest of the runtime can deal with a
larger assumed physical page size than the true size. Hence, there's
little disadvantage to conservatively setting the assumed physical
page size to 64kB on ARM.
This may result in some extra memory use, since we can only return
memory at multiples of the assumed physical page size. However, it is
a simple change that should make Go run on systems configured for
larger page sizes. The following commits will make the runtime query
the actual physical page size from the OS, but this is a simple step
there.
Austin Clements [Sat, 28 May 2016 01:04:40 +0000 (21:04 -0400)]
runtime: bound scanobject to ~100 µs
Currently the time spent in scanobject is proportional to the size of
the object being scanned. Since scanobject is non-preemptible, large
objects can cause significant goroutine (and even whole application)
delays through several means:
1. If a GC assist picks up a large object, the allocating goroutine is
blocked for the whole scan, even if that scan well exceeds that
goroutine's debt.
2. Since the scheduler does not run on the P performing a large object
scan, goroutines in that P's run queue do not run unless they are
stolen by another P (which can take some time). If there are a few
large objects, all of the Ps may get tied up so the scheduler
doesn't run anywhere.
3. Even if a large object is scanned by a background worker and other
Ps are still running the scheduler, the large object scan doesn't
flush background credit until the whole scan is done. This can
easily cause all allocations to block in assists, waiting for
credit, causing an effective STW.
Fix this by splitting large objects into 128 KB "oblets" and scanning
at most one oblet at a time. Since we can scan 1–2 MB/ms, this equates
to bounding scanobject at roughly 100 µs. This improves assist
behavior both because assists can no longer get "unlucky" and be stuck
scanning a large object, and because it causes the background worker
to flush credit and unblock assists more frequently when scanning
large objects. This also improves GC parallelism if the heap consists
primarily of a small number of very large objects by letting multiple
workers scan a large objects in parallel.
Fixes #10345. Fixes #16293.
This substantially improves goroutine latency in the benchmark from
issue #16293, which exercises several forms of very large objects:
Commit 59877bf renamed bitMarked to bitScan, since the bitmap is no
longer used for marking. However, there were several other references
to this strewn about comments and in some other constant names. Fix
these up, too.
David Crawshaw [Fri, 26 Aug 2016 00:29:15 +0000 (20:29 -0400)]
cmd/compile: generate table of main symbol types
For each exported symbol in package main, add its name and type to
go.plugin.tabs symbol. This is used by the runtime when loading a
plugin to return a typed interface{} value.
Change-Id: I23c39583e57180acb8f7a74d218dae4368614f46
Reviewed-on: https://go-review.googlesource.com/27818
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is caused by sqrtsd preserving upper part of destination register.
Which introduces dependency on previous value of X0.
In 1.6 benchmark loop didn't use X0 immediately after call:
Change-Id: I98aa203176f8f2ca2fcca6e334a65bc60d6f824d
Reviewed-on: https://go-review.googlesource.com/28535 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Caio Marcelo de Oliveira Filho [Mon, 5 Sep 2016 18:14:55 +0000 (15:14 -0300)]
cmd/go: use httpGET helper in bug command
Use existing helper function instead of importing "net/http". This
allows the go_bootstrap build to not depend on "net/http" package.
See cmd/go/http.go for details.
The ctxt parameter is always set to 0 on entry into anylit so make this
parameter a literal constant, and where possibly remove ctxt as a parameter
where it is known to be a constant zero.
Passes toolstash -cmp.
This is a re-creation of CL 28221 by Dave Cheney.
That CL was graciously reverted in CL 28480
to make merging other CLs easier.