The comment at https://github.com/golang/go/issues/27171#issuecomment-470134255 describes some problems with the current approach.
One problem is Docker and other repos can have tags that are not valid semver tags but that still match a glob pattern of v[0-9]*.[0-9]*.[0-9]* which are found by 'git describe' but then rejected by cmd/go, and hence those repos currently can end up with v0.0.0 pseudoversions instead of finding a proper semver tag to use as input to building a pseudoversion (when then causes problems when the v0.0.0 pseudoversion is fed into MVS). An example problematic tag is a date-based tag such as 'v18.06.16', which matches the glob pattern, but is not a valid semver tag (due to the leading 0 in '06').
Issues #31673, #31287, and #27171 also describe problems where the '--first-parent' argument to 'git describe' cause the current approach to miss relevant semver tags that were created on a separate branch and then subsequently merged to master.
In #27171, Bryan described the base tag that is supposed to be used for pseudoversions as:
"It is intended to be the semantically-latest tag that appears on any commit that is a (transitive) parent of the commit with the given hash, regardless of branches. (The pseudo-version is supposed to sort after every version — tagged or otherwise — that came before it, but before the next tag that a human might plausibly want to apply to the branch.)"
This CL solves the glob problem and tags-on-other-branches problem more directly than the current approach: this CL gets the full list of tags that have been merged into the specific revision of interest, and then sorts and filters the results in cmd/go to select the semantically-latest valid semver tag.
Fixes #31673
Fixes #31287
Updates #27171
Change-Id: I7c3e6b46b2b21dd60562cf2893b6bd2afaae61d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/174061
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
David Chase [Thu, 1 Nov 2018 21:31:20 +0000 (17:31 -0400)]
runtime: look for idle p to run current goroutine when switching to GC or traceReader
This repairs one of the several causes of pauses uncovered
by a GC microbenchmark. A pause can occur when a goroutine's
quantum expires "at the same time" a GC is needed. The
current M switches to running a GC worker, which means that
the amount of available work has expanded by one. The GC
worker, however, does not call ready, and does not itself
conditionally wake a P (a "normal" thread would do this).
This is also true if M switches to a traceReader.
This is problem 4 in this list:
https://github.com/golang/go/issues/27732#issuecomment-423301252
Russ Cox [Fri, 25 Jan 2019 17:01:53 +0000 (12:01 -0500)]
index/suffixarray: add 32-bit implementation
The original index/suffixarray used 32-bit ints on 64-bit machines,
because that's what 'int' meant in Go at the time. When we changed
the meaning of int, that doubled the space overhead of suffix arrays
for all uses, even though the vast majority of them describe less
than 2 GB of text.
The space overhead of a suffix array compared to the text is not
insignificant: there's a big difference for many uses between 4X and 8X.
This CL adjusts names in qsufsort.go so that a global search and
replace s/32/64/g produces a working 64-bit implementation,
and then it modifies suffixarray.go to choose between the 32-bit
and 64-bit implementation as appropriate depending on the input size.
The 64-bit implementation is generated by 'go generate'.
This CL also restructures the benchmarks, to test different
input sizes, different input texts, and 32-bit vs 64-bit.
The serialized form uses varint-encoded numbers and is unchanged,
so on-disk suffix arrays written by older versions of Go will be
readable by this version, and vice versa.
The 32-bit version runs a up to 17% faster than the 64-bit version
on real inputs, but more importantly it uses 50% less memory.
I have a followup CL that also implements a faster algorithm
on top of these improvements, but these are a good first step.
Than McIntosh [Wed, 1 May 2019 14:40:30 +0000 (10:40 -0400)]
go/internal/gccgoimporter: skip new test with aliases with old gccgo
Add the issue31540 test to the list of tests that needs to be skipped
with old copies of gccgo. Along the way, add an explicit field to the
importer test struct that can be used to tag the test (as opposed to
having special cases by name in the test routine), so as to make it
easier to remember to tag testcases correctly.
Fixes #31764.
Change-Id: Ib9d98fea2df8ce0b51e5a886fb2c4acd6db490ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/174738 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Michael Anthony Knyszek [Tue, 5 Feb 2019 00:06:55 +0000 (00:06 +0000)]
runtime: change the span allocation policy to first-fit
This change modifies the treap implementation to be address-ordered
instead of size-ordered, and further augments it so it may be used for
allocation. It then modifies the find method to implement a first-fit
allocation policy.
This change to the treap implementation consequently makes it so that
spans are scavenged in highest-address-first order without any
additional changes to the scavenging code. Because the treap itself is
now address ordered, and the scavenging code iterates over it in
reverse, the highest address is now chosen instead of the largest span.
This change also renames the now wrongly-named "scavengeLargest" method
on mheap to just "scavengeLocked" and also fixes up logic in that method
which made assumptions about size.
Michael Vogt [Wed, 1 May 2019 07:52:44 +0000 (07:52 +0000)]
net: set DNSError.IsTemporary from addrinfoErrno errors
Today it is not possible (AFAICT) to detect if a DNSError if of type EAI_AGAIN, i.e. if it is something temporary that should be retried. This information is available inside addrinfoErrno but when the DNSError is created this information is lost.
This PR fixes this so that the addinfoErrno.Temporary information is added to DNSError as well. With that a user who gets a DNSError can check now is its a temporary error (for errors that resulted from a addrinfoErrno this is EAI_AGAIN).
Michael Fraenkel [Sat, 6 Oct 2018 17:32:19 +0000 (13:32 -0400)]
net/http: make Transport.MaxConnsPerHost work for HTTP/2
Treat HTTP/2 connections as an ongoing persistent connection. When we
are told there is no cached connections, cleanup the associated
connection and host connection count.
Nir Soffer [Tue, 7 Nov 2017 18:08:52 +0000 (20:08 +0200)]
net/http: add Transport.ReadBufferSize and WriteBufferSize
Previously transport was using the hardcoded bufio.defaultBufSize
(4096), limiting throughput and increasing cpu usage when uploading or
downloading large files.
Add options to allow users to configure the buffer sizes as needed.
I tested the maximum benefit of this change by uploading data from
/dev/zero to a server discarding the bytes. Here is an example upload
using the default buffer size:
$ time ./upload 10 https://localhost:8000/
Uploaded 10.00g in 25.13 seconds (407.49m/s)
real 0m25.135s
user 0m5.167s
sys 0m11.643s
With this change, using 128k buffer size:
$ time ./upload 10 https://localhost:8000/
Uploaded 10.00g in 7.93 seconds (1291.51m/s)
real 0m7.935s
user 0m4.517s
sys 0m2.603s
In real world usage the difference will be smaller, depending on the
local and remote storage and the network.
See https://github.com/nirs/http-bench for more info.
Richard Musiol [Tue, 30 Apr 2019 17:16:23 +0000 (19:16 +0200)]
syscall: on wasm, do not use typed array asynchronously
The underlying buffer of a typed array becomes invalid as soon as we
grow the WebAssembly memory, which can happen at any time while Go code
runs. This is a known limitation, see https://golang.org/cl/155778.
As a consequence, using a typed array with one of the asynchronous
read/write operations of Node.js' fs module is dangerous, since it may
become invalid while the asynchronous operation has not finished yet.
The result of this situation is most likely undefined.
I am not aware of any nice solution to this issue, so this change adds
a workaround of using an additional typed array which is not backed by
WebAssembly memory and copying the bytes between the two typed arrays.
Maybe WebAssembly will come up with a better solution in the future.
Jay Conrod [Sun, 21 Apr 2019 19:21:58 +0000 (15:21 -0400)]
cmd/go: make get -u upgrade only modules providing packages
Currently, 'go get -u' upgrades modules matching command line
arguments and any modules they transitively require. 'go get -u' with
no positional arguments upgrades all modules transitively required by
the main module. This usually adds a large number of indirect
requirements, which is surprising to users.
With this change, 'go get' will load packages specified by
its arguments using a similar process to other commands
('go build', etc). Only modules providing packages will be upgraded.
'go get -u' now upgrades modules providing packages transitively
imported by the command-line arguments. 'go get -u' without arguments
will only upgrade modules needed by the package in the current
directory.
'go get -m' will load all packages within a module. 'go get -m -u'
without arguments will upgrade modules needed by the main module. It
is equivalent to 'go get -u all'. Neither command will upgrade modules
that are required but not used.
Note that 'go get -m' and 'go get -d' both download modules in order
to load packages.
Brian Kessler [Fri, 5 Apr 2019 20:05:07 +0000 (14:05 -0600)]
cmd/compile: add signed divisibility rules
"Division by invariant integers using multiplication" paper
by Granlund and Montgomery contains a method for directly computing
divisibility (x%c == 0 for c constant) by means of the modular inverse.
The method is further elaborated in "Hacker's Delight" by Warren Section 10-17
This general rule can compute divisibilty by one multiplication, and add
and a compare for odd divisors and an additional rotate for even divisors.
To apply the divisibility rule, we must take into account
the rules to rewrite x%c = x-((x/c)*c) and (x/c) for c constant on the first
optimization pass "opt". This complicates the matching as we want to match
only in the cases where the result of (x/c) is not also needed.
So, we must match on the expanded form of (x/c) in the expression x == c*(x/c)
in the "late opt" pass after common subexpresion elimination.
Note, that if there is an intermediate opt pass introduced in the future we
could simplify these rules by delaying the magic division rewrite to "late opt"
and matching directly on (x/c) in the intermediate opt pass.
On amd64, the divisibility check is 30-45% faster.
Keith Randall [Tue, 30 Apr 2019 21:03:07 +0000 (14:03 -0700)]
cmd/compile: fix line numbers for index panics
In the statement x = a[i], the index panic should appear to come from
the line number of the '['. Previous to this CL we sometimes used the
line number of the '=' instead.
Emmanuel T Odeke [Mon, 11 Mar 2019 22:52:20 +0000 (15:52 -0700)]
net/http: make Server return 501 for unsupported transfer-encodings
Ensures that our HTTP/1.X Server properly responds
with a 501 Unimplemented as mandated by the spec at
RFC 7230 Section 3.3.1, which says:
A server that receives a request message with a
transfer coding it does not understand SHOULD
respond with 501 (Unimplemented).
Elias Naur [Tue, 30 Apr 2019 18:00:23 +0000 (20:00 +0200)]
cmd/dist: detect GOHOSTARCH on iOS
cmd/dist defaults to GOHOSTARCH=amd64 on darwin because no other
darwin host could build Go. With the upcoming self-hosted iOS
builders, GOHOSTARCH=arm64 is also possible.
Elias Naur [Tue, 30 Apr 2019 17:57:46 +0000 (19:57 +0200)]
cmd/dist: set the default external linker on platforms without gcc
The go tool already sets -extld to the appropriate compiler. This
CL changes cmd/dist to do the same, to fix bootstrapping on platforms
that only have clang (Android and iOS).
all: refer to map elements as elements instead of values
The spec carefully and consistently uses "key" and "element"
as map terminology. The implementation, not so much.
This change attempts to make the implementation consistently
hew to the spec's terminology. Beyond consistency, this has
the advantage of avoid some confusion and naming collisions,
since v and value are very generic and commonly used terms.
I believe that I found all everything, but there are a lot of
non-obvious places for these to hide, and grepping for them is hard.
Hopefully this change changes enough of them that we will start using
elem going forward. Any remaining hidden cases can be removed ad hoc
as they are discovered.
The only externally-facing part of this change is in package reflect,
where there is a minor doc change and a function parameter name change.
all: add new GOOS=illumos, split out of GOOS=solaris
Like GOOS=android which implies the "linux" build tag, GOOS=illumos
implies the "solaris" build tag. This lets the existing ecosystem of
packages still work on illumos, but still permits packages to start
differentiating between solaris and illumos.
cmd/go/internal/modfetch: fix concurrent read/write race in modfetch
On Windows systems, the failure rate for cmd/go's TestScript/mod_concurrent
is somewhere around 3-10% without this change. With the change, I have yet
to see a failure.
Richard Musiol [Fri, 26 Apr 2019 23:16:10 +0000 (01:16 +0200)]
cmd/internal/obj/wasm: cache SP in a local
We use Wasm global variables extensively for simulating
registers, especially SP. V8 does not handle global variables
efficiently.
This CL reduces global variable accesses by caching the global SP
in a local variable in each function. The local cache is set on
function entry and updated after each call (where the stack could
have moved). Within a function, the SP access will use the local
variable.
Supersedes https://golang.org/cl/173979.
Running on Chrome Version 73.0.3683.103 on darwin/amd64:
Richard Musiol [Sun, 28 Apr 2019 10:16:44 +0000 (12:16 +0200)]
runtime: do not use heap arena hints on wasm
The address space of WebAssembly's linear memory is contiguous, so
requesting specific addresses is not supported. Do not use heap arena
hints so we do not have unused memory ranges.
This fixes go1 benchmarks on wasm which ran out of memory since
https://golang.org/cl/170950.
Ordering the initialization this way limits the lifetime of the
temporaries involved. In particular, for large maps the number of
simultaneously live temporaries goes from ~2*len(m) to ~2. This change
makes the compiler (regalloc, mostly) a lot happier. The compiler runs
faster and uses a lot less memory.
For #26546, changes compile time of a big map from 8 sec to 0.5 sec.
Fixes #26552
Update #26546
Change-Id: Ib7d202dead3feaf493a464779fd9611c63fcc25f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174417
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Joel Sing [Mon, 29 Apr 2019 02:24:30 +0000 (12:24 +1000)]
cmd,runtime: enable cgo for openbsd/arm64
Updates #31656.
Change-Id: Ide6f829282fcdf20c67998b766a201a6a92c3035
Reviewed-on: https://go-review.googlesource.com/c/go/+/174132
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ian Lance Taylor [Fri, 26 Apr 2019 05:39:56 +0000 (22:39 -0700)]
runtime: account for callbacks in checkdead on Windows
When a callback runs on a different thread in Windows, as in the
runtime package test TestCallbackInAnotherThread, it will use the
extra M. That can cause the test in checkdead to fail incorrectly.
Check whether there actually is an extra M before expecting it.
I think this is a general problem unrelated to timers. I think the test
was passing previously because the timer goroutine was using an M.
But I haven't proved that. This change seems correct, and it avoids
the test failure when using the new timers on Windows.
Updates #27707
Change-Id: Ieb31c04ff0354d6fae7e173b59bcfadb8b0464cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174037 Reviewed-by: Keith Randall <khr@golang.org>
Change-Id: Ib102ae95acfd89fc3c9942a4ec82c74362f62045
Reviewed-on: https://go-review.googlesource.com/c/go/+/174299
Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime: make mmap return 0 instead of -1 on aix/ppc64
Most of the platforms are returning 0 instead of -1 when mmap syscalls
is failing. This patch corrects it for AIX in order to fix
TestMmapErrorSign and to improve AIX compatibility.
Change-Id: I1dad88d0e69163ad55c504b2b4a997892fd876cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174297
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
testing: delay flag registration; move to an Init function
Any code that imports the testing package forces the testing flags to be
defined, even in non-test binaries. People work around this today by
defining a copy of the testing.TB interface just to avoid importing
testing.
Fix this by moving flag registration into a new function, testing.Init.
Delay calling Init until the testing binary begins to run, in
testing.MainStart.
Init is exported for cases where users need the testing flags to be
defined outside of a "go test" context. In particular, this may be
needed where testing.Benchmark is called outside of a test.
syscall: don't return EINVAL on zero Chmod mode on Windows
Fixes #20858
Change-Id: I45c397795426aaa276b20f5cbeb80270c95b920c
Reviewed-on: https://go-review.googlesource.com/c/go/+/174320 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
os/exec: always set SYSTEMROOT on Windows if not listed in Cmd.Env
Fixes #25210
Change-Id: If27b61776154dae9b9b67bf4e4f5faa785d98105
Reviewed-on: https://go-review.googlesource.com/c/go/+/174318 Reviewed-by: Ian Lance Taylor <iant@golang.org>
Joel Sing [Sun, 28 Apr 2019 07:59:05 +0000 (17:59 +1000)]
runtime: initialise cpu.HWCap on openbsd/arm64
OpenBSD does not provide auxv, however we still need to initialise cpu.HWCap.
For now initialise it to the bare minimum, until some form of CPU capability
detection is implemented or becomes available - see issue #31746.
Richard Musiol [Thu, 25 Apr 2019 08:55:49 +0000 (10:55 +0200)]
misc/wasm: fix command line arguments containing multi-byte characters
Command line arguments containing multi-byte characters were causing
go_js_wasm_exec to crash (RangeError: Source is too large), because
their byte length was not handled correctly. This change fixes the bug.
BigMikes [Sun, 30 Dec 2018 14:18:58 +0000 (15:18 +0100)]
net: correct docs of KeepAlive field in Dialer type
KeepAlive field used to report the wording "keep-alive period"
which may be misleading. This field does not represent the whole
TCP keepalive time, that is the inactivity period upon which one
endpoint starts probing the other end. But it acctually specifies
the keepalive interval, that is the time between two keepalive
probes.
Dmitri Shuralyov [Mon, 29 Apr 2019 04:23:16 +0000 (00:23 -0400)]
net/http: remove "number:" from Response.Status string
The behavior of Value.String method on non-string JavaScript types has
changed after CL 169757.
Update the implementation of Transport.RoundTrip method to construct the
Response.Status string without relying on result.Get("status").String(),
since that now returns strings like "<number: 200>" instead of "200".
Daniel Martí [Sun, 28 Apr 2019 16:03:35 +0000 (23:03 +0700)]
all: remove a few unused parameters
I recently modified tabwriter to reduce the number of defers due to
flush calls. However, I forgot to notice that the new function
flushNoDefers can no longer return an error, due to the lack of the
defer.
In crypto/tls, hashForServerKeyExchange never returned a non-nil error,
so simplify the code.
Finally, in go/types and net we can find a few trivially unused
parameters, so remove them.
Adds a sample Fuzz test function to package encoding/json following the
guidelines defined in #31309, based on
https://github.com/dvyukov/go-fuzz-corpus/blob/master/json/json.go
This CL adds support for consulting the Go checksum database
when downloading a module that is not already listed in go.sum.
The overall system is described at golang.org/design/25530-sumdb,
and this CL implements the functionality described specifically in
golang.org/design/25530-sumdb#command-client.
Although the eventual plan is to set GOPROXY and GOSUMDB to
default to a Google-run proxy serving the public Go ecosystem,
this CL leaves them off by default.
Alessandro Arzilli [Tue, 5 Feb 2019 07:49:49 +0000 (08:49 +0100)]
runtime: whitelist debugCall32..debugCall65536 in debugCallCheck
Whitelists functions debugCall32 through debugCall65536 in
runtime.debugCallCheck so that any instruction inside those functions
is considered a safe point.
This is useful for implementing nested function calls.
For example when evaluating:
f(g(x))
The debugger should:
1. initiate the call to 'f' until the entry point of 'f',
2. complete the call to 'g(x)'
3. copy the return value of 'g(x)' in the arguments of 'f'
4. complete the call to 'f'
Similarly for:
f().amethod()
The debugger should initiate the call to '.amethod()', then initiate
and complete the call to f(), copy the return value to the arguments
of '.amethod()' and finish its call.
However in this example, unlike the other example, it may be
impossible to determine the entry point of '.amethod()' until after
'f()' is evaluated, which means that the call to 'f()' needs to be
initiated while stopped inside a debugCall... function.
Jason A. Donenfeld [Sat, 27 Apr 2019 09:45:11 +0000 (11:45 +0200)]
syscall: allow setting security attributes on processes
This allows creating processes that can only be debugged/accessed by
certain tokens, according to a particular security descriptor. We
already had everything ready for this but just neglected to pass through
the value from the user-accessible SysProcAttr.
Change-Id: I4a3fcc9f5078aa0058b26c103355c984093ae03f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174197
Run-TryBot: Jason Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Alex Brainman [Sat, 27 Apr 2019 01:02:31 +0000 (11:02 +1000)]
cmd/go/internal/renameio: use ERROR_ACCESS_DENIED to check for errors
CL 172418 added code to check for "Access is denied" error.
But "Access is denied" error will be spelled differently on
non-English version of Windows.
Elias Naur [Sat, 27 Apr 2019 10:15:42 +0000 (12:15 +0200)]
cmd/link/internal/ld,syscall: replace getfsstat64 with getfsstat
getfsstat64 is deprecated but not yet caught by the App Store checks.
Use the supported getfsstat$INODE64 form instead to ensure forward
compatibility.
Change-Id: I0d97e8a8b254debb3de1cfcb3778dbed3702c249
Reviewed-on: https://go-review.googlesource.com/c/go/+/174200
Run-TryBot: Elias Naur <mail@eliasnaur.com> Reviewed-by: Keith Randall <khr@golang.org>
Elias Naur [Sat, 27 Apr 2019 09:36:40 +0000 (11:36 +0200)]
cmd/link/internal/ld,syscall: drop $INODE64 suffixes on simulators
Some libc functions are suffixed with "$INODE64" on macOS.
Unfortunately, the iOS simulator doesn't have the suffixes, so we can't
use GOARCH to distinguish the two platform.
Add linker support for adding the suffix, using the macho platform
to determine whether it is needed.
While here, add the correct suffix for fdopendir on 386. It's
"$INODE64$UNIX2003", believe it or not. Without the suffix,
Brian Kessler [Sun, 10 Mar 2019 04:58:16 +0000 (21:58 -0700)]
cmd/compile: add unsigned divisibility rules
"Division by invariant integers using multiplication" paper
by Granlund and Montgomery contains a method for directly computing
divisibility (x%c == 0 for c constant) by means of the modular inverse.
The method is further elaborated in "Hacker's Delight" by Warren Section 10-17
This general rule can compute divisibilty by one multiplication and a compare
for odd divisors and an additional rotate for even divisors.
To apply the divisibility rule, we must take into account
the rules to rewrite x%c = x-((x/c)*c) and (x/c) for c constant on the first
optimization pass "opt". This complicates the matching as we want to match
only in the cases where the result of (x/c) is not also available.
So, we must match on the expanded form of (x/c) in the expression x == c*(x/c)
in the "late opt" pass after common subexpresion elimination.
Note, that if there is an intermediate opt pass introduced in the future we
could simplify these rules by delaying the magic division rewrite to "late opt"
and matching directly on (x/c) in the intermediate opt pass.
Additional rules to lower the generic RotateLeft* ops were also applied.
On amd64, the divisibility check is 25-50% faster.
Daniel Theophanes [Fri, 26 Apr 2019 18:46:26 +0000 (11:46 -0700)]
database/sql: add NullInt32
It is common for database integers to be represented as int32
internally. Although NullInt64 is already defined,
this should remove some type casts and make working with those eaiser.
For linking large binaries, the slice of Relocs consumes a large
amount of memory. We can reduce this memory consumption by
shrinking the size of the Reloc struct. This CL moves the fields
used only in external linking or only on PPC64 and S390X to a
lazily initialized side struct.
Linking k8s.io/kubernetes/cmd/kube-apiserver on Linux/AMD64,
before:
inuse_space 1240.25MB total
438.11MB 35.32% 35.32% 438.11MB 35.32% cmd/link/internal/objfile.(*objReader).readSlices
after:
inuse_space 1123.39MB total
306.85MB 27.31% 55.03% 306.85MB 27.31% cmd/link/internal/objfile.(*objReader).readSlices
Under GOGC=5 (to simulate system under memory pressure), the max
RSS reduces from ~2.05G to ~1.83G. Even with external linking the
max RSS doesn't increase.
Daniel Theophanes [Fri, 26 Apr 2019 18:09:51 +0000 (11:09 -0700)]
database/sql: check if src is nil before converting to string
A nil src (NULL database value) will result in a "nil" string,
which will never parse correctly in a ParseInt or similar
numeric conversion. The resulting error is confusing. Check
for a nil src prior to converting the value to string
if the resulting string will be parsed after that.
cmd/go: add internal/note, internal/sumweb, internal/tlog from golang.org/x/exp/sumdb
Copied and updated import paths.
Eventually we will probably publish
these packages somewhere in golang.org/x
(as non-internal packages) and then we will
be able to vendor them properly.
For now, copy.
sumweb.globsMatchPath moved to str.GlobsMatchPath.
Change-Id: I4585e6dc5daa423e4ca9669195d41e58e7c8c275
Reviewed-on: https://go-review.googlesource.com/c/go/+/173950 Reviewed-by: Jay Conrod <jayconrod@google.com>
Ian Lance Taylor [Fri, 26 Apr 2019 13:56:58 +0000 (06:56 -0700)]
runtime: test for cgo build tag in TestGdbPythonCgo
Testing whether cgo is enabled in go/build is not the same as testing
whether the go tool supports cgo. They differ, for example, when using
GOARCH=386 on an amd64 system, as for a cross-build cgo is disabled by default.
Change-Id: Ib59106c92a3131b73ac6a91c0f7658a1769acf73
Reviewed-on: https://go-review.googlesource.com/c/go/+/174098
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
AIX doesn't allow to mmap an address range which is already mmap.
Therefore, once the region has been allocated, it must munmap before
being able to play with it.
LE Manh Cuong [Wed, 30 Jan 2019 05:38:15 +0000 (12:38 +0700)]
time: fix misleading error with the leading zero format
When the leading zero format is used, we currently don't handle the
month and year properly.
For the month, we were reporting an out of range error when getnum
returns zero of its own, as it also returns the month 0. That's
confusing, so only check the range when getnum returns a nil error.
For the year, we don't restore the value when parsing error occurs. For
example, with the incorrect input "111-01", "01" will be used to report
an error. So restore the value when an error occurs fix the problem.
Fixes #29918
Fixes #29916
Change-Id: I3145f8c46813a0457766b7c302482e6b56f94ed6
Reviewed-on: https://go-review.googlesource.com/c/go/+/160338
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Expand "go version" from printing just the version of the
go command itself to being able to print the version of any
go binary (when listed on the command line).
This is a simplified version of rsc.io/goversion, which will
now be deprecated in favor of "go version".
(Preparatory changes to runtime, cmd/go, and cmd/link
made this version information easier to find, allowing a
much simpler implementation than in rsc.io/goversion.)
It is useful to be able to dig the Go version out of a binary,
even a stripped binary. rsc.io/goversion does this for x86
binaries by disassembling the binary to find runtime/proc.go's
startup-time reference to runtime.buildVersion's address.
That approach is quite fragile: the implementation doesn't
work for non-x86 and must be updated as the generated
code changes.
rsc.io/goversion finds the module version string by looking
for random 16-byte framing around the actual string.
This is less fragile but fairly kludgy and requires scanning
the entire data segment.
cmd/buildid finds the build ID by looking for an ELF note
or else falling back to scanning the beginning of the text
segment for a magic string. This has proved quite reliable
and doesn't require scanning much of the binary.
This CL makes it possible to find the Go and module versions
using a scan more like the build ID scan: a symbol early in
the writable data segment starts with a magic number and
then has pointers to the two string variables.
Than McIntosh [Thu, 25 Apr 2019 14:35:47 +0000 (10:35 -0400)]
cmd/link: use read-only mmap to back selected symbol name strings
When reading symbol names from an object file, if a name does not
need fixup (conversion of "". to package path), then generate
a string whose backing store is in read-only memory (from an mmap
of the object file), avoiding the need for an allocation. This
yields a modest reduction in total linker heap use.
runtime: switch to P 0 before destroying current P
Ps are strictly numbered from 0 to GOMAXPROCS-1, so if procresize
happens to be running on a P that's being destroyed, it moves itself
to P 0.
However, currently procresize destroys the unused Ps *before* moving
itself to P 0. This means it may briefly run on a destroyed P. This is
basically harmless, but has at least one very confusing consequence:
since destroying a P has write barriers, it may enqueue pointers to a
destroyed write barrier buffer. As far as I can tell, there are no
negative consequences of this, but this seems really fragile.
This CL swaps the order of things, so now procresize moves itself to P
0 if necessary before destroying Ps. This ensures it always has a
valid P.
This is part of refactoring for #10958 and #24543, but is a good
cleanup regardless.