Joe Tsai [Fri, 25 Jul 2025 00:10:54 +0000 (17:10 -0700)]
encoding/json: fix truncated Token error regression in goexperiment.jsonv2
The jsontext.Decoder.ReadToken method reports a non-EOF error,
if the token stream is truncated and does not form a valid JSON value.
In contrast, the v1 json.Decoder.Token method would report EOF
so long as the input was a prefix of some valid JSON value.
Modify json.Decoder.Token to preserve historical behavior.
This only modifies code that is compiled in under goexperiment.jsonv2.
Updates #69449
Fixes #74750
Change-Id: Ifd281c46f118f0e748076013fefc7659f77c56ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/689516 Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Joe Tsai [Wed, 25 Jun 2025 01:56:26 +0000 (18:56 -0700)]
encoding/json/jsontext: preserve buffer capacity in Encoder.Reset
This does the equivalent of CL 681177 for the Encoder.
It preserves the internal buffer between resets.
Change-Id: I5e9353b6d7755e067d4f9a4d1ea3d8f056253027
Reviewed-on: https://go-review.googlesource.com/c/go/+/690375 Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Keith Randall [Thu, 24 Jul 2025 23:41:23 +0000 (16:41 -0700)]
reflect: avoid a bounds check in stack-constrained code
Since CL 682496 we need more stack space to handle bounds checks.
The code modified here normally has no bounds checks, but in -N
builds it still does and thus uses too much stack.
Use unsafe arithmetic to avoid the bounds check.
This will hopefully fix some of the arm64 linux builders.
Change-Id: I5b3096a14b4fb9553e635b7f340e60b8ffba8755
Reviewed-on: https://go-review.googlesource.com/c/go/+/690415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@google.com>
Joe Tsai [Thu, 24 Jul 2025 19:16:35 +0000 (12:16 -0700)]
encoding/json: fix extra data regression under goexperiment.jsonv2
When operating under v1 semantics in the v2 implementation,
a extra data error should take precedence over any semantic error
that could theoretically occur within the value itself.
This change only affects code compiled under goexperiment.jsonv2.
Fixes #74614
Change-Id: I055a606b053fa66b0c766ae205487b8290109285
Reviewed-on: https://go-review.googlesource.com/c/go/+/689919 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Joe Tsai [Thu, 24 Jul 2025 18:34:18 +0000 (11:34 -0700)]
encoding/json: reduce error text regressions under goexperiment.jsonv2
There were minor and unnecessary error text changes
when v1 was implemented using v2.
Reduce divergences if possible.
Of the cases reported in #74713, there are no more differences for:
v1: json: cannot unmarshal number into Go value of type chan int
v2: json: cannot unmarshal number into Go value of type chan int
and
v1: json: cannot unmarshal number into Go value of type error
v2: json: cannot unmarshal number into Go value of type error
However, there is a difference between:
v1: json: cannot unmarshal string into Go struct field .F.V of type int
v2: json: cannot unmarshal string into Go struct field S.F.V of type int
For reasons unclear, the v1 logic was always inconsistent about
whether it could properly record the root struct type,
while the v1 emulation layer under v2 is always able to.
This only modifies code that is compiled in under goexperiment.jsonv2.
Fixes #74713
Change-Id: I9e87323b1810130cb929288fdd86aff4be82d5f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/689918 Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
internal/buildcfg: add ability to get GORISCV64 variable in GOGOARCH
For #61476
Change-Id: I29f4c1c3c3303e70ec2d7f380112eb2d00754018
Reviewed-on: https://go-review.googlesource.com/c/go/+/665655 Reviewed-by: Mark Ryan <markdryan@rivosinc.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Mark Freeman <mark@golang.org> Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Keith Randall [Tue, 1 Jul 2025 21:45:45 +0000 (14:45 -0700)]
runtime: detect successful recovers differently
Use stack unwinding instead of keeping incremental track of the argp
of defers that are allowed to recover.
It's much simpler, and it lets us get rid of the incremental tracking
by wrapper code. (Ripped out in a subsequent CL.)
We only need to stack unwind a few frames to get the right answer, and
only when recover()ing in a panic situation. It will be more expensive
in that case, but cheaper in all others.
Change-Id: Id095807db6864b7ac1e1baf09285b77a07c46d19
Reviewed-on: https://go-review.googlesource.com/c/go/+/685355 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
This is an attempt to see if we can get enough performance that we don't
need to consider assembly like that in CL 681695.
Change-Id: I8250feb797a6b4e7d335c23929f6e3acc8b24840
Reviewed-on: https://go-review.googlesource.com/c/go/+/682778 Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Keith Randall [Wed, 18 Jun 2025 22:06:55 +0000 (15:06 -0700)]
cmd/compile: move arm64 over to new bounds check strategy
For all the static bounds checks in cmd/go, we have:
6877 just a single instruction (the call itself)
139 needs an additional reg-reg move
602 needs an additional constant load
25 needs some other instruction
that's ~90% implemented using just a single instruction.
Reduces the text size of cmd/go by ~0.8%.
Total binary size is just barely smaller, ~0.2%. (The difference
is the new pcdata table.)
Change-Id: I416e9c196f5d8d0e8f08e191e6df3045e11dccbe
Reviewed-on: https://go-review.googlesource.com/c/go/+/682496
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Keith Randall [Wed, 18 Jun 2025 21:50:23 +0000 (14:50 -0700)]
cmd/compile,runtime: remember idx+len for bounds check failure with less code
Currently we must put the index and length into specific registers so
we can call into the runtime to report a bounds check failure.
So a typical bounds check call is something like:
MOVD R3, R0
MOVD R7, R1
CALL runtime.panicIndex
or, if for instance the index is constant,
MOVD $7, R0
MOVD R9, R1
CALL runtime.panicIndex
Sometimes the MOVD can be avoided, if the value happens to be in the
right register already. But that's not terribly common, and doesn't
work at all for constants.
Let's get rid of those MOVD instructions. They pollute the instruction
cache and are almost never executed.
Instead, we'll encode in a PCDATA table where the runtime should find
the index and length. The table encodes, for each index and length,
whether it is a constant or in a register, and which register or
constant it is.
That way, we can avoid all those useless MOVDs. Instead, we can figure
out the index and length at runtime. This makes the bounds panic path
slower, but that's a good tradeoff.
We can encode registers 0-15 and constants 0-31. Anything outside that
range still needs to use an explicit instruction.
This CL is the foundation, followon CLs will move each architecture
to the new strategy.
Change-Id: I705c511e546e6aac59fed922a8eaed4585e96820
Reviewed-on: https://go-review.googlesource.com/c/go/+/682396 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
While ./make.bash uniqued per LOC, there is 2 doublings and 4 halvings.
Change-Id: Ic0727cbf429528a2dbf17cbfc3b0121db8387444
Reviewed-on: https://go-review.googlesource.com/c/go/+/685695
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org>
There are other benefits not tested by this benchmark:
- the math form is usually a couple bytes shorter (ICACHE)
- the math form is usually 0~2 uops shorter (UCACHE)
- the math form has usually less register pressure*
- the math form can sometimes be optimized further
*regalloc rarely find how it can use less registers
As far as pass ordering goes there are many possible options,
I've decided to reorder branchelim before late opt since:
- unlike running exclusively the CondSelect rules after branchelim,
some extra optimizations might trigger on the adds or subs.
- I don't want to maintain a second generic.rules file of only the stuff,
that can trigger after branchelim.
- rerunning all of opt a third time increase compilation time for little gains.
By elimination moving branchelim seems fine.
Change-Id: I869adf57e4d109948ee157cfc47144445146bafd
Reviewed-on: https://go-review.googlesource.com/c/go/+/685676 Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
cmd/compile: learn transitive proofs for safe unsigned subs
I've split this into it's own CL to make git bisect more effective.
Change-Id: I436ff21a3e2362b3924de25a458534eb9947e013
Reviewed-on: https://go-review.googlesource.com/c/go/+/685821 Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
cmd/compile: learn transitive proofs for safe negative signed adds
I've split this into it's own CL to make git bisect more effective.
Change-Id: Ib2c6dbc82fb04f50f2d17fbe6626c9fc322fb478
Reviewed-on: https://go-review.googlesource.com/c/go/+/685820
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
cmd/compile: learn transitive proofs for safe positive signed adds
I've split this into it's own CL to make git bisect more effective.
Change-Id: I3fbb42ec7d29169a29f7f55ef2c188317512f532
Reviewed-on: https://go-review.googlesource.com/c/go/+/685819
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
cmd/compile: learn transitive proofs for safe unsigned adds
I've split this into it's own CL to make git bisect more effective.
Change-Id: Iaab5f0bd2ad51e86ced8c6b8fbd371eb75eeef14
Reviewed-on: https://go-review.googlesource.com/c/go/+/685815 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Mark Freeman <mark@golang.org>
Alexander Musman [Sat, 7 Jun 2025 09:00:11 +0000 (12:00 +0300)]
cmd/compile: fold shift through AND for slice operations
Fold a shift through AND when the AND gets a zero-or-one operand (e.g.
from arithmetic shift by 63 of a 64-bit value) for a common case with
slice operations:
ASR $63, R2, R2
AND R3<<3, R2, R2
ADD R2, R0, R2
As the operands are 64-bit, we can transform it to:
Change-Id: Ibcd5e67173da39b77ceff77ca67812fb8be5a7b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/679895 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Mark Freeman <mark@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
internal/runtime/syscall: rename to internal/runtime/syscall/linux
All code in internal/runtime/syscall is Linux-specific, so better
move it to a new linux sub-directory. This way it will be easier
to factor out runtime syscall code from other platforms, e.g.
Windows.
Updates #51087.
Change-Id: Idd2a52444b33bf3ad576b47fd232e990cdc8ae75
Reviewed-on: https://go-review.googlesource.com/c/go/+/689155 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Alexander Musman [Tue, 13 May 2025 06:44:17 +0000 (09:44 +0300)]
cmd/compile: improve loopRotate to handle nested loops
Enhance loop rotation of nested loops. Currently, loops are processed independently,
resulting in unnecessary jumps between outer and inner loops. By processing inner
loops before their parent loop, we ensure nested loop blocks are
properly placed within their parent loop's block sequence.
There is some code size improvement (as measured on amd64) due to jumps
to/from inner loop are removed by the updated loopRotate block order:
Change-Id: I4b9e993c2be07fad735e6bcf32d062d099d9cfb5
Reviewed-on: https://go-review.googlesource.com/c/go/+/684335 Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Alexander Musman [Sat, 7 Jun 2025 10:18:58 +0000 (13:18 +0300)]
cmd/compile: optimize slice bounds checking with SUB/SUBconst comparisons
Optimize ARM64 code generation for slice bounds checking by recognizing
patterns where comparisons to zero involve SUB or SUBconst operations.
This change adds SSA opt rules to simplify:
(CMPconst [0] (SUB x y)) => (CMP x y)
The optimizations apply to EQ, NE, ULE, and UGT comparisons, enabling
more efficient bounds checking for slice operations.
Code size improvement:
compile: .text: 9088004 -> 9065988 (-0.24%)
etcd: .text: 10500276 -> 10497092 (-0.03%)
Change-Id: I467cb27674351652bcacc52b87e1f19677bd46a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/679915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
internal/poll: remove handle field from Windows' poll.operation
The handle field can be accessed directly wherever needed, there is
no need to store it in the operation struct.
This skims down the size of os.File by 16 bytes.
Change-Id: I87c94cb773437891127b6c36dc7f8883622ffed3
Reviewed-on: https://go-review.googlesource.com/c/go/+/685435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Damien Neil <dneil@google.com>
internal/poll: remove fd field from Windows' poll.operation
There is no need to keep the fd in the poll.operation struct,
given that all usages of this field have direct access to the fd struct.
This skims down the size of os.File by 16 bytes.
Change-Id: I837e345250387f62e294cc1772d752865a04ef6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/685415 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Julian Zhu [Wed, 18 Jun 2025 09:26:26 +0000 (17:26 +0800)]
runtime: add benchmark for small-size memmory operation
On RISC-V and MIPS andarchitectures, misaligned load/store is not mandatory for implementations. Therefore, it's important to handle memory operations involving small sizes or data with a remainder when divided by 8 or 4.
This CL add some benchmark for small-size memmory operation, to ensure that SSA rules do not generate unaligned access traps on such architectures.
Change-Id: I6fcdfdb76e9552d5b10df140fa92568ac9468386
Reviewed-on: https://go-review.googlesource.com/c/go/+/682575 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
kmvijay [Fri, 30 May 2025 10:10:20 +0000 (10:10 +0000)]
cmd/internal/obj/s390x: add MVCLE instruction
MVCLE (Move Long Extended) instruction is used to move large data storage-to-storage.
This change will add MVCLE into the Go asm for s390x architecture.
Upcoming PR of runtime/memmove_s390x.s will use this instruction for performance improvement.
Change-Id: I3bbb6668c736a36849917887398c74cebb1c3a99
Reviewed-on: https://go-review.googlesource.com/c/go/+/677455 Reviewed-by: Srinivas Pokala <Pokala.Srinivas@ibm.com> Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Munday <mikemndy@gmail.com> Reviewed-by: Vishwanatha HD <vishwanatha.hd@ibm.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Change-Id: I272967d001809ac4948e4118df6cdd0e0661ab96
Reviewed-on: https://go-review.googlesource.com/c/go/+/682195 Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
kmvijay [Wed, 18 Jun 2025 10:57:46 +0000 (10:57 +0000)]
runtime: improvement in memclr for s390x
The unrolled loop for sizes >= 4KB is further optimized.
Offsets are computed and included in the XC instruction directly.
This reduces code size and instructions, improves performance.
Change-Id: I2911866fb82777311ec4219600fb48c85f7bf862
Reviewed-on: https://go-review.googlesource.com/c/go/+/682595 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Michael Matloob [Wed, 23 Jul 2025 15:45:01 +0000 (11:45 -0400)]
cmd/go/internal/doc: clean up after merge with cmd/internal/doc
This is done in a separate CL to reduce the diffs from the previous CL.
Merge the main.go and doc.go files, and isolate the bootstrap-tagged
code to one file.
For #74667
Change-Id: I11bf0aa18beeb898937135f49f473c1ba1b7e756
Reviewed-on: https://go-review.googlesource.com/c/go/+/689875 Reviewed-by: Michael Matloob <matloob@google.com>
Auto-Submit: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Matloob [Wed, 23 Jul 2025 15:25:14 +0000 (11:25 -0400)]
cmd/internal/doc: merge with cmd/go/internal/doc
Now that cmd/doc has been removed, cmd/go/internal/doc is the only user
of the cmd/internal/doc code. Merge cmd/internal/doc into
cmd/go/internal/doc.
For #74667
Change-Id: I16bbe7b1f418b54ee411c8d6e6609c0973e40b3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/689836 Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Matloob <matloob@golang.org> Reviewed-by: Michael Matloob <matloob@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Matloob [Wed, 23 Jul 2025 15:25:14 +0000 (11:25 -0400)]
cmd/doc: delete
Users should not invoke "go tool doc" directly, but should instead run
"go doc". Remove "cmd/doc" because it no longer invokes the "doc" tool
but incorporates its logic directly.
Fixes #74667
Change-Id: I357a3d7e0ca075f028df66e34951a41354c08941
Reviewed-on: https://go-review.googlesource.com/c/go/+/689835 Reviewed-by: Michael Matloob <matloob@google.com>
Auto-Submit: Michael Matloob <matloob@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Roland Shoemaker [Wed, 21 May 2025 02:03:44 +0000 (02:03 +0000)]
runtime: randomize heap base address
During initialization, allow randomizing the heap base address by
generating a random uint64 and using its bits to randomize various
portions of the heap base address.
We use the following method to randomize the base address:
* We first generate a random heapArenaBytes aligned address that we use
for generating the hints.
* On the first call to mheap.grow, we then generate a random
PallocChunkBytes aligned offset into the mmap'd heap region, which we
use as the base for the heap region.
* We then mark a random number of pages within the page allocator as
allocated.
Our final randomized "heap base address" becomes the first byte of
the first available page returned by the page allocator. This results
in an address with at least heapAddrBits-gc.PageShift-1 bits of
entropy.
Fixes #27583
Change-Id: Ideb4450a5ff747a132f702d563d2a516dec91a88
Reviewed-on: https://go-review.googlesource.com/c/go/+/674835 Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Robert Griesemer [Wed, 18 Jun 2025 00:09:27 +0000 (17:09 -0700)]
go/types, types2: better error messages for certain type mismatches
When an untyped operand of a (typically binary) operation does not
match the type of the operand and an implicit conversion is not
possible, the error message should report a "type mismatch".
The type-checkers mostly did so, but not for untyped numeric types
to other types (e.g. an untyped int vs a function); in those cases
it reported that the (impossible) conversion failed.
Fix this for numeric types.
This also improves the position and messages for some incorrect
min/max built-in calls.
Fixes #73428.
Change-Id: I8af071918b73fcc72f16cc61858d7baca57fc259
Reviewed-on: https://go-review.googlesource.com/c/go/+/682495
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Mark Freeman <mark@golang.org> Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Keith Randall [Fri, 13 Jun 2025 18:08:53 +0000 (11:08 -0700)]
cmd/compile: use ,ok return idiom for sparsemap.get
Change-Id: I89719b94de74a32402d02309515dffc4989484db
Reviewed-on: https://go-review.googlesource.com/c/go/+/681575
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@google.com>
Keith Randall [Thu, 12 Jun 2025 00:14:59 +0000 (17:14 -0700)]
cmd/compile: use generics for sparse map
So it is easier to reuse this code with different key/value types.
Change-Id: I5a9e669769cf359b32f2fe784594868acdee4d02
Reviewed-on: https://go-review.googlesource.com/c/go/+/681175 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Keith Randall [Thu, 22 May 2025 17:25:49 +0000 (10:25 -0700)]
cmd/api: rerun updategolden
Not sure why, but
go test cmd/api --updategolden
makes changes to the golden files. The changes are just moving things
around, no actual semantic changes. But this is confusing for new
contributors when updating api files.
Change-Id: If27b039f47b94953b7891f8f6b6999ed79953198
Reviewed-on: https://go-review.googlesource.com/c/go/+/675616
Auto-Submit: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Robert Griesemer <gri@google.com>
runtime: drop NetBSD kernel bug sysmon workaround fixed in NetBSD 9.2
The NetBSD releases supported by the NetBSD project as off today are 9.4
and 10.1. The Go project's NetBSD builders are on 9.3. Thus, it is fine
to drop the workaround which was only needed for NetBSD before 9.2.
Fixes #46495
Cq-Include-Trybots: luci.golang.try:gotip-netbsd-arm64
Change-Id: I3c2ec42fb0f08f7dafdfb7f1dbd97853afc16386
Reviewed-on: https://go-review.googlesource.com/c/go/+/687735
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Auto-Submit: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Adi Aloni [Mon, 23 Jun 2025 10:39:20 +0000 (13:39 +0300)]
cmd/go: clean should respect workspaces
The existing implementation doesn't invoke modload.InitWorkfile during
runClean which in turn causes go clean to ignore workspaces and
consequentially workspace vendoring.
Fixes #74337
Change-Id: I295a1fcc5e81d096971c8cee9c9baa840c7725e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/682856 Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Michael Matloob <matloob@google.com>
Filip Petkovski [Mon, 23 Jun 2025 11:47:21 +0000 (11:47 +0000)]
encoding/json/jsontext: preserve buffer capacity in Decoder.Reset
The Decoder.Reset method is not preserving the internal buffer between
resets, causing buffer capacity to be lost and resulting in unnecessary
allocations when reusing decoders. This is particularly problematic when
decoding many small messages.
This commit fixes the Reset method to preserve the internal buffer. It
makes sure aliasing is removed if the buffer currently points to an
internal byte slice of a bytes.Buffer. It adds a TestDecoderReset test
structured into subtests to better validate the different scenarios.
Change-Id: Ia685bff47034598224489173bb7f2ffd48e89da5
GitHub-Last-Rev: 462ddc936409ee3da6d75def7d9de59eb3c65f8d
GitHub-Pull-Request: golang/go#74120
Reviewed-on: https://go-review.googlesource.com/c/go/+/681177 Reviewed-by: Sean Liao <sean@liao.dev> Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Michael Matloob [Fri, 18 Jul 2025 18:14:16 +0000 (14:14 -0400)]
cmd/go: always return the cached path from go tool -n
If we're running go tool -n always return the cached path of the tool.
We can't always use the cached path when running the tool because if we
copied the tool to the cached location in the same process and then try
to run it we'll run into #22315, producing spurious ETXTBSYs.
Fixes #72824
Change-Id: I81f23773b9028f955ccc97453627ae4f2573814b
Reviewed-on: https://go-review.googlesource.com/c/go/+/688895
Auto-Submit: Michael Matloob <matloob@golang.org> Reviewed-by: Michael Matloob <matloob@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Pratt [Wed, 23 Jul 2025 20:56:50 +0000 (16:56 -0400)]
runtime: check TestUsingVDSO ExitError type assertion
Currently this test panics if the error is not an ExitError. We aren't
expecting other errors, but we want to continue to the t.Fatal so the
error contents actually get logged.
For #74672.
Change-Id: I6a6a636cee5ddac500ed7ec549340b02944101ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/689956 Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Wed, 23 Jul 2025 17:47:03 +0000 (17:47 +0000)]
internal/goversion: update Version to 1.26
Go 1.26 is in the process of being opened for development (to be
eventually released). This change marks the very beginning of its
development cycle, updating the Version value accordingly.
For #40705.
For #73829.
Change-Id: Ie4ce2d38160dd6283c08e10ecbd7d3a43ed92b48
Reviewed-on: https://go-review.googlesource.com/c/go/+/689877 Reviewed-by: Mark Freeman <mark@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Lyle Dean [Sun, 20 Jul 2025 23:20:11 +0000 (00:20 +0100)]
net: correct comment for ListenConfig.ListenPacket
Fixes #74634
Change-Id: I21196c4aef1b717bfda0b54e61b35e83cfa9dc1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/689075 Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Mark Freeman <mark@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
cuishuang [Mon, 9 Jun 2025 07:47:15 +0000 (15:47 +0800)]
cmd/internal/objabi: remove redundant word in comment
Change-Id: I9bbce94a2f36d5de28b946c94652876f01907fe6
Reviewed-on: https://go-review.googlesource.com/c/go/+/680115 Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sean Liao <sean@liao.dev>
Not sure why this tends to be darwin-specific, but we'll fix
just darwin for now.
(It isn't quite darwin-only, as it appeared in #66893.
But it is certainly worse on darwin.)
Fixes #73136
Update #66893
Change-Id: If609e909bc6c65c2663dd46b7a9bad4fd291c3da
Reviewed-on: https://go-review.googlesource.com/c/go/+/689315 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Keith Randall [Mon, 21 Jul 2025 17:09:35 +0000 (10:09 -0700)]
cmd/compile: for arm64 epilog, do SP increment with a single instruction
That way, the frame is atomically popped. Previously, for big frames
the SP was unwound in two steps (because arm64 can only add constants
up to 1<<12 in a single instruction).
Fixes #73259
Change-Id: I382c249194ad7bc9fc19607c27487c58d90d49e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/689235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Keith Randall <khr@google.com>
Keith Randall [Sat, 19 Jul 2025 16:34:04 +0000 (09:34 -0700)]
runtime: fix asan wrapper for riscv64
Make sure we keep the M in X21. The instruction at line 87
needs to be loading from the M, not from the gsignal.
Fixes #73979, maybe?
Change-Id: I3aba1c35454636138b305bca26e20dbc8e6b1d6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/688975 Reviewed-by: Joel Sing <joel@sing.id.au> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com> Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Ian Alexander [Fri, 30 May 2025 21:59:34 +0000 (17:59 -0400)]
cmd/go: check subcommand properties
This change corrects the properties checked by Lookup. We were
inspecting the properties of the Command receiver; now we are
inspecting the properties of the subcommand.
Fixes #73864.
Change-Id: Ieb462e489fc4f8f0568aa3a2d404b322d627166c
Reviewed-on: https://go-review.googlesource.com/c/go/+/678655 Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@google.com>
cmd/internal/obj: enable got pcrel itype in fips140 for riscv64
This CL enable R_RISCV_GOT_PCREL_ITYPE in fips140
Fixes #74662
Change-Id: Ic189c4e352517ae74034f207a5f944b610f2eb73
Reviewed-on: https://go-review.googlesource.com/c/go/+/688635 Reviewed-by: Mark Ryan <markdryan@rivosinc.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Julian Zhu <jz531210@gmail.com>
cmd/compile/internal/ssa: restrict architectures for TestDebugLines_74576
CL 687815 recently added TestDebugLines_74576.
The pre-existing debug_lines_test.go file generally restricts the
tested architectures and contains multiple warnings that the
testing approach is useful but fragile, such as:
"These files must all be short because this is super-fragile."
Despite that, initially I wanted to see what happened on the
different architectures on the trybots in case it might show something
surprising, and I let TestDebugLines_74576 run on all architectures.
That seemed to initially work, but the test is now failing on a
linux/risc64 builder (#74669), so it is likely more prudent to be
more conservative and restrict the platforms like many of the
other pre-existing tests, which is what this CL now does.
Fixes #74669
Change-Id: I9e5a7d3ee901f58253cf72e03c2239df338479e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/688856
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com>
Alan Donovan [Fri, 18 Jul 2025 17:16:21 +0000 (13:16 -0400)]
go/types: infer correct type for y in append(bytes, y...)
The type-checking logic for append has a special case for
append(bytes, s...) where the typeset for s contains string.
However, this case was triggering even when the typeset contained
only []byte, causing the creation of Signature types of the form
func([]byte, Y) []byte, with the variadic flag set, where Y
is the type of Y (e.g. a type parameter constrained to ~[]byte).
This is an illegal combination: a variadic signature's last
parameter must be a slice, or its typeset must contain string.
This caused x/tools/go/ssa to crash.
This CL narrows the special case to only typesets that contain
string, and adds a test for the inferred signature.
(There's little point in testing that a subsequent NewSignatureType
call would succeed, because the inferred type plainly has
no free type parameters.)
Fixes #73871
Change-Id: Id7641104133371dd6b0077f281cdaa9db84cc1c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/688815 Reviewed-by: Mark Freeman <mark@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
cmd/compile/internal/escape: improve DWARF .debug_line numbering for literal rewriting optimizations
The literal rewriting optimizations to reduce user allocations
that were implemented in CL 649079 and related CLs like CL 684116
could produce .debug_line tables such that the line numbers
sometimes jumped back to a variable's declaration.
This CL adjusts the positions of the IR nodes to avoid this.
For the first test added here in i74576a.go:
11 func main() {
12 a := 1
13 runtime.Breakpoint()
14 sink = a
15 }
Without this fix, the test reports debug lines of 12, 13, 13, 12, 14.
Note it goes backwards from 13 to a second 12.
With this fix, the test reports debug lines of 12, 13, 13, 14
without going backwards.
The test added in i74576b.go creates a slice via make with a
non-constant argument, which similarly shows debug lines going backwards
before this fix but not after. To address the slice make case, we
create a new BasicLit node to then set its position.
There were some related allocation optimizations for struct literals
such as CL 649555 during the Go 1.25 dev cycle, but at least in some
basic test cases, those optimizations did not seem to produce
debug line issues. The struct literal interface conversion test
added in i74576c.go has the same behavior before and after this change.
Finally, running 'go test ./...' in the delve repo root (4a2a6e1aeb)
seems to have many failures with go1.25rc2, but seems to pass with
this CL.
Fixes #74576
Updates #71359
Change-Id: I31faf5fe4bb352fdcd06bdc8606dbdbc4bbd65f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/687815
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com> Reviewed-by: Keith Randall <khr@google.com>
Michael Anthony Knyszek [Tue, 15 Jul 2025 17:11:18 +0000 (17:11 +0000)]
runtime: fix idle time double-counting bug
This change fixes a bug in the accounting of sched.idleTime. In just the
case where the GC CPU limiter needs up-to-date data, sched.idleTime is
incremented in both the P-idle-time and idle-mark-work paths, but it
should only be incremented in the former case.
Fixes #74627.
Change-Id: If41b03da102d47d25bec48ff750a9da27019b71d
Reviewed-on: https://go-review.googlesource.com/c/go/+/687998 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
cmd/compile/internal/escape: speed up analyzing some functions with many closures
Escape analysis examines functions in batches. In some cases, closures
are in the same batch as their parent function, but in other cases,
the closures are in different batches. This can mean the per-batch
ir.ReassignOracle cache is not as effective.
For example, #74615 has 4,000 closures in a single function that
are all in different batches. For that example, these caches
had an ~80% hit rate.
This CL makes the ir.ReassignOracle cache more broadly scoped, instead
of per batch.
This speeds up escape analysis when a function has many closures
that end up in different batches, including this resolves #74615.
For that example, this cache now has a ~100% hit rate.
In addition, in (*batch).rewriteWithLiterals, we also slightly delay
checking the ir.ReassignOracle cache, which is more natural to do now
compared to when rewriteWithLiterals was first merged. This means we can
avoid consulting or populating the cache in more cases. (We also leave
a new type-related TODO there. If we were to also implement that TODO, a
quick test suggests we could independently resolve the specific example
in #74615 even without making the cache more broadly scoped,
though other conceivable examples would not be helped; the scoping of
the cache is the more fundamental improvement.)
If we look at cumulative time spent via pprof for the #74615 example
using this CL, the work of ir.ReassignOracle escape analysis
cache now typically shows zero cpu samples.
This CL passes "go build -toolexec 'toolstash -cmp' -a std cmd".
Fixes #74615
Change-Id: I3c17c527fbb546ffb8a4fa52cd61e41ff3cdb869
Reviewed-on: https://go-review.googlesource.com/c/go/+/688075
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Cherry Mui [Tue, 26 Dec 2023 20:35:56 +0000 (15:35 -0500)]
cmd/link, runtime: on Wasm, put only function index in method table and func table
In the type descriptor's method table, it contains relative PCs of
the methods (relative to the start of the text section) stored as
32-bit offsets. On Wasm, a PC is PC_F<<16 + PC_B, where PC_F is
the function index, and PC_B is the block index. When there are
more than 65536 functions, the PC will not fit into 32-bit (and
relative to the section start doesn't help). Since there are no
more bits for the function index, and the method table always
targets the entry of a method, we put just the PC_F there, and
rewrite back to a full PC at run time when we need the PC. This
way we can have more than 65536 functions.
The func table also contains 32-bit relative PCs, and it also
always points to function entries. Do the same there, as well
as other places where we use relative text offsets.
Also add the relocation type in the relocation overflow error
message.
Also add check for function too big on Wasm. If a function has
more than 65536 blocks, PC_B will overflow and PC = PC_F<<16 + PC_B
will points to the wrong function.
Fixes #64856.
Change-Id: If9c307e9fb1641f367a5f19c39f88f455805d0bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/552835 Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Cherry Mui [Thu, 29 Feb 2024 19:27:57 +0000 (14:27 -0500)]
cmd/internal/obj/wasm: use 64-bit instructions for indirect calls
Currently, on Wasm, an indirect call is compiled to
// function index = PC>>16, PC is already on stack
I32WrapI64
I32Const $16
ShrU
// set PC_B to 0
...
// actual call
CallIndirect
Specifically, the function index is extracted from bits 16-31 of
the "PC". When there are more than 65536 functions, this will
overflow and wrap around, causing wrong function being called.
This CL changes it to use 64-bit operations to extract the
function index from the "PC", so there are enough bits to for it.
For #64856.
Change-Id: I83c11db4b78cf66250e88ac02a82bd13730a8914
Reviewed-on: https://go-review.googlesource.com/c/go/+/567896
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Michael Anthony Knyszek [Mon, 14 Jul 2025 19:33:13 +0000 (19:33 +0000)]
runtime: use memclrNoHeapPointers to clear inline mark bits
Clearing the inline mark bits with memclrNoHeapPointers is slightly
better than having the compiler insert, e.g. duffzero, since it can take
advantage of wider SIMD instructions. duffzero is likely going away, but
we know things the compiler doesn't, such as the fact that this memory
is nicely aligned. In this particular case, memclrNoHeapPointers does a
better job.
For #73581.
Change-Id: I3918096929acfe6efe6f469fb089ebe04b4acff5
Reviewed-on: https://go-review.googlesource.com/c/go/+/687938 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Michael Anthony Knyszek [Mon, 14 Jul 2025 19:28:57 +0000 (19:28 +0000)]
runtime: only clear inline mark bits on span alloc if necessary
This change modifies initInlineMarkBits to only clear mark bits if the
span wasn't just freshly allocated from the OS, where we know the bits
are already zeroed. This probably doesn't make a huge difference most of
the time, but it's an easy optimization and helps rule it out as a
source of slowdown.
For #73581.
Change-Id: I78cd4d8968bb0bf6536c0a38ef9397475c39f0ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/687937
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Mon, 14 Jul 2025 19:23:12 +0000 (19:23 +0000)]
runtime: have mergeInlineMarkBits also clear the inline mark bits
This is conceptually simpler, as the sweeper doesn't have to worry about
clearing them separately. It also doesn't have a use for them.
This will also be useful to avoiding unnecessary zeroing in
initInlineMarkBits at allocation time. Currently, because it's used in
both span allocation and at sweep time, we cannot blindly trust
needzero.
This change also renames mergeInlineMarkBits to moveInlineMarkBits to
make this change in semantics clearer from the name.
For #73581.
Change-Id: Ib154738a945633b7ff5b2ae27235baa310400139
Reviewed-on: https://go-review.googlesource.com/c/go/+/687936
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Michael Anthony Knyszek [Mon, 14 Jul 2025 19:13:27 +0000 (19:13 +0000)]
runtime: merge inline mark bits with gcmarkBits 8 bytes at a time
Currently, with Green Tea GC, we need to copy (really bitwise-or) mark
bits back into mspan.gcmarkBits, so that it can propagate to
mspan.allocBits at sweep time. This function does actually seem to make
sweeping small spans a good bit more expensive, though sweeping is still
relatively cheap. There's some low-hanging fruit here though, in that
the merge is performed one byte at a time, but this is pretty
inefficient. We can almost as easily perform this merge one word at a
time instead, which seems to make this operation about 33% faster.
For #73581.
Change-Id: I170d36e7a2193199c423dcd556cba048ebd698af
Reviewed-on: https://go-review.googlesource.com/c/go/+/687935 Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Michael Pratt [Tue, 1 Jul 2025 20:36:22 +0000 (16:36 -0400)]
runtime: expand GOMAXPROCS documentation
Expand the GOMAXPROCS documentation to include details of how defaults
are selected, as this is something that inquisitive minds will want to
know. I've added an additional warning that these details may changed.
While we are here, add a bit more structure to make it easier to find
the relevant parts of the documentation.
For #73193.
Change-Id: I6a6a636cae93237e3e3174822490d51805e70990
Reviewed-on: https://go-review.googlesource.com/c/go/+/685318 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Sean Liao <sean@liao.dev> Reviewed-by: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
Joe Tsai [Tue, 1 Jul 2025 22:39:49 +0000 (15:39 -0700)]
encoding/json: decompose legacy options
WARNING: This commit contains breaking changes
for those already using GOEXPERIMENT=jsonv2.
This decomposes FormatBytesWithLegacySemantics as:
* FormatBytesWithLegacySemantics
* FormatByteArrayAsArray
* ParseBytesWithLooseRFC4648
This decomposes FormatTimeWithLegacySemantics as:
* FormatDurationAsNano
* ParseTimeWithLooseRFC3339
In particular, it splits out specific behaviors from the option
that may need to be specified on a finer-grain level.
FormatByteArrayAsArray and FormatDurationAsNano are targeted
to just the default representation of a [N]byte or time.Duration type.
Both of these are not necessary if the `format` tag is explicitly specified.
However, we want to isolate their behavior from other behaviors that used to
be part of FormatBytesWithLegacySemantics and FormatTimeWithLegacySemantics.
ParseBytesWithLooseRFC4648 and ParseTimeWithLooseRFC3339 are targeted
to just historically buggy parsing according to the relevant RFCs,
which may need to be enabled by some services for backwards compatibility.
While FormatTimeWithLegacySemantics is deleted, we still need
FormatBytesWithLegacySemantics to configure highly esoteric
aspects of how v1 used to handle byte slices.
We rename OmitEmptyWithLegacyDefinition as OmitEmptyWithLegacySemantics
to be consistent with other options with the WithLegacySemantics suffix.
Updates #71497
Change-Id: Ic660515fb086fe3af237135f195736de99c2bd33
Reviewed-on: https://go-review.googlesource.com/c/go/+/685395
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Joe Tsai [Tue, 1 Jul 2025 05:17:41 +0000 (22:17 -0700)]
encoding/json/v2: add security section to doc
This follows up CL 684315 with an expanded section in the v2 doc.
Updates #14750
Updates #71845
Change-Id: I1ffa97e030f5f2b709e8142028e3c8e0e38b80ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/685195
Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Joe Tsai [Thu, 10 Jul 2025 00:08:49 +0000 (17:08 -0700)]
encoding/json: remove legacy option to EscapeInvalidUTF8
In the presence of invalid UTF-8, the AllowInvalidUTF8 option
allows such bytes to be present, but silently mangles them
using the Unicode replacement character.
The v2 default is to emit the replacement character verbatim
(which is valid UTF-8 and exactly what it is for).
However, the v1 behavior has historically been to emit
the escaped form of the replacement character.
This behavior was introduced in https://go.dev/cl/11211045
where the documentation says that it is:
replacing invalid bytes with the Unicode replacement rune U+FFFD
but the implementation actually replaces it with
the escaped form of the Unicode replacement rune.
Given that the documentation differs from the implementation,
the actual behavior is likely an oversight.
Given how esoteric of behavior this is,
we change the v1in2 behavior to avoid the unnecesary escaping
and drop support for EscapeInvalidUTF8.
This does not violate the Go compatibility agreement since
we do not document what the exact syntactic output is.
Also, there has already been prior precedence for changing the output:
* [encoding/json: encode \b and \f as '\b' and '\f' in JSON strings](https://go.dev/cl/521675)
* [encoding/json: encode \n in strings as "\n", not "\u000A"](https://go.dev/cl/4678046)
* [encoding/json: encode \t as \t instead of \u0009](https://go.dev/cl/162340043)
* [encoding/json: use standard ES6 formatting for numbers during marshal](https://go.dev/cl/30371)
Fixes #74551
Change-Id: Ib59a873c44713d302f1f6ab103ffba2520d63276
Reviewed-on: https://go-review.googlesource.com/c/go/+/687116
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
In the event that the input is just JSON whitespace,
the underlying jsontext.Decoder treats this as an empty stream
and reports io.EOF.
The logic in unmarshalFull simply casted io.EOF as io.ErrUnexpectedEOF,
which is inconsistent with how all other io.ErrUnexpectedEOF are reported,
which are wrapped within a jsontext.SyntacticError.
Do the same thing for consistency.
We add a v1 test (without goexperiment.jsonv2) to verify that
the behavior is identical to how v1 has always behaved.
We add a v1in2 test (with goexperiment.jsonv2) to verify that
the v1in2 behavior correctly replicates historical v1 behavior.
We also fix a faulty check in v1 Decoder.Decode,
where it tried to detect errUnexpectedEnd and
return an unwrapped io.ErrUnexpectedEOF error.
This is the exact semantic that v1 has always done
in streaming Decoder.Decode (but not non-streaming Unmarshal).
There is a prior bug reported in #25956 about this inconsistency,
but we aim to preserve historical v1 behavior to reduce
the probability of churn when v1 is re-implemented in terms of v2.
Fixes #74548
Change-Id: Ibca52c3699ff3c09141e081c85f853781a86ec8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/687115
Auto-Submit: Joseph Tsai <joetsai@digital-static.net> Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Damien Neil <dneil@google.com>
crypto/rsa: drop contradictory promise to keep PublicKey modulus secret
We claim to treat N as secret (and indeed bigmod is constant time in
relation to the modulus) but at the same time we warn that all inputs to
VerifyPKCS1v15 and Verify are public:
> The inputs are not considered confidential, and may leak through
> timing side channels, or if an attacker has control of part of the
> inputs.
See #67043 (which focuses on the inverse, recovering signatures by
controlling the public key input to Verify), and in particular
https://github.com/golang/go/issues/67043#issuecomment-2079335804.
Stopping the Verify adaptive attack would require significantly more
complexity, the kind that has caused vulnerabilities in the past (e.g.
CVE-2016-2107). On the other hand, assuming that a public key is
confidential is unlikely to work in practice, since it can be recovered
from just two valid (message, signature) pairs. See for example
https://keymaterial.net/2024/06/15/reconstructing-public-keys-from-signatures/.
This comment was introduced in CL 552935, not really due to a need to
specify that N was secret, but rather to clarify that E is not (so it
could be used in variable-time exponentiation).
Change-Id: I6a6a6964f3f8d2dc2fcc13ce938b271c9de9666b
Reviewed-on: https://go-review.googlesource.com/c/go/+/687616 Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
Keith Randall [Fri, 11 Jul 2025 16:10:53 +0000 (09:10 -0700)]
runtime: turn off large memmove tests under asan/msan
Just like we do for race mode. They are just too slow when running
with the sanitizers.
Fixes #59448
Change-Id: I86e3e3488ec5c4c29e410955e9dc4cbc99d39b84
Reviewed-on: https://go-review.googlesource.com/c/go/+/687535 Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
Cherry Mui [Thu, 10 Jul 2025 04:46:00 +0000 (00:46 -0400)]
cmd/link: do size fixups after symbol references are loaded
When we do a size fixup, we need to clone the symbol to an
external symbol so we can modify it. This includes cloning the
relocations, which includes resolving the relocations. If the
symbol being fixed has a relocation referencing a non-Go symbol,
that symbol has not yet been created, it will be resolved to an
empty symbol. Load the references first, so the referenced symbol,
even if it is a non-Go symbol, exists.
Fixes #74537.
Change-Id: I81525bd7c3e232b80eefeb0f18e13ba5331e1510
Reviewed-on: https://go-review.googlesource.com/c/go/+/687315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
Michael Anthony Knyszek [Wed, 9 Jul 2025 15:50:06 +0000 (15:50 +0000)]
runtime: run TestSignalDuringExec in its own process group
TestSignalDuringExec sends a SIGWINCH to the whole process group.
However, it may execute concurrently with other copies of the runtime
tests, especially through `go tool dist`, and gdb version <12.1 has a
bug in non-interactive mode where recieving a SIGWINCH causes a crash.
This change modifies SignalDuringExec in the testprog to first fork
itself into a new process group. To avoid issues with Ctrl+C and the new
process group hanging, the new process blocks on a pipe that is passed
down to it. This pipe is automatically closed when its parent exits,
which should ensure that the subprocess also exits.
Fixes #58932.
Change-Id: I3906afa28cf8b15d22ae612d071bce7f30fc3e6c
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-noswissmap,gotip-linux-amd64-longtest-aliastypeparams,gotip-linux-amd64-longtest,gotip-linux-386-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/686875
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com>
Daniel McCarney [Sat, 28 Jun 2025 18:29:48 +0000 (14:29 -0400)]
crypto/tls: empty server_name conf. ext. from server
When a TLS server uses the information from the server_name extension in
a client hello, and the connection isn't resuming, it should return an
empty server_name extension in its server hello (or encrypted extensions
for TLS 1.3).
For TLS <1.3 we we do this in doFullHandshake(), by setting the
pre-existing serverHelloMsg.serverNameAck bool. We know that the
connection isn't resuming based on the context where this function is
called.
For TLS 1.3, a new encryptedExtensionsMsg.serverNameAck bool is added,
and populated as appropriate in sendServerParameters() based on whether
the conn was resumed or not. The encryptedExtensionsMsg marshalling is
updated to emit the encrypted extension based on that field.
These changes allow enabling the ServerNameExtensionServer-* bogo tests
that verify both the presence and absence of the server_name extension
based on the relevant specifications.
Resolves #74282
Updates #72006
Change-Id: I703bc2ec916b50906bdece7b7483a7faed7aa8e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/684795
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
Roland Shoemaker [Mon, 9 Jun 2025 18:23:46 +0000 (11:23 -0700)]
cmd/go: disable support for multiple vcs in one module
Removes the somewhat redundant vcs.FromDir, "allowNesting" argument,
which was always enabled, and disallow multiple VCS metadata folders
being present in a single directory. This makes VCS injection attacks
much more difficult.
Also adds a GODEBUG, allowmultiplevcs, which re-enables this behavior.
Thanks to RyotaK (https://ryotak.net) of GMO Flatt Security Inc for reporting this issue.
Fixes #74380
Fixes CVE-2025-4674
Change-Id: I5787d90cdca8deb3aca6f154efb627df1e7d2789
Reviewed-on: https://go-review.googlesource.com/c/go/+/686515
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
Commit-Queue: Carlos Amedee <carlos@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
Change-Id: I85a4051bd3413bd843b17d22cf9120f615cfe8db
Reviewed-on: https://go-review.googlesource.com/c/go/+/686295 Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Carlos Amedee <carlos@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
Ian Lance Taylor [Mon, 7 Jul 2025 20:01:34 +0000 (13:01 -0700)]
cmd/compile: document that nosplit directive is unsafe
For #74478
Change-Id: I902e9a92cdacb5ad6dafa9896640f8196ba1d56a
Reviewed-on: https://go-review.googlesource.com/c/go/+/686115
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
prove.go used to make my editor and precomit checks very unhappy.
Change-Id: I25f7ffa2191480bc1b4f91fa91ccf3e4768045fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/685818 Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Michael Anthony Knyszek [Mon, 7 Jul 2025 17:19:17 +0000 (17:19 +0000)]
internal/trace: only test for sync preemption if async preemption is off
Currently, the test change made for the fix to #68090 is flaky. This is
because the sync-point-only goroutine that we expect to be sync
preempted might only ever get async preempted in some circumstances.
This change adds a variant to all trace tests to run with
asyncpreemptoff=1, and the stacks test, the flaky one, only actually
checks for the sync-point in the trace when async preemption is
disabled.
Fixes #74417.
Change-Id: Ib6341bbc26921574b8f0fff6dd521ce83f85499c
Reviewed-on: https://go-review.googlesource.com/c/go/+/686055
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>