From: Russ Cox <rsc@golang.org>
Date: Fri, 7 Aug 2015 17:34:56 +0000 (-0400)
Subject: runtime: make sure heapBitsBulkBarrier cannot be preempted
X-Git-Tag: go1.5~1^2~31
X-Git-Url: http://www.git.cypherpunks.su/?a=commitdiff_plain;h=3ae17043f7f29f0d745aa35b340f8cab80219068;p=gostls13.git

runtime: make sure heapBitsBulkBarrier cannot be preempted

Changes the torture test in #12068 from failing about 1/10 times
to not failing in almost 2,000 runs.

This was only happening in -race mode because functions are
bigger in -race mode, so a few of the helpers for heapBitsBulkBarrier
were not being inlined, and they were not marked nosplit,
so (only in -race mode) the write barrier was being preempted by GC,
causing missed pointer updates.

Filed issue #12069 for diagnosis of any other similar errors.

Fixes #12068.

Change-Id: Ic174d9b050ba278b18b08ab0d85a73c33bd5b175
Reviewed-on: https://go-review.googlesource.com/13364
Reviewed-by: Austin Clements <austin@google.com>
---

diff --git a/src/runtime/mbitmap.go b/src/runtime/mbitmap.go
index c439158f07..fd6b4b11a2 100644
--- a/src/runtime/mbitmap.go
+++ b/src/runtime/mbitmap.go
@@ -109,6 +109,9 @@ func add1(p *byte) *byte {
 
 // subtract1 returns the byte pointer p-1.
 //go:nowritebarrier
+//
+// nosplit because it is used during write barriers and must not be preempted.
+//go:nosplit
 func subtract1(p *byte) *byte {
 	// Note: wrote out full expression instead of calling subtractb(p, 1)
 	// to reduce the number of temporaries generated by the
@@ -245,6 +248,9 @@ func (h heapBits) prefetch() {
 // next returns the heapBits describing the next pointer-sized word in memory.
 // That is, if h describes address p, h.next() describes p+ptrSize.
 // Note that next does not modify h. The caller must record the result.
+//
+// nosplit because it is used during write barriers and must not be preempted.
+//go:nosplit
 func (h heapBits) next() heapBits {
 	if h.shift < 3*heapBitsShift {
 		return heapBits{h.bitp, h.shift + heapBitsShift}
@@ -293,6 +299,9 @@ func (h heapBits) setMarkedNonAtomic() {
 
 // isPointer reports whether the heap bits describe a pointer word.
 // h must describe the initial word of the object.
+//
+// nosplit because it is used during write barriers and must not be preempted.
+//go:nosplit
 func (h heapBits) isPointer() bool {
 	return (*h.bitp>>h.shift)&bitPointer != 0
 }