runtime: map bitmap and spans during heap initialization

We lazily map the bitmap and spans areas as the heap grows. However,
right now we're very slightly too lazy. Specifically, the following
can happen on 32-bit:

1. mallocinit fails to allocate any heap arena, so
   arena_used == arena_alloc == arena_end == bitmap.

2. There's less than 256MB between the end of the bitmap mapping and
   the next mapping.

3. On the first allocation, mheap.sysAlloc sees that there's not
   enough room in [arena_alloc, arena_end) because there's no room at
   all. It gets a 256MB mapping from somewhere *lower* in the address
   space than arena_used and sets arena_alloc and arena_end to this
   hole.

4. Since the new arena_alloc is lower than arena_used, mheap.sysAlloc
   doesn't bother to call mheap.setArenaUsed, so we still don't have a
   bitmap mapping or a spans array mapping.

5. mheap.grow, which called mheap.sysAlloc, attempts to fill in the
   spans array and crashes.

Fix this by mapping the metadata regions for the initial arena_used
when the heap is initialized, rather than trying to wait for an
allocation. This maintains the intended invariant that the structures
are always mapped for [arena_start, arena_used).

Fixes #21044.

Change-Id: I4422375a6e234b9f979d22135fc63ae3395946b0
Reviewed-on: https://go-review.googlesource.com/51714
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

diff --git a/src/runtime/mheap.go b/src/runtime/mheap.go
index bf682ec97f93b4e6b3fae4225c62f51ac7037540..893587e5d2243a0a99f8a131853a500c7d9a5d3f 100644 (file)
--- a/src/runtime/mheap.go
+++ b/src/runtime/mheap.go
@@ -503,6 +503,11 @@ func (h *mheap) init(spansStart, spansBytes uintptr) {
        sp.array = unsafe.Pointer(spansStart)
        sp.len = 0
        sp.cap = int(spansBytes / sys.PtrSize)
+
+       // Map metadata structures. But don't map race detector memory
+       // since we're not actually growing the arena here (and TSAN
+       // gets mad if you map 0 bytes).
+       h.setArenaUsed(h.arena_used, false)
 }
 
 // setArenaUsed extends the usable arena to address arena_used and