runtime: generally allow preemption during concurrent GC phases

Currently, the entire GC process runs with g.m.preemptoff set. In the
concurrent phases, the parts that actually need preemption disabled
are run on a system stack and there's no overall need to stay on the
same M or P during the concurrent phases. Hence, move the setting of
g.m.preemptoff to when we start mark termination, at which point we
really do need preemption disabled.

This dramatically changes the scheduling behavior of the concurrent
mark phase. Currently, since this is non-preemptible, concurrent mark
gets one dedicated P (so 1/GOMAXPROCS utilization). With this change,
the GC goroutine is scheduled like any other goroutine during
concurrent mark, so it gets 1/<runnable goroutines> utilization.

You might think it's not even necessary to set g.m.preemptoff at that
point since the world is stopped, but stackalloc/stackfree use this as
a signal that the per-P pools are not safe to access without
synchronization.

Change-Id: I08aebe8179a7d304650fb8449ff36262b3771099
Reviewed-on: https://go-review.googlesource.com/8839
Reviewed-by: Rick Hudson <rlh@golang.org>

diff --git a/src/runtime/mgc.go b/src/runtime/mgc.go
index 7efd8d6d067df5c87a80ab2f1371ff1ec8165c31..32f9b4d852db907b3811a54dea89b73f43368808 100644 (file)
--- a/src/runtime/mgc.go
+++ b/src/runtime/mgc.go
@@ -433,9 +433,6 @@ func gc(mode int) {
                sweep.nbgsweep++
        }
 
-       mp := acquirem()
-       mp.preemptoff = "gcing"
-       releasem(mp)
        gctimer.count++
        if mode == gcBackgroundMode {
                gctimer.cycle.sweepterm = nanotime()
@@ -537,10 +534,9 @@ func gc(mode int) {
        }
 
        startTime := nanotime()
-       if mp != acquirem() {
-               throw("gcwork: rescheduled")
-       }
 
+       mp := acquirem()
+       mp.preemptoff = "gcing"
        _g_ := getg()
        _g_.m.traceback = 2
        gp := _g_.m.curg