[dev.garbage] runtime: revive sweep fast path

sweep used to skip mcental.freeSpan (and its locking) if it didn't
find any new free objects. We lost that optimization when the
freed-object counting changed in dad83f7 to count total free objects
instead of newly freed objects.

The previous commit brings back counting of newly freed objects, so we
can easily revive this optimization by checking that count (like we
used to) instead of the total free objects count.

Change-Id: I43658707a1c61674d0366124d5976b00d98741a9
Reviewed-on: https://go-review.googlesource.com/22596
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>

diff --git a/src/runtime/mgcsweep.go b/src/runtime/mgcsweep.go
index 82537edaaa6bdfbdf2e5dc2fedd023aaaef7cf40..b8e33897c1c6f5fc6e835a107ece44e9463f68e6 100644 (file)
--- a/src/runtime/mgcsweep.go
+++ b/src/runtime/mgcsweep.go
@@ -296,7 +296,7 @@ func (s *mspan) sweep(preserve bool) bool {
        // But we need to set it before we make the span available for allocation
        // (return it to heap or mcentral), because allocation code assumes that a
        // span is already swept if available for allocation.
-       if freeToHeap || nfree == 0 {
+       if freeToHeap || nfreed == 0 {
                // The span must be in our exclusive ownership until we update sweepgen,
                // check for potential races.
                if s.state != mSpanInUse || s.sweepgen != sweepgen-1 {
@@ -309,7 +309,7 @@ func (s *mspan) sweep(preserve bool) bool {
                atomic.Store(&s.sweepgen, sweepgen)
        }
 
-       if nfree > 0 && cl != 0 {
+       if nfreed > 0 && cl != 0 {
                c.local_nsmallfree[cl] += uintptr(nfreed)
                res = mheap_.central[cl].mcentral.freeSpan(s, preserve, wasempty)
                // MCentral_FreeSpan updates sweepgen