runtime: flush assist credit on goroutine exit

Currently dead goroutines retain their assist credit. This credit can
be used if the goroutine gets recycled, but in general this can make
assist pacing over-aggressive by hiding an amount of credit
proportional to the number of exited (and not reused) goroutines.

Fix this "hidden credit" by flushing assist credit to the global
credit pool when a goroutine exits.

Updates #14812.

Change-Id: I65f7f75907ab6395c04aacea2c97aea963b60344
Reviewed-on: https://go-review.googlesource.com/24703
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

diff --git a/src/runtime/proc.go b/src/runtime/proc.go
index 8adf3b27252697bbd87542ef58666dc6761dd50d..02c092711c9e5577a8683107f6fcc7f6c70f5329 100644 (file)
--- a/src/runtime/proc.go
+++ b/src/runtime/proc.go
@@ -2671,6 +2671,15 @@ func goexit0(gp *g) {
        gp.labels = nil
        gp.timer = nil
 
+       if gcBlackenEnabled != 0 && gp.gcAssistBytes > 0 {
+               // Flush assist credit to the global pool. This gives
+               // better information to pacing if the application is
+               // rapidly creating an exiting goroutines.
+               scanCredit := int64(gcController.assistWorkPerByte * float64(gp.gcAssistBytes))
+               atomic.Xaddint64(&gcController.bgScanCredit, scanCredit)
+               gp.gcAssistBytes = 0
+       }
+
        // Note that gp's stack scan is now "valid" because it has no
        // stack.
        gp.gcscanvalid = true