]> Cypherpunks repositories - gostls13.git/commitdiff
cmd/api: reduce parallel 'go list' invocations to a constant
authorBryan C. Mills <bcmills@google.com>
Wed, 26 Jan 2022 17:45:12 +0000 (12:45 -0500)
committerBryan Mills <bcmills@google.com>
Wed, 26 Jan 2022 19:17:30 +0000 (19:17 +0000)
'go list' has its own internal parallelism, so invoking in in parallel
can produce up to quadratic peak memory usage.

Running 'go list' is also very I/O-intensive, so the higher
parallelism does substantially improve latency; unfortunately, we lack
a good way to balance latency against memory footprint, so we need to
sacrifice some latency for reliability.

Fixes #49957.

Change-Id: Ib53990b46acf4cc67a9141644d97282964d6442d
Reviewed-on: https://go-review.googlesource.com/c/go/+/380994
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
src/cmd/api/goapi.go

index 036aefe4d8b3e280f07cdbcb399243147d8993cc..5ae059e4cecdc42cae4934c5ba021f1fcb3921de 100644 (file)
@@ -459,8 +459,11 @@ type listImports struct {
 
 var listCache sync.Map // map[string]listImports, keyed by contextName
 
-// listSem is a semaphore restricting concurrent invocations of 'go list'.
-var listSem = make(chan semToken, ((runtime.GOMAXPROCS(0)-1)/2)+1)
+// listSem is a semaphore restricting concurrent invocations of 'go list'. 'go
+// list' has its own internal concurrency, so we use a hard-coded constant (to
+// allow the I/O-intensive phases of 'go list' to overlap) instead of scaling
+// all the way up to GOMAXPROCS.
+var listSem = make(chan semToken, 2)
 
 type semToken struct{}