archive/zip: improve BenchmarkCompressedZipGarbage
Before this CL:
$ go test -bench=CompressedZipGarbage -count=5 -run=NONE archive/zip
BenchmarkCompressedZipGarbage-8 50
20677087 ns/op 42973 B/op 47 allocs/op
BenchmarkCompressedZipGarbage-8 100
20584764 ns/op 24294 B/op 47 allocs/op
BenchmarkCompressedZipGarbage-8 50
20859221 ns/op 42973 B/op 47 allocs/op
BenchmarkCompressedZipGarbage-8 100
20901176 ns/op 24294 B/op 47 allocs/op
BenchmarkCompressedZipGarbage-8 50
21282409 ns/op 42973 B/op 47 allocs/op
The B/op number is effectively meaningless. There
is a surprisingly large one-time cost that gets
divided by the number of iterations that your
machine can get through in a second.
This CL discards the first run, which helps.
It is not a panacea. Running with -benchtime=10s
will allow the sync.Pool to be emptied,
which brings the problem back.
However, since there are more iterations to divide
the cost through, it’s not quite as bad,
and running with a high benchtime is rare.
This CL changes the meaning of the B/op number,
which is unfortunate, since it won’t have the
same order of magnitude as previous Go versions.
But it wasn’t really comparable before anyway,
since it didn’t have any reliable meaning at all.
After this CL:
$ go test -bench=CompressedZipGarbage -count=5 -run=NONE archive/zip
BenchmarkCompressedZipGarbage-8 100
20881890 ns/op 5616 B/op 47 allocs/op
BenchmarkCompressedZipGarbage-8 50
20622757 ns/op 5616 B/op 47 allocs/op
BenchmarkCompressedZipGarbage-8 50
20628193 ns/op 5616 B/op 47 allocs/op
BenchmarkCompressedZipGarbage-8 100
20756612 ns/op 5616 B/op 47 allocs/op
BenchmarkCompressedZipGarbage-8 100
20639774 ns/op 5616 B/op 47 allocs/op
Change-Id: Iedee04f39328974c7fa272a6113d423e7ffce50f
Reviewed-on: https://go-review.googlesource.com/22585
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>