net: avoid transiting durations through floats

net: avoid transiting durations through floats

This slightly simplified the code. I stumbled upon this when support was
being added to Fuchsia (and this pattern was initially cargo-culted).

Change-Id: Ica090a118a0056c5c1b51697691bc7308f0d424a
Reviewed-on: https://go-review.googlesource.com/c/go/+/177878
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>