liblink: check for symgrow size too large

liblink: check for symgrow size too large

Many calls to symgrow pass a vlong value.  Change the function
to not implicitly truncate, and to instead give an error if
the value is too large.

R=golang-codereviews, gobot, rsc
CC=golang-codereviews
https://golang.org/cl/54010043