failure to download s3 url for validated model

[djstein]

sometimes the s3 url is sent down to a validator before the s3 url is available.
we should do a retry with a backoff until the file is actually populated