Do not retry an HTTP 408; lock retry cap and timeout values
The OpenAI and Anthropic adapters now report an HTTP 408 as llm.TimeoutStatusError, which still matches llm.ErrTimeout but carries its status. Retrying no longer retries any error carrying a 4xx status, so a vendor's 408 is returned at once while a transport timeout, which has no status, is still retried. Tests: 408 (bare and wrapped) is not retried and a transport timeout is; FromSettings integration cases for an OpenAI and Anthropic 408 and an OpenAI transport timeout; the cap and default-timeout tests assert the literal 20 s and 60 s instead of the constants they lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -22,11 +22,12 @@ const (
|
||||
)
|
||||
|
||||
// Retrying is a Provider that retries transient Generate failures of the
|
||||
// Provider it wraps. It retries ErrRateLimited, ErrTimeout and a 5xx
|
||||
// ErrProvider at most MaxRetries times, and nothing else. A rate-limited
|
||||
// retry waits for the vendor's RetryAfter, capped at MaxRetryAfter; every
|
||||
// other retry, and a rate limit that names no wait, waits a jittered
|
||||
// exponential backoff. The Request's Timeout, or DefaultTimeout, is one
|
||||
// Provider it wraps. It retries ErrRateLimited, a transport ErrTimeout and
|
||||
// a 5xx ErrProvider at most MaxRetries times, and nothing else: an error
|
||||
// carrying a 4xx HTTP status other than a rate limit, such as an HTTP 408
|
||||
// TimeoutStatusError, is not retried. A rate-limited retry waits for the
|
||||
// vendor's RetryAfter, capped at MaxRetryAfter; every other retry, and a
|
||||
// rate limit that names no wait, waits a jittered exponential backoff. The Request's Timeout, or DefaultTimeout, is one
|
||||
// deadline for the whole call, retries and waits included; a caller's
|
||||
// earlier context deadline wins. A retry whose wait would not end before
|
||||
// that deadline is not made: the last error is returned at once.
|
||||
@@ -91,6 +92,10 @@ func (r *Retrying) Generate(ctx context.Context, req Request) (Response, error)
|
||||
// wait returns how long to wait before retrying after err, the retry-th
|
||||
// retry counting from 0, and false when err is not retried.
|
||||
func (r *Retrying) wait(err error, retry int) (time.Duration, bool) {
|
||||
var st interface{ Status() int }
|
||||
if errors.As(err, &st) && st.Status() >= 400 && st.Status() <= 499 {
|
||||
return 0, false
|
||||
}
|
||||
var rl *RateLimitedError
|
||||
if errors.As(err, &rl) && rl.RetryAfter > 0 {
|
||||
return min(rl.RetryAfter, MaxRetryAfter), true
|
||||
|
||||
Reference in New Issue
Block a user