Amazon Bedrock: "Connection timeout or reset on long-running or idle connections": what it means and how to fix it

What it means

API calls fail with connection resets or timeouts, especially for long-running requests such as streaming, extended thinking, or large inference responses, when traffic goes through NAT Gateways, interface VPC endpoints, or Network Load Balancers.

Causes

• NAT Gateways, interface VPC endpoints, and Network Load Balancers drop TCP connections idle longer than 350 seconds without notifying the client

How to fix it

1. 1
   Fix 1.

   Enable TCP keep-alive in your AWS SDK client.

   Snippet

   Copy

   import boto3
   from botocore.config import Config
   
   config = Config(tcp_keepalive=True)
   client = boto3.client("bedrock-runtime", config=config)

2. 2
   Fix 2.

   Lower the OS-level keep-alive interval to fire before the 350-second idle timeout (both settings are required).

   Snippet

   Copy

   sysctl -w net.ipv4.tcp_keepalive_time=45