Proxy Proxy Infrastructure

Rotating proxy

A rotating proxy is a proxy service that automatically cycles through a pool of IP addresses for each HTTP request or each user session. Instead of your scraper making 10,000 requests from the same IP — which triggers rate limits and bans within minutes — a rotating proxy distributes those requests across thousands or millions of different IPs.

Most managed web scraping APIs (ScraperAPI, Zyte, Bright Data) include automatic proxy rotation. You don’t configure the rotation — you call the API, and the service handles which IP to use for each request.

Per-request vs per-session rotation

Per-request rotation: Each HTTP request uses a new IP from the pool. Best for stateless scraping (product pages, public listings) where each request is independent.

Per-session (sticky) rotation: The same IP is held for a defined time window (e.g., 10 minutes) or until the session ends. Required for any workflow that requires state: login flows, multi-step checkout, cookie-based sessions.

ScraperAPI — sticky session (same IP for session duration) 
# Without sticky session — different IP per request
curl "https://api.scraperapi.com/?api_key=KEY&url=https://example.com"

# With sticky session — same IP for the duration of the session
curl "https://api.scraperapi.com/?api_key=KEY&url=https://example.com&session_number=123"

The session_number parameter tells ScraperAPI to reuse the same IP for all requests with the same session number. This is how you maintain authentication state across multiple page requests in a scraping workflow.

Why rotation matters against anti-bot systems

Modern anti-bot systems track request patterns per IP:

  • More than N requests per minute from a single IP → rate limit or temporary block
  • More than M requests per day from a single IP → hard ban
  • Requests with identical headers and timing patterns → fingerprint block, regardless of IP

A rotating pool of residential IPs addresses the first two. Browser fingerprint management (headers, TLS fingerprint, canvas fingerprint) addresses the third. Most production-grade managed scraping APIs handle all three automatically.

Pool size matters

A rotating proxy pool of 1,000 IPs is insufficient for scraping a major e-commerce site at 100,000 requests/day — the same IPs cycle back quickly and get banned faster than the pool refreshes. Bright Data’s 72 million residential IP pool means IP re-use at scale is minimal, which is why their success rates on hard targets are consistently higher.

Related concepts

→ residential proxy → ip rotation → session proxy → anti bot detection