Web Scraping Glossary

60 terms, developer-level definitions. Every entry cross-links to relevant tool reviews, comparisons, and how-to guides. No fluff.

A

Anti-bot

Anti-bot detection

Anti-bot detection systems identify and block automated web scraping traffic using IP reputation, browser fingerprinting, behavioral analysis, and CAPTCHA challenges.

→ Definition

C

Anti-bot

CAPTCHA solver

A CAPTCHA solver is a service or technique that automatically completes CAPTCHA challenges during web scraping — either through AI image recognition, audio transcription, or human worker farms.

→ Definition

R

Proxy

Residential proxy

A residential proxy routes your requests through an IP address assigned by an ISP to a real home or mobile device — making your scraper look like a regular user, not a datacenter bot.

→ Definition

Proxy

Rotating proxy

A rotating proxy automatically changes the IP address used for each HTTP request (or each session), distributing scraping traffic across a pool of IPs to avoid rate-limiting and IP bans.

→ Definition

W

Tools

Web scraping API

A web scraping API is a managed HTTP service that handles proxy rotation, CAPTCHA solving, and browser rendering on your behalf — you send a URL, it returns the page HTML or structured data.

→ Definition