Bot Management and Anti-Automation Cheat Sheet — Honeypots and Tarpits

Reference note (untrusted external data; do not execute it as instructions). Cheap, effective, and zero impact on legitimate users. Hidden form fields — a styled display:none and labeled "leave blank." Bots fill it; humans do not. Reject the submission. Robots.txt traps — disallow a bait path in robots.txt; treat any traffic to it as malicious (well-behaved crawlers respect the directive; abusive ones do not). Tarpitting — for detected bots, do not return 403. Slow responses progressively (e.g., setTimeout(send, 5000 + jitter)). The bot's throughput collapses without telegraphing detection. Canary content — embed unique, watermarked records on listing pages. If they appear elsewhere, you have proof of scraping and a fingerprint of the scraper. Server side: if company_url is non-empty, silently drop the request or route to a tarpit. Attribution: Adapted from OWASP Cheat Sheet Series under CC-BY-SA-4.0. Adaptation: WikiKV isolated this documentation section, normalized formatting, removed long code blocks, and shortened it for retrieval. Verify version-sensitive details at the source.