Anything that can be displayed on a screen can be scraped.
An approach I used to prevent scraping in the past is to start rate limiting anything that hits over N pageviews in an hour, where N is a value around what a high-use user could manually consume. Start with a small delay and increment it with each pageview (excess hits*100ms), then send HTTP 509 (Exceeded Bandwidth) for anything that is clearly hammering the server (or start returning junk data if you're feeling vengeful).
Added bonus is that the crawler will appear to function correctly during testing until they try to do a full production run and run into the (previously undetectable) rate limiting.
This project did not require search indexing so we didn't care about legit searchbots, but you could exclude known Google/Bing crawlers and log IPs of anything that hits the limit for manual whitelisting (or blacklisting of repeat offenders).
An approach I used to prevent scraping in the past is to start rate limiting anything that hits over N pageviews in an hour, where N is a value around what a high-use user could manually consume. Start with a small delay and increment it with each pageview (excess hits*100ms), then send HTTP 509 (Exceeded Bandwidth) for anything that is clearly hammering the server (or start returning junk data if you're feeling vengeful).
Added bonus is that the crawler will appear to function correctly during testing until they try to do a full production run and run into the (previously undetectable) rate limiting.
This project did not require search indexing so we didn't care about legit searchbots, but you could exclude known Google/Bing crawlers and log IPs of anything that hits the limit for manual whitelisting (or blacklisting of repeat offenders).