It probably doesn't matter to you, but just in case....
both these solutions only 'ask' bots not to crawl
It only works if you're talking about the main search engines.
A private bot or scraper will just ignore the request. As I said, probably unnecessary for most sites as they won't be interested enough in your site to bother looking, but if you want to be ultra-careful, you should probably 'physically' stop bots with something like password protection.
both these solutions only 'ask' bots not to crawl
It only works if you're talking about the main search engines.
A private bot or scraper will just ignore the request. As I said, probably unnecessary for most sites as they won't be interested enough in your site to bother looking, but if you want to be ultra-careful, you should probably 'physically' stop bots with something like password protection.

