Well, the first question we should be asking is not ‘why do the bots crawl sites?’, but rather what exactly is a bot? Secondly, this system that systematically travels...