Why Do Robots Visit My Website?

What is a Visiting Robot (Or Bot)?

A robot is a software application designed to do a specific job when visiting a website. Also know as a crawler, spider or bot, a robot can be good news, or not so good news. Helpful bots include those from search engines such as Google, which index your website and then include it in their search results. Unhelpful bots are those which post spam messages or copy your content.
In this article, we look at how to identify if a bot has been to your site and if you should do anything about it.

How to Identify Bot Traffic

As discussed in our article on understanding and using AWStats, information about visiting robots is one of the statistics AWStats provide. This can include the names of visiting bots, the dates they visited as well as the number of pages they visited. AWStats works to identify bots by:
  • their originating IP-address

  • the speed at which pages are requested

  • fast or slow visit lengths

  • high bounce rates

  •  spam form filling

  • traffic spikes from unusual locations

  • sequence they move through a website.

However, some bots pretend to be humans, and as such are not counted as bot traffic. This means that unless a bot is specifically identified as such, your site visit statistics can be somewhat skewed.

Should I Be Worried About Robot Visits?

The short answer is no. Our servers are highly effective at managing bot access to the websites we host. As such they work towards blocking the spam or unhelpful bots, but let through the good bots from search engines. This means you do not need to add any robot.txt files, or other methods to manage bot visits - it is all automatically done for you.

Tags: seo  

Posted: Friday 17 January 2020


 


Menu Sitemap
Categories