Google rolled out a new crawler named Google-CloudVertexBot and this bot aims to crawl for clients of a commercial nature who use Vertex AI.

The Google-CloudVertexBot added to Google’s crawler documentation serves to crawl sites only at the request of the site owners. This crawler also differentiates itself from other listed crawlers in Search Central that work with advertising or in by Google Search.

Google-CloudVertexBot Crawler: More on this crawler

It has been stated that in Vertex AI Agent Builder, different types of data stores have been contained. Furthermore, Google says a data store can contain a single data type. There are around six types of data and two types of site crawling. Advanced Website Indexing and Baisc Website Indexing are the two types of site crawling in Google-CloudVertexBot.

Furthermore, the Changelog also states that the Google-CloudVertexBot has been listed among Google crawlers to assist website owners in identifying ‘new crawler traffic’. The Google documentation also hints that crawlers do not index public websites. However, the changelog says otherwise. It states that website owners will be able to identify new crawler traffic.

Besides the new crawler introduction, Google recently launched its August 2024 core updates. The August 2024 Core updates are meant to foster functional content from smaller and independent publishers. The update has been stated to take a full month to roll out fully. It is targeted towards improving the search result quality by allowing people to look at ‘useful’ content.

Suggested:

Google August 2024 Core Update Has Been Released.

Enhancing SEO with Cross-Domain Canonicals: Google’s New Guidelines.