Measure Your Crawl Cycle

Another important metric for search engine optimization is the frequency of search engine visits and crawls website content. The rate of the crawling can be quantified by crawl cycle (CC) which present the time between two crawls.

SEOmeter.com provides a tool for webmasters in order to monitor search engine’s crawling on their website. Their tool will record each cache update events for each website listed on them and display the history of cache update in a user friendly graph. Currently SEOmeter.com monitors Google only, since only Google who allows public access to the caching timestamp among other major search engine.

here are many benefits by using this tool. Webmasters can show to public how their website is trusted by search engine (by crawling frequency) while SEO marketers can measure their marketing campaigns (by checking the historical trend of search engine crawling rate) of their clients’ site. The detection and analysis any problems related to robot’s crawling is another benefit.

Here is a sample of Crawl Cycle for Google.com

The history graph visualizes how the crawl cycle (CC) of a given website changes over time. The x-axis of each dot represents the timestamp of search engine’s crawling event. The y-axis of the dot is a past one-month average of crawl cycle from that date. So for example, CC=2 on Dec. 20 means that between 11/20 and 12/20, search engine crawled the given website once every other day on average.

For webmasters who want to ramp up the search engine’s crawling rate, SEOmeter.com also shares their tips that describes what determines search engine’s crawl rate.