Google seems to know handle or obey the RateLimit Header Fields for HTTP. Mike Blazer asked John Mueller from Google about this and John said he never heard of it, so he assumes Google Search does not handle it.
Mike Blazer asked, “Is Google respecting the “x-ratelimit-limit” HTTP header when crawling websites?”
John Mueller replied, “I’ve never heard of it, so I guess that’s an answer :-). We document HTTP 429 & 503 for requesting slow-downs.”
Here are those posts on Mastodon:
You can see the full help document on what Google supports with these headers over here.
Forum discussion at Mastodon.
Source link : Seroundtable.com
