Google Explains How 304 (Not Modified) HTTP Status Codes Can Go Very Wrong

Gary Illyes from Google posted how serving a 304 HTTP server status code can “backfire spectacularly,” even though the 304 can be a super useful signal for search engines.

Gary Illyes wrote on LinkedIn, “HTTP 304 (not modified) is super useful to signal crawlers that the content they’re accessing hasn’t changed since it was last crawled, but it can also backfire spectacularly.”

Gary then listed the steps on how this can happen:

• Crawler requests URL
• Server encounters some sort of error and serves an empty page with a 200 (ok) HTTP status code
• Crawler considers that as a transient soft error, schedules a recrawl to verify
• Since the content (that was not actually served) hasn’t changed, the server returns HTTP 304 for the particular URL, without content (as mandated by the HTTP standard RFC 9110)
• The crawler “learns” the error is persistable and doesn’t try to recrawl anymore (it will eventually, but holding your breath is not recommended)

In short, if you have a server or HTML issue where you serve Google a blank page or wrong content and then you tell Google not to come back with a 304 HTTP server status code, that can be bad.

Forum discussion at LinkedIn.