What makes a proper 404 page
A page that does not exist must answer with status 404. If the server sends status 200 instead, the search engine takes the error page for real content.
How it works
Every answer from a web server carries a status code. 200 means “here is the page”, 404 “not found”, 410 “gone”. Visitors do not see the code, only the page that comes with it. Search engines, on the other hand, go by the code.
If an address that does not exist answers with 200 and an error message, that is called a soft 404. The search engine then has to guess from the content that it is an error page. If it fails, empty pages end up in the index, and the crawler spends time on addresses with nothing on them. That time is missing for your real pages (this is known as crawl budget).
bytereport calls up an address that exists on no website, follows any redirects and checks the status of the final answer.
What matters
- Status 404 or 410. The error page can be as pretty and helpful as you like. Only the status code has to be right.
- Do not redirect to the start page. Many sites redirect every unknown address to the start page. That ends with status 200 again, and Google treats it as a soft 404 too.
- Redirect when there is a successor. If a page has moved, the old address redirects to the new one with 301. 404 is for addresses with no replacement.
- Helpful for visitors. A good 404 page says clearly that the page is missing and offers search, navigation and a link to the start page.
How to set it up
In most CMSs the status is correct as long as the error page comes through the intended template, such as 404.php in WordPress. Problems usually come from redirect plugins that send every error to the start page, or from single-page apps that answer every address with the same page and status 200. Switch off the catch-all redirect, or have the server send 404 for unknown paths.
In the web server, you set a custom error page so that the status is kept:
Do not give a full address with https:// there, or nginx and Apache turn it into a redirect. A new check shows whether it worked, as does Google Search Console under Indexing · Pages.