Crawl Errors and How to Find Them
Your site has broken links. We show you how to find them using free tools, then explain which ones actually matter for SEO.
Author
Audit Compass Editorial Team
Written by the Audit Compass Editorial Team, focused on clear, honest guidance for website technical health.
What Are Crawl Errors, Really?
When Google's bot visits your website, it follows links from page to page. Sometimes it hits a dead end—a broken link, a page that doesn't exist anymore, or a server that's not responding. That's a crawl error. And yes, they happen to almost every site.
Here's the thing: not all crawl errors hurt your SEO. Some are completely harmless. But if you've got dozens of them, Google wastes time crawling pages that don't help your rankings. It's like having broken roads in a neighborhood—the delivery driver spends time on routes that lead nowhere.
We're going to walk through the main types, show you exactly how to find them, and then explain which ones you actually need to fix.
The Main Types of Crawl Errors
There aren't that many types, which is good news. Once you know what to look for, you'll recognize them quickly.
404 Errors
The page doesn't exist. Google asked for it, but the server said "sorry, nothing here." This happens when you delete a page without setting up a redirect.
Server Errors (5xx)
Your server had a problem when Google tried to access the page. Could be a temporary issue or something bigger. Either way, Google can't read that page right now.
Blocked by Robots.txt
You told Google not to crawl this page in your robots.txt file. Sometimes you mean to. Sometimes it's a mistake.
Redirect Chains
A link goes to page A, which redirects to page B, which redirects to page C. Google doesn't like following long chains. Ideally, one redirect or none.
How to Find Crawl Errors: The Free Tools
You don't need fancy software. Google gives you what you need for free—it's just not always obvious where to look.
Google Search Console
This is the main one. Go to your Google Search Console account, click on "Coverage" in the left menu. You'll see a breakdown: valid pages, pages with warnings, pages with errors. Click on "Errors" and you'll get a list of every page Google couldn't access. It'll tell you whether it's a 404, server error, or something else.
Bing Webmaster Tools
Similar to Google Search Console. Look for "Crawl Issues" in the menu. Bing sometimes finds errors Google misses, so it's worth checking both. You're looking for the same information—which pages Bing couldn't reach.
Screaming Frog (Free Version)
Download it, point it at your website, and it'll crawl every page like Google does. It'll highlight broken links, server errors, redirects. The free version crawls up to 500 pages, which is plenty for most small-to-medium sites.
Which Errors Actually Matter?
This is where people get confused. Not every crawl error is a crisis. Some are actually fine.
Fix These ASAP
- 404s on important pages (like your main service pages or blog posts)
- Server errors on any page Google thinks is important
- Redirect chains longer than 2 redirects
- Broken internal links (links from one of your pages to another of your pages)
Monitor These
- 404s on old blog posts you've deleted (set up a 301 redirect instead)
- Pages blocked by robots.txt that you don't need indexed anyway
- Occasional server errors (might be temporary)
Ignore These
- 404s on URLs that were never pages on your site (Google sometimes tests random URLs)
- 404s on old tracking pixels or third-party resources
- Robots.txt blocks on admin pages or duplicate content
The Fix Strategy: Quick Wins First
Don't try to fix everything at once. Start with what matters most.
Audit Your Broken Links
Use Screaming Frog or Search Console to get a complete list. Sort by which pages have the most errors.
Prioritize High-Impact Pages
Fix errors on pages that get traffic or are important for conversions first. A broken link on your pricing page matters more than one on an old archived post.
Use 301 Redirects (Not 404s)
If a page is gone, redirect it to the closest relevant page. Don't just let it 404. Google passes link value through 301 redirects.
Re-Check in Search Console
After fixing errors, request Google to re-crawl those pages. It takes a few days, but you'll see the errors disappear from your coverage report.
Keep It Simple
Crawl errors aren't mysterious. They're just pages that Google can't reach. Some matter for SEO, most don't. Your job is simple: find them, prioritize the important ones, and fix them with redirects or by restoring the pages. Check Search Console once a month and you'll stay ahead of it. Most sites don't—that's actually why this stuff gives you an advantage.
This article is informational and educational in nature. It's not a substitute for professional SEO consultation. Every website is different, and what works for one site might not apply to yours. If you need help diagnosing specific crawl errors on your site, consider consulting with an SEO professional or reaching out to our team.