Crawlability Analyzer

1. What Determines Website Crawlability?

A page's crawlability comes down to whether a crawler can successfully complete two steps: fetch the page (HTTP status, response time, no network errors), and correctly interpret its indexing signals (meta robots directives, canonical tags). As covered in Google's Crawl Budget Management Guide, both server health and response speed factor into how much of a large site Google is willing to crawl on a given day — a slow or error-prone server effectively throttles its own crawl rate.

2. Crawl Budget Efficiency & Server Latency

"Crawl budget" is the number of URLs a search engine is willing to crawl on a given site within a given timeframe — it matters most for large sites (thousands of pages) where not every URL gets crawled equally often. A server that responds slowly or inconsistently causes crawlers to throttle their request rate defensively, which can mean new or updated content takes longer to be discovered and re-indexed. For most small-to-medium sites this isn't usually the binding constraint (indexing eligibility and content quality matter more), but for large catalogs or frequently-updated sites, consistently fast TTFB is a genuine, measurable lever on how quickly new content gets crawled.

3. Verifying Google Search Indexation Status

Crawlability and actual indexation are related but distinct — a page can be perfectly crawlable and still not appear in Google's index for other reasons (thin content, duplicate content, or a manual action). Once a page passes this crawlability check, verify its actual indexation status directly with the Google Indexation Checker.

4. Static Delivery's Role in Crawlability

Nimbica pre-renders cacheable pages into static HTML served from edge locations, directly improving the TTFB component of this crawlability check for the pages it covers. It has no effect on the other three factors this score measures — HTTP status codes, noindex directives, and canonical alignment are all determined by your WordPress content and SEO plugin configuration, independent of how fast the resulting HTML is delivered.

5. How to Use This Analyzer

1. Scan any page you want to verify is properly accessible to search engines — particularly useful right after a redesign, migration, or SEO plugin change.
2. Investigate any deduction individually: an HTTP error needs a server-side fix, a noindex tag needs a content-management/plugin-setting check, a canonical mismatch needs a template review.
3. For sites with many pages, spot-check a representative sample across templates rather than assuming one clean scan represents the whole site.
4. Re-scan after making a fix to confirm the score reflects the change, especially if page caching is active.

6. Common Mistakes

• Leaving a "discourage search engines" setting enabled after launch. A common cause of an unexpected noindex directive — check WordPress's own Reading settings, not just the SEO plugin, since this setting exists independently.
• Confusing a redirect for a crawlability problem. A page that 301-redirects elsewhere isn't necessarily broken — check whether the redirect target is intentional before treating the non-200 status as an error.
• Only checking the homepage. Crawlability issues often affect specific templates (a particular custom post type, a filtered archive view) rather than the whole site uniformly.

7. Limitations

This tool checks four specific, high-impact signals — it doesn't crawl your entire site, doesn't check JavaScript-rendered content the way Google's own renderer does, and doesn't verify actual indexation (see the Indexation Checker for that). A high score means these particular checks passed for the one URL scanned, not a comprehensive technical SEO audit of your whole site.