🤖NEW:AI-Powered Incremental Builds — your site updates in under 30 seconds. See what's new →
← All Tools
100% Free • Indexation Auditor

Google Indexation Checker

Audit webpage indexability by scanning meta robots noindex tags, X-Robots-Tag HTTP headers, canonical alignment, and live search index links.

Technical Deep-Dive

Google Indexation Architecture & Crawl Selection Criteria

Last updated: August 2026 • Reviewed by Nimbica Technical SEO Team

1. How Google Selects Pages for Search Indexation

Googlebot crawls pages to decide search index placement. As documented in Google Search Central Indexing Documentation, clean indexability signals prevent pages from dropping out of search results.

2. Meta Robots vs. X-Robots-Tag Headers

`noindex` directives instruct Googlebot not to index a URL, whereas `noarchive` prevents Google from serving cached web page snapshots.

3. Auditing Overall Website Crawl Health

Audit crawl health in our Crawlability Analyzer.

4. Instant Indexation via Nimbica Static Edge

Nimbica pre-renders static HTML snapshots served from 300+ global edge locations, ensuring Googlebot fetches indexable HTML with sub-50ms TTFB.

5. What This Checker Actually Verifies

Enter a URL and the checker fetches the live page and inspects three on-page signals in order: the HTTP response status code, the <meta name="robots"> tag content for a noindex directive, and whether the page's rel="canonical" link is self-referential (points back to the URL you entered) or points elsewhere. A page is flagged fully indexable only when it returns HTTP 200, carries no noindex directive, and has an aligned (or absent) canonical tag. It also builds a one-click site: search link so you can cross-check the result against Google's live index as a secondary signal.

6. Worked Example

Checking a URL that returns HTTP 200, has <meta name="robots" content="index, follow">, and a canonical tag equal to its own URL produces: Meta Robots Directive: Index Allowed, Canonical URL Alignment: Self-Referential / Aligned, and an overall "Page is Fully Indexable" result. Now compare a staging-clone URL that still carries <meta name="robots" content="noindex, follow"> left over from a pre-launch checklist — the same checker returns NOINDEX BLOCK PRESENT and an overall "Page Indexation Blocked" result, even though the page loads fine in a browser. This is the exact class of silent bug (a forgotten staging noindex tag shipped to production) that this tool is built to catch quickly.

7. Google Search Console Index Coverage States

This tool checks on-page signals directly; Search Console's Page Indexing report is the authoritative record of what Google has actually done with a URL. Common states you'll see there include: Indexed, Crawled – currently not indexed (Google visited but chose not to index, often a quality signal), Discovered – currently not indexed (known but not yet crawled, often a crawl-budget issue), Excluded by 'noindex' tag, Duplicate without user-selected canonical, Alternate page with proper canonical tag, Page with redirect, Not found (404), and Blocked by robots.txt. A page that passes every check in this tool can still show as "Crawled – currently not indexed" in Search Console, since that state reflects a quality or content-value judgment Google makes independently of on-page directives.

8. Practical Use Cases

  • Post-launch QA: confirming a staging environment's blanket noindex tag was actually removed before a site relaunch goes live.
  • Publisher spot-checks: news and content sites verifying a freshly published article is indexable within minutes of publishing, before waiting on Search Console's reporting lag.
  • E-commerce catalog audits: spot-checking whether out-of-stock or seasonal product pages were accidentally left with a noindex tag applied by an automated inventory rule.
  • Agency onboarding audits: quickly scanning a new client's key landing pages for indexability blockers before a deeper technical SEO audit begins.

9. Common Mistakes & Limitations

  • Treating site: search as definitive: Google has repeatedly stated that site: operator result counts and presence are an approximate, unreliable signal — useful as a quick sanity check, never as proof of index status. Search Console's URL Inspection tool is the authoritative source.
  • Confusing "indexable" with "indexed": a page with clean signals (200 status, no noindex, aligned canonical) is eligible for indexing, not guaranteed to be indexed. Google still applies independent quality and crawl-budget decisions.
  • Missing X-Robots-Tag HTTP header directives: a noindex instruction can also be delivered via an X-Robots-Tag response header rather than an HTML meta tag — commonly used for PDFs, images, and other non-HTML resources that have no <head> to place a meta tag in. Always check response headers separately for non-HTML file types.
  • Client-side rendered noindex tags: if a JavaScript framework injects the robots meta tag after initial page load, a raw HTML fetch (as used here) may not see it the same way a rendering crawler does.
  • Non-self-referential canonicals aren't always errors: a canonical pointing elsewhere is expected and correct on intentional duplicate/paginated/parameterized pages — it only signals a problem when the target page is unrelated or a mistake.

Give search engine crawlers sub-50ms response times with Nimbica

Transform dynamic PHP rendering bottlenecks into ultra-fast static HTML deployed across 300+ global edge locations.

Frequently Asked Questions

What makes a webpage indexable by Google Search?

A webpage is indexable if it returns an HTTP 200 OK status code, does not contain a `noindex` meta tag or header, and has a self-referential canonical URL tag.

How does site:URL search verification work?

Searching `site:https://yourdomain.com/page` directly in Google confirms whether the URL is currently present in Google's live search index.

What happens if a page has a noindex directive?

A `noindex` meta tag instructs search crawlers not to include the webpage in search result listings.

Is a passing result from this tool a guarantee the page is indexed?

No. A clean result (200 status, no noindex, aligned canonical) means the page is eligible for indexing, not that Google has indexed it. Google still applies independent crawl-budget and content-quality decisions on top of these signals.

What are the common index coverage states in Google Search Console?

Common states include Indexed, Crawled - currently not indexed, Discovered - currently not indexed, Excluded by noindex tag, Duplicate without user-selected canonical, Page with redirect, Not found (404), and Blocked by robots.txt. Search Console is the authoritative source for a URL's real index status.

Can a noindex directive be applied outside the HTML meta tag?

Yes. Noindex can also be delivered via an X-Robots-Tag HTTP response header, which is the required method for non-HTML files like PDFs and images that have no HTML head to place a meta tag in.

Is Google's site: search operator a reliable way to check indexation?

It is a useful quick sanity check, but Google has repeatedly confirmed that site: search result counts and presence are approximate, not a definitive record. The URL Inspection tool in Search Console is the authoritative source.