Guides
How to Check If Google Can Index Your Website
A practical guide to checking indexing: how to tell if a page is in Google, the four signals that control it, and what to do when a page is crawled but not indexed.
Run a fresh DomainLens audit and use the report as your priority list.
How to tell if a page is actually indexed
Indexing and ranking are not the same thing. Indexing is whether Google has stored the page at all; ranking is where it appears once indexed. A page can be perfectly optimised and still get zero traffic if it never made it into the index in the first place.
The fastest check is the site: operator — search Google for site:yourdomain.com/your-page. If the URL appears, it is indexed. For an authoritative answer, use the URL Inspection tool in Google Search Console, which reports the exact index status, the canonical Google chose, and any reason it was excluded.
If a page is missing, the cause is almost always one of a small set of signals that tell Google not to index it — or a quality issue that makes Google choose not to.
The four signals that control indexing
- robots.txt — if the URL is disallowed, Google may not crawl it (and an uncrawled page usually cannot be indexed).
- robots meta tag — a noindex in the page HTML removes it from the index even if it is crawled.
- X-Robots-Tag — the same noindex instruction delivered as an HTTP header; easy to miss because it is not in the HTML.
- Canonical URL — if the canonical points to a different URL, Google indexes that one instead of the page you are checking.
Step by step: check one URL
- Search site:yourdomain.com/path to see if the URL is already in the index.
- Open the URL in Search Console URL Inspection for the definitive index status and chosen canonical.
- Fetch the page as a bot and confirm the rendered HTML has no noindex meta tag.
- Check the HTTP response headers for an X-Robots-Tag: noindex.
- Confirm robots.txt does not disallow the path, and that the canonical points to this URL.
Why a page gets crawled but not indexed
Sometimes Google crawls a page and still chooses not to index it. In Search Console this shows up as "Crawled — currently not indexed" or "Discovered — currently not indexed". The directives are fine; Google has simply decided the page is not worth storing.
This is usually a quality signal: the content is thin, near-duplicate of other templates, or the site is new and has not earned enough trust. The fix is not technical — it is making the page genuinely more useful and unique, and strengthening the site overall.
Common indexing mistakes
- Leaving a staging noindex tag on the production site after launch.
- Blocking a page in robots.txt and expecting noindex to still work — Google cannot read a noindex on a page it is not allowed to crawl.
- Setting a canonical that points to the homepage or another URL by mistake.
- Assuming the page is broken when it is simply too new or too thin to be indexed yet.
How DomainLens helps
DomainLens checks the indexability signals for you in one pass: it reads the robots meta tag, the X-Robots-Tag header, and the canonical URL on the live page, and flags when they would keep the page out of the index.
It cannot see your Search Console data, so pair it with URL Inspection for the final index status — but it catches the common technical blockers before you even open Search Console.