A server log shows that Googlebot requested a URL. A browser displays the page correctly. Search Console still reports that the URL is not indexed. These observations can all be true: each describes a different part of processing a page.
Treating “Google saw it” as a single state hides the question that matters: what was fetched, what could be processed, and which URL was ultimately selected for indexing? The examples here are conceptual cases, not measurements from a live experiment.
Discovery is knowledge of a URL
A URL can be discovered through links or a sitemap before its content is fetched. Discovery does not establish that the server will respond successfully, that crawling is permitted or that the page is worth indexing. A sitemap is a discovery aid rather than an instruction that guarantees inclusion.
When a new page seems invisible, check that its intended URL is linked from useful, accessible pages and that the link actually points to the final destination. A URL known only to an internal search form has a different discovery path from one linked in a resource hub.
Crawling retrieves a response; rendering processes the page
A fetch can return HTML, a redirect, an error or another resource type. Record the status code and response body instead of assuming every successful request delivered the intended content. A 200 response containing an empty shell or an error message is not equivalent to a complete article.
Rendering can execute JavaScript and build a representation of the page. Important content may depend on scripts, API responses or resources that are unavailable to a crawler. Your logged-in browser, with cached assets and cookies, is not a reliable substitute for inspecting what the search system could access.
Google documents crawling, rendering and indexing as related processing stages in its JavaScript SEO guidance. Rendering does not provide a separate promise of indexing, and a fresh browser view does not establish what an earlier crawl contained.
Indexing involves evaluation and canonical selection
A processed page may remain outside the index. It may be excluded by a directive, overlap with another URL or fail to be selected for other reasons. Canonicalization concerns which representative URL a search engine selects among duplicate or substantially similar pages. Your declared canonical is a signal; it is not proof of the selected canonical.
Inspect the selected canonical and the page evidence before rewriting content. If a parameter URL points to a clean version and the clean version is the intended indexed page, the parameter URL’s exclusion can be expected behavior. If distinct pages unexpectedly consolidate, investigate their content and canonical signals.
Do not interchange robots.txt, noindex and canonical
| Control | Main purpose | Common diagnostic mistake |
|---|---|---|
| robots.txt | Controls permitted crawling paths for compliant crawlers | Assuming a blocked URL can never appear in search |
| noindex | Requests exclusion from indexing when the directive can be read | Blocking crawling and expecting the crawler to read a new noindex |
| Canonical declaration | Indicates a preferred representative for similar content | Treating the declaration as a guaranteed selection |
Google’s noindex documentation explains why the crawler must be able to access the page to see that rule. For canonical issues, compare the declaration with the Google-selected canonical. These controls solve different problems.
Use Search Console as evidence with timestamps
Start with URL Inspection for the exact URL. Separate the indexed-state report from a live test. A live test helps investigate current accessibility; it does not prove that the page is in the index or guarantee inclusion after a request.
- Confirm the intended URL and its current response.
- Inspect crawl restrictions and indexing directives.
- Check the rendered content and required resources where relevant.
- Compare declared and selected canonicals.
- Read the reported indexing reason alongside the crawl date.
- Retest the mechanism after fixing it, then allow time for reprocessing.
Keep a record of the response, test date and affected template. “Crawled, currently not indexed” is a reported state that calls for investigation, not a complete diagnosis or evidence of a penalty. Use the broader diagnostic framework when indexing is only one part of the problem, and the glossary when terms in different tools appear to conflict.