The Noindex Tag: When to Use It and How to Check It

Short answer: The noindex directive tells search engines not to show a page in their results. You add it as a <meta name="robots" content="noindex"> tag in the page head or as an X-Robots-Tag: noindex HTTP header, and the page must remain crawlable so search engines can see it. Use it deliberately for pages with no search value, and check your site regularly for accidental noindex tags, which are one of the most common reasons pages disappear from search.

What noindex does

Search engines work in stages. They discover a URL, crawl it, and then decide whether to store it in their index so it can appear in search results. The noindex directive steps in at that last stage. It says: “you may read this page, but please do not show it in search.”

When a search engine crawls a page and finds noindex, it drops the page from its index, or never adds it in the first place. Links on the page can still be followed, unless you also add nofollow, and visitors can still reach the page normally through your menu, a bookmark or a direct link. Noindex affects search visibility only. It does not hide the page from people and it does not protect anything.

This makes noindex a precise tool. It lets you keep useful pages for visitors, such as a thank-you page after a form submission, without cluttering search results with pages nobody would search for.

Two ways to add noindex

1. The robots meta tag. The most common method is a tag placed inside the <head> of the HTML page:

<meta name="robots" content="noindex">

This applies to all search engines. You can target one crawler with its name, for example <meta name="googlebot" content="noindex">, although that is rarely needed. In practice, most site owners never type this tag by hand. SEO plugins and website builders add it when you tick a box such as “Hide from search engines” or “Allow search engines to show this page: No”.

2. The X-Robots-Tag HTTP header. The same instruction can be sent in the server’s response headers:

X-Robots-Tag: noindex

The header is the only option for files that have no HTML head, such as PDFs, images or plain text files. It is also useful for applying noindex to whole folders or file types through server configuration. Google’s documentation on the noindex directive describes both methods.

Both methods have the same effect. Use whichever your platform supports, but avoid sending conflicting instructions, such as an index meta tag and a noindex header on the same page. When directives conflict, search engines follow the more restrictive one.

The crawling trap: never block noindex pages in robots.txt

This is the single most important rule about noindex. Search engines can only obey a noindex instruction if they are allowed to crawl the page and read it.

If you add noindex to a page and also block that page in robots.txt, the crawler never fetches the page and never sees the noindex. If the URL is linked from anywhere, it may stay in the index, sometimes shown with the note that no information is available. Search Console reports such pages as “Indexed, though blocked by robots.txt”.

The correct sequence for removing a page from search is:

1. Add noindex to the page.
2. Make sure robots.txt allows the page to be crawled.
3. Wait for search engines to recrawl it, or request indexing in Search Console to speed it up.
4. Only once the page has dropped out of the index, consider blocking it in robots.txt if you also want to save crawl effort. For most small sites, this last step is unnecessary.

Pages that usually should be noindex

Noindex is right for pages that are useful to visitors but make poor search results. Typical examples:

• Thank-you and confirmation pages shown after a form, order or sign-up.
• Internal search results, which can create unlimited thin pages.
• Cart, checkout and account pages.
• Login, password reset and registration pages, unless people genuinely search for them.
• Thin archive pages, such as tag pages with one post, date archives or author archives on single-author blogs. Some sites choose to keep useful tag pages indexed, so decide based on their content.
• Staging, test and development sites, ideally protected with a password as well.
• Campaign landing pages meant only for paid ads or e-mail, with content duplicated elsewhere.
• Legal boilerplate duplicated across many URLs, if your platform creates it.

Noindex is not the right tool for duplicate versions of an indexable page, such as the same product under two URLs. For those, a canonical tag or a redirect tells search engines which version to keep, and passes signals to it, while noindex would simply discard the duplicate.

Pages that should never be noindex

An accidental noindex on an important page silently removes it from search. Pages that must be indexable include:

• the home page;
• service, product and category pages;
• location pages for local businesses;
• blog posts and guides you want people to find;
• contact and about pages, which people often search for by brand name.

It sounds obvious, but a surprising number of audits find noindex on exactly these pages. The page looks perfect in the browser, visitors see nothing wrong, and only the traffic graph tells the story weeks later.

How accidental noindex happens

Most accidental noindex tags come from a small number of causes:

• Launch leftovers. The site was built with search engines discouraged, and nobody switched it back at launch. In WordPress, this is the “Discourage search engines from indexing this site” option under Settings, Reading.
• Staging copied to live. A staging environment configured with noindex headers is copied over the live site, including its server configuration.
• Plugin defaults. Switching or updating an SEO plugin can change which content types are indexable, for example noindexing all products or all pages of a certain type.
• Template changes. A developer adds noindex to a template for one purpose and it is applied to more pages than intended.
• Bulk edits. A bulk action in a plugin or builder sets search visibility for many pages at once.

The common thread is that noindex is invisible to people browsing the site. That is why it needs to be checked deliberately.

How to check for noindex

There are several quick ways to see whether a page is noindex:

1. View the page source. Search the HTML for noindex. If you find it inside a robots meta tag, the page is excluded.
2. Check the response headers. In the browser’s developer tools, open the Network tab, reload, click the page request and look for X-Robots-Tag in the response headers.
3. Use URL Inspection in Search Console. It tells you whether Google considers the page indexable and, if not, whether a noindex tag is the reason.
4. Read the page indexing report. The “Excluded by ‘noindex’ tag” group lists every URL Google found with noindex. Scan it for pages that should be indexed.
5. Crawl the whole site. A crawler collects robots directives for every page, making it easy to spot an important page among hundreds of intentional exclusions.

Make the check part of every launch, redesign, plugin change and migration. It takes minutes and prevents weeks of lost visibility.

How long noindex takes to work, and how to undo it

Adding noindex does not remove a page instantly. It takes effect when the page is recrawled, which may be days for important pages and weeks for rarely visited ones. If you need a page out of Google’s results urgently, the Removals tool in Search Console hides it temporarily while the noindex takes effect.

Undoing an accidental noindex follows the same logic in reverse. Remove the directive, make sure the page is crawlable and listed in your sitemap, and request indexing in Search Console for the most important URLs. Rankings often recover once the page is reindexed, but there is no guarantee that it returns to exactly the same position, especially if it was missing for a long time.

How Site AI Audit helps

Site AI Audit crawls your site like a search engine and its SEO section checks the sitemap, robots.txt and noindex rules, listing everything that keeps pages out of search results. When an important page carries noindex, the report shows which pages are affected and how to fix it in plain words. Run a free check to make sure nothing important is hidden, and see the plans if you want weekly monitoring that catches changes after launches and updates.

Related reading

• Robots.txt Explained: A Beginner’s Guide With Safe Examples
• XML Sitemaps Explained: How to Create and Submit One

The bottom line

Noindex is the right way to keep low-value pages, such as thank-you pages, internal search and carts, out of search results. Add it through a meta tag or an X-Robots-Tag header, keep the page crawlable so search engines can read the instruction, and use canonicals rather than noindex for duplicates. Above all, check regularly that no important page carries noindex by accident.

الأسئلة الشائعة

What is the difference between noindex and nofollow?

Noindex tells search engines not to show the page in results. Nofollow tells them not to follow the links on the page. They can be used separately or together as “noindex, nofollow”.

Can I use noindex in robots.txt?

No. Google stopped supporting noindex rules in robots.txt in 2019. Use a robots meta tag or an X-Robots-Tag header instead.

How long does it take for a noindex page to disappear from Google?

It disappears after Google recrawls the page, which can take from a few days to several weeks. Requesting indexing in Search Console can speed this up, and the Removals tool can hide it temporarily.

Should tag and category pages be noindex?

It depends on their content. Category pages that group products or articles well are often valuable and should stay indexed. Thin tag pages with one or two items are good candidates for noindex.

Does noindex hurt the rest of my site?

No. Noindexing low-value pages does not harm other pages. It only becomes a problem when important pages are noindexed by mistake.