What is the correct code snippet to prevent search engines from crawling a page in Webflow? And should the "noindex/nofollow" code be added to empty collection pages and the 404 page?
TL;DR
- Access the Pages panel in Webflow's Designer, select a page, navigate to SEO settings, and add
<meta name="robots" content="noindex, nofollow">in the Header Code, then publish. - Apply the same steps to empty collection pages to enhance SEO and verify noindex is set on customized 404 pages.
Understanding how to prevent search engines from crawling specific pages on your Webflow site is important for managing your website's visibility in search results. The noindex/nofollow tags are key to implementing this.
1. Adding "noindex/nofollow" to a Page
- Go to the Pages panel in Webflow’s Designer.
- Select the page you wish to edit.
- In the Page Settings, scroll down to the SEO settings section.
- Find the Header Code field.
- Enter the following meta tag:
<meta name="robots" content="noindex, nofollow">. - Publish your site to apply the changes.
2. Using "noindex/nofollow" for Empty Collection Pages
- Empty collection pages can harm your SEO due to thin content.
- Consider applying the noindex/nofollow tag to prevent them from appearing in search results.
- Follow the same steps as above by accessing the Template page in the Pages panel.
3. Applying "noindex/nofollow" to the 404 Page
- Webflow automatically applies the noindex directive to the default 404 page.
- Verify if custom 404 pages have the noindex set by entering the meta tag in the Header Code field of your 404 page settings.
Summary
To prevent search engines from indexing certain pages on your Webflow site, use the <meta name="robots" content="noindex, nofollow"> tag in the Page Settings. It's advisable to apply this to empty collection pages. Webflow automatically sets the noindex directive for the default 404 page, but if you've customized it, double-check the settings.
