What is the correct code snippet to prevent search engines from crawling a page in Webflow? And should the "noindex/nofollow" code be added to empty collection pages and the 404 page?
TL;DR
- To prevent search engines from indexing pages in Webflow, navigate to the Pages panel, select a page, open Page Settings, add
<meta name="robots" content="noindex, nofollow">in the Custom Code section, save, and publish. - Apply these meta tags to empty collection pages and the 404 page to ensure they aren't indexed.
To prevent search engines from crawling a page in Webflow, you need to add meta tags with the directives "noindex" and "nofollow". Here’s how you can do that and whether you should apply it to specific types of pages:
1. Add Noindex/Nofollow Meta Tags
- In Webflow, navigate to the Pages panel.
- Select the page you want to edit.
- Open the Page Settings by clicking on the gear icon.
- In the Custom Code area, add the following meta tag within the head code section:
<meta name="robots" content="noindex, nofollow">- Save the changes and Publish your site to apply them.
2. Apply to Empty Collection Pages
- Yes, you should add the "noindex, nofollow" meta tags to empty collection pages to prevent search engines from indexing pages that lack content.
3. Apply to 404 Page
- Yes, it's recommended to use the "noindex, nofollow" tags on your 404 page. This prevents search engines from indexing it, as a 404 page does not provide useful content for search results.
Summary
To prevent search engines from indexing specific pages in Webflow, add the noindex, nofollow meta tags to the page's custom code section. This is particularly useful for empty collection pages and the 404 page to ensure they aren't indexed by search engines.