How can I whitelist the IP or user agent of a third-party vendor to crawl my Webflow site for legal compliance archiving purposes?

TL;DR
  • Contact your hosting provider to see if they allow IP/user agent whitelisting and get their guidance on giving vendor access.
  • Gather necessary IP addresses or user agents from the vendor.
  • Configure access settings in Webflow's dashboard and explore third-party tools like CDN services for IP whitelisting.
  • Request vendor to perform a test crawl after configuration to ensure success and compliance.

To allow a third-party vendor to crawl your Webflow site, you must whitelist their IP or user agent specifically for this purpose. This process involves configuration in your hosting settings.

​

1. Contact Your Hosting Provider

  • Reach out to Webflow Support or your hosting provider to determine if they allow whitelisting specific IP addresses or user agents.
  • Ask for guidance on their recommended approach for giving a third-party vendor access.

​

2. Gather Necessary Information

  • Identify the IP addresses (e.g., 192.168.1.1) or user agents that the third-party vendor will use to crawl your site.
  • Request this information directly from the vendor to ensure accuracy.

​

3. Configure Access in Webflow

  • Sign in to Webflow and navigate to your Dashboard.
  • Go to the Project Settings for the website you wish to be crawled.
  • Although Webflow does not allow IP whitelisting through its native interface, be aware of any security features or third-party apps associated with your project that may support this. Alternatively, present these requirements to Webflow Support for further instructions.

​

4. Use Third-Party Tools or Integrations

  • If Webflow itself does not allow for IP or user agent whitelisting directly, you might use services like Cloudflare or similar CDN services that provide added security and access configurations.
  • Ensure that these services or tools are set up correctly with the IP or user agent details provided by your vendor.

​

5. Perform Tests

  • Once configurations are set, request your vendor to conduct a test crawl.
  • Confirm that the crawl is successful and is compliant with your legal archiving needs.

​

Summary

While Webflow does not directly support IP whitelisting through its native interface, reaching out to support and exploring third-party security integrations can facilitate the process. Ensure you have accurate IP or user agent details from the vendor and coordinate with both the vendor and your hosting provider or any additional security tools you use.

Webflow Resources
Webflow templates
Webflow inspiration
Rate this answer

Other Webflow Questions

Explore Logic tutorials
How can I easily redirect my website to a specific URL with language detection using Webflow?
How can I resolve various performance issues, such as eliminating render-blocking resources, deferring unused CSS, ensuring text remains visible during webfont load, minimizing main-thread work, serving static assets with an efficient cache policy, reducing JavaScript execution time, avoiding an excessive DOM size, and minimizing Critical Requests Depth in my Webflow site?
What could be causing a pop-up form to be cropped at the top on iPhones and Safari, despite working properly on Chrome Desktop and Mobile?
Is there a simple code that allows slider heights in Webflow to automatically adjust based on the height of the content within them, even for CMS pages? It would be greatly appreciated as it is preventing the launch of my site. Thank you!
How can I fix the issue of the sound of a YouTube video continuing to play in the background after closing a pop-up modal on my Webflow site?
Does the "fixed" background picture property not work on iPhone 5 when using Webflow?
How can I remove the default greyish gradient styling from the select element in Webflow?
How can I create a centered navigation bar similar to the one on the Pantone website using Webflow?
Does making adjustments to content in the CMS change the original publishing date in Webflow?
Is it illegal for a company to steal a website layout when the website was hosted on Webflow and how did the new company get access to my files without my permission?
How can nofollow tags be added to individual links in Webflow?
How can I make a GIF a background in Webflow using a Div block?
How can I add a table to my Webflow blog? I have tried multiple methods like adding tags and headings, but none of them seem to work. Could someone please assist me?
What is the most effective and reliable way to add a dynamic Instagram feed to Webflow while keeping it cost-effective?
What happens if non-www is set as the default in Webflow and Chrome shows a privacy warning while Safari immediately gives up?
Can I capture a screenshot of a Webflow page on click and convert it to a downloadable PDF?
How can I alternate the layout of the image and heading for each collection item in a two-column format on Webflow?
How can I execute JavaScript code when clicking a specific button with a given ID in a Webflow project?
How can I disable the Webflow success or failure state for a sign-up form and display a custom thank you page using jQuery and the Webflow form submit state?
How can I create a vertical scrolling menu with Webflow, similar to the one on Apple's website, that switches to horizontal scrolling when the menu doesn't fit on one screen?
How can I change the language of the Google Maps embed from English to Arabic in Webflow?
How can I reduce the vertical space between lines within a bullet point in Webflow? Can I replace the bullet points with icons on the "Services" page?
How can the text color be changed for custom input fields on Webflow?
How can I add conditional visibility in Webflow to prevent a div from appearing on a published page if a CMS field is empty?
How can I stop a YouTube video from playing in the background in audio mode when I close a modal in Webflow?
How can I connect and use my JavaScript and CSS files for special functions and styles in Webflow?
How can I fix the "Ensure text remains visible during webfont load" warning in Webflow?
How can I implement a ThreeJS script as a background for my Webflow project using custom code?
How can I send data from a Webflow Form to ActiveCampaign without using Zapier? I have set the form to POST and input the form's action URL, similar to Mailchimp but it redirects me to the admin area of ActiveCampaign without sending the data. Has anyone had success with this method?
Can Webflow be used to create a "blob" elements effect in the header of a website using custom code or JavaScript?