When custom JavaScript code is used in Webflow, it is usually rendered client-side. This means that the resulting HTML code is not relevant for SEO because search engine bots like Googlebot only see the HTML code which is rendered server-side by Webflow.
As a result, the search engine crawlers may not index your website's content correctly, leading to lower search rankings and reduced visibility. To address this issue, a reverse proxy with Cloudflare Workers and prerender.io can be used to ensure that search engine crawlers can access and index your JS-based site content effectively.
Using a reverse proxy with Cloudflare Workers and prerender.io is one of the best solutions to address SEO issues related to JavaScript-heavy websites.
This approach ensures that search engine crawlers can effectively access and index the content of your JavaScript-based site.
Here's a step-by-step guide to setting it up.
Step-by-step guide
1. Sign up for Cloudflare and Prerender.io
First, create an account with both Cloudflare (https://www.cloudflare.com) and Prerender.io (https://prerender.io). If you already have an account with Cloudflare, ensure that your domain is added and configured correctly.
2. Set up Webflow with Cloudflare
To set up a Webflow site with a Cloudflare proxy, follow the following 6 steps:
- Add your domain to Cloudflare and configure it correctly.
- Ensure that the DNS record is set to proxy (orange cloud turned on).
- In Webflow, disable SSL and copy the blank DNS records (without SSL).
- Add the copied records to the DNS settings in Cloudflare.
- Re-enable SSL in the Webflow settings.
- Ignore any warnings in Webflow regarding AAAA-records.
3. Set up a Cloudflare Worker
In the Cloudflare dashboard, navigate to the "Workers" tab and click on "Create a Worker." This will take you to the Worker editor where you can write your JavaScript code.
4. Configure the reverse proxy
Inside the Worker editor, you'll need to write JavaScript code to handle incoming requests and forward them to your origin server or Prerender.io as appropriate.
Here's a sample code snippet to get you started:
Copy this code from GitHub Gist.
Replace 'your_prerender_token’ with your actual Prerender.io API token. This code checks if the request comes from a crawler and, if so, sends it to Prerender.io for rendering. Otherwise, it forwards the request to your origin server.
Note: This is a basic solution! To ensure all search and social media bots are properly captured, I recommend using the full JavaScript provided by prerender.io. You can find it in the following GitHub repository, and detailed instructions for the Cloudflare ESM Syntax Integration are available here.
5. Deploy the Cloudflare Worker
Save and deploy your Cloudflare Worker. The Worker will now run for all requests to your domain.
6. Test your setup
You can use tools like Google's Mobile-Friendly Test or the Chrome Extension User-Agent Switcher to see how your website is being crawled and rendered by Googlebot.
Conclusion
By following these steps, you'll set up a reverse proxy that routes requests from search engine crawlers through Prerender.io, ensuring they receive a fully-rendered version of your JavaScript-heavy website. This should improve your site's SEO performance by making your content more accessible to search engines. All other requests will be directed straight to your Webflow website.
