Please note that we've published the follow-up article
Edge SEO: with great power comes great responsibility.
SEO can be a difficult enough service to sell with spend not being 1:1 with ROI, but a lead time of 6 months for critical changes can cripple an SEO campaign, especially when the decision makers and budget holders aren't always well versed or wholly understanding of the issues being faced.
Edge SEO, and tools such as Sloth, can really make a difference here.
Welcome Dan! For those that don't know you, can you introduce yourself?
My name is Dan Taylor and I'm a senior technical SEO consultant & account director at SALT.agency (opens in a new tab), who are a specialist technical SEO agency. I write a monthly piece for Search Engine Journal, as well as a number of articles on my own website (opens in a new tab).
I speak at conferences around the UK and US, notably at TechSEO Boost in 2018, where at that conference we (SALT) won the inaugural Call for Research competition, backed by Moz, for our research into using Cloudflare Workers for SEO purposes.
At the conference we also unveiled the beta for our own Cloudflare Worker tool called Sloth (opens in a new tab), which allows users to configure and manage Cloudflare Worker bundles for various SEO purposes, including redirects, AB testing, code injection, without needing to touch a line of code.
Can you explain what Edge SEO is?
Edge SEO refers to the technique of implementing SEO recommendations, technical fixes, and navigating platform restrictions through using a serverless application (Cloudflare Workers) on a CDN edge server.
Will Critchlow published a study on Moz in 2016 which found that the majority of SEOs didn't see their recommendations implemented for ~6 months after they were made.
SEO can be a difficult enough service to sell with spend not being 1:1 with ROI, but a lead time of 6 months for critical changes can cripple an SEO campaign, especially when the decision makers and budget holders aren't always well versed or wholly understanding of the issues being faced.
Common issues outlined in the study were that marketing requests fell behind those of other teams in terms of priority, that the platform was too restrictive/wouldn't allow for the changes, or the development backlog was just that long.
Through edge SEO, and tools such as Sloth, a lot of these issues can be navigated and either complete remedies, or temporary fixes put in place.
So how does Edge SEO work in practice?
In practice, this means that we can now modify a webpage and website, without modifying the underlying code base. This is done by modifying both the request and response between the client (either a user or Googlebot), and the server.
Sloth was the first tool of its kind, and whilst the concept of edge SEO is not a new one – we hope that the tool itself and our research will help spark the creativity of others and we find more practical uses for the technology.
Do Cloudflare Workers have to modify every individual response? And if so, how much of an impact does this have on page speed?
Cloudflare Workers modify each request made to the server, however for the initial $5 a month you get 10-million requests, and you can filter out unnecessary requests to reduce volume.
In terms of impact on page load speed and additional latency, from our testing and usage on real-world websites the average overhead has been around the 10ms mark - which for comparative purposes, Distilled's ODN advertises a 15ms latency overhead.
In some extreme examples we've seen up to 50ms of latency added, but this is very much down to the platform and packet size being returned by the server. Generally if you're serving big packets, large images and other uncompressed media (and your site is already fairly sluggish), the additional latency would be a minor amplification of this.
From Google's perspective, can Google know the difference between a page served by the origin and one by the edge?
No, the "version" of the page that is being shown to Google has already been modified by the Cloudflare Workers. This is a single request, so only one version is served and as the site is using Cloudflare's CDN, the request and response must pass through the CDN – which is where the modified Worker bundle modifies the codebase to the desired specification.
What kind of changes can you make with SEO at the edge?
The possibilities of what you can do on the edge, for the purposes of SEO, are almost limitless, through building Sloth we've been able to:
- Implement redirects (opens in a new tab) (for example 301 and 302 redirect)
- Modify the HTTP header response (such as including an X-Robots-Tag)
- Modify the robots.txt file (opens in a new tab) (including the robots.txt file on Shopify)
- Implement a JavaScript pre-rendering solution
- Collecting server log files (opens in a new tab) (including logs from Salesforce CC/Demandware)
- Inject new content
- Overwrite existing content (including meta titles, descriptions and body content)
- Implementing schema markup (opens in a new tab)
- Implementing hreflang (and mapping it across multiple platforms and ccTLDs not on a single database structure)
And we've also been testing AB testing, as well as being able to test website migrations (such as new templates, changing platforms) without doing a full migration.
With edge SEO, it's easy to mess up and accidentally signal the wrong things to Google. Be notified about issues as they happen!
In what situations is Edge SEO useful?
For many, edge SEO offers no tangible value other than a new thing to test, or a new luxury tool. For a number of SEOs and in-house digital teams however, it can revolutionise their SEO campaigns.
There are a number of real-world struggles faced, and a lot of them unique to bespoke platforms, legacy tech stacks or how the stacks are configured. There's also the issue of unhelpful, gatekeeper developers (in some cases).
Edge SEO can help navigate these issues for as long as the domain is on Cloudflare, you can implement changes on the edge agnostic of the tech stack, developer availability, or platform restrictions.
Some of the use cases we've been working with a group of beta testers on include modifying the robots.txt file on Shopify websites, and collecting server log files on Salesforce CommerceCloud / Demandware.
Track every change and be alerted in case of SEO trouble!
How does Sloth compare to Distilled's ODN?
Sloth and ODN are two different tools, as Sloth is just a code generator and management tool that increases the accessibility of edge SEO technologies. Some people are calling this a meta CMS and to an extent it is a management system.
Like Distilled's ODN, Sloth is platform independent and works in the middle ground between the client and the server. We would say that ODN is a good AB testing solution, however Sloth is designed more for tackling complex problems - such as implementing a standard hreflang across 15 different ccTLDs and languages with all of the sites built on different platforms, or implementing redirects when the physical server has a line limit and can't handle the requirement.
You talk a lot about Cloudflare, what about other CDNs?
Obviously other CDNs are available, and you can use Akamai, Imperva Incapsula and MaxCDN (to name a few) to improve your website performance. I've explored how CDNs can improve SEO performance in this article (opens in a new tab).
The reason that I specifically talk about Cloudflare when talking about edge SEO is because only Cloudflare offers a serverless application with these capabilities. If the other CDN providers were to also provide a similar serverless application to Cloudflare Workers, then we would be implement edge SEO practices through those.
What are the downsides of doing SEO at the edge? What risks are involved?
When pushing any form of change to a website there are inherent risks and the potential to introduce bugs, or have changes pushed to a website that are not signed off/detrimental to the website or business itself.
Some of these risks aren't necessarily risks, but they do create the need for new internal processes. Some of the potential risks from a technical perspective include:
- Potential conflicts between the underlying codebase and the changes made on the edge.
- Potential to introduce front-end bugs that are difficult to debug, when there is little/no access to the backend, and when it is unclear what is being modified/injected through stream transformation.
There is also a risk around the cost of the technology. In order to use Cloudflare Workers you need to enable them within your Cloudflare account and this costs $5 per month, for 10-million requests, with each additional 1-million requests costing a further 50c. This doesn't sound like a lot but if you're a major website with lots of requests the costs can add up quickly.
How do you mitigate those risks?
To get around any bugs or additional latency being added to the website, we recommend that edge SEO activities are not done in silo or without structure. They should follow a similar process to "regular" changes on the website and have a thorough changelog and scheduled "push" dates.
It's important that internal processes are developed to incorporate the use of this technology so there is a single point of ownership. One of the features we built into Sloth is that it doesn't store your Cloudflare authentication key.
This can be seen as an inconvenience for the one-click deployment, however it's for 2 core reasons:
- It prevents anyone with access to the Sloth account from pushing changes without regulation, especially important if the account is compromised.
- It can compromise the PCI compliance of the business.
By adding in this step, it helps create a chain of command if the authentication key is seen as the key that unlocks the big red launch button. With anything there should always be an element of testing before the big launch, and if bugs are introduced, Cloudflare Workers and the deployed self-sufficient worker bundle can be turned off just as fast as they were implemented.
To avoid the high costs due to request volume, we recommend scoping deployments thoroughly and implementing filters to reduce unnecessary requests from triggering the Worker bundle – this will keep costs down.
When it comes to monitoring the changes you're making, the key element of this technology is that it can be verified and monitored as you're already used to. Taking Hreflang implementations for example, you can use a standard validation tool such as hreflangchecker.com (opens in a new tab) and validate your Hreflang implementation.
Crawlers, and SEO monitoring systems such as ContentKing can pick up on errors through edge implementations just fine.
Besides flexibility, Edge SEO also adds an extra layer of complexity. Now more than ever it's important to track On-page SEO changes.
Practically speaking, what platforms can you utilize to do SEO at the edge?
As long as you can point your domain to Cloudflare's nameservers and use the Cloudflare product, you can use Cloudflare Workers and perform SEO on the edge.
Likewise, if and when Imperva Incapsula and Akamai create a similar serverless application for deploying self-sufficient bundles onto their networks, the same principle applies.
Continue reading in-depth interviews with SEO specialists
You can check out our previous editions of SEO in Focus here:
Steven is ContentKing's VP of Community. This means he's involved in everything community and content marketing related. Right where he wants to be. He gets a huge kick out of letting websites rank and loves to talk SEO, content marketing and growth.
