What I learned about SEO from using the 10 most used JS frameworks #brightonseo

JavaScript will define and impact the future of most SEO consultants. A big chunk of websites has, is or will move over to a JS framework driven platform. Stack Overflow published an extensive study about the data gathered from an enquiry amongst more than 100.000 professional programmers’ most used Programming, Scripting and Markup languages: read more at Most Popular Technologies The outcome is quite clear, it’s all about JavaScript today:

But JavaScript and search engines are a tricky combination. It turns out there is a fine line between successful and disastrous implementations. Below I will share 10 tips to prevent SEO disasters to happen with your own or your clients sites.

1.      Always go for Server Side Rendering (SSR)

As Google shared earlier this year during Google I/O the pipeline for crawling, indexing and rendering is somewhat different from the original pipeline. Check out https://web.dev/javascript-and-google-search-io-2019 for more context but the diagram below is clear enough to start with: there is a separate track, also known as the second wave, where the rendering of JavaScript takes place. To make sure Google has URLs to be processed and returned to the crawl queue, the initial HTML response needs to include all relevant HTML elements for SEO. This means at least the basic page elements that show up in SERPs and links. It’s always about links right? 🙂

Google showed numerous setups in their article about rendering on the web but forget to include the SEO perspective. That made me publish an alternative table: read more at https://www.notprovided.eu/rendering-on-the-web-the-seo-version/

Server Side Rendering (SSR) is just the safest way to go. There are cons, but for SEO you just don’t want to take a risk Google sees anything else then a fully SEO optimized page in the initial crawl. Don’t forget that the most advanced search engine, Google, can’t handle it well. How about all the other search engines like Baidu, Naver, Bing etcetera?

Since Google openly admits there are some challenges ahead, they have been sharing setups of dynamic rendering. Pick the best suitable scenario for a specific group of users (low CPU power mobile phone users for example) or bots. An example setup could be the following where you make use of the client side rendering setup for most users (not for old browsers, non JS users, slow mobile cell phones etcetera) and sent search engine bots or social media crawlers the fully static rendered HTML version.

Whatever Google tells us, read Render Budget, or: How I Stopped Worrying and and Learned to Render Server-Side by a former Google Engineer.


2.      Tools for checking what search engines do and don’t see

Since most platforms capture user agents for dynamic rendering setups, changing it directly into Chrome is the first thing I always do. Is this 100% proof? No, some setups also match on IPs. But I would target the SSR as broad as possible, also think about social media crawlers wanting to capture OpenGraph tags for example. Targeting a combination of IPs and User Agents will not cover enough. Better cover too much requests and spend some more money on sufficient servers pushing out rendered HTML then missing out on specific platforms possibilities.

Next thing you need to check is if users, bots and other requests get the same elements of content and directives back. I’ve seen example where Googlebot got different titles, H1 headings and content blocks back compared to what the users got to see. A nice Chrome plugin is View Rendered Source that compares the fetched and rendered differences directly.

If you have access to a domain in Google Search Console, of course use the inspection tool. It now also uses an evergreen Googlebot version (like all other Google Search tools) so it represents what Google will actually see during crawling. Check the HTML and screenshots to be sure every important element is covered and is filled with the correct information.

Non-owned URLs that you want to check? Use the Rich Result Tester https://search.google.com/test/rich-results which also shows the rendered HTML version and you can check for Mobile and Desktop versions separately to double check if there are no differences.

3.      The minimal requirement for initial HTML response

It is a simple list of search engine optimization basics, but important for SEO results:

  1. Title and meta tags
  2. Directives like indexing and crawling directives, canonical references and hreflangs annotations.
  3.   All textual content, including a semantically structure set of Hx-headings
  4. Structured data markup

Lazy loading: surely a best practice in modern performance optimization but it turns out that for things like mobile SERP thumbnails and Google Discover Feed, Googlebot likes to have a noscript version. Make sure that Google can find a clean <img src=””> link without the need of any JavaScript.

4.      Data persistence risks

Googlebot is crawling with a headless browser, not passing anything to the next, sucessive URL request. So don’t make use of cookies, local storage or session data to fill in any important SEO elements. I’ve seen examples where products were personalized within category pages and that product links were only loaded based on a specific cookie. Don’t do that or accept a ranking loss.

5.      Unit test SSR

Whatever developers tell you, things can break. Things can go offline due to network failures. It could be due to new release or just some unknown bug that gets introduced while working on completely different things. Below an example of a site were the SSR was broken (just after last year’s #BrightonSEO) causing two weeks of trouble internally.

Make sure you setup unit testing for server side rendering. Testing setups for the most used JavaScript frameworks:

6.      Third party rendering – Setup monitoring

Also third party rendering like prerender.io is not flawless, those can break too. If Amazon crashes their infrastructure, most third parties you’ll use will be offline. Use third party (haha!) tools like Contentking, Little Warden or PageModified. Do consider where they host their services 🙂

Another tactic you can apply to be sure Google doesn’t index empty pages is to start serving a 503 header, load the page and send a signal back to the server once content is loaded and update header status. This is quite tricky and you need to really tune this to not ruin your rankings completely. It is more of a band-aid for unfinished setups.

7.      Performance: reduce JS

Even if every element relevant for SEO is available in the initial HTML response, I have had clients losing traffic due to performance getting worse for both users and search engine bots. First of all, think of real users’ experiences. Google Chrome UX reports are a great way of monitoring the actual performance. And Google can freely use that data to feed it to their monstrous algorithms haha!

Most effective tip is using tree-shaking to simply reduce the amount of JavaScript bytes that needs to be loaded. Making your scripts more clean can also speed up processing which helps a lot with older, slower CPUs. Specifically for older mobile phones this can help speeding up user experiences.

8.      Can Google load all JS scripts?

Make sure you monitor and analyze log files to see if any static JS files are generating any errors. Botify is perfect for this with their separate section monitoring static file responses. The brown 404 trends clearly show an issue with files not being accessible at the moment Google required them.

9.      Prevent analytics views triggered during pre rendering

Make sure you don’t send pageviews into your analytics when pre rendering. Easy way is just blocking all request to the tracking pixel domain. As simple as it can get. Noticed an uplift in traffic? Check your SSR first before reporting massive traffic gains.

10. Some broader SSR risks

Cloaking in the eyes of search engines: they still don’t like it and make sure you don’t accidently cloak. In the case of server side rendering this could mean showing users different content compared to search engines.

Caching rendered pages can be cost effective but think about the effects on the datapoints sent to Google: you don’t want outdated structured data like product markup to be outdated.

Check the differences between Mobile and Desktop Googlebots, a tool like SEO Radar can help you quickly identify differences between the two user agents.

Rendering on the Web – The SEO Version

Last week Google published a handy overview about Rendering on the Web in their developers section. It is aimed at developers needing to make a decision about the architecture they will use for their JavaScript driven applications. The main decision that affects logic within apps is the way rendering is set up. SSR, CSR, Rehydration and prerendering: they describe the different terminology in detail and align the advantages or disadvantages of using the different scenario’s. This overview is resulting in the following handy table:

Rendering and Search Engine Optimization

Both users and search engines are depending on this logic so I am disappointed that the authors didn’t include the perspective of their Organic Search colleagues in their first version. It seems the article was updated (at 08-02-2019) and a short section about SEO impact was included. Besides that, Jason and Addy (Tip: The Cost of JavaScript) are doing great work making developers well aware of some of the consequences of getting more JavaScript driven frameworks and websites. Thanks, the SEO community is thankful for aligning each other’s interest in keeping the web fast.

For optimal SEO results I advice not to take any risk and make the life of Googlebot (and her friends at Bing, Baidu and Yandex 🙂 ) and their render services as easy as possible: server rendering is the way to go. Feed the search engine bots non-cached HTML on the fly. The crawlers don’t want and don’t need JavaScript.

A solution that Google introduced last year and which is also pushed in the article after the last update is Dynamic Rendering: Get Started with dynamic rendering. I do not recommend that due to the different stages where Google needs information for certain tasks. Bartosz Goralewicz shared the risks and issues with setting up dynamic rendering during SMX West: Dynamic Rendering – is this really an SEO silver bullet? The biggest issue I’ve personally had to deal with was the problem with outdated content. Think about rich snippets showing prices, ratings and stock levels.

They also make the conclusion that Google will likely be able to get through your website when the raw HTML found in the mobile friendly test tool will have all the elements. If elements like meta data, textual content and internal links are present there, it should be sufficient. Right, it should be but I have seen so many cases where it harmed the organic rankings of websites that the only way to fix it was going back to serving the bots plain HTML.

There is a number of third party prerendering solutions available and many frameworks are aware of it so they are developing modules to deal with this. Be aware: if you depend on third party prerendering it can cause serious issues if those services are having issues. A power outage, configuration issues or something like that happens and you can’t do anything about it.

Less favourite, compared to server side rendering, solutions to consider are Hybrid Rendering or Static Site Rendering. Hybrid rendering means that the minimal required HTML is first send and then a layer of JavaScript is put on top of it. This also makes that you can save users JS on static pages but can still load interactive elements on the pages where you need it.

Static Site Rendering works well for a small blog but not for complex, dynamic environment since you will generate all HTML files upfront before uploading to the server. That means a single file for every possible URL. That doesn’t sound like a scalable solution, right?

SEO friendly options

Conclusion: keep it simple for search engine bots and use server side rendering and don’t bother bots with JS. The table including SEO Pros and cons:

A number of nice cases where Next.js was used can be found at Next.js showcase

How a medical site recovered from the Google Medic / EAT updates

  • Put all EAT related To-do’s on the backlog
  • Have a meeting with the developers
  • Developers are extremely busy with real improvements
  • Google will put the site back due to EAT-related To-do’s being present in the backlog
  • Happy holidays, Merry Christmas!

SEO Quick wins: finding meta data optimisation opportunities with Data Studio

Many of my clients have great content teams working on optimizing websites. Sending them a spreadsheet of data with some fancy click through ratios for keywords or URLs that perform better or worse than other datapoints will not get them enthousiastic at all. Sending them a fancy interactive graph in Data Studio will get them to work (not a 100% success ratio!) on boring things like optimizing meta descriptions and titles. So how can you spend 5 minutes of your time on Data Studio and make your life as a marketer much easier? Follow the instructions below.

Continue reading