Insights
Technical SEO
When migrating or redesigning a website that involves URL changes, having a 301 redirect strategy is crucial in maintaining link authority which has been built up over time. Without redirects, the new URLs would need to build up authority and trust from scratch and it is likely the site would see a huge drop in traffic.
A 301 redirect is used to permanently redirect site users and search engines from one URL to another. There are many reasons to use a 301 redirect; a website migration, a site redesign, combining and condensing multiple pages into one. When we change a page URL, we can not change all links pointing to that page (for example, links from an external site or from search engine results). Without setting up a 301 redirect, users and search engines would click through this link and end up on an error page.
Implementing a 301 redirect tells search engines that the page has moved to a new URL and directs the search engine to the correct page. This means the search engine can pass on any page trust and link authority to the new page, and users can end up on the page they were looking for.
When working with a site migration or redesign, there can be thousands – if not tens of thousands – of URLs to redirect. Mapping these URLs can be extremely time consuming, as each redirect needs to be individually written manually or managed within the CMS.
The client’s site had a mass duplication problem caused by the fluidity of the sub-folder structure. This meant that pages could have up to four different variations due to the nature of the structure and the number of available sub-folder options, meaning there was a significant number of redirects needed to be mapped.
In addition to the sheer volume of URLs needed, we faced a few more challenges, one of which was the encoding of the URLs. Symbols within the URLs were encoded and presented a different URL once accessed by Google, meaning the URLs both caused errors within the script and were different from the crawled URL.
For example, the %3e in the URL:
https://www.searchlaboratory.com/uk/example_categories/%3example
Would convert into %>:
https://www.searchlaboratory.com/uk/example_categories?%>example
We also needed to match the previous H1 tags to the new H1 equivalent. This presented an additional challenge as the H1s had been implemented using JavaScript based on a device width, which meant Screaming Frog would time out before it identified all the H1s – even when we enabled JavaScript crawling, not all H1s were captured.
We created two scripts using Python which allowed us to overcome both challenges and reduce the time taken to map the URLs from months to just a few days.
The first script allowed us to scrape the client’s website and pull out any H1 tags that were injected using JavaScript and therefore missing from the Screaming Frog crawl.
The second script allowed us to match the old H1 tags with the new much more quickly than if we were to do it manually. The script produced two documents based on whether there was an exact or partial H1 match, with a partial match identifying where keywords were similar but not the exact same, and an exact match highlighting where the new H1 and old H1 were exactly the same.
Once we had reviewed the partial matches document to ensure that the URLs were mapped to the correct alternatives, we were able to input both documents into an excel format and send to the client for implementation.
We are now able to use these scripts to efficiently implement large scale 301 redirects, ensuring there is minimal page authority lost during website migrations.
Get more of our technical SEO insights and tips here.
Need help with your SEO?Get in touch
Insights
Insights
Insights