Google’s Webmaster Tools is one of the most powerful tools for site SEO and improving the visibility of your website. It allows us a complex understanding of how Google is crawling and indexing your site – as well as providing an insight into any issues it has encountered.
Webmaster Tools is ever-changing and as such it’s important to keep up-to-date with any developments through Google’s Webmaster Central blog.
Here are some tips for using Webmaster Tools for your site SEO:
1. Turn on Site-Error Email Alerts
Often Google will update you when there are any major issues with your site. These include:
– DNS server issues.
– Your web server is overloaded, or down.
– Your site’s robots.txt is inaccessible.
– Your site’s robots.txt is blocking important pages on the site.
– You have unnatural links pointing at your website (this is a big one!)
It is important that we set up email forwarding so that we are aware of these issues as soon as they arise. The sooner we are aware of the problem, the sooner we can solve it!
2. Unearth Site-Wide Duplicate Content
We can use the ‘HTML Improvements’ report to unearth instances of site-wide duplicate content. This report allows us to view all duplicate Titles and Descriptions on the site. We often find that where a page has a duplicate Title, often it will also have the same content. As such, this report is great for uncovering duplicate pages which have separate URL’s as a result of tracking parameters, session ID’s or errors within the site CMS. Check these pages and implement Canonicals where suitable.
3. Use the URL Parameters Report to Block Unnecessary Pages
Sometimes there are URL parameters on our site which don’t actually affect the content of a page. These could include tracking parameters such as ‘utm_source’ or ‘gclid’. There might also be URL parameters which do affect page content, but only in a way which is meant to make the site more usable. For instance – parameters which are used to order page content in alphabetical order. We might not always want Google to crawl these pages, as they may cause issues with duplicate content and reduce the efficiency of a crawl.
Any URL parameters Google encounters when crawling your site will be included within the URL Parameters report. If any have not been included, you can add them manually by clicking ‘Add Parameter’.
Through this report we can then tell Google which parameters we would like it to crawl and which we would like it to ignore. Conventionally we might have blocked certain parameters using a wildcard statement in your robots.txt file, but this option is preferred as we can help Google to understand how the parameter works on the site. Any further information we can offer Google to help it understand how your website works is always beneficial, as it will help ensure that Google gives the correct pages the most authority.
To block a query parameter, we need to click ‘edit’ on the right-hand side of the screen. From there we can tell Google how the parameter works – firstly, if it changes page content; and if so, how? For example, ‘utm_source’ is a manual tagging parameter used to track the source of a click within Google Analytics. So, in this instance we would select ‘No. Doesn’t affect page content’. Google will then understand that it does not need to crawl any URL’s which use this parameter as they will not contain unique content.
For a parameter which does change page content but which we don’t require Google to crawl, we can select ‘Yes. Changes, reorders or narrows page content‘. We can then specify how this parameter changes page content (eg. If it orders content alphabetically then we would specify that it ‘sorts’). By then selecting ‘No URL’s‘ under ‘Which URLs with this parameter should Googlebot crawl?’ we can block all duplicate page URL’s which are being created by this function.
It’s advised that you should run through the list of query parameters on your website and decide how you would like Google to treat each.
4. Fetch as Googlebot
This is a great tool because it allows us to gain an insight into how Google ‘sees’ specific pages on our site. If a page isn’t performing well in search results and you think there may be an underlying reason, then this tool might help you figure out why. It’s great for helping to establish the crawlability of your site, and it will return information such as the HTTP Response (handy if you think Googlebot might be being redirected), the page text (can Google see all the content that your users can?) and the page HTML.
5. Blocked URL’s
This is a great tool to make sure that your robots.txt statements are (or aren’t) blocking the URL’s you would like them to. Before adding any statements to your file, use this tool to check your statements against certain URL’s. You can edit the robots.txt file live within the tool and check how it works against specified URL’s. This is especially handy if you are using any advanced syntax such as wildcard characters or exact match statements.
6. Crawl Errors
The Crawl Errors report is one of the most powerful features of Webmaster Tools. There are six main categories of errors: Server errors, Soft 404’s, Access denied, Not found, Not followed and ‘Other’.
Server Errors
Server Errors are typically 500 and 503 response codes caused by internal server errors. These are typically rogue URL’s. A high number of server errors on your website could lead Google to believe that your site provides a bad user experience and this may harm your search engine visibility.
Soft 404’s
Soft 404’s are pages which are technically 404 error pages, but which are not sending the correct 404 response code. For example, this might include ‘Product Not Found’ pages which are sending the normal 200 response code, but which should be sending a 404 response. Google’s stance is that these pages “create a poor experience for searchers and search engines” – so it’s important that we look into these.
Access Denied
This report allows you to see which pages Google is being prevented from accessing – either by statements within your robots.txt file or through areas of the site where user log-in is required.
Not Found
These are effectively 404 errors which are returned when Google attempts to visit a page which no longer exists. While it was previously thought that a large number of 404 errors on your site would harm your rankings, Google has recently clarified its position on this, stating that “Generally, 404 errors don’t impact your site’s ranking in Google, and you can safely ignore them”. These error warnings should therefore be interpreted more as an ‘FYI’ than as something to be actioned.
Not Followed
This category lists URL’s on your site which Googlebot was unable to completely follow. There are generally two reasons why this might happen. Features such as Flash and Javascript might make links difficult to crawl, while search engine spiders sometimes have trouble following unconventional redirects which might result in anomalies such as redirect loops. More information on these errors can be found in Google’s official guidelines around not followed errors.
Other
These include other errors such as 400 Response Codes (Bad Requests). This report is usually best for identifying areas of your site which are causing errors and taking action upon these rogue URL’s.
7. Geo-Targeting Sub-Directories
Generally it is more beneficial in terms of SEO to feature country site variations within sub-directories on one main domain – rather than on separate top-level domains or sub-domains. For instance:
https://www.searchlaboratory.com/de/
rather than http://www.searchlaboratory.de
or http://de.searchlaboratory.com.
The reason for this being that we are consolidating all of our link juice on a single domain. However, how do we then make sure that Google understands which sub-directory is meant to target which country?
We can use the rel=alternate mark-up, but it’s also best to explicitly state this within Webmaster Tools. This is a fairly simple process, though it can be laborious if you have a high number of country based sub-directories. We simply need to verify each sub-directory separately within WMT. This can be done by adding the verification code to the homepage of each sub-directory. Once complete, we can then navigate to ‘Configuration > Settings’, and set the ‘Geographic Target’ as our preferred country. Google will then understand that this sub-directory and all pages contained within it are meant to target your specified country.
Insights
