SEO

URL parameters are the parts of the address of a web page often visible after a question mark (?), ampersand (&), equals (=) or other attribute. Parameters might also be referred to as “query strings” and are commonly used on e-commerce websites.

Examples of different types of URL parameters

Numerous websites make use of query string parameters as a core part of their site architecture for tracking session IDslanguages, powering search queries and more. However, query string parameters can often result in multiple variations of the same URL all serving the same content.

For example, the following URLs would, in theory, all point to the same content: a collection of bikes. The only difference here is that some of these pages might be organised or filtered slightly differently:

  • http://www.example.com/products/bikes.aspx 
  • http://www.example.com/products/bikes.aspx?category=mountain&color=blue
  • http://www.example.com/products/bikes.aspx?category=mountain&type=womens&color=blue

URL parameters like those listed above, are common on e-commerce websites that use faceted navigation systems. Faceted navigation systems or site filter options allow users to browse products using a selection of pre-defined filters. This ultimately helps the user to find what they want, but it can lead to problems.

Example of a bike website's faceted navigation or product filtering options. Each filter option adds a parameter to the URL string.

Example of a bike website’s faceted navigation or product filtering options. Each filter option adds a parameter to the URL string.

 

Using URL parameters to serve content based on filters applied via faceted navigation menus commonly results in URLs similar to those above. While this can be helpful to users trying to drill down to specific product groups within e-commerce sites, they can give search engines a headache if not managed correctly.

Common Issues with URL Parameters

Duplicate Content

The generation of URL parameters based on website filters can create serious on-page issues that can impact the rankings of e-commerce category pages. We have already discussed how filters can be used to sort or narrow the content of a page, thus creating additional URLs that add no real value. For example, if your website enables users to sort content by price or feature (colour, size etc.), and these options aren’t actually changing the content of the page but just narrowing the results; then this could hamper your website’s performance.

One way to check whether this is the case for your site is to test some of the filters available on one of your product category pages to assess whether the content changes significantly after the products have been filtered. For example, let’s imagine that the content of an original category page on a cycling website included a paragraph or two of copy to promote a specific type of bike. Then, when filters were applied using the faceted navigation to select female bikes, the URL of the page changes to include a query string (example.com/bikes?type=female) – if the majority of the page content remained the same, these pages could be classed as duplicate content by Google IF their relationship isn’t made clear by the website owner.

Keyword Cannibalisation

Keyword cannibalisation occurs when multiple pages on a website target the same or similar keywords. This often causes search engine problems when deciding which page is the most appropriate page to rank for a particular search query, which in turn can lead to the “wrong” or “undesired” page ranking for that term.

A prime example of this can be seen on the bike company Halfords’ website www.halfords.com. As is commonplace on e-commerce sites, Halfords.com uses URL parameters to dynamically serve product and search filter results for types of bikes, brands, sizes, styles or kids or adult audiences. The problem for Halfords is that their site doesn’t deal with the numerous dynamically generated URL parameters page which ultimately, creates competing category pages for Google to index.

To better illustrate how parameter URLs can harm organic performance, let’s take a look at the following results for “women’s bikes”. For this search, Halfords’ main bike category appears in the search results rather than their women’s bikes category page.

The Halfords site doesn't rank as well as it could in Google due to URL parameter issues on their website.

The organic results for Decathlon and Evans Cycles are their women’s bike pages, which makes sense as these pages best match the search query “women’s bikes”. This is what I would expect from a properly optimised e-commerce site, but unfortunately for Halfords, their failure to address the numerous duplicate parameter pages for the keyword phrases has resulted in Google not ranking their women’s bikes page.

To understand the extent of Halfords’ problems, I ran a site command search on the Halfords domain (site:halfords.com women’s bikes) to identify competing pages for the term women’s bikes. The search results for this query revealed why the right page isn’t ranking.

I conducted a site command search on the Halfords site for to identify competing womens bikes pages (site:halfords.com "womens bikes"). The results show the duplicate parameter URLs that are holding back Halfords organic performance for this term.

I conducted a site command search on the Halfords site for to identify competing women’s bikes pages (site:halfords.com “womens bikes”). The results show the duplicate parameter URLs that are holding back Halfords’ organic performance for this term.

There are numerous duplicate parameter URLs for women’s bikes and the screenshot above includes two results with product_storeId_10001_catalogId_14552_productId_ in the URL.

The first result from halfords.com is http://www.halfords.com/webapp/wcs/stores/servlet/product_storeId_10001_catalogId_14552_productId_1135992_langId_-1_categoryId_null&cm_re=Cycling+SideNav-_-bikes-_-womens

One of the many womens bikes pages on the Halfords site. An SEO own goal!

One of the many women’s bikes pages on the Halfords site. A prime example of the problem that can be caused by parameter URLs.

The second result URL is a similar URL, but a different Product ID parameter is appended to the same URL as the first result. This is essentially a duplicate page. http://www.halfords.com/webapp/wcs/stores/servlet/product_storeId_10001_catalogId_14552_productId_1135992_langId_-1_categoryId_null&cm_re=Cycling+SideNav-_-bikes-_-womens

Halfords Womens Bikes 2nd page in Google's results is similar to the first but with a different Product ID appended. This is duplicate content and explains why Halfords women's bikes page does not rank.

Halfords Womens Bikes 2nd page in Google’s results is similar to the first but with a different Product ID appended. This is duplicate content and explains why Halfords women’s bikes page does not rank.

Halfords also have their mobile site in Google’s desktop index which isn’t good, as it provides yet another duplicate parameter URL.

http://www.halfords.com/webapp/wcs/stores/servlet/ProductMobileDisplay?langId=-1&storeId=10001&catalogId=14552&categoryId=173014&productId=1311971&cm_re=guidednavbikes-_-bikes-_-womensbikes

Mobile parameter URLs are in Google's index for the Halfords site

Mobile parameter URLs are in Google’s index for the Halfords site. This doesn’t provide a good experience for desktop users.

If we visit the Halfords site, it’s possible to find even more URLs available for women’s bikes queries. The link within the main navigation menu takes you through to a women’s bike page powered by parameters.

Navigation menu on Halfords site takes users to a parameter filter page

Navigation menu on Halfords’ site takes users to a parameter filter page

The product category filter options and additional search filter options are creating further pages that are competing for the same term. Halfords are missing out on organic traffic because they are not using technical SEO techniques to avoid parameter issues and help Google to understand which pages to index. Ideally, your pages should use a search engine friendly URL structure, but you can still use parameters as long as you take the necessary steps to avoid duplication and indexing issues.

The fact that Halfords has multiple pages for women’s bikes with variations of women’s bikes in their page titles and headings, performance is being held back. This is a classic example of keyword cannibalisation, where a website is simply confusing search engines by having two or more pages covering the same topic, meaning search engines will be unsure of which page to rank.

When we originally wrote this post in 2015, we cited Booking.com as a prime example of an accommodation website with parameter issues. Booking.com used URL parameters to display unique hotel results related to specific locations across the globe. It looks like they have solved the parameter issues with their site, but to illustrate the problems that they had and that many sites continue to have, I have referenced our original observations below. These results show the results for the query ‘Booking Hotels in Dubai’.

For this search, Booking.com appeared twice in the search results:

hotels in dubai

The first result from booking.com that appeared for that search query was this page: http://www.booking.com/city/ae/dubai.en-gb.html

dubai

This page uses a search engine friendly URL structure, and serves a customised landing page that acts as a search portal for all hotels they offer in Dubai.

However, Google also indexed a second page from Booking.com’s website for that term: http://www.booking.com/searchresults.en-gb.html?city=-782831

dubai2

 

This page was substantially different to the first page we looked at; and had been developed so that the heading is generated through the name of the city being searched for, and the number of properties being returned for that search. The city attribute had been used to populate the page’s meta title to ensure it’s not seen as a duplicate of the first page:

dubai 3

Now, while these pages won’t be classed as exact duplicates of one another, this doesn’t mean that they won’t cause problems. The fact that Booking.com had two pages that feature the phrase ‘Hotels in Dubai’ in their page titles, headings, and copy and held back the performance of both pages.

Booking.com resolved their duplicate parameter URL content issues, and now we’re going to take a look at a few of the techniques for doing this below.

How to Avoid Pitfalls Associated with URL Parameters

There are a number of solutions to ensure URL parameters don’t cause SEO issues for your website. But before you rush in and implement any of the fixes below, you need to check whether URL parameters could be causing your website problems by asking the following questions:

  • When using a search filter on your website (see faceted navigation), does the URL change and the copy remain the same as the copy on the original URL?
  • When using a search filter on your website, does the URL change and the page title and meta description remain the same or contain the same target keyword?

If you answered yes to one or both of these, URL parameters could be holding back the performance of your site in organic search, and it might be time to take action.

Canonical Tags

Canonical tags are used to indicate to search engines that certain pages should be treated as copies of a certain URL, and that any rankings should actually be credited toward the canonical URL.

canonical tag

Web developers can set the canonical version of a piece of content to the category page URL before a filter is applied. This is a simple solution to help direct search engine robots toward the content you really want to be crawled, while keeping filters on the site to help users find products closely related to their requirements. For more information on implementing rel=”canonical” tags, see our guide on how to use canonical tags properly. We have also written a guide on how to implement canonical tags on Magento e-commerce sites.

URL Parameter Tools

Use the URL Parameters tool within Google Search Console to give Google information about how to handle URLs containing specific parameters. Please do tread carefully when making changes as you can easily exclude the wrong pages from Google’s index which may result in a significant loss of traffic.

Configuring URL parameters in Google Search Console

Configuring URL parameters in Google Search Console

Bing Webmaster Central also provides a tool for ignoring URL parameters. You can find a guide on how to use this tool here.

Robots.txt – Disallowing Query Strings

The robots.txt file can help you remedy a duplicate content situation, by blocking search query parameters from being crawled by search engines. However, before you go ahead and block all query strings, I’d suggest making sure everything you’re disallowing is something you definitely don’t want to be indexed. In the majority of cases, you can specify search engines to ignore any parameter based pages simply by adding the following line to your website’s robots.txt file:

Disallow: /*?*

This will disallow any URLs that feature a question mark. Obviously, this is useful for ensuring any URL parameters are blocked from being crawled by search engines, but you first need to ensure that there aren’t any other areas of your site using parameters within their URL structure.

To do this, I’d recommend carrying out a crawl of your entire website using a tool like Screaming Frog’s SEO spiderexporting the list of your website’s URLs into a spreadsheet and carrying out a search within the spreadsheet for any URLs containing question marks (?).

Common things to look out for here would be the use of URL parameters to serve different language variants of a page, which in itself is a bad idea. If this is the case, you don’t want to block search engines from crawling those variants via robots.txt. You’ll need to instead look into implementing a viable URL structure to target multiple countries.

If you’ve worked through the list of URLs and confirmed that the only pages using URL parameters are those causing duplicate content issues, I’d suggest adding the above command to your website’s robots.txt file.

The image below shows Halfords robot.txt file and we can see that none of the parameter URLs are blocked.

Halfords.com robots.txt file. None of the problematic parameter pages are blocked

Halfords.com robots.txt file. None of the problematic parameter pages are blocked

Conclusion

Using a faceted navigation can prove extremely useful for consumers looking for specific products within your website, but you need to ensure that any URLs generated as a result of filters being applied don’t hold back the performance of your original category pages in organic search results.

While I have detailed three of the most common fixes for URL parameters, every website platform is a little different, and you should therefore take the time to assess each situation on a case by case basis before jumping in and implementing any of the solutions I’ve described.

15 responses to “Avoiding the SEO Pitfalls of URL Parameters”

  1. Victor says:

    Thank you for your article I have been trying to see if I should exclude my specific cruise voyage pages from being indexed. I am getting some duplicate page titles on some of our page. The content on the pages does change slightly for example the prices of the cruise voyage changes and the port we visit sometimes change as well but other than that about 85% of the content remains the same on the page. Should I be excluding the pages with the query string at the end of them? Below is an example of two of our urls that are being indexed by Google currently. http://www.windstarcruises.com/cruise/Mediterranean/Enchanting-Greece-and-the-Amalfi-Coast/ and http://www.windstarcruises.com/Cruise/Mediterranean/Enchanting-Greece-and-the-Amalfi-Coast/?id=92&sid=18946 as you can see the urls are similar except for the added Id and Sid which depending on the cruise date you select it changes to reflect the pricing of that particular voyage. Like I stated earlier though 85% of the content stays the same only the pricing sub tab and sometimes the ports you visit content changes. Any tips you can provide would be greatly appreciate it. Thank you

  2. Rafael Tikva says:

    Ben, I don’t understand why the Booking.com example you give is a bad thing?

    Sure, they’re targeting the same keyword twice – which means they occupy more SERPs real estate by occupying 2 spots in the search results. How exactly is that a bad thing?

  3. Mark says:

    Wanted to say thank you for this article. We actually have an opposite use case and I thought it may help others to discuss. In our situation we use facets to generate our search results (it is a location directory site). Each Facet updates the search parameters to display a subset of results. All are unique, all are location and amenity specific. In order to generate the search the user is required to type in a city name in the search bar that triggers a call to Google Maps API. From there we use the GPS coordinates to structure the “facet” url for that city and generate our results. The user can then filter the results from there.

    We generated 2 sitemap xml files with the top 26,000 cities but Google would not index the content. We already are using jquery to change the page title and page H1 text but nothing was helping. Your article helped a lot. I took your advise on the URL Parameters update and identified that Google WAS already limiting its crawl of any pages starting with the url parameter we generate. I selected “edit” and then selected the appropriate drop downs to instruct Google how to use that content; they provided a preview of how that would impact the index = exactly what we want. Perfect.

    For Canonical this too was off. Our SEO tool was setting a canonical on all of these pages back to the top level domain! Ouch. There was no quick fix to this but because our content is unique I felt comfortable removing canonicals from the site completely. Obviously I will need to watch for unintended consequences on this one.

    Thanks for the article.

  4. It’s hard to come by well-informed people about this subject, however, you seem like you know what you’re talking about! Thanks
    You can learn all about URL meaning here.

  5. Paul Mallam says:

    Great article. We are having some issues with the url parameters and information read here will help me resolve the current site issues.

  6. hipcraft says:

    Thanks for breaking this down. The query string parameters are certainly helpful when digging into the analytics of a site. Great post!

  7. Pete Keyworth Pete Keyworth says:

    Thank you. I’m glad that you found the post useful.

  8. Bill says:

    Thanks, this helps a lot! Few experts thematise this issue.

  9. Kit Builder says:

    Update – the article helped resolve the ongoing issue. Thanks

  10. Awesome stuff nobody write about URL structure. really helpful Thanks

  11. Mike Khorev says:

    Hi Pete. This is an amazing, very comprehensive article on a rarely discussed (yet very important) topic. If I may add, I think the issue of keyword cannibalizing can be fixed with proper SEO tagging (do correct me if I’m wrong), and I also believe URL parameters can work hand in hand with schema markup. Really great article, thanks!

  12. Ashwin says:

    If I open a page url multiple times on regular basis does it rank the page on Google?

  13. Andrea says:

    Hi Pete, thank you for the article! One question: it’s not clear to me which is the preferable solution between canonicalisation and block crawling. In your /toys/trucks example, If you could choose, would you:

    1) canonicalise the urls with parameters as you showed
    2) block the crawling of /toys/trucks?color=blue and /toys/trucks?color=green

    Which solution pass more link juice to the desired url /toys/trucks?
    Thank you!

  14. John Kramer says:

    Thanks Pete. Great article!!

Leave a Reply

Your email address will not be published. Required fields are marked *