The Adworkz Blog

Duplicate content can hurt SEO on your site

September 8, 2010 | by Andrea

Duplicate individuals

If your website content looks too similar to other websites, search engines could filter it out. Image: Flickr/ oskay / CC- BY

When you are trying to optimize your website for search engines, it is worth it to make sure you’re not using duplicate content in a way that will degrade the popularity of your site. Duplicate content can be put to work for you, but it can also harm your search engine optimization if used incorrectly. Understanding how search engines see duplicate content, how it is indexed and the effect of duplicate content is important when you are responsible for or concerned about the rank of a website.

How search engines see duplicate content

Search engines include filters for duplicate content for a multitude of reasons. Search engines are in the business of providing the most relevant search results to users. When multiple websites have the exact same content, a search engine runs the risk of showing multiple results that don’t give additional information. This could be multiple websites that have the same publisher’s description of a book, different URL styles for the same website (www.url.com vs. url.com) or a website that uses copyrighted content. In short, a search engine sees two websites with duplicate content as two websites that should not appear in the same list of search results.

How search engines index content

Indexing the entire internet is a huge task. Search engines do this by sending out robots – programs – that read websites and categorize them according to the keywords they find on sites. This text is stored on the search engine’s servers for comparison with other websites. When the search engine finds two websites that have content that is the exact same or very similar, it flags both websites for filtering. Only one of these filtered websites will likely ever be displayed on search results. Depending on the search engine, which of the filtered websites will be displayed could be a matter of which site gets more traffic, which is more trusted or which had the content up first. Either way, most of the duplicate content websites simply will not show up on search results.

The effects of duplicate content

If your website is found to have duplicate content, a search engine may very well filter it out. While some search engine optimization experts call this the “duplicate content penalty,” it is not a true penalty. It is a filter – which means it is much easier to avoid and work around. Most simply, just avoid putting duplicate content on your website. Even if you get an e-mail or RSS content feed, find a feed that you are allowed to make edits to, then edit the articles. There are also content feed programs that provide unique versions of articles. You can also avoid the duplicate content filter by creating and posting unique content. If you are not sure whether your content is unique, run it through a duplicate content checker to ensure it isn’t seen as duplicate.

Comments(4)   Posted in: Affiliate Marketing

  • Franrose

    You won’t necessarily get “penalized” for duplicate content, but it can seriously hurt your business, especially if the company revenue is going to depend heavily on search traffic. If you do decide on putting duplicate content on your website, however, make sure to add some additional value to that content before republishing it.

  • David

    As a matter of fact, syndicated content can and does hurt many sites. All you have to do is examine the results for top news topics and then check your local newspaper websites that receive AP Wire to see if they have the same content. Then ask yourself why you can’t find them in the search results.

    Another observation you can do is just look through the results for various top level news topics and count the number of articles ranking well that simply just have syndicated content. Likely you will not find many. You could see some that have overwhelming authority that do show like news.yahoo.com which publishes AP. However, unless your site is of that size and magnitude, re-publishing duplicate content is not always the best move.

    The main logic to keep in mind is that Search Engines have a quest, a main goal, and that is to show the most authoritative and trustworthy sites. One easy way for them to determine which sites are authoritative is to keep track of which sites are the original publishers of great content and who are simply “Copy Cats.”

    If you were Google or Bing would you want to give preference to sites with possibly stolen content like many spam sites do? What about sites that simply scrape the web and re-post whatever they find? Do you think the public would best be served if these types of sites were given preference in ranking?

    If your answer is no, then you now understand what “hurt your site” means. If your site is not “preferred” by the search engines you can look at that as a bad thing. Especially in cases where you are in fierce competition with similar quality and size of websites, your site can lose the trust of search engines by publishing syndicated content.

    It is much better to just write your own version of the story and keep away from having pages in your site relegated to the “omitted results” area linked to at the bottom end of many search queries today.

    That being said, if you have little competition and you are only fighting to rank locally then syndicated content can easily give your site an edge that your competitors don’t. That edge is scale. If they are not publishing quality content or even syndicated content, you have a chance to have your site be considered the most authoritative in your small niche.

    If they are publishing either syndicated content or just low quality content you can still beat them if the syndicated content you publish is highly specific to what your site wants to rank for. You have to add original content in here and there but that is pretty easy.

    So, for the most part duplicate content is bad unless you don’t face much competition. In that case, there may not be enough money in that market to be worth spending your time on in the first place. Shying away from vanity rankings is a smart thing to do. ;)

  • Andrea

    Connor,
    You are right – syndicated content can be very helpful to both SEO and site visitors. However, search engines, as a general rule try to filter out large blocks of duplicate content because it is uphelpful to their customers. Google goes into depth explaining this in their webmaster tools article on Duplicate Content. It is true that for startup websites, getting syndicated content can help their SEO. That syndicated content, though, if it is not edited, increases the liklihood that the page with the syndicated content will be filtered out of results.

    As with all SEO and marketing, nothing is absolute — but given the best practices that search engines have given webmasters, and observed results, this is most often the case.

  • Connor Bringas

    This is totally false. Its called syndicated content and it doesnt hurt your seo efforts. WRONG!
    Do some research