Duplicate content

Duplicate content refers to content that is reachable in identical or almost identical form under several URLs, either within one website or across several websites. Search engines then face the question of which version should appear in the search results. In online stores, duplicate content is the norm, not the exception: filters, sorting, variants, category paths and manufacturer texts create it automatically. What matters is controlling it so that Google shows the right page.

Internal and external duplicate content

• Internal: the same content under several URLs on the same domain, such as a category with and without a sorting parameter, or a product reachable via two category paths. The store has these duplicates in its own hands.

• External: the same content on different domains, such as manufacturer texts that hundreds of retailers adopt word for word, your own product texts on marketplaces or content that other websites have copied.

Then there is near-duplicate content: pages that differ only in small details, such as product variants with an identical description or category pages consisting only of a product grid without their own text.

Is there a penalty for duplicate content?

As a rule, no. Google has repeatedly made clear that there is no penalty for duplicate content as long as there is no intent to deceive. Instead, Google filters: from a group of identical pages, one is selected and shown, the others are held back. It is different when content is deliberately copied or mass-reproduced to manipulate search results. That can lead to a manual action.

So the problem isn’t the penalty, but the consequences:

• Google chooses the wrong version: for example the variant with a parameter instead of the clean category URL.

• Signals are spread: links and user signals land on several URLs instead of one.

• Crawling is wasted: Googlebot spends time on duplicates instead of new products.

• Other sites rank with your text: with manufacturer texts, the manufacturer or a large marketplace often wins, not your store.

Typical causes in online stores

How to resolve duplicate content

There is a suitable tool for every cause:

• 301 redirect: when a URL is no longer needed, for example with domain variants or merged pages. The strongest solution, because the duplicate disappears.

• [Canonical tag](can): when both URLs should remain reachable, for example with sorting, parameters, variants and several category paths.

• noindex: when a page has no search value, such as internal search results. More on this under meta tags.

• [robots.txt](rtl): when very many filter combinations shouldn’t be crawled at all. Important: blocked pages can show neither a canonical nor a noindex.

• hreflang: when the same language is served for several countries. hreflang shows Google which version is intended for which country and prevents the country pages from competing against each other as duplicates.

• Your own copy: the only solution against external duplicate content. Replacing manufacturer texts with your own descriptions sets you apart from the competition. More on this under unique content.

• Clean internal linking and [XML sitemap](xs): both should consistently point to the canonical URL.

Manufacturer texts: the underestimated problem

In many stores, most product descriptions come from the manufacturer. The same texts then appear at dozens of retailers, on marketplaces and at the manufacturer itself. Google usually shows only one version prominently, often that of the strongest domain. For a store with less visibility, this means its product pages have hardly any chance of good positions, however good the technology is.

Not every product needs its own text. It makes sense to start with the products and categories that bring the most revenue or have real search demand, and to supplement the description there with your own content: experience, application examples, comparisons, answers to typical customer questions. Support is available via content for online stores.

Duplicate content in Shopware and Shopify

Shopware 6 comes with the most important tools: canonical tags for products and variants, a main category per sales channel for products in several categories and SEO URL templates. Typical problem areas are indexable filter and sorting pages, variants without a clear canonical rule and several sales channels with the same content without hreflang.

Shopify creates its own patterns through its fixed structure: products can be reached via /products/ and via collection paths, variants via ?variant=. The theme usually sets the canonical tag correctly. With Shopify Markets and several countries in the same language, clean hreflang information is decisive.

More on the peculiarities of the systems under Shopware SEO and Shopify SEO.

How to find duplicate content

• Google Search Console: in the indexing report, messages such as “Duplicate, Google chose different canonical than user” or “Duplicate without user-selected canonical” show where Google sees duplicates.

• Search operators: a distinctive sentence from a product description in quotation marks shows who uses the same text. A query such as site:your-store.com inurl:order= shows indexed sorting pages. More queries are in the article on Google search operators.

• Crawling tools: programs that run through the store like a search engine find duplicate titles, descriptions and content.

If another website deliberately copies your content, a request for removal and, if necessary, a copyright complaint to Google helps. More on this in the entry on negative SEO.

Frequently asked questions

Will my store be penalised for duplicate content?

As a rule, no. Google filters duplicate pages instead of penalising them. A manual action only threatens in the case of deliberate manipulation.

How much similarity is allowed?

There is no fixed percentage. What matters is whether a page offers its own added value. The same building blocks, such as shipping information or size charts on many pages, are not a problem.

Are manufacturer texts forbidden?

No, but they make it hard to rank with these pages. Your own texts for the most important products are one of the most effective measures for stores.

Are my country domains for Germany and Switzerland duplicate content?

Without hreflang, they can compete against each other. With correct hreflang information, Google knows which version is intended for which country.

What is the difference between a canonical tag and noindex for duplicates?

The canonical tag consolidates the signals on the main version. noindex removes a page from the index without passing on its signals. For duplicates, the canonical tag is almost always the better choice.