Duplicate content is an issue for SEO because it dilutes ranking signals: when the same page exists at several URLs, or the same text on several pages, links, clicks and relevance get split between copies until Google decides which one counts, and the one Google picks isn’t always the one you’d pick. If this question came from the HubSpot certification exam, that first clause is the answer it wants. The rest of this page is what the sentence means on a real site, because “dilutes” is doing a lot of work in it and most of the pages ranking for this query stop there.
There is no duplicate content penalty, and there hasn’t been one for as long as Google has been writing about it. The 2008 post everyone cites says that duplicates are filtered, not punished, and that the exception is duplication meant to deceive. The current spam policies say the same in different words. What duplicate content costs comes through three ordinary mechanisms instead, each of which can be seen in Search Console by name.
What Google does with duplicate content
When Google finds pages it considers the same, it groups them into a cluster and selects one URL as the canonical; its canonicalization documentation says plainly that the one it selects may not be the one you declared. The canonical is the one that gets indexed and shown; the rest are crawled less often and don’t appear in results. Signals for the cluster, including links pointing at the copies, are consolidated onto the canonical. This is the part the “dilution” answer leaves out: the splitting of signals is what happens before Google clusters the pages, or when it doesn’t recognize them as duplicates at all; once clustered, the signals converge. How common that is nobody measures well: Matt Cutts put duplicate content at 25 to 30 percent of the web in 2013, and a Semrush crawl of 100,000 sites a few years later found it on about half of them, and both numbers count the harmless kind along with the costly. A site’s duplicate content problem is therefore one of two things: pages Google clustered with the wrong canonical, or pages similar enough to compete and different enough to escape clustering.
How to find duplicate content: three statuses, one crawl, one search
Both kinds are visible in the Page indexing report, under three statuses:
- Alternate page with proper canonical tag. The page declares a canonical, Google agrees, the copy is excluded. This is the system working, and nothing on this row needs fixing.
- Duplicate without user-selected canonical. Google found duplicates, none of them declares a preference, and it chose for you. Usually harmless, sometimes the parameter version wins.
- Duplicate, Google chose different canonical than user. The page declares a canonical and Google overruled it. This is the row to read first, because each URL in it is a case where your preference and Google’s disagree, and the reasons are usually a stronger signal pointing the other way: internal links to the other version, a redirect, an hreflang set, or a copy on a stronger domain.
Inspecting a URL from the third row shows both the declared and the Google-selected canonical side by side. Which pages Google treats as duplicates it decides on content, not on the tag, and the tag is a hint it weighs with everything else. A 301 is not a hint, which is why it’s the stronger fix where it’s possible.
The statuses cover what Google clustered. The near-duplicates that escaped clustering take two more reads. Inside the site, a crawl’s near-duplicate report at a 90% body-text threshold pairs the pages Google treats as different and users wouldn’t, and the Performance report filtered to one query shows whether two of your URLs trade places for it week to week. Outside the site, a distinctive sentence from the page searched in quotation marks lists every domain carrying it, which is how a syndicated copy or a scraper outranking you gets found before the status does.
Why duplicate content costs rankings: three mechanisms
The wrong URL gets chosen. A category page with clean URL and a filtered version of it with ?sort=price are duplicates to Google. If internal links, a sitemap entry or a redirect favour the parameter version, that’s the canonical, and the clean URL is the copy. The ranking doesn’t disappear, it attaches to a URL with a worse title, no breadcrumb and a tracking parameter in it, and the category keeps earning a little less than it did for as long as nobody notices. Staging domains left open do the same at site scale, and that case has its own row in the technical SEO audit guide.
Signals split while the pages compete. Two articles on the same question, or a subcategory that lists the same products as a filter, are near-duplicates: not similar enough to cluster, similar enough to rank for the same queries. Search Console shows this as a query whose landing page changes week to week. Neither page accumulates the links or the clicks the one page would have. This is cannibalization, and it’s the form of duplication that content teams create on purpose, one article per keyword variant.
Crawling goes to copies. Google’s crawl budget documentation lists duplicate content among the things that waste crawl capacity, and on a site with a hundred thousand filter combinations it does: the crawler spends its visits on pages that will never be indexed while the new product pages wait. The same documentation says crawl budget is a concern for large sites, roughly a million pages or ten thousand that change daily, and that most sites don’t need to think about it. A 400-page site with tag archives has duplicates and no crawl problem, and “wasted crawl budget” offered as the first reason to every site is a large-site argument repeated to small-site readers.
When it becomes a penalty: scraped and scaled content
Two cases in the spam policies, both of them manual actions rather than anything an algorithm applies on its own. Scraped content: a site built from other sites’ pages, with or without light rewriting. Scaled content abuse: many pages generated to rank rather than to help, which includes a thousand location pages that differ by one city name. In both, what’s being judged is the intent rather than the matching text. A product page that reuses the manufacturer’s description is one of two hundred pages with the same description, nothing more, and Google will show whichever has the most of everything else: reviews, stock, price, a reason to be chosen.
Where the duplicates come from, and what each source looks like
The sources differ by what the page is for. Commercial sites generate duplicate content mechanically, out of the CMS, the filters and the category tree, and the fix is a rule in the template. Blogs generate them editorially, out of similar topics, tags and republishing, and the fix is a decision per page.
| Source | How it shows | Fix |
|---|---|---|
| http and https, www and non-www, trailing slash and none | Every page in the site twice or four times; “Duplicate without user-selected canonical” in bulk | One 301 from every variant to one host, one scheme, one slash rule |
| Filter, sort and tracking parameters on category pages | Thousands of excluded URLs; sometimes a parameter URL as the chosen canonical | Canonical to the clean URL on every filtered view; filters that change the product set enough to deserve a page get a clean URL of their own |
| One product under several category paths | Three product URLs, Google picks one per product, not always the same one across the catalogue | One product URL independent of the category path; the category pages link to it |
| Manufacturer descriptions reused across retailers | Nothing in Search Console; the page is indexed and ranks below the retailers with more on the page | Reviews, specs in your own structure, stock and delivery; rewriting the description helps less than the guides claim |
| Pagination | Page 2 onward excluded or indexed with the same title as page 1 | Each page self-canonical with its own title; never canonical to page 1, which hides the products on page 2 |
| Tag, author and date archives on a blog | Hundreds of thin index pages listing the same excerpts | noindex on archives nobody searches for; keep category pages that have their own text |
| Two or three articles on one question | A query whose landing page alternates in the Performance report | Merge into the stronger one with a 301; the merged text has to be better than either, or the merge only removes a page |
| Syndication to Medium, LinkedIn or a partner site | The copy outranks the original; the original sits in “Duplicate, Google chose different canonical” | The copy carries noindex, or you accept that it may rank instead of you; a canonical on the copy is no longer enough (see below) |
| Scrapers | Usually nothing; occasionally a scraper outranks a weak original | Publish first, get indexed first, link internally; a DMCA request for the persistent cases |
| Staging, dev and PDF copies | A second domain in results, or a PDF ranking for the page’s query | 401 plus X-Robots-Tag: noindex on staging; canonical from the PDF to the HTML page via the same header |
Syndication deserves its own paragraph, because the usual advice on it is out of date. Until 2023, Google’s documentation said a syndicated copy should carry a canonical pointing to the original. That line was removed, and the documentation now says that if you don’t want the copies to appear in results, the partner should block them with noindex. The first section explains why: a canonical is a hint, and when a stronger domain hosts the copy, Google can decide the copy is the canonical regardless of the tag. Medium and LinkedIn do not set noindex on your behalf. Publishing on them is a choice to let them rank for your text.
Canonical, 301 or noindex: which one, when
Three tools, and they do different things, which is why the question “should I canonical or redirect” has an answer rather than a preference.
A 301 removes the duplicate: the URL stops existing as a page, signals pass to the target, and there’s nothing left for Google to choose between. Use it whenever the duplicate doesn’t need to be visited by anyone: host variants, old URLs, merged articles. A canonical keeps the duplicate reachable and asks Google to index another URL instead: filtered views users need to see, printer pages, the product under its second category path. Google may decline. Noindex keeps the page out of the index without consolidating anything; links into a noindexed page stop counting for the rest of the site over time, which makes it the wrong tool for pages with inbound links and the right one for archives and syndicated copies. Robots.txt is not on the list. It stops crawling, not indexing, and a blocked URL with links to it can be indexed without content; for parameter sprawl on a very large site it saves crawl, and that’s the one job it has here.
Before any of them, there’s the question that gets skipped: is the duplicate content a problem at all. “Alternate page with proper canonical tag” with ten thousand URLs in it is the system working and needs no ticket. The rows that need a decision are the ones where Google chose against you, which is a technical ticket, and the near-duplicates that never got clustered because they were written to be different, which is an editorial one that no tag fixes.
Duplicate content FAQ
Is there a Google duplicate content penalty?
No. Duplicates are clustered and one is shown; the others are excluded from the index, and nothing else happens to them. The exceptions, scraped content and scaled content abuse, are spam policies enforced by manual action, and what they’re enforced against is intent to manipulate, which two matching pages don’t show on their own. The same myth exists around redirects, and the 301 redirect penalty question has the same answer for the same reason.
Does syndicating my articles hurt my rankings?
It can, if the copy ranks instead of the original, which happens when the host is stronger than you and the copy isn’t noindexed. A canonical tag on the copy used to be the recommended protection; Google removed that recommendation in 2023 because it didn’t reliably work.
Are manufacturer product descriptions duplicate content?
Yes, across every retailer that uses them, and Google expects that. The page isn’t filtered for it. It ranks on what’s around the description, so the time goes into reviews, structured specs, availability and price rather than into rewriting a paragraph two hundred competitors also have.
How much duplicate content is acceptable?
The share doesn’t matter, and no threshold exists in anything Google has published. A store with ten thousand URLs in “Alternate page with proper canonical tag” has a large share and no problem. The row that matters is “Duplicate, Google chose different canonical than user”: one URL in it that carries revenue is a problem, and a thousand parameter copies clustered correctly are not.
Does duplicate content affect AI Overviews and ChatGPT?
Through the same canonical decision, not a new one. AI Overviews and the answer engines draw on pages that are indexed and selected, so when Google clusters your page with a copy and picks the copy, the copy is what gets quoted; a syndicated article on a stronger domain is cited under that domain’s name. I don’t have a number for how often that happens, and nobody publishing one has shown the sample. What can be read is the same third status, and the fix is the same noindex on the copy.