Semantic similarity between the old and the new website

During a redesign, every URL of the old website must be redirected to the page of the new website that replaces it. When addresses, titles and site structure change, the most reliable clue left is the content: a product page keeps its description, an article keeps its text. Semantic similarity measures how much two pages talk about the same thing. This is what Swan calculates between the pages of the old website and those of the staging site to suggest your redirect plan.

This page explains that calculation and its limits. The semantic analysis of your website before the migration (topics, lexical fields, pages competing with each other, pages to merge) is a different task, described in our article on the semantic analysis of a website before its migration.

Matching pages by their address fails as soon as the site structure changes: rewritten or translated slugs, renamed categories, CMS identifiers in the address, prefixes imposed by the new platform (/products/ and /collections/ on Shopify). Matching by title fails when titles are rewritten or too generic. Content, on the other hand, usually survives the redesign.

Targeting the closest page by content is also what preserves rankings: a redirect to a page on the same topic passes on the relevance of the old one, whereas a redirect to the home page or a generic category may be treated by Google as a page not found. See the SEO risks of a redirect.

Swan crawls the source URLs (the old website, plus the URLs you import from Search Console, a backlink tool or your logs) and those of the test domain, the staging version of the new website. From each page, it extracts the text of the content, leaving out what repeats from page to page, such as the menu, the footer or shared blocks: without this, all the pages of a website would look alike.

Each text is turned into a vector, a series of numbers that places the page in a space where content on the same topic sits close together, even when it does not use exactly the same words. That is the difference with simply counting shared words: two versions of the same product page, one of them rewritten, stay close.

For each source URL, Swan measures how close its vector is to that of each page of the staging site and suggests the closest candidates, ranked, with their score. A high score, clearly above the next candidate, signals a safe match. Two candidates neck and neck, or a low score, signal a URL to look at.

The calculation suggests, it does not decide. You approve the obvious matches in bulk, choose among the candidates when in doubt, enter another target, or set the URL to 410 when its content has no equivalent on the new website. The redirects are then checked on the test domain, then on the production domain, and exported. See the redirect plan.

• A page without text (gallery, form, login page) has no content to compare: its target is chosen by hand.
• Close does not mean equivalent: a category and one of its products can be very close in vocabulary without answering the same intent.
• There is always a least distant candidate, even for a page removed from the new website: the score tells you whether the match makes sense.
• The content must be on the staging site before the crawl: without migrated content, there is nothing to compare. See editorial content migration.

The two are often confused. Semantic similarity between the old and the new website serves the redirect plan: it comes into play once the staging site is built. The semantic analysis of a website comes before: it maps the topics and queries of the old website and decides what the new one will contain, including the pages to merge, whose redirects will then converge on a single target. For that step, read our method for semantic analysis before a migration and our article on migrating websites that merge.