whatsapp icon

An Enterprise Technical SEO Case Study: Machine Learning Solution For Internal Link Structure On Expedia

Expedia already had an existing internal link structure widget which was providing some level of internal linking, however the links were being rapidly outdated by changes in the site caused by ongoing business activity.

Challenge

• 18 Million pages site with too many page templates
• Over 50k links pointing to 404 pages
• Multiple teams working on website
• Inventory changing frequently
• Time consuming update every 6 months
• Needed to use historical solution and include manual overrides

Results

• Over 50k links pointing to 404s removed
• 25% of useless links removed
• Designed a new architecture with approx. 99% cleanliness
• Daily recommendations and automated updates sent

Goals

• Fix the internal link structure for a very large site (25 Million pages indexed) with over 50k links pointing to 404 pages
• Ensure all internal links point to canonical pages and reduce 301 redirects
• Design a dynamic, automated solution to provide ongoing updates to the internal architecture

What To Do
Expedia already has had an existing internal link structure widget which was providing some level of internal linking, however the links were being rapidly outdated by changes in the site caused by ongoing business activity.

Built a Customised Web Crawler
• The currently available crawler solution would not make the cut for this site.
• IMWT has ended up upgrading their own customised web crawler to rapidly and efficiently crawl through the website to identify broken or underperforming links.

Working closely in regards to the business concerns of the client, the agency also built an automated logic engine which identified the best action to take with a broken link – to change its target, remove it, or to flag it for human attention.

Developed a Recommendation Engine
Using some of the agency’s existing Machine Learning solutions, In Marketing We Trust developed an algorithm capable of suggesting the most accurate replacement for an error page and turned a painstaking manual process into an automated one.

The internal link structure is now progressively kept up to date, with fix recommendations being issued every 24 hours. This has brought the number of bad internal links down from tens of thousands to less than a thousand at any given time, and has considerably improved the valid-to-invalid Googlebot crawl ratio, meaning that the site is updated faster and is seeing improved organic results on search engines.

Connected Crawl Rate With Sales Numbers
The next step on this project was to start analysing the landing page conversion outcomes of pages, and influence the internal link structure to prioritise pages with better conversion outcomes, while maintaining the breadth of internal links necessary to ensure that crawlers can still find everything on the site, and that internal links keep high levels of relevancy.


Results

• Over 50k links pointing to 404s removed
• 25% of useless links removed
• Designed a new architecture with approx. 99% cleanliness
• Daily recommendations and automated updates sent
• Integrated with historical system, including manual overrides
• Tied up to business metrics, increasing sales

Want to see your content here?

It’s the right time to go global! List your agency among the leaders of the industry, promote your works, create original content and find your new team members!

SUBMIT CONTENT

WORK | Case Studies

Dosist Redesigned Their Website With Y Media Labs

dosist — recognized as the #2 Hottest Startup by Linkedin (2019), ...

WORK | Case Studies

CSI Media Develop New Websites For The Priory

The Priory approached CSI Media to deliver a new website for ...

WORK | Case Studies

Innovative Solutions For Next Generation Banking

As a result of the data analysis and optimization practices implemented ...