Digital Marketing Agency

Is Your Site Losing Crawl Budget? New Best Practices

Visibility is everything, but with millions of websites vying for attention, how can you ensure Google finds and prioritises yours? The answer lies in understanding and optimising your crawl budget. Think of it as Google's allotted time to browse your site. A healthy crawl budget means Google finds and indexes your new content faster, improving your search rankings and driving more traffic your way.

But here's the thing: Google changed the game's rules, especially for larger websites. Let's dive into the latest update and how you can adapt to stay ahead of the curve.

Understanding Crawl budget

Imagine a busy librarian with limited time to catalogue all the books in a massive library. That's essentially what Googlebot does with your website. It crawls through your pages, following links to discover and index your content.

Several factors influence how much time Googlebot spends on your site (your crawl budget):

• Website size: The more pages you have, the more time it takes to crawl.
• Website structure: A clear and logical site architecture makes crawling more efficient.
• Content update frequency: Fresh content encourages Googlebot to visit more often.
• Technical issues: Broken links, slow loading times, and other errors can hinder crawling.

A poor crawl budget means pages can get buried deep in your site, never to be seen by Google, let alone your potential customers.  This translates to lower search rankings, reduced visibility, and ultimately, fewer leads and sales.

What changed?

Google recently updated its guidelines with a significant focus on mobile-first indexing. This means Google primarily uses the mobile version of your website for indexing and ranking.

The key takeaway from this update?  Link parity. Google now explicitly recommends that large websites provide the same links on their mobile version as on their desktop version.

Why is this so important?  Inconsistencies between mobile and desktop versions can confuse Googlebot, wasting precious crawl budget. Here's what Google has to say:

“If your site uses separate HTML for mobile and desktop versions, provide the same set of links on the mobile version as you have on the desktop version. If it’s not possible to provide the same set of links on the mobile version, ensure that they’re included in a sitemap file. Google only indexes the mobile version of pages, and limiting the links shown there can slow down discovery of new pages.”

Who is most affected?

This update is especially critical for:

• Large websites (1 million+ pages) that update content weekly.
• Medium-sized websites (10,000+ pages) with daily content changes.
• Websites with a significant number of  'Discovered – currently not indexed' URLs in Google Search Console. This often signals a crawl budget problem.

Actionable steps to optimise your Crawl budget

Take control of your crawl budget with these actionable steps:

1. Audit your website: Compare your mobile and desktop versions.  Ensure consistent link structure and navigation.
2. Update your sitemap: A sitemap acts as a roadmap for Googlebot. Ensure all essential pages are included for efficient crawling.
3. Prioritise mobile: Ensure your mobile site is user-friendly and easily crawlable. All critical navigation paths should be accessible on mobile devices.
4. Monitor your crawl stats: Regularly check Google Search Console for crawl errors, indexing issues, and other red flags.

Envigo can help

At Envigo, we're passionate about helping businesses thrive online. Our team of SEO experts stays ahead of the latest algorithm updates and best practices, including crawl budget optimisation.

Need help improving your website's visibility and search rankings?  Contact us today for a free consultation and let us help you unlock your website's true potential.

New Google Trends guide for content marketing

One powerful tool that often gets overlooked is Google Trends. But with the release of Google's new guide on using Google Trends for search marketing, there's no excuse for missing out! Published by Google Search Central, this comprehensive documentation explains how to use Google Trends for search marketing. It's a fantastic resource for newcomers and a helpful refresher for seasoned marketers.

Is your site losing crawl efficiency? Check your URL parameters!

URL parameters are essential for many websites, especially e-commerce sites with filters and sorting options. They help users find exactly what they need, but they can also create headaches for SEOs. Why? Because they can lead to duplicate content and wasted crawl budget, essentially confusing Googlebot (the search engine spider that crawls your site).

Reddit Boosts Ad Performance with New Keyword Targeting

Reddit has finally expanded its keyword targeting capabilities, and it's a big deal. Now, instead of just relying on community interests, you can target users based on the actual words they use in their conversations. This means your ads appear alongside relevant discussions, putting your message in front of a highly engaged audience.

Can AI Write Your Video Ads? Meta Says Yes!

Meta's recent announcements at Advertising Week place video and AI firmly in the spotlight. With 60% of user time on Facebook and Instagram now spent watching videos, Meta is equipping advertisers with new tools to leverage this trend.

Is Your Robots.txt File Outdated? Google's New Policy Explained

Ever feel like the internet is a sprawling maze? For search engines like Google, it is! That's where your robots.txt file comes in. It acts like a helpful signpost, guiding Google's crawlers through your website. But with Google's recent update to its robots.txt policy, you might need to give those signposts a refresh.