Digital Musings

Backlinks are like votes of confidence from other websites. Search engines like Google use them to size up your site's authority and relevance. But if you're getting votes from the wrong crowd (think spammy or irrelevant sites), it can raise red flags.

Visibility is everything, but with millions of websites vying for attention, how can you ensure Google finds and prioritises yours? The answer lies in understanding and optimising your crawl budget. Think of it as Google's allotted time to browse your site. A healthy crawl budget means Google finds and indexes your new content faster, improving your search rankings and driving more traffic your way.

But here's the thing: Google changed the game's rules, especially for larger websites. Let's dive into the latest update and how you can adapt to stay ahead of the curve.

Understanding Crawl budget

Imagine a busy librarian with limited time to catalogue all the books in a massive library. That's essentially what Googlebot does with your website. It crawls through your pages, following links to discover and index your content.

Several factors influence how much time Googlebot spends on your site (your crawl budget):

• Website size: The more pages you have, the more time it takes to crawl.
• Website structure: A clear and logical site architecture makes crawling more efficient.
• Content update frequency: Fresh content encourages Googlebot to visit more often.
• Technical issues: Broken links, slow loading times, and other errors can hinder crawling.

A poor crawl budget means pages can get buried deep in your site, never to be seen by Google, let alone your potential customers.  This translates to lower search rankings, reduced visibility, and ultimately, fewer leads and sales.

What changed?

Google recently updated its guidelines with a significant focus on mobile-first indexing. This means Google primarily uses the mobile version of your website for indexing and ranking.

The key takeaway from this update?  Link parity. Google now explicitly recommends that large websites provide the same links on their mobile version as on their desktop version.

Why is this so important?  Inconsistencies between mobile and desktop versions can confuse Googlebot, wasting precious crawl budget. Here's what Google has to say:

“If your site uses separate HTML for mobile and desktop versions, provide the same set of links on the mobile version as you have on the desktop version. If it’s not possible to provide the same set of links on the mobile version, ensure that they’re included in a sitemap file. Google only indexes the mobile version of pages, and limiting the links shown there can slow down discovery of new pages.”

Who is most affected?

This update is especially critical for:

• Large websites (1 million+ pages) that update content weekly.
• Medium-sized websites (10,000+ pages) with daily content changes.
• Websites with a significant number of  'Discovered – currently not indexed' URLs in Google Search Console. This often signals a crawl budget problem.

Actionable steps to optimise your Crawl budget

Take control of your crawl budget with these actionable steps:

1. Audit your website: Compare your mobile and desktop versions.  Ensure consistent link structure and navigation.
2. Update your sitemap: A sitemap acts as a roadmap for Googlebot. Ensure all essential pages are included for efficient crawling.
3. Prioritise mobile: Ensure your mobile site is user-friendly and easily crawlable. All critical navigation paths should be accessible on mobile devices.
4. Monitor your crawl stats: Regularly check Google Search Console for crawl errors, indexing issues, and other red flags.

Envigo can help

At Envigo, we're passionate about helping businesses thrive online. Our team of SEO experts stays ahead of the latest algorithm updates and best practices, including crawl budget optimisation.

Need help improving your website's visibility and search rankings?  Contact us today for a free consultation and let us help you unlock your website's true potential.

One powerful tool that often gets overlooked is Google Trends. But with the release of Google's new guide on using Google Trends for search marketing, there's no excuse for missing out! Published by Google Search Central, this comprehensive documentation explains how to use Google Trends for search marketing. It's a fantastic resource for newcomers and a helpful refresher for seasoned marketers.

URL parameters are essential for many websites, especially e-commerce sites with filters and sorting options. They help users find exactly what they need, but they can also create headaches for SEOs. Why? Because they can lead to duplicate content and wasted crawl budget, essentially confusing Googlebot (the search engine spider that crawls your site).

Reddit has finally expanded its keyword targeting capabilities, and it's a big deal. Now, instead of just relying on community interests, you can target users based on the actual words they use in their conversations. This means your ads appear alongside relevant discussions, putting your message in front of a highly engaged audience.

Meta's recent announcements at Advertising Week place video and AI firmly in the spotlight. With 60% of user time on Facebook and Instagram now spent watching videos, Meta is equipping advertisers with new tools to leverage this trend.

Ever feel like the internet is a sprawling maze? For search engines like Google, it is! That's where your robots.txt file comes in. It acts like a helpful signpost, guiding Google's crawlers through your website. But with Google's recent update to its robots.txt policy, you might need to give those signposts a refresh.

Starting October 15th, you can wave goodbye to the 60-second limit. YouTube is tripling the maximum length of Shorts, allowing creators to upload videos up to 3 minutes long. This applies to both square and vertical videos, giving you more flexibility in how you present your content.

Before you start panicking and envisioning your website banished to the depths of the internet, let's take a breath. This isn't a total revamp, more like a bit of a spring clean. Think of it as Google giving their spam documentation a good tidy-up – a bit of dusting, rearranging the furniture, and adding a few new bits and bobs.

Remember when you could easily peek into the past versions of a website using Google Cache? It was a handy tool, wasn't it? Whether you were an SEO specialist tracking competitor changes or a curious user wanting to see how a site looked back in the day, Google Cache was your time machine.

Google's been busy tidying up behind the scenes, and their recent overhaul of the crawler documentation is causing a bit of a stir in SEO. You might be wondering, "Does this even matter to me and my website?". In short, yes, it absolutely does.

As data privacy regulations tighten and consumer awareness grows, advertisers are constantly seeking ways to respect user privacy while still delivering effective targeted campaigns. Google's latest move, Confidential Matching, aims to strike this balance.