Reddit blocked Bing and other Search Engines

Microsoft Had Confirmed Reddit Blocked Bing Search

Alright, listen up! Apparently, Reddit has updated its robots.txt file, which will have a major impact on Bing and Other Search Engines. Let’s dive in, shall we?

Reddit Blocked Bing Search
https://www.reddit.com/robots.txt

User-agent: *
Disallow: /

This robots.txt directive tells all web crawlers (user agents) that they are not allowed to access any pages on the website. Essentially, it blocks all search engines from indexing any part of the site.

The Lowdown:

  • On July 1, 2024, Reddit updated its robots.txt file, which is like a set of instructions for search engines on what they can and can’t crawl on the site.
  • And guess what? They’ve gone and prohibited Bing and a bunch of other search engines from crawling the site. But they’ve left Google alone, for now.
  • A Microsoft spokesperson had this to say: “We respect the robots.txt standard. Bing stopped crawling Reddit after they implemented their updated robots.txt file on July 1, which prohibits all crawling of their site.”

The Backstory:

  • Earlier this month, there were rumblings that Reddit had blocked all search engines, including Google. But turns out, that wasn’t the case.
  • Reddit did, however, give the boot to most other search engines, including Bing. And when you filter Reddit results on Bing, you’ll notice there’s a whole lot of nothing there.

The Reason Behind It:

  • According to a Reddit spokesperson, this move is not related to their recent partnership with Google.
  • Nope, they’ve been chatting with a bunch of search engines, but it seems they couldn’t reach an agreement with some of them.
  • Why’s that? Well, Reddit’s saying that some search engines just couldn’t or wouldn’t make solid promises about how they’d use Reddit’s content, including for their AI.

The Bigger Picture:

  • By securing a licensing deal with Google, Reddit’s now in a position to play hardball with other search engines and AI tools.
  • And that’s exactly what they’ve done, blocking most of ’em from crawling their site.
  • Meanwhile, Google’s been driving crazy amounts of traffic to Reddit, and they’re even testing some special treatment for them in their search results.

What It Means:

  • This whole situation makes you wonder if other big-time websites might try to follow in Reddit’s footsteps.
  • I mean, if they can call the shots and decide who gets to access their content, where does that leave smaller publishers and content creators?
  • It’s a bit of a head-scratcher, but one thing’s for sure: Bing users won’t be seeing much new Reddit content anytime soon.

Key Takeaways:

  • Reddit has updated its robots.txt file to prevent Bing and most other search engines from crawling its site.
  • This move is not related to their recent partnership with Google, but rather their inability to reach agreements with some search engines over the use of their content.
  • By securing a licensing deal with Google, Reddit is now in a position to play hardball with other search engines and AI tools.
  • This raises questions about whether other large websites might try a similar approach and the implications for smaller publishers and content creators.
  • Bing users can expect to see a significant drop in new Reddit content in their search results.

Conclusion:
As per the current situation, Google is the only search engine that is allowed to crawl Reddit and index their pages on Google.

Leave a Comment