Bingbot crawling errors, causing 404s Bingbot keeps trying to spider nonexistent URLs on my blog. There are no links leading to these nonexistent URLs, but rather, Bingbot is screwing up. There are specific strings that Bing keeps adding to my URLs, so I set up redirects via .htaccess and now have Bingbot redirected to the following URL every time it does that:
It's very possible that no live person at Bing will ever see that, but I love the irony of redirecting Bing's bots to their competitor, with countless posts about how Bing keeps causing these same errors on other people's sites.
In case anyone runs across this post in a search for Bingbot crawling errors or Bingbot 404s and wants to know how I redirected them, I used RedirectMatch 301.
Those are not the right URLs for registering or administration, and the only one who should be accessing the administration panel is me. I think those last three are game plugins for Wordpress, and while there have been others, those three have been tried multiple times.
Anyway, whenever someone tries to access those URLs, I send them to a page that looks like this (caution: 2mb animated background). The link I posted here won't log anything, but the actual page I send them to logs their IP and other info about them.
For those who don't want to click on the link, it has an animated fire background with the following in a red circle centered on the page:
Your failure to hack or spam has been logged. If you know what's good for you, you will stay away.
I can't just block those particular bots, because their IPs change, and it only says "Mozilla" and nothing else as their user agent. But in addition to being redirected to that page, I also set it up so the bots will receive a "403 Forbidden" code instead of a 404.
Re: Bingbot crawling errors, causing 404s You are beyond clever and ornery! I don't honestly care for Bing.....but I'm glad you are able to redirect them to their competitor...THAT is just freaking hilarious!
---
“Freedom and democracy are dreams you never give up.”
Re: Bingbot crawling errors, causing 404s They would probably need a lot of people constantly reviewing their logs before anyone would spot it, since it's all automated.
Re: Bingbot crawling errors, causing 404s Well, it looks like redirecting those hacker wannabes to a fiery page with a 403 (forbidden) code is making them even more determined to access these pages that don't exist.