Runboard.com
Слава Україні!



🙂       Use the black navigation bar to log in or create your account.

Jump to Page:  1  2  3 ... 7  8  9 

 
Lesigner Girl Profile
Live feed
Blog
Friends
Miscellaneous info

Minerva
Head of Runboard staff

Registered: 11-2005
Posts: 9606
Karma: 132 (+147/-15)
ReplyQuote
Bingbot crawling errors, causing 404s


Bingbot keeps trying to spider nonexistent URLs on my blog. There are no links leading to these nonexistent URLs, but rather, Bingbot is screwing up. There are specific strings that Bing keeps adding to my URLs, so I set up redirects via .htaccess and now have Bingbot redirected to the following URL every time it does that:

http://www.google.com/#output=search&q=Bingbot+errors

It's very possible that no live person at Bing will ever see that, but I love the irony of redirecting Bing's bots to their competitor, with countless posts about how Bing keeps causing these same errors on other people's sites.

In case anyone runs across this post in a search for Bingbot crawling errors or Bingbot 404s and wants to know how I redirected them, I used RedirectMatch 301.

RedirectMatch 301 /(.*)string-they-keep-adding http://URL-to-redirect-them-to

It worked for me, because the string they keep adding doesn't exist in any URL on my site.

---
Runboard Knowledge Base
Runboard Support Forums
Find other message boards
6/10/2013, 12:24 am Link to this post PM Lesigner Girl Read Blog
 
SKOKEY Profile
Live feed
Blog
Friends
Miscellaneous info

Citizen

Registered: 12-2005
Province: Bennington, NH
Posts: 1308
Karma: 38 (+39/-1)
ReplyQuote
Re:


You so damn clever, girl.
6/10/2013, 1:02 am Link to this post PM SKOKEY
 
Lesigner Girl Profile
Live feed
Blog
Friends
Miscellaneous info

Minerva
Head of Runboard staff

Registered: 11-2005
Posts: 9606
Karma: 132 (+147/-15)
ReplyQuote
Re: Bingbot crawling errors, causing 404s


And evil. emoticon

I've had some bots trying to hack and spam there, too, trying to access nonexistent URLs like these:

     /administrator/
     /register.php
     /2009/register.php
     /2009/04/register.php
     /2009/04/xhtml-and-css-glossary-child-element/register.php
     /2009/03/css-border-property/wp-signup.php
     /postavljanje-bombi/
     /pirati/
     /leteci-glavomet/

Those are not the right URLs for registering or administration, and the only one who should be accessing the administration panel is me. I think those last three are game plugins for Wordpress, and while there have been others, those three have been tried multiple times.

Anyway, whenever someone tries to access those URLs, I send them to a page that looks like this (caution: 2mb animated background). The link I posted here won't log anything, but the actual page I send them to logs their IP and other info about them.

For those who don't want to click on the link, it has an animated fire background with the following in a red circle centered on the page:

Your failure to hack or spam has been logged. If you know what's good for you, you will stay away.


I can't just block those particular bots, because their IPs change, and it only says "Mozilla" and nothing else as their user agent. But in addition to being redirected to that page, I also set it up so the bots will receive a "403 Forbidden" code instead of a 404.

---
Runboard Knowledge Base
Runboard Support Forums
Find other message boards
6/10/2013, 1:50 am Link to this post PM Lesigner Girl Read Blog
 
Queenyforever Profile
Live feed
Blog
Friends
Miscellaneous info

Ignore me.

Registered: 01-2007
Province: Just north of the clouds...
Posts: 1467
Karma: 48 (+48/-0)
ReplyQuote
Re: Bingbot crawling errors, causing 404s


 emoticon You are beyond clever and ornery! I don't honestly care for Bing.....but I'm glad you are able to redirect them to their competitor...THAT is just freaking hilarious! emoticon

---

“Freedom and democracy are dreams you never give up.”

6/10/2013, 2:23 pm Link to this post PM Queenyforever Read Blog
 
Lesigner Girl Profile
Live feed
Blog
Friends
Miscellaneous info

Minerva
Head of Runboard staff

Registered: 11-2005
Posts: 9606
Karma: 132 (+147/-15)
ReplyQuote
Re: Bingbot crawling errors, causing 404s


I don't like Bing, either, and I wish their programmers could see that redirection. emoticon

---
Runboard Knowledge Base
Runboard Support Forums
Find other message boards
6/10/2013, 10:59 pm Link to this post PM Lesigner Girl Read Blog
 
Queenyforever Profile
Live feed
Blog
Friends
Miscellaneous info

Ignore me.

Registered: 01-2007
Province: Just north of the clouds...
Posts: 1467
Karma: 48 (+48/-0)
ReplyQuote
Re: Bingbot crawling errors, causing 404s


Who knows......someone might bring it to their attention! emoticon

---

“Freedom and democracy are dreams you never give up.”

6/11/2013, 5:30 pm Link to this post PM Queenyforever Read Blog
 
Lesigner Girl Profile
Live feed
Blog
Friends
Miscellaneous info

Minerva
Head of Runboard staff

Registered: 11-2005
Posts: 9606
Karma: 132 (+147/-15)
ReplyQuote
Re: Bingbot crawling errors, causing 404s


They would probably need a lot of people constantly reviewing their logs before anyone would spot it, since it's all automated.

---
Runboard Knowledge Base
Runboard Support Forums
Find other message boards
6/11/2013, 8:59 pm Link to this post PM Lesigner Girl Read Blog
 
Lesigner Girl Profile
Live feed
Blog
Friends
Miscellaneous info

Minerva
Head of Runboard staff

Registered: 11-2005
Posts: 9606
Karma: 132 (+147/-15)
ReplyQuote
Re: Bingbot crawling errors, causing 404s


Well, it looks like redirecting those hacker wannabes to a fiery page with a 403 (forbidden) code is making them even more determined to access these pages that don't exist. emoticon

06/15/13 03:16 PM	202.46.50.158 	/leteci-glavomet/

06/15/13 04:58 PM 202.46.55.22 /postavjanje-bombi/
06/15/13 04:58 PM 202.46.48.24 /pirati/
06/15/13 04:59 PM 202.46.62.93 /leteci-glavomet/
06/15/13 05:47 PM 119.63.193.196 /leteci-glavomet/
06/15/13 06:32 PM 119.63.193.196 /postavjanje-bombi/
06/15/13 06:33 PM 119.63.193.132 /pirati/
06/15/13 06:33 PM 119.63.193.130 /leteci-glavomet/
06/16/13 10:53 AM 202.46.48.27 /borba-slonova/
06/16/13 12:36 PM 202.46.56.51 /borba-tenkova/
06/16/13 08:21 PM 202.46.57.83 /postavjanje-bombi/
06/16/13 08:21 PM 202.46.54.40 /postavjanje-bombi/
06/16/13 08:21 PM 202.46.61.28 /pirati/
06/16/13 08:21 PM 202.46.56.54 /pirati/
06/16/13 08:21 PM 202.46.62.94 /leteci-glavomet/
06/16/13 08:22 PM 202.46.48.15 /leteci-glavomet/


Maybe I should start redirecting them to this video instead of the flaming 403 page. emoticon



---
Runboard Knowledge Base
Runboard Support Forums
Find other message boards
6/17/2013, 6:22 am Link to this post PM Lesigner Girl Read Blog
 
Lesigner Girl Profile
Live feed
Blog
Friends
Miscellaneous info

Minerva
Head of Runboard staff

Registered: 11-2005
Posts: 9606
Karma: 132 (+147/-15)
ReplyQuote
Re: Bingbot crawling errors, causing 404s


I removed the 403 code and put that video in place of the message I had there before.

Here's how it looks now. Warning: The video will auto-play, and it has the same 2mb fiery background image. emoticon

Last revised by Lesigner Girl, 6/18/2013, 5:45 am


---
Runboard Knowledge Base
Runboard Support Forums
Find other message boards
6/18/2013, 5:45 am Link to this post PM Lesigner Girl Read Blog
 
SKOKEY Profile
Live feed
Blog
Friends
Miscellaneous info

Citizen

Registered: 12-2005
Province: Bennington, NH
Posts: 1308
Karma: 38 (+39/-1)
ReplyQuote
Re:


Catchy tune!
6/18/2013, 11:07 am Link to this post PM SKOKEY
 


Add to this discussion

Jump to Page:  1  2  3 ... 7  8  9 



You are not logged in (login)
Back To Top

This board's time is GMT.