Heal Your Church WebSite


Teaching, rebuking, correcting & training in righteous web design.

How to block spambots, ban spybots, and tell unwanted robots to go to ….

I want to apologize for the lack of posting, and certainly being late with this great topic Mark Pilgrim had addressed this past Wednesday. I’ve been busy rebuilding a more usable Redland Baptist web site and it has consumed much of my free time. Don’t worry, I’m taking notes, and I intend to share them with the rest of you.

About a month ago, Vincent Flanders had emailed me. He was getting harvested hard by various spambots. I was reminded of that dialog when I read Mark Pilgrim’s 26-Feb-03 post entitled How to block spambots, ban spybots, and tell unwanted robots to go to hell. Here is what Mark said on his site — which sounded very similar to the original email I had received from Vincent:

I recently woke up to the staggering level of abuse occurring on this web site. This is old news to some, but we all wake up at different times. I’m talking, of course, about automated robots and spiders. They come at all hours, they take as much as they can, and they leave me with the (bandwidth) bill. They do so without respecting the Robot Exclusion Standard, now almost 10 years old. Some come to gather email addresses, which are then sold to spammers; some come to steal images or other content, and republish it without my consent; some come to spy on me and sell information to their clients about perceived violations of copyright, trademark, or some nebulous concept of brand identity.

None of them act in my best interest, or in the interests of my readers.

Can I hear an Amen? One of the biggest problems I’ve had with running a church web site, is pesky programs that rape and pillage the site for email addresses, sermons and images. Now mind you, I have no problem with a stranger who wants to copy one of Mark Adam’s sermons emailing me and requesting a copy for an upcoming Bible study. That’s why the information is out there free of charge. What I don’t like is ‘bots that come by and broadcast the latest pr0n to my pastors, or repackage and sell the sermon material in spite of our admonishion not to do so on in our Terms of Use. They’re crooks, so what’s breaking one more law? This is why I employ some anti-spam countermeasures — and have discussed them on THIS site.

And this is why I’m adding Mr. Pilgrim’s to the mandatory reading list.

Do not merely listen to the word, and so deceive yourselves. Do what it says. – James 1:22

2 Comments

  1. Psalm 9-5
    You have rebuked the “nations” (aka spamers) and destroyed the wicked;
    you have blotted out their name for ever and ever.

    on note…
    Blog Publishers Stealing Web Limelight
    “Blogs4God offers “…. http://news.excite.com/tech/article/id/58716|technology|03-01-2003::08:07|reuters.html

  2. Actually that’s to be done when the Lord comes again with His judgments upon the wicked. For now we live in the world but not of the world (i.e. like the spammers do, or sink to similar levels). All we need to figure out now is that for every spam done they get a message of the Gospel in their real inbox. I believe that would be worth the effort ;)