Heal Your Church WebSite

Blog

  • Regluar Expressions Editors and Testing Apps.

    McCulloch 1432 Gas Powered Chainsaw
    Regular expressions are like a chain saw … a gas powered chain saw. At first, you’re deathly afraid of even touching it. When you do fire one up for the first time, it’s only after you’ve dressed yourself like a hockey goalie and have your spouse waiting just outside of your roped-off area with his or her their finger on the 911 speed dial button.

    Once you realize that if you practice some common sense, that is you take time to think through what you’re trying to do first, regular expressions become your favorite power tool – just like your chain saw.

    For example, now that I’m pondering a move to WordPress, I’ve got to port Scripturizer from Perl to PHP. Scott Yang has already done much of the work, but the regex needs to be expanded to include multiple references, e.g. Romans 10:9-10, 12:1-2 or Matthew 5:6-7, 9-12.

    Debugging something like this can become incredibly frustrating using the code and test method. Not so much since I have both Perl and PHP installed on my local PC. Still, having tools to help construct, visualize and debug your regular expressions can help reveal hidden bugs, catch typos that snuck past your bleary gaze, and/or teach you how to use your chain saw – I mean regex – without cutting your foot clean off.

    So here are some tools, tutorials and testers I found:

    Desktop Tools:

    • Kiki – a free environment for regular expression testing (ferret). It allows you to write regexes and test them against a sample text, providing extensive output about the results.
    • For those of you who love Linux: the KDE Regular Expression Editor
    • Expresso – A tool chest for building and testing regular expressions for Microsoft Windows .NET.
    • Like PHP but hate regular expressions? Try the RegExpEditor module with PHPEdit.
    • RegEx Coach – don’t let the plain-jane website fool you out of using this Window-based tool.

    Online Tutorials:

    Online Testing Tools:

    Leave a comment if you have a favorite not on the lists. Or if you have tried one of the above cool regex tools and tutorials, let us know what you think.

  • Using MySQL with the MT-Blacklist to Auto Ban IP addresses

    One of the downsides of increased blog popularity is an increase in comment spam attempts. Fortunately, using a variety of tools already at hand, I’m able to identify and block these persistent buggers automagically while I sleep.

    The Problem

    Last week, this blog enjoyed some linkage from the technical ‘A-list’ regarding my analysis SixApart’s since modified licensing schedule for MovableType 3.0. With this recognition came a flood of comment spam attacks. Fortunately, most entries were thwarted using Jay Allen’s MT-Blacklist plug-in; however a few employing intentional typos got through.

    I had pondered some sort of trap for the spammers … in fact I still am. Until then, I need a means of automatically denying them access regardless of typos without denying read access to legitimate visitors. So I began to pour through my various log files for a solution.

    MT-Blacklist Logs to the Rescue

    One of the options the MT-Blacklist offers is to log failed entry attempts. I have this turned on so I can quickly peruse my MovableType activity log. In fact, it was this very feature that brought to my attention the stepped-up attacks on my system. It also brought to my attention that the few successful entries shared IP addresses with many of the failed entries.

    You see, along with being greedy and lazy, spammers, especially comment spammers, are persistent. I know because it took a set of 5 ips four days to finally sneak some comments onto my system … after 87 failed attempts. It was at this point I realized what I could do the temporarily thwart the spamscum:

    Crontab as SQL statement that inserts records into the MovableType mt_ipbanlist table from entries in the mt_log where a denied IP address shows up more than once. I then encapsulated the SQL statement in a Perl program so I could add bells and whistles at a later date.



    Yes, I realize the LEFT JOIN clause is less efficient than a NOT IN subquery, but unfortunately, I couldn’t get this more efficient syntax to work with the crufty version of MySQL on my server.

    The Aforementioned Bells and Whistles

    Next step is to expand this program to find the most egregious sinners determine which ones are not based in North America, and deny them using my .htaccess file. Here’s the MySQL statement I’d use to find these bums:

    SELECT DISTINCT count( log_ip ), log_ip
    FROM mt_log
    WHERE mt_log.log_message
    LIKE “MT-Blacklist comment denial%”
    GROUP BY log_ip
    HAVING count( log_ip ) >4
    ORDER BY 1 DESC

    Perhaps Jay Allen could be so kind as to add an option to the MT-Blacklist to automatically add IP addresses to the banned IP table? Or along the same lines, perhaps there could even create some sort of banned-IP list so I could share it with those using other blogging applications such as WordPress and pMachine?

    That said, if you can juice it up the above code, leave a comment. I’d be interested in how you deal with it.

  • Countering comment spam with mod_rewrite

    Even though you may have installed Jay Allen’s MT-Blacklist to thwart comment spam, wouldn’t it be nice to deny spambots from consuming your system’s bandwidth and CPU by redirecting them to an ‘error page’ especially designed for our vermonous visitors?

    Apache Wunderwerkzeug : mod_rewrite

    A quick tip-toe through our archives and you’ll see that in the past we’ve given some detailed attention to the use of mod_rewrite for a variety of tasks, including:

    Comment spam … sucks

    Like most legit webmasters, I’ve also given the topic of spam quite a bit of coverage on this blog, such as:

    Unix-Girl to the rescue

    So just in time for a weekend project , I found a brilliant bit of mod_rewrite by Kasia Trapszo, a.k.a. Unix-Girl, that concatenates these two topics into an effective way to thwart comment spam.

    On her page, Kasia details a very simple snippet that demands anyone filling out a comment form first be referred by an article on the same site. In other words, bots just can’t come in out of the blue and fill out a form without at least first pretending to have read the article.

    While I believe she, is running Jay Allen’s MT-Blacklist, this form of .htaccess hackery is a good idea as it gives her publishing system and bandwidth a break by stopping this type of scum at the door.

    With a little experimentation, this little code gem could also be used to protect any form on your site. Then again, this is mod_rewrite, so be careful not to shoot your foot off. Oh, and let me know if you come up with something unique based on Kasia’s kool example.

  • Ikea College Park MD – What we can learn from Ikea blowing a $350 sale

    With any automated system, mistakes are bound to happen. Sometimes it is the end user’s fault. Many times it is a bug in the program, a server glitch or someone spilling a Pepsi on the keyboard. The point is simple; you need to anticipate that at some time, someone is going to make a mistake using your church’s website. The question is how does your response reflect your church’s purpose and personality? Being a little proactive may be the difference between a return visitor and someone never gracing the door of your institution ever again.

    Case in point, last night Ikea had $350 of my dollars in their hands, but because of a faulty information infrastructure, a lack of contingencies and a queue system that enslaves the customer to the needs of the merchant, they let it slip through their fingers. Here is what happened:

    – – – – – § – – – – –

    Yesterday, my wife and I make the 40-minute trek from Rockville to the Ikea in College Park Maryland. No small feat as we are bringing along a sweet, but typical 4 year old. We took the pickup truck, which isn’t as comfortable as the Honda Accord, but then again, you can’t pack a computer desk and side unit in the trunk of the latter.

    So we get there, I take the kid to the play area; my wife begins to hunt for the furniture she saw online. I wait 15 minutes with my anxious daughter because the play area staff is short handed.

    Finally, my daughter gets the green light and I try to hail my wife on our two-way radios. One problem, my wife put it down for a second to measure a desk. She turns around, the radio is gone, and some well-intended staff saw it and took it to the lost and found. I admire that to a degree; I would admire it a bit more had the employee first asked anyone within sight of the radio if they had misplaced such an item.

    I lose 15 minutes trying to find my wife, who has to track down the lost and found to recover the radio to find me.

    We get to the office furniture area, we look at alternate models, we decide on the ‘Galant Desk Combination Left‘, but it isn’t a “self shop” item. We get the attention of a very helpful staff person named Condi. She walks us over to a terminal, prints out a paper and instructs to get our self-shop items and then present the paper while we’re ready to purchase.

    It takes 5 minutes to walk the maze from the office area to the self-shop area. There we get a CPU/printer cart to go with my wife’s desk, I get a “Jerker” table.

    The wait in line at the register wasn’t nearly as long as it was on a previous visit, so I didn’t mind my wife had to walk down to the pickup line while I brought the truck around. I park the truck and wait. Time is running out, my daughter’s allotted time in the play area. I leave the truck in the loading area; my wife can see it … she’s still waiting.

    I get in line to wait for my daughter. I get my daughter. I return my daughter to her mother who turns to me and points to a cart that has everything for the ‘Galant‘ desk unit … except the desktop. Looking up from the cart full of table legs, a ‘half round’ and frame, I ask the clerk where the desktop is. He non-chalantly informs us that they must be out stock because they couldn’t find it.

    Now let’s stop here for a minute. First, when Condi pulled the item up on screen, she should have had a bright red flag on her machine tell her the unit was out of stock … yellow if there was only one or two left; at which point she could have called down to the warehouse to confirm the piece in stock, and hold it while removing it from the system. The system should have refused to print the order unless the out of stock unit was removed from the list of the purchase. Second, up on printing page, a message should have been sent to the warehouse to hold that unit … in fact, the entire request should have arrived at the shipping area before we did.

    With that in mind, I inform them that I’m not going to by any of the parts of the desk unless there is a desktop. What is the sense in that? Especially since we were never informed when the desktop would be back in stock … if ever.

    Let’s stop again, at moment the clerk realized someone had just paid for something that was out of stock, he should have informed the warehouse manager, and the manager should approach the customers; perhaps with free shipping when the unit arrives. If for no other reason because Maryland consumer laws are pretty strict about taking a customer’s money for unavailable item without prior agreement of the consumer. Instead, he points to the refund area, in which we must pick a number … 580, while the current number is 574.

    We wait another 15 minutes, we get a pretty jovial guy who doesn’t mind us asking for a manager … we wait a couple more minutes … he comes back and informs us a warehouse manager will be right up. My wife asks the refund clerk if there is any way we can make sure a mistake hasn’t been made in the warehouse. The refund clerk shakes his head no.

    Myself, having worked on a large POS/Inventory/Warehouse data systems for the largest Mac vendor to the Federal Government a few years back, I remark how impossible it seems to me that Condi was allowed to print-out a request for an item not in stock. How it is absurd that there aren’t procedures and mechanisms in place to deal with low quantity non-self-serve items. Especially since everyone is bar-coding everything in a day and age of high-speed data replication.

    The clerk tries to tell me that it could other persons requested the same unit at the same time. Which was odd, we didn’t see a whole lot of people near looking at the desk, nor near Condi’s station nor loading such units in their trucks and vans. Nor does it explain how the person at the register is allowed to ring-up a sale to the out of stock after scanning the barcode. Though it does indicate to me that this young man’s sales talents are being squandered at the returns desk.

    Exasperated, I kvetch to the cheerful young man at the refund counter that I can’t believe the system is so decrepit. He assures me that it is, and then proceeds to say, “I should know, I work with computers.” Hmmm … that must explain why he’s wearing the yellow shirt with the blue Ikea logo on it.

    Don’t worry the refund guy was nice enough that I kept that thought to myself. Moreover, I inform him that he’s taken too many items off the purchase, that is, there was a $99 “Jerker” sitting my truck that I did want to take home … and no matter how frustrated, it would be stealing if he didn’t debit my credit.

    We pay for the “Jerker” and leave the store; still no warehouse manager in site, my wife is frustrated to the point of tears.

    – – – – – § – – – – –

    It’s a long story, but I feel much better after writing it. I also hope you’re a bit wiser in understanding the importance of pre-empting mistakes, and when that doesn’t work, giving your users what they need to gracefully recover from mistakes.

    For example, what happens when your user types in a URL that doesn’t exist on their page? I personally prefer to give them the name of the attempted page, along with an explanation, a menu/method of navigation and the search form.

    So what happens when the user can’t find something on the search form? Have you taken into account they might not know how to enter an effective query? Along with instructions, do you have an “advanced form” which dumbs down some of the options in the forms of additional radio button and/or check box elements?

    Have you instructed members of the church office how to respond to errors and inquiries? Do you provide any type of documentation to help the users and/or the church staff circumnavigates the site?

    Here’s the point, if you don’t address the potential for “oopsies” on your church’s website, then you are probably insuring that a potential visitor is going to visit some other church after they frustrated with your website and leave forever.

    Ikea had $350 in their hand, yet they let it slip through their fingers. Worse, we’re not likely to go back to buy anything else after having to wait in five different lines of 15 minutes each, three of which was to pay for a cash and carry item that was not in stock … or couldn’t be found.

    Don’t let the same type of thing happen to your church’s website.

    • put systems in place that keep everyone on the same page
    • have contigencies to deal with mistakes and/or complaints
    • put the needs of the seeker and/or member above that of the system

  • Mirroring Websites with wget, curl and/or tar

    From time to time, it is a good thing to back up your entire site onto a different computer/server, even if your entire site is database-driven.

    Take my situation two weeks ago, when my former host provider shut down the RBC site because of a false spam report. Even though no spam had actually been sent from redlandbaptist.org, my former host’s upstream provider demanded action. In response, my former shut down the site with no notification despite their existing terms of use to warn on the first time, suspend on the second, etc… After I threatened legal action, the host provider and I found a middle ground where he agreed to put the site online, and I agreed to move my sites elsewhere over the course of a weekend.

    Fortunately, I was already moving sites off this host I personally found troublesome. The Redland site was the last to go. RBC was last because there are legacy portions of it that are not entirely data-driven. That is, with my other sites, I merely dumped the MySQL database off the old machine, install a fresh version of MovableType on the new site, pipe in the data, hit rebuild and viola!

    While there are several PC based options for mirroring a site, I wanted something that would take files from one Linux server and move them to another. I had three choices. First, there is wget, a nice little GNU tool for offline reading and site mirroring. As Jim Roberts writes in his article “Mirroring Websites with wget“, the syntax is insanely simple:

    wget -mirror -w 3 -p -P c:\sokkit\site\rbc ftp://username:password@ftp.redlandbaptist.org

    My only problem with this approach is that much of the legacy stuff at RBC was image related and/or offline because it is seasonal. And though I employed the “-p” option, not all images made the cut, nor did any of our offline archives for obvious reasons. So another solution, at least where images are concerned, would be to use a Perl program that employs another command line download tool simply known as curl:

    perl curlmirror.pl -p -s 800 -o rbc -t /home/backupsite/rbc http://www.redlandbaptist.org

    Unfortunately, since I have several files with a .php extension, I get the same filename mangling that I would with wget’s offline reader syntax:

    wget –mirror -w 2 -p –html-extension –convert-links -P c:\sokkit\site\rbc http://www.redlandbaptist.org

    The above syntax converts urls like http://www.redlandbaptist.org/index.php?sid=123 into http://www.redlandbaptist.org/index.phpsid_123.html. Great for viewing, not so great as a working backup.

    There was another way, one that insured I got all my files, all the correct paths and all the correct file ownerships and permissions. Unfortunately, this method required shell access, and though my former host provider was kind enough to put the site back online, I seriously doubt he would have honored any request to restore ssh access. So I cheated, I downloaded a ‘modified copy’ of the Gamma Web Shell.

    ‘Modified copy?’ The Gamma Web Shell allows an individual to execute shell commands directly from their browser, so you can imagine the security implications of installing such a program. So on my local PC, I first modified the password to something huge and random, I limited the commands allowed, and then I changed the code to accommodate changing the file name from WebShell.cgi to something hard to guess. Once modified, I ftp’d it to the old Redland site, and then entered the following commands:

    mysqldump -uUSRNAME -pPASSWRD -opt mydatabase > databasedump.sql

    tar -zcvf shebang.tar.gz *

    Please be warned, if you go this route, you are putting your site at great risk. Don’t blame me if you get hacked. You have been warned. In fact, I only did it because my back was against the wall. That said, immediately after executing the above backup commands, I deleted the shell program. Then uploaded a text file full of Lorem Ipsum using the same name to make sure it couldn’t be reconstituted from the trash.

    After these security precations, I FTP’s the backup file to the new host, invoked the command “tar -zxvf” and was back in business almost instantly. I also FTP’d the tar.gz backup file to my home PC and ‘burninated‘ a CD as a ‘suspenders and belt‘ precaution.

    So how about you? What’s your method of mirroring and/or moving sites? Remember what I said this past August, “if you fail to plan, then you’re planning to fail.“

  • Tweaking Instapundit’s New Design

    When it comes to issues of the law and punditry, and even to a degree digital photography, �ber-blogger Glenn Reynolds has it all over me. However, when it comes to issues of website usability and accessibility, I think I might have the advantage over the otherwise all knowing Instapundit.Recently, the good Dr.�s site underwent a very nice facelift: bolder fonts, bolder colors, less distracting presentation of his blogads and permalinks, and a useful �Don�t Make Me Think� style navigation/menu bar along the top. In other words, I like it, I like it a lot. That said there is some tweaking I might affect before blessing this redesign as completely done.

    Perhaps the biggest problem I have with the site is the �active pattern� employed as borders. I found my eyes drawn to what I perceived as �motion� along the fringe, instead of keeping my eyes centered on compelling content that has always been king at Dr. Reynolds� site. It makes reading the site for any length of time difficult � almost to the point of a headache.

    Next I�d reword �Stylesheet Switcher� to �Change Font Size.� The former may give the user the impression that they�ll get the old stylesheet, even with the iconic help directly below the words. I�d then take this entire block of code and switch its ‘above-the-fold‘ position with the somewhat buried Search box input form. Most browsers allow the user to change the font size on their own � while server-side searches on a site as chock-full of great content as Instapundit are likely to be in greater demand, or to quote the godfather of usability:

    “Search is the user’s lifeline for mastering complex websites. The best designs offer a simple search box on the home page and play down advanced search and scoping.” – Jakob Nielsen

    I would obfuscate the email address.

    With these quick fixes out of the way, I would strive to use a (generated) tableless layout. Not to prove I�m a hip-cool programmer guy, but primarily because it would allow me to switch to a variety of formats merely by switching the CSS. As it stands now, there is a separate PDA and Print page; the latter of which could be healed with a little bit of �Miracle Print.� I�d make sure the new layout had a footer that offered some simple text hyperlinks all of the useful navigation elements from the top.

    I think I�d then do something about reworking the archives page. There is quite a bit of information that I would think of interest to any one into history or political science. Especially anyone in the process of writing a book needing a roadmap on who said what regarding recent political events.

    If I did move the search form/block to the top, I would then rework the search results page to include revenue-bearing elements such as Google’s Adsense. After all, the user is explicitly conveying their immediate interest in the form of keywords and phrases, the exact information revenue-bearing programs use to determine who sees what ad. As long as you’re going to have ads on the page, go all the way.

    Again, please note, we�re talking tweaks here. Obviously this page has been rendered by a professional who knows what she’s doing. Given the great cloud of witlessness that plagues many weblogs and church web sites, I’d say there are plenty of positive elements on the newly designed Instapundit website that are worth emulating. That said, because the site is the gateway drug of blogging, I would like to see it as addictive as possible.

    What do you think? Leave a love note, lemme know.

  • Generic ADO Connections for ASP.Net

    So you think you want to go the Microsoft Web Server route after reading a NetCraft post entitled “ASP.NET Overtakes JSP and Java Servlets” ??? Okay, I can understand that. I mean even with CPanel and WHM, managing an Apache web server can be a bit daunting. And while MySQL is free, there are some nifty tricks SQLServer has to offer, if you’re willing to pay for it.

    While personally, I feel far more secure behind a Linux-based webserver, I’m required by my day job (you know, the one that pays the bills) to do the .NET/Windows Server dance.

    One of the issues you might face in making the transition from one operating system and/or webserver to another is accessing your data. This is why APIs such as the PHP-based ADOdb and Perl DBI are popular on Linux/Apache platforms.

    Similarly, this is why most competent Microsoft programmers access databases with interfaces such as the ADO, and more recently, OLEdb, unless of course you’ve been asked to use the MSDE 2000 while your crack technical staff obtains a license to and installs THE Microsoft SQL Server.

    Wide Variety of Standards
    Yeah, I know, the above paragraph reminds me of the only joke ever told by professor Frank da Cruz in my netorking class at Columbia University, a joke who’s punchline was “a wide variety of standards.” Took me about 3 seconds to get it, but I was sure to laugh out loud when I did (yes, I know, I’m a shameless suck-up that way, but I digress.)

    In case you missed the reality that made the punch line funny, there are just too many daggoned different ‘connectivity standards’ out there for databases. Probably due to the fact that there are just too many daggoned RDBMS tools out there, many of which are rotting legacy systems that you and I are now compelled to maintain and integrate. I mean, forget deciding whether or not to use VB.Net over C#, how do I seamlessly patch together an enterprise where the the CEO’s contact database is in MS-Access, an HRD system in Oracle, and the sales and tracking system is running on an MS SQL and still have code that is reusable?

    One solution I read about this weekend comes in the form of a nifty GenericADO connection class in VB.Net, based upon a similar generic data access component in C# by the same author.

    Need connection strings?
    This generic approach works fine, once you figure out the connection string … which itself can be a black art when dealing with anything other than a Microsoft database technology. Well, that is once you download, install and then reference the ODBC .NET Data Provider (why this isn’t included in distributions of Visual Studio .NET is beyond me).

    Still, ODBC wasn’t the real show-stopper when I first took-on .Net two years ago … rather it was Sybase. The biggest problem was figuring out the whole ASE thingie, an effort hampered by the fact that Sybase released a buggy version of their Studio installer.

    Once over this hump, and once you’ve spent another half day discovering you need to first create an OLE DB connection object via the Sybase ASE OLE DB Administrator, then all you need to do is hack away until you find a connection string that works.

    At least that’s what I had to do because at that time, there wasn’t an instructive website chock full of examples for every known .NET Data Provider Connection String available to man and geek.

    Lobotomy Time
    If you haven’t figured it out, this post serves some selfish reasons, such as links to websites and tips on how to dive through the flaming hoops of database connectivity in a Microsoft-based enterprise without ‘burninating’ myself into a crisp. Hopefully, some of you guys and gals will find these links useful.

    And for you religious purists who not only know what a “Factory Pattern” is, but are so entirely snooty as to pooh-pooh such utility classes, here is a link to some rather hairy template connection classes that are sure to amaze and confuse your co-workers.

    As always, your mileage may vary.

  • Click here to enter the cool church website!

    Since last week, I’ve enjoyed several comments and even more private emails informing of various church websites in need of healing. A task made increasingly difficult now that my review of “The Coolest Church WebSite, Ever!” comes up numero uno for the Google query ‘cool church website.’ Fortunately, well actually unfortunately, there are enough self-proclaimed “cool church websites” out there to keep us all busy with reviews until Jesus returns.

    Case in point, the website for high-school age students at the Central church of Christ in Moore, Oklahoma.

    If your church doesn't have a spinning cross on its steeple, then why put one on your site?If you haven’t clicked in the hyperlink for today’s example, imagine a kelly green screen with a somewhat grainy, poorly cropped and centered picture of what looks like the side doors and facade for the church, and the parking lot … enough so to catch a glimpse of the sign. Flanking the image’s sides, spinning animated GIF crosses. Above this artwork, in lime green, the phrase that pays: “click here to enter the cool church website.”

    Now a caveat here, today’s example webpage was designed back in 2001 two 16 year old kids. I’m sure now they look at it and roll their eyes, slapping their right hand to their forehead and whispering under their breath “what was I thinking?” It is also for this reason, I’m not going to say anything else about about the design elements … especially in light of the youth websites I first created … mercifully, all evidence of the latter has long since been rendered to the bit-bucket.

    Instead, I want to use this page to talk about some misconceptions we have about youth ministry and what we do online. As someone who for the past 10 years has been involved with youth ministry and teaches a high-school aged Sunday school class, I know one or two things for certain.

    1. If you have to tell people it’s cool, then it probably isn’t. This is why I tell incoming classes that the only time I use their vernacular is when I’m making a parody of them. Let’s face it, nothing is more pathetic than a 40-some year old poop misusing the phrase “word-up.” This means when you write content, write it as you would speak it to them or anyone else. They’ll appreciate it much more than you’re lame, ‘Wonder Bread™’ rendition of hip-hop.
    2. This leads me to my second point: don’t confuse the Internet for television, MTV, movies or other forms of youth-targeted media. Unless you’re as awesomely talented and playfully demented as Mike and Matt Chapman, kids aren’t going to visit your youth website for entertainment. If they want movies, they’re going to go to the movies, if they want music videos, they’re going sneak a peek at MTV (unless you’re like me and are cable-free).

    This isn’t to say your youth website should look and feel like the rest of your church’s website. But if your objective is to bring them in by being cool, then all you’re going to really do is look like some old fool. Instead:

    • make the site informative;
    • make it a place where they can contribute content and feedback;
    • make it relevant with continual posts about things they’re facing, things they like, or upcoming events.

    The bottom line is incredibly simple, if your youth website is informative, moreover it if ministers to the needs of your youth, they’ll think it cool, and they’ll come back for more.