Month: April 2011

  • backUpMySQL.pl – is it cool?

    Originally published on April 29, 2003, I’ve made some formatting corrections & bumped this up a bit.

    Yesterday, Mark Pilgrim’s message of the day was
    You know what’s cool? Backups.

    Well who am I to argue with such coolness? So in the spirit of “what is Hip“, and myself being in a situation where my host is also changing data centers, I would like to share with you a little utility script I run on my system every night ubiquitously entitled “backupMySQL.pl.”

    Basically this little Perl takes advantage of naming conventions used by the standard-fare Apache configuration many of us enjoy. That is, our accounts are usually stored in directories such as “/home/USERNAME” and our database are prefixed with our USERNAME, such as USERNAME_mt. Moreover, a properly configured system will allow you to securely house and run such scripts BELOW the public /public_html &/or /www directory where all your public stuff is published.

    With this configuration in mind, I FTP this script in ASCII mode to my root directory (below HTTP access), a chmod -755 so it would execute. I then created a subdictory entitled /dbs and chmod -755 /dbs so my script can access it. I then went to my control panel and cron’d the job to run every night. Okay, so I lied, I did this all from the command line, but as you can see, you can implement this script without having to bash yourself silly.

    One other optional feature I have in this script is the ability to FTP my backup to a friend who hosts a website on an entirely different server and service. I reciprocate in kind for him. What this does is insures that we have an “off site” backup — based on the principle that if both our servers go down, then we’ve got a much larger issue at hand (how about global thermonuclear war?). So here it is. Use it, tweak it, let me know ho you like it — just make sure to check the files from time to time to make sure your backups can be restored.

    #!/usr/bin/perl -w
    # ———————————————————————–
    # copyright Dean Peters © 2003 – all rights reserved
    # http://www.HealYourChurchWebSite.org
    # ———————————————————————–
    #
    # * Obligatory Legal Stuff *
    #
    # backupmysql.pl is free software. You can redistribute and modify it
    # freely without any consent of the developer, Dean Peters, if and
    # only if the following conditions are met:
    #
    # (a) The copyright info and links in the headers remains intact.
    # (b) The purpose of distribution or modification is non-commercial.
    #
    # Commercial distribution of this product without a written
    # permission from Dean Peters is strictly prohibited.
    # This script is provided on an as-is basis, without any warranty.
    # The author does not take any responsibility for any damage or
    # loss of data that may occur from use of this script.
    #
    # You may refer to our general terms & conditions for clarification:
    # http://www.healyourchurchwebsite.com/archives/000002.shtml
    # For more info. about this code, please refer to the following article:
    # http://www.healyourchurchwebsite.com/archives/000802.shtml
    #
    # * Technical Notes and ASSUMPTIONS (PLEASE READ) *
    #
    # this code assumes a standard Apache configuration where
    # the $HOME directory is a path beneath public_html &/or www
    # and employs a naming scheme such as /home/YOURACCOUNTNAME/…
    #
    # it also assumes that your databases are prefixed with your
    # account name, such as YOURACCOUNTNAME_mt
    #
    # do NOT under any circumstances run this from a directory accessible
    # via HTTP (e.g. public_html/… or www/…)
    # it makes system calls, and although it takes no input, just don’t!
    #
    # this program works best with CRON, e.g.
    # 0 0 * * * /home/YOURACCOUNTNAME/backupmysql.pl
    #
    use DBI;
    use Net::FTP;
    
    # this assumes you have previous created a subdirectory named /dbs
    # and have chmod 777 /dbs
    $path = “/home/$username/dbs/”;
    $file = “/home/$username/dbs.tar.gz”;
    
    # databse connection info …
    $host = “localhost”;
    $username = “YOURACCOUNTNAME”;
    $password = “YOURPASSWORD”;
    
    # connect to the database and retrieve a tuple of your databases
    $dbh = DBI->connect(“DBI:mysql:host=$host”,$username,$password) or die “Bad login info”;
    $sth = $dbh->prepare(“show databases like \’$username\_%\'”);
    $sth->execute();
    
    # for each database … back it up!
    while(@row = $sth->fetchrow_array()) {
    	if(!$row[0]) { die “No dbs to backup!”; }
    	foreach $db (@row) {
    		system(“mysqldump –opt –user=$username –password=$password $db > /home/$username/dbs/$db\.sql”);
    	}
    }
    
    # you’re done with the database
    $sth->finish();
    $dbh->disconnect();
    
    # delete the old version — probably should “grandfather it”
    if(-e $file) { unlink $file; }
    
    # create a single, easy to use and transport file
    system(“tar -cf dbs.tar dbs”);
    system(“gzip dbs.tar”);
    system(“rm $path\$username_*”)	if $path =~ m/home\/$username/;
    
    #
    # OPTIONAL – you can comment this out, or not
    # this assumes you have a friend on a different server
    # with whom you’ve made arrangements to hold backups of each
    # other’s data. This way, if a server fails, you can get it
    # from your friend’s site
    #
    # it also assumes your friend has created a directory for
    # you entitled “/backup” — this of course can be changed
    # if your friend sets up an individual FTP account to a directory
    # … which is what I actually do … I love my friends!
    #
    $ftp = Net::FTP->new(“MYFRIENDSDOMAIN.COM”, Debug => 0);
    if($ftp->login(“FTPUSERNAME”,’FTPPASSWORD’)) {
      $ftp->binary();
      $ftp->cwd(“/backup”);
      $ftp->put(“dbs.tar.gz”, $username.”_dbs.tar.gz”);
      $ftp->quit;
    }
    
    # bye bye
    exit;
    
  • Fun with the Twitter Search API and jQuery

    During my job search last year, I admitted that “yeah, I’m a bit of an API junkie.” Anyone whose followed this site since 2002 probably has gone blind once or twice reading posts about SOAP, XML-RPC, RSS feed and other such programmer protocols and interfaces.

    So why should anyone be surprised that today I’m providing a quick how-to code snippet of some fun I’m having with the Twitter Search API, REST, jQuery and jSON?

    YES, I know I need to get back into providing posts about content management,  effective social media strategies and web campaigns … but for today … please indulge me with one more trip into the land of code.

    Some Context

    In the process of writing some WordPress plugins leveraging the Facebook API, I thought “why not twitter?

    However, there are already a multitude of plugins and widgets out there that’ll show my profile.  So I turned my eyes to Twitter Search.

    My first thought was to simply write this all up using not much else but the jQuery.getJSON() method.  However, this approach doesn’t lend itself well to caching – which in turn would lead to some of you with busy sites getting your widgets blacklisted by Twitter as <a href=”http://apiwiki.twitter.com/Rate-limiting” title=”Twitter API Wiki – Rate Limiting”>their default rate limit</a> for calls to the REST API is 150 requests per hour.

    So now I’m working on a PHP solution inspired in large part by Arron Jorbin’s post “More Twitter Shortcodes for WordPress.” Must read for anyone working with feeds or APIs in the WordPress arena.

    Hey, so where’s the jQuery & jSON?

    All that context aside, I did successfully write a short snippet that used jQuery to call the RESTFul Twitter Search API and then parses the jSON into a dynamic display.

    I did this in part because while I will employ some form of PHP or  Perl to cache the Twitter Search, I still might employ jQuery as the rendering mechanism for said cache. Here’s my test code so far:

    /* a counter outside the context of setCountdown() */
    var seconds2go = 0;
    
    /*
     * the method that sets the visual display of the countdown timer,
     * and triggers getTweet after 2 minutes
     */
    var setCountdown = function() {
      seconds2go--;
      if(seconds2go > 0) {
        $("#countdown").html("Seconds until the next refresh:' +
      ' <span>" + seconds2go + "</span>");
      } else {
        $("#countdown").html("Seconds until the next refresh:' +
      ' <span>0</span>");
        getTweet();
        seconds2go = 120;
      }
    }
    
    /*
     * the method goes out to the Titter A.P.I,
     * then parses the jSON block into the display
     */
    var getTweet = function() {
    
      /* set everything up */
      var url="http://search.twitter.com/search.json" +
           "?rpp=5&callback=?&q=";
      var query = escape( query=$("#twittersearch").val() );
      var display = '<div class="tweetDisplayContainer error">' +
           'no records found</div>';
      var urirex = /(https?):\/\/+([\w\d:#@%\/;$()~_?\+-=\\\.&]*)/g;
      var hashrex = /\#+([\w\d:#@%/;$()~_?\+-=\\\.&]*)/g;
      var thashuri = "http://search.twitter.com/search?q=%23";
      $("#twitterresults").html('');
    
      /*
       * A.J.A.X. happens here -> go get the data, then parse it
       */
      $.getJSON(url+query,function(json){
      $("#twitterresults").html('<h4><a class="searchlink" href="' +
      url.replace('search\.json','search')+query +
      '" title="see the search query via Twitter">Testing: ' +
      url+query + '</a></h4>');
      if(json) {
        display = '<div class="tweetsContainer">' +
      '<dl class="tweets clearfix">';
        $.each(json.results,function(i,tweet){
          ttext = tweet.text.replace(urirex,
      '<a href="$1://$2" title="">$2</a>');
          ttext = ttext.replace(hashrex,
      '<a href="' + thashuri  + '$1" title="">#$1</a>');
          display +=  '<dt class="tweet' + i + '">' +
                '<img src="' + tweet.profile_image_url + '"  />' +
              '</dt>' +
              '<dd class="tweet' + i + '">' +
                ttext + ' <strong>via:</strong>' +
                '<a href="http://twitter.com/' + tweet.from_user +
                '" title="tweets by ' + tweet.from_user +
                '">@' + tweet.from_user + '</a>'
              '</dd>';
            });
        display += '</dl></div>';
      }
        $("#twitterresults").append(display);
    
      });
    }
    
    /*
     * this is where we kick-it all off,
     * assumes seconds2go = 0 initially
     */
    setInterval(setCountdown, 1000);
    

    As you can see, the most difficult part was getting it all to fit in a readable format on this blog! Well, that and some additional fun with regular expressions.

    Well that and what you don’t see in the code are two html elements:

    <h2 id="countdown">Seconds until the next refresh: <span>120</span></h2>
    
    <input type="hidden" id="twittersearch" value="deanpeters #smm" />
    
    <div id="twitterresults">no results yet</div>
    

    Todo: I’m thinking the above script could use a bit of animation easing or some other effect so we don’t simply “flash” new results at the user. It also needs to be objectified and wrapped-up as a plugin. More on that as I work on the widget/plugin.

    Demo Stuff

    I did create a demo page – it’ s not pretty, but it effectively shows how to get it done. I’ll craft up some CSS for it later.

    It’s basically built off a search of  deanpeters  #smm as pictured below:

    twitter search criteria for jQuery test

    I’ve also created a .txt version of the file if you’re interested.

    Additional Reading

    In the meantime, I though I’d list some of the sites I visited while approaching this exercise. Some good people providing some good examples:

    FYI

    Thanks for all the emails and retweets of late. Good stuff!