WRT54G vs WRE54G

Remember a few weeks ago when I was having all the wireless troubles?  The problem seemed to go away for a bit, and then WHAMMO!  It bit me again recently…again when I was downstairs in the living room.

Jessica heard me crying about it and mentioned that she had been having the same problem for the last few weeks.  What?!  Why hadn’t she mentioned this?  Dunno, but at least now I could eliminate my own laptop as the problem and concentrate on debugging a general wifi problem in the house.

The problem really only happened at DHCP lease acquisition time.  Once either of us had a lease we could roam all over the house, but successfully acquiring a lease in the first place was really only working upstairs.  Hmmm….

Well, apologies to all you potential Encyclopedia Browns out there, but you havn’t been given all the facts — because I had forgotten about one very important one: I had installed a Linksys WRE54G Wireless Range Extender for Jessica when she moved her office to the basement about two years ago.  Honestly, I had forgotten all about it.  I mean, I saw it every now and then, but I never bothered to think about what it was doing.  Or what it wasn’t doing.

One unplugged WRE54G later and Jessica and I are both acquiring DHCP leases all over the house again.  Apparently we were trying to associate with the range extender when our wireless interfaces were reconfiguring, and it must not have been bridging correctly with the router since I installed the new firmware.

I don’t know if we even need the range extender anymore since now that I’m running DD-WRT on my WRT54G I’m transmitting at 84 mW (instead of the default 28).  I’ll play around with connectivity in the basement this weekend, and if I still need the range extender I’ll configure WDS Bridging between it and the router so that they play together nicely this time.

Escape from mod_rewrite!

I’ve spent the last few days investigating Plone as a potential replacement for WordPress for the okcomputer.org web site, but, as is frequently the case when experimenting with new technology around here, I’ve been beating my head against the wall trying to get my Plone installation to work behind an Apache proxy. I could successfully pull up the main page’s content in my browser, but none of the Cascading Style Sheets or Javascript files were making it. The result was less than inspiring. So while I was waiting for the tornadoes to pass through tonight, I fired up my browser, tailed my log files and started beating on it.

First off, the mod_rewrite rule from the RewriteRuleWitch wasn’t matching. Fantastic. Here’s what it gave me:

RewriteRule ^($|/.*) \
http://127.0.0.1:8080/VirtualHostBase/\
http/%{SERVER_NAME}:80/okcomputer/VirtualHostRoot$1 [L,P]

The root document was matching, but nothing else was. After setting my RewriteLogLevel to 9, I saw that the URI as it was being tested against the regex didn’t begin with a /. So I removed the slash from the regex and added one to the substitution pattern:

RewriteRule ^($|.*) \
http://127.0.0.1:8080/VirtualHostBase/\
http/%{SERVER_NAME}:80/okcomputer/VirtualHostRoot/$1 [L,P]

Now the RewriteRule was matching, but the proxied requests were generating 400 errors on the Zope HTTP server. The obvious culprits were the spaces in some of the URL’s, specifically in URL’s containing the string “Plone Default”. My browser was (correctly) escaping the spaces as “%20”, but by the time mod_rewrite had processed the request and passed it on to mod_proxy the spaces were back resulting in a malformed HTTP request.

I found the fix for this last bit at Velo World. The trick is to define a RewriteMap and then use mod_rewrite’s built in escape function, like so:

RewriteMap escape int:escape
...
RewriteEngine On
RewriteRule ^($|.*) \
http://127.0.0.1:8081/VirtualHostBase/\
http/%{SERVER_NAME}:80/okcomputer/VirtualHostRoot/${escape:$1} [L,P]

And success! But what a pain in the ass! I’d love to blame mod_rewrite, but I’m sure there’s some valid reason it doesn’t re-escape substitutions by default. Instead I’ll blame Plone. Why? Why do you put spaces in your %$#@! URL’s. And given the number of pages on the web explaining how to set up Plone behind an Apache server with mod_rewrite and mod_proxy — including Plone’s own documentation — how come none of them mentioned this? Very frustrated, but the problem is fixed, the tornadoes have passed, it’s going on 2:00 AM, I’m going to bed.

UPDATE 2/14/2008: Special thanks to Sascha, who’s been helping me debug my RewriteRule issue.  The Witch’s rule should work, I just can’t figure out why apache is stripping that slash of the front of my URI’s.  I’m not using a RewriteBase, and the RewriteRule isn’t an a .htaccess file, so I just have no idea what the problem is.

The Great Photo Salvage

After the filesystem crash back in October I had 11 gigabytes worth of files without meaningful names just sitting in a lost+found directory. I was forced to come up with some creative ways to salvage data.

$ find lost+found/ -type f | wc -l
79370
$ du -sh lost+found/
11G     lost+found/

One of the first things I thought people might want access to would be their digital photographs, so I decided to cook up a little program to sort through the entire lost+found directory, find JPEG’s with a “Camera Model” EXIF tag and then copy them into an appropriate directory based on the value of that tag.

My first attempt used the shell, but that proved unacceptably slow so I fell back to my good friend Python. I installed the Python Image Library (python-imaging in Ubuntu), which I used to identify the JPEG’s (very fast, by the way), and which also has (kludgy, experimental) support for reading EXIF tags.

Here’s the code:

#! /usr/bin/python

import sys
import os
import Image
from ExifTags import TAGS
from shutil import copyfile

INDIR = 'lost+found'
OUTDIR = 'recovered-photos'

def recover_photo(infile, outdir):
    try:
        im = Image.open(infile)

        if im.format == 'JPEG':
            tags = {}
            exif = im._getexif()

            for tag, val in exif.items():
                lookup = TAGS.get(tag, tag)
                tags[lookup] = val

            dirname = tags['Model'].strip()
            dirname = dirname.replace(' ', '_')
            outdirname = os.path.join(outdir, dirname)

            if not os.path.isdir(outdirname):
                os.makedirs(outdirname)

            outfile = os.path.basename(infile)

            if not outfile.endswith(('.jpg', '.JPG')):
                outfile = outfile + '.jpg'

            outfile = os.path.join(outdirname, outfile)

            copyfile(infile, outfile)
            print infile, '->', outfile

    except IOError, v:
        try:
            (errno, errmsg) = v
        except:
            errmsg = v
        print infile, errmsg
    except:
        print infile, 'unknown error'

for root, dirs, files in os.walk(INDIR):
    for name in files:
        infile = os.path.join(root, name)
        recover_photo(infile, OUTDIR)

After running for 28m18s:

$ ls -l recovered-photos/
total 492
drwxr-xr-x 2 matthew matthew  4096 2008-01-26 13:58 C2100UZ
drwxr-xr-x 2 matthew matthew  4096 2008-01-26 14:18 C3100Z,C3020Z
drwxr-xr-x 2 matthew matthew 12288 2008-01-26 14:18 C860L,D360L
drwxr-xr-x 2 matthew matthew 36864 2008-01-26 14:17 C960Z,D460Z
...
$ find recovered-photos/ -type f | wc -l
9293
$ du -sh recovered-photos/
7.3G    recovered-photos/

9293 digital photographs all nicely sorted.

I’ll clean the code up, flesh it out with some handy command-line args and post a link to the updated version here as soon as I have a few spare cycles.

Easy, Empty Gravatars

For some unknown reason, I got vaguely interested in Gravatars this week. I’ll spare you the Google search — Gravatars are Globally Recognized Avatars. (And for you total lame-os, avatars are the little pictures next to people’s names in some forums and blog comments.) The idea is that you upload your avatars to gravatar.com and associate them with your email address (which just happens to be a globally unique identifier that you’ve probably already provided to most systems you use), and then services that are configured to use Gravatars can always pull your latest and greatest avatar from a central repository. Rocket science it ain’t.

PG GravatarG GravatarFirst, I set up a couple of Gravatars myself: one rated PG (I’m holding a beer), another rated G of me playing the ukulele at Ian and Christine’s wedding party. I’m not sure why I bothered as I don’t really post to any forums these days, but they’re there.

Next, I spent about three minutes setting up the Easy Gravatars WordPress plugin over on Kid Amnesiac. Man, does Easy Gravatars live up to its name! Not much flexibility, but a default setting that was perfect for my needs. (The only tweaks I made were to change the URL for the default image for missing Gravatars to one being served off my own server and then to set the default maximum-naughtiness rating to PG.)

Here’s the kicker: not a single person who has posted comments to Kid Amnesiac has a Gravatar. I checked ’em all. Nothing. I even posted a comment myself (since deleted) just to see if the plugin worked (it did). I figured at least one person (*cough* Diana *cough*) would have a Gravatar. Nope. None. So that was a bummer.

I’m leaving the plugin enabled in hopes that someone might read this and go nuts and actually set up a Gravatar. Please, make me feel useful. Oh, and I suppose I should set the plugin up for matthewwhitworth.com, too.

Shredding Serenity

All okcomputer.org services and accounts have been migrated to the new okcomputer.org server, firefly, for a couple of weeks now, so today I started work on decommissioning the old server, serenity. Normally I’d wipe the disks like so:

# dd infile=/dev/urandom outfile=/dev/hda1

But that has been complicated by the annoying fact that the hosting company built the OS on a single disk partition. An attempt to destroy the contents of hda1 would risk causing an
operating system crash before all the contents had been destroyed.* If I had physical access to the server I could boot from a live CD and have a free go at the hard drive, but that isn’t an option. I needed a tool that would give me some precision as to what I was destroying.

I’ve opted to use shred, a program included in the coreutils package that writes random data over a file’s bytes. Learning about shred was funny because there is a lot of misinformation out there. Its primary weakness is that it assumes that the filesystem overwrites data in-place, and not all modern filesystems do that — in fact most modern filesystems don’t do that. serenity used the ext3 filesystem, and there’s quite a number of people screaming on the ‘Net that shred is ineffective on ext3 filesystems. Well, I actually R’ed the FM, where I learned that ext3 filesystems are only problematic to shred in data=journal mode (not the default). Seems I was all clear.

I couldn’t shred the entire filesystem or I would find myself in the same conundrum as if I had used the dd method, so I decided to concentrate on user data, databases, certificates and system passwords. After that I could let loose a more dangerous (but less precise) method and call it a night.

shred’s other weakness is that it was designed to be used on single files (or whole filesystems via their device files) rather than recursing through a directory tree like “rm -Rf”. So I wrapped it in a find statement and then deleted the empty user home directory trees by hand before deleting each user’s account. Actually the whole thing can be neatly wrapped up in some awk:

# for u in `awk -F ':' '$3 <= 1000 && $3 > 65534 { print $1 }' /etc/passwd`; do
> find / -type f -user $u -exec shred -n 3 -zu {} \;
> rm -Rf /home/$u
> deluser $u
> done

So serenity has been wiped clean. I’m going to call the hosting company in just a bit and terminate service. Everything is chugging along on firefly. Our long national nightmare is over.

—

* I don’t really know if this is true, but it seems a good guess. Input appreciated.

Disabling Trackbacks the Hard Way

Kid Amnesiac, Jessica’s blog about Simon, has always been cursed with lots of comment spam, which I had never really bothered to investigate. It was easier to simply mark the offending comments as spam every time I logged in and move on, especially since the spam comments always required moderation and never actually made it out to the viewable part of the blog.

When I had originally built Kid Amnesiac, I had required all comments to come from registered users and forced moderation on every registered users’ first posts. This seemed the most flexible balance between making it easy for users to post comments and not being overrun with spam. Since all of the comment spam was ending up in Moderation I assumed that the spam bots were actually registering as users before they posted, but a quick review of the Users page made it obvious that that was not the case. A few days ago it occurred to me that the spam must be Trackbacks, so I went into the Discussion Options configuration page and disabled them.

But the spam kept coming. In fact, in just the last three days it had dramatically increased. So this morning I decided to check how trackbackability was enabled in the database. “DESCRIBE wp_posts;” showed that there was a column named ping_status, described as “enum(‘open’,’closed’)”. Armed with that information it was easy to see what was going on.

mysql&gt; SELECT post_title, post_date
-&gt; FROM wp_posts
-&gt; WHERE ping_status = 'closed'
-&gt; ORDER BY post_date ASCENDING;
+-----------------------------------+---------------------+
| post_title                        | post_date           |
+-----------------------------------+---------------------+
| Fifteen Months and a Growth Spurt | 2008-01-18 12:19:38 |
| These Shoes Were Made for Walking | 2008-01-19 21:43:23 |
+-----------------------------------+---------------------+
2 rows in set (0.00 sec)

The configuration change I had made had only affected the posts that were published after the modification, so now it was up to me to change the ping_status for all of the previous posts in the database.

mysql&gt; UPDATE wp_posts
-&gt; SET ping_status = 'closed'
-&gt; WHERE ping_status = 'open';
Query OK, 299 rows affected (0.01 sec)
Rows matched: 299  Changed: 299  Warnings: 0

mysql&gt; SELECT post_title, post_date
-&gt; FROM wp_posts
-&gt; WHERE ping_status = 'open';
Empty set (0.00 sec)

Perfecto! Remember, kids, always back up your WordPress database before monkeying around in it. If you screw it up, your wife may stab you.

Network Manager vs Keyring Manager

I decided to use some of my rolled-over vacation days this week and give myself a four-day weekend. It’s been nice, and I’ve crossed a few items off my tasklist: rebuilt the okcomputer.org CA, put new tires on the car, installed DD-WRT on my Linksys WRT54G wireless router.

Things were humming along until about two hours ago, when my laptop started doing that annoying thing where Network Manager just out of the blue asks for your wireless key. This has happened in the past, and sometimes I’ve been able to just give my wireless key and move on, other times I’ve had to log out and log back in before it would work again. It hadn’t happened in quite a while (maybe since the Gutsy upgrade), and I hadn’t thought about it in a while.

Well, tonight it hit me out of nowhere and I’m down hard. Nothing I do seems to fix it. I’ve deleted the login keyring, logged out and back in, but when I provide Network Manager with the key on login it never gets saved to the new keyring.

The logs seem to indicate that when I submit the key Network Manager sends out DHCP requests, but never gets a response. This would make sense if the key isn’t working and the DHCP requests aren’t encrypted correctly. But why wouldn’t the key work? Why did the key stop working in the first place?

Is my wireless card just crapping out? Maybe. It’s over three years old, but I still seem to be getting a decent signal.

Could it have something to do with my installing DD-WRT on the router earlier this evening? I doubt it — I used the router successfully for several hours before the problem started, and two other laptops work just fine — but I feel like I can’t rule out such an obvious potential connection.

I’ve got a few more ideas for debugging this, but I’m going to bed now and I’ve got a full day scheduled for tomorrow. I’ll post an update tomorrow night.

UPDATE (Sun Jan 20 09:24):

Well, it’s working now, but I’m not exactly sure why. I power-cycled the router last night before bed and then booted the laptop this morning after breakfast. As expected, I immediately got the “Wireless Network Key Required” dialog box. So I stopped the gnome-keyring-daemon, deleted ~/.gnome2/keyrings/login.keyring and then rebooted for good measure. This time when I logged in and got the “Key Required” dialog box, I gave it the key, tailed the logs and watched the DHCP process complete successfully. Sure enough, I’m online.

The only differences between this morning and last night are the power-cycled router and my proximity. I’m about three feet away from the router now. Last night I was downstairs in the living room, but I’ve routinely used the laptop there in the past and I was using another wireless laptop from there last night. I’ll just take what I can get now, and then do some distance testing later today when I have more time.

UPDATE (Mon Jan 21 12:09):

Everything has worked fine for the past 27 hours. Honestly, I’ve got no idea what happened or why I was able to fix it on Sunday with techniques that didn’t work on Saturday. The only variable that I can’t account for is the router that I power-cycled. Maybe something was monkeyed up with the leases in the DHCP server….

First Contact Winner!

So ever since I pushed the new matthewwhitworth.com site into production Jessica has been bugging me about whether or not it was getting any hits. Although I kept telling her that I was giving it time and didn’t expect anything for quite a while, I was actually checking the web server logs much more frequently than I care to admit.

A few days ago I had done the old “google yourself” trick and was shocked at how much my sabbatical had caused me to drop off the Internet radar. I haven’t posted to Usenet or the Debian mailing lists for years.  My old web page had been down for months during the crash, and the content had been dormant for over a year. Really, what did I expect?

Anyway, about 10 minutes after Jessica checked in last night, I went to give my email one last perusal before bed (I wasn’t watching the logs — I promise) and sure enough, there was the first mail from my new Contact page, from old middle/high school associate Jay V. So that was cool — I hadn’t heard from him in almost 20 years. And since he was the first person to get in touch with me via the new site, he won an MP3 of “Barbed Wire”, a song by my first band, Sacrilegious Toejam. More on that later….

Shakedown Cruise

I got bold and decided to push matthewwhitworth.com into production today, and the bugs are already starting to shake out. The big problems seem to be in the theme I chose to use, miniBits. I liked it because of its clean layout and minimal graphics, but its small size also gave me the suspicion that some edge cases might not handle so well.

It turns out it’s more than the edge cases. About 15 minutes after pushing the site into DNS I noticed that the simple “view post” action produces invalid xhtml and displays the sidebar all wonky down at the bottom of the page.

So why didn’t I (or maybe the author) notice this earlier? Well, it’s only broken when you’re not logged in! I’ve been doing most of my development so far while logged in, either as myself or as admin, so the bug just wasn’t occurring. It’s also possible that he sidebar was off the bottom of my browser window and consequently I wasn’t seeing how broken it was when I was viewing the first couple of posts. Who knows? But now it’s gonna be out in the open for a while until I can fix it.

(Interestingly, the bug also doesn’t manifest itself at the demo site. The author freely admits that the theme has only been tested with WordPress 2.1 and hasn’t been maintained for a while. Could version compatability be the culprit?)

I also need to do a bit of Internationalization and Localization. The original author appears to be Italian, and the strings in the program are a mixture of Italian and English. That will actually give me a good “learn how to do something” project (as opposed to the more annoying “bug fix” project) .

So I’m pulling the theme code into bzr right now and getting ready to tweak. Hopefully, I’ll knock out the sidebar display bug tonight and get the site linked over to okcomputer.org so it can really go live.

UPDATE: I found and fixed the bug, and it’s a PHP classic: mixing your layout code inside conditionals. Down in the comments.php file I found two </div> tags were inside an if clause when they shouldn’t have been. A quick cut & paste moved them below the endif, and things are suddenly looking much better.

“The Tempest” at ATL

Jessica and I went to see The Tempest at Actors Theater of Louisville today, and I think Jessica spoke for both of us when afterwards she asked, “What was Shakespeare smoking when he wrote this one?”

I’m not gonna bother with a plot synopsis (it’s Shakespeare, for Pete’s sake — buy the Cliff’s Notes), but the production of it was very interesting. First, it was performed in the round and with no real set. In fact, the stage props really consisted of just two sheets, occasionally rearranged on the floor to simulate a seat or a hiding place.

Next, and most interesting to me, was that the music was all performed live by three musicians tucked away in a little viewing area (that just happened to be very visible from my seat). There was a lot of violin and percussion, occasional acoustic and electric guitar, and plenty of other stuff that I couldn’t quite see. Afterwards I discovered that the musicians were all from the band Rachel’s, one of many Louisville post-rock bands that Tony B is always trying to get me interested in.

Third, and most interesting to Jessica, was that Ariel performed almost the entire play flying around on a harness. I was pretty impressed by the fluidity of movement that was required. (I could just imagine myself up there, hanging limp as a fish and generally making an ass of myself.) I think Jessica was just impressed with Ariel’s pecs. Ariel also delivered some of his lines as song. I’m not sure if the lyrics were straight from the script or adapted for this production, but it was a very cool addition.

« Previous Page — Next Page »