know your tools; a story of web forms and why they always suck

If there's a contentious issue in web development (could be other development, too, but I don't know!), it's capturing user input. Forms are irritating - validating, securing, sanitising, storing - no one likes doing it. We write libraries to do the legwork, then hate using those libraries! Forms are counter-intuitive to write and they're a pain in the ass to use. We instinctively, as web developers, try to guess what all our users might enter, then write receivers accordingly. Courteous users will try to tailor their input based on what they think the form wants to know. This is probably based on no previous knowledge at all, although previous use of forms may ingrain a way to enter dates and currency, but it's all input.

Computers are scarcely clever enough to infer the format or context of input (or, at least, I'm not clever enough to write something to do that!) so we have to hand-hold users through the process. We write beautiful copy (that never gets read), we tailor error messages as well as we can based on the nature of their input and how it fails the test (that never get read) and we sit by the support email account and wait for people who can't be bothered to read, but can be bothered to email, to let us know how we're failing. Even if you believed that forms worked 2 minutes ago, you'd be naive to believe that they do now. I am the authority, do not question.

So. Forms suck. Sadly, as a web developer (if you are one), it's your job to make them as accommodating as you possibly can. This is a relatively easy thing to do, and I hate to use badly-written forms when there's really no excuse for it. The catalyst for this post is equal parts simple and irritating, but I'll get to that in a minute. Think about how the data entered in your form is going to be used. If your validation method could ever be broken with valid data, you NEED to rethink it. If I can't enter my email address in your shop's signup form, you can bet I'm buying elsewhere (you'd be surprised how easily some forms get fooled by two-letter TLDs). If you require a phone number, but I'm entering my number right and your form isn't accepting it, it's YOUR FORM that's broken (you can circumvent this by giving clear directions, that way, if they don't get read and you get a complaint, you can walk users through and sarcastically encourage them to read guidelines you've put in place).

The handiest of handy techniques I ever employed in form validation is to teach myself how users enter data in forms. You can do this so easily by having your validation class or method or whatever, send you an email, or log to a file the thing that tripped it up, the contents of post, get, session, time, IP, server variables, debug backtrace, everything you can get, then study it when you're doing improvements. Learning about how users use something is nowhere near as valuable as learning how your users use something. It also gives a great point of reference when walking a user through your app to determine where they're going wrong, and it can even help you to improve your copy.

That being said, the next person who develops something that encourages every user to actually read the great instructions you write will be the first!

Which kinda brings me onto why I'm even ranting in the first place. As a user, I hate using forms. As a developer, I hate creating them (unless they're doing something really cool and interesting). What I hate even more is when they're not fit for purpose. Upon filling in a particular form yesterday, I was asked to enter my country. Country can be pretty contentious in itself - it's normally a bit of a pain, do I choose United Kingdom, Britain, Great Britain? Has my country been prioritised as it's a major customer of this website? Is the list in alphabetical order otherwise? Do I scroll to U, B or G? Whilst these are small issues, when you fill in as many forms as I do, the fluctuation in input method can start to grate!

My favourite kind of way to enter country is by free text field. It's a little risky if you're joining users with a table of countries as you can never be sure that you've joined user input with the correct country, but it does make life easier. So, I entered my country as "United Kingdom" and was met with the following error:


This is, by far, one of the most stupid, avoidable messages that I have ever come across in a form. Number one, if they don't want punctuation in my input, it can be stripped programmatically, even in the javascript that passes the form to the server: string.replace(/[^A-Za-z]*/,'');

If they need it in upper case, that can also be done by the javascript: string.toUpperCase();

So, that's the first 2 of their issues, both of which become entirely moot when you come across their final issue; Choose a country from this list! Seriously?! Have you ever heard of a select list?! I know you have, because I already used one on your website! Justification for this sort of shortsightedness evades me, but I am open to enlightenment.

Short and short, users have the attention span of a gnat, and the temper of a recently-woken bear. Irritating either one of those is an easy way to lose a customer, so please try to make your forms minimally irritating. If I end up using one, I may just have to hunt you down and moan at you.

a rookie mistake, a programming challenge and a fresh start

It's gone. 4 years and 30GB of photos gone because of a stupid lapse in concentration. All my originals are just nowhere - rsync --delete really fucked me this time!

Never the one to look on the dull side of things, this presented an interesting challenge. Seeing as I don't know exactly when it happened, the chance of an undelete was pretty slim. So I turned to Google to see what was available to me in the way of backing up my flickr photos. It's not quite 30GB of originals, but 1145ish decentish resolution exports is much better than nothing.

There are a couple of things available, but the only notable one is Java, slow, crap and doesn't download some photos with no explanation, so that's that. Seeing as I basically only wanted a straight backup of everything of mine on flickr, I figured that there had to be a way that I could just get a list of all my photos so that I could archive them and be done with it. This wasn't really the case, so I opted for using flickr's API calls for everything in sets and everything not in sets. Because, if I ever decide to put it out in public, I wouldn't want it getting abused; I've decided to use flickr's authorisation API to confirm the user as well. It works really well, and is downloading my originals as we speak.

The only potential issue is, because it puts your set photos in their own directory, there will probably be some duplicates, but I'd rather have 2 of one than none. If you want a copy of the script, you'll need OSX/Linux with PHP 4 at least. Leave a comment and I'll email it over.


And, now it's time to start a new era of photos.

Speaking of...

It's fun to purposefully mislead people; luckily, conversation is a great vessel for this.

As well as xkcd's awesome example, I like to say "speaking of [something]" as a precursor to a topic, when said topic has never even so much as been alluded to. It's best to do with confrontational people—you then get to try and convince them that you had definitely mentioned it and they probably weren't listening.

Sometimes boredom is a dangerous thing.

304: new things.


My laptop, rather suddenly, gave up on me yesterday so I've had to get a new one. All technology seems to be against me recently. That's quite an annoyance for a web developer/wannabe programmer.

You're looking at our new bed/part deposit on a house. The worst part is, I'm not even excited. Is that a breakthrough?

The white noise of Twitter.

OK, so I've hit my saturation point with Twitter. I now follow 87 people, and it's too much. I appear to have entered a little sub-social circle, in which everyone re-tweets everyone else's tweets. I've already voiced my displeasure at the phenomenon that is re-tweeting and it's really ruining the whole thing for me now. So now, I'm going to be cleansing my following list. It's nothing personal if you go, you probably just talk too much about stuff that I don't care about.