Error messages

Continuing on my list of programming dictums that I invented over the years.

Error messages are an indication of failure on the part of the programer, not of the user.

If a user isn't supposed to do something, then the program shouldn't let him or her do it. Classic example: If you want the user to type a number, throw away any keystroke that isn't 0-9 or editing keys. Do not allow the user to type in "monday" and then throw an error message on the screen.

What's more, an error message should be short, but contain enough information for a programmer to reproduce them. In my large application at work, I save pretty much every variable I can get my hands on to a log file. I then display the log files name to the user. Users nearly always send me a screen cap of any error message they encounter (embeded in a .doc, but that's another story) so I can get the ID from there. Of course, I can also just look in the log directory for new error logs. Taking this one step further, the app could email me the ID when an error occurs.

An object method may make a decision or change something

Lucs pushed me to start a small series of posts on programming dictums. I'm going to start with the one I thought up myself and post a few more as I think of them/get them written.

The first deals with OO. I do 90% of my coding in OO. In fact, one of my profs back in CEGEP commented that my assembler programs had an OO style.

So, first dictom is something I realised a few years ago: An object method may make a decision or change something.

I never quite got the hang of MVC because I couldn't understand what goes in the Controler, what goes in the Model. The answer, of course, is related to side-effects. Side-effects and decisions should not be mixed up in one method.

Ideally one should have one method per decision. If the method needs to load some data to make the decision, the loading code of course goes in a separate method. These methods may remember their decisions, but that's it as far as side effects go. I like to give these methods names like needs_barcode(), has_done_something(), is_ajax() but that's personal taste. These methods will tend to go in a Controller or be used by the Controller (if one is doing MVC, that is).

Methods that change something means any side effect: saving a file, setting a variable, munging some data. Ideally one wants to cut the side-effects up into small methods that handle related data. These methods will tend to go in a Model.

Done right, top level methods look like the following made up example.

if( $self->is_ajax ) {
    $self->output_json;
}
else {
    if( not $self->has_sent_header ) {
        $self->output_header;
    }
    $self->output_html;
    if( $self->is_streaming )
        $self->queue_next;
}

The benefits to dividing the code as I propose is when it comes to implementing sub-classes. Fine grain methods allow one to overload just the behaviour necessary, without (worse case) copying huge methods just to tweak a small part. It also makes it harder to stumble on hidden interdependencies and unintended consequences.

What's more, smaller methods are always a good thing. Small chunks are easier to stare at and prove to oneself that, yes they are doing what you intended them to do. They also improve readability, create a smaller units to build tests around and make refactoring code easier. What's more, I find that a good half of refactoring is taking large methods and cutting them up into smaller chunks.

CookingItRight.com

Now the idea of the Fricktionless Scams has a new life in the Web 2.0 era: You write the code, users generate the content, advertisers pay the bills. Of course you then have to deal with the Greater Internet Fuckwad, spam and all the drama that an easy facsimile of anonymity brings. This last bit kind of raises the bar you have to leap over to create a web tool. What's more, you now need an smart phone app or three.

But still, with stories like that of Plenty of Fish, it's always tempting to dream of the easy money that would be Mine All Mine if only I could spend 3 months coding up a website that would run itself.

One market that for me hasn't been done right is cooking and recipes.

At one extreme you have cooking blogs. These are small well curated collections of recipes; often with first class food photography. Hooked on Heat is a good example: Excellent recipes, well written and you know they are going to work.

At the other extreme are shovelware sites like cooks.com and barnonedrinks.com. Cooks.com exists solely to place ads in front of you. The recipes are untried and unvetted, the formatting is painful, the site is useless to me. BarNoneDrinks is no where near as bad, and given that mixing up a cocktail takes all of 5 minutes, you won't loose out too badly if it turns out the recipe is a dud. But they have no way of searching for recipes that contain 2 specified ingredients, they have a lot of duplicate recipes and recipes that are basically what a few teens threw together from their father's liquor cabinet and yeah, bro, that's totally a Purple Panty Pooper! In other words, they lack curation.

In between the two are the lifestyle sections of magazines and cooking forums. Forums rely on the Googles to be able to find anything, you have to scan through all the threads to find out if the recipe is broken or not. And of course they all use Ultimate BBS which makes them a nightmare from a UI stand point.

From these examples, a list of what I want: curation or some form of vote or reputation mechanism that would rate a recipe, comments on the recipe and variations of the recipe. Slashdot's moderation system comes to mind, as does StackExchange where some uses can actually modify a question. As to variations, how about generating a table that would allow one to easily compare different related recipes. I mean, you'd be surprised by the variations I've found in the spice mix for pumpkin pie.

The one thing Barnonedrinks is good at is the fact that the ingredients are marked up. That is, they know that creme de menthe is an ingredient, not just some text. With this, you can do something like the Google Cookbook: cross index 3 ingredients you have on hand to find a recipe for supper.

But not only ingredients: techniques (creaming butter, kneading dough), quantities (is that a fluid ounce (30ml) or an ADP once (28g)? Do you want this recipe for 2 or 20 people? In metric or American customary units?), maybe even cooking style (Indian, South-Asian, Bengali and curry can all be inaccurately used as synonyms).

Ideally, one could also rearrange the recipe format. There's the One True Recipe format (a list of ingredients, followed by instructions), the Joy Of Cooking (ingredients are inline in bold with the instructions), and the one I'd like to see: a hybrid of the two (ingredients all in a list at the top, but grouped and linked to the relevant instructions.)

What's more, the instruction themselves could be brief (Cream the butter and sugar) or detailed (Bring the butter to room temp (don't melt!), beat it and the sugar in a mixer until it has a uniform consistency and light yellow texture. About X minutes. This is called creaming.)

These last 2 points (reformatting and instruction level) would mean a lot of markup is needed in a recipe. So my hypothetical CookingItRight.com would entail need a UI that made this a painless as possible.

Lastly you need a dictionary of ingredients and equivalents. What exactly is Chinese 5 spices? Is all wheat flour identical? Paprika is basically food colouring in the USA, but is smoky and spicy in Europe. Are prawns shrimps? Is a red pepper a capsicum?

Which brings us far far from our original goal: the Frictionless Scam.

Hacked

Today I discovered that one of my customer's servers was hacked. Very few of my customers take computer security seriously. These people were worse then the average. Googling around for details on this root kit, I found two articles with a description of some techniques for attack analysis.

Of course, ideally we'd take off and nuke it from orbit. Only way to be sure. I MEAN take that the server out of service and reinstall from scratch.. This is not an option in this case.

One thing this root kit did was replace /usr/bin/ssh with a compromised one that logged all passwords to a file that was mailed out every night. I was depressed to find the file had the root password to my associate's home computer. So I changed the root password, and looked around for more stuff. Most of it was spread around and given important sounding names like klogd1, popauth and dnsquery.

I won't go into all the details because a- they are boring and b- probably only specific to this rootkit, but here is a few commands that will help find compromised files:

rpm -V -a
rpm -V -f /some/file

This assumes you have an RPM-based distro. Go read `man rpm` for details. TL;DR: any line with a 5 has changed since it was installed. If it's a config file, review it to make sure it doesn't do something bad. If it's an executable or shared library, reinstall that package.

ls -lint | sort -g | less

Picked this up from the articles above. The first column is the inode of the file. If you see any inodes that are far from the others in the list (they will be at the bottom of the list) they were installed after the initial setup. This isn't a 100% sign that you've been hacked; it happens when you do a `yum upgrade`. But those files do merit a closer look.

Double joins

I finally groked something I was probably told to do 5 years ago. Actually I have no memory of being told to do this trick, but I should have been. It's so sweet. It's allowed me to rewrite and rethink the database queries for my archiving app.

If you don't know that a table join is, you'll have to learn about that first. So go do that and come back.

OK, now what is so sweet about a double join? First off, I'll point out that I'm not joining 3 tables together, I'm joining 2 tables together TWICE.

Why would I do this? A common idiom is to have a main table that contains all the fields that every widget (documents, articles in your store, whatever) has and another table that contains one or more rows per-widget that contain fields that only that widget or only that class of widget has. The alternative is to have one table that contains a field for every possible field of every possible class of widget, which can be very wasteful. And then you have to MODIFY TABLE each time you want to add or remove a field. Or you could have a table per class of widget, which reduces waste, but increases complexity.

So, a hypothetical ecommerce schema would looks roughly like:

CREATE TABLE main (
    int SKU PRIMARY KEY,
    timestamp created,
    int price,
    int in_stock,
    int category
);

CREATE TABLE extra {
    int SKU PRIMARY KEY,
    varchar(20) field,
    varchar(255) value
};

(Yes, extra.field really should be an ENUM or an INT.)

Now you want to find all the items with the keyword 'tablet', but you want to sort on the french name of the product:

SELECT SKU,E1.value as keyword,ES.value AS sort_field FROM main 
    JOIN extra AS E1 ON main.SKU = extra.SKU AND field = 'keyword'
    JOIN extra AS E2 ON main.SKU = extra.SKU AND field = 'fr'
    WHERE E1.value = 'tablet'
    ORDER BY sort_field;

Look at that, it's like you've temporarily added 2 new fields to main, 'keyword' and 'title'. What's more, to sort in English, I just change 'fr' to 'en'. To sort by manufacturer, change it to 'manufacture.'

Seems to me this invalidates 75% of the reason for NoSQL. The other 25% is that joins like the one above aren't very efficient for huge (aka web-scale) databases, the kind with multimillion records. The answer to that is that RAM IS CHEAP and USE SSDs.

Shitlist

Added: Agrupur.
For dedicating a entire face of a 2 liter milk jug to an ad for a certain privacy invading website who's name begins with eff.

Though I do give them points for the decent use of whitespace.

The Frictionless Scam

When we were much younger, Dylan and I would dream up things we could do to make money with as little effort as possible. The goal is what I call the "Frictionless Scam." That is, you want no contact with the client nor with the police. So it has to be legal and something that doesn't require the client in the room with you. You want to cash the check, hand over the product and everyone walks away happy.

A good example would be computer-generated astrology or biorythm charts. Feed some random or nearly random numbers into a program, print out the chart, mail it to client and cash the check.

Another silly idea I had while stuck in a traffic years back was auditing vehicles for possession by a ghost or extraterestrial or something. Which would involve a box with blinkenlights and dials. Then you create a certificat that guarantees the vehicle is ghost-free. Now given that ghost don't exist, you would be pretty safe from a legal stand-point. But the fact that you have to do a show and dance for the customer means this isn't a frictionless scam.

Neal Stephenson came up with a great Frictionless Scam: plug an Eliza-like chatterbox into voice synth and voice recognition software to build a phone sex service. "Is it because of your massively erect penis that you came to see me?" "Enough about me, what are you wearing?" "Do my large melons excite you?" "What does this tell you?" (If you haven't played with an Eliza, you might not get the previous joke.) Of course, neither voice synth nor voice recog are up to the point where we can do this and you still need a large server somewhere with an expensive T1 to pull it off. What's more, in 2012 it would have to be a 3d generated web cam stream.

Now it seems that Print On Demand and Amazon have created another near fricionlessscam. Take some Wikipedia articles, edit them into a book, sell book on Amazon. And thanks to PoD, you only have to print the book if/when you actually sell it. And given that Wikipedia has good markup, you can pretty much automate the conversion from Wiki to PDF that gets sent to the On Demand printer.