Hacked

Today I discovered that one of my customer's servers was hacked. Very few of my customers take computer security seriously. These people were worse then the average. Googling around for details on this root kit, I found two articles with a description of some techniques for attack analysis.

Of course, ideally we'd take off and nuke it from orbit. Only way to be sure. I MEAN take that the server out of service and reinstall from scratch.. This is not an option in this case.

One thing this root kit did was replace /usr/bin/ssh with a compromised one that logged all passwords to a file that was mailed out every night. I was depressed to find the file had the root password to my associate's home computer. So I changed the root password, and looked around for more stuff. Most of it was spread around and given important sounding names like klogd1, popauth and dnsquery.

I won't go into all the details because a- they are boring and b- probably only specific to this rootkit, but here is a few commands that will help find compromised files:

rpm -V -a
rpm -V -f /some/file

This assumes you have an RPM-based distro. Go read `man rpm` for details. TL;DR: any line with a 5 has changed since it was installed. If it's a config file, review it to make sure it doesn't do something bad. If it's an executable or shared library, reinstall that package.

ls -lint | sort -g | less

Picked this up from the articles above. The first column is the inode of the file. If you see any inodes that are far from the others in the list (they will be at the bottom of the list) they were installed after the initial setup. This isn't a 100% sign that you've been hacked; it happens when you do a `yum upgrade`. But those files do merit a closer look.

Double joins

I finally groked something I was probably told to do 5 years ago. Actually I have no memory of being told to do this trick, but I should have been. It's so sweet. It's allowed me to rewrite and rethink the database queries for my archiving app.

If you don't know that a table join is, you'll have to learn about that first. So go do that and come back.

OK, now what is so sweet about a double join? First off, I'll point out that I'm not joining 3 tables together, I'm joining 2 tables together TWICE.

Why would I do this? A common idiom is to have a main table that contains all the fields that every widget (documents, articles in your store, whatever) has and another table that contains one or more rows per-widget that contain fields that only that widget or only that class of widget has. The alternative is to have one table that contains a field for every possible field of every possible class of widget, which can be very wasteful. And then you have to MODIFY TABLE each time you want to add or remove a field. Or you could have a table per class of widget, which reduces waste, but increases complexity.

So, a hypothetical ecommerce schema would looks roughly like:

CREATE TABLE main (
    int SKU PRIMARY KEY,
    timestamp created,
    int price,
    int in_stock,
    int category
);

CREATE TABLE extra {
    int SKU PRIMARY KEY,
    varchar(20) field,
    varchar(255) value
};

(Yes, extra.field really should be an ENUM or an INT.)

Now you want to find all the items with the keyword 'tablet', but you want to sort on the french name of the product:

SELECT SKU,E1.value as keyword,ES.value AS sort_field FROM main 
    JOIN extra AS E1 ON main.SKU = extra.SKU AND field = 'keyword'
    JOIN extra AS E2 ON main.SKU = extra.SKU AND field = 'fr'
    WHERE E1.value = 'tablet'
    ORDER BY sort_field;

Look at that, it's like you've temporarily added 2 new fields to main, 'keyword' and 'title'. What's more, to sort in English, I just change 'fr' to 'en'. To sort by manufacturer, change it to 'manufacture.'

Seems to me this invalidates 75% of the reason for NoSQL. The other 25% is that joins like the one above aren't very efficient for huge (aka web-scale) databases, the kind with multimillion records. The answer to that is that RAM IS CHEAP and USE SSDs.

Shitlist

Added: Agrupur.
For dedicating a entire face of a 2 liter milk jug to an ad for a certain privacy invading website who's name begins with eff.

Though I do give them points for the decent use of whitespace.

The Frictionless Scam

When we were much younger, Dylan and I would dream up things we could do to make money with as little effort as possible. The goal is what I call the "Frictionless Scam." That is, you want no contact with the client nor with the police. So it has to be legal and something that doesn't require the client in the room with you. You want to cash the check, hand over the product and everyone walks away happy.

A good example would be computer-generated astrology or biorythm charts. Feed some random or nearly random numbers into a program, print out the chart, mail it to client and cash the check.

Another silly idea I had while stuck in a traffic years back was auditing vehicles for possession by a ghost or extraterestrial or something. Which would involve a box with blinkenlights and dials. Then you create a certificat that guarantees the vehicle is ghost-free. Now given that ghost don't exist, you would be pretty safe from a legal stand-point. But the fact that you have to do a show and dance for the customer means this isn't a frictionless scam.

Neal Stephenson came up with a great Frictionless Scam: plug an Eliza-like chatterbox into voice synth and voice recognition software to build a phone sex service. "Is it because of your massively erect penis that you came to see me?" "Enough about me, what are you wearing?" "Do my large melons excite you?" "What does this tell you?" (If you haven't played with an Eliza, you might not get the previous joke.) Of course, neither voice synth nor voice recog are up to the point where we can do this and you still need a large server somewhere with an expensive T1 to pull it off. What's more, in 2012 it would have to be a 3d generated web cam stream.

Now it seems that Print On Demand and Amazon have created another near fricionlessscam. Take some Wikipedia articles, edit them into a book, sell book on Amazon. And thanks to PoD, you only have to print the book if/when you actually sell it. And given that Wikipedia has good markup, you can pretty much automate the conversion from Wiki to PDF that gets sent to the On Demand printer.

How Google Jumped The Shark

I was going to call this "Year end round up" but I'm 2 months late. And there were far more important things going on last year.

Google jumped the shark.

It was the year they declared war on your private life. Here is a great quote from Google CEO Eric Schmidt "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place." It is extremely startling that a man as smart as him doesn't realise everything that is wrong with that statement. Remember, this from a company who's mission statement used to be "Don't do evil." Of course, Schmidt is a rich straight male of european descent and has profited handsomely from that Invisible Knapsack he has.

And of course they chose the wrong side of the Nym Wars.

But for me, the real signal that Google No Longer Gets It was that they pulled the plug on Google Wave. This was the most exciting thing I've seen since... well since the web was won, I guess. Take email, IRC, Wikis and online forums. Remove all the elements of those that make them annoying. Then add in RSS feeds, images, spreadsheets, version control, real time simultaneous editing and more. Take AJAX, Comet and all that, then make something helluva useful out of it.

Observer bias

Another way of putting it:

At its basic level, the Law of Fives is a practical demonstration that perception is intent-sensitive; that is, the perceiver's intentions inform the perception. To whatever extent one considers that perception is identical with reality, then, it has the corollary that reality is intent-sensitive.

A startup

I realised a few months back that while I claim to be a programmer, I could also be called a serial-entrepreneur. I've founded 4 companies over the years (Educatic, ARTware, Awalé and Quaero). Me being an entrepreneur is kinda ironic, if you know me.

Anyway, the real goal of this post is to get a link to my latest companies brand new website. I'd like to brag about how awesome this website is, but it isn't awesome and prolly never will be. I didn't make it so don't blame me for the flash animation and canned newsroom sound.

But the thing is though, the product we sell really kicks ass. A fully configurable automagic document classification and document archiving intranet application. It can handle volumes of over 1 million documents a year. That's a lot of documents. One thing I realised a while back is that we didn't just write an archiving app, we wrote a tool to build an archiving app with. Which is both great and a pain because each install is so customised I have trouble keeping track.

It is noteworthy that this is the very first time one of my companies has even HAD a website. Which is another irony given that I developed websites for a living for over 10 years.

The URL again, for those that don't know how to the clicky-clicky: http://www.quaero.ca

(Full disclosure: I founded Quaero and yes this is a blatant attempt at seeding the search engines.)