Z8NA and lm_sensors, again

So I was setting up another Z8NA and wanted to get sensors working. After running around in circles, I simply installed from source and now it Just Works. Which is so much better, compared to last times results. BTW, I also install lm-sensors 3.3.4 by hand from source and got a z8na.conf from here.

cd ~/work/sys
wget http://khali.linux-fr.org/devel/lm-sensors/config/sensors3-Asus-Z8NA-D6.conf 
sudo cp sensors3-Asus-Z8NA-D6.conf /etc/sensors.d/z8na.conf
wget http://dl.lm-sensors.org/lm-sensors/releases/lm_sensors-3.3.4.tar.bz2
tar jxvf lm_sensors-3.3.4.tar.bz2
cd lm_sensors-3.3.4
make
sudo make install
sudo sensors-detect
sudo sensors -s
sudo sensors

For what its worth, kmod-w83795 RPMs are available in my repo.

Don't write a program that does $X

A much better idea is to write a library that does $X, then write a front end program that uses that library. Your design will be better. Your program will be more flexible. Code reuse is much easier.

Introverts

So I just stumbled across a video about introverts and thought I'd share it.

Creating a DVD

Creating a DVD is easy but also harder then it needs to be. I wish I could just say "I have an NTSC TV. I have an .avi over there. Make it so." But alas there are all sorts of fiddly bits that need dealing with. So I need a GUI front end that understands the fiddly bits. But installing on CentOS one can get complicated.

First you need CentOS 6.

Then you need ffmpeg 0.7, available from the NauLinux school repo. You might already have ffmpeg 0.6.5 installed, in which case you'll need rpm -e ffmpeg-libpostproc --nodeps before doing yum --enablerepo=naulinux-school install ffmpeg. (If you skip this step, you end up with an error that looks like Unrecognized option 'vf'. Or at least you will if can find stderr, which might be in ~/.xsession-errors).

Finally you install devede, which is written in Python but is surprisingly non-fail. Linux tech has a repo. Set up that repo, then run yum --enablerepo=linuxtech-testing --enablerepo=rpmforge install devede.

Vixie is a smart guy

I found out something interesting the other day. A crontab entry has a DOW (day of week) column and a day of month (DOM) column. Because I'm a bad sysadmin, I'd never read the full manpage and assumed that if you specified both your entry would be run once every 7 or 14 years. I also suffer from NIH, so I'm writing a cron replacement in pure-perl that includes all the stuff I think a cron should have, like watching a directory for new files. So I finally read the full man page. And it turns out that Vixie was a smart guy. Having an cron job that runs every 7 or 14 years isn't useful so the DOW and DOM columns don't affect each other.

I other words

0 0 1 * 0 do-something

is in fact the equivalent of

0 0 1 * * do-something
0 0 * * 0 do-something

which is so much more useful.

This is another formulation of the "don't allow the user to do stupid things" principle I talked about a while back.

Total silence

I bought myself a fanless PSU for my birthday. I'd been eying one for a while now. Corey now has no fan on the CPU, a SSD and a fanless PSU; total, deafening silence. I had to move Corey into a different case because the PSU wants to be vented from the top. So now I've changed corey's CPU, motherboard, heatsink, hard drive, RAM, case and PSU. But it's still the same ax!

Of note: when you switch on a computer you've just assembled, one of the first bits of feedback you get is the fans going WUURRRwuuurrrrr. But not if your computer has no fans! It took me at least one panicky minute to figure this our.

Gloating

I just have to share this:

~ (fil@billy:)$ uptime
 13:48:21 up 719 days, 13:48, 11 users,  load average: 0.37, 0.22, 0.18

This means my rented VM host hasn't been rebooted in close to 2 years. Wheee!

Multiple UPSes

OK, so now for a problem that probably only affects me. I have multiple UPSes in my server closet. A large one (APC BackUPS XS-1300 LCD) for most of the computers in the closet and a mid-sized one (APC BackUPS XS-900) for the computers in my office. I have a long wire running through the floor into my office. Why not put the mid-sized one in the office itself? Because it has a noisy fan.

Both these USPes are connected via USB to one computer. This computer assigns nearly randomly the device name, based on where I've plugged them into a USB hub. This resulted in the computers in the basement thinking they were on the 900 and my office thinking it was on the 1300. Not acceptable.

Using lsusb, I see the UPSes are at 2:5 and 2:4

# lsusb
Bus 004 Device 001: ID 0000:0000  
Bus 003 Device 001: ID 0000:0000  
Bus 005 Device 001: ID 0000:0000  
Bus 001 Device 001: ID 0000:0000  
Bus 002 Device 005: ID 051d:0002 American Power Conversion Uninterruptible Power Supply
Bus 002 Device 002: ID 05e3:0604 Genesys Logic, Inc. USB 1.1 Hub
Bus 002 Device 001: ID 0000:0000  
Bus 002 Device 004: ID 051d:0002 American Power Conversion Uninterruptible Power Supply

Again with lsusb, I can find the serial number:

# lsusb -v -s 2:5
Bus 002 Device 005: ID 051d:0002 American Power Conversion Uninterruptible Power Supp
ly
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               1.10
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0         8
  idVendor           0x051d American Power Conversion  
  idProduct          0x0002 Uninterruptible Power Supply
  bcdDevice            1.06
  iManufacturer           3 American Power Conversion
  iProduct                1 Back-UPS XS  900 FW:830.E8 .D USB FW:E8 
  iSerial                 2 BB0842016813  
... SNIP ...

Another way to get the same info:

# udevinfo -a -p /class/usb_device/usbdev2.5
  looking at device '/class/usb_device/usbdev2.5':
    KERNEL=="usbdev2.5"
    SUBSYSTEM=="usb_device"
    SYSFS{dev}=="189:132"

  looking at parent device '/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.3':
    ID=="2-1.3"
    BUS=="usb"
    DRIVER=="usb"
    SYSFS{configuration}==""
    SYSFS{serial}=="BB0842016813  "
    SYSFS{product}=="Back-UPS XS  900 FW:830.E8 .D USB FW:E8 "
    SYSFS{manufacturer}=="American Power Conversion"
    SYSFS{maxchild}=="0"
    SYSFS{version}==" 1.10"
... SNIP ...

Then I created some custom udev rules in /etc/udev/rules.d/99-apc.rules:

KERNEL=="hiddev*", SYSFS{serial}=="BB0842016813*", NAME="apc900"
KERNEL=="hiddev*", SYSFS{serial}=="JB0741003708*", NAME="apc1300"

I could also have used SYSFS{product} for matching, but the serial number uniquely defines these UPSes. I need the * for globbing because the serial number is padded with spaces, as can be seen in the udevinfo output.

Next reload the udev db:

# udevcontrol reload_rules
# udevtrigger
# ls -l /dev/apc*
crw------- 1 root root 180, 97 Oct  5 12:53 /dev/apc1300
crw------- 1 root root 180, 96 Oct  5 12:53 /dev/apc900

YAY!

Today in scamming news

They got shut down.  I don't know if it's the scammers that have phone me and Papy.

More scams

A scammer with a Indian accent just called me, CLID was 760-705-8888. The caller claimed to be from "National Hydro Regulatory something or other" and that he was going to reduce my "hydro bill" by 40%. The connection was really bad and laggy so I didn't get much out of him. He didn't seem to know what he was talking about. When I asked why he sounded so much like a scammer and if the next bit of info he wanted was a credit card number, he called me an asshole and hung up. Had the connection not been as laggy and me less busy I might have tried to get more info out of him.

You now all have a pre-warning of this specific scam.

What's more, if ever someone phones you up, that you've never heard about, asks for CC info or pretty much anything and won't give you a call back number, well HEY IT'S A SCAM.

Error messages

Continuing on my list of programming dictums that I invented over the years.

Error messages are an indication of failure on the part of the programer, not of the user.

If a user isn't supposed to do something, then the program shouldn't let him or her do it. Classic example: If you want the user to type a number, throw away any keystroke that isn't 0-9 or editing keys. Do not allow the user to type in "monday" and then throw an error message on the screen.

What's more, an error message should be short, but contain enough information for a programmer to reproduce them. In my large application at work, I save pretty much every variable I can get my hands on to a log file. I then display the log files name to the user. Users nearly always send me a screen cap of any error message they encounter (embeded in a .doc, but that's another story) so I can get the ID from there. Of course, I can also just look in the log directory for new error logs. Taking this one step further, the app could email me the ID when an error occurs.

An object method may make a decision or change something

Lucs pushed me to start a small series of posts on programming dictums. I'm going to start with the one I thought up myself and post a few more as I think of them/get them written.

The first deals with OO. I do 90% of my coding in OO. In fact, one of my profs back in CEGEP commented that my assembler programs had an OO style.

So, first dictom is something I realised a few years ago: An object method may make a decision or change something.

I never quite got the hang of MVC because I couldn't understand what goes in the Controler, what goes in the Model. The answer, of course, is related to side-effects. Side-effects and decisions should not be mixed up in one method.

Ideally one should have one method per decision. If the method needs to load some data to make the decision, the loading code of course goes in a separate method. These methods may remember their decisions, but that's it as far as side effects go. I like to give these methods names like needs_barcode(), has_done_something(), is_ajax() but that's personal taste. These methods will tend to go in a Controller or be used by the Controller (if one is doing MVC, that is).

Methods that change something means any side effect: saving a file, setting a variable, munging some data. Ideally one wants to cut the side-effects up into small methods that handle related data. These methods will tend to go in a Model.

Done right, top level methods look like the following made up example.

if( $self->is_ajax ) {
    $self->output_json;
}
else {
    if( not $self->has_sent_header ) {
        $self->output_header;
    }
    $self->output_html;
    if( $self->is_streaming )
        $self->queue_next;
}

The benefits to dividing the code as I propose is when it comes to implementing sub-classes. Fine grain methods allow one to overload just the behaviour necessary, without (worse case) copying huge methods just to tweak a small part. It also makes it harder to stumble on hidden interdependencies and unintended consequences.

What's more, smaller methods are always a good thing. Small chunks are easier to stare at and prove to oneself that, yes they are doing what you intended them to do. They also improve readability, create a smaller units to build tests around and make refactoring code easier. What's more, I find that a good half of refactoring is taking large methods and cutting them up into smaller chunks.

CookingItRight.com

Now the idea of the Fricktionless Scams has a new life in the Web 2.0 era: You write the code, users generate the content, advertisers pay the bills. Of course you then have to deal with the Greater Internet Fuckwad, spam and all the drama that an easy facsimile of anonymity brings. This last bit kind of raises the bar you have to leap over to create a web tool. What's more, you now need an smart phone app or three.

But still, with stories like that of Plenty of Fish, it's always tempting to dream of the easy money that would be Mine All Mine if only I could spend 3 months coding up a website that would run itself.

One market that for me hasn't been done right is cooking and recipes.

At one extreme you have cooking blogs. These are small well curated collections of recipes; often with first class food photography. Hooked on Heat is a good example: Excellent recipes, well written and you know they are going to work.

At the other extreme are shovelware sites like cooks.com and barnonedrinks.com. Cooks.com exists solely to place ads in front of you. The recipes are untried and unvetted, the formatting is painful, the site is useless to me. BarNoneDrinks is no where near as bad, and given that mixing up a cocktail takes all of 5 minutes, you won't loose out too badly if it turns out the recipe is a dud. But they have no way of searching for recipes that contain 2 specified ingredients, they have a lot of duplicate recipes and recipes that are basically what a few teens threw together from their father's liquor cabinet and yeah, bro, that's totally a Purple Panty Pooper! In other words, they lack curation.

In between the two are the lifestyle sections of magazines and cooking forums. Forums rely on the Googles to be able to find anything, you have to scan through all the threads to find out if the recipe is broken or not. And of course they all use Ultimate BBS which makes them a nightmare from a UI stand point.

From these examples, a list of what I want: curation or some form of vote or reputation mechanism that would rate a recipe, comments on the recipe and variations of the recipe. Slashdot's moderation system comes to mind, as does StackExchange where some uses can actually modify a question. As to variations, how about generating a table that would allow one to easily compare different related recipes. I mean, you'd be surprised by the variations I've found in the spice mix for pumpkin pie.

The one thing Barnonedrinks is good at is the fact that the ingredients are marked up. That is, they know that creme de menthe is an ingredient, not just some text. With this, you can do something like the Google Cookbook: cross index 3 ingredients you have on hand to find a recipe for supper.

But not only ingredients: techniques (creaming butter, kneading dough), quantities (is that a fluid ounce (30ml) or an ADP once (28g)? Do you want this recipe for 2 or 20 people? In metric or American customary units?), maybe even cooking style (Indian, South-Asian, Bengali and curry can all be inaccurately used as synonyms).

Ideally, one could also rearrange the recipe format. There's the One True Recipe format (a list of ingredients, followed by instructions), the Joy Of Cooking (ingredients are inline in bold with the instructions), and the one I'd like to see: a hybrid of the two (ingredients all in a list at the top, but grouped and linked to the relevant instructions.)

What's more, the instruction themselves could be brief (Cream the butter and sugar) or detailed (Bring the butter to room temp (don't melt!), beat it and the sugar in a mixer until it has a uniform consistency and light yellow texture. About X minutes. This is called creaming.)

These last 2 points (reformatting and instruction level) would mean a lot of markup is needed in a recipe. So my hypothetical CookingItRight.com would entail need a UI that made this a painless as possible.

Lastly you need a dictionary of ingredients and equivalents. What exactly is Chinese 5 spices? Is all wheat flour identical? Paprika is basically food colouring in the USA, but is smoky and spicy in Europe. Are prawns shrimps? Is a red pepper a capsicum?

Which brings us far far from our original goal: the Frictionless Scam.

Hacked

Today I discovered that one of my customer's servers was hacked. Very few of my customers take computer security seriously. These people were worse then the average. Googling around for details on this root kit, I found two articles with a description of some techniques for attack analysis.

Of course, ideally we'd take off and nuke it from orbit. Only way to be sure. I MEAN take that the server out of service and reinstall from scratch.. This is not an option in this case.

One thing this root kit did was replace /usr/bin/ssh with a compromised one that logged all passwords to a file that was mailed out every night. I was depressed to find the file had the root password to my associate's home computer. So I changed the root password, and looked around for more stuff. Most of it was spread around and given important sounding names like klogd1, popauth and dnsquery.

I won't go into all the details because a- they are boring and b- probably only specific to this rootkit, but here is a few commands that will help find compromised files:

rpm -V -a
rpm -V -f /some/file

This assumes you have an RPM-based distro. Go read `man rpm` for details. TL;DR: any line with a 5 has changed since it was installed. If it's a config file, review it to make sure it doesn't do something bad. If it's an executable or shared library, reinstall that package.

ls -lint | sort -g | less

Picked this up from the articles above. The first column is the inode of the file. If you see any inodes that are far from the others in the list (they will be at the bottom of the list) they were installed after the initial setup. This isn't a 100% sign that you've been hacked; it happens when you do a `yum upgrade`. But those files do merit a closer look.

Double joins

I finally groked something I was probably told to do 5 years ago. Actually I have no memory of being told to do this trick, but I should have been. It's so sweet. It's allowed me to rewrite and rethink the database queries for my archiving app.

If you don't know that a table join is, you'll have to learn about that first. So go do that and come back.

OK, now what is so sweet about a double join? First off, I'll point out that I'm not joining 3 tables together, I'm joining 2 tables together TWICE.

Why would I do this? A common idiom is to have a main table that contains all the fields that every widget (documents, articles in your store, whatever) has and another table that contains one or more rows per-widget that contain fields that only that widget or only that class of widget has. The alternative is to have one table that contains a field for every possible field of every possible class of widget, which can be very wasteful. And then you have to MODIFY TABLE each time you want to add or remove a field. Or you could have a table per class of widget, which reduces waste, but increases complexity.

So, a hypothetical ecommerce schema would looks roughly like:

CREATE TABLE main (
    int SKU PRIMARY KEY,
    timestamp created,
    int price,
    int in_stock,
    int category
);

CREATE TABLE extra {
    int SKU PRIMARY KEY,
    varchar(20) field,
    varchar(255) value
};

(Yes, extra.field really should be an ENUM or an INT.)

Now you want to find all the items with the keyword 'tablet', but you want to sort on the french name of the product:

SELECT SKU,E1.value as keyword,ES.value AS sort_field FROM main 
    JOIN extra AS E1 ON main.SKU = extra.SKU AND field = 'keyword'
    JOIN extra AS E2 ON main.SKU = extra.SKU AND field = 'fr'
    WHERE E1.value = 'tablet'
    ORDER BY sort_field;

Look at that, it's like you've temporarily added 2 new fields to main, 'keyword' and 'title'. What's more, to sort in English, I just change 'fr' to 'en'. To sort by manufacturer, change it to 'manufacture.'

Seems to me this invalidates 75% of the reason for NoSQL. The other 25% is that joins like the one above aren't very efficient for huge (aka web-scale) databases, the kind with multimillion records. The answer to that is that RAM IS CHEAP and USE SSDs.

Shitlist

Added: Agrupur.
For dedicating a entire face of a 2 liter milk jug to an ad for a certain privacy invading website who's name begins with eff.

Though I do give them points for the decent use of whitespace.

The Frictionless Scam

When we were much younger, Dylan and I would dream up things we could do to make money with as little effort as possible. The goal is what I call the "Frictionless Scam." That is, you want no contact with the client nor with the police. So it has to be legal and something that doesn't require the client in the room with you. You want to cash the check, hand over the product and everyone walks away happy.

A good example would be computer-generated astrology or biorythm charts. Feed some random or nearly random numbers into a program, print out the chart, mail it to client and cash the check.

Another silly idea I had while stuck in a traffic years back was auditing vehicles for possession by a ghost or extraterestrial or something. Which would involve a box with blinkenlights and dials. Then you create a certificat that guarantees the vehicle is ghost-free. Now given that ghost don't exist, you would be pretty safe from a legal stand-point. But the fact that you have to do a show and dance for the customer means this isn't a frictionless scam.

Neal Stephenson came up with a great Frictionless Scam: plug an Eliza-like chatterbox into voice synth and voice recognition software to build a phone sex service. "Is it because of your massively erect penis that you came to see me?" "Enough about me, what are you wearing?" "Do my large melons excite you?" "What does this tell you?" (If you haven't played with an Eliza, you might not get the previous joke.) Of course, neither voice synth nor voice recog are up to the point where we can do this and you still need a large server somewhere with an expensive T1 to pull it off. What's more, in 2012 it would have to be a 3d generated web cam stream.

Now it seems that Print On Demand and Amazon have created another near fricionlessscam. Take some Wikipedia articles, edit them into a book, sell book on Amazon. And thanks to PoD, you only have to print the book if/when you actually sell it. And given that Wikipedia has good markup, you can pretty much automate the conversion from Wiki to PDF that gets sent to the On Demand printer.

How Google Jumped The Shark

I was going to call this "Year end round up" but I'm 2 months late. And there were far more important things going on last year.

Google jumped the shark.

It was the year they declared war on your private life. Here is a great quote from Google CEO Eric Schmidt "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place." It is extremely startling that a man as smart as him doesn't realise everything that is wrong with that statement. Remember, this from a company who's mission statement used to be "Don't do evil." Of course, Schmidt is a rich straight male of european descent and has profited handsomely from that Invisible Knapsack he has.

And of course they chose the wrong side of the Nym Wars.

But for me, the real signal that Google No Longer Gets It was that they pulled the plug on Google Wave. This was the most exciting thing I've seen since... well since the web was won, I guess. Take email, IRC, Wikis and online forums. Remove all the elements of those that make them annoying. Then add in RSS feeds, images, spreadsheets, version control, real time simultaneous editing and more. Take AJAX, Comet and all that, then make something helluva useful out of it.