ELRepo GPG key

If you get the following error message:

GPG key at file:///etc/pki/rpm-gpg/RPM-GPG-KEY-elrepo.org (0xBAADAE52) is already installed
The GPG keys listed for the "ELRepo.org Community Enterprise Linux Kernel Repository - el8" repository are already installed but they are not correct for this package.
Check that the correct key URLs are configured for this repository.. Failing package is: python3-perf-6.13.2-1.el8.elrepo.x86_64
 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-elrepo.org
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'yum clean packages'.
Error: GPG check FAILED

The answer is:

rpm --import https://www.elrepo.org/RPM-GPG-KEY-v2-elrepo.org

See https://elrepo.org/wiki/doku.php?id=start

grub2 vs xfs

So I just tried

# grub2-install --boot-directory=/boot2 /dev/sdb1
Installing for i386-pc platform.
grub2-install: error: hd0 appears to contain a xfs filesystem which isn't known to reserve space for DOS-style boot.  Installing GRUB there could result in FILESYSTEM DESTRUCTION if valuable data is overwritten by grub-setup (--skip-fs-probe disables this check, use at your own risk).

And then I did

# grub2-install --boot-directory=/boot2 /dev/sdb1 --skip-fs-probe
Installing for i386-pc platform.
grub2-install: warning: File system `xfs' doesn't support embedding.
grub2-install: warning: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..
grub2-install: error: will not proceed with blocklists.

But of course I'm an idiot; I don't want to install the grub loader on sdb1, I want to install it on sdb, where the BIOS can actually find it

# grub2-install --boot-directory=/boot2 /dev/sdb
Installing for i386-pc platform.
Installation finished. No error reported.

thunderbird vs self-signed certs

A default dovecot install on AlmaLinux 9 creates a self-signed SSL certifiate. Thunderbird is now very picky about SSL certs. It used to tell you a certificate wasn't valid and allow you to create an exception. Now it just spins and does nothing. You will see the following in your dovecot logs:

Apr 23 18:47:42 sHOST dovecot[12484]: imap-login: Disconnected: Connection closed: SSL_accept() failed: error:0A000412:SSL routines::sslv3 alert bad certificate: SSL alert number 42 (no auth attempts in 0 secs): user=<>, rip=CLIENTIP, lip=HOSTIP, TLS handshaking: SSL_accept() failed: error:0A000412:SSL routines::sslv3 alert bad certificate: SSL alert number 42, session=<BNl+V8sWFOgKAAAF>

I spent 4-5 hours running around in circles to try and find a solution

First step is to import the key, tell dovecot to listen on port 443 (https) by adding the following lines to the service imap-login stanza in /etc/dovecot/conf.d/10-master.conf:

#service imap-login {
  inet_listener https {
    port = 443
    ssl = yes
  }

Note that you could also set up lighttpd to serve up the cert.

Restart dovecot with:

systemctl restart dovecot

Test the above with:

openssl s_client -connect YOURHOST:443

Then, in Thuderbird, you go into Hamburger > Preferences > Privacy & security > (scroll way down) > Manage Certificates... In the Certificate Manager window, you select the Servers tab and click Add Exception... and enter https://YOURHOST:443. Then click on Get Certificate and Confirm Security Exception.

We now have an exception for YOURHOST:443, but we want YOURHOST:993 (if you are using SSL/TLS) or YOURHOST:143 (if you are using STARTTLS). To fix the port number, you need to close Thunderbird, then modify the Thunderbird profile directly. Under Linux, this is ~USER/.thunderbird/SOMETHING-NON-OBVIOUS. I had a half dozen directories. To find the one you want:

cd ~/.thunderbird
find . -name cert_override.txt | xargs ls -l --sort=time

The most recently modified file is the one you want to edit.

YOURHOST:443    OID.2.16.840.1.101.3.4.2.1      HEX-STRING-HERE U       BASE64-STRING-HERE

Change the :443 on that line to :993 (for SSL/TLS) or :143 (for STARTTLS).

You can confirm you have the correct line by comparing the HEX-STRING-HERE with your dovecot cert's SHA256 fingerprint:

openssl x509 -sha256 -in /etc/pki/dovecot/certs/dovecot.pem -noout -fingerprint

AlmaLinux vs owfs

The Zeroconf interface in owserver will dump core. So you have to compile without zeroconf:

git clone --depth=1 https://github.com/owfs/owfs.git
cd owfs
./configure --prefix=/opt/owfs-v3.2p4/ --disable-avahi --disable-zero
make -j3
sudo make install
sudo joe /usr/lib/systemd/system/owserver.service # configure your 1-wire master
sudo systemctl daemon-reload
sudo systemctl start owserver
sudo systemctl status -l owserver

AI genius part 2

Another REST server generated by AI:

use strict;
use warnings;
use HTTP::Server;
use JSON;
my $port = 8080;
my $server = HTTP::Server->new(
    Host => 'localhost',
    Port => $port,
);
$server->add_handler('/', sub {
    my $request = shift;
    my $method = $request->method();
    if ($method eq 'GET') {
        # GET request
        my $data = get_data();
        return JSON::encode($data);
    } elsif ($method eq 'POST') {
        # POST request
        my $content_type = $request->header('Content-Type');
        if ($content_type =~ /application\/json/) {
            my $body = $request->body();
            my $data = JSON::decode($body);
            process_data($data);
            return 'Data processed successfully';
        } else {
            return 'Invalid content type';
        }
    } else {
        return 'Invalid method';
    }
});
$server->start;
sub get_data {
    # Return some sample data
    my $data = {
        name => 'John Doe',
        age => 30,
        email => 'johndoe@example.com',
    };
    return $data;
}
sub process_data {
    # Process the data and do something with it
    my $data = shift;
    print "Received data: ", JSON::encode($data), "\n";
}

It suffers from the slight problem that HTTP::Server doesn't exist.

AI genis

This is some AI generating a Perl preforking HTTP server.

use IO::Prefork;
use LWP::UserAgent;
use Time::HiRes qw(gettimeofday);
use Socket;

my $prefork = IO::Prefork->new();
$prefork->listen(4000, '127.0.0.1');

sub handle_request {
    my ($req, $env) = @_;
    my $now = gettimeofday();
    print STDERR "Request received: $req\n";
    
    # fork a new process for this request
    my $pid = fork();
    if ($pid == 0) {
        # child process, handle the request
        eval {
            $prefork->handler($req, $env);
        };
        if (WANTED_EXIT_CODE) {
            exit(WANTED_EXIT_CODE);
        }
    } else {
        # parent process, wait for child to finish and handle its result
        my $child_exit_code = wait();
        die "Child process did not exit cleanly: $child_exit_code" if ($child_exit_code != 0);
        
        # send the result of the request back to the client
        my $response = $prefork->result;
        print STDERR "Sending response back to client\n";
        print $response;
    }
    
    # log the time it took to handle this request
    my $elapsed_time = $now[3] - $now[2];
    print STDERR "Request processed in $elapsed_time seconds\n";
}

$prefork->process({ 'id' => 1, 'cmd' => sub { exec("/usr/bin/php-fpm")->out(0); } });

First problem: IO::Prefork doesn't exist. And that functionality wouldn't go in the IO:: namespace anyway.

Second problem: It seems to want to do the forking in handle_request(), which would imply it's forking during each request, not before requests, as asked. I say "seems" because handle_request() isn't used anywhere, anyway.

The cherry on top is the last line, where it just execs php-fpm anyway.

Stochastication

Stochastication: n. The process a LLM uses to stitch shit together and spit out an hallucination. Like masturbation, only statistical. (Courtesy of ology on IRC)

mecab-devel, where are you?

To compile MySQL from srpm on AlmaLinux8, you need mecab-devel, which doesn't seem to exist. After some digging around, this is the solution I found :

sudo yum --enablerepo=powertools group install "Development Tools"
sudo yum install make gcc-c++ rpmbuild

mkdir -pv ~/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
cd ~/rpmbuild/SOURCES
wget 'https://drive.google.com/uc?export=download&id=0B4y35FiV1wh7cENtOXlicTFaRUE' -O mecab-0.996.tar.gz
cd ~/rpmbuild/SPECS
wget https://git.almalinux.org/rpms/mecab/raw/branch/c8-stream-8.0/SPECS/mecab.spec

rpmbuild -ba mecab.spec

cd ~/rpmbuild/RPMS/x86_64/
sudo yum install mecab*.rpm

This isn't perfect. Why would someone host their code on Google drive? But it seems this is what the author wanted.

Scammers

1-450-886-7026 are scammers.

Attribute::Handlers vs Exporter

In Perl, how does one export an attribute handler?

Do the following:

package MyAttrDecl;
use strict;
use warnings;

require Exporter;
our @EXPORT = qw( _ATTR_CODE_MyAttr );
our @ISA = qw( Exporter );

  
sub MyAttr :ATTR(CODE) 
{
    my ($package, $symbol, $referent, $attr, $data, $phase, $filename, $linenum) = @_;
    my $name = *{$symbol}{NAME};
    warn "Adding an attribute to package=$package name=$name";
}

1;

The important part is _ATTR_CODE_MyAttr. Obviously you change this to match your code.

The above is called with the following:

use MyAttrDecl;

sub new :MyAttr( "/pos/v1" )
{
    return bless {}, shift;
}

virsh vs old VMs

So I upgraded my VM server. Fresh new NVMe drives in RAID1 with AlmaLinux 8 on them. I then copied all my VMs over, updated their machines to pc or q35 as needed and launched them to test them. Most booted up fine. Three of them failed with the following error.

qemu-kvm: block/io.c:1438: bdrv_aligned_preadv: Assertion `(offset & (align - 1)) == 0' failed.

Fortunately they weren't important VMs so I could wait until today to fix the problem. The solution was to use qemu-img to convert the rewrite the disk images.

cd /kvm/VM/pool
mkdir bad
mv vda.img bad
qemu-img convert bad/vda.img vda.img -p -O qcow2

That should work for most people. Note that I assume your images are qcow2 (which they should be). If you had some raw images, you could change qcow2 to raw above. Or change types='raw' to type='qcow2' via virsh edit VM.

Here we verify the new images.

modprobe nbd max_part=8
qemu-nbd --connect=/dev/nbd0 /kvm/VM/pool/vda.img
# disconcertingly, Alma will autoactivate any VGs on the image.  Otherwise we'd do
partx -a /dev/nbd0
pvscan /dev/nbd0p2
vgchange -ay VGNAME
# now things should be available
fsck -f /dev/nbd0p1
fsck -f /dev/mapper/VGNAME-lvname
vgchange -an VGNAME
qemu-nbd --disconnect=/dev/nbd0

My image had /boot as partition 1 and / as a LV in in partition2.

If you're like me, you run libvirt on a headless server and look at VM consoles with virt-viewer. You also probably see the following warnings:

Gtk-Message: 14:08:40.348: Failed to load module "canberra-gtk-module"
Gtk-Message: 14:08:40.348: Failed to load module "pk-gtk-module"
Gtk-Message: 14:08:40.477: Failed to load module "canberra-gtk-module"
Gtk-Message: 14:08:40.477: Failed to load module "pk-gtk-module"

The solution is as follows

yum install PackageKit-gtk3-module libcanberra-gtk3

Lighttpd vs Let's Encrypt

If you are getting SSL_ERROR_NO_CYPHER_OVERLAP error with lighttpd and an SSL certificate issued by Let's Encrypt, make sure you are using the latest version of lighttpd, openssl and have your root certs up-to-date.

yum --enable-repo=epel update lighttpd openssl openssl-devel openssl-libs openssl-static ca-certificates

CentOS 6 vs CPAN and Let's Encrypt

Here is the magic to get CPAN CLI to work with https.

# cpan

cpan[1]> o conf urllist https://www.perl.com/CPAN
Please use 'o conf commit' to make the config permanent!

cpan[2]> o conf urllist                                 
    urllist           
        0 [https://www.perl.com/CPAN]
Type 'o conf' to view all configuration items

cpan[3]> o conf commit
commit: wrote '/usr/share/perl5/CPAN/Config.pm'

If it is giving you problems with SSL certificat verification, then you have to upgrade openssl, ca-certificate to the latest version. Perl also maintains it's own SSL certificates in Mozilla::CA, so you might need to do

SSL_CERT_FILE=/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem cpan Mozilla::CA

CentOS 6 vs the world

If, like me, you are a fool and still have CentOS 6 installs you have to maintain, you might run into the following problem:

https://vault.centos.org/6.10/os/x86_64/repodata/repomd.xml: [Errno 14] problem making ssl connection
Trying other mirror.
https://vault.centos.org/6.10/extras/x86_64/repodata/repomd.xml: [Errno 14] problem making ssl connection
Trying other mirror.
https://vault.centos.org/6.10/updates/x86_64/repodata/repomd.xml: [Errno 14] problem making ssl connection
Trying other mirror.

The solution is to update curl and yum by hand:

wget https://vault.centos.org/6.10/os/x86_64/Packages/python-urlgrabber-3.9.1-11.el6.noarch.rpm
wget https://vault.centos.org/6.10/updates/x86_64/Packages/yum-3.2.29-81.el6.centos.0.1.noarch.rpm
wget https://vault.centos.org/6.10/updates/x86_64/Packages/curl-7.19.7-54.el6_10.x86_64.rpm
wget https://vault.centos.org/6.10/updates/x86_64/Packages/libcurl-7.19.7-54.el6_10.x86_64.rpm
sudo rpm -Uvh libcurl-7.19.7-54.el6_10.x86_64.rpm curl-7.19.7-54.el6_10.x86_64.rpm yum-3.2.29-81.el6.centos.0.1.noarch.rpm python-urlgrabber-3.9.1-11.el6.noarch.rpm

Sendmail smart relay with TLS and plain auth

Instructions on how I set up sendmail smart relay with TLS and plain authenetication on CentOS 6.

First, make sure you have enough installed :

yum -y install ca-certificates sendmail sendmail-cf

Create /etc/mail/authinfo:

AuthInfo:YOUR.HOST.COM    "U:YOUR-USER@YOUR.HOST.COM" "I:YOUR-USER" "P:YOUR-PASSWORD" "M:LOGIN PLAIN"

Replace YOUR.HOST.COM, YOUR-USER and YOUR-PASSWORD with the correct stuff. LOGIN PLAIN stays as-is if you are using plaintext logins. Make sure to chmod 0600 this file.

Add the following to /etc/mail/sendmail.mc, making sure you use m4's dumbass `quotation' style

define(`SMART_HOST', `YOUR.HOST.COM')dnl
define(`RELAY_MAILER',`esmtp')dnl
define(`RELAY_MAILER_ARGS', `TCP $h 587')dnl
FEATURE(`authinfo')dnl
define(`confCACERT_PATH', `/etc/pki/tls/certs')dnl
define(`confCACERT', `/etc/pki/tls/certs/ca-bundle.crt')dnl

Note that the above is TCP port 587, which you might need to change.

Finally you restart sendmail and test as you normally woudl.

chmod 0600 /etc/mail/authinfo
service sendmail restart
echo "Testing" | mail -s "Test 1" somebody@example.com
tail -F /var/log/maillog

sshd vs selinux

If you're like me and want to use autossh, you'll ideally want to use ~sshd/.ssh/authorized_keys. This doesn't work out of the box on CentOS 7 at least. SELinux prevents sshd from reading its own authorized_keys. The following fixes it.

semanage fcontext --add -t ssh_home_t '/var/empty/sshd/.ssh(/.*)?'
restorecon -vFR .ssh/

Example error message:

setroubleshoot[58151]: SELinux is preventing /usr/sbin/sshd from read access on the file authorized_keys. For complete SELinux messages run: sealert -l 5a9832e8-d7c0-4e5d-af15-a977db1232e9
SELinux is preventing /usr/sbin/sshd from read access on the file authorized_keys.#012#012*****  Plugin catchall_labels (83.8 confidence) suggests   *******************#012#012If you want to allow sshd to have read access on the authorized_keys file#012Then you need to change the label on authorized_keys#012Do#012# semanage fcontext -a -t FILE_TYPE 'authorized_keys'#012where FILE_TYPE is one of the following: [huge list goes here]

Automatic backups in Windows

The following powershell script copies all .docx files from $global:SRC to $global:DEST with a timestamp.

Save this file to something like "StartClone.ps1". Then launch it with "[right click] > Run with PowerShell." Minimize the resulting window.

### Configuration - CUSTOMIZE THESE
$global:SRC = "C:\Users\fil\test"
$global:DEST = "C:\Users\fil\backup"
$global:LOGFILE = "C:\Users\fil\log.txt"


### SET FOLDER TO WATCH + FILES TO WATCH + SUBFOLDERS YES/NO
$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = $global:SRC
$watcher.Filter = "*.docx"
$watcher.IncludeSubdirectories = $true
$watcher.EnableRaisingEvents = $true  

$rv=Test-Path "$global:DEST"
if ($rv -eq $False) {
    New-Item -ItemType "directory" -Path "$global:DEST"
}

function global:logging ($text) {
    $logline = "$(Get-Date -uformat "%Y/%m/%d %H:%M:%S") - $text"
    Add-content $global:LOGFILE -value $logline
}

### DEFINE ACTIONS AFTER AN EVENT IS DETECTED
$action = { 
    $path = $Event.SourceEventArgs.FullPath
    $changeType = $Event.SourceEventArgs.ChangeType
    global:logging "$changeType $path"    
    $file = Split-Path $path -leaf
    $now = "$(Get-Date -uformat "%Y-%m-%d %H.%M.%S")"
    $dest = "$global:DEST\$now $file"
    # global:logging "file=$file now=$now dest=$dest"
    copy-item "$path" "$dest"
}    
### DECIDE WHICH EVENTS SHOULD BE WATCHED 
echo "Hello world $global:LOGFILE"
Register-ObjectEvent $watcher "Created" -Action $action
Register-ObjectEvent $watcher "Changed" -Action $action
# Register-ObjectEvent $watcher "Deleted" -Action $action
# Register-ObjectEvent $watcher "Renamed" -Action $action
global:logging Started
while ($true) {sleep 60}

Laser printer vs UPS

As an extra wrinkle to the previous post, the new Lexmark is on an UPS. Yes, I can hear you screaming "NEVER PUT A LASER PRINTER ON AN UPS" but I'm a professional, I know what I'm doing. Specifically, I over-engineered my solution.

The problem with a laser printer on a UPS is you might be tempted to print something during a black out. And your UPSes battery will never supply the current your fuser needs. My solution is to build a plug with relays connected to an arduino connected to a computer connected to the UPS. When power goes out, apcupsd on the computer runs a small script that sends a command to the arduino via USB to turn off the relays. The arduino also has a momentary switch. When I push the switch, the arduino turns the relays on. I also have small script that sends the command to turn the relays on.

I probably could have avoided the arduino and used a transistor latch. Power off turns the latch and relays off. Only a button press turns the latch and relays on. But then I couldn't do the following.

One annoying """feature""" of the Lexmark is that it has AirPrint and Wi-fi. I have yet to find out how to turn these off. So the printer spends most of its time turned off. So I wrote a CUPS backend that checks if the printer is on, sends the arduino the "turn on" command if it isn't, then chains to the normal socket backend to actually send the data to the printer.

#!/bin/bash

HWEL=10.0.0.68
LOG=/tmp/hwel-driver.log

if [[ $# == 1 ]] ; then
    exec /usr/lib/cups/backend/socket "$@"
fi

function aping () {
    local host=$1
    if ping -c 1 -q $host >/dev/null ; then
        return 0
    fi
    return 1
}

function ping_wait () {
    local host=$1
    while true ; do
        if ping -c 1 -q $host >/dev/null ; then
            return
        fi
        sleep 2
    done
}


status=$(/sbin/apcaccess status localhost:3552 | grep STATUS | cut -d: -f 2)
if [[ $status =~ ONLINE ]] ; then
    echo "INFO: hwel battery status=$status" | tee -a $LOG >&2
else
    echo "ERROR: hwel battery status=$status" | tee -a $LOG >&2
    exit 17
fi

aping $HWEL || sudo /home/fil/bin/hwel-on
ping_wait $HWEL

export DEVICE_URI=socket://$HWEL:9100
exec /usr/lib/cups/backend/socket "$@"

I put this script in /usr/lib/cups/backend/hwel and tell CUPS to use a new URL.

lpadmin -p hwelraw -v hwel://hwel.localdomain:9100

Yes, my printer is called Hwel. Yes, I name all my printers after Discworld dwarfs.

Windows 10 vs Lexmark

So I have a new Lexmark MB2236adw laser printer. For some reason beyond my ken, Windows 10 keeps saying it's offline, when clearly it isn't. Printer works fine from Linux, so I decided that Windows is going to print via CUPS.

First, create a raw printer on your CUPS host:

lpadmin -p hwelraw -v socket://hwel.localdomain:9100 -E \
     -D "Lexmark on UPS (raw)" -L "Philip's Office" -o raw

Then add the following to cups.conf to allow printing via the network:

<Location /printers>
    Order allow,deny
    Allow localhost
    Allow 10.0.0.*
</Location>

Remember to restart CUPS

service cups reload

Open up the port 631 in iptables.

Now on the windows computer, install the Lexmark drivers.

Then you have to turn on IPP : Control Panel > Programs > Turn Windows Features On or Off > Print and Document Services > Internet Printing Client must be checked.

Now it's time to Add a printer, using the The printer that I want isn’t listed button. You are going to be using an http URL, that will look like http://scott/pritners/hwelraw. Replace scott with your Linux computer's name and hwelraw with the CUPS queue name. Select the printer driver installed previously, print your test page and bask in the glory of getting a computer to do your bidding.

As a bonus step, print a 2 page document to make sure Windows can print double sided because my new printer does cool things like that.