Sphinx and Mysql 5.7

So I need to move my big application from CentOS 6 to CentOS 8. First problem - systemd. Unfortunately there's no way to remove systemd, and I'm not sure I want to move to Slackware. Second problem - mysql 5.6 won't compile with a recent GNU C++ compiler ! OK, I'll move to 5.7. Wait, SphinxSE won't compile with 5.7. It looks like Sphinx has been abandoned, but it's successor Manticore hasn't moved SphinxSE to 5.7 either. It turns out that 5.7 should have been called 6.0. Maybe this is why the next version is called 8.0?

Fortunately some Googling allowed me to find someone who did the hard work. I cleaned up compiler warnings and updated INSTALL. Source is here.

Arduino Nano sizeof()

In case you were wondering, here is the output of sizeof() and other things for an Arduino Nano. What's interesting (to me) is that while the Atmega328p is an 8-bit MCU, sizeof(int)==2 and sizeof(char *)==2. In other words, the compiler is treating it as a 16-bit processor.

Welcome to sizeof!
sizeof(bool)=1
sizeof(char)=1
sizeof(int)=2
sizeof(short)=2
sizeof(long int)=4
sizeof(long long int)=8
sizeof(char*)=2
sizeof(String)=6
sizeof(int8_t)=1
sizeof(uint8_t)=1
sizeof(int16_t)=2
sizeof(uint16_t)=2
sizeof(int32_t)=4
sizeof(uint32_t)=4
sizeof(int64_t)=8
sizeof(uint64_t)=8
__INT_MAX__=7FFF
__WINT_MAX__=FFFF
SHRT_MAX=7FFF
USHRT_MAX=FFFF
INT_MAX=7FFF
UINT_MAX=FFFF
LONG_MAX=7FFFFFFF
ULONG_MAX=FFFFFFFF

Here's the code that produces the above.

#include <avr/power.h>
#include <avr/sleep.h>
#include <limits.h>

#define SO( i ) Serial.print( "sizeof(" #i ")=" ); Serial.println( sizeof(i) )
#define Max( i ) Serial.print(  #i "=" ); Serial.println( i, HEX )

void setup(void)
{
    Serial.begin( 9600 );
    Serial.println( "Welcome to sizeof!" );
    
    SO( bool );
    SO( char );
    SO( int );
    SO( short );
    SO( long int );
    SO( long long int );
    SO( char* );
    SO( String );

    SO( int8_t );
    SO( uint8_t );
    SO( int16_t );
    SO( uint16_t );
    SO( int32_t );
    SO( uint32_t );
    SO( int64_t );
    SO( uint64_t );

    Max( __INT_MAX__ );
    Max( __WINT_MAX__ );
    Max( SHRT_MAX );
    Max( USHRT_MAX );
    Max( INT_MAX );
    Max( UINT_MAX );
    Max( LONG_MAX );
    Max( ULONG_MAX );
#ifdef LLONG_MAX
    Max( LLONG_MAX );
#endif
#ifdef ULLONG_MAX
    Max( ULLONG_MAX );
#endif
/*    Max( LONG_LONG_MAX );
    Max( ULONG_LONG_MAX ); */
}

void loop(void)
{

    delay(1000);    // allow serial buffer to empty
    set_sleep_mode(SLEEP_MODE_PWR_DOWN); // sleep mode is set here
    sleep_mode(); // here the device is actually put to sleep!!
}

Using a custom CA root in C#

Over the last year or so, I've been using C#. I'm surprised at how good a language it is. Yes, it suffers from MuchToMuchCamelCaseSyndrome(tm). Maybe Microsoft learned from all the mistakes of Java and avoided them. Maybe Microsoft Studio is just better then whatever that Java crap was. I did however find one problem : it's not easy to use a custom certificate authority file when using SSL. The following is the best I've put together.

using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
using System;

namespace NS
{
    public class Cls
    {
        static private X509Certificate2 CustomCA = null;
        static public Boolean ssl_verification(Object sender, X509Certificate certificate, 
                    X509Chain chain, SslPolicyErrors sslPolicyErrors) {

            custom_ca_create();
                    
            // remove this line if commercial CAs are not allowed to issue certificate for your service.
            if ((sslPolicyErrors & (SslPolicyErrors.None)) > 0) { return true; }

            // Upgrade the server cert 
            X509Certificate2 cert2 = new X509Certificate2( certificate );

            // Build a chain we'll check ourselves.
            X509Chain p_chain = new X509Chain();
            p_chain.ChainPolicy.RevocationMode = X509RevocationMode.NoCheck;
            p_chain.ChainPolicy.ExtraStore.Add( CustomCA );

            // Maybe no errors occur during this check?
            if ( p_chain.Build( cert2 ) )
                return true;

            bool OK = true;
            foreach ( X509ChainStatus sts in p_chain.ChainStatus ) {
                if( sts.Status == X509ChainStatusFlags.UntrustedRoot ) {
                    // We already know that the custom root CA isn't trusted by .Net
                    // So we make sure the root in our custom chain is the custom root CA
                    X509Certificate2 root = p_chain.ChainElements[p_chain.ChainElements.Count - 1].Certificate;
                    if( root.Thumbprint != CustomCA.Thumbprint ) {
                        OK = false;
                        break;
                    }
                }
                else if ( sts.Status != X509ChainStatusFlags.NoError ) {
                    OK = false;
                    break;
                }
            }

            return OK;
        }

        static public void quaero_ca_create()
        {
            if ( CustomCA == null ) {
                // You must include your certificate.pem fil in your resources.
                Byte[] raw = NS.Properties.Resources.Custom_CA;
                CustomCA = ca_create( raw );
            }
        }
        static public X509Certificate2 ca_create( Byte[] raw)
        { 
            X509Certificate2 cert = new X509Certificate2();
            cert.Import( raw );
            return cert;
        }
    }
}

The above code is used with the following:

    System.Net.ServicePointManager.ServerCertificateValidationCallback = NS.Cls.ssl_verification;

As a comparison, this is how I did the same thing in Perl:

    my $ua = LWP::UserAgent->new;

    $ua->ssl_opts( SSL_ca_file => "/home/dw/prive/dw-app/SSL/Custom.CA.pem" );
    $ua->ssl_opts( verify_hostname => 1 );

The following posts helped me figure this out : https://stackoverflow.com/questions/33627593/c-sharp-net-how-to-allow-a-custom-root-ca-for-https-in-my-application-on, https://stackoverflow.com/questions/9508388/how-to-add-a-trusted-ca-certificate-not-a-client-certificate-to-httpwebrequest and https://social.msdn.microsoft.com/Forums/vstudio/en-US/1966a6e8-b6f4-44d1-9102-ec3a26426789/how-can-i-verify-a-certificate-manually-without-installing-its-parents?forum=clr.

intel drivers on CentOS 6

Furthermore, to get X working nicely on a Shuttle DX30 with CentOS 6, you have to install a newer intel driver:

yum install libpciaccess-devel.x86_64 xorg-x11-server-devel.x86_64 libXfont-devel.x86_64 libXfont2-devel.x86_64
git clone git://anongit.freedesktop.org/xorg/driver/xf86-video-intel
cd xf86-video-intel
git checkout d39197bb10b7d88cb4c456e7a5e8d34c1dc6eeaf
autoreconf -i
./configure
make
sudo mv /usr/lib64/xorg/modules/drivers/intel_drv.so /usr/lib64/xorg/modules/drivers/intel_drv.so.ORIG
sudo rsync -a ./src/.libs/intel_drv.so /usr/lib64/xorg/modules/drivers/intel_drv.so

(Copied from https://www.centos.org/forums/viewtopic.php?t=64780)

CPUs are computers

So I wanted to get a Shuttle DX30 working under LTSP. First step, it uses a pxe-client-id that's 9 bytes long. CentOS's dhcpd throws away anything that doesn't have a 17 byte pxe-client-id. I have no idea why. I tried poking around the RFCs but didn't find much of interest.

Once I patched dhcped, PXE booting worked and I managed to load a kernel. Except it was slow.

Very.

Very.

Very.

Slow.

Just how slow? It took over 30 minutes to get to a XDM prompt slow.

I saw Machine check events going past. To track those down, I put my laptop's 2.5 inch HDD into it. This was also slow. While not as bad, it was clearly not working properly. But I got mcelog to grab the following:

mcelog: failed to prefill DIMM database from DMI data
mcelog: mcelog read: No such device
Hardware event. This is not a software error.
MCE 0
CPU 0 BANK 4 
ADDR fef5d200 
TIME 1531800239 Tue Jul 17 00:03:59 2018
MCG status:
MCi status:
Error overflow
Uncorrected error
MCi_ADDR register valid
Processor context corrupt
MCA: Internal unclassified error: 408
Running trigger `unknown-error-trigger'
STATUS e600000000020408 MCGSTATUS 0
MCGCAP c07 APICID 0 SOCKETID 0 
CPUID Vendor Intel Family 6 Model 92

After some messing around, cursing the Gods, discussing Intel NUCs on #ltsp, general lack of sleep, I found the solution : kernel-ml

yum -y  --enablerepo=elrepo-kernel install kernel-ml
joe /etc/grub.conf # set default=0

It should be noted it took over 10 minutes for dracut to create the initrd for kernel-ml. But once I then booted into the new kernel, everything was fine. I installed kernel-lt as a test, because it needs the CPU and the disk to run smoothly and it took 2 minutes, which is annoying but expected. For reference, installing kernel-ml on my desktop takes 1.5 minutes but it has an SSD.

kernel-lt was a failure, 9m54 to install kernel-ml-4.17.5-1.el6.elrepo.x86_64. I booted back into kernel-ml, removed and reinstalled kernel-ml-4.17.5-1.el6.elrepo.x86_64 and it took 1m52. So kernel-ml is the clear winner.

Now to find out how to install kernel-ml into LTSP.

upstart after rc

I run daemontools from upstart. I ran into a problem that one of my daemons needed dbus to be working, but upstart was starting daemontools before rc had a chance to start it. The bigger problem is that CentOS 6 is only a partial conversion to upstart in that it still uses rc files for most things.

I used to have start on runlevel [345] in /etc/init/daemontools.conf but that means daemontools is started at the same time (or even before) as rc.

I tried start on started rc but that means daemontools runs after rc is launched, not after rc has finished.

The solution is a custom event :

cat <<'SH' >/etc/init.d/rc-done
#!/bin/sh
#
# rc-done Emits events when rc has finished or nearly
#
# chkconfig:   2345 99 01

[ -e /etc/sysconfig/rc-done ] && . /etc/sysconfig/rc-done

case "$1" in
    start)
 initctl emit rc-done
        ;;
    stop)
 initctl emit rc-start
        ;;
    *)
        echo $"Usage: $0 {start|stop}"
        exit 2
esac
exit $?
SH
chmod +x /etc/init.d/rc-done
chkconfig --add /etc/init.d/rc-done

Now we put start on rc-done into daemontools.conf

Of course the interesting part is initctl emit rc-done. We could just as easily have put that at end of rc.local.

gmail G3 SSL certs

The error messsage:

fetchmail: Server certificate verification error: unable to get local issuer certificate
fetchmail: This means that the root signing certificate (issued for /C=US/O=Google Trust Services/CN=Google Internet Authority G3) is not in the trusted CA certificate locations, or that c_rehash needs to be run on the certificate directory. For details, please see the documentation of --sslcertpath and --sslcertfile in the manual page.
139677826279240:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:s3_clnt.c:1178:
fetchmail: SSL connection failed.

The solution:

cd /your/sslcertpath
wget https://pki.goog/gtsr1/GTSR1.crt
wget https://pki.goog/gtsr2/GTSR2.crt
wget https://pki.goog/gtsr3/GTSR3.crt
wget https://pki.goog/gtsr4/GTSR4.crt
wget https://pki.goog/gsr2/GSR2.crt
wget https://pki.goog/gsr4/GSR4.crt
for n in *.crt ; do openssl x509 -in $n -out ${n/crt/der} -outform DER; done
for n in *.der ; do openssl x509 -in $n -inform DER -out ${n/der/pem} -outform PEM; done
c_rehash .
killall fetchmail