Do not try this at home

What you are about to see is bad and wrong. There are many better and easier ways to do this. I'm documenting it here so you can see how hoary it is.

Say you have a system set up with RAID1 on / and /boot, 2 active, 1 spare disks. This system can function even if reduced to a single active disk. Surely one can clone the system just by rebuilding the arrays with new disks?

Short answer is "Yes".

The longer answer is "No, don't do that. Use Clonezilla."

The reason you shouldn't do it this way is that Linux RAID (aka mdadm aka dm) uses UUIDs to identify arrays. It also has a field that contains the hostname the array was created on. What's more LVM has volume group names. These need to be unique if ever these the original and clone array are going to appear on the same system. Even if they you are sure they never will, some admin software you are going to try out some day.

But can't you change the UUIDs, VGs, hostname and so on? Yes you can. I just did it for a client. It was more pain then it was worth.

The following walk-through assumes my normal setup : first partition of each disk is part of a RAID1 3 active, no spares that goes on /boot (called md1 or md126). Second partition is partition of each disk is part of a RAID1 2 active, no spares that goes on / (called md0 or md127).

DO NOT TRY THIS AT HOME. I am a professional sysadmin with years of experience fucking up working systems. The following walk-through is provided without any warranty as to applicability or suitability to a any sane or useful or safe task. Back up your data. Verify your backup. RAID is not a backup. YMMV. HTH. HAND.

1. Make sure the arrays are fully in sync.
2. Do a clean shutdown
3. Make sure you can boot from the first and second drives of the array.
4. Remove the first active and the spare drive from the computer. Label them well and set aside
5. Disconnect the second drive. This will be the first drive on the new clone
6. Boot from a LiveDVD or USB or something. You will need a distro that has mdadm, uuidgen, lvm. I used the CentOS 6.5 LiveDVD.

7. telinit 1 # single user mode
   pstree # make sure nothing unwanted is running
   killall dhclient # kill everything unwanted.  You will need udevd

8. Plug the old-second-new-first drive in and wait for things to settle
9. cat /proc/mdstat Make sure your arrays are inactive. They will have (S) to mean they need to sync with something.
10. This is the hairy bit:

    # get /boot working
    mdadm --stop /dev/md126
    mdadm --assemble --update=uuid --uuid=$(uuidgen) /dev/md126 /dev/sda1
    mdadm --stop /dev/md126
    mdadm --assemble --update=name --name=$(hostname):1 /dev/md126 /dev/sda1
    mdadm --stop /dev/md126
    mdadm --assemble /dev/md126 /dev/sda1 --run
    tune2fs -U $(uuidgen) /dev/md126
    # get / working
    mdadm --stop /dev/md127
    mdadm --assemble --update=uuid --uuid=$(uuidgen) /dev/md127 /dev/sda2
    mdadm --stop /dev/md127
    mdadm --assemble --update=name --name=$(hostname):0 /dev/md127 /dev/sda2
    mdadm --stop /dev/md127
    mdadm --assemble /dev/md127 /dev/sda2 --run
    # activate LVM on /dev/md127
    vgscan
    # rename VG
    vgrename OLDVG NEWVG
    # mount /
    vgchange -a y NEWVG
    tune2fs -U $(uuidgen) /dev/mapper/NEWVG-root
    mount /dev/mapper/NEWVG-root /mnt
    # mount /boot
    mount /dev/md1 /mnt

11. Now comes the really annoying part: You have to update /etc/fstab (CentOS 6 has the UUID of /boot array), /boot/grub/grub.conf (CentOS 6 has the UUID of / array and the VG of /) and and possibly /boot/grub/initramfs-MUTTER.img to use the new UUIDs. The really fun part (for me) is that the LiveDVD doesn't have joe. So I had to write the UUID down on a piece of paper, then write it into grub.conf.

    You can find the UUID of an array with

    mdadm --detail /dev/md0

    If you want more flexibility, do

    mount --bind /proc /mnt/proc
    mount --bind /dev /mnt/dev
    mount --bind /sys /mnt/sys
    mount --bind /tmp /mnt/tmp
    chroot /mnt

    This will allow you to run mkinitrd if you need to. Note that this assumes your live DVD has a kernel that is compatible with your Linux distro.
12. Reboot to new system. Keep your fingers crossed.
13. Now you just insert your 2 other disks and run

    sfdisk -d /dev/sda | sfdisk /dev/sdb
    sfdisk -d /dev/sda | sfdisk /dev/sdc
    mdadm --add /dev/md126 /dev/sdb1
    mdadm --add /dev/md126 /dev/sdc1
    # wait until rebuild is finished
    cat <<GRUB | grub
    device (hd0) /dev/sdb
    root (hd0,0)
    setup (hd0)
    GRUB
    cat <<GRUB | grub
    device (hd0) /dev/sdc
    root (hd0,0)
    setup (hd0)
    GRUB
    mdadm --add /dev/md127 /dev/sdb2
    mdadm --add /dev/md127 /dev/sdc2
    

14. Ask yourself - was this really worth it? Wouldn't Clonezilla have been so much easier?

That didn't seem to hard, you might be saying. What I'm omitting is that when I booted to the new array, I got a lot of checksum errors and a failed fsck. I did fsck -y /dev/md127 a bunch of times until it came up clean.

Also - how do I get my arrays back to md1 and md0? The old method (--update=super-minor) no longer works.