Linux Raid Server

 With my Raspberry Pi project I came in connection to Linux (Ubuntu) again after years of being a Windows nerd. I have tons of data stored on a 3TB Western Digital Red driven Raid 1 Windows system and wanted to migrate it to a Ubuntu Linux LTS system. Why destroying a working backup system and investing time in something new? I was curious, had thirst for a new technological adventure and wanted to learn something new about Linux itself.

My current setup:

  • HP Proliant Microserver N40L (powerfull, cheap and small home server)
  • 2 x 3TB Western Digital Red HDD
  • Windows Server Essentials 2012 (student version)
  • Raid with ReFS

The last point was the critical one because Linux can’t work with the ReFS filesystem and I wanted to migrate my data to an EXT4 one. So days of copying started :-|.

Windows:ReFS >> Windows:NTFS >> Linux:EXT4 >> Linux:Raid

I resolved my existing Raid 1 in Windows, formated the second HDD to NFTS and copied all data. The I used Linux to create the Ext4 out of the ReFS HDD and copied all data from the NTFS to the Ext4. The NTFS has than been formated to a Raid partition and copying from Ext4 to Raid started again. Isn’t that confusing? Thankfully I had to HDD for shifting around data.

Now to the dangerous and detailed part.

Attention: always make backups. If you are loosing data it's your own fault.

Linux Tools

First we have to install some tools:

>> apt-get install mdadm (Raid tool)
>> apt-get install gdisk (partition tool for HDD > 2TB, fdisk can't manage this)

Preparing the new Ext4 partition

I use one NTFS HDD and one EXT4 HDD. The new one will be created with these commands:

>> gdisk -l (get list of devices, notice the total sector size of your new Ext4 partition)
>> gdisk /dev/sda (name of our new ext4 partition)
Command (m for help): d (follow wizard, delete old partition)
Command (m for help): n (follow wizard, create new partition, but select a 20MB minor sector size: total_sectors - 20*1024*sector_size, in my case 20*1024*512 because if one of your Raid HDD gets broken the new one could have another sector size and Raid syncing could get problems)
Command (m for help): w (write partition table)
>> mkfs.ext4 /dev/sda1 (create Ext4 filesystem)

Copying data

Now we can copy our data from NTFS to EXT4.

From NTFS to Ext4 I had the best experiences with the simple copy command tool.

>> cp -ruv /mnt/NTFS/*  /mnt/EXT4/ (recursive, update, verbose)

I also tried the rsync command – but after an interruption it started the whole process new instead of updating only missing files on the new Ext4 partition. One reason could be that the synchronization process has problems in comparing NTFS file attributes with Ext4 ones which is an important criteria of validating data that has to be copied. The cp command has simpler checks and copies only missing data.

Now to the most important task: the RAID!

Prepare HDD

Attention: now all data should be located on the Ext4 HDD, so we can delete data on our NTFS HDD to create the Raid setup.

>> gdisk -l (get list of devices)
>> gdisk /dev/sdb (name of our NTFS partition)
Command (m for help): d (follow wizard, delete old NTFS partition)
Command (m for help): n (follow wizard, create new partition, select same sector size as the first created Ext4 HDD)
Command (m for help): t (select 1 and fd00)
Command (m for help): w (write partition table)

Raid 1 configuration

The plan: create Raid 1 with the new NTFS-to-Ext4 created partition in combination with one missing device (our full of data Ext4 partition). After that copy data from Ext4 to Raid using again the copy command.

>> mdadm -C /dev/md0 --force --level=1 --raid-devices=2 /dev/sdb1 missing
>> mkdir /mnt/raid
>> mkfs.ext4 /mnt/raid (create Ext4 filesystem)
>> mount /dev/md0 /mnt/raid
>> cp -ruv /mnt/ext4/* /mnt/raid (copy data - recursive, update, verbose)

Next we do have to transform our Ext4-data haven to Raid.

Add HDD to Raid

>> gdisk -l (get list of devices)
>> gdisk /dev/sda (name of our new Raid data partition)
Command (m for help): d (follow wizard, delete old partition)
Command (m for help): n (follow wizard, create new partition, Raid sector limits of HDDs should be the same)
Command (m for help): t (select 1 and fd00)
Command (m for help): w (write partition table)

Now we add our just created Raid partition to the existing Raid (the missing HDD will be replace by our new one).

>> mdadm /dev/md0 --add /dev/sda1 (new HDD added, syncing starts)
>> mdadm --detail /dev/md0 (syncing details)

To resolve the device /dev/md0 after a restart we have still some work to do:

>> mdadm --detail
ARRAY /dev/md0 level=raid1 num-devices=2 metadata=00.90 UUID=*.*.*.*

Add this output to mdadm.conf and save it:

>> nano /etc/mdadm/mdadm.conf

For mounting /mnt/raid at system start you have to add your device to /etc/fstab.

Remove HDD from Raid

>> mdadm /dev/md0 --set-faulty /dev/sda1 (set faulty bit)
>> mdadm /dev/md0 -r /dev/sda1 (remove HDD)
>> mdadm --zero-superblock /dev/sda1 (remove Raid partition detection)

Resolve Raid

>> mdadm --stop /dev/md0
>> mdadm --zero-superblock /dev/sda1 (remove Raid partition detection)
>> mdadm --zero-superblock /dev/sdb1 (remove Raid partition detection)

Leave a Reply