[NCLUG] RAID array not started on re-boot

[Stephen Warren]

On 09/09/2013 12:21 PM, Kevin Olson wrote:
> Greetings!
> 
> At work, we are suffering from a strange issue, and I am hoping the
> collective wisdom of the group can provide insight.
> 
> We are running a box with CentOS 6.4, fully updated (well, perhaps minus
> anything in the last week). This machine has two software RAID arrays
> created with mdadm. One is a RAID1, and one is RAID0. In the normal course
> of events, the RAID1 runs on /dev/md1 and the RAID0 on /dev/md2. The UUID
> of each RAID is in /etc/fstab, and mount works when the devices are running.
> 
> Due to a strange combination of effects, in the past two weeks the machine
> has twice lost power. In each instance, when power was restored to the
> machine, the RAID0 was properly built with its two devices, but for some
> reason the RAID1 was not created, and instead each of the two disks were
> made into independent (though running in degraded mode) RAIDs arrays.
> 
> [root at dta ~]$ cat /proc/mdstat
> Personalities : [raid1] [raid0]
> md126  : active (auto-read-only) raid1 sdb[0]
>      976551872 blocks [2/1] [U_]
> 
> md127 : active (auto-read-only) raid1 sda[0]
>     976551872 blocks [2/1] [U_]
> 
> md2 : active raid0 sde1[0] sdf1[1]
>    1953518592 blocks super 1.2 512k chunks
> 
> unused devices: <none>
...
> My questions are:
> * Why upon the restarting of the machine was /dev/md1 not properly created?

Are the partition types still set to Linux RAID auto-detect (0xfd)?

Did the RAID super-blocks get corrupted? Try running the following to
make sure that the UUIDs in the super-blocks match (although I don't
know why the --assemble command would work later if they are):

# sudo mdadm --misc --examine /dev/sda
# sudo mdadm --misc --examine /dev/sdb

> * Why did the system decide to create /dev/md126 and /dev/md127?

I guess it thinks they aren't part of the same RAID array any more?

Perhaps try rebuilding your initrd in case that got corrupt. It seems
unlikely, but you never know.