[NCLUG] Installing ubuntu 12.04 to ZFS root directory

[Phil Marsh]

Hi Jesse,
Perhaps I'm crazy if my top priority were speed. However, a RAIDZ2 can withstand a simultaneous failure of any two drives in the array whereas if both drives of a mirror fail, one will lose data. Yes, a 2X4 mirror can withstand up to two simultaneous drive failures, but only if both failed drives don't belong to a single mirror pair.
Thanks for your post. It jogged me into attempting the math. It turns out that I saw interesting things when I plotted the probability of array failure (Pa) as a function of individual drive failure probability (Pi) for both the mirror (2X4) and RAIDZ2 (both are total 4 drives). Assuming all drives have equal Pi s for the arrays, for a Pi<~0.4, the Pa(RAIDZ2) < Pa(mirror). But for Pi>~0.4 the mirror was more reliable (Pa(mirror) < Pa(RAIDZ2)) than the RAIDZ2!
However, if one attends to the array daily, a Pi near 0.4 will never be reached unless the drives are VERY defective! So, for the mirror to effectively be more reliable than the RAIDZ2, the probability of an individual drive failure (Pi) would have to exceed about 40% each day!
Now, at low Pi/day as would be realistic, Pa(RAIDZ2) is about 4*Pi**3 whereas Pa(mirror) is about 2*Pi**2. I believe for comparison, a RAID5 or RAIDZ1 3-drive Pa is about 3*Pi**2, so the 2X4 mirror is more reliable than the 3-drive RAID5. All the above assume a small Pi, a 2X4 drive mirror, and a 4-drive RAIDZ2.
So, because I value data security in my /home drive more than speed, I'm going with the RAIDZ2 for /home and for speed-critical apps, I'm using my extra disks to set up a 2-drive ZFS mirror and partitions of these 2 drives to host my / using a MDADM mirror excepting /home.

I tried to host / on native ZFS (RAIDZ2) but it appears to boot horribly slowly, and got caught up looking at the floppy drive, and I suspect this isn't due to my RAIDZ2 speed but rather something else - perhaps having to do with the special  ZFS bootloader required and, perhaps mounting the ZFS during the bootloader's search for the kernel which was on the ZFS array? The /root/grub directory was on a standard drive ext4 FS.

Please comment,
Phil

Sent from my iPad

On Aug 20, 2013, at 9:00 AM, Jesse Griffin <jesse at tummy.com> wrote:

> Hello,
> 
> I think you are crazy to be using RAIDZ2 with only 4 drives!  A RAID10 setup
> would give you way better read performance and essentially the same redundancy.
> A basic RAID10 sort of mirrored pool can be created like this:
> 
>    zpool create tank0 mirror sda sdb mirror sdc sdd
> 
> Compression at the default recommended level should actually increase your
> throughput.  This is because less data is stored on disk and your CPU can
> decompress the data faster than your mechanical disk could serve it up.  Of
> course, this assumes that your data set has at least an average level of
> compressibility.
> 
> Though related the Solaris, there is lots of good information here,
> http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide, if you
> haven't come across that yet.
> 
> 
> Thank you,
> Jesse Griffin
> tummy.com, ltd.
> 
> On 08/19/2013 01:43 PM, Phil Marsh wrote:
>> Hi, 
>> I had a question for those who tried ZFS. I am running ZFS RAIDZ2 with four 1TB drives, two are Seagate Barracudas and two are WD blacks. I am booting (MBR + /boot/grub from an older 500GB WD black and otherwise have the rest of the / and /home directories on a ZFS RAIDZ2 pool. My machine has 8GB 800MHz DDR2 RAM and an older quad core Intel (I think 6400 from 2007). The transfer rate of my SATA controllers is, I think, rated at 3GB/sec. The OS is Ubuntu 12.04 (64B)
>> My issue is that I'm noticing my booting is taking about 3x longer than my old Ubuntu 10.04 configuration (booted from a single drive with RAID 5 MDADM) and the machine winds up trying to access the floppy even though it's disabled for boot in the BIOS.
>> Software installation seems to be slow too.
>> Moreover, a check on iostat shows read BW of ~300K and write BW of about 3.5M. That seems slow to me.
>> I'm beginning to wonder if I should just go to MDADM (software) RAID6 for my /home?
>> Could someone kindly comment? Any suggestions and what is your experience with ZFS? Am I crazy to be using RAIDZ2 with four disks and the / directory for the most part, in the RAIDZ2?
>> Thanks,
>> Phil
>> 
>> Sent from my iPad
>> 
>> On Aug 18, 2013, at 5:33 PM, phil marsh <montanaembassy at yahoo.com> wrote:
>> 
>>> Hi folks,
>>> It finally sunk in what Bob was saying to me during the dinner of the last NCLUG meeting.
>>> I recall him saying that one could install GRUB and the /boot directory to an SSD which would need very little space.
>>> Next, one could install / and /home to a ZFS pool while having /boot and the MBR on an SSD.
>>> I went to: 
>>> https://github.com/zfsonlinux/pkg-zfs/wiki/HOWTO-install-Ubuntu-to-a-Native-ZFS-Root-Filesystem
>>> to find out how to install the /, with the exception of /boot which would be installed on a non-zfs filesystem,  and /home directories to a ZFS pool.
>>> Apparently, GRUB and the MBR have to still be installed outside of any ZFS pool to work properly?
>>> I have installed Ubuntu 12.04 to ZFS as described above and in the link. I did notice a few small errors (or differences between my and the author's system) namely, that one should not change the default /mnt/etc/network/interfaces file but instead leave it alone - I could not connect my network card after booting to the desktop GUI with the suggested changes.
>>> 
>>> Also, after installation, I had to update (add the standard repositories to) the /etc/apt/sources.list file and the /etc/apt/source.list.d directory in order to have all the standard repositories that one would have in a normal Ubuntu 12.04 desktop install.
>>> In order to do this, I was lucky to have a box that had Ubuntu 12.04 installed already but if you need, I can provide you my 
>>> /etc/apt/sources.list file and /etc/apt/source.list.d directory
>>> as well as a list of my installed packages.
>>> 
>>> My /boot directory and MBR now occupy a whole 500GB drive which is extremely wasteful to say the least!
>>> My questions are:
>>> 1. Can I move my /boot directory and MBR to a thumb drive which I will permanently place in one of the box's USB ports? I can apparently set the BIOS to boot from the USB (as for a USB live disk). I think this is possible and not too difficult but please comment if I'm mistaken.
>>> 2. Do you think this would be a good idea and also, would it give rise to problems later? I don't think the /boot directory sees many writes but again, I could be mistaken.
>>> Thanks so much,
>>> Phil
>> _______________________________________________
>> NCLUG mailing list       NCLUG at lists.nclug.org
>> 
>> To unsubscribe, subscribe, or modify
>> your settings, go to:
>> http://lists.nclug.org/mailman/listinfo/nclug
> _______________________________________________
> NCLUG mailing list       NCLUG at lists.nclug.org
> 
> To unsubscribe, subscribe, or modify
> your settings, go to:
> http://lists.nclug.org/mailman/listinfo/nclug