6

can zfs do the following?

10 x 3TB disks as 5 mirrored vdevs plus 2 x 2TB as a mirror vdev.

I am looking to use those two 2TB disks that I have for a total of 17TB, and then upgrade down the track.

For example, could I later upgrade those 2x 2TB disks to 2x 6TB?

jakethedog
  • 786

1 Answers1

6

10 x 3TB disks as 5 mirrored vdevs plus 2 x 2TB as a mirror vdev.

You have the following options:

  1. 1 pool (12 x 2), usable space (6 x 2) = 12 TB
  2. 1 pool with (10 x 3) + (2 x 2), usable space (5 x 3) + (1 x 2) = 17 TB
  3. 2 pools with (10 x 3) and (2 x 2), usable space 15 and 2 TB separately

Option 1 seems to be inferior to 2, but if you replace the two small disks and have autoexpand = on set on the pool, it will automatically grow to (12 x 3), resulting in 18 TB, vs. 15 + 3 TB usable for option 2. The size will be the same, but your pool will be faster, more balanced and more fault-tolerant than option 2.

For example, could I later upgrade those 2x 2TB disks to 2x 6TB?

Yes, you could. That would give you 15 + 6 TB = 21 TB, but it would be slower than an equal configuration.

  • You can upgrade each disk of each vdev separately - the whole vdev always has the size of the smallest disk in it. For mirrors this means that both disks must be upgraded before additional space can be used.

  • You can also add and remove disks to and from mirror vdevs, but this only increases/decreases reliability and fault-tolerance, usable space is not changed.

  • Finally, you can always add more vdevs of any type and size to your pool (although similar sizes/types are recommended for performance reasons), so your pool size will grow (span, essentially RAID 0). Be careful though, as you cannot remove vdevs after adding them without destroying the pool. Your pool will fail if any vdev inside fails critically, along with all data in it.

user121391
  • 1,393
  • good answer. can you point me to a source for "Finally, you can always add more vdevs of any type and size to your pool (although similar sizes/types are recommended for performance reasons)" or further explanation of what and how performance is affected? – jakethedog Dec 21 '16 at 20:48
  • 1
    @jakethedog Thank you. I would recommend http://constantin.glez.de/blog/2010/06/closer-look-zfs-vdevs-and-performance for general reading, as well as http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#Storage_Pool_Performance_Considerations Essentially, as all vdevs are accessed in round robin fashion, so the slowest vdev sets the performance. If all are the same, this problem does not occur, because they are all equally slow/fast. – user121391 Dec 22 '16 at 09:49
  • I would personally recommend against autoexpand, if only because I'm a control freak who wants control over what ZFS does (especially given that you can never shrink a vdev after it has been expanded). Better to just zpool online -e when needed and wanted. // cc @jakethedog – user Dec 27 '16 at 12:39
  • @user121391 have you observed significant performance impact if a zfs pool have combination of striped raidz and mirror vdevs ? is iops limited to individual mirror vdev or raidz vdev ? for example zpool status in the link https://pastebin.com/emiHy4U2 – sherpaurgen Dec 12 '18 at 11:09
  • 2
    @satch_boogie I have not tried that because there will be a performance hit. Roughly speaking, your pool performance will be determined by your slowest vdev, in your example this would be the Z1 vdev. Also, your resilience will be determined by the weakest vdev, which is again the Z1 vdev. I would suggest to backup and recreate the pool with 3 mirror vdevs and set the last disk as a hotspare. – user121391 Dec 12 '18 at 12:14
  • @user121391 You mentioned in the last bullet "Finally, you can always add more vdevs of any type and size to your pool ... so your pool size will grow". If the pool is reaching storage capacity and additional vdevs of the same size are added to grow the pool (say you go from 4x 2-way mirror to 6x 2-way mirror), won't any new data striped across the pool mostly (or only) be striped to the new vdevs? And, as a result decrease I/O performance? – Firedan1176 Aug 23 '22 at 14:41