ZFS Installation

From GridPP Wiki
Jump to: navigation, search

More detailed information can be found in this link ---> here

The omreport/omconfig commands are special to Dell machines but there should be similar commands available for other installations.
You should have installed the ZFS for Linux kernel modules using the official repository and its documentation instead of using it from a 3rd party repository.


1) find out which disks are there to be used, IF you have a raid controller only and no HBA, otherwise continue at 3)

  omreport storage pdisk controller=0|grep -E "^ID|Capacity"


2) for the disks id found above create single raid0 IF you have a raid controller only and no HBA:

 for i in $(seq 0 11); 
 do 
   omconfig storage controller controller=NUMBER action=createvdisk raid=r0 size=max pdisk=0:0:$i; 
 done

The pdisk id needs to be adapted!


3) look if all disks are available

  lsblk |grep disk


4) find the correct disks to be put in the storage zpool (example for 2TB disks)

  lsblk |grep 1.8T


5) with the names returned by lsblk create a raidz2 or raidz3 depending on your needs

Be sure not to forget to configure the spare!

 zpool create -f POOLNAME raidz3 sdb sdc sdd sde sdf sdg sdh sdi sdj sdk sdl sdm sdab sdad sdae sdag sdah sdaj spare sdak


6) enable different features of zpool and zfs and configure it correctly

  zfs set compression=lz4 POOLNAME;
  zpool set autoreplace=on POOLNAME;
  zpool set autoexpand=on POOLNAME;
  zfs set relatime=on POOLNAME;
  zfs set xattr=sa POOLNAME;
  zpool export POOLNAME;
  zpool import -d /dev/disk/by-id POOLNAME;


7) be sure to have all needed services enabled

  • on RH/SL6 with chkconfig for
    • zfs-import
    • zfs-mount
    • zfs-share
    • zfs-zed
  • on RH7/SL7/CentOS7 with systemctl for
    • zfs-import-cache.service
    • zfs-mount.service
    • zfs-share.service
    • zfs.target
    • zfs-zed.service

It should look similar like this in the end:

[root@hv2 ~]# systemctl list-unit-files | grep zfs
zfs-import-cache.service                      enabled 
zfs-import-scan.service                       enabled 
zfs-mount.service                             enabled 
zfs-share.service                             enabled 
zfs-zed.service                               enabled 
zfs.target                                    enabled 
[root@hv2 ~]# systemctl | grep zfs
zfs-import-cache.service                                                                         loaded active exited    Import ZFS pools by cache file
zfs-mount.service                                                                                loaded active exited    Mount ZFS filesystems
zfs-share.service                                                                                loaded active exited    ZFS file system shares
zfs-zed.service                                                                                  loaded active running   ZFS Event Daemon (zed)
zfs.target                                                                                       loaded active active    ZFS startup target


8) for usage with DPM do something like:

  zfs create POOLNAME/gridstorage01  #only if you want an extra filesystem
  chown -R dpmmgr:dpmmgr /POOLNAME