How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS

# How to Build a Minimal ZFS NAS without Synology, QNAP, TrueNAS Source: [https://neil.computer/notes/how-to-setup-minimal-zfs-nas-without-truenas/](https://neil.computer/notes/how-to-setup-minimal-zfs-nas-without-truenas/) ## How to Build a Minimal ZFS NAS without Synology, QNAP, TrueNAS Published on:2024\-08\-23 ZFS System by OracleIf you need a basic NAS and don't care about GUI features, it is suprisingly simple to set up a ZFS dataset and share it over the network using Samba\. #### Scope: Scope & RequirementsRaid LevelRAIDZ1 \(1 Drive Redundancy\)Operating SystemDebian 12 BookwormEncryptionNoneZFS ImplementationOpenZFS, zfs\-2\.1\.1CPU4 Cores, Xeon Server CPU can be had for cheapRAMECC RDIMM RAM 16 GBStorage4x4TB NVMe SSDBackupsNot covered, use[ZFS Backup Scheduler](https://github.com/wyager/zfs-backup?ref=neil.computer)SkillsBasic familiarity with LinuxSkill LevelBeginner/EasyI am using this article to document it for future myself, feel free to adopt it for your needs\. Problem with TrueNAS is that it is a full\-featured, supposedly enterprise\-grade, software suite\. While it may be simple to set it up \(I've never tried\), I just don't need any of the bells and whistles it offers\. It's the mismatch between what I need and what it offers; not something inherently wrong with TrueNAS\. There is also something to be said about a system you know everything about and not having to rely on yet another thing\. ## ZFS's best feature that's never explained or written anywhere ZFS filesystem is self contained\. If your OS is nuked suddently, simply take all disks to another machine or install a new OS, install`zfs`, run`zfs import`and get back your data\. This freedom is underrated and not well understood\. It is also not explained anywhere\. It's worth emphasizing: All configuration/details about ZFS is stored on the disks themselves\. If you've setup a RAIDZ2 \(Raid 6\) with 6 disks, they are self contained\. Move them to a new machine with`zfs`tools installed, and simply run`zfs import`\. Boom, they'll show up as RAIDZ2\. This is an amazing feature that no matter what happens to the host OS, machine, etc; as long as the disks are not damaged, your data is fine\. ## Step 1\. Locate and Organize Disks List all disks on a linux machine using`lsblk \-d \-o TRAN,NAME,TYPE,MODEL,SERIAL,SIZE`command\. ``` [root@sys ~]# lsblk -d -o TRAN,NAME,TYPE,MODEL,SERIAL,SIZE TRAN NAME TYPE MODEL SERIAL SIZE sda disk Virtual disk 40G nvme nvme5n1 disk Samsung SSD 990 PRO 4TB XXXXXXXXXXXXXXX 3.6T nvme nvme2n1 disk Samsung SSD 990 PRO 4TB XXXXXXXXXXXXXXX 3.6T nvme nvme6n1 disk Samsung SSD 990 PRO 4TB XXXXXXXXXXXXXXX 3.6T nvme nvme3n1 disk Samsung SSD 990 PRO 4TB XXXXXXXXXXXXXXX 3.6T ``` These are brand new NVMe drives from Samsung so they should be completely unallocated\. The disks are also mapped to an ID, running`ls \-lh /dev/disk/by\-id`: ``` [root@sys ~]# ls -lh /dev/disk/by-id total 0 lrwxrwxrwx. 1 root root 10 May 10 09:34 dm-name-rhel-root -> ../../dm-0 lrwxrwxrwx. 1 root root 10 May 10 09:34 dm-name-rhel-swap -> ../../dm-1 lrwxrwxrwx. 1 root root 10 May 10 09:34 lvm-pv-uuid-DGBpev-Na0C-tY20-YY6E-tpL3-epNA-8Ts3Y0 -> ../../sda3 lrwxrwxrwx. 1 root root 13 May 10 09:34 nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX -> ../../nvme2n1 lrwxrwxrwx. 1 root root 13 May 10 09:34 nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX -> ../../nvme5n1 lrwxrwxrwx. 1 root root 13 May 10 09:34 nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX -> ../../nvme3n1 lrwxrwxrwx. 1 root root 13 May 10 09:34 nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX -> ../../nvme6n1 ``` Notice that they are symlinked to their disk names in`/dev/`\. We can create a`/etc/zfs/vdev\_id\.conf`that maps an alias to these IDs: ``` [root@sys ~]# vim /etc/zfs/vdev_id.conf # Add these lines in the vdev_id.conf file alias nvme0 /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX alias nvme1 /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX alias nvme2 /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX alias nvme3 /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_XXXXXXXXXXXXXXX ``` Run`udevadm trigger`to set the alias \(or you can reboot the machine\)\. We can verify that the aliases have been mapped by running`ls \-lh /dev/disk/by\-vdev`: ``` lrwxrwxrwx. 1 root root 13 May 10 10:28 nvme0 -> ../../nvme2n1 lrwxrwxrwx. 1 root root 13 May 10 10:28 nvme1 -> ../../nvme5n1 lrwxrwxrwx. 1 root root 13 May 10 10:28 nvme2 -> ../../nvme3n1 lrwxrwxrwx. 1 root root 13 May 10 10:28 nvme3 -> ../../nvme6n1 ``` [Alias mapping](https://openzfs.github.io/openzfs-docs/man/v2.2/5/vdev_id.conf.5.html?ref=neil.computer)is completely optional, you can if you'd like use the full ID`/dev/disk/by\-id/nvme\-eui\.002538414143c248`of the disk when creating the zpool as we will do in the next section\. Using an alias makes it nice\. However, please don't use`/dev/nvme1, /dev/nvme2, \.\.\.`as the order is not guaranteed, especially if you mount a new drive to the system\. Creating a`vdev\_id\.conf`ensures that the serial number of the drive is tied to the alias\. Remember when we discussed there is no configuration needed on the host OS?`/etc/zfs/vdev\_id/conf`is not necessary and only used when creating a zpool for convenience\. If your OS gets nuked, and you lose`vdev\_id\.conf`, it won't matter at all\. ## Step 2\. Create ZPOOL For this tutorial, I am creating a RAIDZ1 \(RAID 5\) zpool\. That means 1 drive redundancy in\-case of failure\. It's up to you if you'd like additional redundancy, RAIDZ2 \(RAID 6\) would certainly be more risilient\. First, we need to install zfs on the linux machine\. Please refer to the[OpenZFS documentation on how to install it](https://openzfs.github.io/openzfs-docs/Getting%20Started/index.html?ref=neil.computer)\. It's usually as straight forward as, in my case,`dnf install zfs`on RHEL 9\. I recommend setting the`ashift=12`option when creating the zpool as this is your last chance to do so\. Most disks report 512kB sector size to OS due to backwards compatibility reasons, but large disks such as Samsung 990 Pro has a sector size of 4KB or even 8KB\.`ashift=12`represents a sector size of 4KB which will substantially improve performance\. ``` [root@sys ~]# ls /dev/disk/by-vdev nvme0 nvme1 nvme2 nvme3 [root@sys ~]# zpool create -o ashift=12 s16z1 raidz1 nvme0 nvme1 nvme2 nvme3 [root@sys ~]# zpool status s16z1 pool: s16z1 state: ONLINE config: NAME STATE READ WRITE CKSUM s16z1 ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 nvme0 ONLINE 0 0 0 nvme1 ONLINE 0 0 0 nvme2 ONLINE 0 0 0 nvme3 ONLINE 0 0 0 errors: No known data errors [root@sys ~]# ``` Perfect\! I chose`s16z1`as the name as it describes the size and type of raid\. Most tutorials will use`tank`as the name, as it relates to`pool`\. Corny, I reject this\. We're not done yet\.`zpool`is a disk abstraction, and`zfs`is the file system\. When we ran`zpool create`, it created a`zfs`file system with it\. List all properties of the zfs file system by running:`zfs get all s16z1`\. To check whether our zpool is properly configured with`ashift=12`, we can run: ``` [root@sys ~]# zdb | grep ashift ashift: 12 ``` Before we share the system, configure compression \(optionally\) as well as the default mount point\. ``` [root@sys ~]# zfs set mountpoint=/mnt/s16z1 s16z1 [root@sys ~]# zfs set compression=lz4 s16z1 ``` Next, let's create a couple of zfs datasets under`s16z1`root dataset\. We will share them using Samba in the next section\. ``` [root@sys ~]# zfs create s16z1/docs [root@sys ~]# zfs create s16z1/backups ``` `docs`for documents,`backups`for time machine backup\. You can create as many datasets as you'd like\. Try to keep them at the top level\. If you're wondering what is the difference between just a regular filesystem folder and a dataset—a zfs dataset is a way more than just a folder\. You can manage zillion properties of a dataset, encrypt it, send and replicate a dataset, take snapshots, etc\.—essentially, the entire ZFS feature set\. Therefore, it is a good idea to create individual datasets for large categories of your files\.`docs`or`backups`is a good abstraction level for a dataset\. If you want to send just`docs`to a another remote server as a backup, you can do that without sending the whole`s16z1`root dataset\. We will discuss sharing`s16z1/docs`as a general purpose share as well as ceating a proper Time Machine storage \(for Apple systems\) by sharing`s16z1/backups`with special properties\. We'll use Samba for this, and the type of file sharing system is orthogonal to ZFS\. ZFS doesn't care as long as it is mounted on the host system\. Install Samba: ``` [root@sys ~]# apt install samba ``` Create a UNIX user specifically for samba, we'll call it`john`: ``` [root@sys ~]# useradd -m john ``` and create a UNIX password for`john`: ``` [root@sys ~]# passwd john ``` Next, associate the UNIX user`john`to Samba by creating a Samba password for`john`\. This password will then be used by hosts connecting to the share as`SYS\\john`\. ``` [root@sys ~]# smbpasswd -a john ``` `john`is also added to the Samba user group\. You can verify`john`as a SMB user by running: ``` [root@sys ~]# pdbedit -L -v john Unix username: john NT username: Account Flags: [U ] User SID: S-1-5-21-880039843-1994218806-4034623300-1001 Primary Group SID: S-1-5-21-880039843-1994218806-4034623300-513 Full Name: john Home Directory: \\SYS\john HomeDir Drive: Logon Script: Profile Path: \\SYS\john\profile Domain: SYS Account desc: Workstations: Munged dial: Logon time: 0 Logoff time: Wed, 06 Feb 2036 08:06:39 MST Kickoff time: Wed, 06 Feb 2036 08:06:39 MST Password last set: Fri, 23 Aug 2024 23:46:08 MST Password can change: Fri, 23 Aug 2024 23:46:08 MST Password must change: never Last bad password : 0 Bad password count : 0 Logon hours : FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF ``` To delete`john`SMB user,`pdbedit \-x \-u john` Now that the UNIX user and Samba user is setup, configuring Samba service is extremely straight forward\. Edit`/etc/samba/smb\.conf`\(delete everything and replace with the following\): ``` [docs] path = /mnt/s16z1/docs browseable = yes read only = no guest ok = no valid users = john create mask = 0755 [backups] path = /mnt/s16z1/backups read only = no guest ok = no inherit acls = yes spotlight = yes fruit:aapl = yes fruit:time machine = yes vfs objects = catia fruit streams_xattr valid users = john ``` Test it out on macOS by mounting it with`cmd\+K`in Finder app and using the following url format:`smb://10\.0\.0\.6/docs`and`smb://10\.0\.0\.6/backups`where`10\.0\.0\.6`is the IP of the server sharing the SMB share\. To test it on Debian systems or similar, install`apt install smbclient`and run: ``` [root@sys ~]# smbclient -U john //10.0.0.6/docs -c 'ls' ``` Mount the`smb://10\.0\.0\.6/backups`and it will show up as a Time Machine share on macOS\. Once mounted, start Time Machine backups by adding the share to macOS \> Settings \> General \> Time Machine\. That's all for now\. I plan to write another article expanding on the encryption and very powerful zfs dataset replication features\. Cover image credit to Oracle ZFS systems\. God, it is beautiful:[https://docs\.oracle\.com/cd/E78901\_01/html/E78910/gqtmb\.html](https://docs.oracle.com/cd/E78901_01/html/E78910/gqtmb.html?ref=neil.computer)