Where ZFS Fits in Enterprise Infrastructure Source
Markdown source•
1---2title: "Where ZFS Fits in Enterprise Infrastructure"3date: "2026-09-22"4published: true5tags: ["zfs", "storage", "infrastructure", "proxmox", "linux", "backup"]6author: "Gavin Jackson"7excerpt: "ZFS from its Solaris roots to enterprise storage today: pools, hardware choices, recovery and repository replication, with a Proxmox breakout and a practical ZFS cheat sheet."8---910# Where ZFS Fits in Enterprise Infrastructure11121314*Illustration inspired by the cover of [FreeBSD Mastery: Advanced ZFS](https://www.tiltedwindmillpress.com/product/fmaz/).*1516ZFS has been on my list of things to understand properly for a long time. Coming from conventional enterprise storage, I am used to thinking in arrays, LUNs, datastores and replication jobs. ZFS puts much of that machinery within reach of the operating system and the people administering it.1718That makes it vastly different from a conventional Linux filesystem such as ext4 or XFS. Those filesystems normally sit on top of storage assembled elsewhere. ZFS reaches down into the storage layout itself, combining pools, redundancy and volume management with checksums, copy-on-write snapshots, compression, replication and recovery tooling.1920It is unquestionably enterprise-grade. Many of these are capabilities I would once have expected to buy as part of an expensive commercial storage platform. With ZFS, we would probably own that stack ourselves: disk layout, hardware selection, monitoring, capacity planning and recovery. That control is appealing, but the ownership is part of the cost.2122I am still learning it. What interests me is how those capabilities fit into ordinary infrastructure work: hosting virtual machines, storing backups, serving files and moving large collections of data between systems.2324## From Solaris to OpenZFS2526ZFS began at **Sun Microsystems in 2001**, and its source was released through OpenSolaris in 2005. It shipped in the June 2006 update to Solaris 10. The [project history](https://openzfs.org/w/index.php?title=History&mobileaction=toggle_view_desktop) and [Solaris release notes](https://docs.oracle.com/cd/E19957-01/820-2714/6nea26ql6/index.html) trace those early milestones.2728In September 2013, [OpenZFS launched](https://openzfs.org/wiki/Announcement) to bring together the open-source development work across platforms including Linux, FreeBSD and illumos. OpenZFS is the implementation discussed here; Oracle's Solaris ZFS continued along a separate development path.2930I first took notice of OpenZFS when we were running FreeBSD on production servers. At the time, ZFS felt like one of FreeBSD's big-ticket advantages over Linux. I even bought [FreeBSD Mastery: Advanced ZFS](https://www.tiltedwindmillpress.com/product/fmaz/) by Allan Jude and Michael W. Lucas. I remember thinking the concepts were seriously next-level, but never really got the opportunity to explore them in depth.3132Its commercial use now spans storage appliances, cloud services and integrated infrastructure. [TrueNAS Enterprise](https://www.truenas.com/truenas-enterprise/) builds supported storage appliances around OpenZFS. [Amazon FSx for OpenZFS](https://docs.aws.amazon.com/fsx/latest/OpenZFSGuide/what-is-fsx.html) offers it as a managed NFS file service. [Oxide](https://oxide.computer/product/storage) uses it beneath a distributed storage service with its own replication and failover design.3334## Pools, vdevs, datasets and zvols3536A conventional storage stack might have a RAID controller or array protecting disks, a volume manager allocating space, and a filesystem managing files. ZFS combines those responsibilities and manages the redundancy itself.3738| Term | Role |39| --- | --- |40| **Pool** | Shared storage capacity assembled from one or more vdevs. |41| **Vdev** | A disk or group of disks, such as a mirror or RAIDZ group. Redundancy is defined here. |42| **Filesystem dataset** | A filesystem within the pool, with its own mount point, quotas, compression and snapshots. |43| **Zvol** | A dataset exposed as a block device, suitable for a VM disk with a guest filesystem inside it. |4445A dataset gives a workload its own policies without assigning it dedicated physical disks. The [pool concepts](https://openzfs.github.io/openzfs-docs/man/v2.3/7/zpoolconcepts.7.html) and [dataset concepts](https://openzfs.github.io/openzfs-docs/man/v2.3/7/zfsconcepts.7.html) manuals cover the structure.4647### Where LVM fits4849[LVM](https://sourceware.org/lvm2/) sits between physical storage and a filesystem in a conventional Linux stack. It groups physical volumes into a volume group, then allocates logical volumes that can be formatted with ext4, XFS or another filesystem. LVM manages blocks; the filesystem above it manages files.5051| Conventional Linux stack | ZFS stack |52| --- | --- |53| Disks or RAID → LVM physical volumes → volume group → logical volumes → ext4/XFS | Disks → ZFS vdevs and pool → filesystem datasets or zvols |5455At the host level, LVM and ZFS are usually alternative storage stacks rather than layers to place on top of each other. ZFS takes over the volume-allocation role while also providing the filesystem, redundancy, checksums and snapshots. I would normally give ZFS direct access to its disks instead of putting it on an LVM logical volume, which would hide the underlying device layout behind another layer.5657They can still complement each other. A guest stored on a ZFS zvol can use LVM inside its virtual disk, and a host can use ZFS and LVM on separate storage. In Proxmox, [LVM-thin](https://raw.githubusercontent.com/proxmox/pve-docs/master/pve-storage-lvmthin.adoc) and ZFS are separate local-storage backends. LVM-thin supports efficient VM snapshots and clones, while ZFS provides the broader integrity and filesystem model described here.58596061*Two mirrored vdevs contribute capacity to one pool. Filesystem datasets and virtual disks share that capacity.*6263The pool distributes data across its normal top-level data vdevs. Losing an entire required vdev can lose the pool. In the diagram, either disk in a mirror can fail, but losing both members of the same mirror is fatal to that vdev. Adding an unprotected disk as another data vdev would introduce a single-disk failure point for the whole pool.6465ZFS uses [copy-on-write](https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Copy-on-write.html): changed blocks are written to new locations before references are updated. A snapshot preserves an earlier state without copying the whole dataset. As live data changes, the snapshot retains blocks that would otherwise be freed.6667Checksums detect damaged blocks. Repair requires a healthy additional copy or enough redundant data to reconstruct the original. A scrub can find damage on a single-disk pool, but cannot recover data for which no good copy remains. See the [scrub manual](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-scrub.8.html).6869> **What ZFS adds to Proxmox**70>71> Proxmox VE's [local ZFS backend](https://raw.githubusercontent.com/proxmox/pve-docs/master/pve-storage-zfspool.adoc) supports VM disks and container filesystems, with integrated snapshots, clones, compression and optional thin provisioning. Templates and clones can share unchanged blocks, while ZFS properties can be inherited from a parent dataset. Thin provisioning still needs capacity monitoring.72>73> Its [storage replication framework](https://raw.githubusercontent.com/proxmox/pve-docs/master/pvesr.adoc) uses snapshots to copy guest volumes to another node, then sends only changes. This can reduce migration traffic and support HA recovery using local disks instead of shared storage. Replication runs on a schedule, so a failure can lose changes since the last successful sync; quorum and HA configuration remain separate responsibilities.74>75> Guests keep their own filesystems: ext4 or NTFS inside a ZFS-backed virtual disk is perfectly ordinary. Manage Proxmox-owned disks through Proxmox, and use the ZFS tools to inspect the underlying storage.7677## Choosing a layout and hardware7879Mirrors generally suit small random I/O, including many VM workloads. RAIDZ trades some of that flexibility for capacity efficiency, especially with larger sequential workloads. RAIDZ1, RAIDZ2 and RAIDZ3 tolerate one, two and three failed disks respectively within a vdev. Performance still depends on the media, number of vdevs and workload; the [OpenZFS tuning guide](https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html) is a useful starting point.8081With equal-sized drives, a two-way mirror provides roughly half its raw capacity before overheads. An N-disk RAIDZ2 group starts with roughly N minus two drives' worth. Leave room for metadata, snapshots, growth and recovery work before promising capacity to applications.8283Expansion options include:8485- **Adding another redundant data vdev** to the pool.86- **Replacing members with larger drives.** The additional capacity requires all members of the vdev to be expanded, as described in the [online manual](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-online.8.html).87- **Widening a RAIDZ vdev on supported versions.** OpenZFS 2.3 introduced RAIDZ expansion. It retains the existing parity level, and old blocks retain their previous data-to-parity ratio. The [attach manual](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-attach.8.html) documents the process. Proxmox supports it from VE 9 with ZFS 2.3.3.8889Attaching another disk to a mirror adds redundancy rather than usable capacity. Plan expansion before buying the disks; the installed software version and pool features constrain the available options.9091For hardware, give ZFS direct disk access through suitable controllers or HBAs. [Proxmox's ZFS guidance](https://raw.githubusercontent.com/proxmox/pve-docs/master/local-zfs.adoc) recommends ECC RAM and advises against putting ZFS over a caching hardware RAID controller. Reliable flush behaviour, SSD endurance, power-loss protection and replacement availability belong in the specification too.9293### Avoid buying the wrong kind of cache9495| Mechanism | Purpose |96| --- | --- |97| **ARC** | RAM cache. Budget it alongside applications and VMs. |98| **L2ARC** | Additional read cache on another device; also consumes RAM for bookkeeping. |99| **SLOG** | A separate intent-log device for synchronous writes. It can improve their latency, but is not a general write cache. |100| **Special vdev** | Holds actual metadata and optionally small data blocks. Its loss can lose the pool, so it needs appropriate redundancy. |101102The [OpenZFS cache guide](https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Pool%20Structure/Caching.html) covers the tradeoffs. Measure the bottleneck before adding hardware. A read-cache SSD cannot fix a workload limited by synchronous writes.103104Filesystem `recordsize`, zvol `volblocksize` and physical-sector alignment (`ashift`) also affect performance. Larger backing blocks can amplify small guest writes. Test representative sustained I/O, including while the pool is degraded or rebuilding.105106Keep operating headroom: OpenZFS recommends [more than 10% free](https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html). Set alerts early enough to allow procurement and expansion. Snapshot retention and thin provisioning can consume the margin quickly.107108Compression can reduce space and I/O where data compresses well. Deduplication needs its own memory and workload sizing; I would leave it off initially. The [property reference](https://openzfs.github.io/openzfs-docs/man/v2.3/7/zfsprops.7.html) covers both.109110## Day-to-day operation and recovery111112`zpool` manages pools and devices; `zfs` manages datasets, volumes and snapshots. The cheat sheet below covers everyday administration.113114Schedule scrubs to read allocated data and verify checksums. They generate I/O and repair only what redundancy permits. A resilver reconstructs missing or outdated data after attachment or replacement; ZFS does not run a scrub and resilver together. The [scrub documentation](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-scrub.8.html) explains the difference.115116Record disk identities and enclosure slots before a failure. When replacing a device, follow the resilver through to completion. Boot-pool replacements also need the platform's bootloader procedure.117118For recovery, a filesystem snapshot can be browsed through `.zfs/snapshot/<name>`. Copy out a deleted file, or create a writable clone to inspect an earlier state. [Rollback](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-rollback.8.html) discards later changes, so establish what needs recovering before using it.119120Snapshots are atomic at the storage layer. Databases and guests may still need flushing, quiescing or application-native backups to produce a usable recovery point. Keep retention within the capacity budget: deleting a live file does not free blocks still referenced by snapshots. Test recovery onto another pool or machine as well as local restores.121122Check recovery-host compatibility before enabling new pool features. A [`zpool upgrade`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-upgrade.8.html) can make the pool unreadable by older software.123124## What happens when ZFS goes bad125126ZFS is good at describing unhealthy storage, but somebody still has to collect those signals and act on them. A pool can remain available in a `DEGRADED` state after losing redundancy. If another valid copy exists, ZFS can repair a damaged block; without one, [`zpool status -v`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-status.8.html) reports a permanent data error. Losing a required top-level vdev can make the whole pool unavailable.127128I would treat a degraded pool as urgent even while applications are still running. The replacement and resilver period is exactly when the remaining disks are carrying more risk.129130### Native health and event reporting131132| Signal | What it shows |133| --- | --- |134| `zpool status -vP` | Pool and vdev state, full device paths, read/write/checksum counters, scrub or resilver progress, and known permanent data errors. |135| `zpool status -x` | Only pools with errors or that are unavailable; convenient input for a monitoring check. |136| `zpool events -v` | Recent kernel events including checksum and I/O errors, slow or hung I/O, missing devices, spare activation, scrubs and resilvers. |137| `zpool history -il tank` | Administrative commands and internal pool history useful during an investigation. |138| SMART/NVMe and kernel logs | Media errors, wear, temperature, link resets and controller faults below the ZFS layer. |139140The [`zpool events`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-events.8.html) history comes from the kernel's recent-event queue; it is not a durable log archive. On Linux, the [ZFS Event Daemon](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zed.8.html) listens for those events and runs enabled **zedlets**. Distribution packages commonly provide zedlets for logging or notification, but the exact files and service names vary. Check that ZED is running, inspect its journal or syslog output, and forward those records off the storage host.141142Useful first checks on a systemd-based host are:143144```bash145zpool status -vP146zpool status -x147zpool events -v148zpool history -il tank149systemctl status zfs-zed.service150journalctl -u zfs-zed.service151```152153Do not clear error counters merely to make a dashboard green. Capture the status and event evidence first, identify the device or path involved, and confirm that the repair or replacement completed. `zpool clear` clears recorded errors; it does not repair failing hardware.154155### Alerts I would expect156157| Condition | Monitoring response |158| --- | --- |159| Pool or required vdev is not `ONLINE` | Alert immediately, with the pool, device path or GUID and current state. `DEGRADED` still means redundancy has been reduced. |160| Read, write or checksum counters increase | Open an investigation even if the pool remains online; correlate with ZFS events, SMART/NVMe data and kernel logs. |161| Permanent data errors appear | Critical alert with the affected objects from `zpool status -v`, followed by application-level validation or restore. |162| Free capacity crosses operating thresholds | Warn early enough to expire snapshots, move data or add capacity without entering the last 10% under pressure. Track the trend as well as the current percentage. |163| Scrub is overdue, aborts or completes with errors | Alert on the schedule and result, not merely whether a scrub happens to be running when the poll occurs. |164| Resilver or spare activation starts | Notify the operations team, then alert if progress stalls or the final pool state is not healthy. |165| ZED or the monitoring agent stops reporting | Treat stale monitoring as its own fault; silence is not an `ONLINE` pool. |166| Replication or backup stops advancing | Alert on the age of the last successful snapshot, send and receive, then keep testing restores separately. |167168Pool health is only one layer. Monitor the physical devices, HBA or controller, enclosure, filesystem capacity, snapshot growth and application behaviour as separate services. `zpool iostat` can expose a busy or slow pool, but latency and throughput need workload-specific thresholds rather than a universal healthy value.169170### Checkmk171172Checkmk has official checks for [ZFS filesystem usage](https://checkmk.com/integrations/zfsget), [pool size](https://checkmk.com/integrations/zpool) and [pool status](https://checkmk.com/integrations/zpool_status). The filesystem-usage check lists Linux, Solaris and FreeBSD agents. At the time of writing, the catalogue marks the pool-size and pool-status checks as Solaris agent checks; the status check reads `zpool status -x`, reports pool errors as critical, and warns on extended checksum or other error information.173174For Linux and Proxmox hosts, verify what the installed Checkmk agent and version actually emit before relying on service discovery. If the packaged agent does not provide the health section, a Checkmk local check or maintained extension can wrap `zpool status -x` and expose one service per pool. Community extensions such as the [OPOSS zpool iostat plug-in](https://github.com/oposs/cmk-oposs_zpool_iostat) add capacity, throughput, latency and maintenance metrics, but their compatibility and support remain with the extension maintainer.175176I would pair the ZFS checks with Checkmk services for ZED, SMART/NVMe health, capacity, scrub age and replication freshness. Then force a test state in a lab or test fixture and confirm that discovery, state changes, notification routing and recovery messages all work.177178### Nagios179180[Nagios Core](https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/4/en/plugins.html) deliberately delegates service checks to external plugins. It has no built-in understanding of ZFS, but a plugin can run locally through NCPA or NRPE, or remotely through `check_by_ssh`, and return the usual OK, WARNING, CRITICAL or UNKNOWN state.181182Nagios Exchange has a community [`check_zfs`](https://exchange.nagios.org/directory/plugins/operating-systems/solaris-operating-systems/check_zfs/details/) plugin for pool health, but its listed release dates from 2009 and targets Solaris/OpenSolaris and early FreeBSD. I would treat it as an example rather than deploy it unchanged on a current Linux host.183184A small maintained site plugin can inspect `zpool status -x`, or the JSON form of `zpool status` where the installed OpenZFS version supports it. It should report the affected pool and devices, map degraded state and new error counters to the site's chosen severity, and publish capacity or error counters as performance data. Run it on the storage host with only the permissions it needs. Whatever collects the result, test the complete path from an unhealthy sample through to the person expected to respond.185186## Updating repositories inside a restricted network187188Another team I work with uses ZFS deltas to move software repository updates from an internet-connected system into a restricted network. I believe a data diode is involved, but have not confirmed that detail. One possible design follows.189190Repository-specific tools could populate datasets containing Debian/Ubuntu mirrors, Git repositories, Python/PyPI mirrors or npm content served through something such as Verdaccio. Once an update is complete and coherent, ZFS takes a snapshot. An initial full send establishes the destination, followed by incremental streams through the approved transfer mechanism. The destination receives the data, validates the repository and makes the release available to clients.191192ZFS tracks changed blocks between snapshots, avoiding a fresh walk of the file tree to discover changes. It handles [replication](https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Operations/Send%20and%20Receive.html); the repository tools still handle packages and metadata, and the gateway handles transport across the boundary.193194Here is the basic sequence for one filesystem dataset. Assume existing pools `source` and `restricted`, a populated `source/repos` dataset, and staging directories `/export` and `/import` outside that dataset. Use suitable ZFS privileges and allow space for the stream files. For the initial receive, `restricted/repos` must not already exist.195196```bash197# Source: after the repository update has completed198zfs snapshot source/repos@release-001199zfs send source/repos@release-001 > /export/repos-full.zfs200```201202Check that the send completed, then transfer the finished file through the approved mechanism. At the destination:203204```bash205zfs receive -u -o readonly=on restricted/repos < /import/repos-full.zfs206```207208For the next completed update:209210```bash211# Source212zfs snapshot source/repos@release-002213zfs send -i source/repos@release-001 source/repos@release-002 \214 > /export/repos-001-to-002.zfs215216# Destination, after successful transfer217zfs receive -u restricted/repos < /import/repos-001-to-002.zfs218```219220The [`-i` send](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-send.8.html) contains the changes between those endpoints. On the initial [`receive`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-receive.8.html), `-u` suppresses automatic mounting; `readonly=on` prevents ordinary filesystem writes. Keep logs and local state elsewhere. For later imports, stop serving and unmount the receiving dataset first, or serve a separate validated snapshot or clone. Verify each import and its repository contents before making the release available to clients.221222Retain a shared baseline. The receiver's most recent snapshot must match the incremental starting point, with no independent filesystem changes. If `001` to `002` is missed, a receiver still at `001` cannot apply `002` to `003`: replay the missing transfer or generate `001` to `003` while that source baseline remains. Otherwise, arrange another full transfer. A sender-side [bookmark](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-bookmark.8.html) can preserve an incremental starting point, but the receiver still needs its matching snapshot.223224Record successful imports and check snapshot identities, not just filenames. Do not use `receive -F` as a routine fix for a mismatch: it can discard destination state. Delivery, retries and acknowledgements need to fit the approved one-way design; ordinary bidirectional SSH cannot run directly through a diode.225226A diode controls direction, and ZFS checksums detect damaged streams. Package signatures, publisher verification and import approval remain separate controls, as reflected in [ASD's gateway guidance](https://www.cyber.gov.au/business-government/asds-cyber-security-frameworks/ism/cyber-security-guidelines/guidelines-for-gateways). If a gateway connects the networks, “restricted network” is a more accurate description than “air gap”.227228## When I would choose something else229230| Requirement | Alternative to consider |231| --- | --- |232| An established SAN with supported multipathing and vendor recovery procedures | Keep the array's supported host filesystem or volume stack unless ZFS fills a specific gap. |233| Storage availability across server failures | A supported HA appliance or distributed system such as [Ceph](https://docs.ceph.com/en/latest/rados/). Local ZFS alone does not provide cluster failover. |234| A simple Linux service with adequate backups | ext4 or XFS may cover the requirements with fewer storage-specific procedures. |235| A conventional Linux host needing flexible block volumes beneath ext4 or XFS | LVM, or LVM-thin for local VM storage, keeps the familiar block-storage model. |236| Object APIs and lifecycle policies | An object-storage service; a ZFS filesystem does not supply these by itself. |237| A team needing one accountable support provider | A supported storage product, including ZFS appliances. Budget the operating model as well as the disks. |238239### When LVM makes more sense240241I would lean towards [LVM](https://sourceware.org/lvm2/) when the requirement is to carve flexible block volumes for ext4 or XFS, particularly where a hardware array or Linux software RAID already owns the redundancy. It fits conventional Linux installation and recovery procedures, and may be the simpler choice for a team already operating that stack confidently.242243[LVM-thin](https://raw.githubusercontent.com/proxmox/pve-docs/master/pve-storage-lvmthin.adoc) is also a sensible Proxmox backend for local VM disks when efficient snapshots, clones and thin provisioning are required without adopting the wider ZFS operating model. LVM does not add ZFS-style end-to-end checksums, scrubs or self-healing, so those responsibilities remain with the array or RAID layer, the filesystem, monitoring and backups.244245## Summary246247ZFS feels like a next-generation filesystem, although that description almost sells it short. It combines the filesystem with volume management, redundancy, checksumming, snapshots, replication and several layers of caching. That breadth is what makes it attractive, but it also moves a large amount of storage engineering into the operating system and onto the team running it.248249For a small systems administration team, installing ZFS is the easy part. The harder work is building enough shared knowledge to design vdevs well, understand capacity behaviour, recognise failure modes, replace devices safely and recover data under pressure. Some layout decisions are awkward or expensive to change later. Reskilling also competes with normal operational work, and an outage is a poor time for the second person on the team to learn how pools and resilvers behave.250251I would want more than one person able to operate the environment, backed by standard hardware and layouts, monitoring, reviewed changes, concise runbooks and regular recovery exercises. Starting with a bounded workload such as a backup target or file service gives the team room to learn before placing more critical systems on it.252253Commercial storage vendors still have an important place. A supported array or appliance hides much of the controller, firmware, disk-health, failover and upgrade complexity behind a tested product and a support contract. The customer gives up some control and often pays more, but gains a clearer operational boundary and somebody to call during a difficult recovery. Some of those products use ZFS themselves; the value is in the integration, validation and support surrounding it.254255I can see ZFS fitting well in VM hosts, file services, backup targets and repository distribution. I would choose it where the team wants that control and has the time to build the operational capability around it. Where the team cannot sustain that investment, a supported storage product may be the more responsible engineering choice.256257> ## <span class="cheat-sheet-title">ZFS Cheat Sheet</span>258>259> Linux/OpenZFS examples for routine administration. These are independent examples, not a script: substitute your pool, dataset, snapshot and device names. Changes require root or delegated privileges; add `sudo` where appropriate. Use Proxmox or appliance management tools for storage they own.260>261> `tank` is an existing pool, `tank/data` a filesystem, `@before` a snapshot and `#before` a bookmark. New dataset names and mount points must be unused.262>263> ### Inspect health, space and activity264>265> | Task | Command |266> | --- | --- |267> | Installed version | `zfs version` |268> | Pool capacity and health | `zpool list` |269> | Device health, errors and full paths | `zpool status -vP tank` |270> | Filesystems and volumes below a pool | `zfs list -r tank` |271> | Space used by data, snapshots and children | `zfs list -o space -r tank` |272> | All properties and their source | `zfs get all tank/data` |273> | NFS/SMB sharing properties | `zfs get sharenfs,sharesmb tank/data` |274> | Snapshots, oldest first | `zfs list -t snapshot -r -s creation tank/data` |275> | Pool and device I/O every five seconds | `zpool iostat -v -y tank 5` |276> | Pool administration history | `zpool history tank` |277>278> [`zpool list`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-list.8.html) reports pool allocation; [`zfs list`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-list.8.html) accounts for dataset space. [`iostat -y`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-iostat.8.html) skips the initial since-boot report. Press **Ctrl+C** to stop it. Configure network shares through your platform's supported NFS/SMB tools.279>280> ### Create, rename and mount281>282> | Task | Command |283> | --- | --- |284> | Create a filesystem at a chosen path | `zfs create -o mountpoint=/srv/data tank/data` |285> | Create a 50 GiB block volume | `zfs create -V 50G tank/volume` |286> | Rename a dataset within its pool | `zfs rename tank/data tank/archive` |287> | List mounted ZFS filesystems | `zfs mount` |288> | Mount a filesystem | `zfs mount tank/data` |289> | Unmount a filesystem | `zfs unmount tank/data` |290> | Change its mount point | `zfs set mountpoint=/srv/archive tank/data` |291>292> [`create`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-create.8.html) normally mounts a new filesystem immediately. A zvol is a block device and reserves its size by default. Stop consumers before unmounting, renaming or changing mount points; a [rename](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-rename.8.html) can also change an inherited mount point. Filesystems with `mountpoint=legacy` use the operating system's [mount tools](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-mount.8.html).293>294> ### Compression, quotas and reservations295>296> | Task | Command |297> | --- | --- |298> | Enable LZ4 compression for new writes | `zfs set compression=lz4 tank/data` |299> | Check compression and achieved ratio | `zfs get compression,compressratio tank/data` |300> | Remove a local compression override | `zfs inherit compression tank/data` |301> | Cap dataset, snapshots and descendants | `zfs set quota=100G tank/data` |302> | Cap the dataset alone | `zfs set refquota=80G tank/data` |303> | Reserve capacity for dataset and descendants | `zfs set reservation=10G tank/data` |304> | Reserve capacity for the dataset alone | `zfs set refreservation=10G tank/data` |305> | Remove a quota | `zfs set quota=none tank/data` |306> | Remove a reservation | `zfs set reservation=none tank/data` |307>308> Quotas limit consumption; reservations set capacity aside. `refquota` excludes snapshots and descendants. `none` also clears `refquota` and `refreservation` when used with those property names. [`inherit`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-set.8.html) removes a local override so an inheritable property follows its parent or default. See the [property reference](https://openzfs.github.io/openzfs-docs/man/v2.3/7/zfsprops.7.html).309>310> ### Snapshots and recovery copies311>312> | Task | Command |313> | --- | --- |314> | Snapshot one dataset | `zfs snapshot tank/data@before` |315> | Snapshot it and its descendants together | `zfs snapshot -r tank/data@before` |316> | Show changed paths since a snapshot | `zfs diff -F tank/data@before tank/data` |317> | Create a writable recovery clone | `zfs clone -o mountpoint=/srv/review tank/data@before tank/review` |318> | Reverse the clone's dependency on its origin | `zfs promote tank/review` |319> | Protect a snapshot from deletion | `zfs hold keep tank/data@before` |320> | List its hold tags | `zfs holds tank/data@before` |321> | Release that hold | `zfs release keep tank/data@before` |322>323> For individual files, browse `/srv/data/.zfs/snapshot/before/` and copy out what you need. [`diff`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-diff.8.html) lists changed paths, not file-content differences. Coordinate application consistency before taking [snapshots](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-snapshot.8.html).324>325> A [clone](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-clone.8.html) stays in the same pool and depends on its origin snapshot. [`promote`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-promote.8.html) reverses that dependency and transfers snapshot accounting; it does not rename the datasets or switch application paths. Releasing the last [hold](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-hold.8.html) can allow a snapshot already marked for deferred deletion to disappear.326>327> ### Keys for encrypted datasets328>329> Here `tank/secure` is an existing encryption root; these commands do not enable encryption on an unencrypted dataset.330>331> | Task | Command |332> | --- | --- |333> | Inspect encryption and key status | `zfs get encryption,keystatus tank/secure` |334> | Find the encryption root and key source | `zfs get encryptionroot,keylocation tank/secure` |335> | Load its key | `zfs load-key tank/secure` |336> | Unload its key | `zfs unload-key tank/secure` |337>338> Loading uses the configured `keylocation`; `prompt` asks interactively. It does not mount the filesystem, so use `zfs mount` separately. Before unloading, unmount affected filesystems and close users of encrypted volumes. [Key operations](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-load-key.8.html) also affect datasets inheriting that root's key.339>340> ### Send, receive and bookmarks341>342> These examples assume completed snapshots `@s1` and `@s2`, existing staging directories and enough free space. Check that each send succeeds before transferring its file.343>344> ```bash345> # Full stream from the source346> zfs send tank/data@s1 > /export/data-full.zfs347>348> # Incremental stream from s1 to s2349> zfs send -i tank/data@s1 tank/data@s2 \350> > /export/data-s1-to-s2.zfs351>352> # At the destination, after transferring each completed file:353> # Initial receive creates backup/data in an existing backup pool354> zfs receive -u -o readonly=on backup/data < /import/data-full.zfs355>356> # Later incremental receive357> zfs receive -u backup/data < /import/data-s1-to-s2.zfs358>359> # Preserve a source-side incremental starting point360> zfs bookmark tank/data@s1 tank/data#s1361> zfs list -t bookmark -r tank/data362> ```363>364> The initial destination dataset must be absent. An incremental [`receive`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-receive.8.html) needs the matching baseline as its latest snapshot, without independent filesystem changes. `-u` suppresses automatic mounting; it does not unmount an already mounted destination. Stop consumers and unmount it before updating a served replica, or serve a separate validated copy.365>366> A [bookmark](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-bookmark.8.html) can replace the sender's old snapshot as the [`send -i`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-send.8.html) starting point, using `tank/data#s1`. It retains no recoverable files; the receiver still needs its matching snapshot. The shell needs permission to write the stream file, even when `zfs send` runs under `sudo`.367>368> ### Scrub, import and export369>370> | Task | Command |371> | --- | --- |372> | Start checking allocated data | `zpool scrub tank` |373> | Monitor scrub or resilver progress | `zpool status tank` |374> | Stop a running scrub | `zpool scrub -s tank` |375> | List pools available to import | `zpool import -d /dev/disk/by-id` |376> | Import a pool | `zpool import -d /dev/disk/by-id tank` |377> | Unmount and release a pool for transfer | `zpool export tank` |378>379> A [scrub](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-scrub.8.html) repairs only where healthy redundant data exists. [Import](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-import.8.html) a pool only when it is inactive on other hosts. Stop consumers before [exporting](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-export.8.html); export leaves the data intact.380>381> ### Create pools and manage disks382>383> **Commands in this section change pool topology or initialise disks.** Replace every uppercase device placeholder with a verified path from `zpool status -P` or your disk inventory. New disks must contain no required data. The creation example uses a new pool named `newpool`.384>385> ```bash386> # Preview a new mirrored pool; -n makes no changes387> zpool create -n newpool mirror \388> /dev/disk/by-id/DISK_A /dev/disk/by-id/DISK_B389>390> # Create it after reviewing the devices and layout391> zpool create newpool mirror \392> /dev/disk/by-id/DISK_A /dev/disk/by-id/DISK_B393>394> # Temporarily take a device offline, then bring it back395> zpool offline -t tank /dev/disk/by-id/EXISTING_DISK396> zpool online tank /dev/disk/by-id/EXISTING_DISK397>398> # Replace a device; monitor resilvering with zpool status399> zpool replace tank \400> /dev/disk/by-id/OLD_DISK /dev/disk/by-id/NEW_DISK401>402> # Turn a single-disk vdev into a mirror, or add a mirror leg403> zpool attach tank \404> /dev/disk/by-id/EXISTING_DISK /dev/disk/by-id/NEW_DISK405>406> # Preview adding a separate mirrored vdev for more capacity407> zpool add -n tank mirror \408> /dev/disk/by-id/NEW_DISK_A /dev/disk/by-id/NEW_DISK_B409>410> # Add that vdev after reviewing the preview411> zpool add tank mirror \412> /dev/disk/by-id/NEW_DISK_A /dev/disk/by-id/NEW_DISK_B413>414> # Expand a device to its available size after enlarging/replacing it415> zpool online -e tank /dev/disk/by-id/EXPANDED_DISK416> ```417>418> Before [creating a pool](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-create.8.html), choose sector alignment and feature compatibility for its hardware and recovery hosts. Take a device [offline](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-offline.8.html) only if the remaining redundancy can sustain it. A [replacement](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-replace.8.html) needs usable surviving data; it cannot reconstruct a failed sole disk. Boot pools also require the platform's bootloader procedure.419>420> For the mirror examples, [`attach`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-attach.8.html) adds redundancy; [`add`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-add.8.html) adds a top-level vdev and capacity. All mirror or RAIDZ members must be [expanded](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-online.8.html) before the vdev gains their extra space.421>422> ### Rollback and deletion - destructive423>424> | Task | Command |425> | --- | --- |426> | Discard live changes since the latest snapshot | `zfs rollback tank/data@before` |427> | Preview deleting a snapshot | `zfs destroy -nv tank/data@old` |428> | Delete that snapshot | `zfs destroy tank/data@old` |429> | Preview deleting a dataset and its descendants | `zfs destroy -nvr tank/retired` |430> | Delete that dataset and its descendants | `zfs destroy -r tank/retired` |431> | Destroy an entire pool and its datasets | `zpool destroy retiredpool` |432>433> For [`rollback`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-rollback.8.html), `@before` must be the latest snapshot; subsequent live changes are lost. Rollback has no dry-run option. Adding `-r` would delete newer snapshots and bookmarks, not recursively roll back child datasets.434>435> For [`zfs destroy`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zfs-destroy.8.html), `-n` previews and `-v` lists the proposed actions. Review that output before removing `n`. Avoid uppercase `-R` as a shortcut: it can destroy dependent clones outside the target hierarchy. [`zpool destroy`](https://openzfs.github.io/openzfs-docs/man/v2.3/8/zpool-destroy.8.html) has no equivalent dry run; it destroys the pool rather than exporting it.436>437> *Documentation and command syntax checked in September 2026.*438