7:16 AM - Reconfiguring a ZFS pool
One of our servers was setup in MidnightBSD 0.7 and had an older ZFS configuration. Over time, the drivers were replaced with 4k advanced format disks. Since ZFS won't let you change the ashift setting on an existing pool, the drivers were not running at optimal performance. MidnightBSD 1.0 current warns about this sutation.
How to fix it?
In this case, there were 2 drives in a mirror and a cache disk as well as another backup pool. First, we performed a backup to the other pool just to be safe using zfs send -R mypool@snapshot | zfs receive -F backup/mypool, then we removed one of the disks from the pool using zpool detach ada4p1. We then blew away the partition and created a new gpt partion that was 4k aligned. gpart add -t mnbsd-zfs -a 4k ada4
Next, we created a new pool called tank with the one drive. We then ran zfs send and receive to copy data from mypool to tank.
Once this data is copied, we went into single user mode (shutdown now) and exported both pools. We then imported the new pool tank. zfs export tank; zfs export mypool; zfs import tank. This allowed our mount points to kick in.
Next, we repreated the process of repartitioning the second disk, ada3. We then add it by doing zfs attach tank ada4p1 ada3p1 to include the second disk in the mirror. Finally we did zpool add cache tank ada1 to re-add our ssd cache drive.
The drive will need to rebuild which can take some time.
This approach prevents any data loss.