ZFS is the best file system ever.

NOTE: I will be using the term 'RAID' very loosely later in this post.  When I describe later RAID 1 and RAID 10 style arrays I am using these terms to illustrate how ZFS works in a manner most people are familiar with.

ZFS takes everything I used to know about partitions, volumes and disk space and spins it up in a blender.  Unlike the traditional file system where the size of your disk or array must be specified at the beginning, ZFS allows you to simply pick your parity and add devices as your needs for speed or space increase.  ZFS is similar to RAID but has many advantages over traditional RAID.  I'm not going to go into detail as there are many Sun/Oracle Solaris developer articles on the subject.  You can start reading about it here.

Assuming you have FreeBSD installed and two identical disks that have not been partitioned or sliced in any way named ad8 and ad10 we can get our first disk pool going with zpool.  I'm going to call my disk pool tank because most people do.

# zpool create tank mirror ad8 ad10

That created a RAID 1 style vdev, or virtual device mounted at location /tank.  Note there are other types of vdevs; I will not go into detail on them, but other types include raidz, raidz2 and no parity.   We will be able to use the amount of space equal to one of the disks, and we will have excellent read I/O and acceptable write I/O.  We will be able to continue using the pool in the event of one of the disks failing.

To ensure the pool is mounted at boot time, echo this statement to /etc/rc.conf:

# echo 'zfs_enable=YES'>>/etc/rc.conf

We can see the health of the pool by typing:

# zpool status -v

Let's assume now that we have filled the tank up and we need to add more space.  The best way to do this is to purchase two more disks of the same capacity we had on ad8 and ad10, and add them to the the pool as another mirrored vdev.  Assuming these new disks are identified as ad12 and ad14, type:

# zpool add tank mirror ad12 ad14

Now when you do a status -v on your zpool you will see two mirrors listed.

So wait, that's it?  We just doubled the capacity of /tank?  Yes.  Not only did we double the capacity, but we also doubled the speed.  We effectively converted our "RAID 1" into a "RAID 10".  This will give us excellent read and write I/O, and the ability to continue working if any one disk from any one vdev fails.  If you want it bigger or faster, keep adding vdevs.

We will periodically want to scrub the pool to make sure there aren't any errors.  To do this type:

# zpool scrub tank

This will queue up a disk scrub which we can monitor with the zpool status -v.  You might want to add this to a weekly cron job.  If any errors are discovered, they can be easily cured via the metadata ZFS keeps on the parity disks with the clear switch.

# zpool clear tank

ZFS keeps a lot more metadata than traditional hardware RAID systems, making it much safer for data archiving.  It also allows you to take a snapshot of the data.  This will allow you to retrieve deleted files or rollback to a specific period of time.  Snapshots are stored in the root of the filesystem in a .zfs folder.  For example, /tank/.zfs/snapshot.  To take a snapshot, type:

# zfs snapshot tank@19may2011

Notice there is no leading '/'; you will now have a folder named /tank/.zfs/snapshot/19may2011/ with what looks like everything you had in there before.  In the event that you need to restore something from a snapshot folder, simply cp it to where it was.  For example:

# cp /tank/.zfs/snapshot/19may2011/my_song.mp3 /tank/

Your missing/overwritten file will be restored to the state it was at that snapshot.