The 16x Leap: How I Saved a 15-Year-Old Mail Server
From PERC 6/i bottlenecks to ZFS mastery: Scaling a Dell T410 to 21 million inodes on FreeBSD.
In 2009, I deployed a Dell PowerEdge T410 at the heart of our infrastructure. It was running FreeBSD 8.0 on UFS with a standard Dell PERC 6/i hardware RAID controller.
By 2012, we hit a wall. With over 4 million emails (inodes) stored, the system was crawling. The “Professional” hardware RAID was actually the bottleneck. This is the story of how a hardware swap and a move to ZFS didn’t just fix the problem—it made the system 16 times faster.
The Enemy: Hardware RAID (PERC 6/i)
The PERC 6/i is a “black box.” It forces a RAID layer between the Operating System and the disks. For a modern filesystem like ZFS, this is a disaster. ZFS needs to see the “metal” to manage data integrity and I/O scheduling effectively.
The Solution: The H200 and “IT Mode”
I pulled the PERC 6/i and replaced it with a Dell H200. Crucially, I flashed the H200 to IT (Initiator-Target) Mode. This turned the controller into a simple pass-through (JBOD), giving FreeBSD direct access to the drives.
The Secret Sauce: 3 Tuning Moves
Once FreeBSD could talk to the disks directly, I implemented three specific changes that led to our performance explosion:
Recordsize=16k: Mail servers handle millions of small files. The ZFS default of 128k causes massive “write amplification.” By setting
recordsize=16k, I matched the filesystem to the workload.LZ4 Compression: Text-heavy emails compress beautifully. Using LZ4 reduced the physical data the disk heads had to read by nearly 60%, effectively increasing our bus speed.
Hybrid Storage (ZIL/L2ARC): I added two SSDs—one for the ZFS Intent Log (ZIL) to accelerate synchronous writes, and one for the L2ARC to cache frequent reads.
The Results
The difference was night and day. We moved from a struggling UFS setup to a ZFS powerhouse that eventually scaled to 21.2 million inodes.
That same server stayed in production, upgraded through FreeBSD 9 and 10, until it was finally retired on October 4th, 2025.
Lesson learned: Don’t let your hardware hide your disks from your kernel.