I recently built a FreeNAS ZFS RAIDZ box for my personal backup archives. I wanted something elegant and low powered with the ability to run ZFS. FreeNAS is a NAS appliance built on FreeBSD. It supports the ability to run the ZFS filesystem and can be booted off a small flash storage like USB or Compact Flash.
I selected the following components:
- 6 X Hitachi 2TB 7200rpm Desktars 3.5 " drives
- ASUS E35M1-I Mini-ITX Motherboard with an AMD dual core E-350 and 8GB of RAM.
- LIAN LI Silver Aluminum PC-Q25A Mini case
- FreeNAS-8.2.0-RELEASE-p1-x64 (r11950)
- 8GB internal Patriot USB stick as OS boot
The ASUS E35M1 is a low voltage netbook AMD Fusion CPU (same one found in the Thinkpad X120E) and supports six SATA 3 6Gb/s ports. The LIAN LI PC-Q25 case can hold up to seven 3.5" hard drives. Five of those seven drives can be hot-swappable.
Here are some pictures of my build.The fit-n-finish on the LIAN LI is pretty impressive. The machined aluminium is well made. This case was clearly designed to be a HTPC or NAS box. It fits well in my Apple - Macintosh environment. Except for the logo up front, it is one slick looking piece of gear.
The Asus motherboard has everything I needed. 6 SATA ports for 6 HDD drives! All the data drives are connected to the motherboard while an internal USB stick boots the FreeNAS OS.
It also has a passive cooling heatsink.
This is where the AMD solution shines. I could not find an Intel based Mini-ITX mobo/cpu with 6 SATA III 6Gb/s ports nor one with a passive heatsink. Moreover, none of the Atom boards officially support 8GB of RAM necessary to run ZFS. This is the perfect small form factor board for FreeNAS!
Internal USB header attaches to the motherboard and hides the USB inside the case.
Picture below depicts drives loaded up. There are five hot-swappable bays. I had to put this to real world practice by taking out drives while the OS was running and without rebooting! The backplane is pretty interesting since it uses molex connectors for power.
With 8GB of ram, I have enough to run ZFS and RAIDZ; giving me roughly 9TB useable space.
After my build, I started to notice some degraded RAIDZ errors on my 3rd disk. Disk #3 seemed fine. I zeroed out the data and booted a different OS (Linux Mint) and copied files with no integrity issues. I tried the drive in different computers and everything checked out fine (S.M.A.R.T) and other scans. However ZFS zpool was giving me checksum errors. Well, it turned out to be a case of "silent data corruption." Linux Mint and Ubuntu did not see any problems but FreeNAS was able to give me a good heads up. It turned out to be a bad SATA cable. Once replaced, everything was fine.
Running a short DD benchmark,I was getting 263-268 Megabytes per second on the internal bus. This is pretty decent considering it is a RAIDZ disk array.
[root@RAIDZ] /# dd if=/dev/zero of=/mnt/RAIDZ/test.dd bs=2048k count=10000
10000+0 records in
10000+0 records out
20971520000 bytes transferred in 74.518446 secs (281427232 bytes/sec)
[root@RAIDZ] /#
[root@RAIDZ] /mnt/RAIDZ# dd if=/dev/zero of=/mnt/RAIDZ/test.dd bs=2048k count=10000
10000+0 records in
10000+0 records out
20971520000 bytes transferred in 76.008326 secs (275910826 bytes/sec)
[root@RAIDZ] /mnt/RAIDZ#
In short,
268.3899230957 megabytes
263.1290683746 megabytes
Through the network, I was getting 60-80 MB/s. This may be due to the Realtek 8111E gigabit controller on-board or the fact I was testing during the middle of the day with 60 other people on the network. I was hoping to get closer to Gigabit's theoretical limit of 125 MB/s so I may experiment with a dual NIC Intel card in the future..
Overall, I am very happy with this NAS build. It supports AFP, CIFs, NFS, iSCSI, Rsync and works surprisingly well. I also like the fact I have other FreeNAS boxes that easily sync to this one with just a few click of a mouse.
The only thing I wish for is a motherboard with 7-8 SATA ports so I can use a SSD as a cache accelerator drive.
The NAS even works surprisingly well serving files to my iPad using AFP, Samba or SFTP.
How do you internal mount the flash drive? I've seen others go the same route but I've never quite figured out how to do it. I've got my own FreeNAS box (even used the same MB) and I'd love to not have the flash drive sticking out the back all the time.
ReplyDeleteYou can buy a motherboard USB header adapter on Amazon.
ReplyDeleteIt plugs into one of the motherboard header USB pins near the SATA connectors. There are 2 sets of pins for this motherboard.
See this post: http://fortysomethinggeek.blogspot.com/2012/08/cool-4-gadget-internal-usb-motherboard.html
Great post, I did a similar build myself not too long ago.
ReplyDeleteWhats the wattage of the power supply you used?
What model number of the Hitachi are you using?
ReplyDeleteHitachi 2TB Deskstar HD32000IDK7
Deletehttp://www.frys.com/product/5947254?site=sr:SEARCH:MAIN_RSLT_PG
Thank you.
DeleteThank you! I'm in the process of building an almost-identical machine. I had already picked out the case, drives, and software. I hadn't seen the ASUS E35M1-I before now - it's got everything I need.
ReplyDeleteThanks for posting this is some great information and I enjoyed reading it... I like the idea of the Lian Li Case, it is perfect for this function. I'm thinking of building with the same case only using the Zotac board which gives me a PCIex1 slot not the PCIex16 slot with your ASUS board. This means I can add a Dual Port NIC and do LACP. I'd really like to go with the board you suggested but it just doesn't have the extra PCIcx1 slot I need. Here is the mobo I'm looking at -http://www.zotacusa.com/zotac-m880g-itx-m880gitx-a-e.html
ReplyDeleteYou can use PCIex x1 card in PCIex x16 slot. It will work. No problem. That x16 is actually x4 on those boards.
DeleteYour site is for sure worth bookmarking.look at these guys
ReplyDeleteI am thinking about going similar way.
ReplyDeleteUnfortunately ASUS E35M1-I Mini-ITX Motherboard isn't sold anymore.
I have found asus C60M1-I equipped with 6x SATA III and the CPU power is not so fare from AMD E-350
http://www.asus.com/Motherboards/AMD_CPU_on_Board/C60M1I/#specifications
http://www.cpubenchmark.net/cpu.php?cpu=AMD+E-350&id=249
http://www.cpubenchmark.net/cpu.php?cpu=AMD+C-60+APU&id=244
According following test, it can accept 16GB.
http://www.jarredcapellman.com/2012/11/14/asus-c60m1/#comments
Because of that I am guessing your ASUS E35M1-I can accept it as well
My question is: what RAIDZ type are you using in your NAS?
Petr
I am running single disk parity. C60M1-I is a good deal considering it can be had for $79 on newegg. a few customer reviews show people are using it for freenas
Deletehey guys,
ReplyDeletei just used the asus c60m1-i motherboard with the c60 cpu to build my freenas, but i only use 8 gb of ram and it runs quite fine. even though, transferrates are not as high as yours. i get roughly around 50 mb/s. i use 4xWD RED 4 TB in RaidZ1 and 2x2TB WD caviar green in Raid 1. The overall power consumption under full load is 50ish W and around 26 once all HDDs are spundown and in sleep (energysaving level 1).
A not so nice fact is, that i get some problems with the NIC. The 8111f seems to be not so good, as I get watchdog timeouts when the system is at full traffic (that is writing data to it whilst streaming sd-video-material to one client). So far i havent found a solution, but not to stream whilst copying. more research on the net came up with the idea to use another NIC in the one expansion slot, but i'D rather keep that one for later deployment of another sata controler card.
Did you guys have similar problems with the link going down?
cheers!
If you have not enough SATA ports you can always add more via SATA Port Multiplier. Each SATA2 has theoretical 375MB/s and hard drive can do about 75MB/s so not near of the theoretical limit. So 5x multiplier for 1 or 2 SATA on the motherboard can give you 10 hard drives using only 2 ports. Don't forget that 1Gbs Ethernet port is your bottleneck. It will hold max of 125 MB/s. So, if you are after size and not speed than there is always way out...
ReplyDeleteWhat is the AFP browsing app on your iPad?
ReplyDeleteyour answer here: http://fortysomethinggeek.blogspot.com/2012/10/ipad-iphone-connect-with-freenas-or-any.html
DeleteGreat post! Built my first FREENAS system in the fall (1 drive). Now it's time to really get serious and go for a nice RAID setup to enable hot swapping, and other goodies. Lots to learn, but this post has been incredibly helpful as I get started!
ReplyDeleteHi!
ReplyDeleteI can't get neither the ASUS E35M1-I nor the C60M1-I here in Europe.
Ok I did find ONE seller for the C60M1 but they want $211 for the board!!
Are there any low power and passive cooled alternatives available?
This comment has been removed by the author.
ReplyDeleteNice build! Very nice, but if your nonECC ram goes bad... Your data is gone, all data! even all backups you might have on a separate location would be garbage due to ZFS' nature and silent RAM corruption! Beware.
ReplyDeleteVery informative post. Keep up the good work. I would really look forward to your other posts
ReplyDeleteAsus - 15.6" Laptop - 8GB Memory - 1TB Hard Drive - Dark Gray (X550LADH51)
Asus - 17.3" Laptop - 8GB Memory - 1TB Hard Drive - Dark Gray