Insane Speeds

If you've been reading my blog, you'll know I am a big fan of fast storage.


Here is something to feast your eyes on.

I'm getting close to 700 MB/sec at peak. Seven Hundred Megabytes per second.

This is an undisclosed SAS RAID I am working with. Insane. This kind of speed cost $$$$$.
You can edit some nice 4K HD videos with this speed.

Measuring and optimizing Network Throughput on OSX

I've been cleaning up my home network; optimizing what servers I need to run for certain functions. There are a couple things to speed up and optimize your home network. First of all, you will want to measure your throughput with some tools. My current tool of choice is iperf.

I've been using iperf on Linux for a while and found a pre-built binary for OSX. If you don't care for the command line, a java version is also available called jperf.

It is a very simple too to use.

On the server, simply run the following:

iperf -s

On the client, you connect to the server by:


iperf -c XXX.XXX.XXX.XXX

(XXX.XXX.XXX.XXX = Ip address).

As you can see below, on my home network, I am getting 986 Mbits/sec. Roughly123 Megabytes per second.

You can specify other options such as formatting like this:

iperf -c 172.16.1.39 -f M
The -f flag will show the results in Megabytes per second

iperf is a great cross platform tool. Here, I am running it under Linux on my Thinkpad to check the speed of my hacked Pogoplug running ArchLinux on an ARM processor.
(To install iperf on the pogo, simply run Pacman manager: pacman -Sy iperf).

Jperf gives iPef a graphical UI layer if you don't want to mess with the console. You will simply need Java installed to run it.

As you can see in my screenshot, I am hitting the theoretical limits of Gigabit ethernet. My network is pretty clean and fast.

If your network switch supports jumbo frames, I suggest your try to take advantage of it. You would need to manually configure your network settings. And sometimes, it may not yield the perfect results so you will need to experiment.

I personally set my MTU to Jumbo Frames at 9000.

And how does my network perform? Different network protocol will yield different results. I ran some benchmarks to see which performed better. Different protocols will yield different results due to overhead. My testing was between my iMac and Macbook and my source was running off an SSD.

Samba/CIF was rather poor between two macs at 70 Megabytes per second.

NFS was a whole lot better than running Samba.

AFP (Apple File Protocol) performed the best;hitting Gigabits limits between macs.

Note, this type of scenario is not indicative of all use cases. In this particular setup, AFP was better suited between two macs. I'm sure SAMBA could have been tweaked to gain better performance. However, I am glad to know that my actual network hardware is capable of performing to its highest potential. Thus, this is why it is important to use tools like iperf.

Ramdisk vs SSD on Ubuntu

With the advent of fast SSDs capable of reading/writing 200-500 Mb a second, is there a need for ramdisks?  I decided to try it out in Ubuntu 12.04.

If you are wondering what a ramdisk is, it  is simply using your physical RAM memory as a temporary storage drive. Instead of writing to disk, you are writing files to memory.

And the results of my testing?

Well, I'll let these pictures speak for themselves:

1st. Ramdisk.  Average read 1.2Gb/s. As in Gigabytes per second. An entire full DVD movie worth of data would take less than 5 seconds to copy.

2nd,Corsair F120 Sandforce based SSD. Read speed bench at 234 Mb/s which is no slouch and faster than any platter drive.  The same DVD would take roughly 22 seconds.

Compared to a standard platter HDD drive, a DVD would take  81 seconds at 60Mb/sec. A 10Mb/sec USB stick would take  486 seconds or 8 minutes to copy.
  
I tried a Virtual Box VDI image in ramdisk and an ubuntu 10.10 image loaded in less than 6 seconds.

Here is how you make a ramdisk:

 mkdir -p /tmp/ramdisk  
 sudo mount -t tmpfs -o size=1024M tmpfs /tmp/ramdisk  

Or, you can simply copy files to /dev/shm/ but you risk saturating all your available ram. By using tmpfs, you can set a limit. In my example, my ramdisk is 1GB.

It should be noted that ramdisks are not persistent. They will need to be recreated upon reboot. You lose the data in ramdisk when your power down.

I am currently exploring options for a real-time ffmpeg transcoding system that will write and read quite a bit to disk. I also have another use case scenario with imagemagick/ghostscript writing large temp files of PDFs. A ramdisk may be the way to go.

There is another interesting use of ramdisk. To run a completely private and secure micro servers like tor-ramdisk to evade police authority. Data would simply disappear upon a power down. If the authority seized your equipment, all the data would simply vanish and make it harder  for forensic analysis.

Now, I just need a laptop with 32GB of RAM.