Archive for the 'FreeNas' Category

Building a 32 TB Server: a thought experiment.

August 31, 2008

Daniel Gimpelevich and Holden Aust built a 16 TB server for Christian Einfeld and his Digital Tipping Point Project.  See Linux Journal, Issue 173, September 2008.  I am impressed that these gentlemen built a server with four times the capacity of anything that I have ever attempted.  It’s funny because in Einfeld’s article he mentions it almost in passing.  My jaw was on the floor.  Also I think it’s cool that he’s a lawyer who is also very much into technology.  Moreover, his philanthropic efforts in San Francisco are admirable.

The server they built motivates me to try to build a 32 TB server.  There are three problems that I have not worked out.  One: fitting 16 drives in one box.  I would wait to build the server when 2 TB drives are obtainable.  I am assuming that I can find a case somewhere that will hold 16 drives.  If I can’t, I would have to have some sort of an external enclosure and run SATA cables to it.  Two: I don’t know if FreeNAS can handle 32 TB of storage.  If not, I’d have to use some other platform, but I suspect it could do it or could be made to do it.  Three: I don’t know if you can put three or four SATA cards on one motherboard.  Obviously these gentlemen figure that part out.  It must be possible, I just don’t know how to do it.  It may be as easy as plugging them in.

If I were to succeed, a RAID 5 FreeNAS server would provide 20.8 TB of usable space out of the 32 TB available.  Since you have to do backups anyway, it almost makes sense to have two raid zero 32 TB servers as you would get 27.73 TB of usable space each and faster performance.

At this point, this is just a thought experiment.  In terms of money, when the drives become available, we’re not talking about that much compared to other types of extreme computing.  For example some people will spend in excess of 14 or $15,000 buying an overclocked “ultimate” machine.  A 32 TB server would probably only cost $2-$3000 to build.

I have to admit it is exciting.  I don’t know what I would use it for.  I still have plenty of space on my 4 TB server that only has 2.6 TB of usable space.  Even with an HD TiVo and pulling HD content off of it and putting it on the server, I don’t think I would need anything close to 32 TB of space.  But it would be fun to build.

FreeNAS Idiosyncracies.

July 27, 2008

I enjoy using FreeNAS. I have three FreeNAS servers. Why would anyone need three? It’s because of the idiosyncrasies associated with using the software. For one thing, Vista doesn’t like FreeNAS very much in certain configurations and I have 3 Vista machines. For example, if you set up one of your servers in a RAID 5 configuration, Vista will read it just fine. But it won’t write to it. It won’t write to it because it thinks the disk is full. Vista can’t get an accurate reading of the disk size. So it won’t write to it at all. XP does not have this problem, thankfully. If it did, I would not be able to use FreeNAS in a RAID 5 configuration.

I overcome this idiosyncrasy by having another server set up with its disks mirrored. Vista reads disk mirrors without any trouble. All I have to do is use an XP machine to sync up the data from the disk mirrors with the main server. This way I have all my data on one machine. I use the third server to back up the main server. In a way, it seems ridiculous. However, FreeNAS servers are cheap (the software is free). Moreover on a gigabit network, data transfer is fast. You could accomplish what I do with a Drobo. However, it wouldn’t be nearly as fast. Also I’m not sure that you could stream video from a Drobo. FreeNAS servers make excellent video servers.

I started out using Rsync to keep my servers synced with each other. This quickly crashed. I couldn’t figure out how to fix it either. So, I moved to Allway Sync. This program has worked wonderfully. Of course FreeNAS wouldn’t be FreeNAS with out idiosyncrasy here as well. You have to reboot the servers more than you should. Often, after transferring gigabytes of data, the server will drop out and need to be rebooted. I haven’t lost any data and the servers are fine once you reboot them. Sometimes, with Allway Sync, sinking directionally as opposed to bidirectionally works better.

The latest FreeNAS joy has been having two of the servers spontaneously switch IP addresses. I have no idea why this happened. I thought for a moment that maybe there was some malice involved from a third party. But no. My best guess is that IP addresses opened up and that the servers rebooted and picked those. I’ve been using FreeNAS for a long time and it has never done that.

You might think that is a lot of effort to use these servers. But I can tell you that that is not so. While I have identified the above issues, they are all manageable. The servers have great up time and I can move my data around quickly.  Plus, it is awesome to have all my data in one place.  It’s like having an old car that has problems but you know what the problems are and you know how to fix them. And in the meantime the car gets you where you want to go. With all the redundancy I have with these servers, I believe my data is safer than it has ever been (knock on wood).

Building a 3 TB Backup Server Using FreeNAS.

July 6, 2008

My computer videos are the most popular thing that I do online for fun. Here is my latest.

FreeNAS versus the NetApp Storevault S300 and the Iomega Storcenter Pro-Nas 150d.

February 23, 2008

In the latest issue of PC Magazine, (the link is a past review on the site) they reviewed NAS storage options.  I have to say that I am surprised at the cost of these boxes.  The NetApp is $2500 and the Iomega is $1700.  Both have a storage limit of three TB.  The NetApp gets transfer rates of around 64 Mbps at best.  The Iomega is slower.

The FreeNAS box I built with 4 TB of storage cost less than both boxes.  The upper limit on storage is significantly higher.  Transfer rates can be over 100 Mbps.  In short, a FreeNAS box with old hardware is far superior to these devices and costs less.

They also review and a Windows home server appliance but that’s not even worth mentioning.  I am surprised that dedicated NAS devices aren’t all benchmarked against a FreeNAS server.  It is clearly the home and small business standard.

Making a 4 Terabyte Server with FreeNAS.

January 27, 2008

I built upon my experience with my first FreeNAS server and created a much more powerful version.  You can see what I did this weekend.  As builds go, it really went well.  It is awesome to have one place for all my data at home, one server to rule them all.

At one point I say, “don’t try this at home.”  What I meant was, don’t stick your hand into a computer that is open and running.  Everyone should try and build one of these servers.

Building a FreeNAS file server with 4 1 TB drives.

January 21, 2008

 I ordered parts for a new server today.  With my success in building a FreeNAS based box, I’m going to try to build another one using 4 1 TB drives in RAID 5.  If it works, I’ll finally be able to put all my data in one place.  Oh and I also ordered a metric butt-ton of Cat 6 cable to try to help increase the speed of my supposedly gigabit network.  Goodtimes are ahead!  I decided to wait on building a new quadcore flagship machine, perhaps as late as until May.

I’m really scavenging my KVM lately.  One of the 4 boxes went to build my 1st server, now my Ubuntu machine is being sacrificed to build the second server.  Yet another box is going to be moved behind the TV in the basement to act as a videoserver for that.  I’ll only have one box left!

Maybe in May, instead of just building one flagship machine, I’ll build 4 and fill up my KVM with bad-ass processing power!  Muhahahaha!  Skynet will be born in my basement…

Thaed’s Home Network Upgrade Adventure.

December 30, 2007


Having a RAID 5 file server on a gigabit network for all the computers in my house has been a dream of mine for a long time. In this video, I show you what I’m switching too and some of the issues that I ran into as I completed the upgrade.

The video is sadly dark and practically unwatchable. This does not prevent me from putting it on YouTube. The fact of the matter is that if you want to shoot good video (you know, with lighting) then you have to plan and prepare for that. It means work. It also helps to write what you’re going to say instead of pointing a camera and just talking. I did not put any work into this and, well, it shows. Nonetheless, I suspect that computer geeks like me might enjoy it for what it is.

Finally, I learned something from this that’s not in the video. Plugging in gigabit switches and routers doesn’t give you gigabit transfer speeds out of the box. Windows, Linux and FreeNAS require substantial tweaking before you notice any improvements. This tweaking is not trivial and I’m still working on it.

FreeNAS Glory.

December 16, 2007

So it’s been up for over a day.  I hate to start getting too excited after all I’ve been through.  So here’s what I did two nights ago.  I installed the RAID card I bought from CompUSA.  When I wrote that I could just use the 4 drives without the RAID card, I wasn’t thinking.  I would not have been able to use the CD-ROM drive.  That would not have worked.  Using the RAID card, however, I ended up in the same spot.

I used the card’s BIOS to set up a 0+1 array.  Then I loaded FreeNAS and tried to configure it.  FreeNAS recognized the drives, but not the hardware RAID.  So what I ended up doing was using the software mirroring of RAID 1 built into FreeNAS to make 2 RAID 1 drives.  So I got the benefit of using 4 drives with the cards, but I didn’t get the speed benefit of a 0+1 array.  Considering how little money I put into this, I’m really not unhappy.  I even got Vista to recognize the drives and send data to them.

It’s still early.  I thought I had had success with FreeNAS before, but I was using the beta.  Here, I’m using the stable version, but there could still be problems.  The one thing that I love about FreeNAS is the connectivity.  It just works.  I’m supporting 5 operating systems in this house and they can all talk to FreeNAS without any gymnastics.  I never did get Samba to work in Fedora despite days of effort.

I cannot say enough how cool it is to have a “real” file server.  In the past when I’ve tried to use windows shares or the Netgear toaster or even a USB drive attached to my router, I’ve had problems.  Often, the servers would drop or one particular type of OS wouldn’t see the shares or things would just be really slow or get bogged down.  Here I have great throughput and I’m able to do lots of different things at once with multiple computers.  Ahhh, it’s very satisfying.

CompUSA is going out of business and more RAID stuff.

December 14, 2007

CompUSA is going out of business.  I had a chance to go there today and pick up a few things.  I would not say that it’s like a fire sale or anything, but I did get a decent price on a computer case and Peggle.  When I see computer cases on sale, I always want to buy them.  Computer cases aren’t much fun to buy via mail order because they push up the shipping prices.

I also bought a very cheap RAID card.  Now it’s only RAID 0 or 1, but I was thinking that if I clear out another 250 GB drive, that I can have 4 drives in the machine with 2 & 2 in RAID 1.  This gives me 2 mirrored drives instead of one giant drive, but the card was dirt cheap and I’m not working with high end equipment here.  The result is going to be nearly the same:  a file server with pretty good reliability.  Unless I’m completely missing the boat here, I can also go back and use FreeNAS now (the stable version).  Ironically, it also occurs to me as I type this that I could have probably done the same thing without the card using the stable version of FreeNAS with a software RAID.  Ah well, hardware RAIDs are supposed to be faster and more reliable anyway.

NAS Nightmare.

December 12, 2007

Having smelled a wiff of success with FreeNAS, I’ve now spent the last few nights trying to build my own RAID 5 Server with Linux.  Linux comes in so many flavors.  I’m playing with Fedora now in my vision quest.  I’ve grown quite familiar with Ubuntu and Fedora is kind of alien.  Yet, in some ways it is more powerful.  The primary way this is true is that Fedora lets you set up a software RAID 5.  Ubuntu does not (at least as far as I can tell).

As of right now, I have the Fedora machine set up with a functioning RAID 5 with three 250 GB drives.  I’ve been stalled for hours, however, in something as simple as getting Samba to work.  Samba is what lets the windows machines talk to the Fedora box.

After many hours of head banging and googling, I think I have it figured out.  It seems Fedora comes with it’s own security software called SELinux.  This software doesn’t let Samba or windbind do anything.  Yet disabling this security did nothing.  The Windows machines still can’t see the Fedora box at all.

The next night…
I changed one setting in samba from workgroup to mshome and the shares were recognized by the other machines, ironically in the workgroup folder.  I thought I had finally had my cheap and easy server.  Nope.  I could see the shares with the other machines, but I could not write to them.  I file server that doesn’t let you see the files or write to the files does not do much good I’m afraid.  I am now down to editing the smb.conf file.  This is not where I want to be.  A friend of mine has suggested that he might have an old RAID card lying around that he can give me.  Certainly, that would make things much easier.  In fact, I could probably go back to FreeNAS at that point.

But I press on, continuing to try new things.  I may go with the RAID card though.  We’ll see.