Viraj's Weblog

Well I came across Mysql's default 4gb limit today. I have Apache logging via mod_log_sql the tables are growing very fast. Just a week's worth of data hits 4gb.

When I think of a database limiting you to a 4gb table I think huh? It just doesn't make much sense to me, and makes me want to use PostgreSQL instead. Sure I can do all this alter table stuff to get it larger than 4gb, but why default it to such a low value in the first place? What's worse, comments from users claim such an alter table can lose records.

Gee, that's great.

I read an interesting article in the latest Linux Journal about someone who used a Linksys NSLU2 file server, installed a Linux firmware on it, and created an iTunes streaming music server. He was able to use this small device to replace a full-blown PC for serving music.

The NSLU2 is a small nifty device with 2 USB ports and an Ethernet port. You hook up USB (v2) hard drives to it, and it has a web-based administrative interface to share out the disks via Samba. Once people were able to install Linux on it, all sorts of cool applications came forth.

But I have another cool idea for such a setup. Think about getting a bunch of these Linksys devices, a bunch of USB drives, and creating a cheap clustered file server. Something that could aggregate the storage of many of these units, and possibly provide some way to easily add or remove nodes on the fly. It would be cool if there was some sort of redundancy or failover as well if a node went down.

A 'node' in this case refers to the Linksys device. On each we could connect 2 hard drives, and then use software RAID 1 to mirror them. This would give us redundant data on one node. Now we would need something to consolidate all of these nodes into one mass storage. There are a few clustering filesystems that I've done some reading on.

Probably the simplest setup is to use (E)NBD, or network block device. This allows a Linux system to 'share' out local devices over the network to be mapped on another system. The local 'device' in this case would be our RAID 1 virtual device. The system that would map these NBDs could be another Linksys node that aggregates them into a RAID 0 device. On that same node we'd run Samba or some other filesharing system. It sounds like a homebrew implementation of Coraid.

This will handle single drive failures, but the problem is if one node goes down, that makes one unit of the RAID 0 inaccessible, and the RAID 0 will fail completely. One option is to use RAID 0 striping on the local node with 2 drives. Then the fileserving node would RAID 1 each pair of nodes, and finally RAID 0 stripe over the pairs. I guess this would be termed a RAID 0+1+0 setup . That might be overkill for a home setup where you can tolerate a bit of downtime to replace a node (at least your data is still safe).

There is another driver called ddraid, but I'm not exactly sure how this would differ from an NBD RAID, and it is documented poorly.

An alternative to NBD is using a clustering filesystem. There are a few for Linux, but many seem to target big-spenders, and have high learning curves and jargon. The most simple and straightforward ones I found were Lustre and PVFS2. I like PVFS2 because it's documentation is easy to follow. It seems to provide easy aggregation of data storage on nodes, allowing addition and removal. It seems to create a virtual directory that is just the aggregate of all nodes. It does not provide redundancy on its own, but using software RAID and failover tools it is possible. I'm a bit confused to how it handles low disk space issues though.

Using RAID 0, I don't believe it would be easy to add more storage as you go. Maybe just using RAID 1 on the nodes, and then aggregating them with NBD and LVM would be enough. LVM would let us grow the filesystem as we added more nodes, and the mirroring saves in case of 1 drive failure.

I think creating a farm of slugs, as these Linksys units are referred to, would be a very cool home project to work on. I hope I can get some time.