2

I've been setting a web server on Amazon EC2 over the past couple of weeks (Ubuntu 8.04). Because I want to my site to be load balanced over multiple web-servers at some point I've placed all my data on an EBS volume. In time I can attach this volume to a file server and share it between all web servers.

Anyhow, currently my EBS volume is EXT3. I simply choose this file system because I'm most familiar with it and haven't had any problems with it in the past. I was wondering if there is another file system that anyone could recommend that would perform well on a server type set up?

Data integrity is of course of vital importance and I'm willing to hand in performance in exchange for this.

Luke
  • 3,856
  • 8
  • 36
  • 42
  • I hope you're not planning on hooking that EBS up to multiple EC2 instances at once, because by gum that ain't gonna fly using ext3 (or XFS, or reiserfs, for that matter). – womble Jun 10 '09 at 06:01
  • That's true, @womble. In that case, you'd need a cluster-aware filesystem like GFS and probably some sort of cluster-aware software for the lock manager. – staticsan Jun 10 '09 at 06:21
  • As far as I know AWS only allows you to mount a single volume on a single EC2 instance. I'm planning on attaching it to a file server (one EC2 instance) where my web servers talk to. – Luke Jun 10 '09 at 06:45

3 Answers3

3

Ext3 is a very good, all-purpose filesystem suitable for the vast majority of things you could use a Linux server for. It has only a few known performance quirks, the main one is deleting tens of thousands of files in a hurry. It is also not recommended for storing lots of very large files, such as motion video as used in a DVR system, or lots of very small files (tens of thousands) in a single directory.

Alternatives which cope with those extremes better are XFS and JFS, but if you're not likely to need any of those three scenarios, then ext3 will be fine. (ReiserFS also handles the lots of small files problem and deleting many many files at once problem quite well, but doesn't handle lots of huge files all that robustly.)

And if you're doing disk-heavy activities where speed of the filesystem is going to matter, then EC2 is going to be a bigger problem than the filesystem.

staticsan
  • 1,539
  • Thanks for that. Can you elaborate on the EC2 thing you mention at the end of your answer? I curious to know of why. – Luke Jun 10 '09 at 05:51
  • An EC2 machine is a virtual machine. And the stated spec of an EC2 compute unit is far from a cutting-edge hardware equivalent. – staticsan Jun 10 '09 at 05:56
  • EBS can be quite variable in it's performance -- blazingly fast one minute, glacially slow the next. It's the nature of the virtualised beast. – womble Jun 10 '09 at 06:00
  • 1
    +1. Exotic filesystems have their uses, but it's best to pick boring old ext3 if you can get away with it. – Alex J Jun 10 '09 at 06:30
  • @staticsan. I understand the virtual nature of AWS and I'm fine with that. I thought there might have been implications I didn't know about. Thanks for your answer! – Luke Jun 10 '09 at 06:47
1

EXT3 is hard to beat. There are other nice journaling filesystems, but from a stability standpoint, best to go with tried-and-true. There are known performance issues with storing lots of files in a directory (say, more than 10,000) but some of those issues have been addressed in the latest kernels and will not normally be an issue. And while you're waiting for Ext4, keep in mind that there are many, many tools out there that are Ext2-aware, which means they can pick up a Ext3 filesystem and run with it.

XFS is fast and handles large files quite well, but it does have a tendency to zero out sectors upon a journal recovery. While this may be perfectly acceptable in video and audio stream editing, it might cause grief in other environments. (Don't get me wrong, I like XFS alot, I just think people need to understand the implications of the filesystem they are using).

Reiser3 (and 4) are both really good with small files. The downside to Reiser is that if you have multiple Reiser filesystems on the same volume, and you somehow loose your partition table info, for the love of all things don't try to repair your R3 filesystem - the fsck for Reiser basically scans the disk looking for what appears to be the start of the filesystem, and then rebuilds it from there. If it happens to find a different filesystem than you intended, oh well, it just stitches them together anyways. In other words, it can easily be confused.

Avery Payne
  • 14,664
1

Ext3 is really really slow with large files and many files in the same directory. I don't know how this shows up in EC2, but the seek times really kill me when I have large files on a physical disk.

This is due to it not having extents. A seek to the end of a 100GB file will require a whole bunch of seek-read-seek-read which kill you on a physical disk.

I almost always run XFS. JFS is fast too, but I've found it buggy. It's been a few years since I ran it though so it may be fine now.

XFS has online defrag too. That doesn't mean it's more susceptible to fragmentation than ext3. But when you get your fs fragmented the best way on ext3 is a backup-restore to "defrag". There is a ext2/ext3 defrag, but nobody uses it so I wouldn't count on it. You can count on XFS defrag. Again, I don't know if this matters for EBS since both may end up as fragmented files on a SAN somewhere anyway, and a defrag would cost you in data operations.

But that's all performance.

I've never lost data with XFS. And I've run it a lot. I've lost data with (and due to) UFS (OpenBSD.. agh), JFS, ReiserFS and ext2 (the last being non-journaled). And I've run XFS more and heavier than any of the others.

But I really long for ZFS or similar on Linux.

Final word: ext3 is fine stability wise. If it's good for you performance- and feature-wise, then stay with it.

Thomas
  • 1,516
  • As a side note i've lost a lot of data with ext3 due to power outage and found now way to get the data back. Since this heavy disappointment my own hardware runs XFS and, same as you, i'm eager to have ZFS or similar. – buster Jun 10 '09 at 08:04
  • @buster: You use XFS to avoid data loss? Oh... kay... – womble Jun 10 '09 at 10:45