ZFS gets inline dedupe

[NAS]

Full article here

 

http://www.channelregister.co.uk/2009/11/02/zfs_gets_dedupe/

 

Potentially very very useful.

[Rajahal]

Along these lines, can some present a list of pros and cons of ZFS vs. unRAID?  When I first decided to go with unRAID, ZFS was still immature and largely unsupported.  However, it seems the landscape is changing now.

[NLS]

ZFS immature?

 

We should tell Sun to stop using it in their tiny little Solaris (even deployed in enterprise environments).

 

How do you compare a filesystem (built to cross the boundaries of all other FS) to a nice trick like unRAID is?

 

 

[Rajahal]

Maybe I don't fully understand it.  I know ZFS is a file system, but I also know that whenever I mention unRAID to more knowledgeable people (outside of the unRAID community), they scoff at me and start talking about how ZFS is the end-all-be-all of RAID setups.

 

If ZFS is just a file system, is there any chance of unRAID implementing it in the future?

[WeeboTech]

Maybe I don't fully understand it.  I know ZFS is a file system, but I also know that whenever I mention unRAID to more knowledgeable people (outside of the unRAID community), they scoff at me and start talking about how ZFS is the end-all-be-all of RAID setups.

 

If ZFS is just a file system, is there any chance of unRAID implementing it in the future?

 

ZFS is a filesystem.

unRAID is a raid architecture with parts borrowed from RAID4 and RAID5.

A filesystem is still requried on top of the unRAID MetaDevice.

Hence ReiserFS.

 

If ZFS were to be open source without license issues, I'm sure it would get implemented in linux and subsequently in unRAID.

[NLS]

Well Weebo nails it, although the part of being sure that unRAID would use it if ZFS was open source is a big discussion :D ...it's also a "free" statement as ZFS is not open source.

 

Another free comment (my turn) is that if ZFS was open source Tom (sorry "LimeTech" I mean) would possibly add it for release 6.X due 2137.

 

Let's not open the "underlying FS of unRAID" discussion again.

 

 

[Rajahal]

So ZFS is free but not open source?  Also, any standard RAID5/6 server can be configured to use ZFS, right?

[WeeboTech]

I stand corrected.

I remember there was something holding it all up.

 

But there's a catch. ZFS, like the rest of OpenSolaris, has been available under Sun's Common Development and Distribution License for about two years, and Linux has been under Version 2 of the the GNU General Public License since its first release in 1991. The licenses are incompatible.

 

http://www.linuxworld.com/news/2007/061807-zfs-on-linux.html

http://en.wikipedia.org/wiki/ZFS

 

There also is something called ZFS-fuse. I have not looked into this much yet.

[ReneV]

zfs and unRAID are orthogonal technologies with different aims and means. It makes no sense to talk of unRAID w/zfs or vice versa. 

 

zfs is an integrated storage management system, rather than just a file system. For example, zfs devices are not formatted as such and it's a strong recommendation never to assign just a partition to a zfs pool.

 

I run both at home, and am profoundly happy and grateful for both: unRAID for headless AV backup on an ever-changing eco-system of drives, and zfs for all other storage needs as part of a workstation.

 

One concrete example is that I run my virtual machines off of a 3-way zfs pool using the OCZ USB+eSATA devices [EDIT: the devices are called `Throttle']. With compression on the zfs pool, I get >480MB/s for "dd if=virtual.machine of=/dev/null".

 

Conversely, when it comes to unRAID's strengths, one would expect all/most physical HDDs in a zfs pool to be active during writes and non-cached reads and, IIRC, zfs insists on equal-sized devices for all parity-backed situations, meaning that no device is used beyond the limits of the smallest device it's paritied against.

[starcat]

ReneV, you could onle swap the smallest paritied device to a larger and of the same size as the others. However I agree, it is best havinf equal sized devices in a disk set. But then you can have two and even three parity disks for that set. If you grow, then just add another disk set to the pool. I am sure you know all this.. it is just cool as upgrading in set of disks makes sense to me at least. However ZFS don't spin down single devices in a pool and this is almost a nogo in a home environment unless you have a dedicated airconditioned server room :-) But in exchange to this one gets features like no RAID write hole issues, integrated data integrity checks and recovery, online dedup, online encription, unlimited expandibility (pair this with SATA disks hanging off of SAS port expanders) to name just a few...

[ReneV]

[on zfs' strengths]

 

What I wanted to say (and probably didn't quite manage to bring out) is that it is easy (and justified) to be impressed with the cool technology that zfs delivers, but that that shouldn't prevent us from appreciating the simplicity and fitness-for-purpose of unRAID. Sure, the ability to use old, slow, and even obsolete HDDs is not illustrious but, at least for me, the fact that Lime Technology delivers this and more through the simple premise of parity-but-not striping should win the company an award for insight and sheer beauty in technology 

[NLS]

Well Rene this doesn't stop ZFS from deserving a thread in this forum.

 

Also although I agree with what unRAID delivers, it's not like it was a patent or anything.

It's not like Tom thought of this where nobody else did (in fact I had this "unRAID" like idea more than 10 years ago and it's not that I am too bright, I've heard other had this idea more or less too). It's just that Tom made a product out of this and indeed fits a certain market.

Now many of us are waiting to see the Next Step (since basically since unRAID's first version new "revolutional" additions are few and far apart - if unRAID was MS Windows, then it would be a fictional Windows NT 4.4.2 and we still wait to see 2000, Vista, or 7).

 

Presenting exciting new related technologies within this forum is both interesting and useful (to many directions).

 

 

[ReneV]

Well Rene this doesn't stop ZFS from deserving a thread in this forum.

 

You are arguing against a straw man of your own making. I have at no point argued against the mentioning of zfs on this forum. Indeed, the lounge is explicitly for any storage-related discussion but, even if it wasn't, it would not be my place to moderate anything.

 

Also although I agree with what unRAID delivers, it's not like it was a patent or anything.

 

That is factually correct. So?

 

It's not like Tom thought of this where nobody else did (in fact I had this "unRAID" like idea more than 10 years ago and it's not that I am too bright, I've heard other had this idea more or less too).

 

I can't even bring myself to say how stupid you sound here!

 

Now many of us are waiting to see the Next Step (since basically since unRAID's first version new "revolutional" additions are few and far apart - if unRAID was MS Windows, then it would be a fictional Windows NT 4.4.2 and we still wait to see 2000, Vista, or 7).

 

OK. (Feeling entitled, are we?!)

 

Presenting exciting new related technologies within this forum is both interesting and useful (to many directions).

 

Sure. What's your point?

[NLS]

I am not sure how to answer your questions (being stupid an' all)...

 

I don't even see how my points raise questions at all.

(but we cleared already that this comes from my own stupidity)

 

Hope someone takes this thread back to ZFS.

 

[limetech]

ZFS is a good technology, but like any technology, it has it's strengths and weaknesses.  Same is true of unRAID.  Does it make sense to put ZFS on top of unraid driver?  Probably not, but maybe by 2137 we can try it, and find out it doesn't make sense.

 

On a side note:  For the most part the unRAID Server Community forum has been an extremely helpful and friendly place.  I receive emails all the time from people who tell me as much.  From time-to-time I also receive emails about nastiness in the forums & that I should do something about it.

 

I almost never delete forum posts except for obvious spam that is brought to my attention (thanks to all of you who do this BTW).  However it does become difficult to follow certain threads, in the limited amount of time I have to do so, which start veering wildly off topic.  This has been especially true in the 'beta' release announcement threads & I recently posted some Announcement Board Rules which warns that off-topic posts may be deleted without warning.

 

Next unfortunate step I need to take is to start deleting 'unfriendly' posts & banning 'unfriendly' users.  If you want to post snarky comments on internet discussions boards, try reddit, I will no longer tolerate it here.

 

 

[NAS]

My advice for what its worth...

 

from someone who owned and operated a forum with multiple thousand active members for several years, understand that one determined, disgruntled banned member can cause much chaos. So much that your new job will be one of perpetual defence.

 

I say this with all humility, with such a policy you are going to need some moderators cause one man cant moderate an entire forum and continue to do two jobs. Otherwise you are going to have a scenario of banning what you see or worse banning what people report to you (aka a good way for members to retaliate at other members.)

 

I read over 90% of all posts here and you don't need a policy this heavy handed. Almost every instance of this kind of talk has been about Limetech being MIA.

 

No offense intended its just an observation from a daily visitor. Feel free to ignore.

 

I think its time this thread was closed.