unRAID Project Roadmap Announcements

Protection from power spikes taking out large chunks of your array is ONLY satisfied with backups.  Unraid is not a backup, it is fault tolerance.  Another unraid server (ideally off-site) can be your backup solution, but unraid in and of itself is not a backup. 

Oh yeah and if you are worried about power spikes in your particular power grid system then you better be using true online UPS protection possibly with supplemental power line conditioning.

But of course this then becomes a full blow risk reduction discussion which is not what it is meant to be. It is a discussion about the best way to implement functional fault tolerance such that recovery from a typical use case failure is not at risk of suffering another failure before the recovery is complete.  That is why dual-parity is so desired by anyone running very dense arrays.

Long ago, I suggested a model for multiple arrays, and I'd like to push it a bit again.  If UnRAID supported multiple arrays, of differing types, it would be possible to run 3 arrays in that 24 bay machine, each with its own parity drive and 6 or 7 data drives.  Rebuilds would be much safer, and a little quicker, because any drive failure is isolated to a single smaller array, won't affect the rest of the drives.  Parity checks would be a little quicker, fewer drives per array.  Multiple drive failures can still happen, but would be less likely to cause data loss.  Multiple arrays give you multiple parity drives with some (but not all) of the advantages of dual parity, but without the performance loss.  Multiple arrays will certainly require significant development hours, but once done, opens a variety of advantages.

I hadn't thought about that, but it makes a lot of sense in the unraid context since there's a software layer logically stitching together the user filesystem anyway.  I'd definitely go for this.  My main hope is to get close to some sort of magical 5:1 data:parity ratio anyway.  I'm running out of room to do that right now at home anyway. 

I have no idea how much work would have to go into making it happen, but it *seems* like it might be easier to accomplish than dual parity, without as much computational overhead and *most* of the advantages.

Protection from power spikes taking out large chunks of your array is ONLY satisfied with backups.  Unraid is not a backup, it is fault tolerance.  Another unraid server (ideally off-site) can be your backup solution, but unraid in and of itself is not a backup. 

 

Oh yeah and if you are worried about power spikes in your particular power grid system then you better be using true online UPS protection possibly with supplemental power line conditioning.

 

But of course this then becomes a full blow risk reduction discussion which is not what it is meant to be. It is a discussion about the best way to implement functional fault tolerance such that recovery from a typical use case failure is not at risk of suffering another failure before the recovery is complete.  That is why dual-parity is so desired by anyone running very dense arrays.

I am so glad you wrote this and posted it.

Lime Tech why have you not written a GUI Backup so we can mirror one Unraid system to another Unraid system ?

There is a need for this to happen. The many Unraid users trying to implement Crashplan and Rsync using Linux commands is not getting the job done. How about insuring we can copy all our data to another Unraid machine .

It needs to be GUI driven and include scheduling, drive selection and  incremental or differential backups.

How nice it would be to be able to have two Unraid machines copying data automatically every night.

Stew

Will check with Tom on this today.  To be totally honest, I haven't discussed that feature with him yet, but maybe its something we can add to either 6.0 or 6.1.  If you want to save me a step, can you provide a link to a forum post with Tom where this feature was discussed before?

Any updates yet?

Thanks!

24 drive machines have always seemed risky to many of us, with the current 1 array model, and only one parity drive.  So you can understand the concern with the idea that a hot spare will be the answer to any disabled drive issue, especially with so many drives to abort the automatic rebuild.  I suspect many of us have had a rebuild failure at some point, not often, but common enough with our average size arrays to be concerned about greatly increasing the drive count.  I wish we knew the numbers on how risky it is to rebuild a drive in a 24 drive array, but the failure rate is certainly more than 0% and almost surely higher than 1%.  Is it as much as 20% or 30%?  Probably not, but we don't know.

 

Dual parity will help a lot, but it is going to come with a performance hit.  How much we don't know yet, but I predict some of the enthusiasm for the advantages of dual parity will wear off quickly, once the performance hit is known.  I do hope it's small, and that ways to overlap the increased I/O are found, to minimize the hit.

 

Long ago, I suggested a model for multiple arrays, and I'd like to push it a bit again.  If UnRAID supported multiple arrays, of differing types, it would be possible to run 3 arrays in that 24 bay machine, each with its own parity drive and 6 or 7 data drives.  Rebuilds would be much safer, and a little quicker, because any drive failure is isolated to a single smaller array, won't affect the rest of the drives.  Parity checks would be a little quicker, fewer drives per array.  Multiple drive failures can still happen, but would be less likely to cause data loss.  Multiple arrays give you multiple parity drives with some (but not all) of the advantages of dual parity, but without the performance loss.  Multiple arrays will certainly require significant development hours, but once done, opens a variety of advantages.

 

I think once you get up to that many drives in a single system, you are probably stretching things a bit on even just the system capabilities on most of the commodity hardware.

 

I like the idea of multiple arrays.  What would be nice is to have multiple protection groups each with a parity disk, and then still have them seen as a single filesystem.

I'm currently sitting at 10 drives, 9 data, and 1 parity for 24TB of space.  While I will continue to regularly swap out older, smaller drives for larger ones and grow my storage capability, I will never go beyond 10 disks.  With the SAS controller I have, and the six onboard sata ports I have more than enough connectivity for more drives.  I could add additional drives by buying additional cages, and a bigger power supply.

However, I recently lost a 2TB disk.  And during the rebuild I had another failure.  All in all I was able to recover everything and expand the 2TB disk with a 4TB disk.  However, the more disks we add, the more risk we add.  I think this is a perfect solution to put into place by allowing at least a secondary array to be added.

FWIW, I would have no issues paying for this feature.  Make it a Pro+ upgrade and charge me another $25.  I am perfectly fine with this as it keeps me from having to build another unRaid host, and manage it.

Thanks!

24 drive machines have always seemed risky to many of us, with the current 1 array model, and only one parity drive.  So you can understand the concern with the idea that a hot spare will be the answer to any disabled drive issue, especially with so many drives to abort the automatic rebuild.  I suspect many of us have had a rebuild failure at some point, not often, but common enough with our average size arrays to be concerned about greatly increasing the drive count.  I wish we knew the numbers on how risky it is to rebuild a drive in a 24 drive array, but the failure rate is certainly more than 0% and almost surely higher than 1%.  Is it as much as 20% or 30%?  Probably not, but we don't know.

 

Dual parity will help a lot, but it is going to come with a performance hit.  How much we don't know yet, but I predict some of the enthusiasm for the advantages of dual parity will wear off quickly, once the performance hit is known.  I do hope it's small, and that ways to overlap the increased I/O are found, to minimize the hit.

 

Long ago, I suggested a model for multiple arrays, and I'd like to push it a bit again.  If UnRAID supported multiple arrays, of differing types, it would be possible to run 3 arrays in that 24 bay machine, each with its own parity drive and 6 or 7 data drives.  Rebuilds would be much safer, and a little quicker, because any drive failure is isolated to a single smaller array, won't affect the rest of the drives.  Parity checks would be a little quicker, fewer drives per array.  Multiple drive failures can still happen, but would be less likely to cause data loss.  Multiple arrays give you multiple parity drives with some (but not all) of the advantages of dual parity, but without the performance loss.  Multiple arrays will certainly require significant development hours, but once done, opens a variety of advantages.

 

I think once you get up to that many drives in a single system, you are probably stretching things a bit on even just the system capabilities on most of the commodity hardware.

 

I like the idea of multiple arrays.  What would be nice is to have multiple protection groups each with a parity disk, and then still have them seen as a single filesystem.

 

I'm currently sitting at 10 drives, 9 data, and 1 parity for 24TB of space.  While I will continue to regularly swap out older, smaller drives for larger ones and grow my storage capability, I will never go beyond 10 disks.  With the SAS controller I have, and the six onboard sata ports I have more than enough connectivity for more drives.  I could add additional drives by buying additional cages, and a bigger power supply.

 

However, I recently lost a 2TB disk.  And during the rebuild I had another failure.  All in all I was able to recover everything and expand the 2TB disk with a 4TB disk.  However, the more disks we add, the more risk we add.  I think this is a perfect solution to put into place by allowing at least a secondary array to be added.

 

FWIW, I would have no issues paying for this feature.  Make it a Pro+ upgrade and charge me another $25.  I am perfectly fine with this as it keeps me from having to build another unRaid host, and manage it.

 

 

Thanks!

You know of all the times I've read someone say, "I'd even pay for that" this is probably one of the few times I'd say I agree.  Especially with the idea that it could be a distinguishing feature for the Pro or Pro+ license.  Personally I would probably never use it, but I can surely see the benefit and the "clutchness" of it in the SOHO market.

I can even image one simple way to implement it.  First of course turn-on discovery in each servers, then each share has an option to turn on backup, which servers to use (that could mean multiple), the type of backup (mirror, snapshot, etc), frequency, and anything else I can't think of.

I would also be willing to pay for new features if I consider them to be worthwhile.

I would also be willing to pay for new features if I consider them to be worthwhile.

I'm ready to buy 2 icy docks 5 in 3 cages, and upgrade my license =)  The old 2TB drives I'm replacing can still be used, for less critical data, due to age.  It just want to ensure the stuff I don't want to risk losing is on newer drives. 

Tom, how can we persuade you? =)

Question: Is there anything new or different in beta 7 that might help a Windows VM under Xen run more stable? What about KVM?

I have a windows 7 VM running on Xen with unRAID beta 5a that runs for weeks without a hiccup. On beta 6 it crashes every 12 to 24 hours. Converting it to KVM is useless due to hardware changes, so I did a a fresh install of Windows 7 on KVM with unRAID beta 6 and I have networking problems where I can never get the VM to use a static IP address. Its like every reboot causes network discovery issues and always falls back to DHCP mode. I haven't tried much else because beta 5a works so well.

Also, I have not tried Windows 8 either due to the amount of work required in reinstalling programs and initial setup. If Windows 8 is better than Windows 7 running in a VM (under Xen or KVM), I could try that instead. Note that I'm not doing any hardware pass through here. Just need a stable Windows VM.

Thanks.

Add me to those requesting an update to AFP. I use UnRAID as my TimeMachine source and every 2-3 months need to rebuild my backup from scratch due to inconsistencies. I get that AFP is on the way out, but even in 10.10 it will be required for TimeMachine use. This is one of my main uses of UnRAID, so please consider updating.

Add me to those requesting an update to AFP. I use UnRAID as my TimeMachine source and every 2-3 months need to rebuild my backup from scratch due to inconsistencies. I get that AFP is on the way out, but even in 10.10 it will be required for TimeMachine use. This is one of my main uses of UnRAID, so please consider updating.

Tom has since come out and confirmed that we are updating AFP.

Jonp/Tom

Are you guys able to share any details on a beta7 eta. You guys provided a Q3 target for RTM which at the time gave a 3 month window. We are now through the first month, but haven't really seen any updates from you guys or updated betas.

Do you still believe you are on target for Q3? Do you have an eta on beta7? I know one of you mentioned you were doing internal testing a week or more back, but it would be great to know where everything stands.

Jonp/Tom

 

Are you guys able to share any details on a beta7 eta. You guys provided a Q3 target for RTM which at the time gave a 3 month window. We are now through the first month, but haven't really seen any updates from you guys or updated betas.

 

Do you still believe you are on target for Q3? Do you have an eta on beta7? I know one of you mentioned you were doing internal testing a week or more back, but it would be great to know where everything stands.

It's coming man.  I was hoping to get beta7 out last Friday, now shooting for this Friday (today), may spill into the weekend.  We are working as hard as possible...

Jonp/Tom

 

Are you guys able to share any details on a beta7 eta. You guys provided a Q3 target for RTM which at the time gave a 3 month window. We are now through the first month, but haven't really seen any updates from you guys or updated betas.

 

Do you still believe you are on target for Q3? Do you have an eta on beta7? I know one of you mentioned you were doing internal testing a week or more back, but it would be great to know where everything stands.

 

It's coming man.  I was hoping to get beta7 out last Friday, now shooting for this Friday (today), may spill into the weekend.  We are working as hard as possible...

Fair enough. Glad to hear we are close.

ROFLMAO!

That made me laugh so hard!!!

That needs to be the new LimeTech logo.  Or at least a pop up on an unRAID help or about screen

So glad to see that AFP is being updated.  I too point my TimeMachine to unraid and it fails to connect on a regular basis.  I have to remove the config from my imac and start over - PITA.

Any idea if this new version will go to Release Candidate, or will it go straight from BETA to Final?

Thank you! 

Add me to those requesting an update to AFP. I use UnRAID as my TimeMachine source and every 2-3 months need to rebuild my backup from scratch due to inconsistencies. I get that AFP is on the way out, but even in 10.10 it will be required for TimeMachine use. This is one of my main uses of UnRAID, so please consider updating.

 

Tom has since come out and confirmed that we are updating AFP.

@limetech maybe good will be add to unraid fstrim daemon ??

https://github.com/Saner2oo2/fstrimDemon

So glad to see that AFP is being updated.  I too point my TimeMachine to unraid and it fails to connect on a regular basis.  I have to remove the config from my imac and start over - PITA.

I've always found that rebooting the server clears up my time machine issues. 

which is why it is a server issue - and why we are glad it is being fixed.  You shouldn't have to reboot your server to fix it.

Jonp/Tom

 

Are you guys able to share any details on a beta7 eta. You guys provided a Q3 target for RTM which at the time gave a 3 month window. We are now through the first month, but haven't really seen any updates from you guys or updated betas.

 

Do you still believe you are on target for Q3? Do you have an eta on beta7? I know one of you mentioned you were doing internal testing a week or more back, but it would be great to know where everything stands.

 

It's coming man.  I was hoping to get beta7 out last Friday, now shooting for this Friday (today), may spill into the weekend.  We are working as hard as possible...

Possible to have a release this friday (today)?

Sent from my SM-N9005 using Tapatalk

now shooting for this Friday (today), may spill into the weekend.

Which weekend is that?