4+4... 2x4... and other 8TB algebra

So, the second preclear cycle for 8TB Seagate Archive drives completed successfully (preclear the 8TB Archives via USB 3.0 card speed example), and, not having patience for the third one, I started playing.

The idea was to use 4TB+4TB, or 4x2TB RAID-0 pool for parity and 8TB Archive drives for data.

Well that does not work so far. Edited: yes it works! See further in the thread.

Test setup: Supermicro H8DME-2 with some low-end Opteron and 4GB RAM, unRAID Trial 6.0-beta14b.

Available drives are: 3x8TB Seagate Archives, 2x4TB Seagates, 4x2TB HGSTs/Samsungs... and one 2.5" Hitachi of size 160GB 

Available RAID cards are:

PCI-X:

3Ware AMCC 9550SX-12SI 12 Port SATA PCI-X RAID (teeested - toooooo sloooow?)

Areca ARC-1110 4 Port 64Bit 133MHz PCI-X SATA II RAID fw 1.39 Works 4x2TB=8TB parity, with capacity truncation disabled

Areca ARC-1110 4 Port 64Bit 133MHz PCI-X SATA II RAID firmware updated to 1.49 Works 4+4TB=8TB parity, with capacity truncation disabled

Adaptec Dell 6-Port SATA RAID AAR-2610SA 0H2052 (not tested yet, how to select drives?)

LSI SAS3041X-R 4-Port PCI-X-133 SAS/SATA 3Gb/s RAID  (not tested yet, how to enter setup?)

PCIe x2:

Addonics AD4SA6GPX2 4-port PCIe x2 Test post (no spanning (!), RAID only)

Areca ARC-1210 4 Port PCI-e x8 SATA II RAID fw 1.49 Works 4+4TB=8TB parity, with capacity truncation disabled

On-drive:

3 x Addonics AD2HDDHP6G Drive mount 2-Port 6G HPM Addonics AD2HDDHP6G tests post

On-bracket:

SYBA SY-PCI40037 1:5 SATA II Dip Switch RAID Tests post, results similar to Addonics "on-drive"

PCI:

Sunweit XWT52 PCI SATA 150 RAID card 4 ports (in the mail from China, 3 weeks?)

Preliminary (2015-03-24) conclusions:

User Capacity of 8TB Archive disk is 8,001,563,222,016 bytes. For parity we need to create same or bigger RAID-0 volume. For 4TB+4TB or 4x2TB, the only way to achieve it is by using an Areca card - it's the only card (so far, out of tested) which allows to disable "capacity truncation". Other cards will silently truncate size (round down to nearest 1GB or 10GB, here is why), RAID pool will become smaller than the 8TB Archive data drive, and unRAID will not accept it.

I don't have immediate need for this hardware, so I will continue playing. If some of you would like me to do some specific tests - I will do my best.

Very interesting on the sizes.  Definitely surprised that a 2 x 4TB RAID-0 shows as "smaller" than the 8TB drives.

In any event, I'd expect the parity sync to maintain good speed all the way to completion.  Considering that the writes are all sequential, the drive's firmware should recognize that all sectors in every band are being written, so there shouldn't be any need for band rewrites; and since everything's sequential, the persistent cache won't be used at all.

... thus it will behave just like a PMR drive for this.

The real test will be doing a LOT of random writing to the array (well over the 25GB that the persistent cache can manage).    The problem is that initially everything you write will be sequential, since the drive is empty.  So you'll need to get a significant amount of data on the drive (perhaps 500GB or so);  then delete a good bit at random ... and then write a LOT of small files (so they don't cause full band rewrites by themselves) that total well over the 25GB in the persistent cache.

This may be difficult to do without several data disks that would let you write to different disks (thus causing writes at various locations on the parity drive) => the reality is performance will probably be just fine UNTIL the array has enough "randomness" in the allocations that the persistent cache gets filled during a batch of writes ... at that point performance will probably be abysmal.

... unfortunately with this size drive it's going to take a LONG time to really test it well.    On the other hand, if it's difficult to "hit" the parameters that cause performance to drop drastically that's a good thing ... it means that in typical UnRAID usage the drives might just be fine 

Hi, just did 2x2TB raid0 for my server about a month ago with two RAID cards - HP P400 and DELL Perc H710P. and in both cases 2x2TB in size was smaller than 4TB drive. solution was very simple and are discussed on the forums - make a small HPA on your 8TB drive i have a long story with migrating to that setup cos setting up HPA, you loose any existing data, so do it only with empty drives!!

Hi, just did 2x2TB raid0 for my server about a month ago with two RAID cards - HP P400 and DELL Perc H710P. and in both cases 2x2TB in size was smaller than 4TB drive. solution was very simple and are discussed on the forums - make a small HPA on your 8TB drive i have a long story with migrating to that setup cos setting up HPA, you loose any existing data, so do it only with empty drives!!

Great idea, thanks! I'll definitely try that for testing, although I'm kinda reluctant to do thing like that on a production server - I somehow don't like setup in which I wouldn't be able just to throw in replacement drive and would have to castrate it a little create an HPA on it first...

Very interesting on the sizes.  Definitely surprised that a 2 x 4TB RAID-0 shows as "smaller" than the 8TB drives.

I was half-expecting something like that... but was still hoping. Well, no luck with straightforward, but I could always use 3x3TB=9TB for parity with 8TB Archives for data. I'm currently thinking how can I root up three 3TB drives from my "production" servers... I mean, I can easily do this if I install the two 4TB instead, but that means I wouldn't be able to play with them in the Testt server... need to think more.

A tiny HPA would work very nicely, but I agree it'd be nice if you didn't have to do that.  That would have a couple of consequences:

=>  In the future, when the drive fails, it's the kind of thing that's easy to forget.  So you'd pop in the replacement drive; try to Start the rebuild; get a message about "too small";  say a few choice words;  create the HPA; and then everything would be fine      Clearly not as nice as just popping in a replacement.

=> Doing that on the parity drive also means you'd have to create the same little HPA on all of your 8TB data drives (with the same consequences for replacements)

Simply a better idea to have a "big enough" parity drive => the 3 x 3TB sounds like the way to go; although you could use one of the nifty little boards discussed on another thread that mount to the back of a drive and let you combine two drives to create a 9TB drive from a 4TB plus a 5TB drive.  Depending on how you use your array, however, you may find that the 8TB SMR drive is okay as parity.

Not true about Parity - if a Raid0 array used for parity fails, you could use a whole 8TB replacement drive as it will be a largest drive

just try this HPA way on your test server to test what sweets best for you..

i did that cos of performance - i have 4TB Seagate 5900RPM data drives, and with very fast Raid0 parity i can write to disk shares at full speed of these Seagates now.

... Not true about Parity

Yeah ... I had a senior moment 

It would, of course, be true that you'd have to do it to all the data drives IF you did it to parity -- but clearly it wouldn't be necessary to do that if you simply used one of the 8TB drives as the parity drive.

Yes and we are just waiting for more tests with these new drives - as parity and data drives as well and with Raid0 2x4TB as parity too

thanks in advance!

My Areca 8T RAID0 is 7,814,036,428 k (taken from myMain detail view under "Size (k)" column).

Some RAID cards will round down space to the nearest 1G or 10G. I disabled that feature on my card. Not sure if you have a similar option on your card.

Please post the exact size in k (see above how I measured it) and report for an actual 8T drive for me. (Thanks!)

The RAID "round down" is a residual feature from years ago when drives of the same size often had different exact byte counts ... so RAID cards would round down a bit to ensure that if you mixed drives (different make/model) you wouldn't have a problem with the array construction.

It's been a few years now since I've seen drives with different byte counts -- I guess all the major makers "got their act together" and use the same exact sizes now ... any differences are likely in the number of spare sectors.

I'm surprised, however, that the 8TB is apparently larger than 2 x 4TB.    I'm fairly certain that's not true for smaller sizes -- i.e. I think 2 x 1TB > a single 2TB;  2 x 2TB > a single 4TB; 2 x 3TB > a single 6TB; etc.

The RAID "round down" is a residual feature from years ago when drives of the same size often had different exact byte counts ... so RAID cards would round down a bit to ensure that if you mixed drives (different make/model) you wouldn't have a problem with the array construction.

 

It's been a few years now since I've seen drives with different byte counts -- I guess all the major makers "got their act together" and use the same exact sizes now ... any differences are likely in the number of spare sectors.

 

I'm surprised, however, that the 8TB is apparently larger than 2 x 4TB.    I'm fairly certain that's not true for smaller sizes -- i.e. I think 2 x 1TB > a single 2TB;  2 x 2TB > a single 4TB; 2 x 3TB > a single 6TB; etc.

My RAID0 weighed in a something like 8001.6 GB. I think it will turn out to be larger than the single 8T drives.

... although you could use one of the nifty little boards discussed on another thread that mount to the back of a drive and let you combine two drives to create a 9TB drive from a 4TB plus a 5TB drive. 

BTW, they already arrived (fantastic fast shipping!) so they will be soon for torture. Unfortunately I don't have any 5TB and don't want to buy one... but I was going to try anyway if those on-drive could be daisy-chained.

Depending on how you use your array, however, you may find that the 8TB SMR drive is okay as parity.

So far it looks like quite a viable option. I still, however, like the idea of reusing my 2TB drives in pooled parity. There are 4 of them, and will be many more... I can't just throw them away, gotta find some use even if only for playing...

Hopefully that will be true => but it's not likely unless the Addonics RAID card PKN tried is truncating the size of the component drives.

It'd be nice if that proves to be correct, however 

My Areca 8T RAID0 is 7,814,036,428 k (taken from myMain detail view under "Size (k)" column).

 

Some RAID cards will round down space to the nearest 1G or 10G. I disabled that feature on my card. Not sure if you have a similar option on your card.

 

Please post the exact size in k (see above how I measured it) and report for an actual 8T drive for me. (Thanks!)

I did not install unMenu on this server, can I do it some other way? I have keyboard/monitor attached, so can login.

... I can't just throw them away, gotta find some use even if only for playing...

Do you have a complete set of backups?    Old, smaller drives work well for backups.  I keep a complete set of backups on 1-3TB drives stored in these: http://www.amazon.com/DriveBox-3851-0000-11-Hard-Disk-Case/dp/B004UALLPE

My Areca 8T RAID0 is 7,814,036,428 k (taken from myMain detail view under "Size (k)" column).

 

Some RAID cards will round down space to the nearest 1G or 10G. I disabled that feature on my card. Not sure if you have a similar option on your card.

Don't know yet, my Areca ARC-1110 turned out to have firmware 1.39, and DOS-mode flash utility needs at least 1.43 to flash to 1.49, so I learned and learned, now my bet is on CLI, but I'd like to wait until parity sync completes, in about 2.5 hours, wanna see the resulting numbers.

... I can't just throw them away, gotta find some use even if only for playing...

 

Do you have a complete set of backups?    Old, smaller drives work well for backups.  I keep a complete set of backups on 1-3TB drives stored in these: http://www.amazon.com/DriveBox-3851-0000-11-Hard-Disk-Case/dp/B004UALLPE

Almost complete, like 99% is on optical discs of various kind. And I'm having a heart attack with one thought of ripping it all again 

I did not install unMenu on this server, can I do it some other way? I have keyboard/monitor attached, so can login.

Assuming you're running v6 you can simply get the size info from the main GUI by clicking on the drive -- that will show the detailed SMART info, including the exact byte count.

... And I'm having a heart attack with one thought of ripping it all again 

Which is why it's a good idea to have a complete set of backups 

Sounds like you have an excellent use for all those old 2TB drives 

I did not install unMenu on this server, can I do it some other way? I have keyboard/monitor attached, so can login.

 

Assuming you're running v6 you can simply get the size info from the main GUI by clicking on the drive -- that will show the detailed SMART info, including the exact byte count.

Great, thanks! Here what I've got for the 8TB Archive installed as data drive:

Disk 1 attached to port: sdd
Device Model:	ST8000AS0002-1NA17Z
Serial Number:	Z840120L
LU WWN Device Id:	5 000c50 079f747c3
Firmware Version:	AR13
User Capacity:	8,001,563,222,016 bytes [8.00 TB]
Sector Sizes:	512 bytes logical, 4096 bytes physical
Rotation Rate:	5980 rpm
Device is:	Not in smartctl database [for details use: -P showall]
ATA Version is:	ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:	SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:	Fri Mar 20 08:25:30 2015 PDT
SMART support is:	Available - device has SMART capability.
SMART support is:	Enabled
SMART overall-health :	PASSED

Okay, Brian's 7,814,036,428 k (from MyMain) should be that x 1024 bytes = 8,001,573,302,272 bytes, which is indeed larger than your 8TB.

That's good news => it means with a controller that doesn't truncate the sizes, 2 x 4TB are indeed > the 8TB Seagate, and should work fine as parity.

Brian:  Can you confirm that your actual byte count matches that?

pkn => Did you by chance look in the BIOS for the Addonics card to see if there's any option about whether or not to truncate the drives?    You already noted that the actual options don't match what's on the website marketing description -- but just in case, the web site shows there's a "Safe" mode ... which may be the automatic truncation (there's no description I can find that details what it means).    If that happens to be a setting in the card's BIOS, disable it, and see if that changes the size of your RAID array.

pkn => Did you by chance look in the BIOS for the Addonics card to see if there's any option about whether or not to truncate the drives?    You already noted that the actual options don't match what's on the website marketing description -- but just in case, the web site shows there's a "Safe" mode ... which may be the automatic truncation (there's no description I can find that details what it means).    If that happens to be a setting in the card's BIOS, disable it, and see if that changes the size of your RAID array.

The BIOS setup for this card is rather minimalistic - not to say primitive - and I'm pretty sure there was nothing like that. "Safe" is supposedly one of the RAID "modes", in the same menu with RAID-0, RAID-1, grayed out too, BTW. Will double check though.

... Some RAID cards will round down space to the nearest 1G or 10G. I disabled that feature on my card. Not sure if you have a similar option on your card. ...

Haha!    IT WORKED!  Thank you very much - yes, that was the reason. I disabled "Capacity truncation" in Areca setup and it worked.

So, the setup:

4x2TB=8TB parity in RAID-0 pool by Areca ARC-1110 in PCI-X 100/133 MHz slot.

8TB Archive as data drive in motherboard SATA port.

unRAID accepted.

unRAID reported parity-via-4x2TB-Areca: User Capacity: 8,001,594,327,040 bytes [8.00 TB]

Data drive 8TB Archive: User Capacity: 8,001,563,222,016 bytes [8.00 TB]

Created volume is seen by BIOS and unRAID under name SAreca_ARC-1110-VOL#000000001369145745

"Free falling" speed of writing (parity sync with single 160GB data drive, after the 160GB mark) by Areca into 4x2TB=8TB parity RAID-0 pool is about 250 MB/s. Not as impressive as 340, but good enough, I think.

Looks like to pull out this trick one needs more sophisticated RAID card, like Areca, which provides user with ability to disable "size truncation". Primitive card, like Addonics AD4SA6GPX2, will not work - it will silently truncate size, RAID pool will become smaller than the 8TB Archive data drive, and unRAID will not accept it. 

Can't check the 4+4TB configuration right now, because Areca card still needs firmware update (from current 1.39 to 1.49) to understand drives > 2.2TB. Summation (2015-03-23) of what I know about Areca firmware update: there are 3 ways: DOS utility nflash, CLI (command line interface) utility, and web-based manager called McRAID. CLI and nflash both can not work with firmware below 1.43, so my only option is McRAID. Download archttp64 (Areca http proxy server), and run it on unRAID server, then you can access it at http://my-server-ip:81. Cards config is at http://my-server-ip:82 and in my card this is closed by login/password, Arecas default admin/0000 don't work. Contacting tech support.

To leave it on a happy note all other tests will have to wait till tomorrow, so I configured 4x2TB=8TB parity, single 8TB Archive as data, launched parity sync and went to watch movies