Calculating parity check speeds

[Kaygee]

Been doing some performance testing with various HDDs combinations and HDD controllers.

 

Seems parity check performance is fairly predictable (assuming not bottlenecks or errors occur).

 

My system with 4 Samsung HDDs parity checks at 78MB/s. Hdd read rate is 110MB/s.

It also parity checks at around 54MB/s using Seagate drives. HDD read rate is 75MB/s.

 

Parity checking a 6 HDD system (4 Sammys and 2 seagtes) generated 65MB/s.

I used some dirty math to suppose their parity rate would be around 65MB/s.

 

I have been testing a further Seagate drive today on a PCI card and was getting a 72MB/s read rate. I figured my Parity rate would be

 

72*.7*.5=50.4

110*.7*.5=77

 

(50.4*.5)+(77*.5)=127.4*.5=63.7MB/s

 

The actual result was:

Nov 17 15:37:49 Tower kernel: md: sync done. time=15438sec rate=63270K/sec

 

So to calculate your parity speed.  Start with the smallest HDD size.

slowest drive read speed * .7  *  (smallest_HDDsize/Partity size).

 

For subsequent calculations

slowest drive read speed * .7 *  (next_HDDsize-smallestHDDsize / Partity size)

 

So to calculate parity speed or the whole array

slowest drive read speed * .7  *  (smallestHDDsize/Parity Size) + slowest drive read speed * .7 * (next_HDDsize - PreviousHDDsize / Partity size) + ...repeat for each HDD size in the array.

 

Lets assume we have 3 320GB drives, 2 640GB drives and 3 1TB drives. The slowest 320GB drive has 80MB/s read speed, all the 640GB drives are also 80MB/s and the slowest 1TB drives is 110MB/s.

 

For the smallest drive we get

80 * .7 * 1000/320 =17.92

Next we have

80 * .7 * 1000/320 =17.92

Lastly we have

110 * .7 * 1000/360 = 27.72

 

17.92 + 17.92 + 27.72 = 63.56

So we would expect to see a parity check to run at 64-64MB/s.

 

I've only tried 4 test systems and they all worked to within 1 or 2 MB/s. If you have a bus contention or a drive with errors your parity speed will be significantly slower.

 

Let me know what you guys n girls get and if it works...

 

If it doesnt work post up your system spec and we'll see if we can understand why and possibly improve the formula accordingly (the .7 might be a cpu speed related for example).

 

I'm not sure if UnRaid parity checks its own parity drive? So I'm unsure if the parity drive should be last HDD as my systems all have a data drive the same size as the parity drive. If you have a parity drive of 1.5TB and largest data drive is 1TB does parity run to 1TB or 1.5TB? 

 

 

[Joe L.]

If you have a parity drive of 1.5TB and largest data drive is 1TB does parity run to 1TB or 1.5TB? 

Pretty sure it runs to the size of the parity drive... but it does simulate the reading of "zeros" from all the other drives really fast at that point, so parity check speed for that last .5TB is pretty fast.

 

[mcs]

So to calculate parity speed or the whole array

slowest drive read speed * .7  *  (smallestHDDsize/Parity Size) + slowest drive read speed * .7 * (next_HDDsize - PreviousHDDsize / Partity size) + ...repeat for each HDD size in the array.

 

Lets assume we have 3 320GB drives, 2 640GB drives and 3 1TB drives. The slowest 320GB drive has 80MB/s read speed, all the 640GB drives are also 80MB/s and the slowest 1TB drives is 110MB/s.

 

For the smallest drive we get

80 * .7 * 1000/320 =17.92

Next we have

80 * .7 * 1000/320 =17.92

Lastly we have

110 * .7 * 1000/360 = 27.72

 

 

Are the equations/examples above consistent 

 

 

Using:

slowest drive read speed * .7  *  (smallestHDDsize/Parity Size) + slowest drive read speed * .7 * (next_HDDsize - PreviousHDDsize / Partity size) + ...repeat for each HDD size in the array.

 

The result should be:

 

For the smallest drive we get

80 * .7 * 320/1000 = 17.92

Next we have

80 * .7 * (640-320/1000) != 17.92

Lastly we have

110 * .7 * (1000-640/1000) != 27.72

 

 

Should the equation be::

slowest drive read speed * .7  *  (smallestHDDsize/Parity Size) + (slowest drive read speed * .7 * (next_HDDsize - PreviousHDDsize )/ Partity size) + ...repeat for each HDD size in the array.

 

Which results in:

 

For the smallest drive we get

80 * .7 * 320/1000 = 17.92

Next we have

80 * .7 * (640-320)/1000 = 17.92

Lastly we have

110 * .7 * (1000-640)/1000 = 27.72

 

which are the same results as presented in the example...

[Kaygee]

Should the equation be::

slowest drive read speed * .7  *  (smallestHDDsize/Parity Size) + (slowest drive read speed * .7 * (next_HDDsize - PreviousHDDsize )/ Partity size) + ...repeat for each HDD size in the array.

 

/Embarrassed on

Yes it should.Thanks for correcting it. I always did suck at math, I wrote that equation out like 20 times, jiggling it to make it work. My test system has two hdd sizes 500GB and 1TB. My original dirty sum went like this 110*.7/2+72*.7/2=Parity, it took me 30 mins to work out how-to write it that badly. 

/Embarrassed off