30% Read Performance Improvement Tweak... Still works in unRaid 4.2

Joe,

 

Is there a command I can add to copy a whole movie folder or more than one vob at once. All my DVD's are split and are all 1Gb each or smaller. Right the Ram thing never even dawned on me until I read your post.  Even you look at me text file you will see that is what I was trying to do with SELENA* hoping the wildcard would work.

 

Rgds,

 

Dave

Default:

 

2400698274 bytes (2.4 GB) copied, 97.513 s, 24.6 MB/s

 

and with the command in the OP:

 

2400698274 bytes (2.4 GB) copied, 35.7486 s, 67.2 MB/s

 

 

This is on a SATA drive (WDC_WD6400AAKS)

So, is there any resolution to this?

 

If I'm reading this correctly, it sounds like this tweak wont do anything for user shares.

 

Good job guys!  If you don't mind some more testing... how does increasing read-ahead help read transfer rate via network?  And does it make any difference reading from /mnt/diskX vs. a User share?

Tom,

Sorry to say, it makes a big difference when reading the same file through a "user share"

 

Looks like we drop down to about half of the performance of reading directly from /mnt/diskX

 

Are user-shares buffered?   Perhaps they could use some tweaking too?  They are way less efficient than reading directly from the /mnt/diskX directory.

 

23 to 28 MB/s vs. 47 MB/s.

 

Joe L.

root@Tower:/boot# blockdev --setra 256 /dev/md8
root@Tower:/boot# dd if=/mnt/user/Movies/WIMBLEDON.ISO of=/dev/null
10127896+0 records in
10127896+0 records out
5185482752 bytes (5.2 GB) copied, 181.625 seconds, 28.6 MB/s
root@Tower:/boot# blockdev --setra 2048 /dev/md8
root@Tower:/boot# dd if=/mnt/user/Movies/WIMBLEDON.ISO of=/dev/null
10127896+0 records in
10127896+0 records out
5185482752 bytes (5.2 GB) copied, 220.202 seconds, 23.5 MB/s
root@Tower:/boot# dd if=/mnt/disk8/Movies/WIMBLEDON.ISO of=/dev/null
10127896+0 records in
10127896+0 records out
5185482752 bytes (5.2 GB) copied, 110.238 seconds, 47.0 MB/s

So, is there any resolution to this?

 

If I'm reading this correctly, it sounds like this tweak wont do anything for user shares.

 

Good job guys!  If you don't mind some more testing... how does increasing read-ahead help read transfer rate via network?  And does it make any difference reading from /mnt/diskX vs. a User share?

Tom,

Sorry to say, it makes a big difference when reading the same file through a "user share"

 

Looks like we drop down to about half of the performance of reading directly from /mnt/diskX

 

Are user-shares buffered?   Perhaps they could use some tweaking too?  They are way less efficient than reading directly from the /mnt/diskX directory.

 

23 to 28 MB/s vs. 47 MB/s.

 

Joe L.

root@Tower:/boot# blockdev --setra 256 /dev/md8
root@Tower:/boot# dd if=/mnt/user/Movies/WIMBLEDON.ISO of=/dev/null
10127896+0 records in
10127896+0 records out
5185482752 bytes (5.2 GB) copied, 181.625 seconds, 28.6 MB/s
root@Tower:/boot# blockdev --setra 2048 /dev/md8
root@Tower:/boot# dd if=/mnt/user/Movies/WIMBLEDON.ISO of=/dev/null
10127896+0 records in
10127896+0 records out
5185482752 bytes (5.2 GB) copied, 220.202 seconds, 23.5 MB/s
root@Tower:/boot# dd if=/mnt/disk8/Movies/WIMBLEDON.ISO of=/dev/null
10127896+0 records in
10127896+0 records out
5185482752 bytes (5.2 GB) copied, 110.238 seconds, 47.0 MB/s

It says that back in 2007, on my server, with my IDE based array and my hardware, reading files through the  "user shares" feature is about half as efficient as reading the disk shares directly.  Since "user shares" is another process the data has to move through, it cannot be as efficient as going directly to the disk shares.  If not buffered correctly, it will be way less efficient that it needs to be otherwise.  If it already did buffering of its own, it might be improving performance if you did not set the read-ahead buffering otherwise.

 

You would want to do these same tests on your hardware with today's version of unRAID and report back.  Changing the read-ahead buffering helped with the disks shares, and since the user-shares still have to get to them, it can't hurt.

 

Joe L.

You would want to do these same tests on your hardware with today's version of unRAID and report back.   Changing the read-ahead buffering helped with the disks shares, and since the user-shares still have to get to them, it can't hurt.

 

Joe L.

 

After further reading, I came across this post stating that user share performance does indeed seem to benefit from this tweak (at least as of unRAID 4.3-beta1).

 

I'll be testing this out over the weekend on 4.3.3.

 

Did you do this to your system yet brainbone? I just got a gb switch in. I'm using 4.3.3 and my transfer rates were:

700MB write 100e-1 min 10 seconds, 1000e-30 seconds

700MB read 100e-1 min 10 seconds, 1000e- 18 seconds

 

Trying to find out if this tweak still helps for 4.3.3

I haven't tested the speed over my network, but for disk > /dev/null:

 

Without Tweak on 4.3.3:
root@file-server1:/boot# dd if=/mnt/user/Media/Movies/test.mpg of=/dev/null
3899808+0 records in
3899808+0 records out
1996701696 bytes (2.0 GB) copied, 47.3311 s, 42.2 MB/s

With Tweak on 4.3.3:
root@file-server1:/boot# dd if=/mnt/user/Media/Movies/test.mpg of=/dev/null
3899808+0 records in
3899808+0 records out
1996701696 bytes (2.0 GB) copied, 26.7576 s, 74.6 MB/s

 

 

So, yes. It works quite well on 4.3.3.

 

Just to be sure as of 4.4 this is no longer required?

Just to be sure as of 4.4 this is no longer required?

Only way to know for sure is to remove the line you added to set the re-ahead buffer, reboot, and do a test without it... then set the readahead buffer and test a second time.

 

I've not tried this... perhaps you can give it a try and report back on the results.

 

Joe L.

Version 4.4 has more aggressive read-ahead built in, so should not be necessary.

 

Also, you can force cache invalidation using this:

 

echo 1 > /proc/sys/vm/drop_caches

 

Refer to http://linux-mm.org/Drop_Caches

 

At one point it was discovered it is only necessary to set the read-ahead on one of the "md" devices...

At that time, there was no need to loop through them all, they shared the same buffer setting (and perhaps the same buffer).

 

This has apparently changed in the newest unRAID.     

 

Setting /dev/md1 does not affect /dev/md2 or any of the others.   

Therefore, we DO need to set each device in a loop..

for i in /dev/md*

do

echo Setting $i

  blockdev --setra 2048 $i

  blockdev --getra $i

done

 

PS.

unRAID 4.4-beta2 sets the buffer size to 1024 by default, so the effects of setting it to 2048 may not be as significant as when it was 256 in older versions of unRAID.  1024 might be the optimum value, or not.   Only time will tell as people try different values on their own arrays.

 

Joe L.

 

joe,

 

Can you take a look at my GO script attached. I have been getting a cannot find device /dev/MD* upon bootup just before the fuse.

 

Thanks,

 

Dave

joe,

 

Can you take a look at my GO script attached. I have been getting a cannot find device /dev/MD* upon bootup just before the fuse.

 

Thanks,

 

Dave

About all you can try is a sleep longer than 30 seconds.  It is possible some of your drives are taking longer to spin up and the array is not started within 30 seconds.

 

Unless... somewhere else you have a script somewhere else referencing /dev/MD*  (Capital letters MD* as in your question)

 

Can you capture the exact error? (or take a picture of the screen if you cannot)

 

Joe L.

 

HI Joe,

 

The error is after it loads the fuse module. The last lines in the boot process before the login prompt are

 

loading fuse module

mounting fuse control filesystem

/dev/md*: No such file or directory

 

thanks,

 

Dave

Version 4.4 has more aggressive read-ahead built in, so should not be necessary.

 

Also, you can force cache invalidation using this:

 

echo 1 > /proc/sys/vm/drop_caches

 

Refer to http://linux-mm.org/Drop_Caches

 

I remembered this post and had been meaning to try removing the setra and see if it made any difference.

 

I was running a big read operation using "quickpar" over samba.  Without doing any setra I got throughput of ~14mB/sec (12-17).  After applying the setra 2K, I get ~30 mB/sec (27-33).

 

Tom, if you are reading, you might want to make this default behavior - it makes a huge difference.

 

(I am running 4.4 Final.)

 

One thing to note, different values of read ahead will be the optimum for different operations.

 

Playing a movie may be best with a value of 1K while doing a par rebuild may be faster with 2K.  Only experimentation will tell, and even that will vary with the drives/controllers used.

I gave this a try gust to see what it would do. I have a old WD IDE drive right on the mother board and a Seagate 1.5tb SATA on a promise controller

 

first I copied a 3.6gig file from the SATA drive and then a 2.7gig file from the IDE drive. I assume that this should clear the memory using different files.

 

at 256 I got 24.7 on the SATA and 23.9 on the IDE

 

Then I used UnMenu to change the settings to 2048, I assume this does the same thing. Repeated the same files

 

got 81.6 on the SATA and 25.9 on the IDE

 

SATA saw huge increase and the IDE none, I was expecting the opposite.

Try different values (64 128 256 512 1024 2048 4096).  Also, look at the size of the hardware cache on the drives.

I re ran the test at another speed

 

IDE Drive (on mother board)

@256 = 23.9 mb/s

@1024 = 25.6 mb/s

@2045 = 25.9 mb/s

 

SATA Drive (on controller)

@256 = 24.7 mb/s

@1024 = 65.3 mb/s

@2048 = 81.6 mb/s

 

My Parity and most all my movies are stored on the SATA drives, the IDE's are pictures, music, and back up files

Hey guys, im not Linux savvy at all, and i would like to test the read speeds of my own server drives!!!

 

I first need to know how i can test the current read speed?....

 

Please advise me the command to do a read test on this file:  " \\tower\disk1\Movies\title\title1.mpg " of course in linux language:)....i can telnet to my unraid box, but dont know any linux commands at all!!!

 

Thanks

That is documented in detail on the wiki.

WHOA - I got a HUGE increase in READ Performance from this Tweak.....THANK YOU VERY MUCH.....i went from 24.1 MB/S to a whopping 95.4 MB/Sec!!!...i done a reboot between tests aswell so nothing was held in memory/cache etc

 

My read was good as it were watching streaming HD from my Unraid....cant imagine how good itll be now:)....i may even be able to seek at much faster speeds:).....thanks alot for the tweak guys....you have my thumbs up for this one:)

 

Im now going to try on my other HDD

 

Ive attached my results!

 

NOTE: I just tested a 15GB file on my other disk, and got a 97.9 MB/Sec!!!.....big smile on my face now:) attached another screen

 

Update2: I tried a speed test from a user share....and got a measly ~38 MB/sec, pic attached....obviously reading directly from share is much worst!

On unraid 4.4.2 off of a WD 500GB drive I got 77.8MB/s with 2048, and 78.1MB/s with the default (I think I read in the this thread that it's 1024). What's strange is that I get the same from a usershare, which seems contrary to what others are getting.

Hey all - i just added another Parity drive to my System?....im really curious to know what the read speeds are for the parity dirve, how could i test?

 

Thanks:)

it's a joke, right?