User share and power saving


Recommended Posts

Hi,

 

I just think the user share is a wonderfull feature, but i now begin to wonder if it is against power saving.

 

I have been checking few times now. I first look at the status to verify all the drives are sleeping. Then i browse my share, and guess what.. all the drives of the share wake up, which first take a lot of time and second eats a lot of power. Even the first time it spinup all the drives

 

I know there is some RAM caching, as if i do nothing, wait for some drives to sleep, they won't wake up on next browsing... As long as i don't make any more copy on the server.

 

As i guess it will do the same when reading a file (didn't test that yet), i wonder if the benefit of the share worth the power usage/time, using it. If reading/copying a file fills cache, then each time i will access a file on the share, it will spinup all the drives to locate a single file. That will take a lot more time that just spining only the drive where the file is, not talking the more power taken.

 

There don't seem to be a dedicated cache for the list of files structure of a share.

 

Am i missing something ? If this is right, what do you think : are the pros of "user share" better than the cons ?

Link to comment

Somewhere there are a couple of threads with ideas for combating this, and I'm sure some of those involved with them will pipe up here.  For now, based on their excellent work, here is what I've done, that has helped greatly.  I don't believe there are 100% perfect solutions, because Linux does not seem to have perfect support for caching the directory entries.

 

The following is a script derived from the work of others, customized for myself.  You'll need to customize the shared_drive line, to meet your own needs.

#!/bin/sh
# Edit as needed, if user shares not used, then list /mnt/disk1, /mnt/disk2, etc.
#shared_drive="/mnt/user"
#shared_drive="/mnt/user/Movies /mnt/user/Mp3 /mnt/user/Pictures"
shared_drive="/mnt/disk1/Videos1 /mnt/disk2/Videos2 /mnt/disk3/Videos3 /mnt/disk3/VidLib3 /mnt/disk4/Videos4 /mnt/disk4/VidLib4 /mnt/disk5/Videos5 /mnt/disk6/Videos6 /mnt/disk7/Videos7"

crontab -l >/tmp/crontab
grep -q "frequent ls" /tmp/crontab 1>/dev/null 2>&1
if [ "$?" = "1" ]
then
    echo "# frequent ls to keep directory blocks in memory:" >>/tmp/crontab
    echo "* * * * * ls -R $shared_drive 1>/dev/null 2>&1" >>/tmp/crontab
    crontab /tmp/crontab
fi

I named the script ls-R, because that is what is constantly being repeated, to force a high priority on keeping the directory entries of the folders listed in the shared_drive line.

 

Then you will need to add something like the following to your go script.

# Set cron job for ls -R task, keep directories cached
/boot/ls-r

That makes it happen.  You should also add the following lines to your go script, to 'encourage' the kernel to keep directories cached.

# Increase the directory caching
echo vm.vfs_cache_pressure = 0 >>/etc/sysctl.conf
sysctl -p /etc/sysctl.conf

 

Edit:  My shared_drive line is rather inefficient looking, redundant, but I'm still not ready to turn User Shares on.

Link to comment

This may not be the right place to bring this up, but I found that my periodic ls -Rs caused problems because they lasted longer than their assigned period at the (few) times they were disk-bound.

 

The fix was to (limit the shares they go through and) make them less-frequent, e.g., to have them run every two minutes:

 

   echo "0-59/2 * * * * ls -R $shared_drive 1>/dev/null 2>&1" >>/tmp/crontab

 

I've got 4GB of RAM and transfer off of the server at a rate of 2.5-3GB/min, which would suggest that an ls -R every 2mins is too infrequent to keep the share structure cached, but in practice I do not see spin-ups even during sustained transfers.

Link to comment

Thanks for the feedback

 

I saw these thread doing ls all the time to keep directory structure in cache. But it seems it can lead to trouble in some cases. So this doesn't sound like a nice and clean solution, specialy with my linux level (i can't live without a mouse ;) )

 

As unraid handles all the disc activity, i thought it would keep for himself the list of all the directory structure of shares somewhere. Like this it wouldn't rely on just linux cache for this, as each file access files up the cache and whipes the cached directory structure. It should be handle by unraid, internaly.

I think not doing this gives the "user share" fonctionality a very big cons, as it eliminates some very important advantage of unraid in general (single drive access and sleeping drive). Spinup 11 drives instead of 1 makes a lot of difference in time taken and power.

 

I guess i will then stick to just the drives and disable shares. A pity because this was a wonderfull fonctionality.

Link to comment

I don't think things are necessarily that bad, and working with the issue might be a nice way for you to move beyond your mouse.

 

Fact of the matter is that these sort of things are complicated and that no-one can predict what implications some or another piece of functionality has in the myriad situations that unraid is used in. If I was you, I would log into your server with telnet, type `ls -R /mnt/user' (or better still `ls -R /mnt/user 1>/dev/null 2>&1', in order not to get the whole content of your server thrown up on your terminal) and time how long it takes for the process to finish. If it takes 30-45secs or more, I would first consider whether I want the content of all shares cached at all times. For example, I have a share for pictures, which I only rarely access and that is physically residing on just one disk. I would think it contains many more files than I have movies and guaranteeing that everything in it is cached in order to have easy access to the movies is just pointless. Once you decide which shares you want cached, copy a couple of movies off of your server, run `ls -R /mnt/user/share1 /mnt/user/share2 ...', and time it. If, after all this, the share traversal still takes a long time, put 0-59/n instead of the first * (for some appropriate n), and get the script in play as described. You'll be much happier for it, both in terms of unraid responsiveness and you having conquered the issue ;)

Link to comment

I don't remember exactly where in the threads ,but i remember reading there could be cons in some cases with the constant ls technic (trouble when parity ? spinup all the drives that where sleeping ?). Anyway, reading the posts, it had never be clear to me it could be used without any trouble. I just didn't want to take the risk as if i run into trouble, i would have no way to solve it alone (i don't understand a word of what you want me to type, well i understood before " 1>/dev/null 2>&1").

Now if there is a sure workaround that can't trouble unraid, i am ready to try it.

 

Now for the softwware part, when the "user share" arrived, a lot already has changed to support this. When you access a file from the share, unraid now has to find on which particular drive the file is to get it. This is why i was thinking it would keep an internal listing of all the files on the share drives, so it wouldn't have to search for the file on all drives. That would have solve the trouble without needing any workaround.

 

Now not using "user share" isn't that bad, and that still means to me unraid is a great software. It is just that, as it is, user share take away some very nice unraid features, which i prefer more than user share.

 

I will stil try tonight (it is 11H30 AM in France) at home to list has RenV adviced, to see the result.

 

For information, i only put movies on my unraid server (12 drives, so 11To of data). The HD movies are riped in ISO, so 2 files per movie (.dvd and .iso) and the DVD are plain DVD file structure copy, so much more files. This is why i am sure to empty the cache with movie stream when playing. This is also why i don't want all the drive to wake up when watching just 1 movie as it will take too long to start (less drawback when copying), beside the fact it will eat a lot of power.

So if the workaround is not 100% sure (well beside a bug), i'd rather point each movie to the specific disk than the global share. A litle more pain to do (only one time action for each movie) but that guaranties that i will benefit of all the other unraid advantages  :)

Link to comment

" 1>/dev/null 2>&1"

 

There are two standard ways of giving output in GNU/Linux (and in UNIXes, more generally), referred to as stdout and stderr. Standard out is used for the expected results. Standard error is used to report errors. The >-syntax can be used to redirect those reponses to elsewhere, e.g., a file. The above bit of magic sends stdout to the null device, i.e., it is discarded, and sends stderr to the place where you would normally expect to see stdout. The result is that you do not get filename after filename scrolling in your terminal, but you are made aware of any errors that results from the command having its output so-redirected.

Link to comment

I use this script.  It might work for you if you need to force the "ls" results into the cache more frequently than once a minute.

You will need to edit the list of folders to suit your needs.

 

Joe L.

#!/bin/sh
crontab -l >/tmp/crontab
grep -q "frequent ls" /tmp/crontab 1>/dev/null 2>&1
if [ "$?" = "1" ]
then
    echo "# frequent ls to keep directory blocks in memory:" >>/tmp/crontab
    echo "* * * * * for i in 1 2 3 ;do sleep 20; ls -R /mnt/user/Movies /mnt/user/Mp3 /mnt/user/Pictures 1>/dev/null 2>&1; done" >>/tmp/crontab
    crontab /tmp/crontab
fi

Link to comment

Ok, so i made few tests as you described.

 

The first `ls -R /mnt/user' took 1m47s, without taking into account the time it took to wake up the drive (quite long as i have 11 data drives). I started the chrono when i saw the first listed files on the console.

The second same call took just 14s to run.

After i used `ls -R /mnt/user 1>/dev/null 2>&1' and it took just 2s to run.

 

Knowing that, what timing do you think i should use for the ls script so that i can read files and still keep the directory structure in cache ?

 

if i take RobJ script :

#!/bin/sh
# Edit as needed, if user shares not used, then list /mnt/disk1, /mnt/disk2, etc.
shared_drive="/mnt/user"

crontab -l >/tmp/crontab
grep -q "frequent ls" /tmp/crontab 1>/dev/null 2>&1
if [ "$?" = "1" ]
then
    echo "# frequent ls to keep directory blocks in memory:" >>/tmp/crontab
    echo "* * * * * ls -R $shared_drive 1>/dev/null 2>&1" >>/tmp/crontab
    crontab /tmp/crontab
fi

Where in this script do i see how often it is launched ? From the answer from others, it seems the first * will make it, but when it is a *, what timing does that mean ?

 

I saw to replace it with '0-59/2' for 2 minutes. But do you think in my case i have to change RobJ script ?

 

Now i guess to implement this, i have to stop the array, then make a clean powerdonwn, then pull out the flash and put this ls-r file in the boot directory of the flash. Of course i also have to add to the go script the 2 pieces of code RobJ gave. Is that the right way to do ? As i will create this file under windows, is it ok if it is saved with notepad with windows CR+LF or do i have to convert to unix for the CR to be handled correctly ?

 

I guess that when reading a file, unraid only spinup the right drive and doesn't have to spinup all of them to check where the file is, so maintaining the link between the share file structure and the real structure on each drive. Because if i am wrong, this means i would have to list also all the discs to keep their folder structure in cache.

 

Thanks in advance and sorry for all these maybe stupid questions, but the window guy i am is having a lot of trouble understanding all those cron lines. And as i don't want to break anything, i'd rather ask first to see if what i try to copy is done the right way  ???

Link to comment

Ok, i decided to try to less bother the specialists with my low level questions and to answer most of them.

 

So here are the results of my searches.

 

When all the drives are sleeping, accessing one file using a user share will only wake up the drive containing the file, all the others are still sleeping.

Reading a file will fill up the cache and erase the folder structure in cache. This means the next listing took 2m58s to wake up and list all the drives.

The flash share is in fact the boot, so i can put the files in it using the share, then reboot the server.

 

Now i searched how the crontab worked, and now i know that all stars will mean a launch every minute. If i want 2 minutes, i can write it "*/2 * * * *"

 

Now some computing : an HD movie will eat around 2-3% of my giga card. I imagine max 10%. This makes 10Mo/s. That will take 3 minutes to fill up my 2Go. So i could use */3 for the ls script.

 

And i finish with what i think 2 more advanced questions :

1- Is there a cons of launching the ls script every minute, and pros to find our fill cache limite not to launch it too often. In fact how that behavior will impact unraid

2- In all the cases, a parity check goes at around 30Mo/s with 11 drives that makes more than 300Mo/s, so in few seconds the memory will be full, meaning the ls script will have to go through all the drives to do its work. So how will the ls script going through all the physical drives will impact unraid during a parity check or build

 

Thanks in advance

 

Now some thinking : if unraid keeps an internal filesystem for shares to make the link between the file and its physical location, could it read it when browsing the share. I imagine that would make the ls script and the memory caching useless.

Link to comment

...

 

Now some computing : an HD movie will eat around 2-3% of my giga card. I imagine max 10%. This makes 10Mo/s. That will take 3 minutes to fill up my 2Go. So i could use */3 for the ls script.

...

 

JUST fyi:

 

It is my experience through extensive experimentation that this math does not work and when it fails the script will cause all drives to spin up at once. I currently run every 30 seconds which seems to be a nice balance.

 

No matter what you do this script will fail at some point. The problem is the kernel to which there is no known/simple 100% effective solution.

 

 

Link to comment

It is my experience through extensive experimentation that this math does not work and when it fails the script will cause all drives to spin up at once. I currently run every 30 seconds which seems to be a nice balance.

 

No matter what you do this script will fail at some point. The problem is the kernel to which there is no known/simple 100% effective solution.

 

Thanks for the information. Arg !

 

So lets say it fails, then it will wake up all the drives and do the ls. This is going to take nearly 3 minutes. This means during that time, the ls will be run 5 more times, with the drives trying to wake up and all the other ls script still running. Will this be a problem ?

And during a parity check/build, what will be the impact of twice a minute the ls script that runs ?

 

And if i have to launch it twice a minute, is this the correct syntax for the script ?

    echo "* * * * * for i in 1 2 ;do sleep 28; ls -R $shared_drive 1>/dev/null 2>&1; done" >>/tmp/crontab

I took as model a script that was doing 3 times 20 seconds.

 

What i wonder, and why i put 28seconds and not 30, is that using 30s, it will start the second listing at more than the minute timing for the next ls script to run. Is that a trouble to have 2 or more ls script running at the same time on the unraid server ?

Link to comment

I currently use a bash loop with a timer and echo inode and dentry stats to the shell to keep an eye on it. Its a work in progress and not very useful for most people. It does however let me see whats going on and allow it to be tweaked. I wouldnt worry about multiple ls as it is almost instant for the stuff it has already cached.

 

Dont get me wrong this trick makes a massive difference to unRAID. I can browse my folders without spinning up drives and the reponse is basically instant... its just that if the kernel could be made to do what it is supposed to on paper it would be even better.

Link to comment

Thanks, so you're telling me the script is ok to ls twice a minute.

 

But what are the consequences when it will run during a parity check where the cache will be emptied in less than 30 seconds, so the ls script will have to go through all the physical drives, and that 2 times every minute for 18 hours.

 

Isn't this going to harm unraid ?

Link to comment

Thanks, so you're telling me the script is ok to ls twice a minute.

I do mine 6 times a minute.  (The cron task runs once a minute, but it has a loop with a 10 second sleep between a series of "ls"' commands)

But what are the consequences when it will run during a parity check where the cache will be emptied in less than 30 seconds, so the ls script will have to go through all the physical drives, and that 2 times every minute for 18 hours.

By running it more frequently, it stays in the cache.  The least recently used blocks are re-used from the cache first... those would be the oldest blocks in the parity calc.

Isn't this going to harm unraid ?

No, not at all.  As already said, once the directory blocks are in memory, they will be accessed almost instantly and physical disks stay spun down.

 

Note I only use the "ls" on my Movies, Pictures, and Mp3 folders.  I don't mind the spin-up delay on my "data" folder I use for system backups.

 

My current script looks like this, the line I add to crontab must be entered all on one line, as shown here.:

#!/bin/sh
crontab -l >/tmp/crontab
grep -q "frequent ls" /tmp/crontab 1>/dev/null 2>&1
if [ "$?" = "1" ]
then
    echo "# frequent ls to keep directory blocks in memory:" >>/tmp/crontab
    echo "* * * * * for i in 1 2 3 4 5 6;do sleep 10; ls -R /mnt/user/Movies /mnt/user/Mp3 /mnt/user/Pictures 1>/dev/null 2>&1; done" >>/tmp/crontab
    crontab /tmp/crontab
fi

 

Joe L.

Link to comment

Joe L

Oh, every 10 seconds, sounds scary to me, but if you say that works fine, i'll finaly make the big jump and add my first go script tonight (well in fact your)

 

Thanks a lot for all your detailed answers

 

Reading all this, i will try to go from 2Go to 4Go of RAM when possible, to increase the chance the file structure stays in cache

Link to comment

1) At some point, I started having disks freezing/being reported as frozen at the same time as I was running overlapping instances of ls -R on an array with not all disks already spun up and the share content not cached. Needless to say, that's bad and required the parity to be rebuilt. It happened a few times, until I moved my unRAID box around a bit and, at the same time, backed off the ls -Ring. I'm basically satisfied that the ls -Ring is unrelated to the disk freezing, but I haven't done any further testing and I couldn't vouch for it.

 

2) My parity check runs in the low 40Ks/sec, which amounts to around 2.4GB/min. As I have 12 data disks plus parity that should mean that I have 31.2GB of data passing through memory every min of a parity check. With around 3.5GB of cacheable RAM that should mean that I need to run ls -R more than 9 times/min during a check to prevent the share content becoming uncached. Does anyone have concrete experience with parity checking that's not slowed down due to ls -R? What are the relevant numbers (speed, RAM, frequency)?

Link to comment

    echo "* * * * * for i in 1 2 3 4 5 6;do sleep 10; ls -R /mnt/user/Movies /mnt/user/Mp3 /mnt/user/Pictures 1>/dev/null 2>&1; done" >>/tmp/crontab

 

I just noticed, JoeL, that you may want a plain `ls /mnt/user' (with suitable redirection) in there, too.

 

    echo "* * * * * for i in 1 2 3 4 5 6;do sleep 10; ls /mnt/user 1>/dev/null 2>&1; ls -R /mnt/user/Movies /mnt/user/Mp3 /mnt/user/Pictures 1>/dev/null 2>&1; done" >>/tmp/crontab

 

Link to comment

We really need to be looking at the same time for a non fudge way to do this. The kernel on paper can in theory do this without all this cron stuff we just need someone in the know to talk to the kernel peeps.

 

The kernel solution is the only one that will ever make it to unRIAD proper and we really want everyone to benefit frmo this not just a small subset of hackers and testers

Link to comment

    echo "* * * * * for i in 1 2 3 4 5 6;do sleep 10; ls -R /mnt/user/Movies /mnt/user/Mp3 /mnt/user/Pictures 1>/dev/null 2>&1; done" >>/tmp/crontab

 

I just noticed, JoeL, that you may want a plain `ls /mnt/user' (with suitable redirection) in there, too.

 

    echo "* * * * * for i in 1 2 3 4 5 6;do sleep 10; ls /mnt/user 1>/dev/null 2>&1; ls -R /mnt/user/Movies /mnt/user/Mp3 /mnt/user/Pictures 1>/dev/null 2>&1; done" >>/tmp/crontab

 

No, I do not want /mnt/user.    Yes, it would cache everything immediately under /mnt/user, but to get to the sub directories I did list, the blocks in /mnt/user would have to be traversed anyway, and therefore cached.

 

Joe L.

Link to comment

Arg ! Once i finaly decide myself to test this (i am far from beeing a hacker, this is why i am a litle scared about this), someone pops in with terrible news, even if i didn't understand all point 1 of ReneV.

 

Now during the night i thought about something else with a run every 10 seconds

 

When the server starts, all the disc are spinup, but the first ls will last more than 10 seconds, as it will have to go through all the 11 physical drives to fill the cache the first time.

This means the second ls will run before all the data will be in memory, so i guess it will have to parse all the drives also. Maybe the third ls will do the same. Will that harm unraid ?

 

If this is bad, is it possible in the go script to make a first ls statement alone (not in the crontab), so it will take the time it takes to fill the cache, then run the script that set every 10 seconds the ls ? Is that a good idea ?

 

With Joe L. script every 10 seconds, should i also add the folowing RobJ script in the go script ?

 

You should also add the following lines to your go script, to 'encourage' the kernel to keep directories cached.

# Increase the directory caching
echo vm.vfs_cache_pressure = 0 >>/etc/sysctl.conf
sysctl -p /etc/sysctl.conf

 

And does it have to come before or after the call to the ls-R script ?

 

We really need to be looking at the same time for a non fudge way to do this. The kernel on paper can in theory do this without all this cron stuff we just need someone in the know to talk to the kernel peeps.

 

The kernel solution is the only one that will ever make it to unRIAD proper and we really want everyone to benefit frmo this not just a small subset of hackers and testers

Oh yes, i would feel much more confident if it was out of the box  ;D

Link to comment

Arg ! Once i finaly decide myself to test this (i am far from beeing a hacker, this is why i am a litle scared about this), someone pops in with terrible news, even if i didn't understand all point 1 of ReneV.

 

The import of what I said is that, in all likelihood, you've got nothing to worry about: go ahead. Many people are running periodic ls -Rs, and no problems have been reported. If, however, you do start experiencing disappearing disks, the fix may be as simple as backing off the frequency of ls -Ring, rebooting, and waiting for the parity check to complete.

Link to comment

Like the DOS and Windows dir command, the Linux ls is a high level command.  If listing the directory of a folder, even simultaneously, can cause a crash or errors, then Linux has far more serious problems than anything we're talking about here.  I personally don't see how an overlapping ls could cause trouble, it is just asking for the disk blocks containing the folder directory.  The first ls call will pull those into the cache, and subsequent calls will read them from the cache, so the overlapping ls calls will catch up to the first call, and then wait together for the next physical block to arrive from the disk.  Each ls call, whether first or 100th or last, has to wait for the OS to provide the next set of directory entries.

Link to comment

the Linux ls is a high level command

 

I was speaking to the possibility, e.g., of the md drivers implementing their own time-outs before the unRAID logic gives up on a disk. To be honest with you, I haven't checked how low-level the unRAID disk backend is but I would be surprised if the data disks are not accessed through the standard reiserfs drivers, which would rather suggest that unRAID can give up on a disk before the kernel does.

 

(Btw, ls is probably better described as GNU than Linux.)

Link to comment
I was speaking to the possibility, e.g., of the md drivers implementing their own time-outs before the unRAID logic gives up on a disk. To be honest with you, I haven't checked how low-level the unRAID disk backend is but I would be surprised if the data disks are not accessed through the standard reiserfs drivers, which would rather suggest that unRAID can give up on a disk before the kernel does.

 

Interesting point, had not thought of that.  I would certainly hope that unRAID would not apply stricter rules and timeouts than the kernel or any kernel module though.  And to my mind, any 'fragility' to something so basic as reading a folder directory just does not make sense.

 

(Btw, ls is probably better described as GNU than Linux.)

 

Oops, my Linux noviceness was just exposed!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.