cache_dirs - an attempt to keep directory entries in RAM to prevent disk spin-up

[queeg]

What about -prune option?  Would that work to exclude sub dirs?

EDIT: 
Here's an example that I just figured out.  It prints everything except the packages directory.
find /boot/config \( -name packages -prune \) -o -name "*" -print

And one that prints everything except subdirs ending in "old" or the packages directory.
find /boot \( -name "*old" -prune -o -name packages -prune \) -o -name "*" -print

Cool.  I'll try to integrate this into it.

[Joe L.]

Actually, the ability to add arguments is built in with the "-a" option.

An example:
cache_dirs  -a '-noleaf -name Steam -prune -o -print'

Or, if you have more than one directory to exclude:
cache_dirs -a '-noleaf \( -name Steam -prune -o -name data -prune \) -o -print'

Your example would be
cache_dirs -a '-noleaf \( -name "*old" -prune -o -name packages -prune \) -o -print'

No need to hack at cache_dirs at all.  It was designed to be flexible.  When using the -a option you need to use the -noleaf at the beginning of the -a option argument, and -print at the end to get correct syntax.  The entire -a option argument is enclosed in single quotes as in -a '-noleaf ....... -print'   Individual shell wild-cards "*" can be enclosed in double quotes as in the "*old" example above.

Not too difficult...

Joe L.

[queeg]

Actually, the ability to add arguments is built in with the "-a" option.

An example:
cache_dirs  -a '-noleaf -name Steam -prune -o -print'

Or, if you have more than one directory to exclude:
cache_dirs -a '-noleaf \( -name Steam -prune -o -name data -prune \) -o -print'

Your example would be
cache_dirs -a '-noleaf \( -name "*old" -prune -o -name packages -prune \) -o -print'

No need to hack at cache_dirs at all.  It was designed to be flexible.  When using the -a option you need to use the -noleaf at the beginning of the -a option argument, and -print at the end to get correct syntax.  The entire -a option argument is enclosed in single quotes as in -a '-noleaf ....... -print'   Individual shell wild-cards "*" can be enclosed in double quotes as in the "*old" example above.

Not too difficult...

Joe L.

That sounds easier.  I had added a new -x parameter but trying to put it into all the find's get me tangled up.
This is what I'm going to use.
/boot/cache_dirs -w -u -d 3 -a '-noleaf \( -name "ftproot" -prune -o -name "lost+found" -prune -o -name "iTunes" -prune \) -o -print'

[queeg]

It doesn't seem to be working.  The drive spins up going down the first directory leve.

/boot/cache_dirs -w -u -d 3 -a '-noleaf \( -name "ftproot" -prune -o -name "lost+found" -prune -o -name "iTunes" -prune \) -o -print'

[BRiT]

The first time the script runs, the drive has to be spun up to get the directory listings.

If you mean the drive spuns up after the script is running and the drive has idled out to sping down, how are you browsing the files? If you're using Windows Explorer, make sure it's not trying to read the content of the files to display thumbnails.

[queeg]

What I meant to say was I spun up the drives, ran the cache_dirs command, allowed the drives to spin down (actually just came back 7 hours later) and from windows I explored the directories while also watching the drives on the unRAID menu.  After browsing down past the first directory the drive spun up.  So I don't think cache_dirs with those parameters is working.

[Joe L.]

What I meant to say was I spun up the drives, ran the cache_dirs command, allowed the drives to spin down (actually just came back 7 hours later) and from windows I explored the directories while also watching the drives on the unRAID menu.  After browsing down past the first directory the drive spun up.  So I don't think cache_dirs with those parameters is working.

As already said, it depends on what you are browsing... If you have thumbnail views enabled, windows is looking at file contents, not just directory listings.

[queeg]

The (Win7) explorer is set to Titles which is the default I think.  

Edit:  Tiles (not Titles)

[Joe L.]

The (Win7) explorer is set to Titles which is the default I think.  

Edit:  Tiles (not Titles)

Yes, "tiles" will show you small images of jpg files, and images of documents.  It will be examining file content, not just directory data.  cache_dirs is not opening every file to read them...   

Solution, change the view you are using to a file listing mode.

Joe L.

[queeg]

A little testing and here is what I see.  It seems that even though explorer defaults to Tile it actually does request file info unless you manually set it to Tile.


Open explorer
browse top level folder: works
browse any subdir: spins up drive.


Open explorer
Change view from Tiles to List
browse top level folder: works
browse any subdir: works
change view from List back to Tiles
browse other subdirs:  continues to work.

In any case, cache_dirs is working correctly with the added -a parameter so I'm happy and thanks Joe.

[Joe L.]

A little testing and here is what I see.  It seems that even though explorer defaults to Tile it actually does request file info unless you manually set it to Tile.


Open explorer
browse top level folder: works
browse any subdir: spins up drive.


Open explorer
Change view from Tiles to List
browse top level folder: works
browse any subdir: works
change view from List back to Tiles
browse other subdirs:  continues to work.

In any case, cache_dirs is working correctly with the added -a parameter so I'm happy and thanks Joe.

Don't be fooled by your test.  Once a "file" is read, it is in the cache until it is replaced by something else needing the cache.

To perform your test properly, you'll need to empty the cache between tests.

To do that, type this in between each test:
sync; echo 3 > /proc/sys/vm/drop_caches

[paperclip]

Joe L., thanks for your great script.

I do have a questions about the -d (max depth) parameter. Does the numbering start at 0 or 1?

So if I had a "Music" share with the following structure:
Music/The Eagles/Hotel California/Track01.mp3

What would be the appropriate max depth value?

Does it even matter? I mean, if my folder structure didn't have a max depth exceeding 3, but I set cache_dirs to look 5 folders deep, would there be a performance hit?

Thanks.

[Joe L.]

Joe L., thanks for your great script.

I do have a questions about the -d (max depth) parameter. Does the numbering start at 0 or 1?

So if I had a "Music" share with the following structure:
Music/The Eagles/Hotel California/Track01.mp3

What would be the appropriate max depth value?

The argument is passed to the "find" command.  Its manual page says

    -maxdepth levels
         Descend at most levels (a non-negative integer) levels of direc-
         tories below the command line arguments.    `-maxdepth  0'  means
         only  apply the tests and actions to the command line arguments.

You can experiment by changing the number and testing with this sample "find" command and see what it prints:
find /mnt/disk*/Music -maxdepth 1

The depth is relative to the starting path, so a maxdepth of 0 would only cache the /mnt/disk*/Music folders and nothing in them or below them.
To cache the hierarchy you described you would need at least a depth of 3.

Does it even matter? I mean, if my folder structure didn't have a max depth exceeding 3, but I set cache_dirs to look 5 folders deep, would there be a performance hit?

Thanks.

Since you have no really deep hierarchies, there is no reason to specify any -d option.  There is no penalty of using 5, or using the default of 999.

Edit: fixed directory name in example

[paperclip]

You can experiment by changing the number and testing with this sample "find" command and see what it prints:
find /mnt/disk*/Mp3 -maxdepth 1

The depth is relative to the starting path, so a maxdepth of 0 would only cache the /mnt/disk*/Music folders and nothing in them or below them.
To cache the hierarchy you described you would need at least a depth of 3.

Since you have no really deep hierarchies, there is no reason to specify any -d option.  There is no penalty of using the default of 999.

Joe L., thanks for this information.

[starcat]

Now, if your "browsing" of the directory is writing to them (MAC OS writes ._DS files) or opening files to show thumbnail views (windows will do this) then yes, the disks will spin up, since the file contents are not cached.

Writing of .DS_Store files on network shares can be prevented by running these:
defaults write /Library/Preferences/com.apple.desktopservices DSDontWriteNetworkStores true
defaults write com.apple.desktopservices DSDontWriteNetworkStores true

see this for details:
http://www.macosxhints.com/article.php?story=20051130083652119