bblue

Mover and top level Shares on cache drive

6 posts in this topic Last Reply

Recommended Posts

For the life of me, I don't understand why mover blindly removes all top level share directories when it's done.  I'm writing a lot directly to cache either via ethernet or an NTFS mounted e-sata drive and have to re-create top level directories each time.  I originally tried some . (dot) directories one level below top thinking that they wouldn't be moved and would keep the top level's from being 'empty' on the final find phase.

 

But that didn't work -- mover just transferred them to the target data drive along with everything else.  So I modified the 'find' section thusly

 

(cd /mnt/cache ; \
find . \
     \( -type d  -regex '[.]/[^.]*'                             -exec mkdir -p /mnt/user0/ {} \; \) , \
     \( -type f  -regex '[.]/[^.].*/.*'  ! -exec fuser -s {} \;  -exec echo moving {} \; -exec mv -v {} /mnt/user0/{} \; \) ; \
find . -type d  -regex '[.]/[^.]*'     ! -exec fuser -s {} \;  -empty -delete \
)

 

For both of the -type d operations I removed the dot before the asterisk at the end of each regex expression.  Now, by adding a directory called ".dnr" in each share right below the top level it seems to work the same but not delete the top level shares.

 

Question:  Does this break anything else?  I'm sure it would be blind to *any* lower level dot-directories, but is that of any concern?

 

--Bill

Share this post


Link to post

I guess my question is why you do not want it to delete the top level folders?  They are created and destroyed as needed and if you create a top level folder that has a .(dot) in front of the name it will be ignored.

Share this post


Link to post

I guess my question is why you do not want it to delete the top level folders?  They are created and destroyed as needed and if you create a top level folder that has a .(dot) in front of the name it will be ignored.

 

Well, why not?  I see no advantage to the share directories being deleted periodically.

 

But as I mentioned, the reason I want them to stay is that I do both my network and local mounted 'extra' disks copying directly to the cache drive.  It's much faster than writing to a share (at least on 4.5b6), and writing to a share is the only way the share name will be 'auto' created as needed.

 

--Bill

Share this post


Link to post

For the life of me, I don't understand why mover blindly removes all top level share directories when it's done.  I'm writing a lot directly to cache either via ethernet or an NTFS mounted e-sata drive and have to re-create top level directories each time.  I originally tried some . (dot) directories one level below top thinking that they wouldn't be moved and would keep the top level's from being 'empty' on the final find phase.

 

But that didn't work -- mover just transferred them to the target data drive along with everything else.  So I modified the 'find' section thusly

 

(cd /mnt/cache ; \
find . \
     \( -type d  -regex '[.]/[^.]*'                             -exec mkdir -p /mnt/user0/ {} \; \) , \
     \( -type f  -regex '[.]/[^.].*/.*'  ! -exec fuser -s {} \;  -exec echo moving {} \; -exec mv -v {} /mnt/user0/{} \; \) ; \
find . -type d  -regex '[.]/[^.]*'     ! -exec fuser -s {} \;  -empty -delete \
)

 

For both of the -type d operations I removed the dot before the asterisk at the end of each regex expression.  Now, by adding a directory called ".dnr" in each share right below the top level it seems to work the same but not delete the top level shares.

 

Question:  Does this break anything else?  I'm sure it would be blind to *any* lower level dot-directories, but is that of any concern?

 

--Bill

I think your change does break some things.  Specifically, it will skip a directory with a period in it anywhere other than the first character.

 

The expression can be interpreted as follows

 

For a directory path in the cache drive starting at the current directory of "."  (at /mnt/cache)

Let's take an example path of

./Pictures/Family/June 2001 Joe L. Family Outing/

 

'[.]/[^.].*'

[.]   =   match a single character of the set of characters contained within the square brackets... 

The matched portion of the sample directory is:

./Pictures/Family/June 2001 Joe L. Family Outing/

'[.]/[^.].*'

/    = match the "/" character separating the directories

The matched portion of the sample directory is:

./Pictures/Family/June 2001 Joe L. Family Outing/

'[.]/[^.].*'

[^.]   = match any single character EXCEPT a period

The matched portion of the sample directory is:

./Pictures/Family/June 2001 Joe L. Family Outing/

  This will NOT match directories starting with a leading period, therefore, they are excluded from being moved.

 

'[.]/[^.].*'

.   = match any single character

The matched portion of the sample directory is:

./Pictures/Family/June 2001 Joe L. Family Outing/

 

'[.]/[^.].*'

*   = match any number of character matching the previous expression (in this case, any number of any character)

The matched portion of the sample directory is:

./Pictures/Family/June 2001 Joe L. Family Outing/

With your change, and the removal of the "." before the "*", you change the meaning of the final "*" in the regular expression to match any number of characters that are not a period.  The "." in a regular expression you removed, used that way, would have had it match ANY character, including a period.

 

With your change, only the following portion of the directory path would match the last "*" portion of the regular expression:

./Pictures/Family/June 2001 Joe L. Family Outing/

and since the entire path did NOT match, the directory would not be created, or removed.

 

You can do what you were trying to accomplish without changing the regular expression at all, but by adding a "-mindepth 2" just before the "-type"

Like this in the delete command:

find . -mindepth 2  -type d  -regex '[.]/[^.].*'     ! -exec fuser -s {} \;  -empty -delete

With the addition, the directory path must be at least two deep before a match occurs and the directory removed.

 

You can not do the same on the mkdir command, otherwise, it would not automatically create top level directories on your disks when they exist on the cache drive, but not on the disk drive.

 

Joe L.

 

Edit: fixed typo..

Share this post


Link to post

Well done Joe L. That was a very nice walkthru of regex and find for those unfamiliar with either. I was going to do something similar but you've done it better than I would have.

Share this post


Link to post

I wholeheartedly agree!  That was an excellent description, Joe. 

 

So many years... I completely forgot that in this context, the first four matches were basically setting the stage for a repetition operator, not a wildcard.

 

The '-mindepth 2' expression in 'find' took care of my problem and is a much better solution anyway.

 

Thank You!

 

--Bill

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


Copyright © 2005-2018 Lime Technology, Inc.
unRAID® is a registered trademark of Lime Technology, Inc.