Help with rsync and deltacopy

[cyrnel]

Ah, the joys of Windows vs. nix permissions and Samba... Try adding the option "--chmod=Fo-X" to your rsync command. That keeps the other execute bit (001) clear for files. Samba maps it to the hidden attribute.

rsync -avP --delete --chmod=Fo-X 192.168.1.152::test /mnt/user/photobackup

[wsume99]

I added the --chmod=Fo-X option to the command and now the .exe files are visible in windows explorer. Thanks so much. I'm actually beginning to like the CLI.

I have another question. I assume that only one rsync instance can be running at a time between the client and the server. Is this correct? For example, if I wanted to backup c:/users/Ziggy/Downloads/ and c:/users/Ziggy/My Pictures/ I would have to write a single script like this:

Code: [Select]

#!/bin/bash

rsync -avP --delete --chmod=Fo-X 192.168.1.152::test /mnt/user/photobackups/Downloads/
rsync -avP --delete --chmod=Fo-X 192.168.1.152::test1 /mnt/user/photobackups/My Pictures/
Where rsyncd.conf defines test as /cygdrive/c/Users/Ziggy/Downloads/ and test1 as /cygdrive/c/Users/Ziggy/My Pictures/. I could then place the script into /etc/cron.daily/ and it should execute a backup once a day of the two folders, first test and then test1. Is that correct?

[cyrnel]

Multiple transfers can run simultaneously, up to the limits of max connections (in rsyncd.conf) and your system. Really though, it makes little sense to fill memory and thrash your disks between just two boxes. Better to do them serially as you are now.

I spoke earlier about not publishing an entire drive, but I'll go against that for now. You can make your setup simpler by backing out the module one or more levels. e.g. Change the module path for [test] to /cygdrive/c/Users/John. Then, in your rsync command, use:

rsync -avP --delete --chmod=Fo-X 192.168.1.152::test/Downloads/ /mnt/user/photobackups/Downloads/

If you plan on more than a few paths, better yet would be to put it in a loop using include-from to pull from a file containing your list of backup sources. Then add exclude-from to filter out the unwanted.

Echoing myself, you might also look into the -b and --backup-dir options which make versioning simple. Create a timestamp, use it in a directory name for --backup-dir, and anything that's changed or been removed will be moved there instead of being lost with the --delete option.

[wsume99]

Now I'm having trouble with the spaces in the path /mnt/user/photobackups/My Pictures/. I had this issue before but I just eliminated the spaces from my test directories. That is not really a solution because I don't want to have to rename all my existing directories. I've tried all of the variations below and it still won't work?

Code: [Select]

root@Tower:/etc/cron.hourly# rsync -avP --delete --chmod=Fo-X 192.168.1.152::tes
t1 /mnt/user/photobackups/My Pictures/
Unexpected local arg: /mnt/user/photobackups/My
If arg is a remote file/dir, prefix it with a colon (:).
rsync error: syntax or usage error (code 1) at main.c(1207) [receiver=3.0.2]
root@Tower:/etc/cron.hourly# rsync -avP --delete --chmod=Fo-X 192.168.1.152::tes
t1 "/mnt/user/photobackups/My Pictures/"
@ERROR: chdir failed
rsync error: error starting client-server protocol (code 5) at main.c(1495) [rec
eiver=3.0.2]
root@Tower:/etc/cron.hourly# rsync -avP --delete --chmod=Fo-X 192.168.1.152::tes
t1 '/mnt/user/photobackups/My Pictures/'
@ERROR: chdir failed
rsync error: error starting client-server protocol (code 5) at main.c(1495) [rec
eiver=3.0.2]
root@Tower:/etc/cron.hourly# rsync -avP --delete --chmod=Fo-X 192.168.1.152::tes
t1 /mnt/user/photobackups/My\ Pictures/
@ERROR: chdir failed
rsync error: error starting client-server protocol (code 5) at main.c(1495) [rec
eiver=3.0.2]
root@Tower:/etc/cron.hourly# rsync -avP --delete --chmod=Fo-X 192.168.1.152::tes
t1 /mnt/user/photobackups/My"\" Pictures/
I think I will look into the -b and --backup-dir options. I like the thought of having older versions of files for maybe a month or so. But first thing for me is to get the automation working the way I need it to, then I can play with the options. I never imagined that I would need to do so much experimenting with test files - silly me.

EDIT: I'm such a noob. All the options I tried did work - well at least the space in the path wasn't the problem.  The first error was a result of the space in the path but the rest of them were a problem with the test1 module I've created. I'm not sure what's wrong with it yet but I'll have to look into it.

Also, I found that /mnt/user/photobackups/"My Pictures"/ syntax works as well.

[cyrnel]

I think libraries are getting in your way. There isn't a /users/name/My Pictures from cygwin's perspective. Try navigating from the windows command line to see what cwrsync sees.

Just "Pictures" should do.

rsync -avP --delete --chmod=Fo-X 192.168.1.152::test1 /mnt/user/photobackups/Pictures/

[wsume99]

The problem appears to be the spaces in the path for the module I defined in the rsyncd.conf file on the windows machine.

I have the following:

Code: [Select]

uid = 0
gid = 0
use chroot = false
strict modes = false
hosts allow = 192.168.1.150
log file = rsyncd.log

# Module definitions
# Remember cygwin naming conventions : c:\work becomes /cygwin/c/work
#
[test]
path = /cygdrive/c/Users/John/"My Pictures"/
read only = true
transfer logging = yes
Again the problem is the fact that there are spaces in the path. The folder c:\Users\John\My Pictures\ does exist on the machine. I've read quite a few posts with similar issues but haven't seen a solid solution yet.

[cyrnel]

Can you paste the command prompt output from:

dir c:\Users\John

[wsume99]

Ah Ha - I guess I should have listened better when you warned me about libraries but I didn't think that was the problem because I could see the My Pictures folder in windows explorer. I guess the lesson I should learn here is not to trust what windows explorer says.

Code: [Select]

C:\Users\John>dir
 
 Directory of C:\Users\John

07/14/2011  03:18 AM    <DIR>          .
07/14/2011  03:18 AM    <DIR>          ..
10/10/2010  01:27 PM    <DIR>          .idlerc
04/12/2011  10:04 PM    <DIR>          Contacts
09/04/2011  10:55 PM    <DIR>          Desktop
07/16/2011  02:32 PM    <DIR>          Documents
08/19/2011  09:05 PM    <DIR>          Downloads
04/12/2011  10:04 PM    <DIR>          Favorites
04/12/2011  10:04 PM    <DIR>          Links
02/23/2011  07:43 PM    <DIR>          Logitech
04/12/2011  10:04 PM    <DIR>          Music
09/01/2011  11:00 PM    <DIR>          Pictures
04/12/2011  10:04 PM    <DIR>          Saved Games
04/12/2011  10:04 PM    <DIR>          Searches
03/14/2010  05:58 PM                 0 Sti_Trace.log
04/12/2011  10:04 PM    <DIR>          Videos
09/12/2010  04:51 PM                99 wakeup.bat
               2 File(s)             99 bytes
              15 Dir(s)  13,327,364,096 bytes free
So according to the dir listing there is no My Pictures folder. The proper path should be c:/Users/John/Pictures/. So the rsyncd.conf file should have a module named test1 with this path:

Code: [Select]

path = /cygdrive/c/Users/John/Pictures/
and the rsync command would be:

Code: [Select]

rsync -avP --delete --chmod=Fo-X 192.168.1.152::test1 /mnt/user/photobackups/"whatever I want to call it but I need quotes if there are spaces"/

[cyrnel]

I well remember my own shock at seeing that the first time.

I'm sure it's all about compatibility. With software, not us.

Again, I'd publish the module as /cygdrive/c/Users/John, or even higher up the tree so can get whatever you like without adding more modules. e.g.:

Code: [Select]

If module uses                      rsync uses
/cygdrive/c/Users/John/Pictures     address::module
/cygdrive/c/Users/John              address::module/Pictures
/cygdrive/Users                     address::module/John/Pictures
...

If you get to the point of looping through your list of sources vs. hard-coding individual rsync lines, it's less confusing to have one point of origin.

Code: [Select]

rsync -avP --delete --chmod=Fo-X 192.168.1.152::test1 /mnt/user/photobackups/"whatever I want to call it but I need quotes if there are spaces"/
Yep, quotes or escape them with \.

[dalben]

I've been watching this thread with interest. I've trolled Google and can't find anywhere that explains the syntax and usage of the --chmod switch. If you could point me towards a page or reference I'd be most grateful.

[cyrnel]

It's standard chmod "funky mode" syntax (as opposed to the more obvious numeric mode).

Basically:
D for directory, F for file
ugoa for user, group, other, all
+ or - for set or clear
rwx for read, write, execute

Try googling chmod man.

[wsume99]

Thanks again for all your help cyrnel.  I think I finally have it working the way I want it.

I've done some more experimenting and what I've come to understand is that you cannot have a space in the path when defining a module.  For example I tried this module and rysnc command combination just for kicks:

Code: [Select]

[test1]
path = /cygdrive/c/Users/John/"Saved Games"
read only = true
transfer logging = yes

rsync -avP --delete --chmod=Fo-X 192.168.1.152::test1 /mnt/user/photobackups/"Saved Games"/This did not work. I got the same "chdir failed" error I had gotten previously and if you look back at the dir listing for c:/Users/John/ you'll see that a Saved Games directory does exist.

However if I use this module and rsync command combination it works (just simply taking cyrnel's advice and driving up the tree so that I do not have any spaces in the module path).

Code: [Select]

[test1]
path = /cygdrive/c/Users/John
read only = true
transfer logging = yes

rsync -avP --delete --chmod=Fo-X 192.168.1.152::test1/"Saved Games" /mnt/user/photobackups/
That code works. It created a Saved Games directory in /mnt/user/photobackups/ and copied all the flies from c:/Users/John/Saved Games/ and places it in /mnt/user/photobackups/Saved Games/. I think I may finally understand this. 

[cyrnel]

Good deal!

If I recall, the code that reads rsyncd.conf and builds modules essentially treats those lines as #defines; it isn't like shell. You could hard-code your module in a recompiled rsync but that's getting desperate. Better to do as you are.