CrashPlan

Yes, removing the restriction from wan allowed me to max the connection out, which is obviously running at lan speeds.  I would have thought wan was through the internet.  I know at home between my laptop and server it runs at the 'lan' speeds.  Unfortunately, I will have to remember to turn the 'wan' speed setting down when I am at home so it doesn't max out my cable modem.  Won't really be a problem after it uploads everything initially.  Thanks again.

What does your network look like? Is your laptop on a different subnet than the crashplan server at your work?

You could have a look on the crashplan website and see if you can find out how it defines a WAN connection. But I reckon it must just assume if a machine isn't on the same subnet...

Same subnet, same gateway.  Laptop is on wireless, but that shouldn't matter.  Not a big deal though since it at least is going faster now.

I've got Crashplan setup on my Unraid box in headless mode, and I am able to connect and begin backups - works great.  After some time, it eventually runs out of memory and crashes, taking down other things with it.  I have a feeling it is choking on large video files I am trying to backup.  Now when I start it, it crashes nearly immediately (leading me to believe it is the size of the files that is killing it).  I have 4GB of RAM, and nothing else major running.

I have it setup to store the backups in /mnt/disk5/Backup (as opposed to a user share). 

What am I doing wrong?  Is it just the sheer size of the files I am trying to backup?  I lost my syslog because I just rebooted, sorry. 

Thanks!

I've got Crashplan setup on my Unraid box in headless mode, and I am able to connect and begin backups - works great.  After some time, it eventually runs out of memory and crashes, taking down other things with it.  I have a feeling it is choking on large video files I am trying to backup.  Now when I start it, it crashes nearly immediately (leading me to believe it is the size of the files that is killing it).  I have 4GB of RAM, and nothing else major running.

 

I have it setup to store the backups in /mnt/disk5/Backup (as opposed to a user share). 

 

What am I doing wrong?  Is it just the sheer size of the files I am trying to backup?  I lost my syslog because I just rebooted, sorry. 

 

Thanks!

I run two crashplan servers, one with 2 meg and one with 1 meg and I haven't ran into many problems except for when I was backing up to memory.  That killed everything realy quick.  Most of the files I have backed up so far are 4gigs or less though.  Maybe for some reason it or some other application is filling your memory up with log entries.  Sorry I can't be of more help.

On a side note - I went through my first CrashPlan upgrade.  The upgrades went ok but it didn't start the crashplan engine up automatically after the upgrade.  Is that normal?  Kind of stinks if that is the case.  Thanks.

I have it setup to store the backups in /mnt/disk5/Backup (as opposed to a user share). 

Have you confirmed the data is actually in there? And not living in ram?

Im having an issue where I have to login with the gui with the serviceport set to 4200 and then logging into my account.  I checked the tar is extracting the .identity file to /var/lib/crashplan.  I have installed crashplan to a hidden directory on my cache drive so that's always there.  Everything looks the way it should so I'm out of ideas.

Also I found even if I set the manifest directory somewhere else on my system or if I manually set it in the gui, it would only backup to a folder called backupArchives in the crashplan installation directory, so i created a sym link to where I actually wanted it to save.

I'm using crashplan 3.0.2 according to the gui and everything is running through my home network via lan.

I have it setup to store the backups in /mnt/disk5/Backup (as opposed to a user share). 

 

Have you confirmed the data is actually in there? And not living in ram?

HA!  No I hadn't, and of course that directory looks empty.  So instead of storing it there, it is going into RAM and crashing.  How do I fix that?  I specified /mnt/disk5/Backup upon install, do I need to set it in the GUI as well?

Thanks so much!

Yes, from the gui, click on settings, general tab, enable button and then at the bottom you can pick the location.

Maybe the wiki or directions should be changed.  Because regardless of what you pick at time of installation, it keeps defaulting to memory for the backup location.

Yes- this seems to be a change in behaviour

Check it via the gui, set it if need be, save everything, then retar crashplan to see if the settings stick.

If this is indeed the case with the new install we are going to have to complain to crashplan about this.  either do not ask for a backup location via the command line or when you do at least honor it!!

This also happened with 3.0.1 because I got burned with it as well.

Yes- this seems to be a change in behaviour

Outstanding, thanks icon123 and boof!  I appreciate the lightning quick responses.  I hadn't even thought to check that since the cli seemed to set the backup location.  I'd be happy to update the wiki to help out the next user, if you'd like.

Cheers.

I just got CrashPlan up and running successfully (much to my surprise)...much thanks to boof and prostuff1 for their step by steps.  When Mozy abandoned their customers I took advantage of the 15% CrashPlan discount and got 4 years unlimited...so I figured it was time to set this up.

One question, when I select my files to backup, I see all my disks there, and then there is a 'user' and 'user0' that appear to contain my shares.  Is there any difference between the two and can I use that to specify my backup source?  Or is it better to just use the disk structure?

I just got CrashPlan up and running successfully (much to my surprise)...much thanks to boof and prostuff1 for their step by steps.  When Mozy abandoned their customers I took advantage of the 15% CrashPlan discount and got 4 years unlimited...so I figured it was time to set this up.

 

One question, when I select my files to backup, I see all my disks there, and then there is a 'user' and 'user0' that appear to contain my shares.  Is there any difference between the two and can I use that to specify my backup source?  Or is it better to just use the disk structure?

"user0" is the same view, but without the contents of the cache drive.

I put together an updated guide since I just went through the process and the existing (and extremely helpful) guides by boof and prostuff are nearly 1 1/2 years old now.  I basically took their guides and made notes as I went through the process to hopefully make it easier for people such as myself that have basically zero Linux (or even unRAID) background.  Hopefully this guide can also help the people that just discovered this thread and prevent them from having to read through all 23 pages.  Feel free to put this on the Wiki.

I'm new to Crashplan, but I'm really liking it so far.  I know you can use it for free if you backup to your own (or friends') resources, but I purchased their unlimited cloud storage "plus" plan.  They are offering a 15% discount (good for any of their plans) if you go to http://www.crashplan.com/mozyonover and sign up for one of their pay plans.  If you do the yearly plan, that's $42 for unlimited storage for a year.  Very reasonably priced for unlimited storage if you ask me.  Not sure how long that discount will be good for.

This guide assumes you're using a Windows desktop/laptop, but it probably wouldn't be all that different for a Mac or Linux distro.  For a Mac, prostuff has some scripts in this thread to help with that process.

Here we go...

- Get the latest version of unMENU and install on your unRAID server.  This is the easiest way for the Linux-challenged like myself to get this done the right way.  There is plenty of info on installing unMENU out there so I won't get into that.  To make it even easier, here's the Wiki entry: http://www.lime-technology.com/wiki/index.php?title=UnMENU_documentation

- Once you have unMENU installed, go to http://servername:8080/ (servername being the name of your unRAID server) in a browser on your Windows desktop.  This takes you to the unMENU web interface.

- Once unMENU loads, click the Pkg Manager menu option at the top.

- Go down to the 'openssh' package and click the button 'Select openssh-5.1p1-i486-1.tgz' (that is the version as of this writing so this may change).  Once installed, click 'Enable Re-Install on Re-Boot' button.  This will ensure SSH gets reinstalled on reboot.

- Go back to the Pkg Manager and find the 'openssl-solibs (OpenSSL shared libraries)' package and click the button 'Select openssl-solibs-0.9.8i-i486-1.tgz' (that is the version as of this writing so this may change).  Once installed, click 'Enable Re-Install on Re-Boot' button.  This will ensure openssl gets reinstalled on reboot.

- Go back to the Pkg Manager and find the 'GNU cpio copies files into or out of a cpio or tar archive' package and click the button 'Select cpio-2.5-i486-3.tgz' (that is the version as of this writing so this may change).  Once installed, click 'Enable Re-Install on Re-Boot' button.  This will cpio gets reinstalled on reboot.

Those are the only 3 packages you will need for this (other than the Crashplan client itself).

- Now go to your unRAID server terminal either directly on the machine or via your Telnet client of choice and login to your server (if you don't have a password, set one as SSH does not allow blank passwords.  If you MUST have a blank password on your root account, you can create a new user account on the terminal).

- Time to start SSH so type:

/etc/rc.d/rc.sshd start

- Now create the a public/private key pair by typing:

ssh-keygen -t rsa

- First you will be prompted for a file name.  I just chose 'cpkey'.

- Next you will be prompted for a password.  Create one, and remember it (although I'm not sure what it's used for).

- Now copy the files that have been created by the install of ssh:

cp --preserve=timestamps /etc/ssh/* /boot/custom/etc/ssh

- Now add the following lines of code to your GO script to copy the key files and start SSH on server start.  To update your GO script, I recommend using unMENU.  Just start up unMENU again (http://servername:8080/) and go the the 'Config View/Edit' menu option.  You will see a dropdown there that includes all of your script/config/system files.  Select '/boot/config/go' from the dropdown and that will load your GO script in readonly mode.  To edit it, just click the 'Edit' button at the bottom.

cp --preserve=timestamps /boot/custom/etc/ssh/ssh*key* /etc/ssh
chmod 600 /etc/ssh/ssh*key*
/etc/rc.d/rc.sshd start

- Now install Crashplan on your Windows desktop if it isn't already installed - http://www.crashplan.com/consumer/download.html (while you're there, download the Linux client as well since you'll need that to install on your unRAID box).  I recommend setting up your Crashplan account at this point as well.  You can do that when you run the Windows client.

- Now install the Crashplan Linux client on your unRAID server.  First, copy the TGZ file to your packages folder on your flash drive.  This folder may vary by the unRAID version you're using.  For me, it was 'boot/packages'.

- Now go to your unRAID console, cd to the folder of the TGZ file in the previous step, and Tar the install file by typing:

tar -zxfv CrashPlan_3.0.2_Linux.tgz (Your filename may very depending on version)

- Now run the install file:

install.sh

- Now follow the prompts:

    - Say yes to the Java Runtime install if asked

    - Enter anything for the rc scripts, we won't be using them

    - The manifest directory is where your backups will live. You will need to change this later in the client, but for now put one of your data drives so it doesn't start backing up to memory.  You could do something like /mnt/disk1/Backups.  Personally I backup to their cloud so I set that up later in the client.

    - Accept the defaults for the rest of the prompts.

    - Once finished, Crashplan should now be installed on your unRAID box.

- Now we need to set up your Windows install of Crashplan to connect to the "headless" install on your unRAID box.  So on your Windows desktop, go to your CrashPlan install folder, most likely 'C:\Program Files\CrashPlan', and then go to the 'conf' folder.  Open the 'ui.properties' file for editing, and add the following line to the end of the file and save & close it:

servicePort=4200

- This is the port we will use in the next step when setting up the tunnel to access the headless install.

- Now we need to set up an SSH Tunnel on your Windows desktop.  I use Putty (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html) for this but I'm sure there are other programs that can be used.

- Once you run Putty, go to Connections > SSH > Tunnels.  Enter 4200 for the source port, then enter 4243 for the destination and click 'Add'.

- Now (still in Putty), go back to 'Session' and enter the name or IP address of your unRAID server in the 'Host Name' field.  Leave the port at 22, and select 'SSH' for 'Connection type'.

- So you don't have to set this up every time you want to connect to your unRAID headless Crashplan install, give this session a name (how about 'unRAID'?) and click 'Save'.

- Now click 'Open' to open the tunnel.  You should get a terminal with a prompt for username/password.  Enter your unRAID username and password here.

- Now run the Crashplan client on your Windows desktop and enter your (hopefully already created) login credentials.  If all went well, you should establish a connection and you will be connected to the Crashplan engine on your unRAID box.

- If everything looks good, close the Crashplan client in Windows.  Remember if you ever want to connect to the Crashplan engine on your Windows machine you need to comment out that line in the  'ui.properties' file.

- Now we need to make the Crashplan install on your unRAID box persistent on a reboot.  To do that, first stop the Crashplan engine on your unRAID server cleanly:

/usr/local/crashplan/bin/CrashPlanEngine stop

- Tar the Crashplan folder which at this point should also include a crucial .identity file that was created when you connected to the Crashplan engine via your Windows desktop.  To do this, type:

tar -cvf /boot/packages/crashplan.tar /usr/local/crashplan /var/lib/crashplan

- Now add the following lines to your GO script.  This will install and start the Crashplan engine on startup:

# install crashplan
tar -C / -xvf /boot/packages/crashplan.tar
/usr/local/crashplan/bin/CrashPlanEngine start

- That should do it!  I would do a reboot and make sure everything starts up correctly and try connecting to the Crashplan engine via the Windows client again and make sure the connection still works.  Remember to add that line to your ui.properties file to connect to the unRAID engine (if you removed it).  Also remember every time you want to connect to the unRAID Crashplan engine, you will need to start that SSH tunnel session via PUTTY (or the SSH Telnet of your choosing).

A few notes:

- Make sure you connect to the unRAID engine through your Windows desktop and ensure the backup destination is set properly.  There was a report of someone not doing this and it was storing the data in RAM and eventually crashed.

- It's been said on this thread that the Crashplan engine auto-updates and will cause you to not be able to connect to your unRAID Crashplan Engine.  There is a script in unMENU (Under 'User Scripts') called 'Tar Crashplan File' which will re-tar the new version install of Crashplan on your server and should correct the problem.  Make sure you click the 'Stop Crashplan Engine' script button before doing this.  Then after tar, click the 'Start Crashplan Engine' button.

- One thing I would like to add to this is how to add the Crashplan stop functionality to the Clean Powerdown script so Crashplan stops cleanly when the server is powered down.  I have no idea how to do that.

- One thing I would like to add to this is how to add the Crashplan stop functionality to the Clean Powerdown script so Crashplan stops cleanly when the server is powered down.  I have no idea how to do that.

If you go to your unMENU folder on your flash drive and find the Clean Powerdown script, you can just add this line before the powerdown:

/usr/local/crashplan/bin/CrashPlanEngine stop

Of course, that works until Clean Powerdown is updated. 

- One thing I would like to add to this is how to add the Crashplan stop functionality to the Clean Powerdown script so Crashplan stops cleanly when the server is powered down.  I have no idea how to do that.

If you go to your unMENU folder on your flash drive and find the Clean Powerdown script, you can just add this line before the powerdown:

/usr/local/crashplan/bin/CrashPlanEngine stop

Of course, that works until Clean Powerdown is updated. 

Okay, thanks, I see now.  I was thinking we would have to modify the actual powerdown script, rather than just the one that calls the powerdown script.

Okay, thanks, I see now.  I was thinking we would have to modify the actual powerdown script, rather than just the one that calls the powerdown script.

Not in this case.  The calling script can run any sequence of command line code, just like the GO script.  You might want to add a "sleep 5" or something in there, to make sure crashplan has stopped before powerdown is called.

That is a good question thou. What happens if there is a power outage and unRAID needs to shut down. How is it going to shutdown the array if it can't because CrashPlan is running?

That is a good question thou. What happens if there is a power outage and unRAID needs to shut down. How is it going to shutdown the array if it can't because CrashPlan is running?

I use a hook into rc.samba init script. It allow me to stop, start and reboot using the web GUI.

#!/bin/bash

## SED new routines to rc.samba
sed -i -e  "/case/,/esac/c \
case \"\$1\" in\n'start')\n  samba_start\n  sh /etc/rc.d/rc.custom start 1>/dev/null 2>&1 &\n  ;;\n'stop')\n\
  sh /etc/rc.d/rc.custom stop 1>/dev/null 2>&1\n  samba_stop\n  ;;\n'restart')\n  samba_restart\n  ;;\n\
*)\n  samba_start\nesac" /etc/rc.d/rc.samba

## Create rc.custom
echo >/etc/rc.d/rc.custom
cat >>/etc/rc.d/rc.custom << 'THE_END_OF_SCRIPT'
M#!/bin/bash
MP="$(dirname "$(readlink -f ${BASH_SOURCE[0]})")"
M
Mstart() {
M    for i in 0 1 2 3 4 5 6 7 8 9 10; do
M        if [[ `df 2>&1` == */mnt/user* ]]; then
M            find /mnt/cache/.addons/ -name '*.uis' -type f -print | sort | xargs -n1 -IMYFILES sh -C MYFILES start
M            break
M        fi
M        sleep 3
M    done
M    exit
M}
M
Mstop() {
M    find /mnt/cache/.addons/ -name '*.uis' -type f -print | sort | xargs -n1 -IMYFILES sh -C MYFILES stop
M    sleep 3
M}
M
Minstall() {
M    find /mnt/cache/.addons/ -name '*.uis' -type f -print | sort | xargs -n1 -IMYFILES sh -C MYFILES install
M}
M
Mcase "$1" in
M        'start')
M            start
M        ;;
M        'stop')
M            stop
M        ;;
M        'install')
M            install
Mesac
THE_END_OF_SCRIPT
sed -i "s/^M//" /etc/rc.d/rc.custom
chmod 755 /etc/rc.d/rc.custom

But this approach requires that every addon has a init script, e.g. crashplan.uis, with the start, stop and install routines. Here is a sample of my scripts:

#!/bin/bash
P="$(dirname "$(readlink -f ${BASH_SOURCE[0]})")"

## Commands used to start
start() {
    #start_script_pid <basename>   <dirname>         <init command>                            <pid file>
    start_script_pid "SABnzbd"     "$P/sabnzbd/"     "python SABnzbd.py -s 192.168.0.100:8081" "/var/run/sabnzbd.pid"
    start_script_pid "SickBeard"   "$P/sickbeard/"   "python SickBeard.py --quiet"             "/var/run/sickbeard.pid"
    start_script_pid "CouchPotato" "$P/couchpotato/" "python CouchPotato.py --quiet"           "/var/run/couchpotato.pid"
}

## Commands used to stop
stop() {
    stop_script_pid "SABnzbd" "/var/run/sabnzbd.pid"
    stop_script_pid "SickBeard" "/var/run/sickbeard.pid"
    stop_script_pid "CouchPotato" "/var/run/couchpotato.pid"
}

restart() {
    stop
    sleep 5
    start
}

install() {
    installpkg $P/SABnzbdDependencies-2.1-i486-unRAID.tgz
}

## Do NOT edit from this line bellow

start_script_pid(){
    if [ ! -f "$4" ]; then
        echo "Starting $1 ...."
cd $2
        $3 1>/dev/null 2>&1 &
        if [[ $! -gt 0 ]]; then
                echo $! > $4
                echo -e "\n  $1 started successfully!  [Process: $!]\n"
        else
                echo -e "\n  $1 failed to start. Look at the log file.\n"
                exit 1
        fi
sleep 1
    else
echo -e "  $1 already running. Try to stop first.\n"
    fi
}

stop_script_pid(){
    if [ -f "$2" ]; then
        kill -9 `cat $2` 1>/dev/null 2>&1
        rm $2 1>/dev/null 2>&1
        echo -e "\n  $1 stopped!\n"
        sleep 1
    else
        ps -eo 'pid,cmd' | grep "$1" | cut -d" " -f1 2>&1 | while read line; do
            pid="$line"
            [[ -n $pid ]] && kill -9 $pid 1>/dev/null 2>&1 &&\
            echo -e "\nPID file not found, but $1 was killed at PID: $pid\n"
        done
    fi
}

case "$1" in
'start')
	start
;;
'stop')
	stop
;;
'restart')
	restart
;;
'install')
	install
esac

Now I'm really confused. I was backing up a PC then it finished. I switched to another PC and then it just stopped.

At first I thought I wonder if I did the NO, NO and had it backing up to Ram. I checked the location and there are files on my /mnt/disk4/Downloads folder so I know it was moving them there.

I checked my machine and it says the service is running, but neither of my 2 machines seem to be able to connect and it says waiting for connection. Rebooted my machine just incase something locked up and it still says the same thing. I guess its time to walk down the install again to see if some how something glitched.

I could see if it was a reboot issue, but I rebooted yesterday to make sure it would stick on a reboot and sure enough it did. Who knows LOL

I checked my machine and it says the service is running, but neither of my 2 machines seem to be able to connect and it says waiting for connection.

When you login to Crashplan's website and check your systems there, what does it say about the last backup time?

That is a good question thou. What happens if there is a power outage and unRAID needs to shut down. How is it going to shutdown the array if it can't because CrashPlan is running?

So what was the consensus on this?  I know if crashplan is running or at least 'working', I can not stop the array.

That is a good question thou. What happens if there is a power outage and unRAID needs to shut down. How is it going to shutdown the array if it can't because CrashPlan is running?

 

So what was the consensus on this?  I know if crashplan is running or at least 'working', I can not stop the array.

The powerdown addon through unMenu in conjunction with the apc package will try to stop crashplan, though not as cleanly as sending CrashplanEngine the stop command, so you should be fine.