Support > User Customizations
nzbget 0.8 with par2lib & openssl
neilt0:
EDIT: There is now a 5.x plugin available for nzbget: http://lime-technology.com/forum/index.php?topic=23423.0
Preamble
With the generous help of the developer of nzbget -- hugbug, I have the latest version of nzbget (0.8) up and running on unRAID.
I think an older version was included with BubbaRAID and someone had posted information on how to install another version, but he couldn't get par2lib working, so it was disabled -- not so useful!
With a bit of work, we figured out how to patch par2lib and configure it correctly for the unRAID environment.
I like SABnzbd, but nzbget has some advantages, particularly when running on a server.
SABnzbd runs under Python, an interpreted language, making it relatively slow. nzbget is written in C, so is faster and has a smaller memory footprint.
Some approximate performance info I've found running the two apps on my server:
CPU usage (measured by top)
SABnzbd: 40-50%
nzbget: 7-14%
RAM usage when downloading (1GB installed)
SABnzbd: 5.6%
nzbget: 0.3%
Another benefit to nzget is that it uses par2lib, not par2cmdline.
Running par as a library, rather than an external command means that nzbget only needs to do one verify pass, then it can immediately grab the pars and repair the files, rather than as SABnzbd does: verify, get pars, verify again and repair.
This was really clear to me yesterday as I watched SAB repair a 40GB collection -- it took about 40 minutes for the verify pass, then ran that verify pass again once it had the pars.
Currently par2lib doesn't support multiple cores, but it would be interesting to compare overall repair times, and I might do that at some point.
With recent builds, nzbget is close to the functionality of SABnzbd. You can use it with Sick Beard via Sick Beard's nzb blackhole option.
Prerequisites
The web interface to nzbget is called nzbgetweb and it requires a php server, but it's very simple to set one up with unRAID. Install unRAID-Web, and download the nzbgetweb archive. Unpack nzbgetweb and put the nzbgetweb folder on your cache drive. Once that's done:
http://:tower89/ -> Personal Webserver Setup
Tell unRAID-Web where the files are, and which port you want to use. In my case, I used an .nzbget folder on my cache drive and port: 8084, so used: /mnt/cache/.nzbget/nzbgetweb/
This gives the nzbgetweb interface on: http://tower:8084/
Now, it just needs nzbget to talk to.
Dependencies
To obtain all the packages required for nzbget, I used http://pkgs.org/ .
I appended the name of the package needed to this URL. e.g. http://pkgs.org/package/libpar2 , then grabbed the Slackware 13.1 (or similar) x86 version and put that in /boot/packages/ (the packages folder on /tower/flash).
If there's any interest, I can package up the dependencies and make them available.
Make it so...or....go
I added the following sections to my go script:
--- Code: ---#################################################################
# install packages for nzbget #
#################################################################
cd /boot/packages
installpkg kernel-headers-2.6.33.4_smp-x86-1.txz
installpkg pkg-config-0.23-i486-2.txz
installpkg libxml2-2.7.6-i486-1.txz
installpkg make-3.81-i486-1.txz
installpkg libsigc++-2.2.8-i486-2sl.txz
installpkg binutils-2.20.51.0.8-i486-1.txz
installpkg glibc-2.11.1-i486-3.txz
installpkg perl-5.10.1-i486-1.txz
installpkg gcc-4.2.4-i486-1.tgz
installpkg gcc-g++-4.2.4-i486-1.tgz
installpkg ncurses-5.7-i486-1.txz
installpkg bind-9.4.3_P4-i486-1.txz
installpkg cxxlibs-6.0.13-i486-2.txz
installpkg patch-2.5.4-i486-1.txz
--- End code ---
There may be a package or two that's not strictly needed by nzbget or par2lib, but if so, I'm sure one of the experts here can advise.
openssl was already installed on my server by the SABnzbd script, so if you aren't using that, also add:
--- Code: ---installpkg openssl-0.9.8r-i486-3.txz
--- End code ---
You'll also need libpar2-0.2.tar.gz and nzbget-0.8.0-testing-r394.tar.gz in the packages folder on the flash drive, they are set up later in the script.
Immediately after this, I included the section for par2lib (note that nzbget includes patches for par2lib, so they are included here):
--- Code: ---#################################################################
# Compile par2lib #
#################################################################
cd /tmp
cp /boot/packages/nzbget-0.8.0-testing-r394.tar.gz .
tar -zxf nzbget-0.8.0-testing-r394.tar.gz
cp /boot/packages/libpar2-0.2.tar.gz .
tar -zxf libpar2-0.2.tar.gz
cd libpar2-0.2
cp /tmp/nzbget-0.8.0-testing/libpar2-0.2-*.patch .
patch < libpar2-0.2-bugfixes.patch
patch < libpar2-0.2-cancel.patch
./configure --prefix=/usr
make
make install
--- End code ---
and finally, nzbget:
--- Code: ---#################################################################
# Compile nzbget #
#################################################################
cd /tmp/nzbget-0.8.0-testing
./configure --with-tlslib=OpenSSL
make
make install
/tmp/nzbget-0.8.0-testing/nzbget -c /mnt/cache/.nzbget/nzbget.conf -D
--- End code ---
Now, bear in mind, I'm no Linux expert, so use anything in this post with care, and at your own risk.
I just figured most of this out with trial and error. I'm sure there is a way to build static binaries which can reside on the cache drive, to reduce boot time, or even an unmenu package, so any experts who know who to do this, please weigh in.
I put all my nzbget configuration files in /mnt/cache/.nzbget .
Just about all the settings can be configured in the web interface, but for reference, here are my conf files (with login stuff removed):
nzbget.conf: http://pastebin.com/M9z8YWrL
postprocess.conf:
--- Code: ---# Template configuration file for postprocessing script "postprocess-example.sh".
# Please refer to "postprocess-example.sh" for usage instructions.
##############################################################################
### PATHS ###
# Set the full path to unrar if it is not in your PATH.
UnrarCmd=unrar
##############################################################################
### OPTIONS ###
# Delete rar-files after unpacking (yes, no).
DeleteRarFiles=yes
# Rename img-files to iso (yes, no).
RenameIMG=yes
# Joint TS-files (yes, no).
JoinTS=yes
##############################################################################
### POSTPROCESSING-PARAMETERS ###
# This section defines parameters, which can be set for each nzb-file
# individually using either web-interface or command line.
# Example command line for setting parameter "password" to value "123" for
# nzb-file with id=2:
# nzbget -E G O Password=123 2
# Perform postprocessing (yes, no).
#
# Set to "no" to skip postprocessing for this nzb-file.
PostProcess=yes
# Password for encrypted posts.
#
# If the post requires a password for unpacking.
Password=
# Destination directory.
#
# NOTE: NZBGet must have write-access-rights for that directory.
DestDir=/mnt/cache
--- End code ---
nzgetweb/settings.php: http://pastebin.com/diLhnD2W
BRiT:
When you benchmarked SABnzbd did you precompile/optimize the Python files first?
You can tell if you did this by the presence of '.pyo' files alongside the '.pyc' files. Those are the compiled bytecode files. This is done by using the Python parameter of "-O0". You only need to do this once after installing.
I see nowhere near that heavy of CPU Usage on my system for SABnzbd.
This nzbget is shaping up to be a decent alternative to SABnzbdonce they get more features included.
neilt0:
--- Quote from: BRiT on April 05, 2011, 02:52:27 PM ---When you benchmarked SABnzbd did you precompile/optimize the Python files first?
--- End quote ---
Wow, I didn't even know you could do that. That optimization business doesn't seem to have been covered much in the SABnzbd discussions?
Isn't it "-OO", not "-O0"?
OK, so I ran the optimization, and RAM usage is down quite a bit, to 3.4%. CPU is the same, or possible even higher (these are not scientific tests, but it has certainly not dropped while downloading pretty much the same content).
Bear in mind, I'm running these applications on my HP Microserver with the 15W CPU -- one of the reasons I wanted to implement nzbget is that I knew it had a drastically smaller CPU and RAM footprint than SABnzbd. If you run nzbget on a more powerful machine, "top" might not even notice it's running!
What features do you think should be added to nzbget? I asked the developer to consider prioritisation, and he added that feature in the last Beta. He also added passworded rar extraction a while ago after I asked him about it -- a feature that the Newsbin developers kept fighting. hugbug is a top bloke.
jeff.lebowski:
--- Quote from: BRiT on April 05, 2011, 02:52:27 PM ---When you benchmarked SABnzbd did you precompile/optimize the Python files first?
You can tell if you did this by the presence of '.pyo' files alongside the '.pyc' files. Those are the compiled bytecode files. This is done by using the Python parameter of "-O0". You only need to do this once after installing.
I see nowhere near that heavy of CPU Usage on my system for SABnzbd.
This nzbget is shaping up to be a decent alternative to SABnzbdonce they get more features included.
--- End quote ---
Where are these located at? I looked in the flash drive, but couldn't find them.
neilt0:
--- Quote from: jeff.lebowski on April 05, 2011, 03:31:40 PM ---
--- Quote from: BRiT on April 05, 2011, 02:52:27 PM ---When you benchmarked SABnzbd did you precompile/optimize the Python files first?
You can tell if you did this by the presence of '.pyo' files alongside the '.pyc' files. Those are the compiled bytecode files. This is done by using the Python parameter of "-O0". You only need to do this once after installing.
I see nowhere near that heavy of CPU Usage on my system for SABnzbd.
This nzbget is shaping up to be a decent alternative to SABnzbdonce they get more features included.
--- End quote ---
Where are these located at? I looked in the flash drive, but couldn't find them.
--- End quote ---
Depending on where SABnzbd is installed -- if you used the unMenu conf, they should be in /mnt/cache/.sabnzbd/sabnzbd .
Navigation
[0] Message Index
[#] Next page
Go to full version