Running unRAID with a full Slackware distro

I've wanted to run unRAID on a full Slackware distro for some time, and josetann rekindled that interest.  I have 3 always-on boxes:  unRAID, Trixbox (phone/asterisk) and a Window's box for home automation and some other miscellaneous stuff.  My ULTIMATE goal is to merge them all into one box (unRAID) with Windows XP in a VM.

 

My procedure is not polished yet, but it works without any packages or blind copying.  It is pretty straightforward for anyone with average Linux experience and who is not afraid to build a kernel.

 

Note, I do not use User Shares... so I am not testing them under this config.

 

My goal is to have an unRAID system on a full Slackware install, that I can add additional software too, and compile applications (and add virtual machines).  But, I want to link back to the flash for configuration info, so if something pukes, I can boot from the USB, and have unRAID up and running in an emergency.... just bypassing the had drive boot.

 

================================

Install Slackware and modify the Lilo config

================================

Download Slacky 12.1, burn ISO to DVD

Partition your disk (/dev/hda)

Install Slackware 12.1, full system

Create lilo entry for alternative kernel for recovery if new kernel hangs

* ls -la /boot and see what file vmlinuz is linked to (probably vmlinuz-huge-smp-2.6.24.5-smp)

* vi /etc/lilo.conf and duplicate the linux boot section, changing the “image=” line to point to to the real file vmlinuz was symlinked to.

save and exit

run lilo -v

 

================================

Extract initramfs files from bzroot

================================

Boot back to Slackware, boot alternative kernel entry to make sure it works

mkdir /uraid

cd /uraid

wget http://lime-technology.com/dnlds/unRAID%20Server%204.3-beta6.zip

unzip *.zip bzroot

zcat bzroot | cpio -m -i -d -H newc --no-absolute-filenames

 

================================

Prepare source, and copy unRAID config and source mods

================================

get linux source for 2.6.24.4 and untar it in /usr/src

rename it to linux-2.6.24.4-unraid-special

symlink linux (ln -s  linux-2.6.24.4-unraid-special linux)

cd /usr/src/linux

cp /uraid/usr/src/linux/.config .

cd /usr/src/linux/drivers/md

cp /uraid/usr/src/linux/drivers/md/* .

 

================================

Modify the unRAID config

================================

This is important.  UnRAID uses an initramfs to provide the root file system to the kernel.  We are going to use a hard drive for the root file system, so the drivers for Linux to access that drive have to be built into the kernel, and not left as modules.

 

cd /usr/src/linux

make clean

make menuconfig

You have to include (not modularize) drivers for boot disk.  My boot disk is IDE.

Unselect Initial RAM filesystem and RAM disk (initramfs/initrd) support

Set the following to (*) and not as (M):

- File Systems /  Ext3 journalling file system support

- Device Drivers / ATA/ATAPI/MFM/RLL support /  Include IDE/ATA-2 DISK support

- Enhanced IDE/MFM/RLL disk/cdrom/tape/floppy support

- generic/default IDE chipset support

- JMicron JMB36x support (if you have it)

The above are what works for my system.... you will need to select the necessary drivers for your system.  You only need the drivers for the boot drive, not the rest of the unRAID disks.  So if the boot drive is IDE, and the unRIAD disks are all SATA, you only need the IDE drivers at this point.

You may need to enable something in Serial ATA (prod) and Parallel ATA (experimental) drivers if your boot drive is SATA.

 

save and exit

 

================================

Make new kernel

================================

make

make modules

make modules install

make install

reboot into the new kernel

if it panics, reboot back to the alternate kernel, and check the disk drivers particularly if the panic was a failure when attempting to mount the root fs.

Once your new kernel is booting, shut down and congratulate yourself.

 

================================

Make unRAID flash

================================

Make unRAID USB stick and boot it, and configure.  Shutdown.

Remove flash, boot back to Slackware, and reinsert flash (actually, I just make the disk a higher boot priority than the flash, and leave it installed).

 

================================

Mount USB stick, link to config on unRAID distro

================================

mkdir /mnt/uraid

mount /dev/sda1 /mnt/uraid  (also make an entry in fstab)

cd /boot

rm config

ln -s /mnt/uraid/config config

 

================================

Copy unRAID mods

================================

I diff’d the unRAID distro to the Slackware distro, and all you really need to copy are these directories to your Slackware distro:

/uraid/root

/uraid/usr/local/sbin

/uraid/etc/samba

 

Plus you need these files (or merge their changes from stock into yours) from /uraid/etc/rc.d:

rc.samba

rc.ifplugd

rc.local

rc.syslog

 

Yes, there are some other files with changes, but I found they were not apparently critical to unRAID.  You can diff them and see.  If anyone finds something else important, be sure to post it.

 

I did some preliminary testing with this system, and xfer rates are the same as when booting from the USB stick.

 

Now, I probably missed something, so anyone else who tries this, please chime in with corrections or additions.  After I get some further input, I’ll clean this up and post it in the wiki.  The only hard part was figuring out which ATA/IDE disk drivers to build into the kernel and which to leave as modules.... that’s going to be different for some people.

 

I’m now surfing with KDE and Firefox from the unRAID host, and streaming a movie to the TV, all at the same time.

 

Now to install a Virtualbox, Apache, and some other toys. 

Thanks bubbaQ - This is awesome!  I am going to try installing this soon.

 

If I could somehow get my unRAID box to play MP3's, I could decommission a Windows box!

Define "play"... do you mean play them on the speakers on the sound card inthe unRAID box, or serve them up to another device for playing?

If I could somehow get my unRAID box to play MP3's, I could decommission a Windows box!

 

Look at MPD. (have we had this conversation before?)

 

http://www.filewatcher.com/b/ftp/ftp.slackware.pl/pub/slackbuilds.org/slackbuilds/12.0/multimedia/mpd.0.0.html

Just has to be compiled and  packaged up.

 

Along wioth gathering all the support libs and programs.

We were talking abou BNC.  I will look at MFD.

 

bubbaQ - play = output using the unRAID box soundcard into whole house stereo

I would love to see someone get SABnzbd (http://www.sabnzbd.org) running on unRaid!  then i could turn off that extra windows box running, they have a linux version of this great program.

I successfully installed VirtualBox, created a VM for WindowsXP, and can remote control it from another XP machine via the Windows Remote Desktop Connection, and the VM can reach the Internet, as well as my intranet.

 

I am intentionally testing all this on a modest system (K8N-E, Athlon64, 3200) with only 500MB , and performance (except for Windows Remote Desktop Connection) so far is acceptable.

I successfully installed VirtualBox, created a VM for WindowsXP, and can remote control it from another XP machine via the Windows Remote Desktop Connection, and the VM can reach the Internet, as well as my intranet.

 

I am intentionally testing all this on a modest system (K8N-E, Athlon64, 3200) with only 500MB , and performance (except for Windows Remote Desktop Connection) so far is acceptable.

 

 

I  would like this as well!  Let me know when i can test it out? 

 

Thanks

Please don't misunderstand... this project will not produce any files or packages to download.  It is just instructions to follow, and some steps will have to be interpreted and modified for your hardware.

 

BTW, I tried remote control with Dameware (commercial package I have for client support) and it works very well for remote control of the Windows VM guested on unRAID from another Windows box.

I successfully installed VirtualBox, created a VM for WindowsXP, and can remote control it from another XP machine via the Windows Remote Desktop Connection, and the VM can reach the Internet, as well as my intranet.

 

Great work, bubbaQ!  Which edition of VirtualBox are you running (open source or closed-source)?  Can you tell me if you have SysV IPC support enabled for your kernel?  In my bouncing back and forth between VMware Server and VirtualBox, I ran into some issues trying to get Vbox (closed-source edition) running on my unRAID devel setup.  The errors suggested that SysV IPC support is needed.  Also, is there any particular reason you've chosen Vbox over other virtualization platforms?

 

 

Yes, you have to enable SYSVIPC.  Also add support for IDE CD ( Include IDE/ATAPI CDROM support ) so you can mount CDs in the VM.

 

I'm using Vbox 1.6, and installed via the VirtualBox-1.6.0-Linux_x86.run script.  I chose Vbox over VMware for the smaller memory footprint, and it is overall more straightforward.  It is not as tweakable, granted, but I think it fits this project well.

 

I've got 2GB on a swap partition, and still haven't gotten pushed very far into swap with the VM running.

 

The VM is using NAT, with port forwarding (in the VBox manual) to let Dameware connect to it.  Today, I'm going to try VNC and some other remote options built into KDE (i.e red carpet).

I configured KDE Desktop Sharing on the unRAID system to share the desktop, and then installed TightVNC on my XP system, and voilla... snappy remote control access to the unRAID/KDE desktop.

 

KDE Desktop Sharing does eat some CPU.... but I have been streaming a movie from the unRAID system the whole time, and no hickups.

 

I also shared /mnt on unRAID to the XP guest, and can read and write the unRAID disks via /mnt/disk*

 

Now this gets me back to a caution I have made many times before.  The stock distro of unRAID has stability in its core function (fault-tolerant media file server) as its number 1 goal.  Tom has good reasons for moving carefully with every change to the stock distro.  He has paying clients.  What I am doing is UNSUPPORTED by me or anyone else.  I'm happy to share my experience, but before anyone starts playing with this, they have to recognize that any one of these things could break some part of unRAID on some particular hardware.

 

Please, play with this stuff on a spare system, and not your production unRAID system.

 

Also, I only have one Pro license at this time, and it is on my main production server.  I am doing all of the testing with unRAID Basic, so I have no idea how this works with cache disk, or other features above basic.

 

Also, I don't use User Shares.... so they may not work without more work.  (See josetann's thread about some specific issues with Shares)

I don't want to jinx it, but I also have NFS running now, exporting the /mnt/diskX mount points.  Preliminary throughput tests on a 100mb network READING from unRAID showed about a 7% improvement in speed with NFS, less "spiky" transfers with NFS, and about 10% higher CPU utilization on the unRAID server during the transfer via NFS than via Samba.

Mobo is Asus K8N-E with NFORCE3-250 chipset.

 

When I boot the stock kernel, hard drives on the PATA IDE ports are UDMA 6 and all is good.  dmesg shows:

 

Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
NFORCE3-250: IDE controller (0x10de:0x00e5 rev 0xa2) at  PCI slot 0000:00:08.0
NFORCE3-250: not 100% native mode: will probe irqs later
NFORCE3-250: BIOS didn't set cable bits correctly. Enabling workaround.
NFORCE3-250: BIOS didn't set cable bits correctly. Enabling workaround.
NFORCE3-250: 0000:00:08.0 (rev a2) UDMA133 controller
    ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hda:DMA, hdb:DMA
    ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdc:DMA, hdd:DMA
Probing IDE interface ide0.

 

After changing the appropriate (or what I thought were appropriate) IDE drivers to built-in instead of modules to eliminate the initramfs, something's amiss.  Now when I boot my new kernel, the disks are in PIO mode.  Apparently something other than the NFORCE3-250 driver is snagging the ide0 and ide1 ports before the NFORCE3-250 driver:

 

Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Probing IDE interface ide0...
hdb: JLMS XJ-HD166S, ATAPI CD/DVD-ROM drive
hda: WDC WD2500JB-00EVA0, ATA DISK drive
...
...
...
NFORCE3-250: port 0x01f0 already claimed by ide0
NFORCE3-250: port 0x0170 already claimed by ide1

Anyone know how to make sure the NFORCE3-250 driver grabs the ports first?

Solved it...  I had unintentionally deselected "AMD and nVidia IDE support" under ATA/ATAPI/MFM/RLL support, but the same nforce driver was being loaded for the NIC later.

 

 

I don't want to jinx it, but I also have NFS running now, exporting the /mnt/diskX mount points.  Preliminary throughput tests on a 100mb network READING from unRAID showed about a 7% improvement in speed with NFS, less "spiky" transfers with NFS, and about 10% higher CPU utilization on the unRAID server during the transfer via NFS than via Samba.

 

Jumbo Frames enabled?

I'm not even sure all my switches will support MTU > 1500.  Two of my media players (Pinnaces) can't handle them.  Plus, the Marvel adapter on my test mobo doesn't do them, so it is a moot question for me.

 

My major problem now is a decent NFS client for Windows.  I have NFS running fine, and can copy with much better throughout from unRAID to another Linux box via NFS than I can from Linux to Windows via Samba.... but I need a decent NFS client for Windows to test NFS under Windows.... all I've tried so far were crap.

 

As far as additional apps, I've got pretty much everything I want going now on the unRAID box... VirtualBox, KDE, Apache, Open Show Center, PHP, perl, etc.  Having unRAID installed on a complete development system opens up unlimited possibilities, particularly with VirtualBox added to it.... you just have to be competent enough to configure and compile your own kernel.... just stay away from SMP -- as you know, it is not blessed yet for unRAID.

 

I have no doubt you COULD enable jumbo frames using this setup, but I have no hardware to do it with.

Hey there Bubba!  I was able to get things running thanks to your help and this guide.  I have some items that helped me with the process.

 

make modules install

 

This should be

 

make modules_install

 

Also, when compiling, Make sure the following are included as modules:

 

Under General Setup: System V IPC (If you want apache)

Under File Systems: Filesystem in Userspace support (User shares)

 

Not sure why these aren't included with the unraid .config

 

This HowTo and the one by Josetann were very helpful.  Thanks again!

Hi there I have a question for you Bubba.

 

If I want to add a new user to a full distro, this would cause issues because of shadow passwords, right?

 

What is the best way to add new users with a home directory, shell, etc.

 

Thanks!

 

 

Hi there I have a question for you Bubba.

 

If I want to add a new user to a full distro, this would cause issues because of shadow passwords, right?

 

What is the best way to add new users with a home directory, shell, etc.

 

Thanks!

 

 

 

Shouldn't cause any issues.  The root user is the only one that needs to have shadow passwords disabled (or at least have unRAID believe they're disabled).

BubbaQ, 

 

Any chance you would want to try to install a MythTV backend on your unraid / slackware box?  Here's a link on how someone else set it up on Slackware 12.1 http://www.mnspace.net/articles/mythtv-slack12.1.html  If I could get a Myth backend running on my unraid box that would be great.    I don't need that box to output HD or anything like that.  I have modified XBOX's running XBMC to act as my front end.

 

My next question is, how difficult would it be to "upgrade" my current Unraid box to run on the full Slackware distro?  I currently have 8 x 500GB HD's in there that are all around 50% full.  I could easily add another small IDE disk as the boot drive.  Is it possible to do it fairly easily or would it be better to start from scratch with aSlackware install and then add on Unraid and add in my discs?

 

If you can get MythTV running on the same box as Unraid I'm going to pick up a new cheap box and get started out on it myself.  I'd try it myself now but I'm fairly new to linux so I'm afraid I wouldn't know where to start.

 

Thanks,

 

J

JJ, I don't know anything about MythTV, but there are some Slackware packages at slacky.eu (http://www.slacky.eu/index.php?searchword=myth&option=com_search&Itemid=). 

 

It's probably not a plug-and-play solution, but maybe it'll save you some work.

As the original post stated, this procedure is for people "with average Linux experience and who is not afraid to build a kernel."  There are too many variables and too many choices that have to be made on an ad-hoc basis for this process to reduced to simple 1-2-3 steps.

 

As an analogy, suppose you are telling someone how to get to your house in a car... but their driving experience is so low, you have to tell them when to apply the gas, and how much, and when to apply the brakes, and when to turn the wheel, and how many degrees to turn it.  You have to have someone in the driver's seat with a basic driving skillset.

 

If you do install a full Slackware development system and add unRAID to it, there is practically NOTHING you can't install on top of it.  There may be compatibility issues (such as NFS and user shares) but that's life.

 

If you are new to Linux, I suggest you build a Slackware box, and play with it for a few months.  Once you have accidentally killed it 5 or 6 times, and can recover it instead of wiping it out and reinstalling, mastered tools like vi, and installed and configured applications like MySQL and Apache from the command line, then you'll be ready to follow this procedure.  The instructions that looked like Greek in the original post, you can then read and follow without blinking an eye.

 

 

 

I realize that it's not going to be plug and play and your first page of directions only scare me a little bit, but I understand 95% of it.  I was just looking for someone to say it is possible with a bit of work or MythTV uses XYZ which is completely incompatable with unraid/slackware.  But since you said it should all work eventually I'll give it a go.  I need to learn more about linux anyway 

 

Thanks,

 

J

I've added BubbaQ's post about Linux experience needed (as a link in a warning box) to the Building a custom kernel wiki page.

 

While there, I edited a few typos, and reformatted it for appearance and consistency.  I am requesting WeeboTech especially, but other experienced kernel builders too, to review my editing for accuracy.

 

Something that would be nice to add to it, would be more external links to further info for interested but inexperienced Linux developers.  For example, links to recommended articles on 'building your first kernel', 'customizing Slackware builds', 'guide to modifying .config', etc.  I imagine some of you are aware of good writeups on intro topics like these, as well as good reference materials you keep handy.  You can either add the links or tips to the wiki yourself, or suggest them here, and they will be added to the wiki page.