musicmann Posted December 26, 2008 Share Posted December 26, 2008 We all agree that unRAID is a great product, and a number of people have made their unRAID servers even better by running additional software on theirs. There are a few ways to add additional functionality to the stock unRAID: Precompiled Slackware Packages/Scripts - A number of great add-on packages and scripts already exist for Slackware (the base for unRAID), and many of them are plug-and-play with unRAID. Occasionally, you might have to "redirect" a config file or two, but a lot of additional functionality can be added this way. Other scripts have been specifically developed for unRAID (such as Joe L.'s unMENU) and can typically be found in the forums or in the wiki. Custom unRAID distro - BubbaQ's bubbaRAID works with unRAID and provides some of the features most requested here in the forums. Install unRAID onto hard drive - Probably the most limitless option of adding additional functionality to unRAID is to migrate unRAID to a full Slackware hard-drive install. This way, you can include a ton of things that just aren't practical on a flash-drive install. Although these are all great options, sometimes, they still aren't enough. There may not be a precompiled Slackware package, and running unRAID from a hard drive might not be practical for your setup. This was the situation for me when I wanted to get VMware Server running on my unRAID. In these cases, you may be able to compile a package yourself that will be able to run on your normal unRAID install. For this, you need a development system that mirrors unRAID but that has all the additional software development resources needed so that you can 1) install the desired software on the development system and 2) package that installation in a way that it can be run on the unRAID system. These are the steps that I went through to build a development environment that mirrors unRAID Server 4.4 which is based on Slackware 12.1 and kernel updates to 2.6.27.7. You development environment can be a physical or a virtual machine. I built mine as a virtual machine (VMware Server 2.0 on a Windows XP host). The steps I followed are basically copied from josetann's page at http://www.thetechguide.com/howto/unraid-on-hard-drive.html with only a few modifications and updates. Hopefully, he won't get too mad at me reprinting them here. --------------- These steps are specifically for unRAID Server using the 2.6.27.7 kernel. At the time of this posting, that includes 4.4 and 4.5beta1. The steps can easily be updated for future kernel versions though more care will need to be taken with Step #6 as the attached config file may not work with future kernel versions. 1. Install Slackware 12.1. I simply accepted all the installation defaults. If you want a slimmer development system, you can be more selective in what's included in the install. Josetann gives more detail as to what is needed and what is extra. Since this was a development (not a production) environment, and because I was building it as a vm, I didn't care too much about a little extra bloat. Additionally, I really like have X on here because it just makes somethings a lot easier. 2. Get the updated kernel source, headers, and modules: cd /usr/src wget http://mirrors.easynews.com/linux/slackware/slackware-current/slackware/k/kernel-source-2.6.27.7_smp-noarch-1.tgz wget http://mirrors.easynews.com/linux/slackware/slackware-current/slackware/d/kernel-headers-2.6.27.7_smp-x86-1.tgz wget http://mirros.easynews.com/linux/slackware/slackware-current/slackware/a/kernel-modules-2.6.27.7-i486-1.tgz 3. Get the unRAID source. I'll put the code for downloading it directly, but if you have already downloaded it, save Tomm some bandwidth and try copying it over to your /usr/src directory some other way. wget http://download.lime-technology.com/unRAID%20Server%204.4.zip 4. Install the kernel packages: installpkg kernel-source* installpkg kernel-headers* installpkg kernel-modules* 5. Extract the unRAID source and copy to appropriate location: mkdir unraid mv unRAID* unraid cd unraid unzip unRAID* mkdir bz-mod cd bz-mod zcat ../bzroot | cpio -i -d -H newc --no-absolute-filenames cd /usr/src/unraid/bz-mod/usr/src/linux cp -rf * /usr/src/linux/ cp -rf .config /usr/src/linux/ 6. Prepare the kernel config. cd /usr/src/linux make clean make menuconfig In the last step of #5, we had copied the stock unRAID kernel config to /usr/src/linux. Step #6 is to update this config to work under our development system settings. At the very least, this will probably mean changing the modules for the filesystems, IDE/ATA, and SATA drivers to be compiled directly into the kernel instead of being modules loaded as needed. This means changing the "M" to "*" under menuconfig. When building my development system, I determined that I needed to make a number of changes to the Device Drivers areas of ATA/ATAPI, Serial ATA, and Graphics to get my system to boot (i.e., no kernel panics) and to get X working. What I actually did was compare the settings in my stock Slackware 12.1 install to see what the settings needed to be in the upgraded (2.6.27.7) menuconfig. This is one of the great things about using virtual machines because I was able to have the stock and the upgraded running side-by-side to make changes and test in real time. I also made some changes to my networking drivers, but I think my router was just being flakey at the time and that I probably didn't have to make any changes here. I've attached the .config file that works for my setup. It may work without modification for yours. You will need to rename it from config.txt to .config and put it in the /usr/src/linux directory prior to doing make clean here. Hopefully, you will not need to make any changes under make menuconfig, but once your config is finished, proceed to Step #7. Note, this may turn out to be an iterative process. If your new kernel fails to boot, you'll probably need to come back to #6 to make changes. 7. Compile the kernel: make bzImage cp arch/x86/boot/bzImage /boot/vmlinuz-2.6.27.7-unRAID cp System.map /boot/System.map-2.6.27.7-unRAID make modules make modules_install 8. Update LILO to include the new kernel as an option. The LILO config file is located at /etc/lilo.conf. It is specific to your install, and may look slightly different for different people. Find a set of lines similar to this: # Linux bootable partition config begins image = /boot/vmlinuz root = /dev/hde3 label = Linux read-only # Linux bootable partition config ends The string for root is likely to be different. Add another entry like the one below, below this current entry. Make sure to keep your "root" the same as it was before. (Note, the only changes are on the "image" line and the "label" line. image = /boot/vmlinuz-2.6.27.7-unRAID root = /dev/hde3 label = unRAID read-only Once you have edited and saved lilo.conf, you'll need to update lilo by running: lilo -v Your new kernel should now be added as a boot option. BTW, I also change a couple of other things in my lilo.conf file. I change my timeout value to 150. This means a 15 second delay before booting the default kernel instead of the standard 2 minutes (120 seconds). Also, once I made sure my unRAID kernel booted properly (and that X worked), I reversed the order of my kernel entries, to make unRAID the first entry and therefore the default kernel. That way, if I fire up my development environment and get distracted, it boots into the unRAID kernel not the Slackware 12.1 kernel. If all went well, you should have a booting unRAID kernel and a functional devlopment environment. This was the extent of my development setup for me, but josetann goes on to give directions for getting unRAID (i.e., the array and parity and all) to run under this setup. His goal was running unRAID on a hard drive not just building a development environment. Please visit his site if you need that additional functionality. Edited Jan 25 2009 to update attached config file Quote Link to comment
pjneder Posted January 6, 2009 Share Posted January 6, 2009 musicmann, This is some good stuff. A bunch of steps usually found in different places consolidated into one easy to follow post. I had to do this the hard way, I wished I had had your post a few weeks back. Having said that, I think this can be expanded some. Maybe I can help, maybe you would like to do this, or maybe you want to still leave it as an exercise for the readers. I don't want to step on your toes here. What I think is missing are the steps that take you from this environment, to the bz* images you can put on a flash and then boot up unraid with your added packages. However, maybe I got this wrong. If you are trying to create an environment solely for the purpose of created pkg files that you can install from your "go" script then this is a great guide. Cheers, Paul Quote Link to comment
musicmann Posted January 7, 2009 Author Share Posted January 7, 2009 Paul, I'm a bit lazy. I was just making an environment on which I could build packages that could be installed via the go script, so once I got what I needed, I stopped there. Feel free to add anything that you see fit to this thread. I think there still may be significant interest in home-rolled unRAID distros, and it may help to have it all located in one place. If you'd like me to add your steps to the original post, that can be done too. Quote Link to comment
pjneder Posted January 12, 2009 Share Posted January 12, 2009 I will work on them and send you a PM. You can vet them for sense and completeness. Then, if you like, you can update your original post. Quote Link to comment
musicmann Posted January 26, 2009 Author Share Posted January 26, 2009 I've updated the kernel config file in the original post. This was to address an issue that I had building a VMware Server package in which VMware Server worked but bridged networking for virtual machines didn't (NAT worked). In the original config, I had piled in a lot of network device drivers thinking that, that was being "safe." This version is much more like the stock unRAID wrt network device drivers. I used a physical instead of a virtual development environment this time, but I tried to add support to the virtual NIC that VMware uses, so you should be able to make virtual development environments with networking support using this config file. Also, Slackware 12.2 uses the 2.6.27.7 kernel (used by unRAID 4.4.2) by default, so you can avoid a few steps by installing it versus installing 12.1 and then upgrading to the 2.6.27.7 kernel. Quote Link to comment
smino Posted February 25, 2009 Share Posted February 25, 2009 Any chance of making an Unraid module for slax, that you could add to : http://www.slax.org/build.php Think of the publicity...and how much easier it would be. Quote Link to comment
smino Posted February 26, 2009 Share Posted February 26, 2009 If you want to add raid Drivers, where in this process would you do it? Where in the process would you create your say raid 5 or 10 drive for the Cache Drive? Any chance unraid could also be used as a htpc? Quote Link to comment
Joe L. Posted February 26, 2009 Share Posted February 26, 2009 If you want to add raid Drivers, where in this process would you do it? Where in the process would you create your say raid 5 or 10 drive for the Cache Drive? Any chance unraid could also be used as a htpc? The unRAID "md" device takes the place of the usual "md" (raid5) device. As currently written, you cannot have both in the same kernel. It would need a complete re-write. Joe L. Quote Link to comment
smino Posted March 3, 2009 Share Posted March 3, 2009 Joel, If you were to do Hardware Raid 5 for the cache drive, then do you still need the MD command? Quote Link to comment
Joe L. Posted March 3, 2009 Share Posted March 3, 2009 Joel, If you were to do Hardware Raid 5 for the cache drive, then do you still need the MD command? I have no experience with hardware raid5 on linux. Sorry. From what I understand, there is limited support for some cards. unRAID needs the "md" device to be its own. Joe L. Quote Link to comment
Guest MrKenSan3 Posted March 11, 2009 Share Posted March 11, 2009 Hi All, New here, but have a simple question, and this seems like the right gang to ask... I've not yet dug into the details of implementing musicmann's EXCELLENT tutorial, but I'd like to ask if *all* of the sources are available for the unRAID kernel mods/modules required. I ask because I'd like to recompile and run this on a NAS box with a non-intel processor. Goal would be to support up to 4ish drives on a commercial NAS box (thinking the D-Link DNS series) as I like the form-factor and size of such a solution rather than hacking a machine together from bits... (which I have plenty of already ;-) ) A nod from those in the know would encourage me to dig in and give this a whirl... Of course with the goal of sharing the fruits with the community! Any thoughts, or pointers to other attempts at something like this would be gratefully appreciated! MrKenSan Quote Link to comment
Joe L. Posted March 11, 2009 Share Posted March 11, 2009 Hi All, New here, but have a simple question, and this seems like the right gang to ask... I've not yet dug into the details of implementing musicmann's EXCELLENT tutorial, but I'd like to ask if *all* of the sources are available for the unRAID kernel mods/modules required. I ask because I'd like to recompile and run this on a NAS box with a non-intel processor. Goal would be to support up to 4ish drives on a commercial NAS box (thinking the D-Link DNS series) as I like the form-factor and size of such a solution rather than hacking a machine together from bits... (which I have plenty of already ;-) ) A nod from those in the know would encourage me to dig in and give this a whirl... Of course with the goal of sharing the fruits with the community! Any thoughts, or pointers to other attempts at something like this would be gratefully appreciated! MrKenSan The source for the "md" kernel module is GPL and available, as is the config file. It is written to overlay on top of a GPL Linux distribution of Slackware source. Many have built their own kernel. The source code for the shfs "User-Share" file-system is NOT available, and is not GPL. The same for the "emhttp" management utility that performs all configuration and management of the array. They represent lime-technologies product. Unless those execuitables are runable on the DLINK platform, you will not be able to run either. Many of the smaller form-factor platforms do not include a memory management module... I have no idea about your target, but a quick check shows it uses an ARM processor, so I'm fairly certain you will not get too far with your effort before hitting a snag. A port to a different platform will be a challenge unless Slackware is already ported. Joe L. Quote Link to comment
Guest MrKenSan3 Posted March 12, 2009 Share Posted March 12, 2009 Thanks Joe L. For the very complete response! Turning my attention to a tree with x86 and Slackware-compatible bark! Cheers, MrKenSan Quote Link to comment
eeking Posted March 19, 2009 Share Posted March 19, 2009 The source code for the shfs "User-Share" file-system is NOT available, and is not GPL. The same for the "emhttp" management utility that performs all configuration and management of the array. They represent lime-technologies product. Unless those execuitables are runable on the DLINK platform, you will not be able to run either. Many of the smaller form-factor platforms do not include a memory management module... I have no idea about your target, but a quick check shows it uses an ARM processor, so I'm fairly certain you will not get too far with your effort before hitting a snag. A port to a different platform will be a challenge unless Slackware is already ported.I'll admit I don't really know what's going on under the hood, but if I understand it correctly, the md module does all the work of keeping parity while emhttp does the work of configuring and controlling md. So in theory one could write their own management utility in order to run unRaid on another platform. Still challenging, but nowhere approaching impossible. And while the shfs is great, you could get by with just SMB or NFS disk shares. </arm-chair coder> Quote Link to comment
aaronjb Posted April 19, 2009 Share Posted April 19, 2009 Just getting going with unRAID here, and following this tutorial prompts me to suggest some alterations The paths for the 2.6.27.7 files need changing as 'slackware-current' now points to 2.6.29.1; wget http://mirrors.easynews.com/linux/slackware/slackware-12.2/slackware/k/kernel-source-2.6.27.7_smp-noarch-1.tgz wget http://mirrors.easynews.com/linux/slackware/slackware-12.2/slackware/d/kernel-headers-2.6.27.7_smp-x86-1.tgz wget http://mirrors.easynews.com/linux/slackware/slackware-12.2/slackware/a/kernel-modules-2.6.27.7-i486-1.tgz Also, for those of us setting up a development environment under VMWare, there is an excellent list of what kernel options are necessary here: http://en.gentoo-wiki.com/wiki/HOWTO_Install_Gentoo_on_VMware#Kernel_options It's for Gentoo, and a couple of kernel options have moved (but are still easily locatable based on the information there), but those options should work very well for any VMWare dev environment (mine, for example, is running under VMWare Fusion) Aside from that the instructions were spot on - so thanks very much for writing it all up! Quote Link to comment
SuperPerry Posted April 30, 2009 Share Posted April 30, 2009 Hi, I am able to boot with my new kernel but I cannot connect with the internet. after typing "startx" KDE loads but Firefox is unable to connect to the internet. ifconfig command showing LOOPBACK. I am getting this error message during "make bzImage" on step #7 make[1]: warning: Clock skew detected. Your build may be incomplete. make: warning: Clock skew detected. Your build may be incomplete. Where should I start? FYI: I am using Slackware 12.2 and I skip steps #2 and #4 Thanks! Quote Link to comment
musicmann Posted April 30, 2009 Author Share Posted April 30, 2009 I assume you've tried restarting networking and/or have restarted the machine itself. After that, my first check would be to make sure that the approprate network driver was compiled into the kernel. If you are using a virtual environment under a VMware product, check to make sure the pcnet32 driver has been included in the kernel. If you are using a physical development environment (or if you're using a different virtualization platform...or even a VMware product and pcnet32 is already in there), find out what networking hardware or "hardware" is being used, and make sure that driver is included in the kernel. Quote Link to comment
SuperPerry Posted April 30, 2009 Share Posted April 30, 2009 Thanks! Dumb me, I actually unchecked my driver... from "M" to "nothing"... instead of "*". Its all good now.. VM next... Quote Link to comment
dgirard Posted April 30, 2009 Share Posted April 30, 2009 Hello! First, thanks to all who've worked hard to get the Development Environments up and running! I'm having difficulty getting the kernel modules compiled. I've basically followed the directions, with the following differences: 1. Using 12.2 as the base slackware install 2. Using unraid 4.5 Beta 4 (the latest available right now) 3. Using the 2.6.29.2 kernel sources (the 'current' slackware sources)...I saw somewhere that Beta5 is being prepared with 2.6.29.1, but I couldn't locate that...and I've not been able to locate any info on what Beta 4 is built on...just that it's whatever was current on April 14th... I tried the .config file shipped with beta4, and the .config file previously listed in this thread...and also tried config'ing my own... The Kernel compiles fine, but when I make modules it failes while compiling md I get an error "in function read file error: 'struct tast_struct' has no member named 'fsuid' (a bunch of times) and the build fails. Am I just getting too agressive with my kernel sources? or is there something else to tweak?...should I just wait for Beta 5? I should also mention that I'm doing this on Slackware running on VMWare esxi3.5 (not that it should matter yet...) My goal is to get VMware working on top of unraid (I know, it won't run while running on esxi, but I don't have a dev box to play with this yet). Any easy fixes?... Thanks! David Quote Link to comment
Joe L. Posted April 30, 2009 Share Posted April 30, 2009 Hello! First, thanks to all who've worked hard to get the Development Environments up and running! I'm having difficulty getting the kernel modules compiled. I've basically followed the directions, with the following differences: 1. Using 12.2 as the base slackware install 2. Using unraid 4.5 Beta 4 (the latest available right now) 3. Using the 2.6.29.2 kernel sources (the 'current' slackware sources)...I saw somewhere that Beta5 is being prepared with 2.6.29.1, but I couldn't locate that...and I've not been able to locate any info on what Beta 4 is built on...just that it's whatever was current on April 14th... I tried the .config file shipped with beta4, and the .config file previously listed in this thread...and also tried config'ing my own... The Kernel compiles fine, but when I make modules it failes while compiling md I get an error "in function read file error: 'struct tast_struct' has no member named 'fsuid' (a bunch of times) and the build fails. Am I just getting too agressive with my kernel sources? or is there something else to tweak?...should I just wait for Beta 5? I should also mention that I'm doing this on Slackware running on VMWare esxi3.5 (not that it should matter yet...) My goal is to get VMware working on top of unraid (I know, it won't run while running on esxi, but I don't have a dev box to play with this yet). Any easy fixes?... Thanks! David Your "current kernel sources" probably have a typo. Or, you introduced an error in your editing of the "md" files. I'll bet "tast_struct" should be "task_struct" (expecially since task_struct is a data structure that does have a member named fsuid) Joe L. Quote Link to comment
dgirard Posted April 30, 2009 Share Posted April 30, 2009 Thanks Joe. You're right about the 'k' ....I mistyped into my post...it is 'task_struct' in the actual error message... I'll go re-grab the kernel sources and try again...I don't think I edited anything (other than the config file, and that--at least one of the times--was using make menuconfig...but perhaps I did something wrong...I certainly did not edit any of the md files... I just wanted to make sure I wasn't beating my head against an impossible configuration...I was worried that the sources for md might not work with this kernel source... Thanks again -David Quote Link to comment
Joe L. Posted April 30, 2009 Share Posted April 30, 2009 Thanks Joe. You're right about the 'k' ....I mistyped into my post...it is 'task_struct' in the actual error message... I'll go re-grab the kernel sources and try again...I don't think I edited anything (other than the config file, and that--at least one of the times--was using make menuconfig...but perhaps I did something wrong...I certainly did not edit any of the md files... I just wanted to make sure I wasn't beating my head against an impossible configuration...I was worried that the sources for md might not work with this kernel source... Thanks again -David I've not looked... but do you have the kernel header files? They would have the structure definitions. (I don't have a development env, so I can't tell you if the structure was renamed, or fsuid was renamed... but somehow, I doubt it) Quote Link to comment
dgirard Posted May 1, 2009 Share Posted May 1, 2009 Yup...got the kernel header files...I agree that it seems like something is missing here... I tried dl'ing the packages from thetechguide.com (kernel 2.6.24.4) and they build just fine (and they have the unraid md code in them)...but even taking the ./md folder from them and porting it into the 2.6.29.2 structure doesn't build (same errors)... I'm thinking there must be some other header file I'm missing...but I have no idea how to find it... BTW, the kernel does compile with the 2.6.29.2 stock md, but as soon as I copy over the unraid md source the build dies... Quote Link to comment
musicmann Posted May 1, 2009 Author Share Posted May 1, 2009 To be on the safe side, have your development enviroment at the exact same kernel version as the target unRAID you're building for. I'm not sure if this is vital, but it will remove another variable from the equation. I haven't played around with 4.5b4, so I don't know the exact version but it may be in the release notes (looks like it might be 2.6.28.4). If that doesn't work, just boot the 4.5b4 flash and you should see the kernel version either during the boot process or when you log in. Quote Link to comment
dgirard Posted May 1, 2009 Share Posted May 1, 2009 Thanks musicmann (for both the initial instructions and this note!) Silly question though...where can I get depreciated kernel sources for slackware?...it looks like they freeze the kernel level with a dotted release, and the "slackware-current" folders only contain the most recent...I can't seem to locate any in-between releases... This is my first time compiling kernels under slackware in about 12 years (yikes!---)...of course we used to have to compile them all the time ;-} -- Funny, it doesn't seem to have changed much (except don't have to "make dep" any more) I'm guessing that Beta5 will be out any time now (I think I saw a note a few weeks ago saying it was due out in a few days...), so perhaps I'll just wait for that...then hopefully I'll be able to get "aligned" sources.... Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.