unRAID Server Release 6.0-beta6-x86_64 Available


Recommended Posts

Download

 

Please see

readme.txt

in the release zip file for installation/upgrade instructions.

 

Disclaimer: This is beta software.  While every effort has been made to ensure no data loss, use at your own risk!

 

unRaid OS has undergone major development and there comes a time when you release what you got so far, and then carry on.  With this release we have three different means of "virtualization": Xen, KVM, and docker.  Of course you can run normally without using any virtualization features as well.  I was hoping to release after more webGui refinements, but docker is such a great feature that we felt it would be worth it to release what we have in place for it now.

 

Thanks to Jon Panozzo and Eric Schultz for their many long hours of development and testing.  Jon will modify this post in the next 24 hours with some links to docker guides.

 

Summary of changes from 6.0-beta5a to 6.0-beta6
-----------------------------------------------
- docker: added support for docker version 1.0.0
- emhttp: support btrfs-formatted cache disk
- linux: update to version 3.15.0
- linux: added more features:
  CONFIG_NR_CPUS: changed from 8 to 16 (number of CPU's supported)
  CONFIG_NUMA: Enable NUMA (Non Uniform Memory Access) support.
  CONFIG_SCSI_MVSAS_TASKLET: Support for interrupt tasklet (improves mvsas performance)
  CONFIG_ALX: Qualcomm Atheros AR816x/AR817x support
  CONFIG_I40E: Intel(R) Ethernet Controller XL710 Family support
  CONFIG_AMD_IOMMU: AMD IOMMU support
  CONFIG_INTEL_IOMMU: Support for Intel IOMMU using DMA Remapping Devices
  CONFIG_INTEL_IOMMU_DEFAULT_ON: Enable Intel DMA Remapping Devices by default
  CONFIG_IRQ_REMAP: Support for Interrupt Remapping
  CONFIG_VFIO: VFIO Non-Privileged userspace driver framework
  CONFIG_VFIO_PCI: VFIO support for PCI devices
  CONFIG_VFIO_PCI_VGA: VFIO PCI support for VGA devices
  CONFIG_CIFS_SMB2: SMB2 network file system support
  CONFIG_NET_9P: Plan 9 Resource Sharing Support (9P2000)
  CONFIG_NET_9P_VIRTIO: 9P Virtio Transport
  CONFIG_9P_FS: Plan 9 Resource Sharing Support (9P2000) 
  CONFIG_9P_FS_POSIX_ACL: 9P POSIX Access Control Lists
  CONFIG_MAGIC_SYSRQ
- slack: updated packages with security advisories:
  curl-7.36.0
  gnutls-3.1.25
  ntp-4.2.6p5
  openssh-6.6p1
  openssl-1.0.1h
  openssl-solibs-1.0.1h
  shadow-4.1.5.1
  libtasn1-3.6
  php-5.4.29
  samba-4.1.6
- slack: added packages:
  btrfs-progs
  libcgroup
  sqlite
- slack: minor fixes in rc.S, rc.M, rc.sshd
- qemu: version 2.0.0 with SeaBIOS 1.7.5
- xen: update to version 4.4.0, with security patches through 6-13-2014.
  Eliminated qemu-system-i386 altogether and now just builds qemu-system-x86-64 (qemu 2.0.0)
  Correct typo in /etc/default/xendomains.

 

BEGIN JONP EDITS

 

Boot Modes

In Beta 6, users will have two options for booting into unRAID, just like before:  Basic Mode or Xen Mode.  However, there are now more differences in these two modes:

 

Basic (Docker + libvirt)

Basic mode in beta 6 comes with the same features and capabilities that beta 5a had before it.  In addition, basic mode is what we call "KVM Ready."  This is similar to TVs that were sold back in the day with being "HD Ready."  Essentially this means that if you want VMs in this mode, you can use KVM to create them, but you don't have to.  And unlike Xen-mode, there is no "extra Linux kernel" required to load before unRAID Server OS.  The lack of this extra step makes this mode easier to support.  That said, KVM is only usable via command-line with libvirt (virsh) tool in beta 6.

 

Xen (Docker + Xen/xl)

Selecting the Xen Mode boot option will load the Xen hypervisor before booting into unRAID mode (see http://wiki.xenproject.org/wiki/Hypervisor).  If upgrading from a previous unRAID 6 Beta making use of the Xen hypervisor, your system should automatically boot into this mode.  Xen was the first hypervisor that Lime Tech implemented in the unRAID 6 beta and has undergone some upgrades for this release.  NOTE:  Booting into Xen mode also includes libvirt for VM management.  Existing Xen users can choose to use EITHER tool when booting into Xen mode to create / manage their VMs.  That said, it is not recommended to have a mix of VMs created by both tools running simultaneously (and this is not supported).  One or the other people ;-).

 

Docker:  Requirements and Setup

One of the biggest new improvements to unRAID 6 is the inclusion of Docker.  Docker allows us to safely use applications that were not built to run on unRAID Server OS.  These applications run in a secure, compartmentalized fashion with minimal consumption of hardware resources.  Docker is available in all three Boot Modes (Basic, KVM, and Xen) in unRAID Beta 6d.  Docker does not require any type of virtualization support on the hardware.  Docker performs well when installed on either array or cache drive devices (SATA or SSD).  Instructions for setup and configuration can be found in the Docker Quick-Start Guide in our Lime Tech Forums.

 

Docker for Plugin Authors

When writing an unRAID Plugin, authors have a painful process of development and testing they have to go through in order to both build and maintain the plugin properly.  We believe Docker offers a solution to many of the common problems with plugin design and management.  We believe that improving this platform is a key goal to our success in growing as a company and as a community.

 

Experiment with Docker in Beta 6d and let us know what you think in our Community Forums by imagining the possibilities and sharing your views!

 

unRAID 6 Beta 6 Supplemental Content

Throughout the course of this week, guides and other details will be posted by JonP on how to best utilize this beta and take advantage of everything baked into it.  As they are posted, this post will be updated here to add links directly to these guides.  In addition, forum changes will occur in the weeks ahead to better arrange/organize content.

Link to comment
  • Replies 336
  • Created
  • Last Reply

Top Posters In This Topic

Hi Tom,

 

guess you need to add a 5 at this

 

Upgrade Instructions

====================

 

If you are currently running 6.0-beta1/2/3/4

--------------------------------------------

 

It is only necessary to copy these files from the zip file to the root

of your USB Flash device:

 

  bzimage

  bzroot

  xen

  readme.txt

 

no need to replace syslinux folder i guess

as it is identical as the 5a release

but without having installed the new beta yet

 

how are we going to select docker for example ?

syslinux only chooses between these

 

default /syslinux/menu.c32

menu title Lime Technology

prompt 0

timeout 50

label unRAID OS

  menu default

  kernel /bzimage

  append initrd=/bzroot

label unRAID OS Safe Mode (no plugins)

  kernel /bzimage

  append initrd=/bzroot unraidsafemode

label Memtest86+

  kernel /memtest

label Xen/unRAID OS

  kernel /syslinux/mboot.c32

  append /xen --- /bzimage --- /bzroot

label Xen/unRAID OS Safe Mode (no plugins)

  kernel /syslinux/mboot.c32

  append /xen --- /bzimage --- /bzroot unraidsafemode

 

gues that is only unraid and xen ?

how are we going to choose kvm or docker ?

Link to comment

Do not see anywhere in the release notes that support for KVM has been added even though it was mentioned in this threads opening post!

 

I guess the thing to do is to get this beta up and running to see what it actually provides!  Just downloaded it so off to try that :)

Link to comment
gues that is only unraid and xen ?

how are we going to choose kvm or docker ?

 

My guess is that kvm and docker are both permanently available in the image, or are loaded at run time.  Enabling either/both must be via the 'Extensions' page in the web GUI.

Link to comment

One strange thing I have immediately noticed is that if I boot into Xen mode I can no longer log in to the console.  The last message displayed on the console is "loading libvirtd".  Telnet and webGUI appear to be OK so the system is successfully loaded.  It is almost as if GPU pass-thru has been activated despite the fact I have not set up any specific settings for this. Anyone else seeing this or have any idea of something I should check?

Link to comment

My first 'buglet' report on beta 6! 

 

If you try and start the docker service and the location which docker is to use (defaults to /mnt/cache/docker) is not in btrfs format then an error message is flashed up and then disappears before you really have time to work out what it says.  I only managed to find out what the problem was by looking in the syslog.

 

I think that ideally two things should happen:

  • The error message should persist longer
  • The option to start the service should be disabled if the target location is not in btrfs format if this is a requirement (ideally with a message in the GUI stating this requirement).

Off now to look at clearing current contents of my cache drive so that I can reformat it (or convert it) to btrfs format

Link to comment

My first 'buglet' report on beta 6! 

 

If you try and start the docker service and the location which docker is to use (defaults to /mnt/cache/docker) is not in btrfs format then an error message is flashed up and then disappears before you really have time to work out what it says.  I only managed to find out what the problem was by looking in the syslog.

 

I think that ideally two things should happen:

  • The error message should persist longer
  • The option to start the service should be disabled if the target location is not in btrfs format if this is a requirement (ideally with a message in the GUI stating this requirement).

Off now to look at clearing current contents of my cache drive so that I can reformat it (or convert it) to btrfs format

 

LOL there goes docker...

i have no space in my box for 2 cache drives in btrfs

 

Link to comment

My first 'buglet' report on beta 6! 

 

If you try and start the docker service and the location which docker is to use (defaults to /mnt/cache/docker) is not in btrfs format then an error message is flashed up and then disappears before you really have time to work out what it says.  I only managed to find out what the problem was by looking in the syslog.

 

I think that ideally two things should happen:

  • The error message should persist longer
  • The option to start the service should be disabled if the target location is not in btrfs format if this is a requirement (ideally with a message in the GUI stating this requirement).

Off now to look at clearing current contents of my cache drive so that I can reformat it (or convert it) to btrfs format

 

LOL there goes docker...

i have no space in my box for 2 cache drives in btrfs

As I understand it, it is quite possible to have single drive in btrfs format.  I guess I will be finding out :)  Having said that if I use 2.5" or SSD drives then I could squeeze in two drives into spare space in the box.  I have the SATA connections available, just no space for another 3.5" drive.
Link to comment

My first 'buglet' report on beta 6! 

 

If you try and start the docker service and the location which docker is to use (defaults to /mnt/cache/docker) is not in btrfs format then an error message is flashed up and then disappears before you really have time to work out what it says.  I only managed to find out what the problem was by looking in the syslog.

 

I think that ideally two things should happen:

  • The error message should persist longer
  • The option to start the service should be disabled if the target location is not in btrfs format if this is a requirement (ideally with a message in the GUI stating this requirement).

Off now to look at clearing current contents of my cache drive so that I can reformat it (or convert it) to btrfs format

Would be interesting to know why the docker location has to be on a btfs formatted drive?

 

It's been a while since I had a (brief) play with docker but I'm pretty sure I ran it in an Ubuntu VM on a disk formatted as ext4.

Link to comment

Would be interesting to know why the docker location has to be on a btfs formatted drive?

 

It's been a while since I had a (brief) play with docker but I'm pretty sure I ran it in an Ubuntu VM on a disk formatted as ext4.

I suspect it is to do with the ability for multiple docker containers (or VM's) to only store differences to their base.    It would have a big effect on reducing storage space required.

 

I could see docker being the basis of providing some of the new monitoring/recovery features that are planned without the risk of de-stabilizing the core system. It looks like it would be more robust than the current plugins that live in core unRAID and would also make it easier to provide updates outside major unRAID releases. However that is just a 'guess' rather than any real knowledge of what is planned. 

 

However if docker is going to demand a btrfs device that might become an issue?  I guess resolving this sort of issue is the purpose of a beta program.

Link to comment

Seems like a lot of jumping to conclusions.  It is just as likely a bug since I strongly doubt Docker itself has that requirement.  A quick read of the Docker install says nothing about needing a specific FS for docker (http://docs.docker.com/installation/ubuntulinux/).  I'm not ruling out some unraid-ism, but really, we're already seeing posts in other threads stating this ("docker requires btrfs") as near fact  ???

 

EDIT: or it could be incompatible with RFS, but again ... waiting for official answers is probably best.

Link to comment

Seems like a lot of jumping to conclusions.

I am often good at that :)

  It is just as likely a bug since I strongly doubt Docker itself has that requirement.  A quick read of the Docker install says nothing about needing a specific FS for docker (http://docs.docker.com/installation/ubuntulinux/).

I am sure that is correct.

  I'm not ruling out some unraid-ism, but really, we're already seeing posts in other threads stating this ("docker requires btrfs") as near fact  ???

It appears to be a requirement in beta 6. 

 

EDIT: or it could be incompatible with RFS, but again ... waiting for official answers is probably best.

Why take all the fun out of it :)
Link to comment

My Ubuntu and Arch KVM machine is running under B6 :-)

Needed to remove this line from my xml file

  <input type='keyboard' bus='ps2'/>

But that shall not be necessary for a functional VM.

 

I though I should find a KVM Manager like Xen Manager :-)

 

Continue play with this .....

 

GREAT JOB on B6 LT team!!

Link to comment

Mover still isn't working for me.

 

I have a user share called "download", which is set to use Cache Disk: Only

 

When I start mover manually (because it's not working, and my cache drive is 1/2 full again), I see this in the syslog...

 

Jun 18 06:35:40 media logger: mover started

Jun 18 06:35:40 media logger: skipping JRiver/

Jun 18 06:35:40 media logger: skipping VM/

Jun 18 06:35:40 media logger: moving download/

 

It sees the first 2 shares as cache only, but not the download share.  Not sure why that is.

 

With that said, it tries to move files from this directory to someplace, and I get the following lines in the syslog, indicating I'm out of space, but that's not correct either.  Each drive in my array has a mimimum of 100GB free space right now...

 

Jun 18 06:35:41 media logger: ./download/incomplete/House.of.Cards.S02E02.1080p.BluRay.x264-DEiMOS/__ADMIN__/SABnzbd_nzf_AA7E0I
Jun 18 06:35:41 media shfs/user0: shfs_mkdir: assign_disk: download/incomplete (28) No space left on device
Jun 18 06:35:41 media logger: rsync: recv_generator: mkdir "/mnt/user0/download/incomplete" failed: No space left on device (28)
Jun 18 06:35:41 media logger: *** Skipping any contents from this failed directory ***
Jun 18 06:35:41 media logger: .d..t...... download/
Jun 18 06:35:41 media logger: cd+++++++++ download/incomplete/
Jun 18 06:35:41 media logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.0]
Jun 18 06:35:41 media logger: ./download/incomplete/House.of.Cards.S02E02.1080p.BluRay.x264-DEiMOS/__ADMIN__/SABnzbd_nzf_D88b0C
Jun 18 06:35:41 media shfs/user0: shfs_mkdir: assign_disk: download/incomplete (28) No space left on device
Jun 18 06:35:41 media logger: rsync: recv_generator: mkdir "/mnt/user0/download/incomplete" failed: No space left on device (28)
Jun 18 06:35:41 media logger: *** Skipping any contents from this failed directory ***

 

I have a folder under another share called download, but it's located at /mnt/user/video/download

 

I can't see why this subfolder having the same name as a share should cause an issue, but I don't know what is causing the problems with mover, so this is just a stab in the dark.

 

Full syslog attached.  Also attached are my share.cfg and the download.cfg for the download share

 

I really want to get this resolved, so I can clean up my cache drive, and format it to btrfs and try out Docker.

 

** I just noticed that my download share shows an odd amount of free space.  The other cache only shares show the amount of free space on the cache drive, and the regular shares show the amount of free space on the whole array, but the download share shows something in the middle of these 2 values.  Weird.

 

User Shares
Name         	Free
JRiver 	 	269 GB
VM 	        269 GB
backup 	 	5.36 TB
documents 	5.36 TB
download 	3.23 TB
music 	 	5.36 TB
photos 	 	5.36 TB
video 	 	5.63 TB

 

In looking further, I see there is a compute... option for the size column, and when I click it, I see that the download share points to 3 discs for some reason...

 

Name         	Size         	Free
download    	33.7 GB 	3.23 TB
Cache: 		33.7 GB 	269 GB
Disk 10: 	0 B            	1.79 TB
Disk 5: 	0 B            	1.17 TB

 

Why is it wanting to use these other 2 discs, and why is mover telling me there is no room when I have over 1TB free on each drive?

syslog.zip

share.cfg

download.cfg

Link to comment
Why is it wanting to use these other 2 discs, and why is mover telling me there is no room when I have over 1TB free on each drive?

 

Just a stab in the dark ... are you sure that there are no /mnt/disk5/download and /mnt/disk10/download folders?

 

269GB + 1.79TB + 1.17TB = 3.23TB

 

 

At a guess, it's using the existence of download folders on array disks to make the decision that download is not a cache-only share.  It then attempts to move files from cache drive to array drives and finds that it is not permitted to use the array drives - hence 'no space ...'.

Link to comment

Mover still isn't working for me.

 

I have a user share called "download", which is set to use Cache Disk: Only

 

When I start mover manually (because it's not working, and my cache drive is 1/2 full again), I see this in the syslog...

 

Jun 18 06:35:40 media logger: mover started

Jun 18 06:35:40 media logger: skipping JRiver/

Jun 18 06:35:40 media logger: skipping VM/

Jun 18 06:35:40 media logger: moving download/

 

It sees the first 2 shares as cache only, but not the download share.  Not sure why that is.

 

With that said, it tries to move files from this directory to someplace, and I get the following lines in the syslog, indicating I'm out of space, but that's not correct either.  Each drive in my array has a mimimum of 100GB free space right now...

 

Jun 18 06:35:41 media logger: ./download/incomplete/House.of.Cards.S02E02.1080p.BluRay.x264-DEiMOS/__ADMIN__/SABnzbd_nzf_AA7E0I
Jun 18 06:35:41 media shfs/user0: shfs_mkdir: assign_disk: download/incomplete (28) No space left on device
Jun 18 06:35:41 media logger: rsync: recv_generator: mkdir "/mnt/user0/download/incomplete" failed: No space left on device (28)
Jun 18 06:35:41 media logger: *** Skipping any contents from this failed directory ***
Jun 18 06:35:41 media logger: .d..t...... download/
Jun 18 06:35:41 media logger: cd+++++++++ download/incomplete/
Jun 18 06:35:41 media logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.0]
Jun 18 06:35:41 media logger: ./download/incomplete/House.of.Cards.S02E02.1080p.BluRay.x264-DEiMOS/__ADMIN__/SABnzbd_nzf_D88b0C
Jun 18 06:35:41 media shfs/user0: shfs_mkdir: assign_disk: download/incomplete (28) No space left on device
Jun 18 06:35:41 media logger: rsync: recv_generator: mkdir "/mnt/user0/download/incomplete" failed: No space left on device (28)
Jun 18 06:35:41 media logger: *** Skipping any contents from this failed directory ***

 

I have a folder under another share called download, but it's located at /mnt/user/video/download

 

I can't see why this subfolder having the same name as a share should cause an issue, but I don't know what is causing the problems with mover, so this is just a stab in the dark.

 

Full syslog attached.  Also attached are my share.cfg and the download.cfg for the download share

 

I really want to get this resolved, so I can clean up my cache drive, and format it to btrfs and try out Docker.

 

** I just noticed that my download share shows an odd amount of free space.  The other cache only shares show the amount of free space on the cache drive, and the regular shares show the amount of free space on the whole array, but the download share shows something in the middle of these 2 values.  Weird.

 

User Shares
Name         	Free
JRiver 	 	269 GB
VM 	        269 GB
backup 	 	5.36 TB
documents 	5.36 TB
download 	3.23 TB
music 	 	5.36 TB
photos 	 	5.36 TB
video 	 	5.63 TB

 

In looking further, I see there is a compute... option for the size column, and when I click it, I see that the download share points to 3 discs for some reason...

 

Name         	Size         	Free
download    	33.7 GB 	3.23 TB
Cache: 		33.7 GB 	269 GB
Disk 10: 	0 B            	1.79 TB
Disk 5: 	0 B            	1.17 TB

 

Why is it wanting to use these other 2 discs, and why is mover telling me there is no room when I have over 1TB free on each drive?

 

 

If I had to guess, the download folder at some point made it onto an array disk.  While it is set to cache only now since the download folder does exist on the array it likely is "parsing it" into the folders that need to be moved.

 

Inspect all your disk shares and see if you have a folder called download hanging out on any of them.

Link to comment

... are you sure that there are no /mnt/disk5/download and /mnt/disk10/download folders?

 

If I had to guess, the download folder at some point made it onto an array disk.  While it is set to cache only now since the download folder does exist on the array it likely is "parsing it" into the folders that need to be moved.

 

Inspect all your disk shares and see if you have a folder called download hanging out on any of them.

 

Bing, bing, bing!!!  both correct.

 

I have now eliminated this folder on both drives.  Will report back later.

 

I would like to suggest that this may still be a 'bug' and that changing a share to cache only would move these folders off of any array drives, thus preventing this in the future, and not require me to be as smart as you 2 :)

 

Thanks to you both for your help.

Link to comment

Careful with that suggestion.  Doing what you ask means unraid is responsible for knowing you want to move data from the array into the cache.  I'm not saying that is impossible, but some things are better left to the user to do manually.

 

Personal Opinion: unraid just shouldn't let you change a share to be cache only. It should block that option once the share is created.  Instead you would have to make a new cache-only share, move the data, then delete the old array share.

 

And that could be documented in the webgui "help" when it won't let you change from an array share to a cache-only share.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.