unRAID Server release 4.5-beta11 available

The release notes for beta-8/9/11 state:

- Upgrade pkgtools in order to support slackware packages which have .txz extension.

 

To use that capability, the "xz" compression utility needs to be included as well.  Currently, in beta-11, it is not.   I'm guessing this is an oversight.

 

Joe L.

Well package xz-4.999.8beta-i486-1 is included in beta11, though I didn't test it on the Flash (tested on hard drive based development system).  I'll look into it.

Just added my cache disk last night thanks to the new feature in Plus and it seems to be working pretty well.  Except now that my cache drive has caught up I seem to have a few strange and erroneous messages in my syslog:

 

Nov 22 22:03:58 NAS2 shfs: cache disk full

Nov 22 22:08:09 NAS2 shfs: cache disk full

Nov 22 22:09:11 NAS2 shfs: cache disk full

Nov 22 22:10:10 NAS2 emhttp: shcmd (42): /usr/sbin/hdparm -y /dev/sdg >/dev/null

Nov 22 22:13:37 NAS2 shfs: cache disk full

Nov 22 22:16:04 NAS2 shfs: cache disk full

Nov 22 22:17:15 NAS2 shfs: cache disk full

 

This is when my cache disk is completely empty (well 33meg of 120G empty)  I am doing moves on a user share between two folders when this message is displayed.

 

Rob

OK, I think I've figured out what the problem is.  I filled up the cache drive but it never reset the full flag when mover ran.  I'm finding that no writes to my array go through the cache drive and I continue to get the cache disk full message.

Can anyone else confirm this?  (an advantage of a small cache disk)

Rob

Restarting the array did not clear the full flag.  Writes are still going directly to the array.

Stopping, removing, re-adding and restarting didn't seem to clear the full flag either.

I even tried starting the array without the cache disk and then re-adding the cache disk but no luck.  I'm still getting cache disk full messages even though unmenu shows it as empty.

Restarting the array did not clear the full flag.  Writes are still going directly to the array.

 

Stopping, removing, re-adding and restarting didn't seem to clear the full flag either.

 

I even tried starting the array without the cache disk and then re-adding the cache disk but no luck.  I'm still getting cache disk full messages even though unmenu shows it as empty.

On the 'Shares' page under 'Cache' is a setting called 'Min. free space', which defaults to 2000000.  This is the minimum amount of free space, specified in 1024-byte blocks,  that must exist on the cache disk in order to create a new object on the cache disk.  So a value 2000000 means 2GB.  Did you change this setting?

I might add that this default value is too low if you are creating large ISO files that can be bigger than 2GB.  If during the transfer of a single large file, the cache disk becomes full, the server will terminate the transfer with "Out of Space" error.

Back to your issue... if you look at the cache disk share itself are there any files on the cache disk?

That could very well be the problem.  I did change it but without notification that it is blocks or bytes (or k or m) makes it hard to know what value to set.

I'm very sure that it's WAY too high - so for humans, we should know that it's in K bytes.

I'll check that and it will likely solve my issue.

I can confirm that spinning up drives will interupt read, even when i watch a simple dvd bit rate max 12 Mbit/s the dvd get's interupted on my popcorn hour c200.

I have 4 gig mem in the unraid computer so i was rather suprised. Hope this can be solved as this makes the server rather unusable.

Well I can recreate the stall, my disks :-

parity device:	pci-0000:00:14.1-ide-1:0 (hdc) ata-SAMSUNG_HD154UI_S1Y6J1KS516318
disk1 device:	pci-0000:00:14.1-ide-0:0 (hda) ata-WDC_WD1200JB-00EVA0_WD-WMAEK2829749
disk2 device:	pci-0000:00:14.1-ide-0:1 (hdb) ata-WDC_WD800BB-60CJA1_WD-WMAAC1145603
disk3 device:	pci-0000:00:11.0-scsi-0:0:0:0 (sda) ata-SAMSUNG_HD154UI_S1Y6J1KS616605
disk4 device:	pci-0000:00:11.0-scsi-1:0:0:0 (sdb) ata-SAMSUNG_HD154UI_S1Y6J1KS516537
disk5 device:	pci-0000:00:11.0-scsi-2:0:0:0 (sdc) ata-SAMSUNG_HD154UI_S1Y6J1KS516320
disk6 device:	pci-0000:00:11.0-scsi-3:0:0:0 (sdd) ata-SAMSUNG_HD154UI_S1Y6J1KS516326
disk7 device:	pci-0000:03:06.0-scsi-0:0:0:0 (sde) ata-WDC_WD10EACS-00ZJB0_WD-WCASJ1828278
disk8 device:	pci-0000:03:06.0-scsi-1:0:0:0 (sdf) ata-WDC_WD10EACS-00ZJB0_WD-WCASJ1775088
disk9 device:	pci-0000:03:06.0-scsi-2:0:0:0 (sdg) ata-WDC_WD10EACS-00ZJB0_WD-WCASJ1728696
disk10 device:	pci-0000:03:06.0-scsi-3:0:0:0 (sdh) ata-WDC_WD10EACS-00ZJB0_WD-WCASJ1804275
cache device:	pci-0000:00:14.1-ide-1:1 (hdd) ata-Maxtor_6V300F0_V6030LKG

Doesn't seem to be any rhyme or reason to it, happens whenever a disk spins up it seems. Testing was from Ubuntu 9.10 and Windows 7 using samba, accessing user shares.

Just tested using disk shares on Snow Leopard and its still the same,  Playing video from Disk 4, spin up disk 5 and it paused.  Spin up disk 7 and it paused.

I also have the spin up problem

 

parity device: pci-0000:00:1f.2-scsi-0:0:0:0 (sda) ata-SAMSUNG_HD103UJ_S13PJ1EQ504937

disk1 device: pci-0000:00:1f.2-scsi-1:0:0:0 (sdb) ata-SAMSUNG_HD103UJ_S13PJ1EQ504936

disk2 device: pci-0000:00:1f.2-scsi-2:0:0:0 (sdc) ata-SAMSUNG_HD103UJ_S13PJ1BQA07919

disk3 device: pci-0000:00:1f.2-scsi-4:0:0:0 (sde) ata-SAMSUNG_HD103UJ_S13PJ1EQ504935

disk4 device: pci-0000:00:1f.2-scsi-5:0:0:0 (sdf) ata-SAMSUNG_HD103UJ_S13PJ90Q909841

disk5 device: pci-0000:00:1f.2-scsi-3:0:0:0 (sdd) ata-WDC_WD10EADS-00M2B0_WD-WCAV51184422

 

 

From what Tom said earlier you should not be having this problem, if I understood him correctly.

 

Please provide a little more details for us and Tom.  Your Motherboard, amount of RAM, version of unRAID (I assume you are running the current beta since this is the beta thread), and anything else you can think of.

 

Sorry for the late reply.

 

I'm running 4.5 beta 11 and I know that from what Tom wrote I shouldnt have the problem but I have.

 

I've foiund 2 scenarios where it happens everytime I try.

 

1: When all drives are spin down besides the one I  play any media music/movie from and I delete or start a copy that have to spin up data and parity drive .

 

2: When playing a HD movie from 1 disk and I start another HD movie from another disk.

 

 

Hardware is Asus board Limetec used to use, 1GB ram and Celeron 430.

 

I think the way I outlined "how to tell" if multiple drives are on the same controller is not reliable.  Please post the output of this command:

 

v /sys/bloock

total 0

lrwxrwxrwx 1 root root 0 Nov 25 18:07 fd0 -> ../devices/platform/floppy.0/block/fd0/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 loop0 -> ../devices/virtual/block/loop0/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 loop1 -> ../devices/virtual/block/loop1/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 loop2 -> ../devices/virtual/block/loop2/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 loop3 -> ../devices/virtual/block/loop3/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 loop4 -> ../devices/virtual/block/loop4/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 loop5 -> ../devices/virtual/block/loop5/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 loop6 -> ../devices/virtual/block/loop6/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 loop7 -> ../devices/virtual/block/loop7/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 md1 -> ../devices/virtual/block/md1/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 md2 -> ../devices/virtual/block/md2/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 md3 -> ../devices/virtual/block/md3/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 md4 -> ../devices/virtual/block/md4/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 md5 -> ../devices/virtual/block/md5/

lrwxrwxrwx 1 root root 0 Nov 25 16:43 sda -> ../devices/pci0000:00/0000:00:1f.2/host1/target1:0:0/1:0:0:0/block/sda/

lrwxrwxrwx 1 root root 0 Nov 25 16:44 sdb -> ../devices/pci0000:00/0000:00:1f.2/host2/target2:0:0/2:0:0:0/block/sdb/

lrwxrwxrwx 1 root root 0 Nov 25 16:43 sdc -> ../devices/pci0000:00/0000:00:1f.2/host3/target3:0:0/3:0:0:0/block/sdc/

lrwxrwxrwx 1 root root 0 Nov 25 16:44 sdd -> ../devices/pci0000:00/0000:00:1f.2/host4/target4:0:0/4:0:0:0/block/sdd/

lrwxrwxrwx 1 root root 0 Nov 25 16:44 sde -> ../devices/pci0000:00/0000:00:1f.2/host5/target5:0:0/5:0:0:0/block/sde/

lrwxrwxrwx 1 root root 0 Nov 25 16:44 sdf -> ../devices/pci0000:00/0000:00:1f.2/host6/target6:0:0/6:0:0:0/block/sdf/

lrwxrwxrwx 1 root root 0 Nov 25 18:07 sdg -> ../devices/pci0000:00/0000:00:1d.7/usb2/2-3/2-3:1.0/host0/target0:0:0/0:0:0:0/block/sdg/

I don't remember the spin/pause problem existing in the past, if this is true then what changed to cause it?

Doesn't seem to be any rhyme or reason to it,  happens whenever a disk spins up it seems.  Testing was from Ubuntu 9.10 and Windows 7 using samba, accessing user shares. 

 

Just tested using disk shares on Snow Leopard and its still the same,  Playing video from Disk 4, spin up disk 5 and it paused.  Spin up disk 7 and it paused.

Just to be clear, this is the behaviour I reported. Access any disk using share over Samba and I get a pause. I have 3 data drives and have single disk user shares on each of these individual drives. Accessing a user share on either of the other 2 data drives while playing back a file from the 3rd data drive will cause a pause.

Spinning up and down the other 2 data drives or the parity drive using unMENU does not cause studder even if I spin-up/spin down as fast as possible using the web interface.

Peter

Doesn't seem to be any rhyme or reason to it,  happens whenever a disk spins up it seems.  Testing was from Ubuntu 9.10 and Windows 7 using samba, accessing user shares. 

 

Just tested using disk shares on Snow Leopard and its still the same,  Playing video from Disk 4, spin up disk 5 and it paused.  Spin up disk 7 and it paused.

 

Just to be clear, this is the behaviour I reported. Access any disk using share over Samba and I get a pause. I have 3 data drives and have single disk user shares on each of these individual drives. Accessing a user share on either of the other 2 data drives while playing back a file from the 3rd data drive will cause a pause.

 

Spinning up and down the other 2 data drives or the parity drive using unMENU does not cause studder even if I spin-up/spin down as fast as possible using the web interface.

 

Quick 2nd test, and yes, same deal here, seems to be samba freezing when a disk spins up, doing a manual spin up from the web page doesn't do anything.

I want to try and clear up some confusion regarding stutter/freeze of media data being read off the unRAID server.  First let's define some terms:

"stutter" refers to a momentary (less than 1 second) 'glitch' in the displayed video/audio (or just audio if media stream is audio-only).  Also included would be a series of "stutters".

"freeze" refers to the complete stop of video/audio playback for many seconds, typically coincident with another disk in the server array getting spun for some other access (either to read directories, copy files, playback other media, etc).

Ok... stutter is something I really want to eliminate in-so-far as it's being caused by the unRAID server.  Actually as of current release, there are no known causes of stutter generated by the unRAID server, except for one possibility: the hard drive itself takes a long time for error recovery.

Other sources of stutter include, most obviously, a glitch in the display client.  For example, I can make a video stream stutter on a Vista HTPC by doing certain operations which seem to take a lot of resources (like a virus scan).  This can be fixed by increasing the video playback process priority.  Other sources of stutter are network problems, eg., bad network cabling, switches, congestion, etc.

So if you think there is a case of unRAID causing 'stutter', I'd like to know how you come to that conclusion, bearing in mind that I can only fix what I can reproduce.  My own HTPC setup is rock sold (unless Vista goes nutts doing something in the background).

As for freezing... This is always caused by spinning up a disk other than the one you are currently reading media from.  I know of two causes of this:

1. Attempt to access any disk/user share on the same Windows PC that is currently also playing the media on the already-spun-up disk.  This is a problem with Windows & I have not tried to find a fix for it yet.  I think this happens in Mac OS as well.

2. Attempt to access a disk which happens to be on the same hardware host controller as the already-spun-up-disk.  This is a nasty problem and I will be posting -beta12 very soon which fixes this by spinning up both (or all) disks that are on the same host controller on-demand.

I think many reports of 'freezing' are of case 1 above.  If you think you have found a third case of unRAID-caused 'freeze', I'd like to know how you came to that conclusion, bearing in mind that I can only fix what I can reproduce.

So if you think there is a case of unRAID causing 'stutter', I'd like to know how you come to that conclusion, bearing in mind that I can only fix what I can reproduce.  My own HTPC setup is rock sold (unless Vista goes nutts doing something in the background).

I've actually never managed to make my HTPC freeze or stutter,  plex on mac mini 20mb read ahead cache.  (OS X 10.6)

Confirmed - reservation larger than cache disk size.

I was only off by a factor of 1024.  (I tried to reserve 4G, but because I didn't know the units, ended up reserving 4T I guess)

Thanks!

Is there any known problems with Samba?

I've noticed that changing mode on a file by adding executable bit to other (chmod o+x file) makes the file disappear from the browse list.

How to reproduce:

- Open a folder listing in Windows explorer

- login to unraid server and change directory to that folder

- touch a file (it will appear in the explorer view)

- chmod o+x on the file - the file disappears.

Same problem if you try to use smbclient, so it is not specific to Windows explorer.

Looks like a bug...

It is not a bug. It depends on how you configured SAMBA to map additional non-unix attributes, like the WinOS HIDDEN flag.

Look into the SAMBA options of "map hidden", "map archive", and "map system" within the smb.conf man pages.

As for freezing... This is always caused by spinning up a disk other than the one you are currently reading media from.  I know of two causes of this:

1. Attempt to access any disk/user share on the same Windows PC that is currently also playing the media on the already-spun-up disk.  This is a problem with Windows & I have not tried to find a fix for it yet.  I think this happens in Mac OS as well.

This is what happens to me but its not happening on any PC.

My popcorn hour freezes when I force sleeping drives to spin up via user share access.

It is not a bug. It depends on how you configured SAMBA to map additional non-unix attributes, like the WinOS HIDDEN flag.

 

Look into the SAMBA options of "map hidden", "map archive", and "map system" within the smb.conf man pages.

 

Thanks for the explanation.

In any case, this should not be set by default. Simply because it causes unexpected behavior.

It is not a bug. It depends on how you configured SAMBA to map additional non-unix attributes, like the WinOS HIDDEN flag.

 

Look into the SAMBA options of "map hidden", "map archive", and "map system" within the smb.conf man pages.

 

 

Thanks for the explanation.

In any case, this should not be set by default. Simply because it causes unexpected behavior.

LOL - If it was not "set by default" then not being able to set a file 'hidden' or have it's archive bit set via Windows would be "unexpected behavior". 

My popcorn hour freezes when I force sleeping drives to spin up via user share access.

Since a user share typically spans several drives, is it possible one of the drives of the user share shares a host controller with the drive being read by the popcorn hour?

Thanks for the explanation.

In any case, this should not be set by default. Simply because it causes unexpected behavior.

LOL - If it was not "set by default" then not being able to set a file 'hidden' or have it's archive bit set via Windows would be "unexpected behavior". 

I am glad I made you laugh...

None of *nix based distributions I've ever touched has the mapping in default config.

What happens now, you rsync files to unraid system from elsewhere with the bit set (rsync preserves permissions, remember?), than you try to look at those files from a windows machine and you don't see anything. And you still consider it to be expected behavior...

My popcorn hour freezes when I force sleeping drives to spin up via user share access.

 

Since a user share typically spans several drives, is it possible one of the drives of the user share shares a host controller with the drive being read by the popcorn hour?

very possible..I posted my  v /sys/block here

http://lime-technology.com/forum/index.php?topic=4674.msg43347#msg43347

If its sharing I can't tell ..sata configured in achi mode.

 

I am glad I made you laugh...

None of *nix based distributions I've ever touched has the mapping in default config.

What happens now, you rsync files to unraid system from elsewhere with the bit set (rsync preserves permissions, remember?), than you try to look at those files from a windows machine and you don't see anything. And you still consider it to be expected behavior...

Here is the hack I applied for myself.

In case somebody needs to do the same.

This will disable mapping to hidden files.

SCRIPT_DIR=/boot/custom/bin

[ -d "$SCRIPT_DIR" ] || mkdir -p $SCRIPT_DIR

SCRIPT=$SCRIPT_DIR/fix_samba_config.sh

cat >$SCRIPT <<EOF

sed -i -e 's/hidden\(.*\)[Yy]es\(.*\)/hidden\1no\2/' /etc/rc.d/rc.samba; /etc/rc.d/rc.samba restart

EOF

chmod +x $SCRIPT

echo "$SCRIPT" >>/boot/config/go

$SCRIPT

Well let me clarify, my doughter watched a dvd this is a 8mbit stream and at the same time i took out my laptop and wanted to clean up some stuff on my server. With every copy action i did to different disks there was a short stutter, but after activating 3 shares by copying things to different shares the c200 gave up and frooze completely.

On my normal windows nas i never experienced these problems i did let windows put the non used drives to sleep, and i never ever experienced a drop or stutter in a sd stream or hd stream when a drive got spun up ,especialy since sd is only 8mbit/s.

So apperently as the machine has 4gig of memory the whole unraid server must be shortly unresponsive as a disk is spun up, in my case all the spin up drives where on the same controller. Also as i experienced the stutter 4 times it seems to happen after every single spin up.

i do not know what httpc setup u have for your own, but if it's a windows based one u indeed probably will not see a stutter as the huge cache these machine's use will need a huge break in a data stream before u will notice it.

In my case when a drive will spin up the server will be unresponsive for about 2-3 seconds in that time i can do nothing to the server in windows or on my popcorn and 2-3 seconds is to much for the small cache my popcorn hour c200 has or for that matter any hardware based media player.

Maybe it's already explained but why does the unraid server become completely unresponisve after every drive spin-up ?

I am glad I made you laugh...

None of *nix based distributions I've ever touched has the mapping in default config.

What happens now, you rsync files to unraid system from elsewhere with the bit set (rsync preserves permissions, remember?), than you try to look at those files from a windows machine and you don't see anything. And you still consider it to be expected behavior...

My point is that some people (actually you are the first one) will complain about this setting causing "unexpected behavior", so if I change it another whole group of people will complain about a different "unexpected behavior"... so matter how it's set, someone will complain 

The reason the "default" is the way it is, is because unRAID is designed to work as smoothly as possible in a Windows Networking (SMB) environment.  Other protocols: NFS, rsync, AFP, etc., though important, at this point are secondary.