Too many files are currently in use?

Sometime when copying large group of (small) files from my workstation onto the server (even if the transfer is server to server), I get the following Windows error:

"Too many files are currently in use. Quit one or more programs, then try again". If I click on "try again" the copy goes through, but it's likely to give the same error with a different file a while later. This problem only seem to happen when I copy several small files.

The fact that this behavior may happen without anything substantial running in the (windows) background, makes me think that this is not a matter of a lack resources, as the error message implies.

I am running windows 7 Ult. x64. The unraid server is 4.5 final, but I remember seeing this error with some of the pre 4.5 betas. Event viewer does not report this problem. Has anyone encountered this, and is there anything I can look at on the server side?

You copy server to server through your workstation? Not using rsync directly on your unRAID box? Something like

# rsync -av --stats --progress fromDirectory toDirectory

...even if the transfer is server to server...

For server to server, it's easiest to telnet into your server and use the mc command.

I am running windows 7 Ult. x64.

Go shoot some microsoft programmers and you'll feel much better.

I saw this problem today copying from the unRAID drive to my desktop machine. It wasn't server to server and I'm on Win7 64Bit as well.

Must be something windows related as I am using 4.5 myself, straight from first avail and never had this issue. However, I am not using windows at all. Sorry, can't be of more help!

You copy server to server through your workstation? Not using rsync directly on your unRAID box? Something like

# rsync -av --stats --progress fromDirectory toDirectory

I am not familiar with linux, my bad. I will try to use rsync next time. BTW, why rsynch and not cp or mv? However, this problem happens copying from windows as well. Yes, I know, ditch Windows...  I realize that this is windows related, OTOH I get this error only when I am using the unraid server.

You copy server to server through your workstation? Not using rsync directly on your unRAID box? Something like

# rsync -av --stats --progress fromDirectory toDirectory

 

I am not familiar with linux, my bad. I will try to use rsync next time. BTW, why rsynch and not cp or mv? However, this problem happens copying from windows as well. Yes, I know, ditch Windows...  I realize that this is windows related, OTOH I get this error only when I am using the unraid server.

Same. I have this problem copying to/from the unRAID only. If I copy the same subdirectory to my Drobo unit I don't have this problem.

Is the Drobo connected via Network or directly via USB/Firewire?

Is the Drobo connected via Network or directly via USB/Firewire?

Droboshare (Ethernet) and uses Samba too.

[bug 462172] Re: samba "Too many files are currently in use."

Bug Watch Updater

Wed, 30 Dec 2009 17:41:00 -0800

** Changed in: samba

      Status: In Progress => Fix Released

[bug 462172] Re: samba "Too many files are currently in use."

Bug Watch Updater

Wed, 30 Dec 2009 17:41:00 -0800

 

** Changed in: samba

       Status: In Progress => Fix Released

 

Pardon me, but where do you go to look for these bugs and where do I go to get the fix? Thx!

[bug 462172] Re: samba "Too many files are currently in use."

Bug Watch Updater

Wed, 30 Dec 2009 17:41:00 -0800

 

** Changed in: samba

       Status: In Progress => Fix Released

 

 

Pardon me, but where do you go to look for these bugs and where do I go to get the fix? Thx!

You might try creating a file named

smb-extra.conf

and in it, put this

[global]

max open files = 20000

Put that smb-extra.conf file in the config folder on your flash drive.   If there is already a file there, add those two lines to it.

Then type

smbcontrol smbd reload-config

to have it read the new contents of the file you created.

Apparently, there is a limit to the number of open files in SAMBA, and files are not being closed in a given session, so it runs out.  Upping the limit might be a work-around.

Linux itself has its own limit, you can see it with

sysctl  fs.file-max

You can set it with

sysctl  fs.file-max=50000

Don't get too crazy... you can run out of other resources if too many files are kept open.  But the odds are you'll be able to get past the file copy you were attempting until the newer version of unRAID, with a newer version of SAMBA, can be deployed.

This bug is described here (with the work-around): https://bugs.launchpad.net/samba/+bug/462172

Joe L.

edit: according to the fix, the old limit was 10,000 open files.  Win7 needs 16,384 + a few extra (20 in the patch)

So.. setting the max open files to 16500 should handle it. 

Actual patch is here:

https://bugzilla.samba.org/attachment.cgi?id=5098&action=edit

My output is greater than your command of 50000, did you mean to suggest 500000? I didn't set that value, it was what was already there

root@unRAID:/boot/config# sysctl fs.file-max

fs.file-max = 346736

root@unRAID:/boot/config#

I'm still getting errors after putting in the smb-extra.conf file you suggested and the other commands.

I tried renaming the file to smb.conf since google suggested smbcontrol smbd reload-config would try to read the smb.conf file. I also bumped the max open files up to 50000 without any success.

abeta, you should have already had a file names /etc/samba/smb.conf. It is generated by unRAID upon boot. It should have some lines in it like the following, which specifies to use additional configuration file snippets.:

[global]
        # configurable identification
        include = /etc/samba/smb-names.conf

<< ... snip ... snip ... >>

        # hook for user-defined samba config
        include = /etc/samba/smb-extra.conf

        # auto-configured shares
        include = /etc/samba/smb-shares.conf

abeta, you should have already had a file names /etc/samba/smb.conf. It is generated by unRAID upon boot. It should have some lines in it like the following, which specifies to use additional configuration file snippets.:

 

[global]
        # configurable identification
        include = /etc/samba/smb-names.conf

<< ... snip ... snip ... >>

        # hook for user-defined samba config
        include = /etc/samba/smb-extra.conf

        # auto-configured shares
        include = /etc/samba/smb-shares.conf

Doh! My apologies. I'll put it back to smb-extra.conf as indicated but it still doesn't seem to like it for me.

You might try creating a file named

smb-extra.conf

and in it, put this

[global]

max open files = 20000

 

Put that smb-extra.conf file in the config folder on your flash drive.   If there is already a file there, add those two lines to it.

 

Then type

smbcontrol smbd reload-config

to have it read the new contents of the file you created.

 

Apparently, there is a limit to the number of open files in SAMBA, and files are not being closed in a given session, so it runs out.  Upping the limit might be a work-around.

 

Linux itself has its own limit, you can see it with

sysctl  fs.file-max

 

You can set it with

sysctl  fs.file-max=50000

Don't get too crazy... you can run out of other resources if too many files are kept open.  But the odds are you'll be able to get past the file copy you were attempting until the newer version of unRAID, with a newer version of SAMBA, can be deployed.

Thanks guys. I applied the fix Joe L described; hope that does it. This fs.file-max business reminds me of the FILES= command in DOS' config.sys... 

abeta, you should have already had a file names /etc/samba/smb.conf. It is generated by unRAID upon boot. It should have some lines in it like the following, which specifies to use additional configuration file snippets.:

 

[global]
        # configurable identification
        include = /etc/samba/smb-names.conf

<< ... snip ... snip ... >>

        # hook for user-defined samba config
        include = /etc/samba/smb-extra.conf

        # auto-configured shares
        include = /etc/samba/smb-shares.conf

 

Doh! My apologies. I'll put it back to smb-extra.conf as indicated but it still doesn't seem to like it for me.

also, the file name that unRAID "includes" from under /boot/config changed at some point.

If using 4.5 unRAID, the file is /boot/config/smb-extra.conf

After adding the lines to the file, you must either stop and re-start the array, OR issue the

smbcontrol smbd reload-config

command to have it re-read the new values.

To see if it was entered correctly, type

testparm -s | head -20

and it will print all the global parameters.

You should see the "max open files" parameter in the listing.

Joe L.

I used mcedit to edit the file.

It looks like it is correct with the max open files line but I'm still getting errors when I copy that subdirectory. See cut/paste:

Loaded services file OK.

Server role: ROLE_STANDALONE

[global]

        workgroup = MSHOME

        server string = Media server

        security = SHARE

        null passwords = Yes

        passdb backend = smbpasswd

        guest account = root

        syslog = 0

        syslog only = Yes

        unix extensions = No

        max open files = 20000

        load printers = No

        printcap name = /dev/null

        disable spoolss = Yes

        show add printer wizard = No

        local master = No

        force user = root

        create mask = 0711

        directory mask = 0711

        guest only = Yes

root@unRAID:/boot/config#

ETA: I bumped it up to 100000 to see if that mattered, I still get errors.

Just to report that the max files does not seem to help matters. The problem occurs with the same frequency as before.

Put that smb-extra.conf file in the config folder on your flash drive.   If there is already a file there, add those two lines to it.

 

Then type

smbcontrol smbd reload-config

to have it read the new contents of the file you created.

I've noticed that I don't really need to issue the `smbcontrol smbd reload-config` command.

Looks like unRAID's samba reloads its settings every time the smb-extra.conf changes.

I was having this same issue with my Win7 x64 + UnRAID 4.5

I created the smb-extra.conf file with the params. mentioned here, but the issue was still ocurring...

Then I executed this command:

root@Tower:~# ulimit -n 50000
root@Tower:~# ulimit -n
50000
root@Tower:~#

And now I am not having issues anymore... looks like it fixed the problem.

Another solution would be use rsync from inside the UnRAID, find bellow a example that I used before (assuming your copying to/from a Windows share already mounted inside your UnRAID under /mnt/winshare)

# rsync --numeric-ids --perms --owner --group -D --times --block-size=2048 --recursive --checksum-seed=32761 --progress -v --modify-window=1 /mnt/winshare/* /mnt/disk1/

I normally do my copies using Windows Explorer and then run this rsync command to check that everything was really transfered. Windows copy seems to be faster then rsync to direct copy my files... I don't know why.

Regards,

chopeta

Then I executed this command:

root@Tower:~# ulimit -n 50000
root@Tower:~# ulimit -n
50000
root@Tower:~#

 

And now I am not having issues anymore... looks like it fixed the problem.

Executing that command does not accomplish anything globally:

ulimit is used to set the resource usage limits of the shell.

Per shell.

To see what I am talking about, just after you've executed your command in your shell,

open another shell (another telnet session) and see what it says when you type ulimit -n

In the new session:

root@Tower:~# ulimit -n
1024

It doesn't have anything to do with samba.

I normally do my copies using Windows Explorer and then run this rsync command to check that everything was really transfered.

If that is the case, then do yourself a favor and get TeraCopy.

You can set it up to do CRC checks after it finishes all copying.

Not to mention that it does indeed copy stuff much faster than windows explorer.

And, it also has nice speed indicators.  

Executing that command does not accomplish anything globally:

ulimit is used to set the resource usage limits of the shell.

 

Per shell.

 

To see what I am talking about, just after you've executed your command in your shell,

open another shell (another telnet session) and see what it says when you type ulimit -n

 

In the new session:

root@Tower:~# ulimit -n
1024

 

It doesn't have anything to do with samba.

All right, sorry for possibly create confusion. I will test TeraCopy right now. Thanks for the tip!

Regards,

chopeta

I'm having the same issues... no fix yet...