sabnzbd docker locking up tower?

hi all, running unraid 6 with only 3 dockers, plex, sickbeard, and sab

every few days, my sab docker cannot be accessed, but plex and sickbeard can be.  also, my shares cannot be accessed.  only thing that helps is a hard reboot, can't even shutdown from putty

i'm not sure if sab is the problem, or just a symptom

i'm attaching my diagnostics but it may not help since these were captured after a hard reboot.  any thoughts on how to troubleshoot?  i've had this issue ever since upgrading to unraid 6 about a month ago...

tower-diagnostics-20160106-1811.zip

This problem seems to be a long running problem and has been reported here https://lime-technology.com/forum/index.php?topic=39510.0

I also had this problem and and it directly seems to be connected with SAB.  My issue was that I was having a recurring issues with the GIU becoming unresponsive.  I tried a number of things - specifically CPU pinning http://lime-technology.com/forum/index.php?topic=36257.0 - which solved the issue for a time - however the hanging did start again (over longer periods).

I ended up setting a daily CRON job to restart my dockers.  This stopped my GUI hang problem. 

UNDER

Flash (folder) - created folder custom

flash\custom

created the following script  docker_restart.sh

#!/bin/bash

#Stop plugin services located in /etc/rc.d/
# enter in plugins to stop here, if any

#stop dockers
docker stop $(docker ps -a -q)

#Start plugin services
# enter in plugins to start here, if any

#start dockers
/etc/rc.d/rc.docker start

Unraid supports CRON jobs now.

I then created a CRON job under

flash\config\plugins\dockrestart (dockrestart was folder I created to store the file)

docker_restart.cron

# Daily Docker Restart - 5:10am
10 5 * * * /boot/custom/docker_restart.sh &> /dev/null

Obviously the code - /boot/custom/docker_restart.sh is referring to the location and script name where you placed and named the docker_restart.sh script.

You have to restart the server for the job to be loaded. 

Since doing the above my server has been back to good ole rocksolid.  I have not had a GUI hang since, however I dare say I have not fixed the underlying problem (it will still be there) - and on the basis of the defect thread - it is clear I do not possess the skill or technical ability to fix it. 

For your info the dockers I run are

Couchpotato

Delugevpn

Plex

Sabnzbd

Daapd

Sonarr

Ubooquity

So my daily restarting them is not an issue.  I do the restart when the server is run up about 15 minutes after mover has started. It has been working flawlessly.  The issue of my daily restarting the dockers is not issues - but it might be something you want to explore dependant on what dockers you are running - and the viability if they can be restarted daily.

Hope this helps. I am no guru by any means - if there are any errors in the above - can a more knowledgeable user point it out.

Credit here -

https://lime-technology.com/forum/index.php?topic=43431.msg414779#msg414779

http://www.nncron.ru/help/EN/working/cron-format.htm - use this to assist in determinng the time you want for the CRON job to start.  Explains the time format

wow thanks for the great reply, i will try this for sure!

two clarifications:

1.  for the docker_restart.sh, can i just paste the code into windows notepad, and then save the file as docker_restart.sh?

2.  for the cron job, do i make a .plg file or something? 

so excited try this!  surprised more people aren't having this problem...

 

1.  for the docker_restart.sh, can i just paste the code into windows notepad, and then save the file as docker_restart.sh?

 

2.  for the cron job, do i make a .plg file or something? 

 

Yes you can use notepad to create the files and name the file as required.

docker_restart.sh

#!/bin/bash

#Stop plugin services located in /etc/rc.d/
# enter in plugins to stop here, if any

#stop dockers
docker stop $(docker ps -a -q)

#Start plugin services
# enter in plugins to start here, if any

#start dockers
/etc/rc.d/rc.docker start

docker_restart.cron

# Daily Docker Restart - 5:10am
10 5 * * * /boot/custom/docker_restart.sh &> /dev/null

I missed the .cron on the original post.  Edited now.

I use a program called notepad++ - grab it here https://notepad-plus-plus.org/

 

Yes you can use notepad to create the files and name the file as required.

 

ok thanks all, i will be so happy if this fixes the unexpected lockup

i guess one final question- is there any way for me to tell that the docker has been restarted, ie to tell that i created the files correctly?

ok thanks all, i will be so happy if this fixes the unexpected lockup

 

i guess one final question- is there any way for me to tell that the docker has been restarted, ie to tell that i created the files correctly?

See the attachment.

oops sorry... i know where to go to see if they are running, i'm trying to ask- how do i know i did the script correctly?

if they are running before i go to bed, and running when i wake up, there are two options:

they restarted or

they never restarted, and just stayed on

wonder if there's a way to tell?

I believe his suggestion is to watch the uptime on the specific docker, to make sure it reboot overnight.

i see thanks, i did not notice the advanced view option, that's great

i did noticed though that it said up 16 hours which is not a good sign.  it is noon right now, and they should have restarted at 5am, which should then be up 7 hours.

i did have a reboot yesterday around 8pm, so that's how its counting to 16.  i must have something wrong

I will attach my two files

docker_restart.sh is in flash/custom

docker_restart.cron is in flash/config/plugins/dockrestart

Desktop.zip

Where both files created in notepad++?  Just checking that you wrote the files on the server *BEFORE* restart?

You are using the Docker Tab - Advanced View to see the up time.  You are correct that the uptime should always count back to it being restarted at 5:10am each morning.  From your settings it is clear that it didn't restart for some reason. I would suggest to check your folder names to make sure they are correct.

In saying that I have attached a zip file with my files to show you my exact setup. 

Hope that helps.

If you are wanting to test, without having to wait all day.  What I did was change the CRON file time to say 10 minutes ahead of current time.  Wrote the file.  Then restarted Unraid.  Waited past the time and checked to see if the files worked.

example:- Time is now 9:00am. 

changed the cron file time to 

# Daily Docker Restart - 9:10am
10 9 * * * /boot/custom/docker_restart.sh &> /dev/null

Understanding that

*    *    *  *    *        command to be executed

-    -    -  -    -

|    |    |  |    |

|    |    |  |    +----- day of week (0 - 6) (Sunday=0)

|    |    |  +------- month (1 - 12)

|    |    +--------- day of        month (1 - 31)

|    +----------- hour (0 - 23)

+------------- min (0 - 59)

Then log back in and check after that time on the Docker tab to see if it worked.  That might be a quicker way for you to test instead of waiting each day. 

If the job worked - change the CRON time to what ever you prefer. Restart your Unraid and check again the following day to make sure it has worked. If it doesn't work on the test come and report back.

Unraid_Docker_Restart.zip

ok great thanks.  i did create my files with notepad++, and did place the files in the locations, then restarted. 

i'm going to try using your files, including the folders you have.  i basically copied both unzipped folders right in the root of flash.  i edited your file to make the restart sooner as you suggested for testing.

rebooted, now i wait about half an hour to see.

it did occur to me- do i have to enable cron somewhere?  or just by having the .cron file in the right folder, unraid will 'use' it?

EDIT:  SUCCESS!!!  i really swear i did exactly as you instructed, but i must have goofed something.  using your files, with a slight change to time worked!  in fact i was doing a workout using plex, and expecting plex to 'disconnect' which it did right on cue.  then checked tower docker, and could see that it restarted the dockers 30 seconds prior.  Thanks so much time to help me, this should be stickied for others!!!

That's great news.  I am happy you were able to get it working.  Although you got this working - hopefully the daily restarting will stop SAB from locking up and taking down DOCKER, which in turn (for me anyway) - was taking down my system.

Please report back in a week or so and let us know if the lock-up has returned.  It is definitely something to do with SAB locking up DOCKER - then taking down UNRAID - something I thought was supposed to not be able to occur by using DOCKER.  The problem then compounded for me where I couldn't run DIAGNOSTICS (no access to GUI and Telnet command hung).

Anyway - daily restarting DOCKER via this method has stopped the 5 to 6 day lockup that I was having.  I hope it works for you as well.

sorry to report i had another hard lockup today.  this fix on this page likely works, i probably just have another problem.  seems like there are a ton of posts on random hard lockups...

just in terms of an update, i had a hard lockup on jan 15, but have not had one since ie 7 days.  its the longest i've had without a lockup, so very pleased.

i guess technically i did have two after installing this docker restart script, but seems to be working now

edit:  now feb 5, still have not had a hard lockup!

I was having this issue about 2 months ago that was taking down everything like you describe as well. For me it looked like the unpacking and rebuilding of the rar files was pegging the processor at 100% and then locking everything else up. I could not watch plex or anything. Since this box is my primary source of entertainment (I only have Plex and no other method of watching TV at home) I set up a schedule in SAB to only operate at night between 3 and 7am. I also changed the mover script, which was seeming to fight with SAB and get pissed off when there was no processor power for the mover to run its own thing, to only run once a week at a time when SAB will be paused. My server runs for weeks at a time without reboots now. Glorious...

I hope you are past the lockups now. I know what it's like to struggle with this. I am not saying what I did will help you, but I just wanted to share with someone who would listen!

I was having this issue about 2 months ago that was taking down everything like you describe as well. For me it looked like the unpacking and rebuilding of the rar files was pegging the processor at 100% and then locking everything else up. I could not watch plex or anything. Since this box is my primary source of entertainment (I only have Plex and no other method of watching TV at home) I set up a schedule in SAB to only operate at night between 3 and 7am. I also changed the mover script, which was seeming to fight with SAB and get pissed off when there was no processor power for the mover to run its own thing, to only run once a week at a time when SAB will be paused. My server runs for weeks at a time without reboots now. Glorious...

 

I hope you are past the lockups now. I know what it's like to struggle with this. I am not saying what I did will help you, but I just wanted to share with someone who would listen!

thanks very much for the suggestions... good to know there's other things to try, if my current 'fix' stops working.  i do not have a large cache drive, therefore i don't think i could afford to have it run only once per week....and plex is my only source for tv also, so i guess having sab run only once per day means you are at least another day delayed in watching shows, which is not the end of the world!

I was having this issue about 2 months ago that was taking down everything like you describe as well. For me it looked like the unpacking and rebuilding of the rar files was pegging the processor at 100% and then locking everything else up. I could not watch plex or anything. Since this box is my primary source of entertainment (I only have Plex and no other method of watching TV at home) I set up a schedule in SAB to only operate at night between 3 and 7am. I also changed the mover script, which was seeming to fight with SAB and get pissed off when there was no processor power for the mover to run its own thing, to only run once a week at a time when SAB will be paused. My server runs for weeks at a time without reboots now. Glorious...

 

I hope you are past the lockups now. I know what it's like to struggle with this. I am not saying what I did will help you, but I just wanted to share with someone who would listen!

 

thanks very much for the suggestions... good to know there's other things to try, if my current 'fix' stops working.  i do not have a large cache drive, therefore i don't think i could afford to have it run only once per week....and plex is my only source for tv also, so i guess having sab run only once per day means you are at least another day delayed in watching shows, which is not the end of the world!

Exactly! We found at my house that we normally didn't watch shows until the next day anyhow so my wife never even noticed I changed the download times to have it queue up the stuff during the day and evening hours, but not pull it until the middle of the night. We watch most things next day or over the weekend. She just knows that the server doesn't lockup anymore to where she has to yell at me to reboot it.... Again... Happy wife, happy life....

I was having this issue about 2 months ago that was taking down everything like you describe as well. For me it looked like the unpacking and rebuilding of the rar files was pegging the processor at 100% and then locking everything else up. I could not watch plex or anything. Since this box is my primary source of entertainment (I only have Plex and no other method of watching TV at home) I set up a schedule in SAB to only operate at night between 3 and 7am. I also changed the mover script, which was seeming to fight with SAB and get pissed off when there was no processor power for the mover to run its own thing, to only run once a week at a time when SAB will be paused. My server runs for weeks at a time without reboots now. Glorious...

 

I hope you are past the lockups now. I know what it's like to struggle with this. I am not saying what I did will help you, but I just wanted to share with someone who would listen!

 

thanks very much for the suggestions... good to know there's other things to try, if my current 'fix' stops working.  i do not have a large cache drive, therefore i don't think i could afford to have it run only once per week....and plex is my only source for tv also, so i guess having sab run only once per day means you are at least another day delayed in watching shows, which is not the end of the world!

Exactly! We found at my house that we normally didn't watch shows until the next day anyhow so my wife never even noticed I changed the download times to have it queue up the stuff during the day and evening hours, but not pull it until the middle of the night. We watch most things next day or over the weekend. She just knows that the server doesn't lockup anymore to where she has to yell at me to reboot it.... Again... Happy wife, happy life....

great server (and wife) management skills!