Docker disk image utilization is High, not exactly sure why

UnRAID 6.1.6 with a 240GB SSD cache and 3 4TB NAS HDD (1 parity). My docker.img is in /mnt/cache/ and its 100GB. I keep getting emails that my utilization is high. I am running Deluge, Plex, CouchPotato and Sonarr. CP went crazy on my and snatched 150+ movies. I paused all of them and deleted a bunch to get about 60 GB back but today I am back to 85%+. My mappings are below, I think I should /mnt/cache instead of /mnt/user but I still cant figure out why Docker usage is over 80GB. The 3 files Deluge has open are only about 5 GB.

/config /mnt/user/appdata/deluge/config

/data /mnt/user/appdata/deluge/data

/download /mnt/user/appdata/downloads/

You must have one or more of your dockers writing data to some folders that aren't mapped outside the docker. There are some other threads about this, but yours has got to be one of the worst. I've never heard of anyone giving 100GB to docker.img and still using it up. I have about a dozen dockers installed and only have 20GB docker.img with a steady 13GB used for many months.

See this thread for some ideas on what you need to do to fix your setup.

See this thread for a method for cleaning things up.

Map your /data and /download outside of your appdata folder on one of your non-cache shares

Yeah I'm realizing that on the downloads.  So use /mnt/user0/downloads. As I remember user0 bypasses the cache

Thanks trurl. I'll take a look

It's normal to not have downloads on your parity protected array as that spins up parity and reduces disk speed writes.  Rather download to cache then process and move to array afterwards. 

You can user either /mnt/user/downloads or /mnt/cache/downloads/ but either way you must set the downloads share to cache only within the webui

Wow,  that's amazing. I'm still sitting at a constant usage of 1.8gb out of 15gb after  6 months or more when I added pyTivo to the picture. Before that I've been at a constant 1.3gb  usage.

As others have said, you have something obviously configured wrong or using a very bad and misbehaving Docker container. Read those linked threads to figure out what is wrong and how to correct it.

There are some other threads about this, but yours has got to be one of the worst. I've never heard of anyone giving 100GB to docker.img and still using it up.

Lol, do I get an award?

It's normal to not have downloads on your parity protected array as that spins up parity and reduces disk speed writes.  Rather download to cache then process and move to array afterwards. 

 

You can user either /mnt/user/downloads or /mnt/cache/downloads/ but either way you must set the downloads share to cache only within the webui

I definitely had something mapped incorrectly. My files we getting downloaded in /var/lib/docker/somethingoranother/longhexstring.

Want to make sure i am understanding the cache function correctly. Basically its your scratch paper. This is where you want to write down your problem and do your math etc. The final answer is what you want written to the user share. For example i download a torrent with extra files i dont want (.nfo and .srt so on and so forth) and then rename it to the format I want and put it in the folder I want. The Mover runs at night and writes it the array.

By this logic I should map my appdata to /mnt/cache/appdata, this info should be relatively small but will have a lot of read writes since they are the working files  for docker. Since it is active data this share should be set to cache only.  I should create a downloads folder OUTSIDE of appdata, but still on the cache. This will be where the large media files are downloaded, keeping them outside of appdata to avoid docker.img utilization. Ideally there is some level of automation to rename and move these files before the mover is invoked. The idea being to avoid an additional unnecessary write to the array.  Finally, i map /mnt/user/media drive to the respective folder (tv, movies, music etc) for each docker app.

One concept I am not 100%  on is the spin down. I understand the concept of spinning down the disk to reduce usage, cycles, heat etc. I just dont understand A if UnRAID spins down automatically and B how it can do so without losing the ability to present all files and C the logic UnRAID uses to do this. Any clarification or corrections are appreciated, thanks guys.

One

also, it appears the once I get my docker.img file under control I need to reduce, but alas a docker image cannot be reduced. As such i would have to blow it away and start from scratch. Kinda poopy but not the end of world, Ill chalk it up to the learning curve. Just want to make sure I am understanding correctly before I do anything rash.

Thanks again

That’s not as bad as it sounds. You can disable Docker and the delete the old docker.img file and create a new img file.

Add your containers again by using User defined templates, these contains all your settings so there is no need to redo any configuration work.

This fixed the excessive usage in my case:

http://lime-technology.com/forum/index.php?topic=45249.0

The official Limetech Sync docker had a log file 7GB  in size after a day

That’s not as bad as it sounds. You can disable Docker and the delete the old docker.img file and create a new img file.

 

Add your containers again by using User defined templates, these contains all your settings so there is no need to redo any configuration work.

not bad at all! Didn't realize UnRAID saved your Docker templates automatically. Thanks