Default Docker Volume Mapping Preferences Coming in unRAID 6.2


jonp

Recommended Posts

Anybody know the answer to this

...I have also read on the forum that /mnt/user does not work with hard links. Is this true and could this be an issue when specifying /mnt/user/appdata?

since /mnt/user/appdata was part of the proposal?

 

i've seen issues that have been resolved in containers by using /mnt/cache/appdata in place of /mnt/user/appdata

 

To bump up this thread and provide my $0.02

 

I concur with sparkly's statement. Many of my containers do not work if the appdata is on a user share. In other words, /mnt/user/appdata does not work, but /mnt/cache/appdata works and /mnt/diskX/appdata works as well (likely due to links, but did not have a chance to confirm).

 

If that's the case, we need bug reports opened and logs so we can resolve.

 

Another thing I wanted to highlight is that mapping /mnt or /mnt/user by default is not a good idea.

 

Why?  Users will have the option to NOT have this happen with a toggle under the Docker Settings page (a checkbox to auto-map all user shares to all new containers).  It will be on by default, but can be toggled off.

 

Why not leave that decision up to the container authors and make it optional? That way if a container needs access to the array (for instance plex definitely does) then the author can put that option in.

 

Authors shouldn't be determining where data goes on a user's system.  Instead, the user should have that choice.  That's how installing applications on ANY OS would work for fairly logical reasons.

 

There are numerous issues with letting an author specify the host path mapping for containers.  I've seen examples where authors path to /mnt/somethingrandom instead of a common path that is on all systems.  If a user added a container and just accepted that as a default value, then whatever would be written to that path would literally be going to RAM.  Nothing would persist and memory would slowly get chewed up until eventually the system would start to have issues because of low memory.

 

To go a step further, authors don't all use the same container path name for common paths.  E.g. /data vs. /appdata or /media vs. /movies.  So now the user has to remember, on an app by app basis for all their containers, what volume mappings tie to what hostmappings.  If we have a universal rule that /unraid maps to /mnt/user or /mnt for ALL containers, then when users are leveraging ANY application in a container and need to path to some share on the host, it would be the same method (just browse to /unraid).

 

For instance my duckdns container has absolutely no need for /mnt mapping. In fact I could get away with not even having an appdata folder as all the necessary info is in the template xml (it needs the duckdns subdomain and the api key, which are passed as environment variables so it can run a cron script every 5 minutes to update the dynamic dns address).

 

But what does it hurt for it to be there?  I get that some folks have OCD when it comes to data access, but keep in mind that for years we have had plugins, which have unrestricted access to the entire system (all data paths both on persistent storage, the flash device, and the root ram filesystem).  No one seemed to mind.  So now we have docker where we CAN restrict that, but should we?  For experienced users, managing data access on a container by container basis is easy, but for new users, this just adds extra confusion that isn't necessary.  We have to care about the new user experience because our growth as a company is entirely reliant on new users at this point.  For the duckdns container to have access to /mnt/user or /mnt/appdata isn't a big deal.

 

Which brings me to my third point which may be out of scope for this thread, and if so please let me know the best place to discuss that. The reason the duckdns container still has the appdata folder is because during a container update, the xml was updated to include the environment variables in the template. In the old method, the user had to modify a config file in the appdata folder to input the subdomain and the api key). However, since the container xml's do not get updated when unraid updates a container, I had to provide a failsafe option that relies on the config file if environment variables are not set. I had to do a bunch of extra coding to keep both methods in just because I knew that existing users would not read the changelog and would not realize the missing env variables.

 

Can we add a method to update the container xml's in dockerman? One suggestion I can come up with is, dockerman can tag the user entered (manually) variables so they are distinguishable from the dev added ones. So during a container update, dockerman can check for differences in the dev modified variables and prompt the user for changes or confirmation.

 

That way, when I updated the duckdns container, the new xml would contain two new dev entered variables that do not exist on the dockerman saved one and it could prompt the user and ask them to enter that new info.

 

Brain is too fried from CES right now to even begin addressing this, but I will speak with Eric and get his input on the best way to handle it.

 

off the top of my head, there seem to be issues with containers with sqlite and postgres when mapped to a user share.

and as far as mapping /mnt to a container en masse, i've seen at least one container on CA in the past that did an operation on the whole mount point.. and god knows what it's like on the hub at large.

Then don't use that container. That's a bad container.

 

i know that, but the unitiated user of docker on unraid, who to be fair you're targeting with the /mnt thing to start with, isn't going to know that.

Link to comment

Anybody know the answer to this

...I have also read on the forum that /mnt/user does not work with hard links. Is this true and could this be an issue when specifying /mnt/user/appdata?

since /mnt/user/appdata was part of the proposal?

 

i've seen issues that have been resolved in containers by using /mnt/cache/appdata in place of /mnt/user/appdata

 

To bump up this thread and provide my $0.02

 

I concur with sparkly's statement. Many of my containers do not work if the appdata is on a user share. In other words, /mnt/user/appdata does not work, but /mnt/cache/appdata works and /mnt/diskX/appdata works as well (likely due to links, but did not have a chance to confirm).

 

If that's the case, we need bug reports opened and logs so we can resolve.

 

Another thing I wanted to highlight is that mapping /mnt or /mnt/user by default is not a good idea.

 

Why?  Users will have the option to NOT have this happen with a toggle under the Docker Settings page (a checkbox to auto-map all user shares to all new containers).  It will be on by default, but can be toggled off.

 

Why not leave that decision up to the container authors and make it optional? That way if a container needs access to the array (for instance plex definitely does) then the author can put that option in.

 

Authors shouldn't be determining where data goes on a user's system.  Instead, the user should have that choice.  That's how installing applications on ANY OS would work for fairly logical reasons.

 

There are numerous issues with letting an author specify the host path mapping for containers.  I've seen examples where authors path to /mnt/somethingrandom instead of a common path that is on all systems.  If a user added a container and just accepted that as a default value, then whatever would be written to that path would literally be going to RAM.  Nothing would persist and memory would slowly get chewed up until eventually the system would start to have issues because of low memory.

 

To go a step further, authors don't all use the same container path name for common paths.  E.g. /data vs. /appdata or /media vs. /movies.  So now the user has to remember, on an app by app basis for all their containers, what volume mappings tie to what hostmappings.  If we have a universal rule that /unraid maps to /mnt/user or /mnt for ALL containers, then when users are leveraging ANY application in a container and need to path to some share on the host, it would be the same method (just browse to /unraid).

 

For instance my duckdns container has absolutely no need for /mnt mapping. In fact I could get away with not even having an appdata folder as all the necessary info is in the template xml (it needs the duckdns subdomain and the api key, which are passed as environment variables so it can run a cron script every 5 minutes to update the dynamic dns address).

 

But what does it hurt for it to be there?  I get that some folks have OCD when it comes to data access, but keep in mind that for years we have had plugins, which have unrestricted access to the entire system (all data paths both on persistent storage, the flash device, and the root ram filesystem).  No one seemed to mind.  So now we have docker where we CAN restrict that, but should we?  For experienced users, managing data access on a container by container basis is easy, but for new users, this just adds extra confusion that isn't necessary.  We have to care about the new user experience because our growth as a company is entirely reliant on new users at this point.  For the duckdns container to have access to /mnt/user or /mnt/appdata isn't a big deal.

 

Which brings me to my third point which may be out of scope for this thread, and if so please let me know the best place to discuss that. The reason the duckdns container still has the appdata folder is because during a container update, the xml was updated to include the environment variables in the template. In the old method, the user had to modify a config file in the appdata folder to input the subdomain and the api key). However, since the container xml's do not get updated when unraid updates a container, I had to provide a failsafe option that relies on the config file if environment variables are not set. I had to do a bunch of extra coding to keep both methods in just because I knew that existing users would not read the changelog and would not realize the missing env variables.

 

Can we add a method to update the container xml's in dockerman? One suggestion I can come up with is, dockerman can tag the user entered (manually) variables so they are distinguishable from the dev added ones. So during a container update, dockerman can check for differences in the dev modified variables and prompt the user for changes or confirmation.

 

That way, when I updated the duckdns container, the new xml would contain two new dev entered variables that do not exist on the dockerman saved one and it could prompt the user and ask them to enter that new info.

 

Brain is too fried from CES right now to even begin addressing this, but I will speak with Eric and get his input on the best way to handle it.

 

off the top of my head, there seem to be issues with containers with sqlite and postgres when mapped to a user share.

and as far as mapping /mnt to a container en masse, i've seen at least one container on CA in the past that did an operation on the whole mount point.. and god knows what it's like on the hub at large.

Then don't use that container. That's a bad container.

 

i know that, but the unitiated user of docker on unraid, who to be fair you're targeting with the /mnt thing to start with, isn't going to know that.

Let's drop that container from CA then.

Link to comment

Somewhat OT for the topic heading, but something that is problematic for even the experienced docker users is keeping track of what ports are already being used when you want to add yet another docker to the many you already have. When you're on the page to add a container, you can't see the port mappings of your other dockers. I suppose you could print them out, but the column heading for Port Mappings seems to sort on the first port mapped for the app, which is totally useless since it is the host ports we are really trying to manage, and some dockers map multiple ports.

 

Don't really have any proposed solution for this just thought I would put it out there.

Link to comment

i believe we (linuxserver) took that container on and it's no longer in CA, but the point still stands about stuff on the wider hub.

 

it's  the wild wild west in terms of quality etc, i think it's sheer madness to open up the whole array like that.

Madness?  Please tell me that's exaggeration or sarcasm.  It's great that docker offers the user the ability to really lock down container access, and we are still locking it down better than plugins do as we are limiting the container from accessing the root RAM filesystem and the flash drive.

 

Also, remember that CA itself is also a plugin and that all containers inside it (aside from LT official containers) are always use at your own risk anyway. The urbackup issues are a prime example of that.

 

Authors should take care in testing their containers for issues like that before adding them to CA and users need to accept that those containers are made by the community who is not liable for issues they may cause (same as plugins).

Link to comment

Somewhat OT for the topic heading, but something that is problematic for even the experienced docker users is keeping track of what ports are already being used when you want to add yet another docker to the many you already have. When you're on the page to add a container, you can't see the port mappings of your other dockers. I suppose you could print them out, but the column heading for Port Mappings seems to sort on the first port mapped for the app, which is totally useless since it is the host ports we are really trying to manage, and some dockers map multiple ports.

 

Don't really have any proposed solution for this just thought I would put it out there.

Not slated for resolution in 6.2, but port management is definitely something we have a plan to handle in a future release.

Link to comment

Authors should take care in testing their containers for issues like that before adding them to CA and users need to accept that those containers are made by the community who is not liable for issues they may cause (same as plugins).

Which is why I added that disclaimer to the start of CA (which if you haven't updated over the last week then you haven't seen it yet).  Started thinking that if something goes completely haywire within an app I didn't want anyone yelling at me for it.

 

As for mapping /mnt/user, I'm all for making the whole thing a ton easier for new users, as quite honestly I get sick of answering the same questions over and over (and over) again.  But, I myself would never run my system like that. 

 

I have to say that unless there is a sure fire way to explain exactly how mapping works to a new user and have them understand it that there's not really any other choice than doing /mnt/user

 

I took my best shot a explaining mappings in the docker FAQ, and TBH I have yet to see anyone else come up with something better.  And mappings are the #1 problem around here. 

 

/mnt/user while it does pose some security risks is a solution to this nagging problem.

 

As a middle ground discussion, what about the possibility of making the default mappings of /mnt/user to be read-only?

 

 

Like I said, I don't agree with /mnt/user, but I don't see an easy alternative from a support perspective.

Link to comment

Somewhat OT for the topic heading, but something that is problematic for even the experienced docker users is keeping track of what ports are already being used when you want to add yet another docker to the many you already have. When you're on the page to add a container, you can't see the port mappings of your other dockers. I suppose you could print them out, but the column heading for Port Mappings seems to sort on the first port mapped for the app, which is totally useless since it is the host ports we are really trying to manage, and some dockers map multiple ports.

 

Don't really have any proposed solution for this just thought I would put it out there.

Not slated for resolution in 6.2, but port management is definitely something we have a plan to handle in a future release.

 

I agree with trurl, and pointed this out somewhere else as well.  It's not a serious problem, more of an annoyance.

Link to comment
  • 4 months later...

With our new "type" attribute for the <Volume> tag, it could look like this:

 

  <Data>

    <Volume type=“appdata”>

      <HostDir>/mnt/user/appdata/TaskBoard</HostDir>

      <ContainerDir>/var/www/TaskBoard</ContainerDir>

      <Mode>rw</Mode>

    </Volume>

  </Data>

 

If the user didn’t have a preference entered under Docker Settings for their preferred appdata location, then the <HostDir> element would be used.  If the user did have a preference specified, the <HostDir> element would be replaced with that preference.

I can't get this working. It just shows up as an regular path (path 1), and not appdata path.

Could someone take a look?

https://github.com/bjonness406/Dev-Templates/blob/master/Dev/amp.xml

 

Also, if I do this " <Volume type=“appdata”>", will I still be able to use the docker on 6.1.9?

Link to comment

With our new "type" attribute for the <Volume> tag, it could look like this:

 

  <Data>

    <Volume type=“appdata”>

      <HostDir>/mnt/user/appdata/TaskBoard</HostDir>

      <ContainerDir>/var/www/TaskBoard</ContainerDir>

      <Mode>rw</Mode>

    </Volume>

  </Data>

 

If the user didn’t have a preference entered under Docker Settings for their preferred appdata location, then the <HostDir> element would be used.  If the user did have a preference specified, the <HostDir> element would be replaced with that preference.

I can't get this working. It just shows up as an regular path (path 1), and not appdata path.

Could someone take a look?

https://github.com/bjonness406/Dev-Templates/blob/master/Dev/amp.xml

 

Also, if I do this " <Volume type=“appdata”>", will I still be able to use the docker on 6.1.9?

Reading this: http://lime-technology.com/wiki/index.php/DockerTemplateSchema it looks like that was an idea the jonp had that never actually got integrated.

 

How you should be generating XML's is by using 6.2 and filling out all of the appropriate entries and then clicking Save Template (and then copy & paste the screen to a file)

 

The resulting template contains both the PITA (but nice) sections for 6.2 and the legacy 6.1 sections. 

 

Manually editing an XML file (short of adding in the extra tags for CA use) is highly discouraged as typo's can be very easily made, and you must also change both the 6.2 sections and the 6.1 sections (hence why its a major PITA the way that gfjardim / LT decided to add additional functionality to the XML via attributes instead of extra tags -> hence the duplication of efforts required to make something that is both 6.1 and 6.2 compatible    :o  :-\  )

Link to comment

With our new "type" attribute for the <Volume> tag, it could look like this:

 

  <Data>

    <Volume type=“appdata”>

      <HostDir>/mnt/user/appdata/TaskBoard</HostDir>

      <ContainerDir>/var/www/TaskBoard</ContainerDir>

      <Mode>rw</Mode>

    </Volume>

  </Data>

 

If the user didn’t have a preference entered under Docker Settings for their preferred appdata location, then the <HostDir> element would be used.  If the user did have a preference specified, the <HostDir> element would be replaced with that preference.

I can't get this working. It just shows up as an regular path (path 1), and not appdata path.

Could someone take a look?

https://github.com/bjonness406/Dev-Templates/blob/master/Dev/amp.xml

 

Also, if I do this " <Volume type=“appdata”>", will I still be able to use the docker on 6.1.9?

Reading this: http://lime-technology.com/wiki/index.php/DockerTemplateSchema it looks like that was an idea the jonp had that never actually got integrated.

 

How you should be generating XML's is by using 6.2 and filling out all of the appropriate entries and then clicking Save Template (and then copy & paste the screen to a file)

 

The resulting template contains both the PITA (but nice) sections for 6.2 and the legacy 6.1 sections. 

 

Manually editing an XML file (short of adding in the extra tags for CA use) is highly discouraged as typo's can be very easily made, and you must also change both the 6.2 sections and the 6.1 sections (hence why its a major PITA the way that gfjardim / LT decided to add additional functionality to the XML via attributes instead of extra tags -> hence the duplication of efforts required to make something that is both 6.1 and 6.2 compatible    :o  :-\  )

Thanks! :)

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.