[6.2.1] plugin development - WebUI "loading" forever after script completes

Description:

I've been struggling with this behavior for years (since unraid v5 RC's), and after some additional debugging effort over the past week, am convinced this is a problem with unraid and not with my plugin.

I'm developing a plugin which executes a script that invokes "encfs" to mount an encfs fuse encrypted store.  If the script executes an encfs mount command, the webUI will "spin forever" and never finish loading.

The encfs command does not appear to be generating any output.  I've tried numerous things (redirect all output to /dev/null, launch encfs in the background using "&", launch encfs even more in the background using "nohup", launch the command in a subshell [inside parens in the bash script], encfs mount using an 'expect' script to provide the password, encfs mount using pipe'd in password, tee the output to a file and verified that the file contains 0 characters, long sleeps, no sleeps, more output, no output, etc) -- the result is all the same that the WebUI does not respond correctly if encfs is invoked.

If I change nothing in the script except to replace the encfs mount command with an empty or logging command, the page loads properly.

Relevant script lines:

  logger `whoami`
  logger `pwd`
  logger "starting encfs mount"
  echo "$1" | encfs -S --public "$ENCDIR0" "$DECDIR0"  -- -o uid=99 -o gid=100 2>&1 | tee /var/log/plugins/encfs_mount_output
  sleep 2
  echo "Mount command done"
  logger "encfs mount command has returned"

WebUI shows "Mount command done", but "is loading" forever.  And the mount *does* complete successfully and is completely usable.

Oct 26 16:40:40 nickserver emhttp: cmd: /usr/local/emhttp/plugins/encfs/scripts/rc.encfs mount test

Oct 26 16:40:40 nickserver root: root

Oct 26 16:40:40 nickserver root: /

Oct 26 16:40:40 nickserver root: starting encfs mount

Oct 26 16:40:43 nickserver root: encfs mount command has returned

# ls -lh /var/log/plugins/encfs_mount_output

-rw-rw-rw- 1 root root 0 Oct 26 16:40 /var/log/plugins/encfs_mount_output

Executing the same command as the same user from the same CWD (ie, " /usr/local/emhttp/plugins/encfs/scripts/rc.encfs mount test" as root from /), the script finishes quickly and returns to the command prompt as expected.

If I change nothing except the encfs line, e.g. as follows:

  logger `whoami`
  logger `pwd`
  logger "starting encfs mount"
#  echo "$1" | encfs -S --public "$ENCDIR0" "$DECDIR0"  -- -o uid=99 -o gid=100 2>&1 | tee /var/log/plugins/encfs_mount_output
  echo "$1" | logger "null command" | tee /var/log/plugins/encfs_mount_output
  sleep 2
  echo "Mount command done"
  logger "encfs mount command has returned"

The page finishes loading as expected.

Oct 26 16:48:45 nickserver emhttp: cmd: /usr/local/emhttp/plugins/encfs/scripts/rc.encfs mount test

Oct 26 16:48:45 nickserver root: root

Oct 26 16:48:45 nickserver root: /

Oct 26 16:48:45 nickserver root: starting encfs mount

Oct 26 16:48:45 nickserver root: null command

Oct 26 16:48:47 nickserver root: encfs mount command has returned

How to reproduce:

Install encfs plugin from my debugging branch: https://github.com/nick5429/unraid-encfs/blob/pipe/encfs.plg

Enable plugin, configure directories to something different if you want, follow directions for manually setting up the encrypted store.  Use "testing" as the password.

Use the plugin UI with your password to attempt to mount

Expected results:

Script finishes, WebUI loads

Actual results:

Script finishes (as seen by logger results), but the WebUI remains loading forever.

If you hit "esc" to stop loading, then browse back to the plugin page (or list the directory at the command line) -- you'll see that the command did complete successfully and the encrypted store was mounted, even though the WebUI never finished loading.

Other information:

This exact same behavior has been occurring since unRAID v5

Side note: it'd be really great if there were a way to convey the password from the webUI form to the mounting script without logging it in syslog or storing it as a permanent file somewhere.  This is a new problem as of the unraid v6.1 plugin system requirements -- I need a mode to disable logging of webUI command parameters.

Note that while this was moved from Defect Reports to Programming, I do still believe this is an unraid issue, not a "plugin development" issue -- though I'm willing to be proven wrong if anyone knows how to make this work.

This is a known limitation when using the plugin system and you need to make use of a workaround by starting the intended command on a scheduled base.

The following example - hopefully - makes it clearer

First a temporary script is created:

<!-- WORKAROUND -->
<FILE Name="/tmp/start_encfs" Mode="0770">
<INLINE>
#!/bin/bash
logger "starting encfs mount"  
echo "$1" | encfs -S --public "$ENCDIR0" "$DECDIR0"  -- -o uid=99 -o gid=100 2>&1 | tee /var/log/plugins/encfs_mount_output
echo "Mount command done"
logger "encfs mount command has returned"
</INLINE>
</FILE>

Next in your main script instead of executing the command directly, use the 'at' command to start the temporary script

at -M -f /tmp/start_encfs now 2>/dev/null

This lets the command execute in a completely detached manner. I.e. your PLG script will not hang.

Got it, thanks!

Woulda been nice if that'd been clearly documented somewhere.... 

Curiosity: what causes this, what triggers it, why does THIS command hit it and other commands in my plugin don't, etc?

Got it, thanks!

 

Woulda been nice if that'd been clearly documented somewhere.... 

 

 

Curiosity: what causes this, what triggers it, why does THIS command hit it and other commands in my plugin don't, etc?

We are talking "corner case" scenario here.

I remember there is an explanation given here on the forum about the issue and the workaround, perhaps a search can help you.

We are talking "corner case" scenario here.

It can't be THAT much of a corner case if my plugin hits it every time the command is executed, and several of your scripts apparently hit it as well.

Though that was really more an oblique comment about how there's zero documentation available on the plugin system at all beyond "look at what I did and try to reverse engineer how it works. good luck."

It can't be THAT much of a corner case if my plugin hits it every time the command is executed, and several of your scripts apparently hit it as well.

 

Though that was really more an oblique comment about how there's zero documentation available on the plugin system at all beyond "look at what I did and try to reverse engineer how it works. good luck."

With corner case I mean MOST plugins do not have an issue or need special requirements.

And there is documentation (only you didn't find it). See http://lime-technology.com/forum/index.php?topic=34970.0

And there is documentation (only you didn't find it). See http://lime-technology.com/forum/index.php?topic=34970.0

I found that and it's helpful, it's just inadequate.  Not trying to pick a fight or anything but...

a) that's all from someone's efforts at writing stuff down while they reverse engineered the plugin system. Helpful, certainly, but incomplete. E.g., even simple things like "what is the list of unraid events my plugin can operate on?" are missing (both from there and from anywhere, as far as I could find)

b) it's outdated and some of what IS there is obsolete.

Some things can be pieced together from other places, but scattered partially-deprecated tidbits in forum posts don't add up to documentation of an API, especially if you want to encourage people to extend the ecosystem you're selling

And there is documentation (only you didn't find it). See http://lime-technology.com/forum/index.php?topic=34970.0

 

I found that and it's helpful, it's just inadequate.  Not trying to pick a fight or anything but...

 

a) that's all from someone's efforts at writing stuff down while they reverse engineered the plugin system. Helpful, certainly, but incomplete. E.g., even simple things like "what is the list of unraid events my plugin can operate on?" are missing (both from there and from anywhere, as far as I could find)

b) it's outdated and some of what IS there is obsolete.

 

Some things can be pieced together from other places, but scattered partially-deprecated tidbits in forum posts don't add up to documentation of an API, especially if you want to encourage people to extend the ecosystem you're selling

Best documentation is to study existing code, existing plugins, etc.  Also look at scripts in /usr/local/sbin.  For example 'emhttp_event' has info on events.  Agreed, we should take the time to fully document all API's - it's on the "todo" list.  But  always ask yourself, is this something better suited as a container?

But  always ask yourself, is this something better suited as a container?

Yes it certainly would be.

However in this case, my understanding is that this is impossible due to technical limitations in docker

I've tried to collect all of the plugin docs here:  Plugin system documentation