Event Handling

[Joe L.]

What was the result of those discussions with counsel?

Yes, let's not get bogged down on stuff that will rarely be used.

I like the .conf as it is a bit cleaner, and more self-explanatory.  It allows changing of a .conf w/o risking hosing the script, and vice versa.

Modding a .conf programatically is a lot easier than doing it to a script.

OK.

Since it is not an .ini file, is it OK to go with
#define MIN_EXCLUSIVE_TIME = 5
#define MAX_RUN_TIME = 60

Or do you have a different preference?

It is really easy to parse "#define" in a shell script.

rc=$script_name
min_exclusive_time=`grep "^#define MIN_EXCLUSIVE_TIME" "$rc" | sed 1q | tr -d " " | cut -d"=" -f2-`
max_run_time=`grep "^#define MAX_RUN_TIME" "$rc" | sed 1q | tr -d " " | cut -d"=" -f2-`

Joe L.

[bubbaQ]

I suggest:

#define_unraid MIN_EXCLUSIVE_TIME =

just to make clear to anyone else that this is an unraid-specific line in the script.

[WeeboTech]

I suggest:

#define_unraid MIN_EXCLUSIVE_TIME =

just to make clear to anyone else that this is an unraid-specific line in the script.

What does MIN_EXCLUSIVE_TIME mean?

[Joe L.]

I suggest:

#define_unraid MIN_EXCLUSIVE_TIME =

just to make clear to anyone else that this is an unraid-specific line in the script.

What does MIN_EXCLUSIVE_TIME mean?

A given "rc" script will be allowed to run in parallel with the next in the same event after this period of time.  The objective is to keep from running all in sequence which would slow down everything, or all in parallel, possible running into deadlocks/resource contention.

A default of a few seconds of (mostly) exclusive time will make a orderly start possible.  The event scripts will progress as fast as they can be invoked and completed, so it only affect those running more than the min-exclusive_time, but less than the max_run_time.

Perhaps I mis-understood the earlier posting.  What did -t XX -T YY represent?

Joe L.

[WeeboTech]

I don't want to worry about parallel processing.
If need be, the event script can fork itself into the background.

the loggerexec program will have two timeouts.

1. -T number of seconds the executed command has to "COMPLETE" before it is forcefully killed. This can be a really large number.

2. -t number of seconds the logger exec will wait for some kind of input (from the command's stdout and/or stderr)  every time the child program writes something, this timer is reset. So if if loggerexec stops receiving input for some arbitrary number of seconds, it will assume something went wrong with the command and it is hung.

If the child provides stdout updates resetting the -t seconds value, then the -T seconds value kicks in as that is a master timer which says if the program does not end in this time period, make it end!

Neither of these is mandatory in the program itself.
We can add default mandatory values in the rc script.


I don't want to get into parallel processing of events at this point in time.
To do that you have to have a master program fork children and keep an internal table of the children.  Then update the table as each child finishes. Keep timers and control things in a managerial role.

At this point. I would like to have an event triggered by emhttp.
This calls a script, which runs each respective script in an event directory.
As protection, a tool will be build to insure that each respective script finishes in a reasonable time frame.

What you are suggesting in regards to parallel processing could have been handled in the "register to an event" model that Tom suggested a while back.  Before we go down that path, let's keep it as simple as possible with good documentation and a standard model to go by.

[bubbaQ]

Do we have a prototype of the /etc/rc.d/rc.emhttp script for Tom?

Let's go simple for starters... no timeouts... just log the event and traverse the directories calling the +x scripts for now.

[WeeboTech]

I'm working on something, but it's not quite the rc.emhttp.
I started working on something called rc.event to prototype with. I hooked it into the rc.local and rc.local_shutdown scripts.
I'm having issues with hooking it into rc.local right now. I have a package that updates rc.local from /boot/extra.
But since rc.local is what is loading boot/extra, it never picks up the added line to run rc.event.
My next step is to update the go script automatically from the /boot/extra.

Then package it up for a review.
Meanwhile I've merged some code from redhat's initlog into util-linux-ng logger.c and have to test it out.
I'm not touching rc.emhttp yet.
I started to, but then decided on a more generic event layout to replate the whole power-control/powerdown package then see how it works out.

Once that is done borrowing the framework and renaming things for rc.emhttp should provide a fast path.

[BRiT]

Here's the events currently listed in the 5.0 beta 2, taken directly from /usr/local/emhttp/emhttp_event:

driver_loaded
#   Occurs early in emhttp initialization.
#   Can also occur as a result of init-config and device slot change.
#   Status information is valid.

array_started
#   Occurs during cmdStart execution.
#   The 'md' devices are valid.

disks_mounted
#   Occurs during cmdStart execution.
#   The disks and user shares (if enabled) are mounted.

svcs_started
#   Occurs during cmdStart execution.
#   The network services (samba, nfs, etc.) are started.

started
#   Signals end of cmdStart execution.

svcs_restarted
#   Occurs as a result of changing/adding/deleting a share.
#   The network services are started and may be exporting different share(s).

stopping_svcs
#   Occurs during cmdStop exeuction.
#   Nothing has actually been stopped yet, about to stop network services.

unmounting_disks
#   Occurs during cmdStop execution.
#   The network services have been stopped, about to unmount the disks.

stopping_array
#   Occurs during cmdStop exection.
#   The disks have been unmounted, about to stop the array.

stopped
#   Occurs at end of cmdStop execution.
#   The array has been stopped.

[bubbaQ]

OP updated with release of -beta2

[bubbaQ]

Have we actually developed the master event script (/usr/local/emhttp/emhttp_event) yet?  I'm ready to update my plugins to use it, but since there is no active code in the unRAID stock sript, I don't want to overwrite what it there, since I'm sure a future version will have it.

To start with, we could simply use:

Code:

find ./plugins -type f -perm /+x -maxdepth 2 -name event.sh -exec {} $1 $2  \;

[Joe L.]

Have we actually developed the master event script (/usr/local/emhttp/emhttp_event) yet?  I'm ready to update my plugins to use it, but since there is no active code in the unRAID stock sript, I don't want to overwrite what it there, since I'm sure a future version will have it.

To start with, we could simply use:

Code:

find ./plugins -type f -perm /+x -maxdepth 2 -name event.sh -exec {}

\;

I've been working on one that implimented a lot of the ideas presented.
From my recollection, and based on this thread, and specifically, this post ,Tom was looking for the script names to be
event_<eventname>.sh

So, we would need to modify your simple line to be
find ./plugins -type f -perm/+x -maxdepth 2 -name event_${1}.sh -exec {} \;


Now, we could certainly experiment with both standards, and have two lines as follows:
find ./plugins -type f -perm /+x -maxdepth 2 -name event_${1}.sh -exec {} \;
find ./plugins -type f -perm /+x -maxdepth 2 -name event.sh -exec {} $1 \;


Note: I added $1 to the end of the exec on the line you suggested, since if only one script is involved, it would need to know the event being passed.

These would not be fool-proof, but it will get us running.     

For a start,we can put the two lines suggested above...and worry about synchronization when we get to a package that requires it.

We also have WeeboTech's suggestion for a central directory for those events that must be synchronized and his proposed
find /usr/local/emhttp/event.d/$1.d -type f -follow -perm /u+x -printf "%p $1\n" | sort | xargs -L1 /bin/sh -c

I think that depending on the naming used, the third approach suggesteg by WeeboTech might have an event called twice if it is linked to emhttp/event.d/.  We'll need to think that through based on Tom's post.

I've been working on a more fool-proof version, but wanted to wait to see what 5.2 offered first.  Never found the time to get it changed to use the scheme Tom implemented in 5.2.   I'll try to get to it later today.

Joe L.

[bubbaQ]

Note: I added $1 to the end of the exec on the line you suggested, since if only one script is involved, it would need to know the event being passed.

I actually caught that omission and changed it before your post

After working with this a while, I really don't see a need to have multiple files, when the files are all native to one plugin.

[Joe L.]

Note: I added $1 to the end of the exec on the line you suggested, since if only one script is involved, it would need to know the event being passed.

I actually caught that omission and changed it before your post

After working with this a while, I really don't see a need to have multiple files, when the files are all native to one plugin.

I think the same way.  One script is sufficient.  (I've never needed to disable just one event, unless in development mode, and for that we can just edit plugin/event.sh )

So... one line is all we need for now to get everything started.
find ./plugins -type f -perm /+x -maxdepth 2 -name event.sh -exec {} $1 \;

Joe L.

[WeeboTech]

Like         -> find ./plugins -type f -perm /+x -maxdepth 2 -name event.sh -exec {} $1 \;
Dont Like -> find ./plugins -type f -perm /+x -maxdepth 2 -name event_${1}.sh -exec {} \;

Teach the newbies to fish.
A good example.model to work with is worth the time and effort.

I would like to see this in addition.
find /usr/local/emhttp/event.d/$1.d -type f -follow -perm /u+x -printf "%p $1\n" | sort | xargs -L1 /bin/sh -c

This really comes in to play for scriptlets that are not gui oriented plugins.

[Joe L.]

Like         -> find ./plugins -type f -perm /+x -maxdepth 2 -name event.sh -exec {} $1 \;

So do I.

Dont Like -> find ./plugins -type f -perm /+x -maxdepth 2 -name event_${1}.sh -exec {} \;

Only reason I included it was because of Tom's post.   I prefer it all in one script with $1 to define the "event" occurring.

Teach the newbies to fish.
A good example.model to work with is worth the time and effort.

True.

I would like to see this in addition.
find /usr/local/emhttp/event.d/$1.d -type f -follow -perm /u+x -printf "%p $1\n" | sort | xargs -L1 /bin/sh -c

This really comes in to play for scriptlets that are not gui oriented plugins.

I agree.  It also has its place.

That gives us (to start)
find ./plugins -type f -perm /+x -maxdepth 2 -name event.sh -exec {} $1 \;
find /usr/local/emhttp/event.d/$1.d -type f -follow -perm /u+x -printf "%p $1\n" | sort | xargs -L1 /bin/sh -c

Joe L.