20% Project 2 - Let Me Know
The bulk of time in my research projects is spent collecting experimental data; for which the worflow is approximately:
- Launch a script on a remote server.
- Go have a coffee and live life.
- Log on to remote server, see the script has finished / crashed.
- If not, GOTO 2.
- If it has, review output and decide on next action.
Runtimes for these scripts can vary from hours to weeks, and with a dozen experiments running concurrently it’s easy to become swamped.
A rudimentary solution for this is to have the job email you its output upon completion:
$ bash ./experiements.sh 2>&1 | mail -s "$USER@$(hostname) done" $EMAIL
There are two downsides to this: you lose the (often critically important) returncode of the job, and it requires a configured mail stack which often varies widly from one system to the next.
For this week’s 20% project I decided to partially re-invent the wheel and build a compact, standalone tool for sending these notifications. The resulting program Let Me Know (lmk) acts as a drop in replacement for the mail command:
$ bash ./experiements.sh 2>&1 | lmk -
However, by flipping the order around and having lmk run the process, we can preserve that returncode information:
$ lmk 'bash ./experiements.sh'
Upon task completion, you get a nice, HTML formatted email of the program’s output, it’s returncode, and runtime:
Compared to last week, this was a small scope project, but still with some valuable learning lessons. I began to see the dark side of writing Python 2+3 compatbile programs, and I spent more time than I’d care to admit learning about Unicode handling as a result of my use of emojis 😉.