The bulk of time in my research projects is spent collecting experimental data; for which the worflow is approximately:
Runtimes for these scripts can vary from hours to weeks, and with a dozen experiments running concurrently it’s easy to become swamped.
A rudimentary solution for this is to have the job email you its output upon completion:
$ bash ./experiements.sh 2>&1 | mail -s "[email protected]$(hostname) done" $EMAIL
There are two downsides to this: you lose the (often critically important) returncode of the job, and it requires a configured mail stack which often varies widly from one system to the next.
For this week’s 20% project I decided to partially re-invent the wheel and build a compact, standalone tool for sending these notifications. The resulting program Let Me Know (
lmk) acts as a drop in replacement for the
$ bash ./experiements.sh 2>&1 | lmk -
However, by flipping the order around and having
lmk run the process, we can preserve that returncode information:
$ lmk 'bash ./experiements.sh'
Upon task completion, you get a nice, HTML formatted email of the program’s output, it’s returncode, and runtime:
Compared to last week, this was a small scope project, but still with some valuable learning lessons. I began to see the dark side of writing Python 2+3 compatbile programs, and I spent more time than I’d care to admit learning about Unicode handling as a result of my use of emojis 😉.