NAME

       nq — job queue utility

SYNOPSIS

       nq [-c] [-q] command line ...
       nq -t job id ...
       nq -w job id ...

DESCRIPTION

       The  nq  utility  provides  a  very  lightweight  queuing  system  without  requiring setup, maintenance,
       supervision or any long-running processes.

       Job order is enforced by a timestamp nq gets immediately when started.  Synchronization happens on  file-
       system  level.   Timer  resolution  is milliseconds.  No sub-second file system time stamps are required.
       Polling is not used.  Exclusive execution is maintained strictly.

       You enqueue(!) new jobs into the queue by running

             nq command line ...

       The job id (a file name relative to NQDIR, which defaults to the current  directory)  is  output  (unless
       suppressed  using  -q)  and nq detaches from the terminal immediately, running the job in the background.
       Standard output and standard error are redirected into the  job  id  file.   nqtail(1)  can  be  used  to
       conveniently watch the log files.

       The options are as follows:

       -c      Clean up job id file when process exited with status 0.

       -q      Suppress output of the job id after spawning new job.

       -t      Enter  test  mode:  exit with status 0 when all of the listed job ids are already done, else with
               status 1.

       -w      Enter waiting mode: wait in the foreground until all listed job ids are done.

ENVIRONMENT

       NQDIR      Directory where lock files/job output resides.  Each NQDIR can be considered a separate queue.
                  The current working directory is used when NQDIR is unset.  NQDIR is created if needed.

       NQDONEDIR  When set, specifies a directory (must be  on  the  same  file  system  as  NQDIR)  where  lock
                  files/job output is moved to after successful execution of the job.

                  Ignored when -c is used.

       NQFAILDIR  When  set,  specifies  a  directory  (must  be  on  the  same file system as NQDIR) where lock
                  files/job output is moved to after unsuccessful execution of the job.

       NQJOBID    The job id of the currently running job, exposed to the job itself.

FILES

       nq expects to control all files in NQDIR (respectively .) which start with “,” or “.,”.  These files  are
       created according to the following scheme:

             ,hexadecimal-time-stamp.pid

EXIT STATUS

       The  nq  utility  exits  0 on success, and >0 if an error occurs; unless test mode is used, in which case
       exit status 1 means there is a job running.

       On fatal errors, exit codes 111 and 222 are used.

EXAMPLES

       Build make(1) targets clean, depends, all, without occupying the terminal:

             % nq make clean
             % nq make depends
             % nq make all
             % nqtail
             ... look at output, can interrupt with C-c any time
             without stopping the build ...

       Simple download queue, accessible from multiple terminals:

             % alias qget='NQDIR=/tmp/downloads nq wget'
             % alias qwait='NQDIR=/tmp/downloads nqtail -q'
             window1% qget http://mymirror/big1.iso
             window2% qget http://mymirror/big2.iso
             window3% qget http://mymirror/big3.iso
             % qwait
             ... wait for all downloads to finish ...

       As nohup(1) replacement (The benchmark will run in background, every run gets a  different  output  file,
       and the command line you ran is logged too.):

             % ssh remote
             remote% nq ./run-benchmark
             ,14f6f3034f8.17035
             remote% ^D
             % ssh remote
             remote% nqtail
             ... see output, nqtail exits when job finished ...

TRICKS

       The  "file  extension"  of  the  log file is actually the PID of the job.  nq runs all jobs in a separate
       process group, so you can kill an entire job process tree at once using  kill(1)  with  a  negative  PID.
       Before the job is started, it is the PID of nq, so you can cancel a queued job by killing it as well.

       Thanks  to  the  initial  exec  line  in the log files, you can resubmit a job by executing it as a shell
       command file, i.e. running

             sh job id

       Creating nq wrappers setting NQDIR to provide different queues for different purposes is encouraged.

INTERNALS

       Enforcing job order works like this:
       -   every job has an output file locked using flock(2) and named according to “FILES”.
       -   every job starts only after all earlier flocked files are unlocked.
       -   the lock is released by the kernel after the job terminates.

ASSUMPTIONS

       nq will only work correctly when:

       -   NQDIR (respectively .) is writable.

       -   flock(2) works correctly in NQDIR (respectively .).

       -   gettimeofday(2) behaves monotonic (using CLOCK_MONOTONIC would  create  confusing  file  names  after
           reboot).

       -   No other programs put files matching ,* into NQDIR (respectively .).

SEE ALSO

       nqtail(1), nqterm(1).

       Alternatives to the nq system include batch(1), qsub(1), schedule(1), srun(1), and ts(1).

AUTHORS

       Leah Neukirchen <leah@vuxu.org>

CAVEATS

       All  reliable queue status information is in main memory only, which makes restarting a job queue after a
       reboot difficult.

LICENSE

       nq is in the public domain.

       To the extent possible under law, the creator of this work  has  waived  all  copyright  and  related  or
       neighboring rights to this work.

       http://creativecommons.org/publicdomain/zero/1.0/

Debian                                            July 3, 2024                                             NQ(1)