NAME

       weaver - workflow engine for executing distributed workflows

SYNOPSIS

       weaver [options] <weaverfile>

DESCRIPTION

       Weaver  is  a  high  level  interface  to  makeflow.  A  weaver input file is written in python, with the
       definition of functions to be applied on sets of files. weaver interprets this input file and generates a
       workflow specification that can be executed by makeflow. This allows an straightforward implementation of
       different workflow execution patterns, such as MapReduce, and AllPairs.

                                         /--------                        +-+ Python |
                                       | ---+----/
               +---------------------------------+ |     | Generate DAG
               |          Weaver          +-+     v
               +---------------------------------+   /-------|        Makeflow         +---+  DAG  |
               +--------+-----------+-----+------+   -------/
               | Condor | WorkQueue | SGE | Unix +-+     | Dispatch Jobs
               +--------+-----------+-----+------+ |     v
                                       | /-------                    +-+ Jobs  |
                                         -------/

OPTIONS

       By default, running weaver on a <weaverfile> generates an input file for makeflow, <Makeflow>, and a  di‐
       rectory, <_Stash>, in which intermediate files are stored.

       General options:

        -h    Give help information.

        -W    Stop on warnings.

        -g    Include debugging symbols in DAG.

        -I    Do not automatically import built-ins.

        -N    Do not normalize paths.

        -b <options>
              Set batch job options (cpu, memory, disk, batch, local, collect).

        -d <subsystem>
              Enable debugging for subsystem.

        -o <log_path>
              Set log path (default: stderr).

        -O <directory>
              Set stash output directory (default <_Stash>).

              Optimization Options:

        -a    Automatically nest abstractions.

        -t <group_size>
              Inline tasks based on group size.

              Engine Options:

        -x    Execute DAG using workflow engine after compiling.

        -e <arguments>
              Set arguments to workflow engine when executing.

        -w <wrapper>
              Set workflow engine wrapper.

EXIT STATUS

       On success, returns zero.  On failure, returns non-zero.

EXAMPLES

       Weaver expresses common workflow patterns succinctly. For example, with only the following three lines of
       code we can express a map pattern, in which we convert some images to the jpeg format:

               convert = ParseFunction('convert {IN} {OUT}')
               dataset = Glob('/usr/share/pixmaps/*.xpm')
               jpgs    = Map(convert, dataset, '{basename_woext}.jpg')

       Please refer to cctools/doc/weaver_examples for further information.

       The  Cooperative  Computing  Tools are Copyright (C) 2022 The University of Notre Dame.  This software is
       distributed under the GNU General Public License.  See the file COPYING for details.

CCTools 7.14.5 FINAL                                                                                   weaver(1)