diff --git a/doc/Makefile.am b/doc/Makefile.am index ac381f19b7b1..cba4a8d516ff 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -33,7 +33,6 @@ MAN1_FILES_PRIMARY = \ man1/flux-queue.1 \ man1/flux-cron.1 \ man1/flux-event.1 \ - man1/flux-mini.1 \ man1/flux-submit.1 \ man1/flux-run.1 \ man1/flux-bulksubmit.1 \ @@ -342,9 +341,7 @@ $(RST_FILES): \ man1/common/submit-process-resource-limits.rst \ man1/common/submit-exit-status.rst \ man1/common/submit-other-options.rst \ - man1/common/submit-bulksubmit.rst \ man1/common/submit-shell-options.rst \ - man1/common/submit-submission-directives.rst \ man3/common/json_pack.rst \ man3/common/json_unpack.rst endif @@ -403,9 +400,6 @@ EXTRA_DIST = \ guide/glossary.rst \ $(RST_FILES) \ man1/index.rst \ - man1/index_general.rst \ - man1/index_job.rst \ - man1/index_kvs.rst \ man1/common/nodeset.rst \ man1/common/submit-job-parameters.rst \ man1/common/submit-standard-io.rst \ @@ -416,9 +410,7 @@ EXTRA_DIST = \ man1/common/submit-process-resource-limits.rst \ man1/common/submit-exit-status.rst \ man1/common/submit-other-options.rst \ - man1/common/submit-bulksubmit.rst \ man1/common/submit-shell-options.rst \ - man1/common/submit-submission-directives.rst \ man3/index.rst \ man3/index_general.rst \ man3/index_idset.rst \ diff --git a/doc/man1/common/submit-bulksubmit.rst b/doc/man1/common/submit-bulksubmit.rst deleted file mode 100644 index 7c49b4b61576..000000000000 --- a/doc/man1/common/submit-bulksubmit.rst +++ /dev/null @@ -1,97 +0,0 @@ -:man1:`flux-bulksubmit` allows rapid bulk submission of jobs using -an interface similar to GNU parallel or ``xargs``. The command takes -inputs on stdin or the command line (separated by ``:::``), and submits -the supplied command template and options as one job per input combination. - -The replacement is done using Python's ``string.format()``, which is -supplied a list of inputs on each iteration. Therefore, in the common case -of a single input list, ``{}`` will work as the substitution string, e.g.:: - - $ seq 1 4 | flux bulksubmit echo {} - bulksubmit: submit echo 1 - bulksubmit: submit echo 2 - bulksubmit: submit echo 3 - bulksubmit: submit echo 4 - -With ``--dry-run`` ``bulksubmit`` will print the args and command which -would have been submitted, but will not perform any job submission. - -The ``bulksubmit`` command can also take input lists on the command line. -The inputs are separated from each other and the command with the special -delimiter ``:::``:: - - $ flux bulksubmit echo {} ::: 1 2 3 4 - bulksubmit: submit echo 1 - bulksubmit: submit echo 2 - bulksubmit: submit echo 3 - bulksubmit: submit echo 4 - -Multiple inputs are combined, in which case each input is passed as a -positional parameter to the underlying ``format()``, so should be accessed -by index:: - - $ flux bulksubmit --dry-run echo {1} {0} ::: 1 2 ::: 3 4 - bulksubmit: submit echo 3 1 - bulksubmit: submit echo 4 1 - bulksubmit: submit echo 3 2 - bulksubmit: submit echo 4 2 - -If the generation of all combinations of an input list with other inputs is not -desired, the special input delimited ``:::+`` may be used to "link" the input, -so that only one argument from this source will be used per other input, -e.g.:: - - $ flux bulksubmit --dry-run echo {0} {1} ::: 1 2 :::+ 3 4 - bulksubmit: submit 1 3 - bulksubmit: submit 2 4 - -The linked input will be cycled through if it is shorter than other inputs. - -An input list can be read from a file with ``::::``:: - - $ seq 0 3 >inputs - $ flux bulksubmit --dry-run :::: inputs - bulksubmit: submit 0 - bulksubmit: submit 1 - bulksubmit: submit 2 - bulksubmit: submit 3 - -If the filename is ``-`` then ``stdin`` will be used. This is useful -for including ``stdin`` when reading other inputs. - -The delimiter ``::::+`` indicates that the next file is to be linked to -the inputs instead of combined with them, as with ``:::+``. - -There are several predefined attributes for input substitution. -These include: - - - ``{.%}`` returns the input string with any extension removed. - - ``{./}`` returns the basename of the input string. - - ``{./%}`` returns the basename of the input string with any - extension removed. - - ``{.//}`` returns the dirname of the input string - - ``{seq}`` returns the input sequence number (0 origin) - - ``{seq1}`` returns the input sequence number (1 origin) - - ``{cc}`` returns the current ``id`` from use of ``--cc`` or ``--bcc``. - Note that replacement of ``{cc}`` is done in a second pass, since the - ``--cc`` option argument may itself be replaced in the first substitution - pass. If ``--cc/bcc`` were not used, then ``{cc}`` is replaced with an - empty string. This is the only substitution supported with - :man1:`flux-submit`. - -Note that besides ``{seq}``, ``{seq1}``, and ``{cc}`` these attributes -can also take the input index, e.g. ``{0.%}`` or ``{1.//}``, when multiple -inputs are used. - -Additional attributes may be defined with the ``--define`` option, e.g.:: - - $ flux bulksubmit --dry-run --define=p2='2**int(x)' -n {.p2} hostname \ - ::: $(seq 0 4) - bulksubmit: submit -n1 hostname - bulksubmit: submit -n2 hostname - bulksubmit: submit -n4 hostname - bulksubmit: submit -n8 hostname - bulksubmit: submit -n16 hostname - -The input string being indexed is passed to defined attributes via the -local ``x`` as seen above. diff --git a/doc/man1/common/submit-constraints.rst b/doc/man1/common/submit-constraints.rst index 399c0ae9468b..afda410c933f 100644 --- a/doc/man1/common/submit-constraints.rst +++ b/doc/man1/common/submit-constraints.rst @@ -1,7 +1,8 @@ CONSTRAINTS =========== -**--requires=CONSTRAINT** +.. option:: --requires=CONSTRAINT** + Specify a set of allowable properties and other attributes to consider when matching resources for a job. The **CONSTRAINT** is expressed in a simple syntax described in RFC 35 (Constraint Query Syntax) which is @@ -29,7 +30,7 @@ CONSTRAINTS The full specification of Constraint Query Syntax can be found in RFC 35. - Currently, **--requires** supports the following operators: + Currently, :option:`--requires` supports the following operators: properties Require the set of specified properties. Properties may be diff --git a/doc/man1/common/submit-dependencies.rst b/doc/man1/common/submit-dependencies.rst index 2b12f01f5304..3a2012e3f40b 100644 --- a/doc/man1/common/submit-dependencies.rst +++ b/doc/man1/common/submit-dependencies.rst @@ -8,7 +8,8 @@ DEPENDENCIES Dependencies may be specified on the command line using the following options: -**--dependency=URI** +.. option:: --dependency=URI + Specify a dependency of the submitted job using RFC 26 dependency URI format. The URI format is **SCHEME:VALUE[?key=val[&key=val...]]**. The URI will be converted into RFC 26 JSON object form and appended to @@ -17,7 +18,7 @@ Dependencies may be specified on the command line using the following options: submitted job will be rejected with an error message indicating this fact. - The ``--dependency`` option may be specified multiple times. Each use + The :option:`--dependency` option may be specified multiple times. Each use appends a new dependency object to the ``attributes.system.dependencies`` array. diff --git a/doc/man1/common/submit-environment.rst b/doc/man1/common/submit-environment.rst index 5dbd4a0cb5f2..9f53b668fb5b 100644 --- a/doc/man1/common/submit-environment.rst +++ b/doc/man1/common/submit-environment.rst @@ -5,27 +5,30 @@ By default, these commands duplicate the current environment when submitting jobs. However, a set of environment manipulation options are provided to give fine control over the requested environment submitted with the job. -**--env=RULE** +.. option:: --env=RULE + Control how environment variables are exported with *RULE*. See - *ENV RULE SYNTAX* section below for more information. Rules are + the `ENV RULES`_ section below for more information. Rules are applied in the order in which they are used on the command line. This option may be specified multiple times. -**--env-remove=PATTERN** +.. option:: --env-remove=PATTERN + Remove all environment variables matching *PATTERN* from the current generated environment. If *PATTERN* starts with a ``/`` character, then it is considered a :linux:man7:`regex`, otherwise *PATTERN* is treated as a shell :linux:man7:`glob`. This option is equivalent to - ``--env=-PATTERN`` and may be used multiple times. + :option:`--env=-PATTERN` and may be used multiple times. + +.. option:: --env-file=FILE -**--env-file=FILE** Read a set of environment *RULES* from a *FILE*. This option is - equivalent to ``--env=^FILE`` and may be used multiple times. + equivalent to :option:`--env=^FILE` and may be used multiple times. ENV RULES ========= -The ``--env*`` options allow control of the environment exported to jobs +The `ENVIRONMENT`_ options allow control of the environment exported to jobs via a set of *RULE* expressions. The currently supported rules are * If a rule begins with ``-``, then the rest of the rule is a pattern @@ -74,8 +77,8 @@ via a set of *RULE* expressions. The currently supported rules are ``PATH``, ``FLUX_*_PATH``, ``/^OMP.*/`` Since we always starts with a copy of the current environment, -the default implicit rule is ``*`` (or ``--env=*``). To start with an -empty environment instead, the ``-*`` rule or ``--env-remove=*`` option +the default implicit rule is ``*`` (or :option:`--env=*`). To start with an +empty environment instead, the ``-*`` rule or :option:`--env-remove=*` option should be used. For example, the following will only export the current ``PATH`` to a job: @@ -85,8 +88,9 @@ should be used. For example, the following will only export the current Since variables can be expanded from the currently built environment, and -``--env`` options are applied in the order they are used, variables can -be composed on the command line by multiple invocations of ``--env``, e.g.: +:option:`--env` options are applied in the order they are used, variables can +be composed on the command line by multiple invocations of :option:`--env`, +e.g.: :: diff --git a/doc/man1/common/submit-exit-status.rst b/doc/man1/common/submit-exit-status.rst index ccbf4ce45870..d6b69860fc74 100644 --- a/doc/man1/common/submit-exit-status.rst +++ b/doc/man1/common/submit-exit-status.rst @@ -7,6 +7,6 @@ the job exit status is 128+signo. The ``flux-job attach`` command exits with the job exit status. -In addition, :man:`flux-run` runs until the job completes and exits +In addition, :man1:`flux-run` runs until the job completes and exits with the job exit status. diff --git a/doc/man1/common/submit-job-parameters.rst b/doc/man1/common/submit-job-parameters.rst index 3a3ac3df079b..ca20e9fcf38b 100644 --- a/doc/man1/common/submit-job-parameters.rst +++ b/doc/man1/common/submit-job-parameters.rst @@ -9,19 +9,22 @@ Common resource options These commands take the following common resource allocation options: -**-N, --nodes=N** +.. option:: -N, --nodes=N + Set the number of nodes to assign to the job. Tasks will be distributed evenly across the allocated nodes, unless the per-resource options (noted below) are used with *submit*, *run*, or *bulksubmit*. It is an error to request more nodes than there are tasks. If unspecified, the number of nodes will be chosen by the scheduler. -**-x, --exclusive** +.. option:: -x, --exclusive + Indicate to the scheduler that nodes should be exclusively allocated to this job. It is an error to specify this option without also using - *-N, --nodes*. If *--nodes* is specified without *--nslots* or *--ntasks*, - then this option will be enabled by default and the number of tasks - or slots will be set to the number of requested nodes. + :option:`--nodes`. If :option:`--nodes` is specified without + :option:`--nslots` or :option:`--ntasks`, then this option will be enabled + by default and the number of tasks or slots will be set to the number of + requested nodes. Per-task options @@ -33,13 +36,16 @@ The most common form uses the total number of tasks to run along with the amount of resources required per task to specify the resources for the entire job: -**-n, --ntasks=N** +.. option:: -n, --ntasks=N + Set the number of tasks to launch (default 1). -**-c, --cores-per-task=N** +.. option:: -c, --cores-per-task=N + Set the number of cores to assign to each task (default 1). -**-g, --gpus-per-task=N** +.. option:: -g, --gpus-per-task=N + Set the number of GPU devices to assign to each task (default none). Per-resource options @@ -50,19 +56,23 @@ with the number of tasks per core or node set on the command line. It is an error to specify any of these options when using any per-task option listed above: -**--cores=N** +.. option:: --cores=N + Set the total number of cores. -**--tasks-per-node=N** +.. option:: --tasks-per-node=N + Set the number of tasks per node to run. -**--gpus-per-node=N** - With -N, --nodes, request a specific number of GPUs per node. +.. option:: --gpus-per-node=N + + With :option:`--nodes`, request a specific number of GPUs per node. + +.. option:: --tasks-per-core=N -**--tasks-per-core=N** Force a number of tasks per core. Note that this will run *N* tasks per *allocated* core. If nodes are exclusively scheduled by configuration or - use of the ``--exclusive`` flag, then this option could result in many + use of the :option:`--exclusive` flag, then this option could result in many more tasks than expected. The default for this option is effectively 1, so it is useful only for oversubscribing tasks to cores for testing purposes. You probably don't want to use this option. @@ -75,13 +85,16 @@ therefore job parameters are specified in terms of resource slot size and number of slots. A resource slot can be thought of as the minimal resources required for a virtual task. The default slot size is 1 core. -**-n, --nslots=N** +.. option:: -n, --nslots=N + Set the number of slots requested. This parameter is required. -**-c, --cores-per-slot=N** +.. option:: -c, --cores-per-slot=N + Set the number of cores to assign to each slot (default 1). -**-g, --gpus-per-slot=N** +.. option:: -g, --gpus-per-slot=N + Set the number of GPU devices to assign to each slot (default none). Additional job options @@ -89,25 +102,30 @@ Additional job options These commands also take following job parameters: -**-q, --queue=NAME** +.. option:: -q, --queue=NAME + Submit a job to a specific named queue. If a queue is not specified and queues are configured, then the jobspec will be modified at ingest to specify the default queue. If queues are not configured, then this option is ignored, though :man1:`flux-jobs` may display the queue name in its rendering of the ``{queue}`` attribute. -**-t, --time-limit=MINUTES|FSD** +.. option:: -t, --time-limit=MINUTES|FSD + Set a time limit for the job in either minutes or Flux standard duration (RFC 23). FSD is a floating point number with a single character units - suffix ("s", "m", "h", or "d"). The default unit for the ``--time-limit`` - option is minutes when no units are otherwise specified. If the time - limit is unspecified, the job is subject to the system default time limit. + suffix ("s", "m", "h", or "d"). The default unit for the + :option:`--time-limit` option is minutes when no units are otherwise + specified. If the time limit is unspecified, the job is subject to the + system default time limit. + +.. option:: --job-name=NAME -**--job-name=NAME** Set an alternate job name for the job. If not specified, the job name will default to the command or script executed for the job. -**--flags=FLAGS** +.. option:: --flags=FLAGS + Set comma separated list of job submission flags. The possible flags are ``waitable``, ``novalidate``, and ``debug``. The ``waitable`` flag will allow the job to be waited on via ``flux job wait`` and similar API calls. diff --git a/doc/man1/common/submit-other-options.rst b/doc/man1/common/submit-other-options.rst index c1a78fc4616f..6d1c1d0023c0 100644 --- a/doc/man1/common/submit-other-options.rst +++ b/doc/man1/common/submit-other-options.rst @@ -1,10 +1,12 @@ OTHER OPTIONS ============= -**--cwd=DIRECTORY** +.. option:: --cwd=DIRECTORY + Set job working directory. -**--urgency=N** +.. option:: --urgency=N + Specify job urgency, which affects queue order. Numerically higher urgency jobs are considered by the scheduler first. Guests may submit jobs with urgency in the range of 0 to 16, while instance owners may submit jobs @@ -12,20 +14,23 @@ OTHER OPTIONS numerical values, the special names ``hold`` (0), ``default`` (16), and ``expedite`` (31) are also accepted. -**-v, --verbose** +.. option:: -v, --verbose + *(run,alloc,submit,bulksubmit)* Increase verbosity on stderr. For example, - currently ``flux run -v`` displays jobid, ``-vv`` displays job events, - and ``-vvv`` displays exec events. ``flux alloc -v`` forces the command - to print the submitted jobid on stderr. - The specific output may change in the future. + currently :option:`flux run -v` displays jobid, :option:`-vv` displays job + events, and :option:`-vvv` displays exec events. :option:`flux alloc -v` + forces the command to print the submitted jobid on stderr. The specific + output may change in the future. + +.. option:: -o, --setopt=KEY[=VAL] -**-o, --setopt=KEY[=VAL]** Set shell option. Keys may include periods to denote hierarchy. VAL is optional and may be valid JSON (bare values, objects, or arrays), otherwise VAL is interpreted as a string. If VAL is not set, then the - default value is 1. See SHELL OPTIONS below. + default value is 1. See `SHELL OPTIONS`_ below. + +.. option:: --setattr=KEY[=VAL] -**--setattr=KEY[=VAL]** Set jobspec attribute. Keys may include periods to denote hierarchy. If KEY does not begin with ``system.``, ``user.``, or ``.``, then ``system.`` is assumed. VAL is optional and may be valid JSON (bare @@ -34,7 +39,8 @@ OTHER OPTIONS character, then VAL is interpreted as a file, which must be valid JSON, to use as the attribute value. -**--add-file=[NAME=]ARG** +.. option:: --add-file=[NAME=]ARG + Add a file to the RFC 37 file archive in jobspec before submission. Both the file metadata and content are stored in the archive, so modification or deletion of a file after being processed by this option will have no @@ -54,15 +60,16 @@ OTHER OPTIONS files, binaries, and directories, the :man1:`flux-shell` ``stage-in`` plugin will be more appropriate. -**--conf=FILE|KEY=VAL|STRING|NAME** - The ``--conf`` option allows configuration for a Flux instance started - via ``flux-batch(1)`` or ``flux-alloc(1)`` to be iteratively built on - the command line. On first use, a ``conf.json`` entry is added to the +.. option:: --conf=FILE|KEY=VAL|STRING|NAME + + The :option:`--conf`` option allows configuration for a Flux instance + started via ``flux-batch(1)`` or ``flux-alloc(1)`` to be iteratively built + on the command line. On first use, a ``conf.json`` entry is added to the internal jobspec file archive, and ``-c{{tmpdir}}/conf.json`` is added - to the flux broker command line. Each subsequent use of the ``--conf`` + to the flux broker command line. Each subsequent use of the :option:`--conf` option updates this configuration. - The argument to ``--conf`` may be in one of several forms: + The argument to :option:`--conf` may be in one of several forms: * A multiline string, e.g. from a batch directive. In this case the string is parsed as JSON or TOML:: @@ -91,7 +98,8 @@ OTHER OPTIONS ``XDG_CONFIG_DIRS`` is not set. Note that ``XDG_CONFIG_DIRS`` may be a colon-separated path. -**--begin-time=+FSD|DATETIME** +.. option:: --begin-time=+FSD|DATETIME + Convenience option for setting a ``begin-time`` dependency for a job. The job is guaranteed to start after the specified date and time. If argument begins with a ``+`` character, then the remainder is @@ -101,21 +109,23 @@ OTHER OPTIONS is accepted, e.g. ``2021-06-21 8am``, ``in an hour``, ``tomorrow morning``, etc. -**--signal=SIG@TIME** +.. option:: --signal=SIG@TIME + Send signal ``SIG`` to job ``TIME`` before the job time limit. ``SIG`` can specify either an integer signal number or a full or abbreviated signal name, e.g. ``SIGUSR1`` or ``USR1`` or ``10``. ``TIME`` is specified in Flux Standard Duration, e.g. ``30`` for 30s or ``1h`` for 1 hour. Either parameter may be omitted, with defaults of ``SIGUSR1`` - and 60s. For example, ``--signal=USR2`` will send ``SIGUSR2`` to - the job 60 seconds before expiration, and ``--signal=@3m`` will send + and 60s. For example, :option:`--signal=USR2` will send ``SIGUSR2`` to + the job 60 seconds before expiration, and :option:`--signal=@3m` will send ``SIGUSR1`` 3 minutes before expiration. Note that if ``TIME`` is greater than the remaining time of a job as it starts, the job will be signaled immediately. The default behavior is to not send any warning signal to jobs. -**--taskmap=SCHEME[:VALUE]** +.. option:: --taskmap=SCHEME[:VALUE] + Choose an alternate method for mapping job task IDs to nodes of the job. The job shell maps tasks using a "block" distribution scheme by default (consecutive tasks share nodes) This option allows the @@ -137,17 +147,20 @@ OTHER OPTIONS check the current job shell configuration for a full list of supported taskmap schemes. -**--dry-run** +.. option:: --dry-run + Don't actually submit job. Just emit jobspec on stdout and exit for ``run``, ``submit``, ``alloc``, and ``batch``. For ``bulksubmit``, emit a line of output including relevant options for each job which would have been submitted, -**--debug** +.. option:: --debug + Enable job debug events, primarily for debugging Flux itself. The specific effects of this option may change in the future. -**--bg** +.. option:: --bg + *(alloc only)* Do not interactively attach to the instance. Instead, print jobid on stdout once the instance is ready to accept jobs. The instance will run indefinitely until a time limit is reached, the @@ -157,33 +170,39 @@ OTHER OPTIONS interactively attach to the job (though it will print any errors or output). -**-B, --broker-opts=OPT** +.. option:: -B, --broker-opts=OPT + *(batch only)* For batch jobs, pass specified options to the Flux brokers of the new instance. This option may be specified multiple times. -**--wrap** - *(batch only)* The ``--wrap`` option wraps the specified COMMAND and ARGS in - a shell script, by prefixing with ``#!/bin/sh``. If no COMMAND is present, - then a SCRIPT is read on stdin and wrapped in a /bin/sh script. +.. option:: --wrap + + *(batch only)* The :option:`--wrap` option wraps the specified COMMAND and + ARGS in a shell script, by prefixing with ``#!/bin/sh``. If no COMMAND is + present, then a SCRIPT is read on stdin and wrapped in a /bin/sh script. + +.. option:: --cc=IDSET -**--cc=IDSET** *(submit,bulksubmit)* Replicate the job for each ``id`` in ``IDSET``. ``FLUX_JOB_CC=id`` will be set in the environment of each submitted job to allow the job to alter its execution based on the submission index. - (e.g. for reading from a different input file). When using ``--cc``, + (e.g. for reading from a different input file). When using :option:`--cc`, the substitution string ``{cc}`` may be used in options and commands and will be replaced by the current ``id``. -**--bcc=IDSET** - *(submit,bulksubmit)* Identical to ``--cc``, but do not set +.. option:: --bcc=IDSET + + *(submit,bulksubmit)* Identical to :option:`--cc`, but do not set ``FLUX_JOB_CC`` in each job. All jobs will be identical copies. - As with ``--cc``, ``{cc}`` in option arguments and commands will be + As with :option:`--cc`, ``{cc}`` in option arguments and commands will be replaced with the current ``id``. -**--quiet** +.. option:: --quiet + *(submit,bulksubmit)* Suppress logging of jobids to stdout. -**--log=FILE** +.. option:: --log=FILE + *(submit,bulksubmit)* Log command output and stderr to ``FILE`` instead of the terminal. If a replacement (e.g. ``{}`` or ``{cc}``) appears in ``FILE``, then one or more output files may be opened. @@ -191,53 +210,64 @@ OTHER OPTIONS flux submit --cc=1-4 --log=job{cc}.id hostname -**--log-stderr=FILE** +.. option:: --log-stderr=FILE + *(submit,bulksubmit)* Separate stderr into ``FILE`` instead of sending - it to the terminal or a ``FILE`` specified by ``--log``. + it to the terminal or a ``FILE`` specified by :option:`--log`. + +.. option:: --wait -**--wait** *(submit,bulksubmit)* Wait on completion of all jobs before exiting. - This is equivalent to ``--wait-event=clean``. + This is equivalent to :option:`--wait-event=clean`. + +.. option:: --wait-event=NAME -**--wait-event=NAME** *(run,submit,bulksubmit)* Wait until job or jobs have received event ``NAME`` before exiting. E.g. to submit a job and block until the job begins - running, use ``--wait-event=start``. *(submit,bulksubmit only)* If ``NAME`` - begins with ``exec.``, then wait for an event in the exec eventlog, e.g. - ``exec.shell.init``. For ``flux run`` the argument to this option + running, use :option:`--wait-event=start`. *(submit,bulksubmit only)* If + ``NAME`` begins with ``exec.``, then wait for an event in the exec eventlog, + e.g. ``exec.shell.init``. For ``flux run`` the argument to this option when used is passed directly to ``flux job attach``. -**--watch** - *(submit,bulksubmit)* Display output from all jobs. Implies ``--wait``. +.. option:: --watch -**--progress** - *(submit,bulksubmit)* With ``--wait``, display a progress bar showing - the progress of job completion. Without ``--wait``, the progress bar + *(submit,bulksubmit)* Display output from all jobs. Implies :option:`--wait`. + +.. option:: --progress + + *(submit,bulksubmit)* With :option:`--wait`, display a progress bar showing + the progress of job completion. Without :option:`--wait`, the progress bar will show progress of job submission. -**--jps** - *(submit,bulksubmit)* With ``--progress``, display throughput statistics - (jobs/s) in the progress bar. +.. option:: --jps + + *(submit,bulksubmit)* With :option:`--progress`, display throughput + statistics (jobs/s) in the progress bar. + +.. option:: --define=NAME=CODE -**--define=NAME=CODE** *(bulksubmit)* Define a named method that will be made available as an attribute during command and option replacement. The string being processed is available as ``x``. For example:: $ seq 1 8 | flux bulksubmit --define=pow="2**int(x)" -n {.pow} ... -**--shuffle** +.. option:: --shuffle + *(bulksubmit)* Shuffle the list of commands before submission. -**--sep=STRING** +.. option:: --sep=STRING + *(bulksubmit)* Change the separator for file input. The default is to separate files (including stdin) by newline. To separate by - consecutive whitespace, specify ``--sep=none``. + consecutive whitespace, specify :option:`--sep=none`. + +.. option:: --dump=[FILE] -**--dump=[FILE]** *(batch,alloc)* When the job script is complete, archive the Flux instance's KVS content to ``FILE``, which should have a suffix known to :linux:man3:`libarchive`, and may be a mustache template as described - above for ``--output``. The content may be unarchived directly or examined - within a test instance started with the :man1:`flux-start` ``--recovery`` - option. If ``FILE`` is unspecified, ``flux-{{jobid}}-dump.tgz`` is used. + above for :option:`--output`. The content may be unarchived directly or + examined within a test instance started with the + :option:`flux-start --recovery` option. If ``FILE`` is unspecified, + ``flux-{{jobid}}-dump.tgz`` is used. diff --git a/doc/man1/common/submit-process-resource-limits.rst b/doc/man1/common/submit-process-resource-limits.rst index 867d203d1af5..801d1237f38b 100644 --- a/doc/man1/common/submit-process-resource-limits.rst +++ b/doc/man1/common/submit-process-resource-limits.rst @@ -4,14 +4,15 @@ PROCESS RESOURCE LIMITS By default these commands propagate some common resource limits (as described in :linux:man2:`getrlimit`) to the job by setting the ``rlimit`` job shell option in jobspec. The set of resource limits propagated can be controlled -via the ``--rlimit=RULE`` option: +via the :option:`--rlimit=RULE` option: + +.. option:: --rlimit=RULE -**--rlimit=RULE** Control how process resource limits are propagated with *RULE*. Rules are applied in the order in which they are used on the command line. This option may be used multiple times. -The ``--rlimit`` rules work similar to the ``--env`` option rules: +The :option:`--rlimit` rules work similar to the :option:`--env` option rules: * If a rule begins with ``-``, then the rest of the rule is a name or :linux:man7:`glob` pattern which removes matching resource limits from @@ -36,13 +37,13 @@ The ``--rlimit`` rules work similar to the ``--env`` option rules: that are propagated by default). We start with a default list of resource limits to propagate, -then applies all rules specified via ``--rlimit`` on the command line. +then applies all rules specified via :option:`--rlimit` on the command line. Therefore, to propagate only one limit, ``-*`` should first be used to -start with an empty set, e.g. ``--rlimit=-*,core`` will only propagate the -``core`` resource limit. +start with an empty set, e.g. :option:`--rlimit=-*,core` will only propagate +the ``core`` resource limit. The set of resource limits propagated by default includes all those except ``memlock``, ``ofile``, ``msgqueue``, ``nice``, ``rtprio``, ``rttime``, and ``sigpending``. To propagate all possible resource limits, use -``--rlimit=*``. +:option:`--rlimit=*`. diff --git a/doc/man1/common/submit-shell-options.rst b/doc/man1/common/submit-shell-options.rst index 852c2b2a4ded..d322c362d886 100644 --- a/doc/man1/common/submit-shell-options.rst +++ b/doc/man1/common/submit-shell-options.rst @@ -4,39 +4,48 @@ SHELL OPTIONS These options are provided by built-in shell plugins that may be overridden in some cases: -**mpi=spectrum** +.. option:: -o mpi=spectrum + Load the MPI personality plugin for IBM Spectrum MPI. All other MPI plugins are loaded by default. -**cpu-affinity=per-task** +.. option:: -o cpu-affinity=per-task + Tasks are distributed across the assigned resources. -**cpu-affinity=off** +.. option:: -o cpu-affinity=off + Disable task affinity plugin. -**gpu-affinity=per-task** +.. option:: -o gpu-affinity=per-task + GPU devices are distributed evenly among local tasks. Otherwise, GPU device affinity is to the job. -**gpu-affinity=off** +.. option:: -o gpu-affinity=off + Disable GPU affinity for this job. -**verbose** +.. option:: -o verbose + Increase verbosity of the job shell log. -**nosetpgrp** +.. option:: -o nosetpgrp + Normally the job shell runs each task in its own process group to facilitate delivering signals to tasks which may call :linux:man2:`fork`. With this option, the shell avoids calling :linux:man2:`setpgrp`, and each task will run in the process group of the shell. This will cause signals to be delivered only to direct children of the shell. -**pmi=off** +.. option:: -o pmi=off + Disable the process management interface (PMI-1) which is required for bootstrapping most parallel program environments. See :man1:`flux-shell` for more pmi options. -**stage-in** +.. option:: -o stage-in + Copy files previously mapped with :man1:`flux-filemap` to $FLUX_JOB_TMPDIR. See :man1:`flux-shell` for more *stage-in* options. diff --git a/doc/man1/common/submit-standard-io.rst b/doc/man1/common/submit-standard-io.rst index e8948b3da6fc..acd75f9e0124 100644 --- a/doc/man1/common/submit-standard-io.rst +++ b/doc/man1/common/submit-standard-io.rst @@ -7,12 +7,14 @@ KVS, where they may be accessed with the ``flux job attach`` command. In addition, :man1:`flux-run` processes standard I/O in real time, emitting the job's I/O to its stdout and stderr. -**--input=FILENAME|RANKS** +.. option:: --input=FILENAME|RANKS + Redirect stdin to the specified filename, bypassing the KVS. As a special case for ``flux run``, the argument may specify an idset of task ranks in to which to direct standard input. -**--output=TEMPLATE** +.. option:: --output=TEMPLATE + Specify the filename *TEMPLATE* for stdout redirection, bypassing the KVS. *TEMPLATE* may be a mustache template which supports the following tags: @@ -31,20 +33,22 @@ emitting the job's I/O to its stdout and stderr. To force output to KVS so it is available with ``flux job attach``, set *TEMPLATE* to *none* or *kvs*. -**--error=TEMPLATE** +.. option:: --error=TEMPLATE + Redirect stderr to the specified filename *TEMPLATE*, bypassing the KVS. *TEMPLATE* is expanded as described above. -**-u, --unbuffered** +.. option:: -u, --unbuffered + Disable buffering of standard input and output as much as practical. Normally, stdout from job tasks is line buffered, as is stdin when running a job in the foreground via :man1:`flux-run`. Additionally, job output may experience a delay due to batching of output - events by the job shell. With the ``--unbuffered`` option, + events by the job shell. With the :option:`--unbuffered` option, ``output.*.buffer.type=none`` is set in jobspec to request no buffering of output, and the default output batch period is reduced greatly, to make output appear in the KVS and printed to the standard output - of :man1:`flux-run` as soon as possible. The ``--unbuffered`` option + of :man1:`flux-run` as soon as possible. The :option:`--unbuffered` option is also passed to ``flux job attach``, which makes stdin likewise unbuffered. Note that the application and/or terminal may have additional input and output buffering which this option will not @@ -53,6 +57,7 @@ emitting the job's I/O to its stdout and stderr. process run without a terminal may be fully buffered when using libc standard I/O streams (See NOTES in :linux:man3:`stdout`). -**-l, --label-io** +.. option:: -l, --label-io + Add task rank prefixes to each line of output. diff --git a/doc/man1/common/submit-submission-directives.rst b/doc/man1/common/submit-submission-directives.rst deleted file mode 100644 index c600b4e7a26f..000000000000 --- a/doc/man1/common/submit-submission-directives.rst +++ /dev/null @@ -1,51 +0,0 @@ - -.. _submission_directives: - -SUBMISSION DIRECTIVES -===================== - -The :man1:`flux-batch` command supports submission directives -mixed within the submission script. The submission directive specification -is fully detailed in RFC 36, but is summarized here for convenience: - - * A submission directive is indicated by a line that starts with - a prefix of non-alphanumeric characters followed by a tag ``FLUX:`` or - ``flux:``. The prefix plus tag is called the *directive sentinel*. E.g., - in the example below the sentinel is ``# flux:``: :: - - #!/bin/sh - # flux: -N4 -n16 - flux run -n16 hostname - - * All directives in a file must use the same sentinel pattern, otherwise - an error will be raised. - * Directives must be grouped together - it is an error to include a - directive after any non-blank line that doesn't start with the common - prefix. - * The directive starts after the sentinel to the end of the line. - * The ``#`` character is supported as a comment character in directives. - * UNIX shell quoting is supported in directives. - * Triple quoted strings can be used to include newlines and quotes without - further escaping. If a triple quoted string is used across multiple lines, - then the opening and closing triple quotes must appear at the end of the - line. For example :: - - # flux: --setattr=user.conf=""" - # flux: [config] - # flux: item = "foo" - # flux: """ - -Submission directives may be used to set default command line options for -:man1:`flux-batch` for a given script. Options given on the command line -override those in the submission script, e.g.: :: - - $ flux batch --job-name=test-name --wrap <<-EOF - > #flux: -N4 - > #flux: --job-name=name - > flux run -N4 hostname - > EOF - ƒ112345 - $ flux jobs -no {name} ƒ112345 - test-name - - diff --git a/doc/man1/flux-alloc.rst b/doc/man1/flux-alloc.rst index 3768db96c652..d40b62aa4e5f 100644 --- a/doc/man1/flux-alloc.rst +++ b/doc/man1/flux-alloc.rst @@ -5,7 +5,6 @@ flux-alloc(1) ============= - SYNOPSIS ======== @@ -14,29 +13,31 @@ SYNOPSIS DESCRIPTION =========== -**flux-alloc** runs a Flux subinstance with *COMMAND* as the initial program. -Once resources are allocated, *COMMAND* executes on the first node of the -allocation with any free arguments supplied as *COMMAND* arguments. When +.. program:: flux alloc + +:program:`flux alloc` runs a Flux subinstance with *COMMAND* as the initial +program. Once resources are allocated, *COMMAND* executes on the first node of +the allocation with any free arguments supplied as *COMMAND* arguments. When *COMMAND* exits, the Flux subinstance exits, resources are released to the -enclosing Flux instance, and **flux-alloc** returns. +enclosing Flux instance, and :program:`flux alloc` returns. If no *COMMAND* is specified, an interactive shell is spawned as the initial program, and the subinstance runs until the shell is exited. -If the *--bg* option is specified, the subinstance runs without an initial -program. **flux-alloc** prints the jobid and returns as soon as the -subinstance is ready to accept jobs. The subinstance runs until it exceeds +If the :option:`--bg` option is specified, the subinstance runs without an +initial program. :program:`flux alloc` prints the jobid and returns as soon as +the subinstance is ready to accept jobs. The subinstance runs until it exceeds its time limit, is canceled, or is shut down with :man1:`flux-shutdown`. Flux commands that are run from the subinstance (e.g. from the interactive shell) refer to the subinstance. For example, :man1:`flux-run` would launch work there. A Flux command run from the subinstance can be forced to refer -to the enclosing instance by supplying the :man1:`flux` --parent option. +to the enclosing instance by supplying the :option:`flux --parent` option. Flux commands outside of the subinstance refer to their enclosing instance, -often a system instance. flux-proxy(1) establishes a connection to a running -subinstance by jobid, then spawns a shell in which Flux commands refer to -the subinstance, for example +often a system instance. :man1:`flux-proxy` establishes a connection to a +running subinstance by jobid, then spawns a shell in which Flux commands +refer to the subinstance, for example :: diff --git a/doc/man1/flux-batch.rst b/doc/man1/flux-batch.rst index a512b93d0be9..e439b926e519 100644 --- a/doc/man1/flux-batch.rst +++ b/doc/man1/flux-batch.rst @@ -17,7 +17,9 @@ SYNOPSIS DESCRIPTION =========== -**flux-batch** submits *SCRIPT* to run as the initial program of a Flux +.. program:: flux batch + +:program:`flux-batch` submits *SCRIPT* to run as the initial program of a Flux subinstance. *SCRIPT* refers to a file that is copied at the time of submission. Once resources are allocated, *SCRIPT* executes on the first node of the allocation, with any remaining free arguments supplied as *SCRIPT* @@ -26,8 +28,8 @@ released to the enclosing Flux instance. If there are no free arguments, the script is read from standard input. -If the *--wrap* option is used, the script is created by wrapping the free -arguments or standard input in a shell script prefixed with ``#!/bin/sh``. +If the :option:`--wrap` option is used, the script is created by wrapping the +free arguments or standard input in a shell script prefixed with ``#!/bin/sh``. If the job request is accepted, its jobid is printed on standard output and the command returns. The job runs when the Flux scheduler fulfills its resource @@ -36,7 +38,7 @@ allocation request. :man1:`flux-jobs` may be used to display the job status. Flux commands that are run from the batch script refer to the subinstance. For example, :man1:`flux-run` would launch work there. A Flux command run from the script can be forced to refer to the enclosing instance by supplying -the :man1:`flux` *--parent* option. +the :option:`flux --parent` option. Flux commands outside of the batch script refer to their enclosing instance, often a system instance. :man1:`flux-proxy` establishes a connection to a @@ -69,7 +71,7 @@ automatically. For example: $ Batch scripts may contain submission directives denoted by ``flux:`` -as described in RFC 36. See :ref:`submission_directives` below. +as described in RFC 36. See `SUBMISSION DIRECTIVES`_ below. The available OPTIONS are detailed below. @@ -91,7 +93,53 @@ The available OPTIONS are detailed below. .. include:: common/submit-shell-options.rst -.. include:: common/submit-submission-directives.rst +SUBMISSION DIRECTIVES +===================== + +The :program:`flux batch` command supports submission directives +mixed within the submission script. The submission directive specification +is fully detailed in RFC 36, but is summarized here for convenience: + + * A submission directive is indicated by a line that starts with + a prefix of non-alphanumeric characters followed by a tag ``FLUX:`` or + ``flux:``. The prefix plus tag is called the *directive sentinel*. E.g., + in the example below the sentinel is ``# flux:``: :: + + #!/bin/sh + # flux: -N4 -n16 + flux run -n16 hostname + + * All directives in a file must use the same sentinel pattern, otherwise + an error will be raised. + * Directives must be grouped together - it is an error to include a + directive after any non-blank line that doesn't start with the common + prefix. + * The directive starts after the sentinel to the end of the line. + * The ``#`` character is supported as a comment character in directives. + * UNIX shell quoting is supported in directives. + * Triple quoted strings can be used to include newlines and quotes without + further escaping. If a triple quoted string is used across multiple lines, + then the opening and closing triple quotes must appear at the end of the + line. For example :: + + # flux: --setattr=user.conf=""" + # flux: [config] + # flux: item = "foo" + # flux: """ + +Submission directives may be used to set default command line options for +:program:`flux batch` for a given script. Options given on the command line +override those in the submission script, e.g.: :: + + $ flux batch --job-name=test-name --wrap <<-EOF + > #flux: -N4 + > #flux: --job-name=name + > flux run -N4 hostname + > EOF + ƒ112345 + $ flux jobs -no {name} ƒ112345 + test-name + RESOURCES diff --git a/doc/man1/flux-broker.rst b/doc/man1/flux-broker.rst index 9d068d3fef80..973728d08a1a 100644 --- a/doc/man1/flux-broker.rst +++ b/doc/man1/flux-broker.rst @@ -11,7 +11,9 @@ SYNOPSIS DESCRIPTION =========== -flux-broker(1) is a distributed message broker daemon that provides +.. program:: flux broker + +:program:`flux broker` is a distributed message broker daemon that provides communications services within a Flux instance. It may be launched as a parallel program under Flux or other resource managers that support PMI. @@ -28,7 +30,7 @@ using Flux API services. A logging service aggregates Flux log messages across the instance and emits them to a configured destination on rank 0. -After its overlay network has completed wire-up, flux-broker(1) +After its overlay network has completed wire-up, :program:`flux broker` starts the initial program on rank 0. If none is specified on the broker command line, an interactive shell is launched. @@ -36,16 +38,20 @@ the broker command line, an interactive shell is launched. OPTIONS ======= -**-h, --help** +.. option:: -h, --help + Summarize available options. -**-v, --verbose** +.. option:: -v, --verbose + Be annoyingly chatty. -**-S, --setattr**\ =\ *ATTR=VAL* +.. option:: -S, --setattr=ATTR=VAL + Set initial value for broker attribute. -**-c, --config-path=**\ =\ *PATH* +.. option:: -c, --config-path=PATH + Set the PATH to broker configuration. If PATH is a directory, then read all TOML files from that directory. If PATH is a file, then load configuration as JSON if the file extension is ``.json``, otherwise diff --git a/doc/man1/flux-bulksubmit.rst b/doc/man1/flux-bulksubmit.rst index e03fe38cbbec..9063306c6f9e 100644 --- a/doc/man1/flux-bulksubmit.rst +++ b/doc/man1/flux-bulksubmit.rst @@ -15,7 +15,106 @@ SYNOPSIS DESCRIPTION =========== -.. include:: common/submit-bulksubmit.rst +.. program:: flux bulksubmit + +:program:`flux bulksubmit` allows rapid bulk submission of jobs using +an interface similar to GNU parallel or ``xargs``. The command takes +inputs on stdin or the command line (separated by ``:::``), and submits +the supplied command template and options as one job per input combination. + +The replacement is done using Python's ``string.format()``, which is +supplied a list of inputs on each iteration. Therefore, in the common case +of a single input list, ``{}`` will work as the substitution string, e.g.:: + + $ seq 1 4 | flux bulksubmit echo {} + bulksubmit: submit echo 1 + bulksubmit: submit echo 2 + bulksubmit: submit echo 3 + bulksubmit: submit echo 4 + +With :option:`--dry-run` :program:`flux bulksubmit` will print the args and +command which would have been submitted, but will not perform any job +submission. + +The :program:`flux bulksubmit` command can also take input lists on the command +line. The inputs are separated from each other and the command with the +special delimiter ``:::``:: + + $ flux bulksubmit echo {} ::: 1 2 3 4 + bulksubmit: submit echo 1 + bulksubmit: submit echo 2 + bulksubmit: submit echo 3 + bulksubmit: submit echo 4 + +Multiple inputs are combined, in which case each input is passed as a +positional parameter to the underlying ``format()``, so should be accessed +by index:: + + $ flux bulksubmit --dry-run echo {1} {0} ::: 1 2 ::: 3 4 + bulksubmit: submit echo 3 1 + bulksubmit: submit echo 4 1 + bulksubmit: submit echo 3 2 + bulksubmit: submit echo 4 2 + +If the generation of all combinations of an input list with other inputs is not +desired, the special input delimited ``:::+`` may be used to "link" the input, +so that only one argument from this source will be used per other input, +e.g.:: + + $ flux bulksubmit --dry-run echo {0} {1} ::: 1 2 :::+ 3 4 + bulksubmit: submit 1 3 + bulksubmit: submit 2 4 + +The linked input will be cycled through if it is shorter than other inputs. + +An input list can be read from a file with ``::::``:: + + $ seq 0 3 >inputs + $ flux bulksubmit --dry-run :::: inputs + bulksubmit: submit 0 + bulksubmit: submit 1 + bulksubmit: submit 2 + bulksubmit: submit 3 + +If the filename is ``-`` then ``stdin`` will be used. This is useful +for including ``stdin`` when reading other inputs. + +The delimiter ``::::+`` indicates that the next file is to be linked to +the inputs instead of combined with them, as with ``:::+``. + +There are several predefined attributes for input substitution. +These include: + + - ``{.%}`` returns the input string with any extension removed. + - ``{./}`` returns the basename of the input string. + - ``{./%}`` returns the basename of the input string with any + extension removed. + - ``{.//}`` returns the dirname of the input string + - ``{seq}`` returns the input sequence number (0 origin) + - ``{seq1}`` returns the input sequence number (1 origin) + - ``{cc}`` returns the current ``id`` from use of :option:`--cc` or + :option:`--bcc`. Note that replacement of ``{cc}`` is done in a second + pass, since the :option:`--cc` option argument may itself be replaced in + the first substitution pass. If :option:`--cc`/:option:`--bcc` were not + used, then ``{cc}`` is replaced with an empty string. This is the only + substitution supported with :man1:`flux-submit`. + +Note that besides ``{seq}``, ``{seq1}``, and ``{cc}`` these attributes +can also take the input index, e.g. ``{0.%}`` or ``{1.//}``, when multiple +inputs are used. + +Additional attributes may be defined with the :option:`--define` option, e.g.:: + + $ flux bulksubmit --dry-run --define=p2='2**int(x)' -n {.p2} hostname \ + ::: $(seq 0 4) + bulksubmit: submit -n1 hostname + bulksubmit: submit -n2 hostname + bulksubmit: submit -n4 hostname + bulksubmit: submit -n8 hostname + bulksubmit: submit -n16 hostname + +The input string being indexed is passed to defined attributes via the +local ``x`` as seen above. The available OPTIONS are detailed below. diff --git a/doc/man1/flux-cancel.rst b/doc/man1/flux-cancel.rst index 9cd1928e1141..1b68569d03ed 100644 --- a/doc/man1/flux-cancel.rst +++ b/doc/man1/flux-cancel.rst @@ -14,39 +14,46 @@ SYNOPSIS DESCRIPTION =========== -flux-cancel(1) cancels one or more jobs by raising a job exception of +.. program:: flux cancel + +:program:`flux cancel` cancels one or more jobs by raising a job exception of type=cancel. An optional message included with the cancel exception may be -provided via the *-m, --message=NOTE* option. Canceled jobs are immediately +provided via the :option:`--message` option. Canceled jobs are immediately sent SIGTERM followed by SIGKILL after a configurable timeout (default=5s). -flux-cancel(1) can target multiple jobids by either taking them on the -command line, or via the selection options *--all*, *-u, --user*, or *-S, ---states=STATES*. It is an error to provide jobids on the command line +:program:`flux cancel` can target multiple jobids by either taking them on the +command line, or via the selection options :option:`--all`, :option:`--user`, +or :option:`--states`. It is an error to provide jobids on the command line and use one or more of the selection options. -By default *--all* will target all jobs for the current user. To target all -jobs for all users, use *--user=all* (only the instance owner is allowed -to use *--user=all*). To see how many jobs flux-cancel(1) would kill, -use the *-n --dry-run* option. +By default :option:`--all` will target all jobs for the current user. To +target all jobs for all users, use :option:`--user=all` (only the instance +owner is allowed to use :option:`--user=all`). To see how many jobs +:program:`flux cancel` would kill, use the :option:`--dry-run` option. OPTIONS ======= -**-n, --dry-run** +.. option:: -n, --dry-run + Do not cancel any jobs, but print a message indicating how many jobs would have been canceled. -**-m, --message=NOTE** +.. option:: -m, --message=NOTE + Set an optional exception note. -**-u, --user=USER** +.. option:: -u, --user=USER + Set target user. The instance owner may specify *all* for all users. -**-S, --states=STATES** +.. option:: -S, --states=STATES + Set target job states (default: active). Valid states include depend, priority, sched, run, pending, running, active. -**-q, --quiet** +.. option:: -q, --quiet + Suppress output if no jobs match RESOURCES diff --git a/doc/man1/flux-config.rst b/doc/man1/flux-config.rst index c2ef46669428..b8e7269ec72d 100644 --- a/doc/man1/flux-config.rst +++ b/doc/man1/flux-config.rst @@ -20,53 +20,62 @@ SYNOPSIS DESCRIPTION =========== -``flux config reload`` tells :man1:`flux-broker` to reload its TOML +.. program:: flux config get + +:program:`flux config reload` tells :man1:`flux-broker` to reload its TOML configuration. Further details of Flux configuration are described in :man5:`flux-config`, including some caveats on configuration updates. -On Flux instances started with :linux:man1:`systemd`, ``systemctl reload flux`` -invokes this command. This command is restricted to the instance owner. - -``flux config get`` queries the TOML configuration for a given Flux broker. -if *NAME* is unspecified, it dumps the entire configuration object. Otherwise, -*NAME* is expected to be a period-delimited path name representing a TOML key. -Return values are printed in string-encoded JSON form, except for string values, -which are printed without quotes to simplify their use in shell scripts. - -``flux config builtin`` prints compiled-in Flux configuration values. -See BUILTIN VALUES below for a list of builtin +On Flux instances started with :linux:man1:`systemd`, +:program:`systemctl reload flux` invokes this command. This command is +restricted to the instance owner. + +:program:`flux config get` queries the TOML configuration for a given Flux +broker. if *NAME* is unspecified, it dumps the entire configuration object. +Otherwise, *NAME* is expected to be a period-delimited path name representing +a TOML key. Return values are printed in string-encoded JSON form, except for +string values, which are printed without quotes to simplify their use in shell +scripts. + +:program:`flux config builtin` prints compiled-in Flux configuration values. +See `BUILTIN VALUES`_ below for a list of builtin configuration key names. This command is available to all users. .. note:: - ``flux config get`` and ``flux config builtin`` refer to disjoint key - namespaces. Flux behavior is determined by a combination of these values, - :man7:`flux-broker-attributes`, and other factors. This disjoint - configuration scheme is subject to change in future releases of Flux. + :program:`flux config get` and :program:`flux config builtin` refer to + disjoint key namespaces. Flux behavior is determined by a combination of + these values, :man7:`flux-broker-attributes`, and other factors. This + disjoint configuration scheme is subject to change in future releases of + Flux. .. note:: - ``flux config builtin`` uses a heuristic to determine if :man1:`flux` + :program:`flux config builtin` uses a heuristic to determine if :man1:`flux` was run from the flux-core source tree, and substitutes source tree specific values if found to be in tree. This enables Flux testing without requiring installation. -``flux config load`` replaces the current config with an object read from -standard input (JSON or TOML), or from ``*.toml`` in *PATH*, if specified. +:program:`flux config load` replaces the current config with an object read +from standard input (JSON or TOML), or from ``*.toml`` in *PATH*, if specified. GET SUBCOMMAND OPTIONS ====================== -**-h, --help** +.. option:: -h, --help + Display subcommand help. -**-d, --default**\ =\ *VALUE* +.. option:: -d, --default=VALUE + Substitute *VALUE* if *NAME* is not set in the configuration, and exit with a return code of zero. -**-q, --quiet** - Suppress printing of errors if *NAME* is not set and *--default* was not - specified. This may be convenient to avoid needing to redirect standard +.. option:: -q, --quiet + + Suppress printing of errors if *NAME* is not set and :option:`--default` was + not specified. This may be convenient to avoid needing to redirect standard error in a shell script. -**-t, --type**\ =\ *TYPE* +.. option:: -t, --type=TYPE + Require that the value has the specified type, or exit with a nonzero exit code. Valid types are *string*, *integer*, *real*, *boolean*, *object*, and *array*. In addition, types of *fsd*, *fsd-integer*, and *fsd-real* ensure @@ -78,7 +87,8 @@ GET SUBCOMMAND OPTIONS BUILTIN VALUES ============== -The following configuration keys may be printed with ``flux config builtin``: +The following configuration keys may be printed with +:program:`flux config builtin`: **rc1_path** The rc1 script path used by :man1:`flux-broker`, unless overridden by diff --git a/doc/man1/flux-content.rst b/doc/man1/flux-content.rst index 6441e64f4a67..20569eb74312 100644 --- a/doc/man1/flux-content.rst +++ b/doc/man1/flux-content.rst @@ -17,25 +17,27 @@ SYNOPSIS DESCRIPTION =========== +.. program:: flux content load + Each Flux instance implements an append-only, content addressable storage service, which stores blobs of arbitrary content under message digest keys termed "blobrefs". -**flux content store** reads data from standard input to EOF, stores it +:program:`flux content store` reads data from standard input to EOF, stores it (possibly splitting into multiple blobs), and prints blobref(s) on standard output, one per line. -**flux content load** reads blobrefs from standard input, one per line, or -parses blobrefs on the command line (but not both). It then loads the +:program:`flux content load` reads blobrefs from standard input, one per line, +or parses blobrefs on the command line (but not both). It then loads the corresponding blob(s), and concatenates them on standard output. After a store operation completes on any rank, the blobs may be retrieved from any other rank. The content service includes a cache on each broker which improves -scalability. The **flux content flush** command initiates store requests +scalability. The :program:`flux content flush` command initiates store requests for any dirty entries in the local cache and waits for them to complete. -This is mainly used in testing. The **flux content dropcache** command +This is mainly used in testing. The :program:`flux content dropcache` command drops all non-essential entries in the local cache; that is, entries which can be removed without data loss. @@ -45,11 +47,13 @@ These operations are only available to the Flux instance owner. OPTIONS ======= -**-b, --bypass-cache** +.. option:: -b, --bypass-cache + Bypass the in-memory cache, and directly access the backing store, if available (see below). -**--chunksize**\ =\ *N* +.. option:: --chunksize=N + Split a blob into chunks of *N* bytes (store only). diff --git a/doc/man1/flux-cron.rst b/doc/man1/flux-cron.rst index 09334ab7b693..9cd756c3c2f6 100644 --- a/doc/man1/flux-cron.rst +++ b/doc/man1/flux-cron.rst @@ -21,7 +21,7 @@ reached. As with :man1:`flux-exec`, these tasks run as direct children of the flux-broker and run outside of the control of any loaded job scheduling service. -The flux-cron(1) utility offers an interface to create, stop, start, +The :program:`flux cron` utility offers an interface to create, stop, start, query, and destroy these entries in the Flux cron service. For a detailed description of the cron service operation and how @@ -38,12 +38,12 @@ COMMANDS **sync** [--epsilon=\ *delay*] [*topic*] Query and modify the current **sync-event** behavior for the cron module. If a sync-event is set, the cron module will defer all task execution - until an event matching the sync-event *topic* is received. With *--epsilon* - the cron module will **not** delay task execution if the task is normally - scheduled to run within *delay* of the matching event. Without any - *topic* supplied on command line, *flux cron sync* displays the current - setting for sync. If a task is deferred due to sync-event, the - *stats.deferred* statistic is incremented. + until an event matching the sync-event *topic* is received. With + :option:`--epsilon` the cron module will **not** delay task execution if + the task is normally scheduled to run within *delay* of the matching event. + Without any *topic* supplied on command line, :program:`flux cron sync` + displays the current setting for sync. If a task is deferred due to + sync-event, the *stats.deferred* statistic is incremented. **interval** [OPTIONS] *interval* *command* Create a cron entry to execute *command* every *interval*, where *interval* @@ -51,128 +51,162 @@ COMMANDS seconds, *m* for minutes, *h* for hours and *d* for days. Options: - --name=\ *STRING*; -n *STRING* +.. program:: flux cron interval + +.. option:: -n, --name=STRING + Set a name for this cron entry to *STRING*. - --after=\ *TIME*; -a *TIME* +.. option:: -a, --after=TIME + The first task will run after a delay of *TIME* instead of *interval*. After the first task the entry will continue to execute every *interval*. - --count=\ *N*; -c *N* +.. option:: -c, --count=N + The entry will be run a total of *N* times, then stopped. - --options=\ *LIST*; -o *LIST* - The *--options* option allows a comma separated list of extra options to be - passed to the flux-cron service. See EXTRA OPTIONS below. +.. option:: -o, --options=LIST + + The :option:`--options` option allows a comma separated list of extra + options to be passed to the flux-cron service. See EXTRA OPTIONS below. + +.. option:: -E, --preserve-env + + The :option:`--preserve-env` option allows the current environment to be + exported and used for the command being executed as part of the cron job. + Normally, the broker environment is used. - --preserve-env; -E - The *--preserve-env* option allows the current environment to be exported - and used for the command being executed as part of the cron job. Normally, - the broker environment is used. +.. option:: -d, --working-dir=DIR - --working-dir=\ *DIR*; -d *DIR* - The *--working-dir* option allows the working directory to be set for the command - being executed as part of the cron job. Normally, the working directory of - the broker is used. + The :option:`--working-dir` option allows the working directory to be set + for the command being executed as part of the cron job. Normally, the + working directory of the broker is used. **event** [OPTIONS] *topic* *command* Create a cron entry to execute *command* after every event matching *topic*. - --name=\ *STRING*; -n *STRING* +.. program:: flux cron event + +.. option:: -n, --name=STRING + Set a name for this cron entry to *STRING*. - --nth=\ *N*; -n *N* - If *--nth* is given then *command* will be run after each *N* events. +.. option:: -n, --nth=N + + If :option:`--nth` is given then *command* will be run after each *N* + events. + +.. option:: -c, --count=N - --count=\ *N*; -c *N* - With *--count*, the entry is run *N* times then stopped. + With :option:`--count`, the entry is run *N* times then stopped. + +.. option:: -a, --after=N - --after=\ *N*; -a *N* Run the first task only after *N* matching events. Then run every event - or *N* events with *--nth*. + or *N* events with :option:`--nth`. + +.. option:: -i, --min-interval=T - --min-interval=\ *T*; -i *T* Set the minimum interval at which two cron jobs for this event will be run. - For example, with --min-interval of 1s, the cron job will be at most run - every 1s, even if events are generated more quickly. + For example, with :option:`--min-interval` of 1s, the cron job will be + at most run every 1s, even if events are generated more quickly. + +.. option:: -o, --options=LIST - --options=\ *LIST*; -o *LIST* Set comma separated EXTRA OPTIONS for this cron entry. - --preserve-env; -E - The *--preserve-env* option allows the current environment to be exported - and used for the command being executed as part of the cron job. Normally, - the broker environment is used. +.. option:: -E, --preserve-env - --working-dir=\ *DIR*; -d *DIR* - The *--working-dir* option allows the working directory to be set for the command - being executed as part of the cron job. Normally, the working directory of - the broker is used. + The :option:`--preserve-env` option allows the current environment to be + exported and used for the command being executed as part of the cron job. + Normally, the broker environment is used. + +.. option:: -d, --working-dir=DIR + + The :option:`--working-dir` option allows the working directory to be + set for the command being executed as part of the cron job. Normally, + the working directory of the broker is used. **tab** [OPTIONS] [*file*] Process one or more lines containing crontab expressions from *file* (stdin by default) Each valid crontab line will result in a new cron entry registered with the flux-cron service. The cron expression format - supported by ``flux cron tab`` has 5 fields: *minutes* (0-59), *hours* + supported by :program:`flux cron tab` has 5 fields: *minutes* (0-59), *hours* (0-23), *day of month* (1-31), *month* (0-11), and *day of week* (0-6). Everything after the day of week is considered a command to be run. - --options=\ *LIST*; -o *LIST* +.. program:: flux cron tab + +.. option:: -o, options=LIST + Set comma separated EXTRA OPTIONS for all cron entries. **at** [OPTIONS] *string* *command* -Run *command* at specific date and time described by *string* + Run *command* at specific date and time described by *string* + +.. program:: flux cron at + +.. option:: -o, --options=LIST ---options=\ *LIST*; -o *LIST* Set comma separated EXTRA OPTIONS for all cron entries. ---preserve-env; -E - The *--preserve-env* option allows the current environment to be exported - and used for the command being executed as part of the cron job. Normally, - the broker environment is used. +.. option:: -E, --preserve-env + + The :option:`--preserve-env` option allows the current environment to be + exported and used for the command being executed as part of the cron job. + Normally, the broker environment is used. ---working-dir=\ *DIR*; -d *DIR* - The *--working-dir* option allows the working directory to be set for the command - being executed as part of the cron job. Normally, the working directory of - the broker is used. +.. option:: -d, --working-dir=DIR - **list** - Display a list of current entries registered with the cron module and - their current state, last run time, etc. + The :option:`--working-dir` option allows the working directory to be set + for the command being executed as part of the cron job. Normally, the + working directory of the broker is used. - **stop** *id* - Stop cron entry *id*. The entry will remain in the cron entry list until - deleted. +**list** + Display a list of current entries registered with the cron module and + their current state, last run time, etc. - **start** *id* - Start a stopped cron entry *id*. +**stop** *id* + Stop cron entry *id*. The entry will remain in the cron entry list until + deleted. - **delete** [--kill] *id* - Purge cron entry *id* from the flux-cron entry list. If *--kill* is used, - kill any running task associated with entry *id*. +**start** *id* + Start a stopped cron entry *id*. - **dump** [--key=KEY] *id* - Dump all information for cron entry *id*. With *--key* print only the value - for key *KEY*. For a list of keys run *flux cron dump ID*. +**delete** [--kill] *id* + Purge cron entry *id* from the cron entry list. If :option:`--kill` is + used, kill any running task associated with entry *id*. + +**dump** [--key=KEY] *id* + Dump all information for cron entry *id*. With :option:`--key` print only + the value for key *KEY*. For a list of keys run + :program:`flux cron dump ID`. EXTRA OPTIONS ============= -For ``flux-cron`` commands allowing ``--options``, the following EXTRA OPTIONS -are supported: +.. program:: flux cron tab + +For :program:`flux cron` commands allowing :option:`--options`, the following +EXTRA OPTIONS are supported: + +.. option:: -o timeout=N -timeout=\ *N* Set a timeout for tasks invoked for this cron entry to *N* seconds, where N can be a floating point number. Default is no timeout. -rank=\ *R* +.. option:: -o rank=R + Set the rank on which to execute the cron command to *R*. Default is rank 0. -task-history-count=\ *N* +.. option:: -o task-history-count=N + Keep history for the last *N* tasks invoked by this cron entry. Default is 1. -stop-on-failure=\ *N* +.. option:: -o stop-on-failure=N + Automatically stop a cron entry if the failure count exceeds *N*. If *N* is zero (the default) then the cron entry will not be stopped on failure. @@ -191,17 +225,17 @@ total number of times to execute the cron job before stopping, and the All cron entries also support a less common list of options, which may be set at creation time via a comma-separated list of *option=value* -parameters passed to the *-o*, *--option=OPTS*. These options are described +parameters passed to :option:`-o, --option=OPTS`. These options are described in the EXTRA OPTIONS section at the end of this document. -Currently, flux-cron supports only two types of entries. The *interval* +Currently, Flux cron supports only two types of entries. The *interval* entry supports executing a command once every configured duration, optionally starting after a different time period. More detailed information about the interval type can be found in the documentation for -the flux-cron *interval* command above. The *event* type entry supports +the :program:`flux cron interval` command above. The *event* type entry supports running a command once every N events matching the configured event topic. More information about this type can be found in the documentation for -*flux cron event*. +:program:`flux cron event`. The Flux cron module additionally keeps a common set of statistics for each entry, regardless of type . These include the creation time, last @@ -212,14 +246,15 @@ the *flux cron dump* subcommand *stats.\** output. When registered, cron entries are automatically *started*, meaning they are eligible to run the configured command when the trigger condition -is met. Entries may be *stopped*, either by use of the *flux cron stop* +is met. Entries may be *stopped*, either by use of the :program:`flux cron stop` command, or if a *stop-on-failure* value is set. Stopped entries are -restarted using *flux cron start*, at which point counters used for +restarted using :program:`flux cron start`, at which point counters used for repeat and stop-on-failure are reset. -Stopped entries are kept in the flux cron until deleted with *flux -cron delete*. Active cron entries may also be deleted, with currently -executing tasks optionally killed if the *--kill* option is provided. +Stopped entries are kept in the flux cron until deleted with +:program:`flux cron delete`. Active cron entries may also be deleted, with +currently executing tasks optionally killed if the :option:`--kill` option is +provided. TASK EXECUTION @@ -238,18 +273,19 @@ On task failure, the cron job is stopped if *stop-on-failure* is set, and the current failure count exceeds the configured value. By default, *stop-on-failure* is not set. -By default, flux-cron module keeps information for the last task executed +By default, the Flux cron module keeps information for the last task executed for each cron entry. This information can be viewed either via the -*flux cron list* or *flux cron dump ID* subcommands. Data such as -start and end time, exit status, rank, and PID for the task is available. -The number of tasks kept for each cron entry may be individually tuned -via the *task-history-count* option, described in the EXTRA OPTIONS section. +:program:`flux cron list` or :program:`flux cron dump ID` subcommands. Data +such as start and end time, exit status, rank, and PID for the task is +available. The number of tasks kept for each cron entry may be individually +tuned via the :option:`--task-history-count` option, described in the +EXTRA OPTIONS section. Commands are normally executed immediately on the interval or event -trigger for which they are configured. However, if the *sync-event* +trigger for which they are configured. However, if the :option:`--sync-event` option is active on the cron module, tasks execution will be deferred until the next synchronization event. See the documentation above -for *flux cron sync* for more information. +for :program:`flux cron sync` for more information. RESOURCES diff --git a/doc/man1/flux-dmesg.rst b/doc/man1/flux-dmesg.rst index af7174f1b60c..d32249e85848 100644 --- a/doc/man1/flux-dmesg.rst +++ b/doc/man1/flux-dmesg.rst @@ -12,37 +12,47 @@ SYNOPSIS DESCRIPTION =========== +.. program:: flux dmesg + Each broker rank maintains a circular buffer of log entries -which can be printed using flux-dmesg(1). +which can be printed using :program`flux dmesg`. OPTIONS ======= -**-C, --clear** +.. option:: -C, --clear + Clear the ring buffer. -**-c, --read-clear** +.. option:: -c, --read-clear + Clear the ring buffer after printing its contents. -**-f, --follow** +.. option:: -f, --follow + After printing the contents of the ring buffer, wait for new entries and print them as they arrive. -**-n, --new** +.. option:: -n, --new + Follow only new log entries. -**-H, --human** - Display human-readable output. See also **--color** and **--delta**. +.. option:: -H, --human + + Display human-readable output. See also :option:`--color` and + :option:`--delta`. -**-d, --delta** - With **--human**, display the time delta between messages instead +.. option:: -d, --delta + + With :option:`--human`, display the time delta between messages instead of a relative offset since the last absolute timestamp. -**-L, --color**\ *[=WHEN]* +.. option:: -L, --color[=WHEN] + Colorize output. The optional argument *WHEN* can be *auto*, *never*, or *always*. If *WHEN* is omitted, it defaults to *always*. The default - value when the **--color** option is not used is *auto*. + value when the :option:`--color` option is not used is *auto*. EXAMPLES ======== diff --git a/doc/man1/flux-dump.rst b/doc/man1/flux-dump.rst index 0bf32c835258..f625c0e9812d 100644 --- a/doc/man1/flux-dump.rst +++ b/doc/man1/flux-dump.rst @@ -12,12 +12,14 @@ SYNOPSIS DESCRIPTION =========== -The ``flux-dump`` command writes a KVS snapshot to a portable archive format, -usually read by :man1:`flux-restore`. +.. program flux dump + +The :program:`flux dump` command writes a KVS snapshot to a portable archive +format, usually read by :man1:`flux-restore`. The snapshot source is the primary namespace of the current KVS root by default. -If *--checkpoint* is specified, the snapshot source is the last KVS checkpoint -written to the content backing store. +If :option:`--checkpoint` is specified, the snapshot source is the last KVS +checkpoint written to the content backing store. The archive is a file path or *-* for standard output. If standard output, the format is POSIX *ustar* with no compression. Otherwise the format is @@ -50,21 +52,26 @@ support: OPTIONS ======= -**-h, --help** +.. option:: -h, --help + Summarize available options. -**-v, --verbose** +.. option:: -v, --verbose + List keys on stderr as they are dumped instead of a periodic count of dumped keys. -**-q, --quiet** +.. option:: -q, --quiet + Don't show periodic count of dumped keys on stderr. -**--checkpoint** +.. option:: --checkpoint + Generate snapshot from the latest checkpoint written to the content backing store, instead of from the current KVS root. -**--no-cache** +.. option:: --no-cache + Bypass the broker content cache and interact directly with the backing store. This may be slightly faster, depending on how frequently the same content blobs are referenced by multiple keys. @@ -74,13 +81,13 @@ OTHER NOTES =========== KVS commits are atomic and propagate to the root of the namespace. Because of -this, when ``flux-dump`` archives a snapshot of a live system, it reflects one -point in time, and does not include any changes committed while the dump is -in progress. +this, when :program:`flux dump` archives a snapshot of a live system, it +reflects one point in time, and does not include any changes committed while +the dump is in progress. -Since ``flux-dump`` generates the archive by interacting directly with the -content store, the *--checkpoint* option may be used to dump the most recent -state of the KVS when the KVS module is not loaded. +Since :program:`flux dump` generates the archive by interacting directly with +the content store, the :option:`--checkpoint` option may be used to dump the +most recent state of the KVS when the KVS module is not loaded. Only regular values and symbolic links are dumped to the archive. Directories are not dumped as independent objects, so empty directories are omitted from @@ -93,11 +100,11 @@ The KVS path separator is converted to the UNIX-compatible slash so that the archive can be unpacked into a file system if desired. The modification time of files in the archive is set to the time that -``flux-dump`` is started if dumping the current KVS root, or to the timestamp -of the checkpoint if *--checkpoint* is used. +:program:`flux dump` is started if dumping the current KVS root, or to the +timestamp of the checkpoint if :option:`--checkpoint` is used. The owner and group of files in the archive are set to the credentials of the -user that ran ``flux-dump``. +user that ran :program:`flux-dump`. The mode of files in the archive is set to 0644. diff --git a/doc/man1/flux-env.rst b/doc/man1/flux-env.rst index 6c3bc56d2ee6..1367ee9111c9 100644 --- a/doc/man1/flux-env.rst +++ b/doc/man1/flux-env.rst @@ -14,9 +14,10 @@ SYNOPSIS DESCRIPTION =========== -flux-env(1) dumps a list of all environment variables as set by flux if run -without a command, when run with a command the environment is set and the -command is run as it would be by the :linux:man1:`env` utility. +:program:`flux env` dumps a list of all environment variables as set by +:man1:`flux` if run without a command, when run with a command the +environment is set and the command is run as it would be by the +:linux:man1:`env` utility. RESOURCES diff --git a/doc/man1/flux-event.rst b/doc/man1/flux-event.rst index 6c87a1222c0a..118c53f802cf 100644 --- a/doc/man1/flux-event.rst +++ b/doc/man1/flux-event.rst @@ -28,19 +28,19 @@ COMMANDS **pub** [-r] [-l] [-s] [-p] *topic* [*payload*] Publish an event with optional payload. If payload is specified, - it is interpreted as raw if the *-r* option is used, otherwise it is + it is interpreted as raw if the :option:`-r` option is used, otherwise it is interpreted as JSON. If the payload spans multiple arguments, the arguments are concatenated with one space between them. - If *-s* is specified, wait for the event's sequence number to be + If :option:`-s` is specified, wait for the event's sequence number to be assigned before exiting. - If *-l* is specified, subscribe to the published event and wait for - it to be received before exiting. *-p* causes the privacy flag to + If :option:`-l` is specified, subscribe to the published event and wait for + it to be received before exiting. :option:`-p` causes the privacy flag to be set on the published event. **sub** *[-c N]* [*topic*] [*topic*\ …​] Subscribe to events matching the topic string(s) provided on the - command line. If none are specified, subscribe to all events. - If *-c N* is specified, print the first *N* events on stdout and exit; + command line. If none are specified, subscribe to all events. If + :option:`-c N` is specified, print the first *N* events on stdout and exit; otherwise continue printing events until a signal is received. Events are displayed one per line: the topic string, followed by a tab, followed by the payload, if any. diff --git a/doc/man1/flux-exec.rst b/doc/man1/flux-exec.rst index 711d3718ddda..4efca62b573e 100644 --- a/doc/man1/flux-exec.rst +++ b/doc/man1/flux-exec.rst @@ -10,20 +10,22 @@ SYNOPSIS DESCRIPTION =========== -flux-exec(1) runs commands across one or more flux-broker ranks using +.. program:: flux exec + +:program:`flux exec` runs commands across one or more Flux broker ranks using the *broker.exec* service. The commands are executed as direct children of the broker, and the broker handles buffering stdout and stderr and -sends the output back to flux-exec(1) which copies output to its own +sends the output back to :program:`flux exec` which copies output to its own stdout and stderr. -On receipt of SIGINT and SIGTERM signals, flux-exec(1) shall forward +On receipt of SIGINT and SIGTERM signals, :program:`flux exec` shall forward the received signal to all currently running remote processes. In the event subprocesses are hanging or ignoring SIGINT, two SIGINT signals (typically sent via Ctrl+C) in short succession can force -flux-exec(1) to exit. +:program:`flux exec` to exit. -flux-exec(1) is meant as an administrative and test utility, and cannot +:program:`flux exec` is meant as an administrative and test utility, and cannot be used to launch Flux jobs. @@ -31,45 +33,53 @@ EXIT STATUS =========== In the case that all processes are successfully launched, the exit status -of flux-exec(1) is the largest of the remote process exit codes. +of :program:`flux exec` is the largest of the remote process exit codes. -If a non-existent rank is targeted, flux-exec(1) will return with +If a non-existent rank is targeted, :program:`flux exec` will return with code 68 (EX_NOHOST from sysexits.h). -If one or more remote commands are terminated by a signal, then flux-exec(1) -exits with exit code 128+signo. +If one or more remote commands are terminated by a signal, then +:program:`flux exec` exits with exit code 128+signo. OPTIONS ======= -**-l, --label-io** +.. option:: -l, --label-io + Label lines of output with the source RANK. -**-n, --noinput** +.. option:: -n, --noinput + Do not attempt to forward stdin. Send EOF to remote process stdin. -**-d, --dir**\ *=DIR* +.. option:: -d, --dir=DIR + Set the working directory of remote *COMMANDS* to *DIR*. The default is to propagate the current working directory of flux-exec(1). -**-r, --rank**\ *=NODESET* +.. option:: -r, --rank=NODESET + Target specific ranks in *NODESET*. Default is to target "all" ranks. - See NODESET FORMAT below for more information. + See `NODESET FORMAT`_ below for more information. + +.. option:: -v, --verbose -**-v, --verbose** Run with more verbosity. -**-q, --quiet** +.. option:: -q, --quiet + Suppress extraneous output (e.g. per-rank error exit status). -**--with-imp** - Prepend ``/path/to/flux-imp run`` to *COMMANDS*. This option is mostly - meant for testing or as a convenience to execute a configured ``prolog`` - or ``epilog`` command under the IMP. Note: When this option is used, - or if ``flux-imp`` is detected as the first argument of *COMMANDS*, - flux-exec(1) will use ``flux-imp kill`` to signal remote commands - instead of the normal builtin subprocess signaling mechanism. +.. option:: --with-imp + + Prepend the full path to :program:`flux-imp run` to *COMMANDS*. This option + is mostly meant for testing or as a convenience to execute a configured + ``prolog`` or ``epilog`` command under the IMP. Note: When this option is + used, or if :program:`flux-imp` is detected as the first argument of + *COMMANDS*, :program:`flux exec` will use :program:`flux-imp kill` to + signal remote commands instead of the normal builtin subprocess signaling + mechanism. NODESET FORMAT diff --git a/doc/man1/flux-filemap.rst b/doc/man1/flux-filemap.rst index 5c2a5a752006..34ca69d0367c 100644 --- a/doc/man1/flux-filemap.rst +++ b/doc/man1/flux-filemap.rst @@ -18,13 +18,16 @@ SYNOPSIS DESCRIPTION =========== -``flux-filemap`` uses :linux:man2:`mmap` to map files into the rank 0 broker -*content cache*. After mapping, the files may be extracted on any broker rank, -taking advantage of scalability properties of the distributed cache to move the -data. The files are treated as read-only and must not change while mapped. +.. program:: flux filemap -``flux-filemap map`` maps one or more file *PATH* arguments. It must be run -on the rank 0 broker, such as within a batch script, and the files must be +:program:`flux filemap` uses :linux:man2:`mmap` to map files into the rank 0 +broker *content cache*. After mapping, the files may be extracted on any +broker rank, taking advantage of scalability properties of the distributed +cache to move the data. The files are treated as read-only and must not +change while mapped. + +:program:`flux filemap map` maps one or more file *PATH* arguments. It must be +run on the rank 0 broker, such as within a batch script, and the files must be directly accessible by the rank 0 broker. If a PATH refers to a directory, the directory is recursively mapped. If a file is encountered that is not readable, or has a type other than regular file, directory, or symbolic link, @@ -32,16 +35,16 @@ a fatal error occurs. Sparse files such as file system images for virtual machines are mapped efficiently. File discretionary access permission are preserved, but file attributes, ACLs, and group ownership are not. -``flux-filemap list`` lists mapped files. Optionally, a :man7:`glob` pattern -may be specified to filter the list. +:program:`flux filemap list` lists mapped files. Optionally, a :man7:`glob` +pattern may be specified to filter the list. -``flux-filemap get`` extracts mapped files and may be run on any broker or -across all brokers using :man1:`flux-exec`. Optionally, a :man7:`glob` pattern -may be specified to filter the list. When extracting mapped files in parallel, -take care to specify a *--directory* that is not shared and is not on a network -file system without considering the ramifications. +:program:`flux filemap get` extracts mapped files and may be run on any broker +or across all brokers using :man1:`flux-exec`. Optionally, a :man7:`glob` +pattern may be specified to filter the list. When extracting mapped files in +parallel, take care to specify a :option:`--directory` that is not shared and +is not on a network file system without considering the ramifications. -``flux-filemap unmap`` unmaps mapped files. +:option:`flux filemap unmap` unmaps mapped files. The ``stage-in`` shell plugin described in :man1:`flux-shell` may be used to extract previously mapped files into $FLUX_JOB_TMPDIR or another directory. @@ -49,49 +52,60 @@ extract previously mapped files into $FLUX_JOB_TMPDIR or another directory. OPTIONS ======= -**-h, --help** +.. option:: -h, --help + Display options and exit -**-T, --tags=LIST** +.. option:: -T, --tags=LIST + Specify a comma separated list of *tags*. If no tags are specified, the *main* tag is assumed. -**-C, --directory=DIR** +.. option:: -C, --directory=DIR + Change to the specified directory before performing the operation (*map* and *get* subcommands only). -**-v, --verbose=[LEVEL]** +.. option:: -v, --verbose=[LEVEL] + Increase output verbosity (*map* and *get* subcommands only). -**-l, --long** +.. option:: -l, --long + Include more detail in file listing (*list* subcommand only). -**--small-file-threshold=N** +.. option:: --small-file-threshold=N + Set the threshold in bytes over which a regular file is mapped through the distributed content cache. Set to 0 to always use the content cache. N may be specified as a floating point number with multiplicative suffix k,K=1024, M=1024\*1024, or G=1024\*1024\*1024 up to ``INT_MAX``. The default is 4K (*map* subcommand only). -**--disable-mmap** +.. option:: --disable-mmap + Never map a regular file through the distributed content cache. -**--chunksize=N** +.. option:: --chunksize=N + Limit the content mapped blob size to N bytes. Set to 0 for unlimited. N may be specified as a floating point number with multiplicative suffix k,K=1024, M=1024\*1024, or G=1024\*1024\*1024 up to ``INT_MAX``. The default is 1M (*map* subcommand only). -**--direct** +.. option:: --direct + Avoid indirection through the content cache when fetching the top level data for each file. This may be fastest for a single or small number of clients, but will scale poorly when performed in parallel (*get* subcommand only). -**--blobref** +.. option:: --blobref + List blobrefs (*list* subcommand only). -**--raw** +.. option:: --raw + List RFC 37 file system objects (*list* subcommand only). EXAMPLE diff --git a/doc/man1/flux-getattr.rst b/doc/man1/flux-getattr.rst index f639f2f2277f..0c7993140c6c 100644 --- a/doc/man1/flux-getattr.rst +++ b/doc/man1/flux-getattr.rst @@ -22,12 +22,12 @@ Flux broker attributes are both a simple, general-purpose key-value store with scope limited to the local broker rank, and a method for the broker to export information needed by Flux services and utilities. -flux-getattr(1) retrieves the value of an attribute. +:program:`flux getattr` retrieves the value of an attribute. -flux-setattr(1) assigns a new value to an attribute, or optionally +:program:`flux setattr` assigns a new value to an attribute, or optionally removes an attribute. -flux-lsattr(1) lists attribute names, optionally with their values. +:program:`flux lsattr` lists attribute names, optionally with their values. RESOURCES diff --git a/doc/man1/flux-job.rst b/doc/man1/flux-job.rst index f832a01ac22c..44c85dc8689b 100644 --- a/doc/man1/flux-job.rst +++ b/doc/man1/flux-job.rst @@ -44,13 +44,15 @@ flux-job(1) performs various job related housekeeping functions. ATTACH ====== -A job can be interactively attached to via ``flux job attach``. This is +.. program:: flux job attach + +A job can be interactively attached to via :program:`flux job attach`. This is typically used to watch stdout/stderr while a job is running or after it has completed. It can also be used to feed stdin to a job. -When ``flux job attach`` is run interactively -- that is all of ``stdout``, -``stderr`` and ``stdin`` are attached to a tty -- the command may display -a status line while the job is pending, e.g +When :program:`flux job attach` is run interactively -- that is all of +``stdout``, ``stderr`` and ``stdin`` are attached to a tty -- the command may +display a status line while the job is pending, e.g :: @@ -59,183 +61,233 @@ a status line while the job is pending, e.g This status line may be suppressed by setting ``FLUX_ATTACH_NONINTERACTIVE`` in the environment. -**-l, --label-io** +.. option:: -l, --label-io + Label output by rank -**-u, --unbuffered** +.. option:: -u, --unbuffered + Do not buffer stdin. Note that when ``flux job attach`` is used in a terminal, the terminal itself may line buffer stdin. -**-i, --stdin-ranks=RANKS** +.. option:: -i, --stdin-ranks=RANKS + Send stdin to only those ranks in the **RANKS** idset. The standard input for tasks not in **RANKS** will be closed. The default is to broadcast stdin to all ranks. -**--read-only** +.. option:: --read-only + Operate in read-only mode. Disable reading of stdin and capturing of signals. -**-v, --verbose** +.. option:: -v, --verbose + Increase verbosity. -**-w, --wait-event=EVENT** +.. option:: -w, --wait-event=EVENT + Wait for event *EVENT* before detaching from eventlog. The default is ``finish``. -**-E, --show-events** +.. option:: -E, --show-events + Show job events on stderr. This option also suppresses the status line if enabled. -**-X, --show-exec** +.. option:: -X, --show-exec + Show exec eventlog events on stderr. -**--show-status** +.. option:: --show-status + Force immediate display of the status line. -**--debug*** +.. option:: --debug + Enable parallel debugger attach. CANCEL ====== -One or more jobs by may be canceled with ``flux job cancel``. An optional -message included with the cancel exception may be provided via the *-m, ---message=NOTE* option or after the list of jobids. The special argument -*"--"* forces the end of jobid processing and can be used to separate the -exception message from the jobids when necessary. +.. program:: flux job cancel + +One or more jobs by may be canceled with :program:`flux job cancel`. An +optional message included with the cancel exception may be provided via the +:option:`--message=NOTE` option or after the list of jobids. The special +argument *"--"* forces the end of jobid processing and can be used to separate +the exception message from the jobids when necessary. + +.. option:: -m, --message=NOTE -**-m, --message=NOTE** Set the optional exception note. It is an error to specify the message via this option and on the command line after the jobid list. -Jobs may be canceled in bulk with ``flux job cancelall``. Target jobs are -selected with: +.. program:: flux job cancelall + +Jobs may be canceled in bulk with :program:`flux job cancelall`. Target jobs +are selected with: + +.. option:: -u, --user=USER -**-u, --user=USER** Set target user. The instance owner may specify *all* for all users. -**-S, --states=STATES** +.. option:: -S, --states=STATES + Set target job states (default: ACTIVE). -**-f, --force** +.. option:: -f, --force + Confirm the command -**-q, --quiet** +.. option:: -q, --quiet + Suppress output if no jobs match STATUS ====== +.. program:: flux job status + Wait for job(s) to complete and exit with the largest exit code. -**-e, --exception-exit-code=N** +.. option:: -e, --exception-exit-code=N + Set the exit code for any jobs that terminate with an exception (e.g. canceled jobs) to ``N``. -**-j, --json** +.. option:: -j, --json + Dump job result information from job eventlog. -**-v, --verbose** +.. option:: -v, --verbose + Increase verbosity of output. WAIT ==== -``flux job wait`` behaves like the UNIX :linux:man2:`wait` system call, +.. program:: flux job wait + +:program:`flux job wait` behaves like the UNIX :linux:man2:`wait` system call, for jobs submitted with the ``waitable`` flag. Compared to other methods of synchronizing on job completion and obtaining results, it is very lightweight. The result of a waitable job may only be consumed once. This is a design -feature that makes it possible to call ``flux job wait`` in a loop until all -results are consumed. +feature that makes it possible to call :program:`flux job wait` in a loop +until all results are consumed. .. note:: Only the instance owner is permitted to submit jobs with the ``waitable`` flag. -When run with a jobid argument, ``flux job wait`` blocks until the specified -job completes. If the job was successful, it silently exits with a code of -zero. If the job has failed, an error is printed on stderr, and it exits with -a code of one. If the jobid is invalid or the job is not waitable, ``flux job wait`` -exits with a code of two. This special exit code of two is used to differentiate -between a failed job and not being able to wait on the job. +When run with a jobid argument, :program:`flux job wait` blocks until the +specified job completes. If the job was successful, it silently exits with a +code of zero. If the job has failed, an error is printed on stderr, and it +exits with a code of one. If the jobid is invalid or the job is not waitable, +:program:`flux job wait` exits with a code of two. This special exit code of +two is used to differentiate between a failed job and not being able to wait +on the job. -When run without arguments, ``flux job wait`` blocks until the next waitable -job completes and behaves as above except that the jobid is printed to stdout. -When there are no more waitable jobs, it exits with a code of two. The exit code -of two can be used to determine when no more jobs are waitable when using -``flux job wait`` in a loop. +When run without arguments, :program:`flux job wait` blocks until the next +waitable job completes and behaves as above except that the jobid is printed +to stdout. When there are no more waitable jobs, it exits with a code of two. +The exit code of two can be used to determine when no more jobs are waitable +when using :program:`flux job wait` in a loop. -``flux job wait --all`` loops through all the waitable jobs as they complete, -printing their jobids. If all jobs are successful, it exits with a code of zero. -If any jobs have failed, it exits with a code of one. +:option:`flux job wait --all` loops through all the waitable jobs as they +complete, printing their jobids. If all jobs are successful, it exits with a +code of zero. If any jobs have failed, it exits with a code of one. + +.. option:: -a, --all -**-a, --all** Wait for all waitable jobs and exit with error if any jobs are not successful. -**-v, --verbose** +.. option:: -v, --verbose + Emit a line of output for all jobs, not just failing ones. SIGNAL ====== -One or more running jobs may be signaled by jobid with ``flux job kill``. +.. program:: flux job kill + +One or more running jobs may be signaled by jobid with :program:`flux job kill`. + +.. option:: -s, --signal=SIG -**-s, --signal=SIG** Send signal SIG (default: SIGTERM). -Running jobs may be signaled in bulk with ``flux job killall``. In addition -to the option above, target jobs are selected with: +.. program:: flux job killall + +Running jobs may be signaled in bulk with :program:`flux job killall`. In +addition to the option above, target jobs are selected with: + +.. option:: -u, --user=USER -**-u, --user=USER** Set target user. The instance owner may specify *all* for all users. -**-f, --force** +.. option:: -f, --force + Confirm the command. EXCEPTION ========= -An exception may raised on one or more jobids with ``flux job raise``. +.. program:: flux job raise + +An exception may raised on one or more jobids with :program:`flux job raise`. An optional message included with the job exception may be provided via -the *-m, --message=NOTE* option or after the list of jobids. The special +the :option:`--message=NOTE` option or after the list of jobids. The special argument *"--"* forces the end of jobid processing and can be used to separate the exception message from the jobids when necessary. -**-m, --message=NOTE** +.. option:: -m, --message=NOTE + Set the optional exception note. It is an error to specify the message via this option and on the command line after the jobid list. -**-s, --severity=N** + +.. option:: -s, --severity=N + Set exception severity. The severity may range from 0=fatal to 7=least severe (default: 0). -**-t, --type=TYPE** +.. option:: -t, --type=TYPE + Set exception type (default: cancel). -Exceptions may be raised in bulk with ``flux job raiseall``, which requires a -type (positional argument) and accepts the following options: +Exceptions may be raised in bulk with :program:`flux job raiseall`, which +requires a type (positional argument) and accepts the following options: + +.. program:: flux job raiseall + +.. option:: -s, --severity=N -**-s, --severity=N** Set exception severity. The severity may range from 0=fatal to 7=least severe (default: 7). -**-u, --user=USER** +.. option:: -u, --user=USER + Set target user. The instance owner may specify *all* for all users. -**-S, --states=STATES** +.. option:: -S, --states=STATES + Set target job states (default: ACTIVE) -**-f, --force** +.. option:: -f, --force + Confirm the command. TASKMAP ======= +.. program:: flux job taskmap + The mapping between job task ranks to node IDs is encoded in the RFC 34 Flux Task Map format and posted to the job's ``shell.start`` event in the -exec eventlog. The ``flux job taskmap`` utility is provided to assist in +exec eventlog. The :program:`flux job taskmap` utility is provided to assist in working with these task maps. When executed with a jobid argument and no options, the taskmap for the job @@ -246,19 +298,24 @@ a nodeid to a list of tasks, or to query on which node or host a given taskid ran. The command may also be used to convert between different support task mapping formats: -**--taskids=NODEID** +.. option:: --taskids=NODEID + Print an idset of tasks which ran on node *NODEID* -**--ntasks=NODEID** +.. option:: --ntasks=NODEID + Print the number of tasks which ran on node *NODEID* -**--nodeid=TASKID** +.. option:: --nodeid=TASKID + Print the node ID that ran task *TASKID* -**--hostname=TASKID** +.. option:: --hostname=TASKID + Print the hostname of the node that rank task *TASKID* -**--to=raw|pmi|multiline** +.. option:: --to=raw|pmi|multiline + Convert the taskmap to *raw* or *pmi* formats (described in RFC 34), or *multiline* which prints the node ID of each task, one per line. @@ -267,34 +324,43 @@ One one of the above options may be used per call. TIMELEFT ======== -The ``flux job timeleft`` utility reports the number of whole seconds left -in the current or specified job time limit. If the job has expired or is +.. program:: flux job timeleft + +The :program:`flux job timeleft` utility reports the number of whole seconds +left in the current or specified job time limit. If the job has expired or is complete, then this command reports ``0``. If the job does not have a time limit, then a large number (``UINT_MAX``) is reported. -If ``flux job timeleft`` is called outside the context of a Flux job, or +If :program:`flux job timeleft` is called outside the context of a Flux job, or an invalid or pending job is targeted, then this command will exit with an error and diagnostic message. Options: -**-H, --human** +.. option:: -H, --human + Generate human readable output. Report results in Flux Standard Duration. PURGE ===== -Inactive job data may be purged from the Flux instance with ``flux job purge``. -Specific job ids may be specified for purging. If no job ids are -specified, the following options may be used for selection criteria: +.. program:: flux job purge + +Inactive job data may be purged from the Flux instance with +:program:`flux job purge`. Specific job ids may be specified for purging. +If no job ids are specified, the following options may be used for selection +criteria: + +.. option:: --age-limit=FSD -**--age-limit=FSD** Purge inactive jobs older than the specified Flux Standard Duration. -**--num-limit=COUNT** +.. option:: --num-limit=COUNT + Purge the oldest inactive jobs until there are at most COUNT left. -**-f, --force** +.. option:: -f, --force + Confirm the command. Inactive jobs may also be purged automatically if the job manager is diff --git a/doc/man1/flux-jobs.rst b/doc/man1/flux-jobs.rst index 00157510c1da..f7f3b8e266b0 100644 --- a/doc/man1/flux-jobs.rst +++ b/doc/man1/flux-jobs.rst @@ -13,7 +13,9 @@ SYNOPSIS DESCRIPTION =========== -flux-jobs(1) is used to list jobs run under Flux. By default only +.. program:: flux jobs + +:program:`flux jobs` is used to list jobs run under Flux. By default only pending and running jobs for the current user are listed. Additional jobs and information can be listed using options listed below. Alternately, specific job ids can be listed on the command line to @@ -23,80 +25,94 @@ only list those job IDs. OPTIONS ======= -**-a** +.. option:: -a + List jobs in all states, including inactive jobs. - This is shorthand for *--filter=pending,running,inactive*. + This is shorthand for :option:`--filter=pending,running,inactive`. + +.. option:: -A -**-A** - List jobs of all users. This is shorthand for *--user=all*. + List jobs of all users. This is shorthand for :option:`--user=all`. + +.. option:: -n, --no-header -**-n, --no-header** For default output, do not output column headers. -**-u, --user**\ *=[USERNAME|UID]* +.. option:: -u, --user=[USERNAME|UID] + List jobs for a specific username or userid. Specify *all* for all users. -**--name**\ *=[JOB NAME]* +.. option:: --name=[JOB NAME] + List jobs with a specific job name. -**--queue**\ *=[QUEUE]* +.. option:: --queue=[QUEUE] + List jobs in a specific queue. -**-c, --count**\ *=N* +.. option:: -c, --count=N + Limit output to N jobs (default 1000) -**--since**\ *WHEN* +.. option:: --since=WHEN + Limit output to jobs that have been active since a given timestamp. In other words, jobs that are currently pending, currently running, or became inactive - since the given timestamp. This option implies ``-a`` if no other - ``--filter`` options are specified. If *WHEN* begins with ``-`` character, - then the remainder is considered to be a an offset in Flux standard duration - (RFC 23). Otherwise, any datetime expression accepted by the Python - `parsedatetime `_ module is - accepted. Examples: "-6h", "-1d", "yesterday", "2021-06-21 6am", + since the given timestamp. This option implies :option:`-a` if no other + :option:`--filter` options are specified. If *WHEN* begins with ``-`` + character, then the remainder is considered to be a an offset in Flux + standard duration (RFC 23). Otherwise, any datetime expression accepted by + the Python `parsedatetime `_ module + is accepted. Examples: "-6h", "-1d", "yesterday", "2021-06-21 6am", "last Monday", etc. It is assumed to be an error if a timestamp in the future is supplied. -**-f, --filter**\ *=STATE|RESULT* +.. option:: -f, --filter=STATE|RESULT + List jobs with specific job state or result. Multiple states or - results can be listed separated by comma. See JOB STATUS below for + results can be listed separated by comma. See `JOB STATUS`_ below for additional information. Defaults to *pending,running*. -**-o, --format**\ *=NAME|FORMAT* +.. option:: -o, --format=NAME|FORMAT + Specify a named output format *NAME* or a format string using Python's - format syntax. See OUTPUT FORMAT below for field names. Named formats - may be listed via ``--format=help``. An alternate default format can be set - via the FLUX_JOBS_FORMAT_DEFAULT environment variable. Additional named - formats may be registered with ``flux jobs`` via configuration. See the - CONFIGURATION section for more details. A configuration snippet for an - existing named format may be generated with ``--format=get-config=NAME``. - -**--json** + format syntax. See `OUTPUT FORMAT`_ below for field names. Named formats + may be listed via :option:`--format=help`. An alternate default format can + be set via the FLUX_JOBS_FORMAT_DEFAULT environment variable. Additional + named formats may be registered with :program:`flux jobs` via configuration. + See the `CONFIGURATION`_ section for more details. A configuration snippet + for an existing named format may be generated with + :option:`--format=get-config=NAME`. + +.. option:: --json + Emit data for selected jobs in JSON format. The data for multiple matching jobs is contained in a ``jobs`` array in the emitted JSON object, unless a single job was selected by jobid on the command line, in which case a JSON object representing that job is emitted on - success. With ``-R, --recursive``, each job which is also an instance + success. With :option:`--recursive`, each job which is also an instance of Flux will will have any recursively listed jobs in a ``jobs`` array, and so on for each sub-child. - Only the attributes which are available at the time of the flux-jobs - query will be present in the returned JSON object for a job. For - instance a pending job will not have ``runtime``, ``waitstatus`` or - ``result`` keys, among others. A missing key should be considered + Only the attributes which are available at the time of the + :program:`flux jobs` query will be present in the returned JSON object for + a job. For instance a pending job will not have ``runtime``, ``waitstatus`` + or ``result`` keys, among others. A missing key should be considered unavailable. - The ``--json`` option is incompatible with ``--stats`` and - ``--stats-only``, and any ``--format`` is ignored. + The :option:`--json` option is incompatible with :option:`--stats` and + :option:`--stats-only`, and any :option:`--format` is ignored. + +.. option:: --color[=WHEN] -**--color**\ *[=WHEN]* Control output coloring. The optional argument *WHEN* can be *auto*, *never*, or *always*. If *WHEN* is omitted, it defaults to *always*. Otherwise the default is *auto*. -**--stats** +.. option:: --stats + Output a summary of job statistics before the header. By default - shows global statistics. If ``--queue`` is specified, shows + shows global statistics. If :option:`--queue` is specified, shows statistics for the specified queue. May be useful in conjunction with utilities like :linux:man1:`watch`, e.g.:: @@ -106,51 +122,58 @@ OPTIONS running jobs, updated every 2 seconds. Note that all job failures, including canceled and timeout jobs, - are collectively counted as "failed" in ``--stats``. + are collectively counted as "failed" in :option:`--stats`. + +.. option:: --stats-only -**--stats-only** Output a summary of job statistics and exit. By default shows - global statistics. If ``--queue`` is specified, shows statistics - for the specified queue. ``flux jobs`` will exit with non-zero - exit status with ``--stats-only`` if there are no active jobs. This + global statistics. If :option:`--queue` is specified, shows statistics + for the specified queue. :program:`flux jobs` will exit with non-zero + exit status with :option:`--stats-only` if there are no active jobs. This allows the following loop to work:: $ while flux jobs --stats-only; do sleep 2; done - All options other than ``--queue`` are ignored when - ``--stats-only`` is used. + All options other than :option:`--queue` are ignored when + :option:`--stats-only` is used. Note that all job failures, including canceled and timeout jobs, - are collectively counted as "failed" in ``--stats-only``. + are collectively counted as "failed" in :option:`--stats-only`. + +.. option:: -R, --recursive -**-R, --recursive** List jobs recursively. Each child job which is also an instance of Flux is prefixed by its jobid "path" followed by the list of jobs, - recursively up to any defined ``-L, --level``. If the ``--stats`` + recursively up to any defined :option:`--level`. If the :option:`--stats` option is used, then each child instance in the hierarchy is listed with its stats. -**--recurse-all** - By default, jobs not owned by the user running ``flux jobs`` are - skipped with ``-R, --recursive``, because normally Flux instances +.. option:: --recurse-all + + By default, jobs not owned by the user running :program:`flux jobs` are + skipped with :option:`--recursive`, because normally Flux instances only permit the instance owner to connect. This option forces the command to attempt to recurse into the jobs of other users. Implies - ``--recursive``. - -**-L, --level**\ *=N* - With ``-R, --recursive``, stop recursive job listing at level **N**. - Levels are counted starting at 0, so ``flux jobs -R --level=0`` is - equivalent to ``flux jobs`` without ``-R``, and ``--level=1`` would - limit recursive job listing to child jobs of the current instance. - -**--threads**\ *=N* - When ``flux jobs`` recursively queries job lists (with ``--recursive``) - or fetches info for jobs that are also instances (see - ``instance.*`` fields), a pool of threads is used to parallelize + :option:`--recursive`. + +.. option:: -L, --level=N + + With :option:`--recursive`, stop recursive job listing at level **N**. + Levels are counted starting at 0, so :option:`flux jobs -R --level=0` is + equivalent to :program:`flux jobs` without :option:`-R`, and + :option:`--level=1` would limit recursive job listing to child jobs of the + current instance. + +.. option:: --threads=N + + When :program:`flux jobs` recursively queries job lists (with + :option:`--recursive`) or fetches info for jobs that are also instances + (see ``instance.*`` fields), a pool of threads is used to parallelize the required RPCs. Normally, the default number of ThreadPoolExecutor - threads is used, but by using the ``--threads``, a specific number + threads is used, but by using the :option:`--threads`, a specific number of threads can be chosen. +.. _flux_jobs_job_status: JOB STATUS ========== @@ -173,11 +196,13 @@ FAILED, CANCELED, or TIMEOUT. Under the *status_abbrev* field name, these are abbreviated as D, S, R, C, CD, F, CA, and TO respectively. +.. _flux_jobs_output_format: + OUTPUT FORMAT ============= -The *--format* option can be used to specify an output format to -flux-jobs(1) using Python's string format syntax. For example, the +The :option:`--format` option can be used to specify an output format to +:program:`flux jobs` using Python's string format syntax. For example, the following is the format used for the default format: :: @@ -211,7 +236,7 @@ by including a ``+`` character when truncation occurs. If both *h* and Additionally, the custom job formatter supports a set of special conversion flags. Conversion flags follow the format field and are used to transform the value before formatting takes place. Currently, -the following conversion flags are supported by *flux-jobs*: +the following conversion flags are supported by :program:`flux jobs`: **!D** convert a timestamp field to ISO8601 date and time (e.g. 2020-01-07T13:31:00). @@ -469,10 +494,12 @@ the state of the job or other context: will also be provided such as the exit ``returncode`` or cancellation message. +.. _flux_jobs_configuration: + CONFIGURATION ============= -The ``flux-jobs`` command supports registration of named output formats +The :program:`flux jobs` command supports registration of named output formats in configuration files. The command loads configuration files from ``flux-jobs.EXT`` from the following paths in order of increasing precedence: @@ -497,12 +524,12 @@ key per format pointing to a table or dictionary with the keys: **description** (optional) A short description of the named format, displayed with - ``flux jobs --format=help`` + :option:`flux jobs --format=help` If a format name is specified in more than one config file, then the last -one loaded is used. Due to the order that ``flux-jobs`` loads config files, -this allows user configuration to override system configuration. It is an -error to override any internally defined formats (such as ``default``). +one loaded is used. Due to the order that :program:`flux jobs` loads config +files, this allows user configuration to override system configuration. It is +an error to override any internally defined formats (such as ``default``). If a format name or string is not specified on the command line the internally defined format ``default`` is used. @@ -519,7 +546,7 @@ Example:: """ It may be helpful to start with an existing named format by using the -``--format=get-config=NAME`` option, e.g.:: +:option:`--format=get-config=NAME` option, e.g.:: $ flux jobs --format=get-config=default >> ~/.config/flux/flux-jobs.toml @@ -530,7 +557,7 @@ error to redefine the default format string. EXAMPLES ======== -The default output of flux-jobs(1) will list the pending and running +The default output of :program:`flux jobs` will list the pending and running jobs of the current user. It is equivalent to: :: @@ -538,7 +565,7 @@ jobs of the current user. It is equivalent to: $ flux jobs --filter=pending,running To list all pending, running, and inactive jobs, of the current user, -you can use *--filter* option or the *-a* option: +you can use :option:`--filter` option or the :option:`-a` option: :: @@ -548,7 +575,7 @@ you can use *--filter* option or the *-a* option: $ flux jobs --filter=pending,running,inactive -To alter which user's jobs are listed, specify the user with *--user*: +To alter which user's jobs are listed, specify the user with :option:`--user`: :: @@ -562,7 +589,7 @@ will list the jobs that have failed or were canceled: $ flux jobs --filter=failed,canceled -The *--format* option can be used to alter the output format or output +The :option:`--format` option can be used to alter the output format or output additional information. For example, the following would output all jobids for the user in decimal form, and output any annotations the scheduler attached to each job: diff --git a/doc/man1/flux-jobtap.rst b/doc/man1/flux-jobtap.rst index 1b59cf2ba4de..741f67cc16db 100644 --- a/doc/man1/flux-jobtap.rst +++ b/doc/man1/flux-jobtap.rst @@ -11,7 +11,7 @@ SYNOPSIS DESCRIPTION =========== -The flux-jobtap(1) command is used to query, load, and remove *jobtap* +The :program:`flux jobtap` command is used to query, load, and remove *jobtap* plugins from the Flux job-manager module at runtime. COMMANDS @@ -19,12 +19,12 @@ COMMANDS **list** [*-a, --all*] Print the currently loaded list of plugins. Builtin plugins will only - be displayed when the *--all* option is used. Plugins built in to the + be displayed when the :option:`--all` option is used. Plugins built in to the job manager have a leading ``.`` in the name, e.g. ``.priority-default``. **load** [*-r*, *--remove=NAME*] PLUGIN [KEY=VAL, KEY=VAL...] Load a new plugin into the job-manager, optionally removing plugin NAME - first. With *--remove* NAME may be a :linux:man7:`glob` pattern + first. With :option:`--remove` NAME may be a :linux:man7:`glob` pattern match. Optional KEY=VAL occurring after PLUGIN will set config KEY to VAL for PLUGIN. diff --git a/doc/man1/flux-keygen.rst b/doc/man1/flux-keygen.rst index 947547006db5..0bc99b470890 100644 --- a/doc/man1/flux-keygen.rst +++ b/doc/man1/flux-keygen.rst @@ -12,7 +12,9 @@ SYNOPSIS DESCRIPTION =========== -flux-keygen(1) generates a long-term CURVE certificate used to secure +.. program:: flux keygen + +:program:`flux keygen` generates a long-term CURVE certificate used to secure the overlay network of a Flux system instance. The Flux overlay network implements cryptographic privacy and data integrity @@ -31,14 +33,16 @@ public keys are exchanged with PMI. OPTIONS ======= -``flux-keygen`` accepts the following options: +:program:`flux keygen` accepts the following options: + +.. option:: -n, --name=NAME -**-n, --name=NAME** Set the certificate metadata ``name`` field. The value is logged when :man1:`flux-broker` authenticates a peer that presents this certificate. A cluster name might be appropriate here. Default: the local hostname. -**--meta=KEY=VAL** +.. option:: --meta=KEY=VAL + Set arbitrary certificate metadata. Multiple key-value pairs may be specified, separated by commas. This option may be specified multiple times. diff --git a/doc/man1/flux-kvs.rst b/doc/man1/flux-kvs.rst index d828a5708e64..2ad571073813 100644 --- a/doc/man1/flux-kvs.rst +++ b/doc/man1/flux-kvs.rst @@ -14,7 +14,7 @@ DESCRIPTION The Flux key-value store (KVS) is a simple, distributed data storage service used a building block by other Flux components. -flux-kvs(1) is a command line utility that operates on the KVS. +:program:`flux kvs` is a command line utility that operates on the KVS. It is a very thin layer on top of a C API. The Flux KVS stores values under string keys. The keys are @@ -39,7 +39,7 @@ default KVS namespace "primary". An alternate namespace can be specified in most kvs commands via the *--namespace* option, or by setting the namespace in the environment variable FLUX_KVS_NAMESPACE. -flux-kvs(1) runs a KVS *COMMAND*. The possible commands and their +:program:`flux kvs` runs a KVS *COMMAND*. The possible commands and their arguments are described below. diff --git a/doc/man1/flux-logger.rst b/doc/man1/flux-logger.rst index 8dd98671e67e..c22805c2ceba 100644 --- a/doc/man1/flux-logger.rst +++ b/doc/man1/flux-logger.rst @@ -11,7 +11,9 @@ SYNOPSIS DESCRIPTION =========== -flux-logger(1) appends Flux log entries to the local Flux +.. program:: flux logger + +:program:`flux logger` appends Flux log entries to the local Flux broker's circular buffer. Log entries are associated with a :linux:man3:`syslog` style severity. @@ -29,10 +31,12 @@ message when it is created. OPTIONS ======= -**-s, --severity**\ =\ *SEVERITY* +.. option:: -s, --severity=SEVERITY + Specify the log message severity. The default severity is *info*. -**-n, --appname**\ =\ *NAME* +.. option:: -n, --appname=NAME + Specify a user-defined application name to associate with the log message. The default appname is *logger*. diff --git a/doc/man1/flux-mini.rst b/doc/man1/flux-mini.rst deleted file mode 100644 index 78584f634505..000000000000 --- a/doc/man1/flux-mini.rst +++ /dev/null @@ -1,124 +0,0 @@ -============ -flux-mini(1) -============ - - -.. warning:: - - This command is deprecated. Please use :man1:`flux-submit`, - :man1:`flux-run`, :man1:`flux-batch`, :man1:`flux-alloc`, and - :man1:`flux-bulksubmit` instead. - - -SYNOPSIS -======== - -**flux** **mini** **submit** [OPTIONS] [*--ntasks=N*] COMMAND... - -**flux** **mini** **bulksubmit** [OPTIONS] [*--ntasks=N*] COMMAND... - -**flux** **mini** **run** [OPTIONS] [*--ntasks=N*] COMMAND... - -**flux** **mini** **batch** [OPTIONS] *--nslots=N* SCRIPT... - -**flux** **mini** **alloc** [OPTIONS] *--nslots=N* [COMMAND...] - - -DESCRIPTION -=========== - -flux-mini(1) submits jobs to run under Flux. In the case of **submit** -or **run** the job consists of *N* copies of COMMAND launched together -as a parallel job, while **batch** and **alloc** submit a script or launch -a command as the initial program of a new Flux instance. - -If *--ntasks* is unspecified, a value of *N=1* is assumed. Commands that -take *--nslots* have no default and require that *--nslots* or *--nodes* -be specified. - -The **submit** and **batch** commands enqueue the job and print its numerical -Job ID on standard output. - -The **run** and **alloc** commands do the same interactively, blocking until -the job has completed. - -The **bulksubmit** command enqueues one job each for a set of inputs read -on either stdin, or given on the command line. The inputs are optionally -substituted in ``COMMAND`` and/or many submission options. See more in the -:ref:`bulksubmit` section below. - -The **flux-mini batch** command submits a script to run as the initial -program of a job running a new instance of Flux. The SCRIPT given on the -command line is assumed to be a file name unless the *--wrap* option is -used, in which case the free arguments are consumed as the script, with -``#!/bin/sh`` as the first line. If no SCRIPT is provided, then one will -be read from standard input. The script may contain submission directives -denoted by ``flux:`` or ``FLUX:`` as described in RFC 36, see -:ref:`submission_directives` below. - -For **flux-mini batch**, the SCRIPT given on the command line is assumed -to be a file name, unless the *--wrap* option used, and the script -file is read and submitted along with the job. If no SCRIPT is -provided, then one will be read from *stdin*. - -**flux-mini alloc** works similarly to **batch**, but instead blocks until -the job has started and interactively attaches to the new Flux instance -(unless the ``--bg`` option is used). By default, a new shell is spawned -as the initial program of the instance, but this may be overridden by -supplying COMMAND on the command line. - -The intent is for the "mini" commands to remain simple with stable interfaces -over time, making them suitable for use in scripts. - -The available OPTIONS are detailed below. - -.. note:: - - These commands target the *enclosing instance*. For example, within - a *SCRIPT* submitted with ``flux mini batch``, a ``flux mini`` command - will submit a job to the batch instance, not the parent instance under - which the batch instance is running. To target the parent instance, - the ``--parent`` option of :man1:`flux` should be used, e.g:: - - flux --parent mini batch [OPTIONS].. ARGS.. - - -.. include:: common/submit-job-parameters.rst - -.. include:: common/submit-standard-io.rst - -.. include:: common/submit-constraints.rst - -.. include:: common/submit-dependencies.rst - -.. include:: common/submit-environment.rst - -.. include:: common/submit-process-resource-limits.rst - -.. include:: common/submit-exit-status.rst - -.. include:: common/submit-other-options.rst - -.. _bulksubmit: - -BULKSUBMIT -========== - -.. include:: common/submit-bulksubmit.rst - -.. include:: common/submit-shell-options.rst - -.. include:: common/submit-submission-directives.rst - - -RESOURCES -========= - -Flux: http://flux-framework.org - - -SEE ALSO -======== - -:man1:`flux-submit`, :man1:`flux-run`, :man1:`flux-alloc`, :man1:`flux-batch`, -:man1:`flux-bulksubmit` diff --git a/doc/man1/flux-module.rst b/doc/man1/flux-module.rst index aecee8f3c134..0e60568b244b 100644 --- a/doc/man1/flux-module.rst +++ b/doc/man1/flux-module.rst @@ -12,7 +12,9 @@ SYNOPSIS DESCRIPTION =========== -flux-module(1) manages dynamically loadable :man1:`flux-broker` modules. +.. program:: flux module stats + +:program:`flux module` manages dynamically loadable :man1:`flux-broker` modules. COMMANDS @@ -23,13 +25,14 @@ COMMANDS the new module has entered the running state (see LIST OUTPUT below). **remove** [--force] *name* - Remove module *name*. If *-f, --force* is used, then do not fail if - module *name* is not loaded. + Remove module *name*. If :option:`-f, --force` is used, then do not fail + if module *name* is not loaded. **reload** [--force] *name* [*module-arguments* …​] - Reload module *name*. This is equivalent to running *flux module remove* - followed by *flux module load*. It is a fatal error if module *name* is - not loaded during removal unless the ``-f, --force`` option is specified. + Reload module *name*. This is equivalent to running + :program:`flux module remove` followed by :program:`flux module load`. It + is a fatal error if module *name* is not loaded during removal unless the + :option:`-f, --force` option is specified. **list** [*service*] List loaded :man1:`flux-broker` modules. @@ -47,25 +50,31 @@ COMMANDS STATS OPTIONS ============= -**-p, --parse**\ *=OBJNAME* +.. option:: -p, --parse=OBJNAME + OBJNAME is a period delimited list of field names that should be walked to obtain a specific value or object in the returned JSON. -**-t, --type**\ *=int|double* +.. option:: -t, --type=int|double + Force the returned value to be converted to int or double. -**-s, --scale**\ *=N* +.. option:: -s, --scale=N + Multiply the returned (int or double) value by the specified floating point value. -**-R, --rusage** +.. option:: -R, --rusage + Return a JSON object representing an *rusage* structure returned by :linux:man2:`getrusage`. -**-c, --clear** +.. option:: -c, --clear + Send a request message to clear statistics in the target module. -**-C, --clear-all** +.. option:: -C, --clear-all + Broadcast an event message to clear statistics in the target module on all ranks. @@ -73,19 +82,25 @@ STATS OPTIONS DEBUG OPTIONS ============= -**-c, --clear** +.. program:: flux module debug + +.. option:: -c, --clear + Set debug flags to zero. -**-S, --set**\ *=MASK* +.. option:: -S, --set=MASK + Set debug flags to MASK. The value may be prefixed with 0x to indicate hexadecimal or 0 to indicate octal, otherwise the value is interpreted as decimal. -**-c, --clearbit**\ *=MASK* +.. option:: -c, --clearbit=MASK + Clear the debug bits specified in MASK without disturbing other bits. The value is interpreted as above. -**-s, --setbit**\ *=MASK* +.. option:: -s, --setbit=MASK + Set the debug bits specified in MASK without disturbing other bits. The value is interpreted as above. @@ -93,6 +108,8 @@ DEBUG OPTIONS LIST OUTPUT =========== +.. program:: flux module list + The *list* command displays one line for each unique (as determined by SHA1 hash) loaded module. diff --git a/doc/man1/flux-overlay.rst b/doc/man1/flux-overlay.rst index c874ab002985..e02e34931926 100644 --- a/doc/man1/flux-overlay.rst +++ b/doc/man1/flux-overlay.rst @@ -21,7 +21,9 @@ SYNOPSIS DESCRIPTION =========== -``flux-overlay status`` reports the current status of the tree based +.. program:: flux overlay status + +:program:`flux overlay status` reports the current status of the tree based overlay network. The possible status values are: full @@ -113,43 +115,54 @@ Finally, translation between hostnames and broker ranks is accomplished with OPTIONS ======= -``flux-overlay status`` accepts the following options: +:program:`flux overlay status` accepts the following options: + +.. option:: -h, --help -**-h, --help** Display options and exit. -**-r, --rank=[RANK]** +.. option:: -r, --rank=[RANK] + Check health of sub-tree rooted at NODEID (default 0). -**-v, --verbose=[LEVEL]** +.. option:: -v, --verbose=[LEVEL] + Increase reporting detail: 1=show time since current state was entered, 2=show round-trip RPC times. -**-t, --timeout=FSD** +.. option:: -t, --timeout=FSD + Set RPC timeout, 0=disable (default 0.5s) -**--summary** +.. option:: --summary + Show only the root sub-tree status. -**--down** +.. option:: --down + Show only the partial/degraded sub-trees. -**--no-pretty** +.. option:: --no-pretty + Do not indent entries and use line drawing characters to show overlay tree structure -**--no-ghost** +.. option:: --no-ghost + Do not fill in presumed state of nodes that are inaccessible behind offline/lost overlay parents. -**-L, --color=WHEN** +.. option:: -L, --color=WHEN + Colorize output when supported; WHEN can be 'always' (default if omitted), 'never', or 'auto' (default). -**-H, --highlight=TARGET** +.. option:: -H, --highlight=TARGET + Highlight one or more targets and their ancestors. -**-w, --wait=STATE** +.. option:: -w, --wait=STATE + Wait until sub-tree enters *STATE* before reporting (full, partial, offline, degraded, lost)> diff --git a/doc/man1/flux-pgrep.rst b/doc/man1/flux-pgrep.rst index ad7f1ce7de1b..d12141a51665 100644 --- a/doc/man1/flux-pgrep.rst +++ b/doc/man1/flux-pgrep.rst @@ -17,7 +17,9 @@ SYNOPSIS DESCRIPTION =========== -*flux-pgrep* lists jobids that match a supplied expression. The +.. program:: flux pgrep + +:program:`flux pgrep` lists jobids that match a supplied expression. The expression may contain a pattern which matches the job name, or a range of jobids in the form ``jobid1..jobid2``. If both a pattern and jobid range are supplied then both must match. @@ -29,37 +31,45 @@ prefixing the pattern with ``name:``, e.g. ``name:fr..123``. By default, only active jobs for the current user are considered. -*flux-pkill* cancels matching jobs instead of listing them. +:program:`flux pkill` cancels matching jobs instead of listing them. OPTIONS ======= -**-a** +.. option:: -a + Include jobs in all states, including inactive jobs. - This is shorthand for *--filter=pending,running,inactive*. + This is shorthand for :option:`--filter=pending,running,inactive`. (pgrep only) -**-A** - Include jobs for all users. This is shorthand for *--user=-all*. +.. option:: -A + + Include jobs for all users. This is shorthand for :option:`--user=-all`. + +.. option:: -u, --user=USER -**-u, --user**\ *=USER* Fetch jobs only for the given user, instead of the current UID. -**-f, --filter**\ *=STATE|RESULT* +.. option:: -f, --filter=STATE|RESULT + Include jobs with specific job state or result. Multiple states or results can be listed separated by comma. See the JOB STATUS section of the :man1:`flux-jobs` manual for more detail. -**-q, --queue**\ *=QUEUE* +.. option:: -q, --queue=QUEUE + Only include jobs in the named queue *QUEUE*. -**-c, --count**\ *=N* +.. option:: -c, --count=N + Limit output to the first *N* matches (default 1000). -**--max-entries**\ *=N* +.. option:: --max-entries=N + Limit the number of jobs to consider to *N* entries (default 1000). -**-o, --format**\ *=NAME|FORMAT* +.. option:: -o, --format=NAME|FORMAT + Specify a named output format *NAME* or a format string using Python's format syntax. An alternate default format can be set via the FLUX_PGREP_FORMAT_DEFAULT environment variable. For full documentation of @@ -69,10 +79,12 @@ OPTIONS builtin named formats include *default*, *full*, *long*, and *deps*. The default format emits the matched jobids only. (pgrep only) -**-n, --no-header** +.. option:: -n, --no-header + Suppress printing of the header line. (pgrep only) -**-w, --wait** +.. option:: -w, --wait + Wait for jobs to finish after cancel. (pkill only) EXIT STATUS diff --git a/doc/man1/flux-ping.rst b/doc/man1/flux-ping.rst index 7a20fd3613a2..f5f02908df28 100644 --- a/doc/man1/flux-ping.rst +++ b/doc/man1/flux-ping.rst @@ -12,53 +12,61 @@ SYNOPSIS DESCRIPTION =========== -flux-ping(1) measures round-trip latency to a Flux service implementing +.. program:: flux ping + +:program:`flux ping` measures round-trip latency to a Flux service implementing the "ping" method in a manner analogous to ping(8). The ping response is essentially an echo of the request, with the route taken to the service added by the service. This route is displayed in the output and can give insight into how various addresses are routed. *target* may be the name of a Flux service, e.g. "kvs". -flux-ping(1) will send a request to "kvs.ping". As a shorthand, +:program:`flux ping` will send a request to "kvs.ping". As a shorthand, *target* can include a rank or host prefix delimited by an exclamation point. -"flux ping 4!kvs" is equivalent to "flux ping --rank 4 kvs" (see --rank -option below). Don't forget to quote the exclamation point if it is -interpreted by your shell. +:program:`flux ping 4!kvs` is equivalent to :option:`flux ping --rank 4 kvs` +(see :option:`--rank` option below). Don't forget to quote the exclamation +point if it is interpreted by your shell. As a shorthand, *target* may also simply be a rank or host by itself indicating that the broker on that rank/host, rather than a Flux -service, is to be pinged. "flux ping 1" is equivalent to -"flux ping --rank 1 broker". +service, is to be pinged. :command:`flux ping 1` is equivalent to +:option:`flux ping --rank 1 broker`. OPTIONS ======= -**-r, --rank**\ *=N* +.. option:: -r, --rank=N + Find target on a specific broker rank. Special case strings “*any*” and “*upstream*” available to ping FLUX_NODEID_ANY and FLUX_NODEID_UPSTREAM respectively. Default: send to “*any*”. -**-p, --pad**\ *=N* +.. option:: -p, --pad=N + Include in the payload a string of length *N* bytes. *N* may be a floating point number with optional multiplicative suffix k,K=1024, M=1024\*1024, or G=1024\*1024\*1024. The payload will be echoed back in the response. This option can be used to explore the effect of message size on latency. Default: no padding. -**-i, --interval**\ *=N* +.. option:: -i, --interval=Ns + Specify the delay, in seconds, between successive requests. A value of zero is valid and indicates that there should be no delay. Requests are sent without waiting for responses. Default: 1.0 seconds. -**-c, --count**\ *=N* +.. option:: -c, --count=N + Specify the number of requests to send, and terminate the command once responses have been received for all the requests. Default: unlimited. -**-b, --batch** +.. option:: -b, --batch + Begin processing responses after all requests are sent. Requires --count. -**-u, --userid** +.. option:: -u, --userid + Include userid and rolemask of original request, which are echoed back in ping response, in ping output. diff --git a/doc/man1/flux-proxy.rst b/doc/man1/flux-proxy.rst index 159273655bd7..7bfa4906d64f 100644 --- a/doc/man1/flux-proxy.rst +++ b/doc/man1/flux-proxy.rst @@ -15,7 +15,9 @@ SYNOPSIS DESCRIPTION =========== -**flux proxy** connects to the Flux instance identified by *TARGET*, +.. program:: flux proxy + +:program:`flux proxy` connects to the Flux instance identified by *TARGET*, then spawns a shell with FLUX_URI pointing to a local:// socket managed by the proxy program. As long as the shell is running, the proxy program routes messages between the instance and the @@ -28,14 +30,14 @@ URI, or a resolvable URI with a scheme supported by a ``flux uri`` plugin. See :man1:`flux-uri` for details. If the connection to the Flux instance is lost, for example when the -target instance terminates, **flux proxy** will emit an error message, +target instance terminates, :program:`flux proxy` will emit an error message, send ``SIGHUP`` and ``SIGCONT`` to the spawned shell or other process, and wait for it to terminate before exiting. The delivery of signals -can be disabled with the ``-n, --nohup`` option, but be aware that Flux +can be disabled with the :option:`--nohup` option, but be aware that Flux commands running under a **flux proxy** which has lost its connection will likely result in errors. -The purpose of **flux proxy** is to allow a connection to be reused, +The purpose of :program:`flux proxy` is to allow a connection to be reused, for example where connection establishment has high latency or requires authentication. @@ -43,20 +45,23 @@ requires authentication. OPTIONS ======= -**-f, --force** +.. option:: -f, --force + Allow the proxy command to connect to a broker running a different version of Flux with a warning message instead of a fatal error. -**-n, --nohup** - When an error occurs in the proxy connection, **flux proxy** will +.. option:: -n, --nohup + + When an error occurs in the proxy connection, :program:`flux proxy` will normally shut down the proxy and send ``SIGHUP`` and ``SIGCONT`` to - the spawned shell or command. If the ``-n, --nohup`` option is used, + the spawned shell or command. If the :option:`--nohup` option is used, the ``SIGHUP`` and ``SIGCONT`` signals will not be sent. - **flux proxy** will still wait for the spawned shell or command to + :program:`flux proxy` will still wait for the spawned shell or command to exit before terminating to avoid having the child process reparented and possibly lose its controlling tty. -**--reconnect** +.. option:: --reconnect + If broker communication fails, drop the current connection and try to reconnect every 2 seconds until the connection succeeds. Any event subscriptions and service registrations that were made on behalf of diff --git a/doc/man1/flux-pstree.rst b/doc/man1/flux-pstree.rst index 4cd91af5a524..69041612c64e 100644 --- a/doc/man1/flux-pstree.rst +++ b/doc/man1/flux-pstree.rst @@ -14,26 +14,28 @@ SYNOPSIS DESCRIPTION =========== -flux-pstree(1) displays a tree of running jobs by job name, similar to +.. program:: flux pstree + +:program:`flux pstree` displays a tree of running jobs by job name, similar to what the :linux:man1:`pstree` command does for system processes. -Like pstree(1), identical leaves of the job tree are combined, which +Like :command:`pstree`, identical leaves of the job tree are combined, which results in a more compact output when many jobs within a Flux instance share the same job name. -The flux-pstree(1) command supports custom labels for jobs, including +The :program:`flux pstree` command supports custom labels for jobs, including separately labeling parent jobs, using the same format string syntax supported by :man1:`flux-jobs`. -The command lists actively running jobs by default, but a ``-a, --all`` +The command lists actively running jobs by default, but a :option:`--all` option lists all jobs in all states for the current user. In the case -that ``-a`` is used, the job labels will automatically be amended to +that :option:`-all` is used, the job labels will automatically be amended to include the job status (i.e. ``{name}:{status_abbrev}``), though this can be overridden on the command line. -The flux-pstree(1) command additionally supports listing extended -job information before the tree display with the ``-x, --extended``, -``-d, --details=NAME``, or ``--detail-format=FORMAT`` options, e.g. +The :program:`flux pstree` command additionally supports listing extended +job information before the tree display with the :option:`--extended`, +:option:`--details=NAME`, or :option:`--detail-format=FORMAT` options, e.g. :: @@ -46,7 +48,7 @@ job information before the tree display with the ``-x, --extended``, ƒe1j54K user1 R 1 1 8.539s └── flux ƒ2MYrwzf user1 R 1 1 4.736s └── sleep -Several detail formats are available via the ``-d, --details=NAME`` +Several detail formats are available via the :option:`-d, --details=NAME` option, including progress, resources, and stats. For example, the ``progress`` display attempts to show the overall progress and utilization of all Flux instances in a hierarchy by displaying the @@ -68,89 +70,105 @@ resources divided by total available resources): ƒ2b6cPMS 1 0:00:01 └── sleep -By default, flux-pstree(1) truncates lines that exceed the current +By default, :program:`flux pstree` truncates lines that exceed the current value of the ``COLUMNS`` environment variable or the terminal width -if ``COLUMNS`` is not set. To disable truncation, use the ``-l, --long`` +if ``COLUMNS`` is not set. To disable truncation, use the :option:`--long` option. By default, the enclosing Flux instance, or root of the tree, is included in output, unless extended details are displayed as when any of the -``-x, --extended``, ``-d, --details=NAME`` or ``--detail-format=FORMAT`` -options are used, or if one or more jobids are directly targeted with -a ``JOBID`` argument. This behavior can be changed via the -``--skip-root=[yes|no]`` option. +:option:`--extended`, :option:`--details=NAME` or +:option:`--detail-format=FORMAT` options are used, or if one or more jobids +are directly targeted with a ``JOBID`` argument. This behavior can be changed +via the :option:`--skip-root=[yes|no]` option. OPTIONS ======= -**-a, --all** +.. option:: -a, --all + Include jobs in all states, including inactive jobs. - This is shorthand for *--filter=pending,running,inactive*. + This is shorthand for :option:`--filter=pending,running,inactive`. + +.. option:: -c, --count=N -**-c, --count**\ *=N* Limit output to N jobs at every level (default 1000). -**-f, --filter**\ *=STATE|RESULT* +.. option:: -f, --filter=STATE|RESULT + Include jobs with specific job state or result. Multiple states or - results can be listed separated by comma. See the JOB STATUS section - of the :man1:`flux-jobs` manual for more detail. + results can be listed separated by comma. See the :ref:`flux_jobs_job_status` + section of the :man1:`flux-jobs` manual for more detail. + +.. option:: -l, --long -**-l, --long** Do not truncate long lines at ``COLUMNS`` characters. -**-p, --parent-ids** +.. option:: -p, --parent-ids + Prepend jobid to parent labels. -**-L, --level** *=N* +.. option:: -L, --level=N + Only descend *N* levels of the job hierarchy. -**-x, --extended** +.. option:: -x, --extended + Print extended details before tree output. This is the same as - ``--details=default``. + :option:`--details=default`. + +.. option:: -d, --detail=NAME -**-d, --detail**\ *=NAME* Select a named extended details format. The list of supported names - can be seen in ``flux pstree --help`` output. + can be seen in :option:`flux pstree --help` output. + +.. option:: -n, --no-header -**-n, --no-header** For output with extended details, do not print header row. -**-X, --no-combine** +.. option:: -X, --no-combine + Typically, identical child jobs that are leaves in the tree display are combined as ``n*[label]``. With this option, the combination of like jobs is disabled. -**-o, --label**\ *=FORMAT* +.. option:: -o, --label=FORMAT + Specify output format for node labels using Python format strings. Supports all format fields supported by :man1:`flux-jobs`. -**--parent-label**\ *=FORMAT* +.. option:: --parent-label=FORMAT + Label tree parents with a different format than child jobs. -**--detail-format**\ *=FORMAT* +.. option:: --detail-format=FORMAT + Specify an explicit details format to display before the tree part. Care should be taken that each line of the format is the same width to ensure that the tree display is rendered correctly (i.e. by judicious use of format field widths, e.g. ``{id.f58:>12}`` instead of just ``{id.f58}``. -**--skip-root**\ *=yes|no* +.. option:: --skip-root=yes|no + Explicitly skip (yes) or force (no) display of the enclosing instance, or root of the tree, in output. -**-C, --compact** +.. option:: -C, --compact + Use compact tree connectors. Usefully for deep hierarchies. -**--ascii** +.. option:: --ascii + Use ascii tree connectors. EXAMPLES ======== -The default output of flux-pstree(1) shows all running jobs for the +The default output of :program:`flux pstree` shows all running jobs for the current user by name, including any running sub-jobs. If there are currently no running jobs for the current user, only the enclosing instance is displayed as a ``.``, to indicate the root of the tree: diff --git a/doc/man1/flux-queue.rst b/doc/man1/flux-queue.rst index bf3d1a7ee031..707c546b4f85 100644 --- a/doc/man1/flux-queue.rst +++ b/doc/man1/flux-queue.rst @@ -28,99 +28,112 @@ SYNOPSIS DESCRIPTION =========== -The ``flux-queue`` command controls Flux job queues. +.. program:: flux queue + +The :program:`flux queue` command controls Flux job queues. Normally, Flux has a single anonymous queue, but when queues are configured, all queues are named. At this time, the *disable*, *enable*, *stop*, *start*, and *status* subcommands can be applied to a single, named queue. The rest affect all queues. -``flux-queue`` has the following subcommands: +:program:`flux queue` has the following subcommands: disable Prevent jobs from being submitted to the queue, with a reason that is shown to submitting users. If multiple queues are configured, either the - *--queue* or the *--all* option is required. + :option:`--queue` or the :option:`--all` option is required. enable Allow jobs to be submitted to the queue. If multiple queues are configured, - either the *--queue* or the *--all* option is required. + either the :option:`--queue` or the :option:`--all` option is required. status Report the current queue status. If multiple queues are configured, - all queues are shown unless one is specified with *--queue*. + all queues are shown unless one is specified with :option:`--queue`. stop Stop allocating resources to jobs. Pending jobs remain enqueued, and running jobs continue to run, but no new jobs are allocated - resources. If multiple queues are configured, either the *--queue* - or the *--all* option is required. + resources. If multiple queues are configured, either the :option:`--queue` + or the :option:`--all` option is required. start Start allocating resources to jobs. If multiple queues are - configured, either the *--queue* or the *--all* option is required. + configured, either the :option:`--queue` or the :option:`--all` option is + required. drain Block until all queues become empty. It is sometimes useful to run after - ``flux queue disable``, to wait until the system is quiescent and can be - taken down for maintenance. + :program:`flux queue disable`, to wait until the system is quiescent and can + be taken down for maintenance. idle Block until all queues become `idle` (no jobs in RUN or CLEANUP state, and no outstanding alloc requests to the scheduler). It may be useful to run - after ``flux queue stop`` to wait until the scheduler and execution system - are quiescent before maintenance involving them. + after :program:`flux queue stop` to wait until the scheduler and execution + system are quiescent before maintenance involving them. list - Show queue defaults and limits. The *-n,--no-header* option suppresses header - from output, *-o,--format=FORMAT*, customizes output formatting (see below). + Show queue defaults and limits. The :option:`--no-header` option suppresses + header from output, :option:`--format=FORMAT`, customizes output formatting + (see below). OPTIONS ======= -**-h, --help** +.. option:: -h, --help + Summarize available options. -**-q, --queue**\ =\ *NAME* +.. option:: -q, --queue=NAME + Select a queue by name. -**-v, --verbose** +.. option:: -v, --verbose + Be chatty. -**--quiet** +.. option:: --quiet + Be taciturn. -**-a, --all** +.. option:: -a, --all + Use with *enable*, *disable*, *stop*, or *start* subcommands to signify intent to affect all queues, when queues are configured but - *--queue* is missing. + :option:`--queue` is missing. + +.. option:: --nocheckpoint -**--nocheckpoint** Use with *stop*, to not checkpoint that a queue has been stopped. This is often used when tearing down a flux instance, so that the a queue's start state is not assumed to be stopped on a restart. -**--timeout** \ =\ *FSD* +.. option:: --timeout=FSD + Limit the time that ``drain`` or ``idle`` will block. -**-n, --no-header** +.. option:: -n, --no-header + Do not output column headers in ``list`` output. -**-o, --format** \ =\ *FORMAT* +.. option:: -o, --format=FORMAT + Specify output format in ``list`` using Python's string format syntax. - See OUTPUT FORMAT below for field names. + See `OUTPUT FORMAT`_ below for field names. OUTPUT FORMAT ============= -The *--format* option can be used to specify an output format using Python's -string format syntax or a defined format by name. For a list of built-in and -configured formats use ``-o help``. An alternate default format can be set via -the FLUX_QUEUE_LIST_FORMAT_DEFAULT environment variable. A configuration -snippet for an existing named format may be generated with -``--format=get-config=NAME``. See :man1:`flux-jobs` *OUTPUT FORMAT* section for -a detailed description of this syntax. +The :option:`--format` option can be used to specify an output format using +Python's string format syntax or a defined format by name. For a list of +built-in and configured formats use :option:`-o help`. An alternate default +format can be set via the FLUX_QUEUE_LIST_FORMAT_DEFAULT environment variable. +A configuration snippet for an existing named format may be generated with +:option:`--format=get-config=NAME`. See :man1:`flux-jobs` *OUTPUT FORMAT* +section for a detailed description of this syntax. The following field names can be specified: diff --git a/doc/man1/flux-resource.rst b/doc/man1/flux-resource.rst index 74dae2b2cf44..b3b4fd113c3e 100644 --- a/doc/man1/flux-resource.rst +++ b/doc/man1/flux-resource.rst @@ -14,19 +14,19 @@ SYNOPSIS DESCRIPTION =========== -flux-resource(1) lists and manipulates Flux resources. The resource inventory -is maintained and monitored by the resource service. The scheduler acquires -a subset of resources from the resource service to allocate to jobs, and relies -on the resource service to inform it of status changes that affect the -usability of resources by jobs as described in RFC 27. +:program:`flux resource` lists and manipulates Flux resources. The resource +inventory is maintained and monitored by the resource service. The scheduler +acquires a subset of resources from the resource service to allocate to jobs, +and relies on the resource service to inform it of status changes that affect +the usability of resources by jobs as described in RFC 27. -The flux-resource(1) **list** subcommand queries the scheduler for its view +The :program:`flux resource list` subcommand queries the scheduler for its view of resources, including allocated/free status. -The other flux-resource(1) subcommands operate on the resource service and -are primarily of interest to system administrators of a Flux system instance. -For example, they can show whether or not a node is booted, and may be used to -administratively drain and undrain nodes. +The other :program:`flux resource` subcommands operate on the resource service +and are primarily of interest to system administrators of a Flux system +instance. For example, they can show whether or not a node is booted, and may +be used to administratively drain and undrain nodes. A few notes on drained nodes: @@ -35,9 +35,9 @@ A few notes on drained nodes: - When an instance is restarted, drained nodes remain drained. - The scheduler may determine that a job request is *feasible* if the total resource set, including drained nodes, would allow it to run. -- In ``flux resource status`` and ``flux resource drain``, the drain state - of a node will be presented as "drained" if the node has no job allocations, - and "draining" if there are still jobs running on the node. +- In :program:`flux resource status` and :program:`flux resource drain`, the + drain state of a node will be presented as "drained" if the node has no job + allocations, and "draining" if there are still jobs running on the node. - If a node is drained and offline, then "drained*" will be displayed. Some further background on resource service operation may be found in the @@ -50,49 +50,49 @@ COMMANDS **list** [-n] [-o FORMAT] [-s STATE,...] [-i TARGETS] Show scheduler view of resources. - With *-s,--states=STATE,...*, the set of resource states is restricted - to a list of provided states. Valid states include "up", "down", + With :option:`-s,--states=STATE,...`, the set of resource states is + restricted to a list of provided states. Valid states include "up", "down", "allocated", "free", and "all". Note that the scheduler represents offline, excluded, and drained resources as "down" due to the simplified interface with the resource service defined by RFC 27. - With *-i, --include=TARGETS*, the results are filtered to only include - resources matching **TARGETS**, which may be specified either as an idset - of broker ranks or list of hosts in hostlist form. It is not an error to - specify ranks or hosts which do not exist, the result will be filtered + With :option:`-i, --include=TARGETS`, the results are filtered to only + include resources matching **TARGETS**, which may be specified either as an + idset of broker ranks or list of hosts in hostlist form. It is not an error + to specify ranks or hosts which do not exist, the result will be filtered to include only those ranks or hosts that are present in *TARGETS*. - The *-o,--format=FORMAT* option may be used to customize the output - format (See OUTPUT FORMAT section below). + The :option:`-o,--format=FORMAT` option may be used to customize the output + format (See `OUTPUT FORMAT`_ section below). - The *-n,--no-header* option suppresses header from output, + The :option:`-n,--no-header` option suppresses header from output, **info** [-s STATE,...] [-i TARGETS] Show a brief, single line summary of scheduler view of resources. - With *-s, --states=STATE,...*, limit the output to specified resource - states as with ``flux resource list``. By default, the *STATE* reported - by ``flux resource info`` is "all". + With :option:`-s, --states=STATE,...`, limit the output to specified + resource states as with :program:`flux resource list`. By default, the + *STATE* reported by :program:`flux resource info` is "all". - With *-i, --include=TARGETS*, the results are filtered to only include - resources matching **TARGETS**, which may be specified either as an idset - of broker ranks or list of hosts in hostlist form. It is not an error to - specify ranks or hosts which do not exist, the result will be filtered - to include only those ranks or hosts that are present in *TARGETS*. + With :option:`-i, --include=TARGETS`, the results are filtered to only + include resources matching **TARGETS**, which may be specified either as + an idset of broker ranks or list of hosts in hostlist form. It is not an + error to specify ranks or hosts which do not exist, the result will be + filtered to include only those ranks or hosts that are present in *TARGETS*. **R** [-s STATE,..] [-i TARGETS] Emit an RFC 20 Resource Set on stdout. - With *-s,--states=STATE,...*, the set of resource states is restricted - to a list of provided states. Valid states include "up", "down", + With :option:`-s,--states=STATE,...`, the set of resource states is + restricted to a list of provided states. Valid states include "up", "down", "allocated", "free", and "all". Note that the scheduler represents offline, excluded, and drained resources as "down" due to the simplified interface with the resource service defined by RFC 27. - With *-i, --include=TARGETS*, the results are filtered to only include - resources matching **TARGETS**, which may be specified either as an idset - of broker ranks or list of hosts in hostlist form. It is not an error to - specify ranks or hosts which do not exist, the result will be filtered + With :option:`-i, --include=TARGETS`, the results are filtered to only + include resources matching **TARGETS**, which may be specified either as an + idset of broker ranks or list of hosts in hostlist form. It is not an error + to specify ranks or hosts which do not exist, the result will be filtered to include only those ranks or hosts that are present in *TARGETS*. **status** [-n] [-o FORMAT] [-s STATE,...] [-i TARGETS] [--skip-empty] @@ -106,51 +106,54 @@ COMMANDS (excluded by configuration), "draining" (drained but still allocated), or "drained". - With *-s,--states=STATE,...*, the set of resource states is restricted - to a list of provided states or offline/online status. With "online" or - "offline", only nodes with the provided status will be displayed. Other - valid states include "avail", "exclude", "draining", "drained", and "all". - The special "drain" state is shorthand for "drained,draining". + With :option:`-s,--states=STATE,...`, the set of resource states is + restricted to a list of provided states or offline/online status. With + "online" or "offline", only nodes with the provided status will be + displayed. Other valid states include "avail", "exclude", "draining", + "drained", and "all". The special "drain" state is shorthand for + "drained,draining". - With *-i, --include=TARGETS*, the results are filtered to only include - resources matching **TARGETS**, which may be specified either as an idset - of broker ranks or list of hosts in hostlist form. It is not an error to - specify ranks or hosts which do not exist, the result will be filtered - to include only those ranks or hosts that are present in *TARGETS*. + With :option:`-i, --include=TARGETS`, the results are filtered to only + include resources matching **TARGETS**, which may be specified either as + an idset of broker ranks or list of hosts in hostlist form. It is not an + error to specify ranks or hosts which do not exist, the result will be + filtered to include only those ranks or hosts that are present in *TARGETS*. - The *-o,--format=FORMAT* option customizes output formatting (See the - OUTPUT FORMAT section below for details). + The :option:`-o,--format=FORMAT` option customizes output formatting (See + the `OUTPUT FORMAT`_ section below for details). - With *-n,--no-header* the output header is suppressed. + With :option:`-n,--no-header` the output header is suppressed. - Normally, ``flux resource status`` skips lines with no resources, - unless the ``-s, --states`` option is used. Suppression of empty lines - can may be forced with the ``--skip-empty`` option. + Normally, :program:`flux resource status` skips lines with no resources, + unless the :option:`-s, --states` option is used. Suppression of empty lines + can may be forced with the :option:`--skip-empty` option. **drain** [-n] [-o FORMAT] [-i TARGETS] [-f] [-u] [targets] [reason ...] If specified without arguments, list drained nodes. In this mode, - *-n,--no-header* suppresses header from output and *-o,--format=FORMAT* - customizes output formatting (see below). The *targets* argument is an - IDSET or HOSTLIST specifying nodes to drain. Any remaining arguments - are assumed to be a reason to be recorded with the drain event. - - With *-i, --include=TARGETS*, **drain** output is filtered to only include - resources matching **TARGETS**, which may be specified either as an idset - of broker ranks or list of hosts in hostlist form. It is not an error to - specify ranks or hosts which do not exist, the result will be filtered + :option:`-n,--no-header` suppresses header from output and + :option:`-o,--format=FORMAT` customizes output formatting (see below). + The *targets* argument is an IDSET or HOSTLIST specifying nodes to drain. + Any remaining arguments are assumed to be a reason to be recorded with the + drain event. + + With :option:`-i, --include=TARGETS`, **drain** output is filtered to only + include resources matching **TARGETS**, which may be specified either as an + idset of broker ranks or list of hosts in hostlist form. It is not an error + to specify ranks or hosts which do not exist, the result will be filtered to include only those ranks or hosts that are present in *TARGETS*. - By default, **flux resource drain** will fail if any of the *targets* + By default, :program:`flux resource drain` will fail if any of the *targets* are already drained. To change this behavior, use either of the - *-f, --force* or *-u, --update* options. With *--force*, the *reason* for - all existing drained targets is overwritten. If *--force* is specified - twice, then the timestamp is also overwritten. With *--update*, - only those ranks that are not already drained or do not have a *reason* set - have their *reason* updated. + :option:`-f, --force` or :option:`-u, --update` options. With + :option:`--force`, the *reason* for all existing drained targets is + overwritten. If :option:`--force` is specified twice, then the timestamp is + also overwritten. With :option:`--update`, only those ranks that are not + already drained or do not have a *reason* set have their *reason* updated. - Resources cannot be both excluded and drained, so **flux resource drain** - will also fail if any *targets* are currently excluded by configuration. - There is no option to force an excluded node into the drain state. + Resources cannot be both excluded and drained, so + :program:`flux resource drain` will also fail if any *targets* are + currently excluded by configuration. There is no option to force an + excluded node into the drain state. This command, when run with arguments, is restricted to the Flux instance owner. @@ -161,23 +164,26 @@ COMMANDS **reload** [-x] [-f] PATH Reload the resource inventory from a file in RFC 20 format, or if the - *-x,--xml* option, a directory of hwloc ``.xml`` files. If - *-f,--force*, resources may contain invalid ranks. This command is + :option:`-x,--xml` option, a directory of hwloc ``.xml`` files. If + :option:`-f,--force`, resources may contain invalid ranks. This command is primarily used in test. OUTPUT FORMAT ============= -The *--format* option can be used to specify an output format using Python's -string format syntax or a defined format by name. For a list of built-in and -configured formats use ``-o help``. An alternate default format can be set via -the FLUX_RESOURCE_STATUS_FORMAT_DEFAULT, FLUX_RESOURCE_DRAIN_FORMAT_DEFAULT, and -FLUX_RESOURCE_LIST_FORMAT_DEFAULT environment variables (for ``flux resource -status``, ``flux resource drain``, and ``flux resource list`` respectively). A -configuration snippet for an existing named format may be generated with -``--format=get-config=NAME``. See :man1:`flux-jobs` *OUTPUT FORMAT* section for -a detailed description of this syntax. +The :option:`--format` option can be used to specify an output format using +Python's string format syntax or a defined format by name. For a list of +built-in and configured formats use :option:`-o help`. An alternate default +<<<<<<< HEAD +format can be set via the FLUX_RESOURCE_STATUS_FORMAT_DEFAULT, +FLUX_RESOURCE_DRAIN_FORMAT_DEFAULT, and FLUX_RESOURCE_LIST_FORMAT_DEFAULT +environment variables (for :program:`flux resource status`, +:program:`flux resource drain`, and :program:`flux resource list` +respectively). A configuration snippet for an existing named format may be +generated with :option:`--format=get-config=NAME`. See :man1:`flux-jobs` +:ref:`flux_jobs_output_format` section for a detailed description of this +syntax. Resources are combined into a single line of output when possible depending on the supplied output format. Resource counts are not included in the @@ -262,13 +268,13 @@ The following field names can be specified for the **list** subcommand: CONFIGURATION ============= -Similar to :man1:`flux-jobs`, the ``flux-resource`` command supports loading -a set of config files for customizing utility output formats. Currently +Similar to :man1:`flux-jobs`, the :program:`flux resource` command supports +loading a set of config files for customizing utility output formats. Currently this can be used to register named format strings for the ``status``, ``list``, and ``drain`` subcommands. -Configuration for each ``flux-resource`` subcommand is defined in a separate -table, so to add a new format ``myformat`` for ``flux resource list``, +Configuration for each :program:`flux resource` subcommand is defined in a +separate table, so to add a new format ``myformat`` for ``flux resource list``, the following config file could be used:: # $HOME/.config/flux/flux-resource.toml @@ -276,8 +282,8 @@ the following config file could be used:: description = "My flux resource list format" format = "{state} {nodelist}" -See :man1:`flux-jobs` *CONFIGURATION* section for more information about the -order of precedence for loading these config files. +See :man1:`flux-jobs` :ref:`flux_jobs_configuration` section for more +information about the order of precedence for loading these config files. RESOURCE INVENTORY ================== diff --git a/doc/man1/flux-restore.rst b/doc/man1/flux-restore.rst index 223c592cf0f3..02c975dcc9e5 100644 --- a/doc/man1/flux-restore.rst +++ b/doc/man1/flux-restore.rst @@ -12,48 +12,57 @@ SYNOPSIS DESCRIPTION =========== -The ``flux-restore`` command reads a KVS snapshot from a portable archive -format, usually written by :man1:`flux-dump`. +.. program:: flux restore + +The :program:`flux restore` command reads a KVS snapshot from a portable +archive format, usually written by :man1:`flux-dump`. The archive source may be specified as a file path or *-* for standard input. The format of the archive may be any of the formats supported by :linux:man3:`libarchive` and is determined on the fly based on the archive content. -The snapshot may be restored to a KVS key if *--key=NAME* is used and the -KVS service is running, or as a checkpoint in the content backing store -if *--checkpoint* is used, without the KVS running. One of those two options -is required. +The snapshot may be restored to a KVS key if :option:`--key=NAME` is used and +the KVS service is running, or as a checkpoint in the content backing store +if :option:`--checkpoint` is used, without the KVS running. One of those two +options is required. OPTIONS ======= -**-h, --help** +.. option:: -h, --help + Summarize available options. -**-v, --verbose** +.. option:: -v, --verbose + List keys on stderr as they are restored instead of a periodic count of restored keys. -**-q, --quiet** +.. option:: -q, --quiet + Don't show a periodic count of restored keys on stderr. -**--checkpoint** +.. option:: --checkpoint + After restoring the archived content, write the final root blobref to the KVS checkpoint area in the content backing store. The checkpoint is used as the initial KVS root when the KVS module is loaded. Unload the KVS module before restoring with this option. -**--key**\ =\ *NAME* +.. option:: --key=NAME + After restoring the archived content, write the final root blobref to a KVS key, so the key becomes the restored root directory. -**--no-cache** +.. option:: --no-cache + Bypass the broker content cache and interact directly with the backing store. Performance will vary depending on the content of the archive. -**--size-limit**\ =\ *SIZE* +.. option:: --size-limit=SIZE + Skip restoring keys that exceed SIZE bytes (default: no limit). SIZE may be specified as a floating point number with an optional multiplicative suffix k or K=1024, M=1024\*1024, or G=1024\*1024\*1024 (up to diff --git a/doc/man1/flux-run.rst b/doc/man1/flux-run.rst index 0b6a310c722e..fd810bfb402e 100644 --- a/doc/man1/flux-run.rst +++ b/doc/man1/flux-run.rst @@ -15,11 +15,13 @@ SYNOPSIS DESCRIPTION =========== -flux-run(1) submits a job to run interactively under Flux, blocking until -the job has completed. The job consists of *N* copies of COMMAND launched -together as a parallel job. +.. program:: flux run -If *--ntasks* is unspecified, a value of *N=1* is assumed. +:program:`flux run` submits a job to run interactively under Flux, blocking +until the job has completed. The job consists of *N* copies of COMMAND +launched together as a parallel job. + +If :option:`--ntasks` is unspecified, a value of *N=1* is assumed. The available OPTIONS are detailed below. diff --git a/doc/man1/flux-shell.rst b/doc/man1/flux-shell.rst index 348f935bbd52..4ed0b0678086 100644 --- a/doc/man1/flux-shell.rst +++ b/doc/man1/flux-shell.rst @@ -11,10 +11,12 @@ SYNOPSIS DESCRIPTION =========== -flux-shell(1), the Flux job shell, is the component of Flux which manages -the startup and execution of user jobs. flux-shell(1) runs as the job user, -reads the jobspec and assigned resource set R for the job from the KVS, -and using this data determines what local job tasks to execute. While +.. program:: flux shell + +:program:`flux shell`, the Flux job shell, is the component of Flux which +manages the startup and execution of user jobs. :program:`flux shell` runs as +the job user, reads the jobspec and assigned resource set R for the job from +the KVS, and using this data determines what local job tasks to execute. While job tasks are running, the job shell acts as the interface between the Flux instance and the job by handling standard I/O, signals, and finally collecting the exit status of tasks as they complete. @@ -22,47 +24,49 @@ collecting the exit status of tasks as they complete. The design of the Flux job shell allows customization through a set of builtin and runtime loadable shell plugins. These plugins are used to handle standard I/O redirection, PMI, CPU and GPU affinity, debugger -support and more. Details of the flux-shell(1) plugin capabilities and -design can be found in the PLUGINS section below. +support and more. Details of the :program:`flux shell` plugin capabilities and +design can be found in the `PLUGINS`_ section below. -flux-shell(1) also supports configuration via a Lua-based configuration +:program:`flux shell` also supports configuration via a Lua-based configuration file, called the shell ``initrc``, from which shell plugins may be loaded -or shell options and data examined or set. The flux-shell(1) initrc may +or shell options and data examined or set. The :program:`flux shell` initrc may even extend the shell itself via simple shell plugins developed directly -in Lua. See the SHELL INITRC section below for details of the ``initrc`` +in Lua. See the `SHELL INITRC`_ section below for details of the ``initrc`` format and features. OPTIONS ======= -**-h, --help** +.. option:: -h, --help + Summarize available options. -**--reconnect** +.. option:: --reconnect + Attempt to reconnect if broker connection is lost. OPERATION ========= -When a job has been granted resources by a Flux instance, a flux-shell(1) -process is invoked on each broker rank involved in the job. The job -shell runs as the job user, and will always have ``FLUX_KVS_NAMESPACE`` -set such that the root of the job shell's KVS accesses will be the guest -namespace for the job. +When a job has been granted resources by a Flux instance, a +:program:`flux shell` process is invoked on each broker rank involved in the +job. The job shell runs as the job user, and will always have +``FLUX_KVS_NAMESPACE`` set such that the root of the job shell's +KVS accesses will be the guest namespace for the job. -Each flux-shell(1) connects to the local broker, fetches the jobspec and -resource set **R** for the job from the job-info module, and uses this +Each :program:`flux shell` connects to the local broker, fetches the jobspec +and resource set **R** for the job from the job-info module, and uses this information to plan which tasks to locally execute. Once the job shell has successfully gathered job information, the -flux-shell(1) then goes through the following general steps to manage +:program:`flux shell` then goes through the following general steps to manage execution of the job: * register service endpoint specific to the job and userid, typically ``-shell-`` * load the system default ``initrc.lua`` (``$sysconfdir/flux/shell/initrc.lua``), unless overridden by - configuration (See JOBSPEC OPTIONS and INITRC sections below) + configuration (See `JOBSPEC OPTIONS`_ and `SHELL INITRC`_ sections below) * call ``shell.init`` plugin callbacks * change working directory to the cwd of the job * enter a barrier to ensure shell initialization is complete on all shells @@ -113,20 +117,21 @@ These callbacks are defined by "topic strings" to which plugins can the shell option is set before the builtin attempts to read them. Simple plugins may also be developed directly in the shell ``initrc.lua`` -file itself (see INITRC section, ``plugin.register()`` below) +file itself (see `SHELL INITRC`_ section, ``plugin.register()`` below) -By default, flux-shell supports the following plugin callback topics: +By default, :program:`flux shell` supports the following plugin callback +topics: **taskmap.SCHEME** Called when a taskmap scheme *SCHEME* is requested via the taskmap - shell option or corresponding ``--taskmap`` option of :man1:`flux-submit` - and related commands. Plugins that want to offer a different taskmap - scheme than the defaults of ``block``, ``cyclic``, and ``manual`` can - register a ``taskmap.*`` plugin callback and then users can request this - mapping with the appropriate ``--taskmap=name`` option. The default block - taskmap is passed to the plugin as "taskmap" in the plugin input arguments, - and the plugin should return the new taskmap as a string in the output args. - This callback is called before ``shell.init``. + shell option or corresponding :option:`flux submit --taskmap` option. + Plugins that want to offer a different taskmap scheme than the defaults of + ``block``, ``cyclic``, and ``manual`` can register a ``taskmap.*`` plugin + callback and then users can request this mapping with the appropriate + :option:`flux submit --taskmap=name`` option. The default block taskmap is + passed to the plugin as "taskmap" in the plugin input arguments, and the + plugin should return the new taskmap as a string in the output args. This + callback is called before ``shell.init``. **shell.connect** Called just after the shell connects to the local Flux broker. (Only @@ -148,7 +153,7 @@ By default, flux-shell supports the following plugin callback topics: **task.fork** Called for each task after the task if forked from the parent - process (flux-shell process) + process (:program:`flux shell` process) **task.exit** Called for each task after it exits and wait_status is available. @@ -174,10 +179,10 @@ available in a given shell instance. JOBSPEC OPTIONS =============== -On startup, ``flux-shell`` will examine the jobspec for any shell specific -options under the ``attributes.system.shell.options`` key. These options -may be set by the :man1:`flux-submit` and related commands ``-o, --setopt=OPT`` -option, or explicitly added to the jobspec by other means. +On startup, :program:`flux shell` will examine the jobspec for any shell +specific options under the ``attributes.system.shell.options`` key. These +options may be set by the :option:`flux submit -o, --setopt=OPT` option, +or explicitly added to the jobspec by other means. Job shell options may be switches to enable or disable a shell feature or plugin, or they may take an argument. Because jobspec is a JSON document, @@ -185,9 +190,9 @@ job shell options in jobspec may take arguments that are themselves JSON objects. This allows maximum flexibility in runtime configuration of optional job shell behavior. In the list below, if an option doesn't include a ``=``, then it is a simple boolean option or switch and may be -specified simply with ``-o option`` in commands like :man1:`flux run`. +specified simply with :option:`flux submit -o OPTION`. -Options supported by ``flux-shell`` proper include: +Options supported by :program:`flux shell` proper include: **verbose**\ =\ *INT* Set the shell verbosity to *INT*. A larger value indicates increased @@ -200,14 +205,14 @@ Options supported by ``flux-shell`` proper include: delivered only to direct children of the shell. **initrc**\ =\ *FILE* - Load flux-shell initrc.lua file from *FILE* instead of the default + Load :program:`flux shell` initrc.lua file from *FILE* instead of the default initrc path. For details of the job shell initrc.lua file format, - see the INITRC section below. + see the `SHELL INITRC`_ section below. Job shell plugins may also support configuration via shell options in the jobspec. For specific information about runtime-loaded plugins, see the documentation for the specific plugin in question. The following -options are supported by the builtin plugins of ``flux-shell``: +options are supported by the builtin plugins of :program:`flux shell`: **pty** Allocate a pty to all task ranks for non-interactive use. Output @@ -226,7 +231,7 @@ options are supported by the builtin plugins of ``flux-shell``: **pty.interactive** Enable a a pty on rank 0 that is set up for interactive attach by - a front-end program (i.e. ``flux job attach``). With no other **pty** + a front-end program (i.e. :program:`flux job attach`). With no other **pty** options, only rank 0 will be assigned a pty and output will not be captured. These defaults can be changed by setting other **pty** options after **pty.interactive**, e.g. @@ -370,8 +375,8 @@ options are supported by the builtin plugins of ``flux-shell``: SHELL INITRC ============ -At initialization, flux-shell(1) reads a Lua initrc file which can be used -to customize the shell operation. The initrc is loaded by default from +At initialization, :program:`flux shell` reads a Lua initrc file which can be +used to customize the shell operation. The initrc is loaded by default from ``$sysconfdir/flux/shell/initrc.lua`` (or ``/etc/flux/shell/initrc.lua`` for a "standard" install), but a different path may be specified when launching a job via the ``initrc`` shell option. @@ -461,8 +466,8 @@ supported. Job shell specific functions and tables are described below: ``cpu-affinity`` shell option to ``per-task``. **shell.options.verbose** - Current flux-shell verbosity. This value may be changed at runtime, - e.g. ``shell.options.verbose = 2`` to set maximum verbosity. + Current :program:`flux shell` verbosity. This value may be changed at + runtime, e.g. ``shell.options.verbose = 2`` to set maximum verbosity. **shell.info** Returns a Lua table of shell information obtained via @@ -473,7 +478,7 @@ supported. Job shell specific functions and tables are described below: **rank** The rank of the current shell within the job. **size** - The number of flux-shell processes participating in this job. + The number of :program:`flux shell` processes participating in this job. **ntasks** The total number of tasks in this job. **service** diff --git a/doc/man1/flux-shutdown.rst b/doc/man1/flux-shutdown.rst index 9aaa20e6b2c4..cc2962e18b21 100644 --- a/doc/man1/flux-shutdown.rst +++ b/doc/man1/flux-shutdown.rst @@ -12,7 +12,9 @@ SYNOPSIS DESCRIPTION =========== -The ``flux-shutdown`` command causes the default Flux instance, or the +.. program:: flux shutdown + +The :program:`flux shutdown` command causes the default Flux instance, or the instance specified by *TARGET*, to exit RUN state and begin the process of shutting down. *TARGET* may be either a native Flux URI or a high level URI, as described in :man1:`flux-uri`. @@ -36,36 +38,42 @@ are printed. A Flux system instance requires offline KVS garbage collection to remove deleted KVS content and purged job directories, which accrue over time and increase storage overhead and restart time. It is recommended that the -*--gc* option be used on a routine basis to optimize Flux. +:option:`--gc` option be used on a routine basis to optimize Flux. OPTIONS ======= -``flux-shutdown`` accepts the following options: +:program:`flux shutdown` accepts the following options: + +.. option:: -h, --help -**-h, --help** Display options and exit -**--background** +.. option:: --background + Start the shutdown and exit immediately, without monitoring the process and displaying log messages. -**--quiet** +.. option:: --quiet + Show only error log messages (severity level <= LOG_WARNING level). -**--verbose=[LEVEL]** +.. option:: --verbose=[LEVEL] + Increase output verbosity. Level 1 shows all log messages. Higher verbosity levels are reserved for future use. -**--dump=PATH** +.. option:: --dump=PATH + Dump a checkpoint of KVS content to *PATH* using :man1:`flux-dump` after the KVS has been unloaded. The dump may be restored into a new Flux instance using :man1:`flux-restore`. Dump creation adds time to the shutdown - sequence, proportional to the amount of data in the KVS. ``--dump=auto`` - is a special case equivalent to ``--gc``. + sequence, proportional to the amount of data in the KVS. + :option:`--dump=auto` is a special case equivalent to :option:`--gc`. + +.. option:: --gc -**--gc** Prepare for offline KVS garbage collection by dumping a checkpoint of KVS content to ``dump/.tgz`` in *statedir*, if defined, otherwise in the broker's current working directory. Create a symbolic link named @@ -74,10 +82,12 @@ OPTIONS the dump, and the link is removed. :linux:man8:`systemd-tmpfiles` automatically cleans up dump files in ``/var/lib/flux/dump`` after 30 days. -**-y, --yes** +.. option:: -y, --yes + Answer yes to any yes/no questions. -**-n, --no** +.. option:: -n, --no + Answer no to any yes/no questions. diff --git a/doc/man1/flux-start.rst b/doc/man1/flux-start.rst index cc8be0fd196a..fff858a00c0f 100644 --- a/doc/man1/flux-start.rst +++ b/doc/man1/flux-start.rst @@ -13,90 +13,108 @@ SYNOPSIS DESCRIPTION =========== -flux-start(1) launches a new Flux instance. By default, flux-start -execs a single :man1:`flux-broker` directly, which will attempt to use -PMI to fetch job information and bootstrap a flux instance. +.. program:: flux start -If a size is specified via *--test-size*, an instance of that size is to be -started on the local host with flux-start as the parent. +:program:`flux start` launches a new Flux instance. By default, +:program:`flux start` execs a single :man1:`flux-broker` directly, which +will attempt to use PMI to fetch job information and bootstrap a flux instance. + +If a size is specified via :option:`--test-size`, an instance of that size is +to be started on the local host with :program:`flux start` as the parent. A failure of the initial program (such as non-zero exit code) -causes flux-start to exit with a non-zero exit code. +causes :program:`flux start` to exit with a non-zero exit code. OPTIONS ======= -**-o, --broker-opts**\ =\ *option_string* +.. option:: -o, --broker-opts=OPTIONS + Add options to the message broker daemon, separated by commas. -**-v, --verbose**\ =\ *[LEVEL]* +.. option:: -v, --verbose=[LEVEL] + This option may be specified multiple times, or with a value, to - set a verbosity level. See VERBOSITY LEVELS below. + set a verbosity level. See `VERBOSITY LEVELS`_ below. + +.. option:: -X, --noexec -**-X, --noexec** Don't execute anything. This option is most useful with -v. -**--caliper-profile**\ =\ *PROFILE* +.. option:: --caliper-profile=PROFILE + Run brokers with Caliper profiling enabled, using a Caliper configuration profile named *PROFILE*. Requires a version of Flux - built with --enable-caliper. Unless CALI_LOG_VERBOSITY is already + built with :option:`--enable-caliper`. Unless CALI_LOG_VERBOSITY is already set in the environment, it will default to 0 for all brokers. -**--rundir**\ =\ *DIR* - (only with *--test-size*) Set the directory that will be +.. option:: --rundir=DIR + + (only with :option:`--test-size`) Set the directory that will be used as the rundir directory for the instance. If the directory does not exist then it will be created during instance startup. If a DIR is not set with this option, a unique temporary directory will be created. Unless DIR was pre-existing, it will be removed when the instance is destroyed. -**--wrap**\ =\ *ARGS,…​* +.. option:: --wrap=ARGS + Wrap broker execution in a comma-separated list of arguments. This is useful for running flux-broker directly under debuggers or valgrind. -**-s, --test-size**\ =\ *N* +.. option:: -s, --test-size=N + Launch an instance of size *N* on the local host. -**--test-hosts**\ =\ *HOSTLIST* +.. option:: --test-hosts=HOSTLIST + Set FLUX_FAKE_HOSTNAME in the environment of each broker so that the broker can bootstrap from a config file instead of PMI. HOSTLIST is assumed to be in rank order. The broker will use the fake hostname to find its entry in the configured bootstrap host array. -**--test-exit-timeout**\ =\ *FSD* +.. option:: --test-exit-timeout=FSD + After a broker exits, kill the other brokers after a timeout (default 20s). -**--test-exit-mode**\ =\ *MODE* +.. option:: --test-exit-mode=MODE + Set the mode for the exit timeout. If set to ``leader``, the exit timeout - is only triggered upon exit of the leader broker, and the flux-start exit - code is that of the leader broker. If set to ``any``, the exit timeout - is triggered upon exit of any broker, and the flux-start exit code is the - highest exit code of all brokers. Default: ``any``. + is only triggered upon exit of the leader broker, and the + :program:`flux start` exit code is that of the leader broker. If set to + ``any``, the exit timeout is triggered upon exit of any broker, and the + :program:`flux start` exit code is the highest exit code of all brokers. + Default: ``any``. + +.. option:: --test-start-mode=MODE -**--test-start-mode**\ =\ *MODE* Set the start mode. If set to ``all``, all brokers are started immediately. If set to ``leader``, only the leader is started. Hint: in ``leader`` mode, - use ``--setattr=broker.quorum=1`` to let the initial program start before - the other brokers are online. Default: ``all``. + use :option:`--setattr=broker.quorum=1` to let the initial program start + before the other brokers are online. Default: ``all``. + +.. option:: --test-rundir=PATH -**--test-rundir**\ =\ *PATH* Set the directory to be used as the broker rundir instead of creating a temporary one. The directory must exist, and is not cleaned up unless - ``--test-rundir-cleanup`` is also specified. + :option:`--test-rundir-cleanup` is also specified. -**--test-rundir-cleanup** - Recursively remove the directory specified with ``--test-rundir`` upon - completion of flux-start. +.. option:: --test-rundir-cleanup + + Recursively remove the directory specified with :option:`--test-rundir` upon + completion of :program:`flux start`. + +.. option:: --test-pmi-clique=MODE -**--test-pmi-clique**\ =\ *MODE* Set the pmi clique mode, which determines how ``PMI_process_mapping`` is set in the PMI server used to bootstrap the brokers. If ``none``, the mapping is not created. If ``single``, all brokers are placed in one clique. If ``per-broker``, each broker is placed in its own clique. Default: ``single``. -**-r, --recovery**\ =\ *[TARGET]* +.. option:: -r, --recovery=[TARGET] + Start the rank 0 broker of an instance in recovery mode. If *TARGET* is a directory, treat it as a *statedir* from a previous instance. If *TARGET* is a file, treat it as an archive file from :man1:`flux-dump`. @@ -104,11 +122,12 @@ OPTIONS In recovery mode, any rc1 errors are ignored, broker peers are not allowed to connect, and resources are offline. -**--sysconfig** - Run the broker with ``--config-path`` set to the default system instance - configuration directory. This option is unnecessary if ``--recovery`` - is specified without its optional argument. It may be required if - recovering a dump from a system instance. +.. option:: --sysconfig + + Run the broker with :option:`--config-path` set to the default system + instance configuration directory. This option is unnecessary if + :option:`--recovery` is specified without its optional argument. It may + be required if recovering a dump from a system instance. VERBOSITY LEVELS ================ diff --git a/doc/man1/flux-startlog.rst b/doc/man1/flux-startlog.rst index 83c8fdd84d3d..275873508902 100644 --- a/doc/man1/flux-startlog.rst +++ b/doc/man1/flux-startlog.rst @@ -12,6 +12,8 @@ SYNOPSIS DESCRIPTION =========== +.. program:: flux startlog + List the Flux instance's start and stop times, by interpreting the contents of the KVS ``admin.eventlog``. @@ -31,17 +33,21 @@ This command is not available to guest users. OPTIONS ======= -**-h, --help** +.. option:: -h, --help + Summarize available options. -**--check** +.. option:: --check + If the instance has most recently restarted from a crash, exit with a return code of 1, otherwise 0. -**--quiet** +.. option:: --quiet + Suppress non-error output. -**-v, --show-version** +.. option:: -v, --show-version + Show the flux-core software version associated with each start event. diff --git a/doc/man1/flux-submit.rst b/doc/man1/flux-submit.rst index b547f94bdd8e..2bc7464f1acf 100644 --- a/doc/man1/flux-submit.rst +++ b/doc/man1/flux-submit.rst @@ -15,11 +15,13 @@ SYNOPSIS DESCRIPTION =========== -flux-submit(1) enqueues a job to run under Flux and prints its numerical -jobid on standard output. The job consists of *N* copies of COMMAND +.. program:: flux submit + +:program:`flux submit` enqueues a job to run under Flux and prints its +numerical jobid on standard output. The job consists of *N* copies of COMMAND launched together as a parallel job. -If *--ntasks* is unspecified, a value of *N=1* is assumed. +If :option:`--ntasks` is unspecified, a value of *N=1* is assumed. The available OPTIONS are detailed below. diff --git a/doc/man1/flux-top.rst b/doc/man1/flux-top.rst index 4becb021215e..61352567cc5b 100644 --- a/doc/man1/flux-top.rst +++ b/doc/man1/flux-top.rst @@ -15,34 +15,39 @@ SYNOPSIS DESCRIPTION =========== -The ``flux-top`` command provides a dynamic view of Flux instance status +.. program:: flux top + +The :program:`flux top` command provides a dynamic view of Flux instance status and running jobs. *TARGET*, if specified, selects a Flux instance other than the default, and may be either a native Flux URI or a high level URI, as described in :man1:`flux-uri`. -The ``flux-top`` display window is divided into two parts: the summary pane, -and the job listing pane, which are described in detail below. +The :program:`flux top` display window is divided into two parts: the summary +pane, and the job listing pane, which are described in detail below. OPTIONS ======= -**-h, --help** +.. option:: -h, --help + Summarize available options. -**--color**\ *[=WHEN]* +.. option:: --color[=WHEN] + Colorize output. The optional argument *WHEN* can be *auto*, *never*, or *always*. If *WHEN* is omitted, it defaults to *always*. The default - value when the **--color** option is not used is *auto*. + value when the :option:`--color` option is not used is *auto*. + +.. option:: -q, --queue=NAME -**-q, --queue**\ =\ *NAME* Limit status and jobs to specific queue. KEYS ==== -``flux-top`` responds to the following key presses: +:program:`flux top` responds to the following key presses: j, down-arrow Move cursor down in the job listing. @@ -58,8 +63,8 @@ d enter Open the job at the current cursor position. Only Flux instances (colored - blue in the job listing) owned by the user running ``flux-top`` may be - opened. The display changes to show a new Flux instance, with its jobid + blue in the job listing) owned by the user running :program:`flux top` may + be opened. The display changes to show a new Flux instance, with its jobid added to the path in the summary pane. Nothing happens if the selected job cannot be opened. @@ -68,7 +73,7 @@ q if any. If the original Flux instance is being displayed, quit the program. control-l - Force a redraw of the ``flux-top`` window. + Force a redraw of the :program:`flux top` window. SUMMARY PANE @@ -81,7 +86,7 @@ The summary pane shows the following information: - The amount of time until the job's expiration time, in Flux Standard Duration format. If the expiration time is unknown, the infinity symbol is - displayed (see CAVEATS below). + displayed (see `CAVEATS`_ below). - The nodes bargraph, which shows the fraction of used and down/excluded nodes vs total nodes. The graph of used nodes is colored yellow and extends from @@ -124,15 +129,16 @@ as :man1:`flux-jobs`. The newest jobs are shown at the top of the display. -``flux-top`` subscribes to job state update events, and tries to update its -display within 2s of receiving new job information. +:program:`flux top` subscribes to job state update events, and tries to update +its display within 2s of receiving new job information. CAVEATS ======= -``flux-top`` employs a few UTF-8 characters to maximize cuteness. If your -heart emoji looks like a cartoon expletive, consult your system administrator. +:program:`flux top` employs a few UTF-8 characters to maximize cuteness. If +your heart emoji looks like a cartoon expletive, consult your system +administrator. The infinity symbol in the expiration field does not really mean the Flux instance will run forever. The field width of the timestamp portion of the diff --git a/doc/man1/flux-update.rst b/doc/man1/flux-update.rst index 439d19c0302a..81a2efbdded0 100644 --- a/doc/man1/flux-update.rst +++ b/doc/man1/flux-update.rst @@ -12,8 +12,10 @@ SYNOPSIS DESCRIPTION =========== -flux-update(1) requests an update of one or more attributes for an active -(pending or running) job. Updates are permitted and validated by the job +.. program:: flux update + +:program:`flux update` requests an update of one or more attributes for an +active (pending or running) job. Updates are permitted and validated by the job manager before being applied to the job. Keys are expressed as period-delimited strings corresponding to an attribute @@ -26,8 +28,8 @@ the key is assumed to be prefixed with ``attributes.system.``, such that:: would request an update of ``attributes.system.myattr`` to the string value ``"value"``. -The flux-update(1) command may also support other convenient key aliases. -Key aliases are listed in the SPECIAL KEYS section below. +The :program:`flux update` command may also support other convenient key +aliases. Key aliases are listed in the `SPECIAL KEYS`_ section below. Updates will be sent to the job manager update service, which checks that the current user is permitted to apply all updates, and that all updates @@ -50,18 +52,20 @@ updates to the job by the guest user are denied with an error message:: This is necessary to prevent possible unintended bypass of limits or other checks on a job by a guest. -The flux-update(1) command may also support special handling of values +The :program:`flux update` command may also support special handling of values for specific keys. Those special cases are documented in the SPECIAL KEYS section below. OPTIONS ======= -**-n, --dry-run** +.. option:: -n, --dry-run + Do not send update to job manager, but print the updates in JSON to stdout. -**-v, --verbose** +.. option:: -v, --verbose + Print updated keys on success. SPECIAL KEYS diff --git a/doc/man1/flux-uptime.rst b/doc/man1/flux-uptime.rst index 30a37332ae20..7a76a3970a05 100644 --- a/doc/man1/flux-uptime.rst +++ b/doc/man1/flux-uptime.rst @@ -15,7 +15,7 @@ SYNOPSIS DESCRIPTION =========== -The ``flux-uptime`` command displays the following information about the +The :program:`flux uptime` command displays the following information about the current Flux instance, on one or two lines: - The current wall clock time. diff --git a/doc/man1/flux-uri.rst b/doc/man1/flux-uri.rst index dcab219cb2fe..5445280fefd8 100644 --- a/doc/man1/flux-uri.rst +++ b/doc/man1/flux-uri.rst @@ -11,6 +11,8 @@ SYNOPSIS DESCRIPTION =========== +.. program:: flux uri + Connections to Flux are established via a Uniform Resource Identifier (URI) which is passed to the :man3:`flux_open` API call. These *native* URIs indicate the "connector" which will be used to establish the @@ -26,13 +28,13 @@ connecting to the enclosing instance. However, connecting to a *different* Flux instance will require discovery of the fully-resolved URI for that instance. -**flux uri** attempts to resolve its *TARGET* argument to a native local +:program:`flux uri` attempts to resolve its *TARGET* argument to a native local or remote URI. The *TARGET* is itself a URI which specifies the method to use in URI resolution via the scheme part, with the path and query parts passed to a plugin which implements the resolution method. As a convenience, if *TARGET* is specified with no scheme, then the scheme -is assumed to be ``jobid``. This allows ``flux uri`` to be used to look +is assumed to be ``jobid``. This allows :program:`flux uri` to be used to look up the URI for a Flux instance running as a job in the current enclosing instance with: @@ -53,21 +55,23 @@ all *TARGET* URIs support the special query arguments ``local`` or would return the ``local://`` URI for *JOBID* (if the URI can be resolved). A special environment variable FLUX_URI_RESOLVE_LOCAL will force -``flux uri`` to always resolve URIs to local form. This is often useful if -the local connector is known to be on the local system (i.e. within a test +:program:`flux uri` to always resolve URIs to local form. This is often useful +if the local connector is known to be on the local system (i.e. within a test Flux instance), and ssh to localhost does not work. A list of supported URI schemes will be listed at the bottom of -``flux uri --help`` message. For a description of the URI resolver schemes -included with Flux, see the URI SCHEMES and EXAMPLES sections below. +:option:`flux uri --help` message. For a description of the URI resolver +schemes included with Flux, see the URI SCHEMES and EXAMPLES sections below. OPTIONS ======= -**--remote** +.. option:: --remote + Return the *remote* (``ssh://``) equivalent of the resolved URI. -**--local** +.. option:: --local + Return the *local* (``local://``) equivalent of the resolved URI. Warning: the resulting URI may be invalid for the current system if the network host specified by an ``ssh`` URI is not the current @@ -81,7 +85,7 @@ The following URI schemes are included by default: jobid:ID[/ID...] This scheme attempts to get the URI for a Flux instance running as a job in the current enclosing instance. This is the assumed scheme if no - ``scheme:`` is provided in *TARGET* passed to ``flux uri``, so the + ``scheme:`` is provided in *TARGET* passed to :program:`flux uri`, so the ``jobid:`` prefix is optional. A hierarchy of Flux jobids is supported, so ``f1234/f3456`` will resolve the URI for job ``f3456`` running in job ``f1234`` in the current instance. This scheme will raise an error @@ -93,7 +97,7 @@ pid:PID then the scheme reads ``FLUX_URI`` from the broker's initial program or another child process since ``FLUX_URI`` in the broker's environment would refer to *its* parent (or may not be set at all in the case of a - test instance started with ``flux start --test-size=N``). + test instance started with :option:`flux start --test-size=N`). slurm:JOBID This scheme makes a best-effort to resolve the URI of a Flux instance diff --git a/doc/man1/flux-version.rst b/doc/man1/flux-version.rst index 51788e358083..e4a1c7e03990 100644 --- a/doc/man1/flux-version.rst +++ b/doc/man1/flux-version.rst @@ -14,7 +14,7 @@ SYNOPSIS DESCRIPTION =========== -flux-version(1) prints version information for flux components. +:program:`flux version` prints version information for flux components. At a minimum, the version of flux commands and the currently linked libflux-core.so library is displayed. If running within an instance, the version of the flux-broker found and FLUX_URI are also included. diff --git a/doc/man1/flux-watch.rst b/doc/man1/flux-watch.rst index 2ecfbace9abc..8955ab3181e6 100644 --- a/doc/man1/flux-watch.rst +++ b/doc/man1/flux-watch.rst @@ -1,8 +1,8 @@ .. flux-help-section: jobs -============ -flux-jobs(1) -============ +============= +flux-watch(1) +============= SYNOPSIS @@ -13,11 +13,13 @@ SYNOPSIS DESCRIPTION =========== -The flux-watch(1) command is used to monitor the output and state of one -or more Flux jobs. The command works similarly to the :man1:`flux-submit` -``--watch`` option, but can be used to monitor even inactive jobs. For -example, to copy all job output to the terminal after submitting a series -of jobs with :man1:`flux-submit` or :man1:`flux-bulksubmit`, use +.. program:: flux watch + +The :program:`flux watch` command is used to monitor the output and state of +one or more Flux jobs. The command works similarly to the +:option:`flux submit --watch` option, but can be used to monitor even inactive +jobs. For example, to copy all job output to the terminal after submitting a +series of jobs with :man1:`flux-submit` or :man1:`flux-bulksubmit`, use :: @@ -30,55 +32,62 @@ batch job. OPTIONS ======= -**-a, --active** +.. option:: -a, --active + Watch all active jobs. This is equivalent to *--filter=pending,running*. -**-A, --all** +.. option:: -A, --all + Watch all jobs. This is equivalent to *--filter=pending,running,inactive*. -**-c, --count**\ *=N* +.. option:: -c, --count=N + Limit output to N jobs (default 1000). This is a safety measure to - protect against watching too many jobs with the ``--all`` option. The - limit can be disabled with ``--count=0``. + protect against watching too many jobs with the :option:`--all` option. The + limit can be disabled with :option:`--count=0`. + +.. option:: --since=WHEN -**--since**\ *=WHEN* Limit output to jobs that have been active since a given timestamp. - This option implies ``-a`` if no other ``--filter`` options are specified. - If *WHEN* begins with ``-`` character, then the remainder is considered - to be a an offset in Flux standard duration (RFC 23). Otherwise, + This option implies :option:`-a` if no other :option:`--filter` options are + specified. If *WHEN* begins with ``-`` character, then the remainder is + considered to be a an offset in Flux standard duration (RFC 23). Otherwise, any datetime expression accepted by the Python `parsedatetime `_ module is accepted. Examples: "-6h", "-1d", "yesterday", "2021-06-21 6am", "last Monday", etc. It is assumed to be an error if a timestamp in the future is supplied. -**-f, --filter**\ *=STATE|RESULT* +.. option:: -f, --filter=STATE|RESULT + Watch jobs with specific job state or result. Multiple states or results can be listed separated by comma. See the JOB STATUS section in the :man1:`flux-jobs` manual for additional information. -**--progress** +.. option:: --progress + Display a progress bar showing the completion progress of monitored jobs. Jobs that are already inactive will immediately have their progress updated in the progress bar, with output later copied to the terminal. The progress bar by default includes a count of pending, running, complete and failed jobs, and an elapsed timer. The elapsed timer is initialized at the submit time of the earliest job, or the - starttime of the instance with ``--all``, in order to reflect the real + starttime of the instance with :option:`--all`, in order to reflect the real elapsed time for the jobs being monitored. -**--jps** - With ``--progress``, display throughput statistics (job/s) in the +.. option:: --jps + + With :option:`--progress`, display throughput statistics (job/s) in the progress bar instead of an elapsed timer. Note: The throughput will be calculated based on the elapsed time as described in the description - of the ``-progress`` option. + of the :option:`-progress` option. EXIT STATUS =========== -The exit status of ``flux watch`` is 0 if no jobs match the job selection -options or if all jobs complete with success. Otherwise, the command exits -with the largest exit status of all monitored jobs, or 2 if there is an +The exit status of :program:`flux watch` is 0 if no jobs match the job +selection options or if all jobs complete with success. Otherwise, the command +exits with the largest exit status of all monitored jobs, or 2 if there is an error during option processing. RESOURCES diff --git a/doc/man1/flux.rst b/doc/man1/flux.rst index f04abc5b2fa5..ed1c15524c2d 100644 --- a/doc/man1/flux.rst +++ b/doc/man1/flux.rst @@ -12,11 +12,13 @@ SYNOPSIS DESCRIPTION =========== +.. program:: flux + Flux is a modular framework for resource management. -flux(1) is a front end for Flux sub-commands. -"flux -h" summarizes the core Flux commands. -"flux help *CMD*" displays the manual page for *CMD*. +:program:`flux` is a front end for Flux sub-commands. +:option:`flux -h` summarizes the core Flux commands. +:program:`flux help CMD` displays the manual page for *CMD*. If *CMD* contains a slash "/" character, it is executed directly, bypassing the sub-command search path. @@ -25,25 +27,29 @@ bypassing the sub-command search path. OPTIONS ======= -**-h, --help** +.. option:: -h, --help + Display help on options, and a list of the core Flux sub-commands. -**-p, --parent** +.. option:: -p, --parent + If current instance is a child, connect to parent instead. Also sets *FLUX_KVS_NAMESPACE* if current instance is confined to a KVS namespace in the parent. This option may be specified multiple times. -**-v, --verbose** +.. option:: -v, --verbose + Display command environment, and the path search for *CMD*. -**-V, --version** +.. option:: -V, --version + Convenience option to run :man1:`flux-version`. SUB-COMMAND ENVIRONMENT ======================= -flux(1) uses compiled-in install paths and its environment +:program:`flux` uses compiled-in install paths and its environment to construct the environment for sub-commands. Sub-command search path diff --git a/doc/man1/index.rst b/doc/man1/index.rst index f11b93ef72cc..f2f7986077af 100644 --- a/doc/man1/index.rst +++ b/doc/man1/index.rst @@ -5,7 +5,6 @@ Section 1 - Flux Commands .. toctree:: :maxdepth: 1 + :glob: - index_job - index_kvs - index_general + * diff --git a/doc/man1/index_general.rst b/doc/man1/index_general.rst deleted file mode 100644 index 9f45ff13bc02..000000000000 --- a/doc/man1/index_general.rst +++ /dev/null @@ -1,27 +0,0 @@ -General Commands -================ - -.. toctree:: - :maxdepth: 1 - - flux-broker - flux-config - flux-cron - flux-dmesg - flux-env - flux-event - flux-exec - flux-keygen - flux-getattr - flux-logger - flux-module - flux-overlay - flux-ping - flux-proxy - flux-shutdown - flux-start - flux-startlog - flux-uri - flux-version - flux - diff --git a/doc/man1/index_job.rst b/doc/man1/index_job.rst deleted file mode 100644 index df366d3ea40d..000000000000 --- a/doc/man1/index_job.rst +++ /dev/null @@ -1,23 +0,0 @@ -Job-related Commands -==================== - -.. toctree:: - :maxdepth: 1 - - flux-alloc - flux-batch - flux-bulksubmit - flux-cancel - flux-job - flux-jobs - flux-jobtap - flux-mini - flux-pstree - flux-pgrep - flux-queue - flux-resource - flux-run - flux-shell - flux-submit - flux-top - flux-uptime diff --git a/doc/man1/index_kvs.rst b/doc/man1/index_kvs.rst deleted file mode 100644 index d29f27558d13..000000000000 --- a/doc/man1/index_kvs.rst +++ /dev/null @@ -1,11 +0,0 @@ -KVS-related Commands -==================== - -.. toctree:: - :maxdepth: 1 - - flux-content - flux-dump - flux-restore - flux-filemap - flux-kvs diff --git a/doc/manpages.py b/doc/manpages.py index 70243e402928..9e7540797835 100644 --- a/doc/manpages.py +++ b/doc/manpages.py @@ -52,7 +52,6 @@ ('man1/flux-keygen', 'flux-keygen', 'generate keys for Flux security', [author], 1), ('man1/flux-kvs', 'flux-kvs', 'Flux key-value store utility', [author], 1), ('man1/flux-logger', 'flux-logger', 'create a Flux log entry', [author], 1), - ('man1/flux-mini', 'flux-mini', 'Minimal Job Submission Tool', [author], 1), ('man1/flux-submit', 'flux-submit', 'submit a job to a Flux instance', [author], 1), ('man1/flux-run', 'flux-run', 'run a Flux job interactively', [author], 1), ('man1/flux-bulksubmit', 'flux-bulksubmit', 'submit jobs in bulk to a Flux instance', [author], 1),