Skip to content

Commit

Permalink
Condense, improve and clean up Nextflow logging (#25)
Browse files Browse the repository at this point in the history
* Condense, improve and clean up Nextflow logging
- tries to remedy visual issues in ANSI console by condensing and
  shortening line lengths of log output
- improves some wording, consistency and whitespacing of log output
- refactor some log output code into functions
- adds ksrates version to log

* Don't clean up if ksrates config file newly generated
There won't be anything to clean up, and onComplete would print
irrelevant warning about empty focal_species parameter.

* Single '/' in logs folder path when focal_species empty

* Some renaming of channels and variables
- Correction -> Adjustment
- peakCalling -> paralogsAnalyses
  • Loading branch information
lohausr authored Aug 23, 2021
1 parent 257c697 commit 29a9753
Show file tree
Hide file tree
Showing 8 changed files with 318 additions and 361 deletions.
Binary file modified doc/source/_images/error_box2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion doc/source/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ The Nextflow configuration file is used to configure various settings for the *k

* **withName** defines settings for individual processes in the *ksrates* Nextflow pipeline.

There are 11 processes in the pipeline, 6 of which (``checkConfig``, ``setupAdjustment``, ``setParalogAnalysis``, ``setOrthologAnalysis``, ``doRateAdjustment`` and ``drawTree``) are by default run locally because they execute minimal calculations. The remaining 5 processes (``estimatePeak``, ``plotOrthologDistrib``, ``paralogsAnalyses``, ``wgdParalogs`` and ``wgdOrthologs``) are instead run by default on a cluster, if available, and can be configured under this section of the Nextflow configuration file. ``wgdParalogs`` and ``wgdOrthologs`` are the most computationally demanding and it is advised to assign them a higher computational power than the other processes. If available, we suggest to configure about 10 CPUs/cores/slots/threads and about 20GB memory (or, on average, about 2GB per configured CPU) for each of these two processes.
There are 11 processes in the pipeline, 6 of which (``checkConfig``, ``setupAdjustment``, ``setParalogAnalysis``, ``setOrthologAnalysis``, ``doRateAdjustment`` and ``drawTree``) are by default run locally because they execute minimal calculations. The remaining 5 processes (``estimatePeaks``, ``plotOrthologDistrib``, ``paralogsAnalyses``, ``wgdParalogs`` and ``wgdOrthologs``) are instead run by default on a cluster, if available, and can be configured under this section of the Nextflow configuration file. ``wgdParalogs`` and ``wgdOrthologs`` are the most computationally demanding and it is advised to assign them a higher computational power than the other processes. If available, we suggest to configure about 10 CPUs/cores/slots/threads and about 20GB memory (or, on average, about 2GB per configured CPU) for each of these two processes.

Settings can be tailored to your configured executor (see above) through the use of Nextflow process directives (for a complete list and detailed descriptions see the `Nextflow documentation <https://www.nextflow.io/docs/latest/process.html#process-directives>`__), such as:

Expand Down
2 changes: 1 addition & 1 deletion doc/source/template_docker_nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ process {
clusterOptions = '' // e.g. '-l h_vmem=2G'
}

withName: 'estimatePeak' {
withName: 'estimatePeaks' {
memory = '' // e.g. '2GB'
clusterOptions = '' // e.g. '-l h_vmem=2G'
}
Expand Down
2 changes: 1 addition & 1 deletion doc/source/template_nocontainer_nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ process {
beforeScript = '' // e.g. 'module load python paml muscle blast'
}

withName: 'estimatePeak' {
withName: 'estimatePeaks' {
memory = '' // e.g. '2GB'
clusterOptions = '' // e.g. '-l h_vmem=2G'
beforeScript = '' // e.g. 'module load python'
Expand Down
2 changes: 1 addition & 1 deletion doc/source/template_singularity_nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ process {
clusterOptions = '' // e.g. '-l h_vmem=2G'
}

withName: 'estimatePeak' {
withName: 'estimatePeaks' {
memory = '' // e.g. '2GB'
clusterOptions = '' // e.g. '-l h_vmem=2G'
}
Expand Down
2 changes: 1 addition & 1 deletion example/nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ process {
clusterOptions = '' // e.g. '-l h_vmem=2G'
}

withName: 'estimatePeak' {
withName: 'estimatePeaks' {
memory = '' // e.g. '2GB'
clusterOptions = '' // e.g. '-l h_vmem=2G'
}
Expand Down
Loading

0 comments on commit 29a9753

Please sign in to comment.