Skip to content

Commit

Permalink
Update docs/working-with-files.md
Browse files Browse the repository at this point in the history
Co-authored-by: Ben Sherman <[email protected]>
Signed-off-by: Chris Hakkaart <[email protected]>
  • Loading branch information
christopher-hakkaart and bentsherman authored Dec 13, 2024
1 parent ea8db45 commit 622033d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/working-with-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ Additional configuration may be necessary for cloud object storage, such as auth

When a remote file is passed as an input to a process, Nextflow stages the file into the work directory using an appropriate Java SDK.

Remote files are staged in a subdirectory of the work directory with form `stage-<session-id>/<hash>/<filename>`, where `<hash>` is determined by the remote file path. If multiple tasks request the same remote file, the file will be downloaded once and reused by each task. These files can be reused by resumed runs with the same session ID.
Remote files are staged in a subdirectory of the work directory with the form `stage-<session-id>/<hash>/<filename>`, where `<hash>` is determined by the remote file path. If multiple tasks request the same remote file, the file will be downloaded once and reused by each task. These files can be reused by resumed runs with the same session ID.

:::{note}
Remote file staging can be a bottleneck during large-scale runs, particularly when input files are stored in object storage but need to be staged in a shared filesystem work directory. This bottleneck occurs because Nextflow handles all file transfers.
Expand Down

0 comments on commit 622033d

Please sign in to comment.