diff --git a/Gemfile b/Gemfile index 9f77398..6a4d713 100644 --- a/Gemfile +++ b/Gemfile @@ -28,4 +28,4 @@ gem "wdm", "~> 0.1.1", :platforms => [:mingw, :x64_mingw, :mswin] # do not have a Java counterpart. gem "http_parser.rb", "~> 0.6.0", :platforms => [:jruby] - +gem "mutex_m", "~> 0.3.0" diff --git a/Gemfile.lock b/Gemfile.lock index 32cc104..2c077dc 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -46,7 +46,7 @@ GEM logger faraday-net_http (3.1.1) net-http - ffi (1.17.0-arm64-darwin) + ffi (1.17.1-arm64-darwin) filesize (0.2.0) forwardable-extended (2.6.0) gemoji (4.1.0) @@ -251,9 +251,10 @@ GEM jekyll-paginate (~> 1.1) jekyll-sitemap (~> 1.3) minitest (5.24.1) + mutex_m (0.3.0) net-http (0.4.1) uri - nokogiri (1.16.7-arm64-darwin) + nokogiri (1.18.2-arm64-darwin) racc (~> 1.4) octokit (4.25.1) faraday (>= 1, < 3) @@ -310,6 +311,7 @@ DEPENDENCIES jekyll-sitemap jemoji minimal-mistakes-jekyll + mutex_m (~> 0.3.0) tzinfo-data wdm (~> 0.1.1) webrick (~> 1.8) diff --git a/_data/navigation.yml b/_data/navigation.yml index 6a1eec8..0ca4669 100644 --- a/_data/navigation.yml +++ b/_data/navigation.yml @@ -1,15 +1,11 @@ main: - - title: Home - url: / - title: "About" url: /about - title: "Getting Started" - url: /getting-started - - title: "WDL Development" - url: /wdl-dev + url: https://docs.openwdl.org/getting-started/overview.html - title: "Specification" url: /spec - title: "Documentation" url: https://docs.openwdl.org - title: "Blog" - url: /blog \ No newline at end of file + url: /blog diff --git a/_pages/getting-started.md b/_pages/getting-started.md deleted file mode 100644 index 9965029..0000000 --- a/_pages/getting-started.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -permalink: /getting-started/ -title: "Getting Started" -toc: true ---- - -The WDL community depends on your involvement to thrive. You are encouraged to ask questions, help other users, and make contributions where you can. Interactions occur primarily on [GitHub](https://github.com/openwdl/wdl) and Slack(https://join.slack.com/t/openwdl/shared_invite/zt-ctmj4mhf-cFBNxIiZYs6SY9HgM9UAVw). The WDL community also has an official [blog](/blog) where announcements are made. - -The [WDL language specification](https://github.com/openwdl/wdl/blob/wdl-1.1/SPEC.md) contains all relevant information for users and developers, including those wanting to implement an execution engine. - -## Learning WDL - -### WDL Resources -* [wdl-docs](https://docs.openwdl.org/en/stable/) -* [learn-wdl](https://github.com/openwdl/learn-wdl) -* [WDL Resources](https://support.terra.bio/hc/en-us/sections/360007274612-WDL-Documentation) provided by Terra (a product of the Broad Institute) -* [WDL Template github repo](https://github.com/openwdl/workflow-template-wdl) - -### Asking a Question - -- Search in the [discussions](https://github.com/openwdl/wdl/discussions) to see if the question has been asked already; if not, start a new discussion. -- Join our [Slack](https://join.slack.com/t/openwdl/shared_invite/zt-ctmj4mhf-cFBNxIiZYs6SY9HgM9UAVw) and ask in the `#support` channel. -- Search the [Bioinformatics Stack Exchange](https://bioinformatics.stackexchange.com/search?q=wdl) for previously answered questions, or ask a new question. -- Search the [Google Group](https://groups.google.com/a/openwdl.org/forum/#!forum/community) for previously answered questions. This group is largely inactive, so you're encoraged to ask new questions in one of the above places instead. - - -### Bugs and New Features - -- Search for an existing [issue](https://github.com/openwdl/wdl/issues). If your issue has not already been reported, create a new one. -- For feature reqeusts, you are encoraged to first start a discussion at one of the places listed above to get feedback from the community. -- If you'd like to provide a fix/implementation for an issue, please read about [contributing](#contributing) before submitting a [pull request](https://github.com/openwdl/wdl/pulls). - -### Published Workflows - -The following are collections of open-source WDL workflows. -The WDL task or workflow you need may already be available in one of these repositories, or you may find a similar workflow and customize it to your needs. - -- [Dockstore](https://dockstore.org/search?entryType=workflows&descriptorType=WDL&searchMode=files) -- [BioWDL](https://github.com/biowdl) -- [Broad Institute WARP](https://broadinstitute.github.io/warp/docs/get-started/) -- [GATK Workflows](https://github.com/gatk-workflows/) -- [ENCODE](https://www.encodeproject.org/pipelines/) -- [Terra](https://app.terra.bio) (requires registration) - diff --git a/_pages/wdl-dev.md b/_pages/wdl-dev.md deleted file mode 100644 index 16713f2..0000000 --- a/_pages/wdl-dev.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -permalink: /wdl-dev/ -title: "WDL Development" -toc: true ---- - -## Execution Engines and Platforms - -WDL does not have an official implementation. -Third parties are relied upon to provide installable software or hosted platforms that interpret and execute WDL workflows and tasks. -Although WDL does not yet have an official compliance program or certification process, implementers are expected to design their tools according to the specification so as to maximize the portability of workflows across implementations. -Nonetheless, implementers may provide additional optional features specific. -Please see the documentation associated with each tool/platform for information on available execution options and support. - -| Implementation | Requires Installation | Local Execution | HPC | Cloud | -| ------------------------------------------------------------------------------ | --------------------- | --------------- | ----- | --------------------- | -| [AWS HealthOmics](https://docs.aws.amazon.com/omics/latest/dev/workflows.html) | No | No | No | AWS | -| [Cromwell](https://github.com/broadinstitute/cromwell) * | Yes | Yes | Many | AWS Batch, Azure, GCP | -| [dxCompiler](https://github.com/dnanexus/dxCompiler) | Yes | No | No | DNAnexus | -| [MiniWDL](https://github.com/chanzuckerberg/miniwdl) | Yes | Yes | SLURM | AWS Batch | -| [Terra](https://terra.bio/) | No | No | No | Azure, GCP | - -\* Also see [WDL Runner](https://github.com/broadinstitute/wdl-runner), a script for launch WDL workflows on GCP using Cromwell - -## Grammars, Parsers, and Language Support - -- The WDL [parsers repository](https://github.com/openwdl/wdl-parsers/) provides grammar definitions in various formats and generated parsers for various languages. -- [MiniWDL](https://github.com/chanzuckerberg/miniwdl) provides python bindings for WDL as well as commandline tools for validation, linting, and generating workflow input templates. -- [WOMTool](https://cromwell.readthedocs.io/en/stable/WOMtool/) is a standalone Java tool for WDL parsing, validating, linting, and diagramming. -- [wdlTools](https://github.com/dnanexus/wdlTools) - provides 1) a parser Java/Scala library, based on the [ANTLR4 grammars](https://github.com/openwdl/wdl-parsers) grammars, for WDL draft-2, 1.0, 1.1, and 2.0; and 2) command-line tools for sytanx checking, type-checking, linting, code formatting (including upgrading from older to newer WDL versions), generating documentation, and executing WDL tasks locally. -- [sprocket](https://github.com/stjude-rust-labs/sprocket) a package manager and linter for WDL files. - -## IDE Support - -| IDE | Tool | -| ------------------ | ------------------------------------------------------------------------------------------------ | -| Emacs | [poly-wdl](https://github.com/jmonlong/poly-wdl) | -| Emacs | [wdl-mode](https://github.com/zhanxw/wdl-mode) | -| JetBrains | [Winstanly](https://plugins.jetbrains.com/plugin/8154-winstanley-wdl) | -| Sublime | [WDL Syntax Highlighter](https://github.com/broadinstitute/wdl-sublime-syntax-highlighter) | -| Vim | [vim-wdl](https://github.com/broadinstitute/vim-wdl) | -| Visual Studio Code | [WDL Syntax Highlighter](https://marketplace.visualstudio.com/items?itemName=broadinstitute.wdl) | - -## Documentation - -- [wdl-aid](https://github.com/biowdl/wdl-aid) generates documentation for the inputs of WDL workflows, based on the parameter_meta information defined in the WDL file. -- [wdldoc](https://github.com/stjudecloud/wdldoc) - -## Testing - -- [wdl-tests](https://github.com/openwdl/wdl-tests) is a collection of test cases for WDL implementations. A specification is provided for writing new tests that are compatible with automated testing frameworks. -- [Pytest-workflow](https://github.com/LUMC/pytest-workflow) is a implementation-agnostic workflow tester. It can be used with both Cromwell and MiniWDL. Uses pytest as the underlying test framework. Tests can be specified in either YAML format or python code. -- [Pytest-wdl](https://github.com/EliLillyCo/pytest-wdl) is a plugin for the pytest unit testing framework that enables testing of WDL tasks and workflows. Tests can be specified in either YAML format or python code. - -## Packaging - -- [miniwdl zip](https://miniwdl.readthedocs.io/en/latest/zip.html) generates a ZIP file including a given WDL source code file and any other WDL files it imports. The ZIP file can be supplied directly to miniwdl run, which can extract it automatically. -- [wdl-packager](https://github.com/biowdl/wdl-packager) packages a workflow and all of its imports into a zip file. The zip can be used as an imports zip package for cromwell. The utility can add non-WDL files (such as the license) to the zip package and provides options to package the zip in a binary reproducible way. diff --git a/index.html b/index.html index d91d7be..a031401 100644 --- a/index.html +++ b/index.html @@ -5,15 +5,12 @@ overlay_color: "#000" overlay_filter: "0.5" overlay_image: /media/logo-preview.png - actions: - - label: " Getting Started" - url: "/getting-started" -intro: - - excerpt: 'The Workflow Description Language (WDL) is an open standard for describing data processing workflows with a human-readable and writeable syntax. - WDL makes it straightforward to define analysis tasks, connect them together in workflows, and parallelize their execution. - The language strives to be accessible and understantable to all manner of users, including programmers, analysts, and operators of a production system. - The language enables common patterns, such as scatter-gather and conditional execution, to be expressed simply. - WDL is designed for portability, and there are several implementations to choose from that run in a variety of environments, including HPC systems and cloud platforms.' +intro: + - excerpt: 'The Workflow Description Language (WDL) is an open standard for describing data processing workflows with a human-readable and writeable syntax. + WDL makes it straightforward to define analysis tasks, connect them together in workflows, and parallelize their execution. + The language strives to be accessible and understandable to all manner of users, including programmers, analysts, and operators of a production system. + The language enables common patterns, such as scatter-gather and conditional execution, to be expressed simply. + WDL is designed for portability, and there are several implementations to choose from that run in a variety of environments, including HPC systems and cloud platforms.' --- -{% include feature_row id="intro" type="center" %} \ No newline at end of file +{% include feature_row id="intro" type="center" %}