2.0.0-alpha.4

⚠️	This is a pre-release. It can contain bugs and significant changes which are not yet finalized. Changes may appear without notice. We recommend to try the pre-releases to learn about upcoming features. For important projects, use stable releases.

Commit history

(click to expand)

• [c8f3174] docs(dev): add ad note about custom dataset server into dev guide

• [4e86cd3] Merge pull request #822 from nextstrain/docs/dev-guide-datasets

• [719f217] chore(ci): attempt to push git tag after web build

• [e923bf6] chore(ci): create git tags on for release web builds

• [7ebae91] chore(ci): use full dev script command names in ci [skip ci]

• [ba28c64] chore: remove simplealign example

• [4263547] Merge pull request #823 from nextstrain/chore/remove-simplealign

• [28ae075] fix(web): ensure truncated text is not repeated in insertions tooltip

This is a port of 79e1f3b, which got lost during rust transition

• [939b57b] Merge pull request #824 from nextstrain/fix/insetions-tooltip

• [d0db03a] feat(web): limit tooltips to 20 insertions max

• [474e633] fix: ensure aa muts are attached to the tree

This fixes an omission: only nuc mutations were attached to the tree JSON object, and aa mutations were missing.

• [82bb53b] feat(web): display nuc insertions in sequence views

• [b7b51bb] feat(web): display aa insertions in sequence views

• [9f31264] feat(web): display total number of markers and limit in the message

• [48b5132] feat(web): increase seq marker limit to 3000

• [0217b06] Merge pull request #828 from nextstrain/feat/web-show-marker-limit

• [7dcc25e] Merge remote-tracking branch 'origin/master' into feat/web-ins-markers

• [27a58c9] feat(web): account insertions in seq marker limit

• [0008108] refactor(web): move insertion colors into theme

• [58f529f] feat(web): make insertion markers black

• [c721968] feat(web): limit number of aa mut badges in nuc gap tooltips

This limits number of aminoacid mutation badges in the tooltip of the nucleotide gap marker in sequence view.

This avoid screen overflow when there's too many mutations.

This happens with aa deletions mostly, so ideally we need to group aa deletions into ranges. But it have negative impact to other places which need access to individual deletions and is more work overall. Maybe later.

• [88e2483] fix: accumulate aa node mutations correctly in preprocess

This bug went unnoticed because we only every diffed the nextclade.tsv
Since we don't surface nextclade.json which contains private muts, this problem didn't show in automated equivalence testing.

• [d407a23] Merge pull request #826 from nextstrain/fix/tree-aa-muts

fix: ensure aa muts are attached to the tree

• [4b0247c] Merge pull request #829 from nextstrain/fix/web-limit-tooltip-badges

• [3180e8b] web: make insertions makers purple/yellow (IGV uses purple)

• [a6de7ba] Merge pull request #827 from nextstrain/feat/web-ins-markers

feat(web): display nuc and aa insertions in sequence views

• [4ce81cf] chore(build): speedup debug binaries

In order to improve deb experience, this configures build to optimize dependency packages even in dev mode. Nextclade packages themselves stay unoptimized, so that full debug information is available.

Also increases number of codegen units to speedup incremental build process.

• [c4ce28e] Merge pull request #831 from nextstrain/chore/faster-debug-build

• [4b7ac6f] Merge pull request #825 from nextstrain/feat/limit-insertions

feat(web): limit tooltips to 20 insertions max

• [81d1cef] chore(ci): make sure .env file is read properly

• [d330857] chore: release cli 2.0.0-alpha.4

2.0.0-alpha.3

⚠️	This is a pre-release. It can contain bugs and significant changes which are not yet finalized. Changes may appear without notice. We recommend to try the pre-releases to learn about upcoming features. For important projects, use stable releases.

Commit history

(click to expand)

• [6ba9532] feat: add new field "score_weight" to qcconfig, with default 75

• [c5f1b34] feat: use custom score_weight in fs/stop qc rules

• [661fb27] fix: implement Default trait to avoid accidental incorrect values

Explicitly implements Default trait for the structs modified in this PR. When used programmatically, currently the default on score_weight is set to 0.0, due to #[derive(Default)] attribute.

Instead, I implemented Default trait with the correct value. With the struct container attribute #[serde(default)], serde during deserialization will substitute missing values from the implementation of the Default trait.

This solution is more safe than setting the default for serde only.

Other structs in this file also need to be audited for:

• whether they need Default trait (i.e. it is required somewhere else in code)

• whether they need an explicit implementation of the Default trait (in cases where the default values provided by #[derive(Default)] are incorrect)

• [d907d90] Merge branch 'master' into custom-score-weight-stop-fs

• [2bd5612] chore: improve docs for release script

• [3d4a4b8] feat(web): configure sequence markers height

This adds toggles to the "Settings" dialog, which allow to toggle markers in sequence views between full-height, half-height at the bottom, half-height at the top and turned off entirely. The toggles are separate for missing, gaps, mutations and unsequenced markers.

The values are remembered in local storage and persist across app launches.

Right now only the nuc view is configurable and only for 4 of hese categories of vertical markers (and not frame shifts for example), but we can add similar toggles to peptide view and other markers later.

• [f96667d] Merge branch 'custom-score-weight-stop-fs'

• [fede327] Merge pull request #811 from nextstrain/feat/config-seq-markers

• [1dcb44b] feat(cli): use index_v2.json

See the corresponding PR in data repo: nextstrain/nextclade_data#26

• [42d41c7] feat(web): user index_v2.json

• [40adcf1] feat(web): use friendly names from the dataset

• [be3c21a] feat(cli): use friendly names from the dataset

• [d5a0a11] fix(web): treat dataset default gene field as optional

When viewedGene was not set in the dataset, causing "Gene is missing in gene map" text appearing in the sequence view, due to gene view being switched to peptide view for undefined gene.

Let's treat the viewedGene field as optional and, if it's undefined, default to nuc view.

• [a50a7b0] feat(web): limit number of markers that can be displayed in seq views

This add a setting that defined maximum number of markers (mutations, deletions, missing, frame shifts) that can be displayed in nucleotide sequence views.

If this number is exceeded, a message is displayed instead.

The value is 300 by default and can be changed in settings. Value is persisted in local storage.

• [5314d08] fix(web): silence false "Duplicate atom key" recoil errors

• [a9a3fa3] Merge pull request #816 from nextstrain/feat/web-seq-marker-limit

• [b9cda4f] Merge branch 'master' into fix/default-gene

• [7481312] Merge branch 'master' into feat/cli-index-v2

• [1a9f69f] Merge branch 'master' into feat/web-index-v2

• [3186c45] fix(web): crash when accessing peptide warnings

• [35628d3] Merge pull request #817 from nextstrain/fix/web0crash-peptide-warnings

• [ff7aaf4] Merge pull request #814 from nextstrain/fix/default-gene

• [5c32330] fix(web): ensure dataset is read from URL

This ensures that the dataset is properly changed if the launch happens from URL.

With this, the previous dataset, persisted in local storage is used, which causes analysis errors if the requested dataset is not the same as the previously used dataset.

• [983a99e] fix(web): ensure seq name column and tooltip are shown for failed seq

For sequences that fail alignment the name column was blank and no tooltip was shown. This was due to incorrect conditional logic in the component.

Now the name column and its tooltip are displayed correctly.

• [f912dff] Merge pull request #820 from nextstrain/fix/web-name-col-on-errors

• [613637e] Merge pull request #818 from nextstrain/fix/web-dataset-from-url

• [c03ca81] Merge remote-tracking branch 'origin/master' into feat/cli-index-v2

• [1db84e5] Merge remote-tracking branch 'origin/master' into feat/web-index-v2

• [6587895] chore(deps): bump auspice version to 2.37.2

https://github.com/nextstrain/auspice/blob/master/CHANGELOG.md#version-2372---20220527

• [2dc3292] Merge pull request #821 from nextstrain/chore/bump-auspice

chore(deps): bump auspice version to 2.37.2

• [6c2da0c] feat(web): adapt to grouped params in index v2 json

• [129b74a] feat(cli): adapt to grouped params in index v2 json

• [cbb0a44] Merge pull request #812 from nextstrain/feat/web-index-v2

• [41968bb] Merge pull request #813 from nextstrain/feat/cli-index-v2

• [aedabb5] chore: release cli 2.0.0-alpha.3

2.0.0-alpha.2

⚠️	This is a pre-release. It can contain bugs and significant changes which are not yet finalized. Changes may appear without notice. We recommend to try the pre-releases to learn about upcoming features. For important projects, use stable releases.

Commit history

(click to expand)

• [66937c1] fix: make GapAlignmentSide values in JSON to resemble the ones in CLI

Uses serde's rename_all container attribute to convert variants of enum GapAlignmentSide to kebab case during (de)serialization, e.g. in virus_proerties.json.

The enum variants in Rust are by convention in PascalCase, and serde uses them as is. But we want to make the values in virus_proerties.json to resemble the values in CLI arguments, and clap seems to be converting enums to kebab case.

If merged, this change requires simultaneous adjustment of datasets using the gap_alignment_side param.

• [a5f9ab8] feat: warn if --genes requested not found in --genemap

• [022496a] feat: warn if non-empty genemap supplied but no genes found

• [e984954] chore(ci): create git tag on web releases [skip ci]

• [b00f086] chore(ci): create web releases [skip ci]

• [48229e3] chore: pre-release web v2.0.0-alpha.1 (on master)

• [4f81bfc] chore(ci): fix web release script, comment and document release process

• [d6f2706] chore(ci): fix typo in web deployment step

• [9e860b4] chore(ci): attempt to fix git tags for web

• [a0d0fef] Merge pull request #805 from nextstrain/fix/gap-alignment-side-kebab-case

fix: make GapAlignmentSide values in JSON to resemble the ones in CLI

• [c00634a] Merge branch 'master' into better-genemap-warnings

• [34876b0] refactor: avoid collecting, straight to string

• [1033081] refactor: bubble Err immediately

• [ae59a3c] chore: update lockfile using cargo build

• [597d8e4] Revert "chore: update lockfile using cargo build"

This reverts commit ae59a3c.

• [99f1235] chore: improve release script error messages

• [bad7333] chore: cargo build --locked in tip

• [0fe47e9] Merge pull request #806 from nextstrain/better-genemap-warnings

• [5c15a09] chore: release cli 2.0.0-alpha.2

2.0.0-alpha.1

⚠️	This is a pre-release. It can contain bugs and significant changes which are not yet finalized. Changes may appear without notice. We recommend to try the pre-releases to learn about upcoming features. For important projects, use stable releases.

Commit history

(click to expand)

• [a979442] feat: add example alignment with bio-rust

• [6f307df] feat: read sequences from files

• [8a80676] perf: add custom allocator and perf tweaks

• [c7016c4] feat: tweak params

• [d9899f1] docs: add readme

• [db6ff2a] Merge remote-tracking branch 'origin/master' into nextalign-rs

• [4deb48f] wip: reimplement alignment in rust

• [e5fe230] Merge remote-tracking branch 'origin/master' into nextalign-rs

• [fbbc1fb] refactor(rust): reimplement fasta parser

• [9ff0086] Merge remote-tracking branch 'origin/master' into nextalign-rs

• [a554fe3] fix: avoid underflow by using signed integers

• [a7a05ba] refactor: extract seed matching function into separate file

• [fc5531d] refactor: extract seed alignment function into separate file

• [9dbcd8a] refactor: rename macro to avoid name collision

• [36ab9cc] test: add unit tests for align pairwise

• [cabbf55] Merge remote-tracking branch 'origin/master' into nextalign-rs

• [5a12b89] test: test higher level function

• [e08d762] refactor: pass primitive by value

• [1a678ba] fix: correct indexing in score matrix

• [de6fc1f] Merge remote-tracking branch 'origin/master' into nextalign-rs

• [be15a84] chore: setup rust wasm

• [698110e] docs: fix image urls

• [ebe8a09] docs: add performance tips section

• [e1378be] docs: fix typos

• [d8321b6] docs: crosslink

• [b094aee] docs: align list of steps with the list in the ToC

• [4a9c2b4] docs: typo

• [968b19f] docs: link to qc section rather than reexplain

• [fea8edb] docs: reword

• [311e339] docs: make link more specific

• [06b4cf4] docs: format link, reword

• [7541b9f] chore: reorganize rust crates

• [14e4b53] chore: make sure package build and rebuild runs correctly

• [f2867ea] feat: serialize-deserialize auspice tree json

• [21f59e1] feat: add auspice tree traversal

• [69791b9] feat: run rust wasm-bindgen module in a webworker pool

• [dd899f4] refactor: ensure type safety across ts and rust using pojos

• [82e886d] refactor: move helper functions to a separate module

• [1f1428d] fix: clear buffer before reading more fasta

• [a0109c0] feat: write aligned fasta

• [c3fb9cd] fix: avoid integer underflow

• [dab5121] fix: incorrect size in alignment

• [e849a3e] fix: off by one errors in alignment

• [012c373] fix: typo in alignment test

• [233b242] feat: port insertions stripping

• [fa90a13] feat: port gff3 parsing

• [96e7975] feat: continue on alignment failure

• [6de1287] chore: update lockfile

• [cee163b] chore: cleanup

• [20130ca] fix: ensure gff3 parsing errors are not ignored

• [bb3ede0] feat: scaffold translation functions

• [c1d4a59] Merge remote-tracking branch 'origin/master' into docs/fixes

• [bf42062] Merge pull request #753 from nextstrain/docs/fixes

• [ccf2a04] chore(build): reenable nextalign smoke test

• [e7d3184] feat: port nextalign translation

• [e70cad8] feat: port nextalign command-line arguments

• [2dd67aa] feat: improve error message

• [b669d5a] fix: array out-of-bounds error in gaps counter

• [1aece1d] fix: ensure correct coord conversion in frame shift translation

• [c73c6e5] refactor: extract functions, rename, document

• [8073e97] feat: add --silent flag

• [c8d50d9] feat: make some of the cli args optional, same as in cpp version

• [474155b] fix: write peptides to the correct dir

• [e0bb45b] refactor: extract fasta reading function

• [194fdfe] feat: port insertions.csv

• [[79cdefd](https://github.com/nextstrain/nextclade/commit/79cdefd68739fcae01b...

1.11.0

Nextclade Web 1.14.0, Nextclade CLI 1.11.0, Nextalign CLI 1.11.0 (2022-03-15)

[Feature] Nextclade Pango classifier (SARS-CoV-2 specific)

With the latest dataset, Nextclade now assigns sequences a pango lineage, similar to how clades are assigned. The classifier is about 98% accurate for sequences from the past 12 months. Older lineages are deprioritised, and accuracy is thus worse. Read more about the method and validation against pangoLEARN and UShER in this report: Nextclade as pango lineage classifier: Methods and Validation.

[Feature] Better tree extensions #741

The Nextclade extensions fields in reference tree JSON now contain more information. This makes custom clade-like columns on the results page more informative.

[Feature] Update Auspice

The tree rendering component, Auspice, was updated to version 2.34.1. Refer to its changelog for details.

[Fix] Mutation badges showing incorrect position for first nucleotide #745

We fixed a bug where mutations at position 1 were incorrectly showing position 0 in Nextclade Web

1.10.3

Nextclade Web 1.13.2, Nextclade CLI 1.10.3, Nextalign CLI 1.10.3 (2022-02-15)

This is a bug fix release.

[Fix] Ignore private reversions of deletions when calculating divergence

Since their introduction, individual private reversions of deletions contributed extra to divergence. This was unexpected, so we removed reversions of deletions from consideration when calculating divergence in Nextclade.

[Fix] Sort private substitutions on the tree

Private substitutions are now sorted by position when they are displayed on the tree page

[Fix] "truncated" text in the insertions tooltip

In Nextclade Web, long insertions no longer have a visual bug when the "truncated" text is displayed twice.

1.10.2

Nextclade Web 1.13.1, Nextclade CLI 1.10.2, Nextalign CLI 1.10.2 (2022-02-01)

This is a bug fix release.

[Fix] Exclude reversions of deletions from consideration in "SNP clusters" QC rule

Since introduction of reversions in Nextclade Web 1.13.0 and Nextclade CLI 1.10.0, "SNP clusters" QC rule have been including reversions of deletions when counting clustered private mutations. This was unexpected and produced false-positives for some of the sequences. To fix that, we removed the reversions of deletions from consideration of this QC rule, so that it behaves as previously.

[Fix] Center markers in sequence view in Nextclade Web

In this version we improved the display of various colored markers (mutations, ranges etc.) in sequence and peptide views on the Results page of Nextclade Web. The individual markers are now centered around their position in the sequence (previously left-aligned). Although markers have moved by just a few pixels, this makes positioning more consistent, and ensures that different types of markers are correctly aligned across table rows.

1.10.1

Nextclade CLI 1.10.1 (2022-01-26)

[Fix] Improve error message when the virus properties file is missing #704

Since version 1.10.0 Nextclade CLI have introduced a new required input file, virus_properties.json and datasets and documentation were updated to match. However, users who don't use datasets might have encountered breakage due to a missing file: when running Nextclade CLI without either --input-dataset of --input-virus-properties flag provided, it would stop with an unclear error message. In this release we improve the error message, making sure that that explains the problem and offers a solution.

This does not affect Nextclade Web or Nextalign CLI.

In order to facilitate upgrades, for most users, we recommend to:

• download the latest dataset before each Nextclade CLI session (e.g. in the beginning of an automated workflow, or once you start a batch of experiments manually) using nextclade dataset get command
• use --input-dataset flag instead of individual --input-* flags for dataset files when issuing nextclade run command
• if necessary, override some of the individual input files using corresponding --input-* flags

[Fix] Add information about virus_properties.json or --input-virus-properties to changelog

In the excitement of bringing the new features, we forgot to mention virus_properties.json or --input-virus-properties in the changelog when Nextclade CLI 1.10.0 was released. We now added this information retroactively.

1.10.0

Nextclade Web 1.13.0, Nextclade CLI 1.10.0, Nextalign CLI 1.10.0 (2022-01-24)

💥 [BREAKING CHANGE] Nextclade: new required input file: virus_properties.json #689

This version introduces a new required input file for Nextclade, called virus_properties.json. This file contains additional information necessary for the "Detailed split of private mutations" feature (see below). The new versions of Nextclade datasets were released to account for this change.

How it affects different tools in the Nextclade family and how to upgrade:

• Nextclade Web - requires the new file. Migration path: no action is needed. Nextclade Web always uses the latest dataset automatically.

• Nextclade CLI - requires the new file. Migration path:

  • Download the latest dataset with nextclade dataset get command (dataset tagged 2022-01-18T12:00:00Z or more recent is required)
  • If using --input-dataset flag: the new file will be be picked up automatically from the latest dataset. No further action is needed.
  • If not using --input-dataset flag: add --input-virus-properties flag to point to virus_properties.json file from the dataset.

• Nextalign CLI - not affected: it does not use virus_properties.json. Migration path: no action is needed.

[Feature] Detailed split of private mutations (Nextclade) #689

Private mutations (differences between a query sequence and nearest neighbour in reference tree) are now split into three categories:

1. Reversion to reference genotype
2. (SARS-CoV-2 only for now) Mutation to a genotype common in at least 1 clade get labeled with that clade
3. Mutations that are neither reversions nor labeled (called "unlabeled")

Which category a mutation belongs to is visible by hovering over the "Mut." column in Nextclade Web and in various "privateNucMutations" fields in csv/tsv/json outputs.

[Change] "Private mutations" QC rule now accounts for reversions and labeled mutations

Reversions and labeled mutations (see feature above) are particularly common in contaminated samples, coinfections and recombination. To draw the user's attention to such sequences, both types of private mutation now get higher weights in the "Private mutations" QC rule (denoted as "P" in Nextclade Web, and qc.privateMutations in output files).

[Feature] Insertions now also available as amino acids #692

Aminoacid insertions in the query peptides relative to the corresponding reference peptide are now displayed in the "Ins." column in Nextclade Web and are emitted as "aaInsertions" and "totalAminoacidInsertions" fields in Nextalign and Nextclade output files. Note, that similarly to nucleotide insertions, aminoacid insertions are stripped from the output alignment.

[Fix] Gaps in query sequences are now stripped correctly #696

When query sequences contained gaps (-), e.g. when inputting aligned sequences, gaps were not stripped correctly since v1.7.0 (web v1.10.0), which could lead to - showing up in insertions.

1.9.0

Nextclade Web 1.12.0, Nextclade CLI 1.9.0, Nextalign CLI 1.9.0 (2022-01-11)

[Feature] Handle "-" strand gene translation

The strand column in the gene map file was previously ignored. Now the "-" strand genes are correctly reverse-complemented before translation. Thanks @dnanto for the contribution.

[Feature] Update SARS-CoV-2 clade schema

The schema that illustrates the tree of SARS-CoV-2 clades on the main page of Nextclade Web was updated to account for recent clade changes.

[Fix] Center mutation markers in sequence views

Previously the mutation markers in sequence views in results table of Nextclade Web were anchored to their position in the sequence view on their left edge. They are now correctly centered around their position, such that the center of marker is at the corresponding position in the sequence.

[Fix] Correct exit codes

Nextclade CLI and Nextalign CLI could sometime exit with incorrect exit code. This has been fixed.

[Fix] Correctly handle empty peptides

The alignment algorithm in Nextclade CLI and Nextalign CLI could sometimes produce translation that is longer than expected, when the translated sequence is empty. Now the empty peptides are discarded and a warning is issued.

[Fix] Ensure array boundaries

In rare cases Nextclade and Nextalign algorithms could sometimes read past the end of arrays, which previously went undetected. This is now fixed.