Repository navigation
Optimize the Nextclade workflow - #25
Merged
Merged
Conversation
The converter now translates each codon with all of its substitutions applied and writes MNV_CORRECTED/MNV_ORIGINAL_HGVS_P itself, so the Nextclade branch no longer runs fix_mnv_codons.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nextclade_run_batch now writes every sample's VCF from the batch JSON, replacing the per-sample loop in nextclade_to_vcf_batch. A segment where no sample aligns publishes no VCFs instead of failing the task. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nextclade_post_process.py derives identity from the batch TSV instead of realigning every segment with Biopython. On 519 test segments 169 match exactly and the mean shift is -0.09 points; a segment Nextclade cannot align is now FAIL with identity NA. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
splitFasta and collectFile write each segment's batch FASTA in the head job, replacing a split_fasta task per sample and the header rewrite loop in nextclade_run_batch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mutation_watch runs once per sample over all its VCFs and watchlists, and merge_flumut_mutations.py annotates a whole segment in one call. Published outputs are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FASTA_PREP runs hash_files, prepare_consensus and flumut once and feeds PAFTOOLS and NEXTCLADE. The two branch-named consensus QC aggregates become aggregate/consensus_qc_summary.tsv. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Output is byte-identical to the previous commit on the 40-sample batch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nextclade's peptide alignment can gap a codon its nucleotide alignment keeps: at the LPAI H5 cleavage site it called K345- for an ACA (Thr) codon. Output now matches main except two NA deletion rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rrtp3W2XVRynKEUHkxs6e9
The merge reads only Nextclade VCFs, which never carry the flag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rrtp3W2XVRynKEUHkxs6e9
Every short-read segment wrote {sample}_{list}_watchlist_hits.tsv, so
only the last segment's hits survived.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rrtp3W2XVRynKEUHkxs6e9
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cuts the NEXTCLADE branch's per-(sample, segment) jobs. The converter corrects MNV codons itself and converts a whole segment batch in one call. Identity QC comes from Nextclade's own alignment. Segment batches are built with
splitFasta/collectFile. Watchlists are scanned once per sample and FluMut is merged once per segment. FASTA prep runs once for both FASTA branches. The last four commits come from review: they simplify the converter's codon logic, translate shared codons from the nucleotide alignment, drop an unreachable merge check, and name short-read watchlist hits by segment.Where to look:
bin/nextclade_to_freebayes_vcf.py(_codon_aa_change,_ann_field_for_sub): codon-aware HGVS.p and theMNV_*fields fluscan-db ingests. A codon with 2+ substitutions is translated from the nucleotide alignment; Nextclade'snucToAaMutscall is used only for a lone substitution, because its peptide alignment can gap a codon the nucleotide alignment keeps (HAK345-for an ACA codon at the LPAI H5 cleavage site).bin/nextclade_post_process.py(identity_pct): identity from the Nextclade TSV columns.modules/shared/mutation_watch.nfand its three call sites.workflows/fasta-prep.nfand themain.nfdispatch.Output changes:
aggregate/{paftools,nextclade}_consensus_qc_summary.tsvbecome oneaggregate/consensus_qc_summary.tsv.mainexcept two NA deletion rows in EPI-ISL-63641 (POS 152, 182), whichmain'sfix_mnv_codons.pyhad rewritten fromp.P3fsto missensep.H44Y/p.I54K.1.0/60.0/-6.000instead of1/60/-6, and the header dropsMNV_UNRESOLVEDand rewords theMNV_*lines. cyvcf2 (fluscan-db) reads the same values; text or checksum comparisons with older runs will differ.MNV_UNRESOLVED, which Nextclade VCFs never carry.{sample}_{SEGMENT}_{list}_watchlist_hits.tsvper segment. Before, every segment wrote the same name and only one survived.FAIL/NAinstead ofWARN.nextclade_run_batchrecord per segment.indexfields follow the new (sorted) batch order.Testing:
test_run_snpeff.pyalso fail onmain.mainvs this branch on 40 GISAID consensus FASTAs (macOS): tasks 1,971 → 685, wall time 22m → 9m33s. Published file sets match apart from the rename above. PAFTOOLS VCFs and PAFTOOLS/Nextclade watchlist hits are byte-identical, and everyFLUMUT=value matches. Of 63,640 rows in each Nextclade VCF kind, 2 differ in annotation (the NA deletion rows above).N329Khit is kept; under the old shared name the NS scan, which runs last, overwrote it. MultiQC andshort_read_make_pileupwere set to ignore for this run: MultiQC crashes on a polars bug in the local env, andmake_pileup.pyhas been missing frombin/since Improve QC checking and reporting #15, so SHORT_READ fails onmaintoo.test_current_vcf_compatibility.py: 9 of 10 pass on bothmainand this branch.test_ha_mat_peptides_get_their_own_rowsfails on both because the GsGD bundle names the HA mature peptides by accession, notHA1.Not yet run on the cluster.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Rrtp3W2XVRynKEUHkxs6e9