This is a Nextflow pipeline for Influenza A mutation analysis from either short-read FASTQ data or pre-built consensus FASTA files.
The short-read workflow performs trimming, assembly, mapping, LoFreq variant calling, SnpEff annotation, and consensus generation. In FASTA mode (--fasta_input), the pipeline currently launches both a PAFTOOLS branch and a NEXTCLADE branch: PAFTOOLS validates the supplied FASTA, selects a subtype-matched reference with BLAST, aligns per segment with minimap2, calls differences with paftools.js, annotates with SnpEff, and fixes multi-nucleotide variant codons; NEXTCLADE validates the same input, batches sequences by segment for nextclade run, and converts the results to VCF, annotating substitutions that share a codon with their combined codon change.
- Read trimming & QC: fastp
- Primer removal with cutadapt
- FASTQ quality reporting with FastQC
- Aggregate the reports with MultiQC
- De-novo assembly
- Automated reference selection from built-in database via BLAST
- Reference-based read mapping (BWA or minimap2)
- Nucleotide variant calling using LoFreq
- Annotation of amino acid variant calls using SnpEff
- MNV codon correction
- Detection of mutations of interest via customisable watchlist scanning of the final VCF
- Consensus sequence generation
- Detection of influenza molecular markers with FluMut
- Consensus FASTA validation and header normalisation
- Automated reference selection from built-in database via BLAST
- Per-segment alignment using minimap2 (
asm20mode) - Variant calling using
paftools.js - Annotation of amino acid variant calls using SnpEff
- MNV codon correction
- Alignment coverage QC with run-level aggregation
- Detection of influenza molecular markers with FluMut
- Consensus FASTA validation and header normalisation
- Per-segment batching across samples
- Per-segment
nextclade runagainst built-in datasets - Per-segment QC from the Nextclade alignment; a segment that fails gets no VCF
- Conversion to freebayes-style VCF, with substitutions that share a codon annotated as one codon change
- Detection of influenza molecular markers with FluMut
- Optional (
--merge_flumut_mutations, default off): reference coordinate conversion — builds a run-level, idempotent nucleotide/amino-acid coordinate map lifting flumut's fixed reference onto the selected Nextclade reference dataset coordinate system (bin/build_reference_coordinate_map.py) — followed by FluMutDB annotation of the VCF, which tags each fluscan amino-acid call that matches a curated FluMutDB marker with aFLUMUT=INFO field carrying the mutation name and its published effects, split into those directly attributable to that mutation and those associated with it through a multi-mutation marker, each qualified by the subtype it was observed in. No rows are added or removed, so a marker is only reported where fluscan called a difference from the Nextclade reference. Because FluMutDB numbers HA by its HA1/HA2 subunits, HA markers are only annotated when the Nextclade dataset also exposes that split — either as two dedicatedHA1/HA2CDS features, or asmature_protein_region_of_CDSchildren of a single HA CDS (the NCBI/RefSeq model, which the GsGD dataset uses). A dataset offering only a bareHACDS is flaggedstale_annotationand passes through with its HA rows unannotated
flowchart TD
input[Consensus FASTA input directory]
dispatch[main.nf FASTA dispatch]
input --> dispatch
dispatch --> prepare
subgraph FASTA_PREP[FASTA_PREP workflow]
direction TB
prepare[prepare_consensus]
prep_flumut[flumut]
prepare --> prep_flumut
end
subgraph PAFTOOLS[PAFTOOLS workflow]
direction TB
blastn[blastn reference selection]
covagg[aggregate coverage summary]
paf_final[Final PAFTOOLS VCFs]
subgraph paftools_variants[paftools_variants, one task per sample]
direction TB
minimap[minimap2 asm20]
covqc[coverage QC]
paftools_call[paftools.js call]
snpeff[SnpEff]
paf2fb[paftools_to_freebayes_vcf.py]
fixmnv_paf[fix_mnv_codons.py]
minimap --> covqc
minimap --> paftools_call
paftools_call --> snpeff
snpeff --> paf2fb
covqc --> paf2fb
paf2fb --> fixmnv_paf
end
prepare --> blastn
prepare --> minimap
blastn --> minimap
covqc --> covagg
fixmnv_paf --> paf_final
end
subgraph NEXTCLADE[NEXTCLADE workflow]
direction TB
groupseg[collect each segment across samples: splitFasta + collectFile]
nc_run[nextclade run batch]
nc2vcf[nextclade_to_freebayes_vcf.py: QC + VCF]
nc_final[Final NEXTCLADE VCFs]
coord_map[coordinate_conversion: build_reference_coordinate_map.py]
coord_final[aggregate/nextclade/db/reference_remap]
flumut_merge[merge_flumut_mutations: merge_flumut_mutations.py]
flumut_annotated[FluMutDB-annotated VCFs]
prepare --> groupseg
groupseg --> nc_run
nc_run --> nc2vcf
nc2vcf --> nc_final
dispatch -.->|"flumutdb reference + db nextclade datasets, sample-independent (if merge_flumut_mutations)"| coord_map
coord_map --> coord_final
coord_map -.->|"per-segment shard (if merge_flumut_mutations)"| flumut_merge
nc_final -.-> flumut_merge
flumut_merge --> flumut_annotated
end
Short-read workflow: Illumina paired-end FASTQ files (*.fastq.gz, *.fq.gz, *.fastq, *.fq), or a sample sheet (below).
PAFTOOLS/NEXTCLADE workflows: Consensus FASTA files (one multi-FASTA per sample, one sequence per influenza segment). Two header formats are accepted:
- Underscore-delimited (legacy):
>SAMPLE_ID_segmentN_SEGMENTe.g.>WIN-AH-2024-FAV-0415_segment1_PB2 - Pipe-delimited (GISAID-style):
>EPI_ISL_ACCESSION|subtype|SEGMENT|...|clade_SEGMENTe.g.>EPI_ISL_20399898|A_/_H5N1|NA|6|EPI5261179|2.3.4.4b_NA
Cleaned output headers ({sample_id}.fasta) are normalised to {sample_id}_{SEGMENT} (e.g. EPI-ISL-20399898_NA).
--samplesheet_input takes a CSV in place of --fastq_input or --fasta_input. Its header picks the workflow:
ID,R1,R2
SAMPLE-01,/path/to/SAMPLE-01_R1.fastq.gz,/path/to/SAMPLE-01_R2.fastq.gz
SAMPLE-02,/path/to/SAMPLE-02_R1.fastq.gz,/path/to/SAMPLE-02_R2.fastq.gzruns the short-read workflow, and
ID,FASTA
SAMPLE-01,/path/to/SAMPLE-01.fasta
SAMPLE-02,/path/to/SAMPLE-02.fastaruns PAFTOOLS and NEXTCLADE. A FASTA sheet may add a DB column naming each sample's
reference bundle, so one run can mix subtypes:
ID,FASTA,DB
SAMPLE-01,/path/to/SAMPLE-01.fasta,/path/to/refs/gsgd
SAMPLE-02,/path/to/SAMPLE-02.fasta,/path/to/refs/wisconsin588A blank DB cell falls back to --db; see docs/database_bundles.md. Other columns are ignored.
- Each
IDis cleaned by the same rule as a FASTA filename (see the note under Usage), so24-1234-IP7-SAMPLE_01is reported asSAMPLE-01. After cleaning, an ID may contain only letters, digits,.and-. - Relative paths resolve against the directory
nextflow runis launched from; absolute paths are safer. - The run stops before any task starts if a column or value is missing, a file does not exist, or two IDs clean to the same sample name.
run_name, which prefixes the run-level reports, is the sample sheet's filename without.csv.aggregate/sample_ids.csvmaps each input ID (or FASTA filename) to its cleanedsample_idand bundle name.- The short-read workflow takes a single
--dband rejects a sheet with aDBcolumn. - Give only one of
--fastq_input,--fasta_inputand--samplesheet_input.
Every reference the pipeline needs for one reference set lives in a single directory — a
bundle — passed as --db:
<db>/
├── db.yml optional metadata (name, version, …)
├── blastn_db.fa nt reference; headers accession|subtype|segment
├── nextclade/ one uppercase directory per segment (HA/, NA/, …)
└── snpeff/ snpEff.config + <genome>/
Bundles may be partial — each workflow validates only the components it consumes, and
fails at the start of the run, before any task is submitted, naming the path it expected.
See docs/database_bundles.md for the full layout, the
db.yml schema, and how to build and validate a bundle.
scripts/validate_db.py /path/to/refs/gsgdExample command (short-read):
nextflow run BCCDC-PHL/fluscan-nf \
-r v0.1.0 \
-profile conda \
--cache ~/.conda/envs \
--fastq_input /path/to/your_fastqs \
--db /path/to/refs/gsgd \
--outdir /path/to/output_dir
To run from a sample sheet, replace --fastq_input (or --fasta_input below) with --samplesheet_input /path/to/samplesheet.csv; see Sample sheet input.
Optional short-read parameters:
--aligner—bwa(default) orminimap2
Optional in every workflow:
--watchlist_dir— directory of*watchlist.csvfiles (gene,mutation[,note]) scanned out of the final VCF into a published TSV. Defaults toassets/watchlists/; set toNO_FILEto disable.HA1/HA2entries are matched against the reference annotation's mature-peptide coordinates, so they work whether or not the VCF annotates the subunits directly. See docs/watchlists.md
FASTA-mode example (runs both PAFTOOLS and NEXTCLADE):
nextflow run BCCDC-PHL/fluscan-nf \
-r v0.1.0 \
-profile conda \
--cache ~/.conda/envs \
--fasta_input /path/to/consensus_fastas \
--db /path/to/refs/gsgd \
--merge_flumut_mutations \
--outdir /path/to/output_dir
Note: In FASTA mode the sample name comes from the input filename (or the sample sheet
ID, cleaned by the same rule), and it names every output. The filename is taken without its extensions, a trailing_consensusis dropped, underscores become dashes, and a leading submitter prefix of the form24-1234-or24-1234-IP7-is stripped — so24-1234-IP7-SAMPLE_A_consensus.fastais reported asSAMPLE-A. Input FASTA headers are read only for the segment label; they do not affect naming.FASTA mode launches both branches, so the bundle must carry all three components: PAFTOOLS requires
blastn_db.faandsnpeff/, NEXTCLADE requiresnextclade/with all eight segment datasets.In the short-read workflow only
blastn_db.fais required. SnpEff annotation — and the MNV correction and watchlist scan that read its output — runs when the bundle carries asnpeff/directory, and is skipped when it does not; consensus and FluMut complete either way.
--merge_flumut_mutations(boolean, defaultfalse) gates the two final NEXTCLADE steps — coordinate conversion and FluMutDB annotation of the VCF (*_flumut_annotated.vcf). Thecoordinate_conversionstep lifts FluMut's own reference, read from theflumutdbconda package, onto the--dbdatasets. The marker set comes from the same package, so the two always match and the annotation step does not consume the per-sampleflumutoutput.
--emit_no_mature_vcf(boolean, defaultfalse) additionally writes each NEXTCLADE VCF a second time with no mature-peptideANN=entries, for consumers that require HA annotated as HA0 only. The two files carry the same records, alleles and order; only theANN=sub-field differs, and segments without mature peptides in their annotation come out identical. The published copy lands innextclade/no_mature_peptide/. Nothing downstream in the pipeline reads it:mutation_watchandmerge_flumut_mutationsresolve HA markers through the HA1/HA2Feature_IDand stay on the standard VCF.
Package management:
- Conda is used for dependency management. The
-profile condaflag and--cacheparameter are required on BCCDC systems. - If environment resolution fails on first run, ensure
mambais available (the pipeline usesconda.useMamba = true).
Outputs are written to the directory specified with the --outdir parameter.
Sample-level outputs are written to one folder per sample, and some workflows also emit run-level aggregate outputs.
| File | Description |
|---|---|
*.fastp.html |
Per-sample fastp HTML report |
*.fastp.json |
Per-sample fastp JSON report |
*_contigs.fasta |
De-novo assembled contigs from SPAdes |
*_consensus.fasta |
Multi-FASTA containing the consensus sequences for each segment |
blast/*_ref.fasta |
Multi-FASTA containing the optimal reference sequences selected for each segment |
blast/*_blastn.tsv |
BLAST search results of assembled contigs against the reference database |
blast/*_blastn_filter.tsv |
Filtered BLAST search results |
*.sort.filter.bam |
Sorted, filtered alignment BAM used for downstream calling and coverage calculations |
*_coverage.png |
Line plots describing mapping coverage across all 8 flu segments |
*_pileup.tsv |
Pileup TSV with raw counts of all 4 nucleotides and gaps |
*_depth.tsv |
Per-position alignment depth (samtools depth) |
*_low_coverage.bed |
Low-coverage regions masked during consensus generation |
short_read/*_final.vcf |
Per-segment final VCF after SnpEff annotation and MNV codon correction |
watchlist/<list>/*_<list>_sr_watchlist_hits.tsv |
Optional (--watchlist_dir): watchlist substitutions found in the final VCF, one file per segment |
flumut/*_flumut_markers.tsv |
FluMut table of molecular markers identified in the consensus sequences |
flumut/*_flumut_mutations.tsv |
FluMut long-format table of amino acid mutations at positions of interest |
flumut/*_flumut_mutations_wide.tsv |
FluMut wide-format table of amino acid mutations at positions of interest |
flumut/*_flumut_literature.tsv |
FluMut literature references associated with the identified markers |
*_short_read_*_provenance.yml |
Aggregated per-sample provenance file for the short-read workflow |
The PAFTOOLS and NEXTCLADE VCFs declare and write the placeholder FORMAT/GQ as an integer.
| File | Description |
|---|---|
${sample_id}.fasta |
Validated consensus FASTA with headers normalised to {sample_id}_{SEGMENT} |
qc/*_qc_report.tsv |
QC report from consensus preparation |
blast/*_ref.fasta |
Multi-FASTA containing the optimal reference sequences selected for each segment |
blast/*_blastn.tsv |
BLAST search results against the reference database |
blast/*_blastn_filter.tsv |
Filtered BLAST search results |
qc/*_coverage_qc.tsv |
Per-segment PAFTOOLS consensus coverage QC with coverage percentage and PASS/WARN/FAIL status. A segment with no BLAST reference, or that minimap2 cannot align, is FAIL and has no VCF |
paftools/*_final.vcf |
Per-segment final VCF after paftools.js, SnpEff, freebayes-style conversion, and MNV codon correction |
watchlist/<list>/*_<list>_pf_watchlist_hits.tsv |
Optional (--watchlist_dir): watchlist substitutions found in the final VCF, one file per segment |
flumut/*_flumut_markers.tsv |
FluMut table of molecular markers identified in the consensus sequences |
flumut/*_flumut_mutations.tsv |
FluMut long-format table of amino acid mutations at positions of interest |
flumut/*_flumut_mutations_wide.tsv |
FluMut wide-format table of amino acid mutations at positions of interest |
flumut/*_flumut_literature.tsv |
FluMut literature references associated with the identified markers |
*_paftools_*_provenance.yml |
Aggregated per-sample provenance file for the PAFTOOLS workflow |
| File | Description |
|---|---|
nextclade/*_final.vcf |
Per-segment final VCF from the Nextclade-to-VCF conversion, written only for segments that do not FAIL QC (see aggregate/nextclade_qc_summary.tsv). Rows in a codon holding more than one substitution carry the combined change, plus MNV_CORRECTED and MNV_ORIGINAL_HGVS_P where it differs from the call before codon correction (Nextclade's, otherwise the single-base change) |
nextclade/no_mature_peptide/*_nc_nomat_final.vcf |
Optional (--emit_no_mature_vcf): the same records annotated without mature peptides, so HA carries only its HA0 ANN= entry and no HA1/HA2 ones |
nextclade/flumut_annotated/*_flumut_annotated.vcf |
Optional (--merge_flumut_mutations): the same VCF with a FLUMUT= INFO field (Protein|AA_Pos|Alt_AA|Mutation_Name|DirectEffect|AssociatedEffect) on rows whose amino-acid change matches a curated FluMutDB marker |
watchlist/<list>/*_<list>_nc_watchlist_hits.tsv |
Optional (--watchlist_dir): watchlist substitutions found in the final VCF, one file per segment, beside the PAFTOOLS (_pf_) ones. Independent of --merge_flumut_mutations: FLUMUT= is a curated annotation written into the VCF, this is a custom list scanned out of it |
flumut/*_flumut_markers.tsv |
FluMut table of molecular markers identified in the consensus sequences |
flumut/*_flumut_mutations.tsv |
FluMut long-format table of amino acid mutations at positions of interest |
flumut/*_flumut_mutations_wide.tsv |
FluMut wide-format table of amino acid mutations at positions of interest |
flumut/*_flumut_literature.tsv |
FluMut literature references associated with the identified markers |
*_nextclade_*_provenance.yml |
Aggregated per-sample provenance file for the NEXTCLADE workflow |
| File | Description |
|---|---|
aggregate/${run_name}_fluscan-nf_multiqc_report.html |
MultiQC report summarising fastp, cutadapt, and FastQC results for all samples (short-read workflow only) |
aggregate/paftools_qc_summary.tsv |
Run-level tabular summary of PAFTOOLS alignment coverage across all FASTA samples and segments |
aggregate/nextclade_qc_summary.tsv |
One row per sample and segment Nextclade saw (NEXTCLADE workflow). status is FAIL, with no VCF written, when Nextclade could not align the segment (not_aligned), reports a frameshift (frameshift), or scores its stop codons bad (stop_codons); WARN when coverage_pct is below --consensus_coverage_pass_pct (default 90); otherwise PASS. coverage_pct is the share of the reference covered by called (non-N) bases, so a truncated or N-heavy segment WARNs but keeps its VCF. identity_pct is exact matches over the called bases plus insertions; it has no threshold and is low for a segment from a different lineage than the reference |
aggregate/sample_ids.csv |
input_id,sample_id,db: each input's sample sheet ID (or FASTA filename), its cleaned sample_id, and the bundle it ran against (FASTA mode) |
aggregate/consensus_qc_summary.tsv |
Every *_qc_report.tsv row from consensus preparation, concatenated across the run. A sample rejected at preparation — duplicate segment, or headers naming more than one sample — contributes no rows and is reported only in the Nextflow log |
aggregate/nextclade/<db>/<SEGMENT>/batch.* |
Raw nextclade run --output-all output for one bundle and segment across every sample in the run (NEXTCLADE workflow) |
aggregate/nextclade/<db>/reference_remap/coordinate_map.json |
Optional (--merge_flumut_mutations): primary machine-readable nucleotide/amino-acid coordinate map lifting flumut's reference (source) onto bundle <db>'s Nextclade reference datasets (target) |
aggregate/nextclade/<db>/reference_remap/coordinate_map.nt.tsv |
Flat per-position nucleotide coordinate map (source/flumut position/base -> target/nextclade position/base, per segment) |
aggregate/nextclade/<db>/reference_remap/coordinate_map.aa.tsv |
Flat per-residue amino-acid coordinate map (source/flumut position/residue -> target/nextclade position/residue, per segment/gene) |
aggregate/nextclade/<db>/reference_remap/coordinate_map.qc.tsv |
Per-segment/per-gene alignment QC: identity, coverage, the number of co-optimal alignments the map's gap placement was chosen from, and status (ok/warning/missing_in_source/extra_in_source/stale_annotation) |
aggregate/nextclade/<db>/reference_remap/flumut_alignment.fasta |
Target/source (nextclade/flumut) nucleotide alignment blocks, one per segment |
aggregate/nextclade/<db>/reference_remap/flumut_aa_alignment.fasta |
Target/source (nextclade/flumut) amino-acid alignment blocks, one per gene |
${run_name}_fluscan-nf_nextflow_report.html |
Nextflow execution report |
${run_name}_fluscan-nf_nextflow_timeline.html |
Nextflow execution timeline |
${run_name}_fluscan-nf_nextflow_trace.txt |
Nextflow task trace |
All three workflows publish one aggregated ${sample_id}_<workflow>_<timestamp>_provenance.yml file per sample, where <workflow> is short_read, paftools, or nextclade. FASTA mode runs both PAFTOOLS and NEXTCLADE over the same input, so each sample gets two files; the workflow label is what tells them apart.
Every record opens with the pipeline and reference-database header, followed by the SHA-256 of each input file — the paired FASTQs in short-read mode, the consensus FASTA in FASTA mode. A sample whose prepare_consensus or blastn step fails produces no provenance file at all, since those processes run errorStrategy 'ignore' and the collection joins are inner joins.
Note the example below is contrived.
- pipeline_name: BCCDC-PHL/fluscan-nf
pipeline_version: 0.1.0
nextflow_session_id: 59fdd919-fc28-4af5-99e0-60355b11807c
nextflow_run_name: hopeful_bhaskara
timestamp_analysis_start: 2024-07-15T16:30:11.150887-07:00
- input_filename: sample-01_R1.fastq.gz
file_type: fastq_input
sha256: 2c1d5b310a5ca11cc2b1665c094a064fc3aa597e06f392985dac639bd2ab4d81
- input_filename: sample-01_R2.fastq.gz
file_type: fastq_input
sha256: 73745eed4badc3594cdd8554e90c75ae4b9b4599ca255064415ded433e598749
- process_name: fastp
tools:
- tool_name: fastp
tool_version: 0.23.2
- process_name: cutadapt
tools:
- tool_name: cutadapt
tool_version: 4.4
- process_name: assembly
tools:
- tool_name: spades
tool_version: v3.15.3
- process_name: lofreq
tools:
- tool_name: samtools
tool_version: 1.18
- tool_name: lofreq
tool_version: 2.1.5
- tool_name: bcftools
tool_version: 1.18
- process_name: flumut
tools:
- tool_name: flumut
tool_version: 0.8.0