Author: Shao Mengjie; Liang Qian; Xiaoting Hua
Email: 1437819081@qq.com; norah-liang@dmicrobe.com; xiaotinghua@zju.edu.cn
This program is designed for annotation of antimicrobal resistance(AMR), pdif site and pdif-ARGs module in bacteria.
PdifFinder is a python3.X script, running on linux.
You should install BLAST and add it in environment variable, you can download from https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/. BLAST version is 2.10.1 in pdifFinder.
- One:
You can download from github by
git clone https://github.com/mjshao06/pdifFinder.git. Then executecd pdifFinder. Last executepip install .. - Two:
You can install PdifFinder from PyPI by
pip install PdifFinder.
PdifFinder can accept FASTA and GENBANK format file(single or multi sequences in one file). Attention on GENBANK format file, it should follow standard format. There are four input parameters: "-i" means FASTA, "-g" means GENBANK, "-d" means input dir contains FASTA or GENBANK, "-s" sets circular sequence mode.
- Simply, you can just run:
pdifFinder -i FASTA -o outdir
pdifFinder -g GENBANK -o outdir
pdifFinder -d inputdir -o outdir
pdifFinder -i FASTA -o outdir -m pwm
pdifFinder -i FASTA -o outdir -m hall -e POS1 POS2
- For more parameters, you can run:
pdifFinder -h
- You can also use long form flags:
pdifFinder --inFile FASTA --outdir outdir
- Here are some important parameters:
| parameter | description |
|---|---|
| --inFile(-i) | FASTA file |
| --genbankFile(-g) | GENBANK file |
| --indir(-d) | input dirname |
| --outdir(-o) | output dirname |
| --circle(-c) | output graph format, default is circle |
| --circular-seq(-s) | treat sequence as circular (auto|true|false), default auto-detect from genbank |
Here are databases structure:
.
├── AMRDB
│ ├── sequence.fasta Resistance gene reference sequences in FASTA format
│ │ sequence id must be database name~~~gene~~~accession~~~description,
│ │ eg: ncbi~~~1567214_ble~~~NG_047553.1~~~BLEOMYCIN BLMA family bleomycin binding protein
│ ├── Res.nhr
│ ├── Res.nin
│ └── Res.nsq
│ └── Res.ndb
│ └── Res.not
│ └── Res.nto
│ └── Res.ntf
└── data
├── redundant.seed.fa Pdif site reference sequences in FASTA format
sequence id must be database >plasmid accession number in NCBI
eg: >KY984047.1_1 ACTGCGCATAAGAGATTTTATGTTAAAT
├── pdifdatabase.fasta ALL pdif sites from 481 plasmids
└── genecolor.txt
| filename | description |
|---|---|
| AMRgene.txt | resistance gene annotation |
| pdif_site.txt | pdif site annotation |
| pdifmodule_list.txt | pdif-ARGs module annotation |
| pdifmoduleseq.fasta | pdif-ARGs module sequence |
| pdifmodule.svg | pdif-ARGs figure |
| plasmid.html | circular graph for above features |
PdifFinder uses a sliding window algorithm to detect 28bp pdif sites in bacterial plasmids. The algorithm implements the following features:
Whole Plasmid Scanning:
- Scans entire plasmid sequences instead of just feature-end fragments
- Uses a 28bp sliding window to examine all possible pdif site locations
- Maintains biological parameters: XerC binding site (positions 0-11), XerD binding site (positions 17-28)
Performance Optimizations:
- 45% speedup through early exit strategies in mismatch counting
- Pre-calculated sequence limits to reduce computational overhead
- Efficient loop structures for XerC and XerD mismatch checking
Bug Fixes:
- Critical loop range bug fixed (now scans all possible starting positions)
- Variable name conflicts resolved to prevent control flow corruption
- Spelling errors corrected throughout codebase
- Mathematical error in batch processing fixed
- Redundant comparisons eliminated
Thread Safety:
- File operations made thread-safe with locking mechanisms
- Supports concurrent processing of multiple sequences
Circular Genome Support:
- Detects circular topology from genbank files automatically
- --circular-seq flag allows manual override (auto|true|false)
- Circular-aware sliding window wraps around sequence boundaries
- Properly handles XerC/XerD orientation detection for circular plasmids
Detection Requirements:
- pdif site detection requires nearby resistance genes for module formation
- Sites must be properly paired (XerC-XerD orientation) for detection
- The algorithm has 0% detection rate on isolated pdif sequences without resistance gene context
Performance Characteristics:
- Time complexity: O(n×s×k) where n=sequence length, s=seed pairs, k≤11
- Memory usage: Linear with sequence length
- Suitable for plasmids up to 200kbp in length
PdifFinder includes a comprehensive test suite:
Unit Tests:
- Bug fix verification tests
- Performance optimization tests
- Algorithm correctness tests
Integration Tests:
- End-to-end pipeline validation
- Input/output format testing
- Large sequence handling tests
Validation Tests:
- Real pdif sequence validation
- False positive rate testing
- Biological parameter verification
All tests can be run with pytest tests/ from the project root directory.
A dedicated benchmark suite that compares pdifFinder's pdif site detection against manually curated reference data from published literature. It measures recall, precision, positional accuracy, and false positive rate.
Validate pdifFinder's core pdif detection algorithm against independently reported sites from peer-reviewed studies. The benchmark catches regressions, quantifies detection quality, and identifies systematic failure modes (seed mismatch, pairing failure, positional drift).
| Mode | Description | Requirements |
|---|---|---|
pdif_only |
Tests pdif site detection in isolation. Mocks AMR gene detection so the sliding-window algorithm runs without real BLAST hits. | None beyond pdifFinder itself |
full_pipeline |
End-to-end test with real blastn-based AMR gene detection. Calls singleThread() with no mocking. |
blastn must be installed and in PATH |
- pdif_only mode: No additional dependencies beyond pdifFinder and its standard Python packages (BioPython, etc.)
- full_pipeline mode: requires BLAST+ (version 2.10.1+), with
blastnavailable in the system PATH. The pipeline checks availability at runtime and skips gracefully if missing.
Run the benchmark from the project root:
# pdif-only mode (quick, no blastn required)
python -m tests.benchmark_literature --mode pdif_only
# full pipeline mode (requires blastn)
python -m tests.benchmark_literature --mode full_pipeline
# Filter to a single plasmid
python -m tests.benchmark_literature --mode pdif_only --plasmid KY984047.1
# Verbose debug output
python -m tests.benchmark_literature --mode pdif_only --verboseResults are written to tests/benchmark_data/results/<timestamp>/ as JSON.
| Metric | Target | Description |
|---|---|---|
| Detection Rate (Recall) | >= 0.80 | Fraction of literature reference sites that were matched by pdifFinder within the position tolerance |
| Precision | >= 0.70 | Fraction of pdifFinder-detected sites that correspond to a known reference site |
| Position Median Error | <= 5 bp | Median absolute deviation between reported and reference pdif site coordinates for matched sites |
| FPR per kb | <= 0.01/kb | False positive rate per kilobase, measured on negative-control plasmids (plasmids with no known pdif sites) |
A site is considered "matched" when both start and end coordinates fall within 5 bp of the reference (the POSITION_TOLERANCE constant) and the 28 bp sequence matches either directly or as reverse complement.
Benchmark results are saved as JSON with the following structure:
{
"mode": "pdif_only",
"timestamp": "2026-05-10T22:46:41",
"plasmid_filter": null,
"comparisons": [
{
"plasmid_accession": "KY984047.1",
"matched_pairs": [
{"ref_index": 0, "det_index": 3, "ref_start": 1312, "ref_end": 1339,
"det_start": 1312, "det_end": 1339}
],
"missed_references": [],
"unvalidated_detections": [],
"failure_categories": {}
}
],
"failures": [],
"metrics": {
"detection_rate": 0.95,
"precision": 0.88,
"position_median_error": 0.0,
"position_mean_error": 0.5,
"position_max_error": 3.0,
"fpr_per_kb": -1.0,
"per_plasmid_breakdown": [
{
"plasmid_accession": "KY984047.1",
"reference_count": 8,
"matched_count": 8,
"missed_count": 0,
"unvalidated_count": 1,
"detection_rate": 1.0,
"failure_categories": {}
}
],
"failure_category_summary": {},
"total_reference_sites": 43,
"total_matched_sites": 41,
"total_missed_sites": 2,
"total_unvalidated": 5
},
"output_dir": "tests/benchmark_data/results/20260510_224641"
}Key output fields:
- comparisons: Per-plasmid comparison results showing matched, missed, and unvalidated sites
- metrics.detection_rate: Overall recall across all reference sites
- metrics.precision: Overall precision across all detected sites
- metrics.position_median_error: Median positional deviation on matched sites (bp)
- metrics.fpr_per_kb: False positive rate (
-1.0when no negative controls are included) - metrics.per_plasmid_breakdown: Per-accession breakdown of reference/matched/missed counts
- metrics.failure_category_summary: Aggregated counts of why sites were missed (NOT_SOUGHT, NO_SEED_MATCH, NO_PAIR, WRONG_POSITION)
- failures: List of failure category strings for each missed reference
The reference dataset is curated from three published studies:
| Source | PMID | Sites | Plasmids | Description |
|---|---|---|---|---|
| Cameranesi et al. 2018 (Front Microbiol) | 29434581 | 17 | 3 (KY984045.1, KY984046.1, KY984047.1) | XerC/D-like sites on Acinetobacter baumannii Ab242 plasmids; coordinates from Table 2 |
| Blackwell & Hall 2017 (Antimicrob Agents Chemother) | 28533235 | 8 | 1 (KY617771.1) | pdif sites from pS30-1; extracted from GenBank misc_recomb features |
| Shao et al. 2023 (Brief Bioinform) | 36426893 | 18 | 8 | pdif sites from the pdifFinder publication; seed sequences from redundant.seed.fa |
Total: 43 pdif site entries across 12 plasmids.
Data files live in tests/benchmark_data/:
literature_reference.csv-- curated pdif site coordinates and sequencesplasmids/-- FASTA and GenBank files for each reference plasmidresults/-- timestamped benchmark output directories
v2 uses Position Weight Matrices (PWM) built from 46 gold-standard XerC/XerD seed sequences to independently scan both DNA strands for pdif sites. It replaces fixed-threshold seed matching with quantitative motif scoring and adds CR-aware module detection.
Key Features:
| Feature | Description |
|---|---|
| PWM arm scanning | XerC and XerD detected independently on forward + reverse strands |
| C/D motif scoring | Quantitative orientation via XerC/XerD affinity scores |
| DC validation | Validate GenBank DC annotations against motif polarity |
| Hall-style degraded search | Three-tier detection with seed-based fallback |
| CR-aware module detection | Pairs sites by Central Region compatibility |
| Canonical normalization | All sites standardized to XerC-spacer-XerD |
Usage:
**CLI:**
```bash
# pwm scan
python -m PdifFinder.pdif_pwm_scanner -f plasmid.fasta
# Hall degraded search (with expected positions)
python -m PdifFinder.pdif_pwm_scanner -f plasmid.fasta -m hall -e 3239 5268 6120Python API:
from PdifFinder.pdif_pwm_scanner import build_pwms, scan_sequence
pwms = build_pwms()
sites, modules = scan_sequence(sequence, *pwms)Testing: pytest tests/test_pwm_scanner.py -v (18 tests)
from PdifFinder.pdif_pwm_scanner import hall_degraded_scan
r = hall_degraded_scan(seq, *pwms, expected_positions=[3239, 5268])
# Returns {high: [...], medium: [...], degraded: [...], missed: [...]}import PdifFinder.pdifFinder as pf
pf.findPdif(fasta_path, outdir, blastnPath, seedDB, resistanceGeneList)| Method | Detection | Speed | Best For |
|---|---|---|---|
| v1 CLI | 100% | Fast | Standard plasmid screening |
| pwm | 75-88% | Moderate | Variant/novel pdif detection |
| Hall | 100% | Moderate | Comprehensive + degraded rescue |
For developers interested in contributing or understanding the algorithm implementation:
- Algorithm Analysis: See
docs/algorithm_analysis.mdfor detailed analysis of the original algorithm - Algorithm Design: See
docs/algorithm_design.mdfor design decisions and implementation details - Test Data: Synthetic test sequences available in
tests/test_data/ - Performance Profiling: Use
profile_scanning.pyfor algorithm performance analysis