results

DeepCatch Results Summary

This directory contains the key results from DeepCatch validation runs.

---

Key Results at a Glance

Head-to-Head: AUC vs ctDNA Fraction

ctDNA Fraction	Bie (THEMIS)	CAPP-Seq	iDES	DeepCatch Multi-Modal	Δ vs Best
1.000%	0.8176	0.8474	0.5138	0.9610 ⭐	+0.1136
0.500%	0.8259	0.7951	0.5067	0.9390 ⭐	+0.1131
0.250%	0.8751	0.7179	0.5038	0.9334 ⭐	+0.0583
0.100%	0.9214	0.5960	0.5008	0.9273	+0.0059
0.050%	0.9172	0.5504	0.5025	0.9281	+0.0109
0.025%	0.9170	0.5242	0.5004	0.9167	-0.0003
0.010%	0.9150	0.5109	0.5004	0.9190	+0.0040
0.001%	0.9197	0.5047	0.5000	0.9277	+0.0080

⭐ = Statistically significant over Bie (p < 0.05, DeLong test). Bie et al. 2023 Nat Commun achieved AUC 0.966 on their own data — Bie AUC values shown here are from our re-implementation on DeepCatch's simulation framework.

Comparison to Published Clinical Assays

Assay	Sensitivity	Specificity	LOD	Cancer Types	Clinical Validation
Guardant360	85.3%	99.6%	0.01%	50	✅ >200K samples
Grail Galleri	51.5%	99.5%	N/A	50+	✅ NHS trial (140K)
CancerSEEK	70.0%	99.0%	N/A	8	✅
DeepCatch multi-modal	71.0%*	99.0%*	≤0.001%*	8	❌ Sim only

*Simulation-estimated. Cannot be directly compared to clinical results.

Longitudinal CET Performance

Metric	Value
Sensitivity	2.5%
Specificity (Overall)	97.0%
AUC	0.4926
Median Detection Time	1077 days
dual-target (sens≥70% + spec≥95%)	❌ NOT MET

---

Honest Limitations

DeepCatch is research-stage software. Here's what you need to know:

What's Honest

• ✅ All AUC values come from reproducible simulations with fixed seeds (seed=42)
• ✅ Real TCGA data used where possible (mutation frequencies from COSMIC v99)
• ✅ 6 literature-parameterized confounders applied (CHIP, shedding variability, error rates, etc.)
• ✅ Published methods re-implemented and benchmarked on the same data (fair comparison)

What's Not

• ❌ ZERO clinical samples — all results are simulation-based
• ❌ No head-to-head on the same plasma samples as Guardant360 or Grail
• ❌ Standard PCR/GC/sample-degradation effects not modeled
• ❌ No tissue-of-origin (TOO) prediction
• ❌ Requires 50,000× sequencing depth (10× clinical standard)

What Would Prove Reality

1. Partner with a clinical lab — run on real plasma from n≥50 cancer + 50 healthy
2. Head-to-head vs Guardant360 on the same blood draws
3. Independent validation at a second institution
4. Pre-registered analysis plan

Bottom line: DeepCatch's ideas are novel, but simulation ≠ clinical reality. The code is open; the clinical validation is the critical missing piece.

---

How to Reproduce

Full Python Pipeline

cd /path/to/deepcatch
bash RUN_ALL.sh

Quick Run (Reduced Data)

bash RUN_ALL.sh --quick

Node.js Real-Data Validation

cd validation/node
node runRealFinal.js

Docker

docker build -t deepcatch:latest .
docker run --rm -v $(pwd)/results:/app/results deepcatch:latest

Random Seeds

All experiments use seed=42 with 5-fold stratified cross-validation and 2,000 bootstrap iterations for confidence intervals.

---

File Index

File	Description
FINAL_REAL_DATA_REPORT.md	Comprehensive real-data validation with 8 cancer types
literature_review.md	Systematic review of 21 papers in cfDNA cancer screening
node/	Node.js validation outputs (runAll, headToHead, CET, etc.)
*.json	Raw validation results (ignored by git — regenerate with scripts)

---

Last updated: 2026-04-28. Results generated with honest intent.