EndoComparer: Comparative Surgical Video Question Answering with
Pseudo-Siamese Visual Adaptation and Perceive-then-Compare Reasoning

Shi Li^1,2, Lalith Sharan^1,2, Joël L. Lavanchy^3,4, Kun Yuan^1,2, Lorenzo Arboit^1,2, Nabani Banik^1,2, Elif Yazici^1,2, Maria Vannucci^2,5, Vinkle Srivastav^1,2, Pietro Mascagni^1,2,6, Nicolas Padoy^1,2

¹ICube, University of Strasbourg, CNRS, INSERM, France ²IHU Strasbourg, France ³Clarunis, Basel, Switzerland ⁴University of Basel, Switzerland ⁵University of Turin, Italy ⁶Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Rome, Italy

EndoComparer is a vision-language model focused on cross-case surgical video comparison: given two surgical videos from different cases, it answers questions about whether and how they differ in anatomy, tools, actions and skill. It gives each video its own visual encoding and reasoning path, which makes the model more robust and accurate against video-binding confusion, where the content of one video is attributed to the other.

🔥 News

• [2026.11.04] ⭐️ Code and model release.

🕹️ Quick start

1. Install

pip install -r requirements.txt            

2. Get the base model and the EndoComparer checkpoint

hf download Qwen/Qwen3.5-9B --local-dir models/Qwen3.5-9B
hf download ethanshili/EndoComparer --local-dir models/EndoComparer

The checkpoint holds the language-model LoRA (adapter_model.safetensors) and the Pseudo-Siamese Visual LoRA (dual_vision_lora.pt, psvl_config.json); both are loaded onto the frozen base at inference.

3. Run inference on the demo item

demo/ ships one EndoDiffVQA test item: two cholecystectomy clips and the question "Is the gallbladder coagulated in both videos?" with its four options.

bash scripts/evaluate.sh --jsonl demo/items.jsonl \
    --model models/Qwen3.5-9B --weights models/EndoComparer \
    --out results/demo/results.json

The model first states what each video shows, then gives the letter:

Video 1: coagulated. Video 2: not coagulated. (b)

results/demo/results.json holds the record (id, prediction, raw_prediction);

🚀 Training

bash scripts/train.sh --from-dataset --dataset-root data/EndoDiffVQA --out runs/endocomparer

🔮 Evaluation

bash scripts/evaluate.sh --dataset-root data/EndoDiffVQA --split test --out results/test/results.json

Runs EndoComparer on the test split with the shared frame protocol and writes Detect / Specify / Strict next to the results. Add --controls full,swap for the swap control, or --jsonl items.jsonl for your own items.

Citation

To be added.

License

This repository is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license. See the LICENSE file for details.

Acknowledgements

EndoDiffVQA is built on CholecT50, MultiBypass140 and CholeScore. EndoComparer builds on Qwen3.5-9B.