Skip to content
ivelinPublic

About

Open-source federated learning for robot arms: distributed operators train one shared SmolVLA-class model on private LeRobot datasets — only model updates move, data never leaves the operator's disk. Sim-first, with measured physics and honest evaluation gates.

Topics

Resources

Code of conduct

Contributing

Stars

4 stars

Watchers

1 watching

Forks

Latest commit

 

History

297 Commits

Folders and files

Repository files navigation

zk0 — Federated learning for robot arms: one shared model, trained on data that never leaves the operator’s disk

The bet: many operators, one shared model — only model updates move; data never leaves the operator’s disk.

License: Apache-2.0 Hugging Face: zk0 Simulation Assets collection Discord: join the community Python 3.10+

zk0: Federated Learning for Robotics AI

Open-source federated learning platform for SmolVLA models on SO-100 datasets using Flower framework.

Start here

What Where
📦 Simulation assets zk0 Simulation Assets collection — free, open-licensed, verified
🤖 Model checkpoint ivelin/zk0-smolvla-fl
🗺️ Roadmap zk0.bot/docs/ROADMAP.html
💬 Community Discord · GitHub Discussions
🦾 Node operators Become a node operator
🛠️ For AI assistants AGENTS.md — the operating contract for assistants working on this repo

See it

The SO-101 model sweeping its joints through their range of motion — wide and gripper close-up views

The motion above is the SO-101 model’s joint range-of-motion sweep from the asset package — wide and gripper close-up views of the same recorded trajectory, rendered from the zk0bot/so101-realistic model files.

The SO-101 arm model in reach pose

The SO-101 arm model in reach pose — hero still from the same asset package.

Contents: Latest Model Release · Direction · Latest findings (Nov 2026) · Simulation assets · Quick Start · Production Deployment · Project Status · Contributing · Get involved

Latest Model Release

  • Model: ivelin/zk0-smolvla-fl
  • Training: 250 rounds FedProx (μ=0.01), dynamic LR/MU scheduling
  • Final Policy Loss: 0.495
  • Clients: 4 clients on diverse SO-100 tasks
  • Framework: Flower + SmolVLA + SO-100 datasets
  • Maintenance (2026-11-16): the checkpoint was repaired so it loads cleanly with current LeRobot. The published weights are unchanged — verified tensor-for-tensor against the original.
from transformers import AutoModel
model = AutoModel.from_pretrained("ivelin/zk0-smolvla-fl")

Direction

zk0 trains one shared model across many independent robot-arm operators:

  • Distributed operators train a shared SmolVLA-class model on their private LeRobot datasets. Only model updates move — training data never leaves the operator's machine.
  • Sim-first, with measured physics. Simulation is our proving ground, but only when the physics is honest (see the audit below).
  • Provenance is never blurred. Every result is marked for what it actually is: sim-trained, sim-validated, or real-validated.
  • The current goal is concrete: the first external node running zk0 on data that never leaves its operator's disk.
  • All node software is open source (Apache-2.0).

Schematic of the zk0 bet: independent operators train at home on private data and send only model updates to one shared model, which comes back to every operator

The bet, as a diagram: many operators, one shared model — only model updates move; training data never leaves the operator’s disk.

Public roadmap: zk0.bot/docs/ROADMAP.html

Latest findings: we caught our own simulator lying (Nov 2026)

In November 2026 we audited the standard SO-101 simulation stack end-to-end — engines, arm model, object assets, and our own scoring code. The scoring code was the weakest link:

  • Final position is not grasp quality. Our task scorer only checked where objects ended up. We built a grasp-quality gate that also measures the carry itself — object tilt, in-gripper slip, grip force relative to object weight, and actuator saturation. Re-scoring our recorded results under the gate flipped 5 previously "passing" verdicts to FAIL.
  • One "pass" was a physics exploit. A mug grasp that scored as a success was being held by a ~47× body-weight stall clamp, squeezed through gripper contacts that a demo-tuned arm model ships with — finger friction that overrides the grasped object's own material. The real arm cannot do that; the score was fiction.
  • Engine scorecard. On pre-registered, weighted criteria — scored by an independent verifier from the raw logs, not by the team that ran the tests — MuJoCo ranked first of six engine configurations (82.6/100). Our working stack is MuJoCo + the Newton OpenUSD bridge + an open-licensed object shelf, probe-measured asset by asset before it enters a scene. Standing qualification: a sim score is not evidence of real-world transfer, and we never quote it as any.
  • An honest servo changes the answers. We added a fitted STS3215 servo model (BAM) as a switchable profile: under load it sags 24.8–65.1 mm where the stock model sags under 1 mm, and grasps that "pass" under the stock servo fail under it. That is the point — evaluation should use the servo the hardware actually has.
  • Friction belongs to the object's coating, and clearance is a budget. The same grasp strategy holds or sheds depending on the mug's surface class (glazed vs slippery vs rubbery), and one placement task flipped from fail to complete with 20 mm more clearance — while still failing the carry-quality gate. Both effects are now first-class fields in our asset database, not footnotes.

Full write-up: linked from zk0.bot. The assets in the next section are the ones these findings were measured on.

Simulation assets on Hugging Face

Free, open-licensed, and verified the hard way — every asset below ships with its verification sheet, and this project's scenes load only from the collection:

zk0 Simulation Assets (collection)

  • zk0bot/so101-realistic — SO-101 arm realism variant, v0.2.0: de-tuned gripper contacts so the grasped object's material governs friction, a measured elbow joint limit, and the BAM fitted-servo profile as a switchable option. Apache-2.0.
  • zk0bot/ycb-025-mug — the YCB mug, converted for simulation around its measured mass (0.118 kg — most object libraries guess), and probe-passed against our asset battery. CC BY 4.0.
  • zk0bot/triposg-mug — a generated mug, and the collection's one sealed asset: its files are hash-stamped against the exact test record that validated them, so anyone can check the proof instead of re-running it. CC BY 4.0.
  • ivelin/zk0-smolvla-fl — the federated SmolVLA checkpoint (see Latest Model Release).

Quick Start

For detailed setup, see docs/INSTALLATION.

Prerequisites

  • Python 3.10+, Conda, Git.
  • NVIDIA GPU recommended.

Clone and Setup

git clone https://github.com/ivelin/zk0.git
cd zk0

# Create conda env
conda create -n zk0 python=3.10 -y
conda activate zk0
conda install ffmpeg=7.1.1 -c conda-forge

# Install deps
pip install -e .

# Env vars
cp .env.example .env  # Edit as needed (e.g., HF_TOKEN)

Run the Simulation

See docs/INSTALLATION for full instructions.

# Tiny test run (1 round)
./train-fl-simulation.sh --tiny

Production Deployment

For production use with multiple distributed nodes, use the zk0bot CLI tool for orchestrated federated learning:

# Server operator: Start the central coordinator
zk0bot server start

# Client operators (each with their private dataset)
zk0bot client start hf:yourusername/your-private-dataset
# or
zk0bot client start local:/path/to/your/dataset

# Monitor status
zk0bot status

# Stop services
zk0bot server stop
zk0bot client stop

The CLI uses Flower's Deployment Engine (SuperLink, SuperNodes, SuperExec) for stateless, insecure-mode operation. Server runs continuously, automatically managing FL sessions based on connected clients. See docs/NODE-OPERATORS.md for detailed setup and security notes.

For run details, outputs, experiment tracking, and model pushing, see docs/RUNNING. For repository branches and contributing guidelines, see CONTRIBUTING.

Project Status

Current Stage: Beta

The project is ready for local simulation testing with multiple clients and datasets.

In Progress

  • Preparing client and server modules for production deployment
  • Recruiting the first external node operators — see Contributing below
  • Simulation validation program: measured-physics assets and honest evaluation gates — see Latest findings above
  • Longer-term research: selective ZK techniques (aggregator and norm proofs); transparency-log provenance

Config: 12 clients available (4 active); 500 rounds; policy loss metric; FedProx (μ=0.01); server-side evaluation.

For full details, see docs/ARCHITECTURE.

Contributing

We welcome contributions from the community! At this Beta stage, we're particularly interested in:

Node Operators

Requirements

  • Hardware: LeRobot SO100 or SO101 robotic arm. Contributors can either:
  • Compute: Local machine with RTX 3090 GPU or better, compatible with LeRobot library
  • Network: Stable internet connection for federated communication
  • Data: Unique training data from your robotics setup

If you meet these requirements, we'd love for you to join as a node operator. Your unique training data and compute resources will help improve the federated learning system. For detailed setup instructions, see CONTRIBUTING.

Other Ways to Contribute

There are several ways you can contribute to this project:

  1. Node Operators: Join the federated network with your hardware and data
  2. Code Contributors: Improve the codebase, add features, fix bugs
  3. Documentation: Help improve documentation and tutorials
  4. Testing: Report bugs, test new features, improve test coverage
  5. Feedback: Share your experience and suggestions

For more details on each, see CONTRIBUTING.

Working with AI assistants

Day-to-day work on this repo — setup, upkeep, deployment, issue triage, and PR preparation — is done by AI assistants working under the maintainer's supervision, the same way zk0 node operators will work with their own bots. Assistants can prepare a change end to end, but merges, releases, and public claims stay human-approved, and whoever builds a change never certifies it: every PR carries its verification evidence. The operating contract for assistants is AGENTS.md — humans directing an assistant should point it there first.

Get involved

  • Try the assets — everything in the zk0 Simulation Assets collection is free and open-licensed. Download a mug, stress our physics, tell us what you find.
  • Star / watch this repo — follow along as verified results and releases land.
  • Join the community — Discord for operators and builders; GitHub Discussions for longer-form questions.
  • Own an SO-100/SO-101 arm? zk0 is recruiting its first external node operators — the milestone this whole project is pointed at. Start with the node-operator application or say hello in Discord.

About

Open-source federated learning for robot arms: distributed operators train one shared SmolVLA-class model on private LeRobot datasets — only model updates move, data never leaves the operator's disk. Sim-first, with measured physics and honest evaluation gates.

Topics

Resources

Code of conduct

Contributing

Stars

4 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages